2024-03-22T10:15:17.604 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-22T10:15:17.605 DEBUG:teuthology.run:Teuthology command: teuthology --name mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --owner mchangir --archive /home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273 --verbose --description fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273/orig.config.yaml 2024-03-22T10:15:17.656 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T10:15:17.747 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273 branch: wip-mchangir-testing-main-20240318.032620 description: fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: mchangir@redhat.com first_in_suite: false job_id: '7616273' kernel: flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.0 ' name: mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 8781 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 7/100 suite: fs:upgrade suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 targets: smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOZzXRohABkKc0eyosDcZ7Pj4wmSn9EHedxgyfbWG3nNkhXPgeagRbsOfQFehm0lfJc5UBWiZEDSOG1Krl1X61s= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLZMjNswqm/B+vSDPOQOn4rTgjwKsbCB0d+IcJtfWnVO0J9bHO/l18dA42MwhBfuTiLS24cB6BBzzA8kpzoNHGs= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.0 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.0 roleless: true - print: '**** done end installing v18.2.0 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/mchangir/src/git.ceph.com_ceph-c_wip-mchangir-testing-main-20240318.032620/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_09:46:06 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-22T10:15:17.747 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-22T10:15:17.748 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-22T10:15:17.748 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-22T10:15:17.750 INFO:teuthology.task.internal:Checking packages... 2024-03-22T10:15:17.773 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-22T10:15:17.774 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T10:15:17.774 INFO:teuthology.packaging:ref: None 2024-03-22T10:15:17.774 INFO:teuthology.packaging:tag: None 2024-03-22T10:15:17.774 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-22T10:15:17.774 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:15:17.774 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-22T10:15:17.919 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-22T10:15:17.920 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-22T10:15:17.927 INFO:teuthology.task.internal:no buildpackages task found 2024-03-22T10:15:17.927 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-22T10:15:17.964 INFO:teuthology.task.internal:Saving configuration 2024-03-22T10:15:17.982 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-22T10:15:17.988 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T10:15:18.015 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 10:10:00.380312', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOZzXRohABkKc0eyosDcZ7Pj4wmSn9EHedxgyfbWG3nNkhXPgeagRbsOfQFehm0lfJc5UBWiZEDSOG1Krl1X61s='} 2024-03-22T10:15:18.037 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 10:10:00.378428', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLZMjNswqm/B+vSDPOQOn4rTgjwKsbCB0d+IcJtfWnVO0J9bHO/l18dA42MwhBfuTiLS24cB6BBzzA8kpzoNHGs='} 2024-03-22T10:15:18.038 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-22T10:15:18.044 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-22T10:15:18.044 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-22T10:15:18.044 INFO:teuthology.run_tasks:Running task console_log... 2024-03-22T10:15:18.171 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa833a4c0d0>, signals=[15]) 2024-03-22T10:15:18.171 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-22T10:15:18.178 INFO:teuthology.task.internal:Opening connections... 2024-03-22T10:15:18.178 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:15:18.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:15:18.271 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-03-22T10:15:18.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:15:18.365 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-22T10:15:18.371 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-03-22T10:15:18.388 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-03-22T10:15:18.388 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-03-22T10:15:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-03-22T10:15:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-03-22T10:15:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-03-22T10:15:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-03-22T10:15:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-03-22T10:15:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-03-22T10:15:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-22T10:15:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-03-22T10:15:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-03-22T10:15:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-22T10:15:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-03-22T10:15:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T10:15:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-22T10:15:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-22T10:15:18.446 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-03-22T10:15:18.467 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-03-22T10:15:18.485 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-03-22T10:15:18.485 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-03-22T10:15:18.541 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-22T10:15:18.542 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-22T10:15:18.543 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-03-22T10:15:18.567 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-22T10:15:18.576 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-22T10:15:18.610 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-22T10:15:18.611 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-03-22T10:15:18.613 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-03-22T10:15:18.629 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-22T10:15:18.641 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-22T10:15:18.641 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-03-22T10:15:18.669 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-03-22T10:15:18.764 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-22T10:15:18.818 INFO:teuthology.run_tasks:Running task kernel... 2024-03-22T10:15:18.846 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-22T10:15:18.846 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}}, timeout 300 2024-03-22T10:15:18.847 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T10:15:18.847 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T10:15:18.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T10:15:18.864 DEBUG:teuthology.task.kernel:sha1 for host.a is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T10:15:18.864 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-22T10:15:18.864 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-03-22T10:15:18.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T10:15:18.865 DEBUG:teuthology.task.kernel:sha1 for host.b is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T10:15:18.865 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-22T10:15:18.866 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-03-22T10:15:18.921 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-361.el9.x86_64 2024-03-22T10:15:18.921 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T10:15:18.921 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-22T10:15:18.921 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T10:15:18.926 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-361.el9.x86_64 2024-03-22T10:15:18.926 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T10:15:18.926 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-22T10:15:18.927 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T10:15:19.076 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum upgrade -y linux-firmware 2024-03-22T10:15:19.193 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum upgrade -y linux-firmware 2024-03-22T10:15:21.231 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-22T10:15:21.344 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-22T10:15:22.606 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2024-03-22T10:15:22.720 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-03-22T10:15:23.538 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-03-22T10:15:23.577 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-22T10:15:27.030 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-03-22T10:15:28.003 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 19 MB 00:03 2024-03-22T10:15:32.572 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.1 MB 00:02 2024-03-22T10:15:32.746 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.1 MB 00:01 2024-03-22T10:15:33.981 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-03-22T10:15:35.063 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-03-22T10:15:35.436 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-22T10:15:36.306 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-03-22T10:15:40.341 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-22T10:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-22T10:15:42.563 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:15:42.571 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:15:42.571 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-03-22T10:15:42.571 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:15:42.571 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-03-22T10:15:42.571 INFO:teuthology.orchestra.run.smithi115.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-22T10:15:42.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:15:42.571 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:15:42.572 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:15:42.572 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 1 Package 2024-03-22T10:15:42.572 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:15:42.572 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 375 M 2024-03-22T10:15:42.572 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:15:43.465 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:15:43.480 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:15:43.480 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-03-22T10:15:43.481 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:15:43.481 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-03-22T10:15:43.481 INFO:teuthology.orchestra.run.smithi150.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-22T10:15:43.481 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:15:43.481 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:15:43.481 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:15:43.481 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 1 Package 2024-03-22T10:15:43.481 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:15:43.481 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 375 M 2024-03-22T10:15:43.482 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:15:54.390 INFO:teuthology.orchestra.run.smithi150.stdout:linux-firmware-20240219-143.el9.noarch.rpm 34 MB/s | 375 MB 00:10 2024-03-22T10:15:54.391 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:15:54.391 INFO:teuthology.orchestra.run.smithi150.stdout:Total 34 MB/s | 375 MB 00:10 2024-03-22T10:15:54.391 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:15:54.496 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:15:54.496 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:15:56.723 INFO:teuthology.orchestra.run.smithi115.stdout:linux-firmware-20240219-143.el9.noarch.rpm 26 MB/s | 375 MB 00:14 2024-03-22T10:15:56.724 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:15:56.724 INFO:teuthology.orchestra.run.smithi115.stdout:Total 26 MB/s | 375 MB 00:14 2024-03-22T10:15:56.724 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:15:56.819 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:15:56.819 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:15:56.987 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:15:56.988 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:15:59.298 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:15:59.298 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:16:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:16:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-22T10:16:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-22T10:16:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-22T10:16:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-22T10:16:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-22T10:16:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:16:05.380 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-03-22T10:16:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-22T10:16:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:16:05.380 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:16:05.670 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.b... 2024-03-22T10:16:05.671 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T10:16:05.749 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:16:05.918 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-22T10:16:05.918 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-22T10:16:05.921 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi150.front.sepia.ceph.com to complete... 2024-03-22T10:16:05.988 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-22T10:16:06.072 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-22T10:16:07.301 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-22T10:16:07.301 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-22T10:16:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-22T10:16:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:16:07.702 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-03-22T10:16:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-22T10:16:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:16:07.703 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:16:08.026 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.a... 2024-03-22T10:16:08.026 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T10:16:08.265 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-22T10:16:08.266 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-22T10:16:08.268 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi115.front.sepia.ceph.com to complete... 2024-03-22T10:17:22.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22 10:17:22 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-22T10:17:22.094 INFO:teuthology.orchestra.run.smithi115.stderr:FINISHED --2024-03-22 10:17:22-- 2024-03-22T10:17:22.095 INFO:teuthology.orchestra.run.smithi115.stderr:Total wall clock time: 1m 14s 2024-03-22T10:17:22.095 INFO:teuthology.orchestra.run.smithi115.stderr:Downloaded: 1 files, 321M in 1m 13s (4.36 MB/s) 2024-03-22T10:17:22.098 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.a... 2024-03-22T10:17:22.099 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-22T10:17:24.051 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying... ######################################## 2024-03-22T10:17:24.110 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing... ######################################## 2024-03-22T10:17:24.191 INFO:teuthology.orchestra.run.smithi115.stdout:Updating / installing... 2024-03-22T10:17:31.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22 10:17:31 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-22T10:17:31.136 INFO:teuthology.orchestra.run.smithi150.stderr:FINISHED --2024-03-22 10:17:31-- 2024-03-22T10:17:31.136 INFO:teuthology.orchestra.run.smithi150.stderr:Total wall clock time: 1m 25s 2024-03-22T10:17:31.137 INFO:teuthology.orchestra.run.smithi150.stderr:Downloaded: 1 files, 321M in 1m 25s (3.79 MB/s) 2024-03-22T10:17:31.140 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.b... 2024-03-22T10:17:31.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-22T10:17:31.743 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-22T10:17:33.070 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying... ######################################## 2024-03-22T10:17:33.139 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing... ######################################## 2024-03-22T10:17:33.218 INFO:teuthology.orchestra.run.smithi150.stdout:Updating / installing... 2024-03-22T10:17:40.681 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-22T10:17:46.374 INFO:teuthology.orchestra.run.smithi115.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T10:17:55.750 INFO:teuthology.orchestra.run.smithi150.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T10:18:08.807 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-22T10:18:08.807 DEBUG:teuthology.orchestra.run.smithi115:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-22T10:18:08.878 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:08.878 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-22T10:18:08.878 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-22T10:18:08.878 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-22T10:18:08.878 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-22T10:18:08.879 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-22T10:18:08.880 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-22T10:18:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-22T10:18:08.882 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-22T10:18:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-22T10:18:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-22T10:18:08.885 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-22T10:18:08.886 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-22T10:18:08.887 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-22T10:18:08.887 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-22T10:18:08.887 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-22T10:18:08.887 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-22T10:18:08.887 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-22T10:18:08.887 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-22T10:18:08.887 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-22T10:18:08.887 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-22T10:18:08.888 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-22T10:18:08.889 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-22T10:18:08.889 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-22T10:18:08.889 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-22T10:18:08.889 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-22T10:18:08.889 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-22T10:18:08.889 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-22T10:18:08.889 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-22T10:18:08.890 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-22T10:18:08.890 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-22T10:18:08.890 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-22T10:18:08.890 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-22T10:18:08.890 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-22T10:18:08.890 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-22T10:18:08.890 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-22T10:18:08.891 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-22T10:18:08.892 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-22T10:18:08.892 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-22T10:18:08.892 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-22T10:18:08.892 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-22T10:18:08.892 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-22T10:18:08.892 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-22T10:18:08.892 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-22T10:18:08.892 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-22T10:18:08.892 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-22T10:18:08.893 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-22T10:18:08.895 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-22T10:18:08.895 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-22T10:18:08.895 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-22T10:18:08.895 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-22T10:18:08.895 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-22T10:18:08.895 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-22T10:18:08.896 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-22T10:18:08.896 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-22T10:18:08.896 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-22T10:18:08.896 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-22T10:18:08.896 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-22T10:18:08.896 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-22T10:18:08.896 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-22T10:18:08.897 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-22T10:18:08.897 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-22T10:18:08.897 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-22T10:18:08.897 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-22T10:18:08.897 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-22T10:18:08.897 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-22T10:18:08.897 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-22T10:18:08.898 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-22T10:18:08.898 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-22T10:18:08.898 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-22T10:18:08.898 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-22T10:18:08.898 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-22T10:18:08.898 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-22T10:18:08.899 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-22T10:18:08.899 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-22T10:18:08.899 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-22T10:18:08.899 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-22T10:18:08.899 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-22T10:18:08.899 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-22T10:18:08.900 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-22T10:18:08.900 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-22T10:18:08.900 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-22T10:18:08.900 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-22T10:18:08.900 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-22T10:18:08.900 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-22T10:18:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-22T10:18:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-22T10:18:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-22T10:18:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-22T10:18:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-22T10:18:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-22T10:18:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-22T10:18:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-22T10:18:08.902 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-22T10:18:08.902 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-22T10:18:08.902 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-22T10:18:08.902 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-22T10:18:08.902 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-22T10:18:08.903 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-22T10:18:08.904 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-22T10:18:08.905 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-22T10:18:08.906 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-22T10:18:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-22T10:18:08.908 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-22T10:18:08.909 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-22T10:18:08.910 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-22T10:18:08.911 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-22T10:18:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-22T10:18:08.913 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-22T10:18:08.914 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-22T10:18:08.915 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-22T10:18:08.916 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-22T10:18:08.916 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-22T10:18:08.916 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-22T10:18:08.916 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-22T10:18:08.916 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-22T10:18:08.916 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-22T10:18:08.917 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-22T10:18:08.917 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-22T10:18:08.917 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-22T10:18:08.917 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-22T10:18:08.917 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-22T10:18:08.918 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-22T10:18:08.919 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-22T10:18:08.920 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-22T10:18:08.921 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-22T10:18:08.922 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-22T10:18:08.923 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-22T10:18:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-22T10:18:08.925 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-22T10:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-22T10:18:08.927 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-22T10:18:08.928 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-22T10:18:08.929 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-22T10:18:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-22T10:18:08.931 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-22T10:18:08.932 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-22T10:18:08.933 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-22T10:18:08.934 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-22T10:18:08.935 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-22T10:18:08.936 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-22T10:18:08.937 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-22T10:18:08.938 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-22T10:18:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-22T10:18:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-22T10:18:08.941 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-22T10:18:08.942 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-22T10:18:08.943 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-22T10:18:08.944 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-22T10:18:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-22T10:18:08.946 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-22T10:18:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-22T10:18:08.948 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-22T10:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-22T10:18:08.950 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-22T10:18:08.951 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-22T10:18:08.952 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-22T10:18:08.953 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-22T10:18:08.954 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-22T10:18:08.955 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-22T10:18:08.956 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-22T10:18:08.957 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-22T10:18:08.958 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-22T10:18:08.958 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-22T10:18:08.958 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-22T10:18:08.958 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-22T10:18:08.958 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-22T10:18:08.958 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-22T10:18:08.958 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-22T10:18:08.959 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-22T10:18:08.959 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-22T10:18:08.959 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-22T10:18:08.959 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-22T10:18:08.959 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-22T10:18:08.959 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-22T10:18:08.959 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-22T10:18:08.959 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-22T10:18:08.959 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-22T10:18:08.960 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-22T10:18:08.960 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-22T10:18:08.960 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-22T10:18:08.960 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-22T10:18:08.960 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-22T10:18:08.960 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-22T10:18:08.961 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-22T10:18:08.962 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-22T10:18:08.963 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-22T10:18:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-22T10:18:08.965 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-22T10:18:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-22T10:18:08.967 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-22T10:18:08.968 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-22T10:18:08.968 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-22T10:18:08.968 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-22T10:18:08.968 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-22T10:18:08.968 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-22T10:18:08.968 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-22T10:18:08.968 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-22T10:18:08.968 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-22T10:18:08.969 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-22T10:18:08.970 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-22T10:18:08.971 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-22T10:18:08.971 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-22T10:18:08.971 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-22T10:18:08.971 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-22T10:18:08.971 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-22T10:18:08.971 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-22T10:18:08.971 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-22T10:18:08.971 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-22T10:18:08.972 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-22T10:18:08.972 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-22T10:18:08.972 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-22T10:18:08.972 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-22T10:18:08.972 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-22T10:18:08.978 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-22T10:18:08.979 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-22T10:18:08.980 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-22T10:18:08.981 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-22T10:18:08.982 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-22T10:18:08.983 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-22T10:18:08.984 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-22T10:18:08.985 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-22T10:18:08.986 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-22T10:18:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-22T10:18:08.988 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-22T10:18:08.989 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-22T10:18:08.990 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-22T10:18:08.991 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-22T10:18:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-22T10:18:08.993 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-22T10:18:08.994 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-22T10:18:08.995 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-22T10:18:08.996 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-22T10:18:08.997 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-22T10:18:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-22T10:18:08.999 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-22T10:18:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-22T10:18:09.001 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-22T10:18:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-22T10:18:09.003 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-22T10:18:09.004 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-22T10:18:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-22T10:18:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-22T10:18:09.007 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-22T10:18:09.008 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-22T10:18:09.009 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-22T10:18:09.010 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-22T10:18:09.011 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-22T10:18:09.012 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-22T10:18:09.013 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-22T10:18:09.014 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-22T10:18:09.015 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-22T10:18:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-22T10:18:09.017 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-22T10:18:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-22T10:18:09.019 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-22T10:18:09.020 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-22T10:18:09.021 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-22T10:18:09.022 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-22T10:18:09.023 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-22T10:18:09.024 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-22T10:18:09.025 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-22T10:18:09.026 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-22T10:18:09.027 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-22T10:18:09.028 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-22T10:18:09.029 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-22T10:18:09.030 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-22T10:18:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-22T10:18:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-22T10:18:09.033 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-22T10:18:09.034 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-22T10:18:09.035 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-22T10:18:09.036 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-22T10:18:09.037 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-22T10:18:09.038 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-22T10:18:09.042 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:09.042 DEBUG:teuthology.orchestra.run.smithi115:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-22T10:18:09.067 INFO:teuthology.orchestra.run.smithi115.stdout:postinstall scriptlet (using /bin/sh): 2024-03-22T10:18:09.067 INFO:teuthology.orchestra.run.smithi115.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-22T10:18:09.067 INFO:teuthology.orchestra.run.smithi115.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-22T10:18:09.067 INFO:teuthology.orchestra.run.smithi115.stdout:fi 2024-03-22T10:18:09.067 INFO:teuthology.orchestra.run.smithi115.stdout:for file in vmlinuz System.map config; do 2024-03-22T10:18:09.067 INFO:teuthology.orchestra.run.smithi115.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-22T10:18:09.067 INFO:teuthology.orchestra.run.smithi115.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout: fi 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:done 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:fi 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:09.068 INFO:teuthology.orchestra.run.smithi115.stdout:fi 2024-03-22T10:18:09.069 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 68.el9 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8264254 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-22T10:18:09.148 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:18:09.149 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2024-03-22T10:18:09.150 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-22T10:18:09.150 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:09.150 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-22T10:18:09.757 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-03-22T10:18:11.083 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-22T10:18:11.119 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-03-22T10:18:11.122 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-03-22T10:18:11.136 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.QbtmFxOJCj 2024-03-22T10:18:11.136 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QbtmFxOJCj 2024-03-22T10:18:11.204 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.QbtmFxOJCj 2024-03-22T10:18:11.349 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.QbtmFxOJCj is 6KB 2024-03-22T10:18:11.398 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.QbtmFxOJCj 2024-03-22T10:18:11.412 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2024-03-22T10:18:11.477 INFO:teuthology.orchestra.run.smithi115.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-22T10:18:11.477 INFO:teuthology.orchestra.run.smithi115.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-22T10:18:11.477 INFO:teuthology.orchestra.run.smithi115.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-22T10:18:11.477 INFO:teuthology.orchestra.run.smithi115.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-22T10:18:11.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:11.614 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-03-22T10:18:18.157 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-22T10:18:18.157 DEBUG:teuthology.orchestra.run.smithi150:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-22T10:18:18.225 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-22T10:18:18.226 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-22T10:18:18.227 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-22T10:18:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-22T10:18:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-22T10:18:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-22T10:18:18.231 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-22T10:18:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-22T10:18:18.233 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-22T10:18:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-22T10:18:18.235 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-22T10:18:18.236 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-22T10:18:18.237 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-22T10:18:18.238 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-22T10:18:18.239 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-22T10:18:18.240 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-22T10:18:18.241 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-22T10:18:18.242 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-22T10:18:18.243 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-22T10:18:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-22T10:18:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-22T10:18:18.246 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-22T10:18:18.247 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-22T10:18:18.248 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-22T10:18:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-22T10:18:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-22T10:18:18.251 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-22T10:18:18.252 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-22T10:18:18.253 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-22T10:18:18.254 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-22T10:18:18.255 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-22T10:18:18.256 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-22T10:18:18.257 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-22T10:18:18.258 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-22T10:18:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-22T10:18:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-22T10:18:18.261 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-22T10:18:18.262 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-22T10:18:18.263 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-22T10:18:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-22T10:18:18.265 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-22T10:18:18.266 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-22T10:18:18.267 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-22T10:18:18.268 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-22T10:18:18.269 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-22T10:18:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-22T10:18:18.271 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-22T10:18:18.272 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-22T10:18:18.273 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-22T10:18:18.274 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-22T10:18:18.275 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-22T10:18:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-22T10:18:18.277 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-22T10:18:18.278 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-22T10:18:18.279 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-22T10:18:18.280 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-22T10:18:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-22T10:18:18.282 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-22T10:18:18.283 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-22T10:18:18.284 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-22T10:18:18.285 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-22T10:18:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-22T10:18:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-22T10:18:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-22T10:18:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-22T10:18:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-22T10:18:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-22T10:18:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-22T10:18:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-22T10:18:18.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-22T10:18:18.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-22T10:18:18.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-22T10:18:18.289 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-22T10:18:18.290 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-22T10:18:18.291 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-22T10:18:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-22T10:18:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-22T10:18:18.294 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-22T10:18:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-22T10:18:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-22T10:18:18.297 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-22T10:18:18.298 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-22T10:18:18.299 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-22T10:18:18.300 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-22T10:18:18.301 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-22T10:18:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-22T10:18:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-22T10:18:18.304 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-22T10:18:18.305 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-22T10:18:18.306 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-22T10:18:18.307 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-22T10:18:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-22T10:18:18.309 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-22T10:18:18.310 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-22T10:18:18.311 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-22T10:18:18.312 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-22T10:18:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-22T10:18:18.314 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-22T10:18:18.315 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-22T10:18:18.316 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-22T10:18:18.317 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-22T10:18:18.318 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-22T10:18:18.319 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-22T10:18:18.320 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-22T10:18:18.321 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-22T10:18:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-22T10:18:18.323 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-22T10:18:18.324 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-22T10:18:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-22T10:18:18.326 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-22T10:18:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-22T10:18:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-22T10:18:18.329 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-22T10:18:18.330 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-22T10:18:18.331 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-22T10:18:18.332 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-22T10:18:18.333 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-22T10:18:18.334 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-22T10:18:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-22T10:18:18.336 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-22T10:18:18.337 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-22T10:18:18.338 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-22T10:18:18.339 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-22T10:18:18.340 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-22T10:18:18.341 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-22T10:18:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-22T10:18:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-22T10:18:18.344 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-22T10:18:18.345 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-22T10:18:18.346 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-22T10:18:18.347 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-22T10:18:18.348 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-22T10:18:18.349 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-22T10:18:18.350 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-22T10:18:18.351 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-22T10:18:18.352 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-22T10:18:18.353 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-22T10:18:18.354 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-22T10:18:18.355 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-22T10:18:18.356 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-22T10:18:18.357 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-22T10:18:18.358 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-22T10:18:18.359 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-22T10:18:18.360 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-22T10:18:18.361 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-22T10:18:18.362 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-22T10:18:18.363 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-22T10:18:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-22T10:18:18.365 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-22T10:18:18.366 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-22T10:18:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-22T10:18:18.371 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:18.371 DEBUG:teuthology.orchestra.run.smithi150:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-22T10:18:18.393 INFO:teuthology.orchestra.run.smithi150.stdout:postinstall scriptlet (using /bin/sh): 2024-03-22T10:18:18.393 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-22T10:18:18.393 INFO:teuthology.orchestra.run.smithi150.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-22T10:18:18.393 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-03-22T10:18:18.393 INFO:teuthology.orchestra.run.smithi150.stdout:for file in vmlinuz System.map config; do 2024-03-22T10:18:18.393 INFO:teuthology.orchestra.run.smithi150.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-22T10:18:18.393 INFO:teuthology.orchestra.run.smithi150.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-22T10:18:18.393 INFO:teuthology.orchestra.run.smithi150.stdout: fi 2024-03-22T10:18:18.393 INFO:teuthology.orchestra.run.smithi150.stdout:done 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:18.394 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-03-22T10:18:18.395 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -qi grub2-tools 2024-03-22T10:18:18.471 INFO:teuthology.orchestra.run.smithi150.stdout:Name : grub2-tools 2024-03-22T10:18:18.471 INFO:teuthology.orchestra.run.smithi150.stdout:Epoch : 1 2024-03-22T10:18:18.471 INFO:teuthology.orchestra.run.smithi150.stdout:Version : 2.06 2024-03-22T10:18:18.471 INFO:teuthology.orchestra.run.smithi150.stdout:Release : 68.el9 2024-03-22T10:18:18.471 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: x86_64 2024-03-22T10:18:18.471 INFO:teuthology.orchestra.run.smithi150.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-22T10:18:18.471 INFO:teuthology.orchestra.run.smithi150.stdout:Group : Unspecified 2024-03-22T10:18:18.471 INFO:teuthology.orchestra.run.smithi150.stdout:Size : 8264254 2024-03-22T10:18:18.471 INFO:teuthology.orchestra.run.smithi150.stdout:License : GPLv3+ 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:Packager : builder@centos.org 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:Vendor : CentOS 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:Summary : Support tools for GRUB. 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:Description : 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:hardware devices. 2024-03-22T10:18:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:18:18.473 INFO:teuthology.orchestra.run.smithi150.stdout:This subpackage provides tools for support of all platforms. 2024-03-22T10:18:18.473 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-22T10:18:18.474 INFO:teuthology.task.kernel:Updating grub on smithi150 to boot 6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:18.474 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-22T10:18:19.040 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2024-03-22T10:18:20.474 INFO:teuthology.orchestra.run.smithi150.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-22T10:18:20.493 INFO:teuthology.orchestra.run.smithi150.stderr:done 2024-03-22T10:18:20.496 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-03-22T10:18:20.511 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.HglOfQ43S9 2024-03-22T10:18:20.511 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HglOfQ43S9 2024-03-22T10:18:20.580 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.HglOfQ43S9 2024-03-22T10:18:20.720 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.HglOfQ43S9 is 6KB 2024-03-22T10:18:20.769 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.HglOfQ43S9 2024-03-22T10:18:20.783 DEBUG:teuthology.orchestra.run.smithi150:> sudo /bin/ls /boot/loader/entries || true 2024-03-22T10:18:20.848 INFO:teuthology.orchestra.run.smithi150.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-22T10:18:20.848 INFO:teuthology.orchestra.run.smithi150.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-22T10:18:20.848 INFO:teuthology.orchestra.run.smithi150.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-22T10:18:20.849 INFO:teuthology.orchestra.run.smithi150.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-22T10:18:20.850 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-22T10:18:20.984 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2024-03-22T10:18:41.636 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-22T10:18:41.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-03-22T10:18:41.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:18:50.996 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-22T10:18:50.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-22T10:18:50.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:18:59.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-03-22T10:19:09.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-03-22T10:19:09.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:19:12.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-03-22T10:19:24.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-03-22T10:19:24.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:19:25.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-03-22T10:19:34.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-22T10:19:34.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:19:42.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-03-22T10:19:52.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-03-22T10:19:57.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-03-22T10:19:57.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:19:59.541 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-03-22T10:20:00.174 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-03-22T10:20:00.174 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.8.0-g5ec3e6e88b2e"... 2024-03-22T10:20:00.174 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-03-22T10:20:00.213 INFO:teuthology.orchestra.run.smithi115.stdout:6.8.0-g5ec3e6e88b2e 2024-03-22T10:20:00.213 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-22T10:20:00.213 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T10:20:00.213 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-22T10:20:01.213 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-22T10:20:01.214 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-22T10:20:01.345 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-03-22T10:20:01.386 DEBUG:teuthology.parallel:result is None 2024-03-22T10:20:04.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-22T10:20:04.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:20:08.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-03-22T10:20:23.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-22T10:20:23.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:20:23.472 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-03-22T10:20:24.027 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-22T10:20:24.028 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.8.0-g5ec3e6e88b2e"... 2024-03-22T10:20:24.028 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-03-22T10:20:24.068 INFO:teuthology.orchestra.run.smithi150.stdout:6.8.0-g5ec3e6e88b2e 2024-03-22T10:20:24.069 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-22T10:20:24.069 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T10:20:24.069 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-22T10:20:25.069 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-22T10:20:25.070 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-22T10:20:25.223 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2024-03-22T10:20:25.304 DEBUG:teuthology.parallel:result is None 2024-03-22T10:20:25.305 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-22T10:20:25.317 INFO:teuthology.task.internal:Creating test directory... 2024-03-22T10:20:25.317 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T10:20:25.326 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T10:20:25.377 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-22T10:20:25.385 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-22T10:20:25.430 INFO:teuthology.task.internal:Creating archive directory... 2024-03-22T10:20:25.430 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T10:20:25.437 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T10:20:25.628 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-22T10:20:25.635 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-22T10:20:25.636 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T10:20:25.643 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T10:20:25.734 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T10:20:25.749 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T10:20:25.768 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T10:20:25.794 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T10:20:25.799 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-22T10:20:25.806 INFO:teuthology.task.internal:Configuring sudo... 2024-03-22T10:20:25.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T10:20:25.811 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T10:20:25.890 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-22T10:20:25.900 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-22T10:20:25.900 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T10:20:25.904 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T10:20:25.948 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T10:20:26.024 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T10:20:26.126 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T10:20:26.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T10:20:26.297 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:20:26.297 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T10:20:26.382 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T10:20:26.489 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T10:20:26.635 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T10:20:26.745 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T10:20:26.885 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:20:26.886 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T10:20:27.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-03-22T10:20:27.032 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-03-22T10:20:27.172 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T10:20:27.249 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T10:20:27.750 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-22T10:20:27.758 INFO:teuthology.task.internal:Starting timer... 2024-03-22T10:20:27.758 INFO:teuthology.run_tasks:Running task pcp... 2024-03-22T10:20:27.813 INFO:teuthology.run_tasks:Running task selinux... 2024-03-22T10:20:27.848 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-22T10:20:27.848 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-03-22T10:20:27.982 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-03-22T10:20:27.987 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-03-22T10:20:28.080 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-03-22T10:20:28.084 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-22T10:20:28.085 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-03-22T10:20:28.146 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-03-22T10:20:28.146 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-03-22T10:20:28.210 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-03-22T10:20:28.210 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2024-03-22T10:20:28.210 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-22T10:20:28.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T10:20:28.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-22T10:20:28.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T10:20:28.381 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-22T10:20:28.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-03-22T10:20:28.468 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-03-22T10:20:28.574 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-22T10:20:28.585 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-22T10:20:28.590 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-22T10:20:28.744 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-22T10:20:28.759 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-22T10:20:28.761 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi115.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-22T10:26:25.324 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2024-03-22T10:26:25.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-03-22T10:26:25.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:26:25.470 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-03-22T10:26:25.625 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-03-22T10:26:25.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-22T10:26:25.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T10:26:25.772 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-03-22T10:26:25.927 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-22T10:26:25.928 INFO:teuthology.run_tasks:Running task clock... 2024-03-22T10:26:25.941 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-22T10:26:25.941 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T10:26:25.942 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T10:26:25.949 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T10:26:25.950 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T10:26:26.046 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T10:26:26.051 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T10:26:26.099 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T10:26:26.102 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T10:26:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-03-22T10:26:26.196 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-03-22T10:26:26.233 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-03-22T10:26:26.236 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-03-22T10:26:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T10:26:26.292 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T10:26:26.335 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T10:26:26.347 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T10:26:26.460 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-03-22T10:26:26.468 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T10:26:26.468 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-03-22T10:26:26.468 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T10:26:26.468 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T10:26:26.468 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T10:26:26.468 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T10:26:26.478 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-03-22T10:26:26.485 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T10:26:26.485 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-03-22T10:26:26.486 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T10:26:26.486 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T10:26:26.486 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T10:26:26.486 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T10:26:26.487 INFO:teuthology.run_tasks:Running task install... 2024-03-22T10:26:26.496 DEBUG:teuthology.task.install:project ceph 2024-03-22T10:26:26.496 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-22T10:26:26.497 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.0', 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-22T10:26:26.497 INFO:teuthology.task.install:Using flavor: default 2024-03-22T10:26:26.503 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-22T10:26:26.503 INFO:teuthology.task.install:extra packages: [] 2024-03-22T10:26:26.503 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-03-22T10:26:26.504 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-22T10:26:26.504 INFO:teuthology.packaging:ref: None 2024-03-22T10:26:26.504 INFO:teuthology.packaging:tag: v18.2.0 2024-03-22T10:26:26.504 INFO:teuthology.packaging:branch: None 2024-03-22T10:26:26.504 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:26:26.854 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.0^{} -> None 2024-03-22T10:26:26.854 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-22T10:26:29.399 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.0^{} -> 5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-03-22T10:26:29.399 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-03-22T10:26:29.402 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-03-22T10:26:29.402 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-22T10:26:29.402 INFO:teuthology.packaging:ref: None 2024-03-22T10:26:29.403 INFO:teuthology.packaging:tag: v18.2.0 2024-03-22T10:26:29.403 INFO:teuthology.packaging:branch: None 2024-03-22T10:26:29.403 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:26:29.403 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-22T10:26:29.403 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-03-22T10:26:29.562 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-03-22T10:26:29.562 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-03-22T10:26:29.566 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-03-22T10:26:29.566 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-03-22T10:26:29.601 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T10:26:29.602 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:26:29.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T10:26:29.615 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T10:26:29.615 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:26:29.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T10:26:29.699 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-22T10:26:29.699 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-22T10:26:29.699 INFO:teuthology.packaging:ref: None 2024-03-22T10:26:29.699 INFO:teuthology.packaging:tag: v18.2.0 2024-03-22T10:26:29.699 INFO:teuthology.packaging:branch: None 2024-03-22T10:26:29.699 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:26:29.700 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T10:26:29.716 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-22T10:26:29.716 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-22T10:26:29.716 INFO:teuthology.packaging:ref: None 2024-03-22T10:26:29.717 INFO:teuthology.packaging:tag: v18.2.0 2024-03-22T10:26:29.717 INFO:teuthology.packaging:branch: None 2024-03-22T10:26:29.717 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:26:29.717 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T10:26:29.836 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T10:26:29.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T10:26:29.973 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T10:26:29.985 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T10:26:30.057 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-03-22T10:26:30.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-03-22T10:26:30.069 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-03-22T10:26:30.073 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-03-22T10:26:30.411 INFO:teuthology.orchestra.run.smithi115.stdout:74 files removed 2024-03-22T10:26:30.426 INFO:teuthology.orchestra.run.smithi150.stdout:74 files removed 2024-03-22T10:26:30.449 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw 2024-03-22T10:26:30.466 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw 2024-03-22T10:26:31.174 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 395 kB/s | 76 kB 00:00 2024-03-22T10:26:31.186 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 321 kB/s | 76 kB 00:00 2024-03-22T10:26:31.450 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 44 kB/s | 9.3 kB 00:00 2024-03-22T10:26:31.475 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 41 kB/s | 9.3 kB 00:00 2024-03-22T10:26:31.784 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 8.4 kB/s | 2.2 kB 00:00 2024-03-22T10:26:31.819 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 7.0 kB/s | 2.2 kB 00:00 2024-03-22T10:26:32.504 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-22T10:26:32.647 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.7 MB 00:00 2024-03-22T10:26:34.213 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - BaseOS 5.7 MB/s | 2.0 MB 00:00 2024-03-22T10:26:34.357 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-03-22T10:26:37.192 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.0 MB 00:02 2024-03-22T10:26:38.044 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.0 MB 00:03 2024-03-22T10:26:39.260 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-22T10:26:40.778 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-03-22T10:26:43.790 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.1 MB 00:00 2024-03-22T10:26:45.394 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-03-22T10:26:46.087 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.1 MB 00:01 2024-03-22T10:26:46.887 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-22T10:26:47.913 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-03-22T10:26:49.912 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-22T10:26:52.703 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-22T10:26:55.333 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:26:55.341 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:26:55.341 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-03-22T10:26:55.341 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:26:55.341 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:26:55.341 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-03-22T10:26:55.341 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-03-22T10:26:55.341 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-03-22T10:26:55.341 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-03-22T10:26:55.341 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-22T10:26:55.342 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-03-22T10:26:55.343 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-22T10:26:55.344 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-22T10:26:55.344 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:26:55.344 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:26:55.344 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:26:55.344 INFO:teuthology.orchestra.run.smithi115.stdout:Install 28 Packages 2024-03-22T10:26:55.344 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 2 Packages 2024-03-22T10:26:55.344 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:26:55.345 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 53 M 2024-03-22T10:26:55.345 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:26:55.673 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-22T10:26:55.713 INFO:teuthology.orchestra.run.smithi115.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 17 MB/s | 5.2 MB 00:00 2024-03-22T10:26:55.771 INFO:teuthology.orchestra.run.smithi115.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 417 kB/s | 24 kB 00:00 2024-03-22T10:26:55.847 INFO:teuthology.orchestra.run.smithi115.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 8.6 MB/s | 653 kB 00:00 2024-03-22T10:26:55.905 INFO:teuthology.orchestra.run.smithi115.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 8.1 MB/s | 474 kB 00:00 2024-03-22T10:26:56.089 INFO:teuthology.orchestra.run.smithi115.stdout:(5/30): librgw2-18.2.0-0.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-03-22T10:26:56.156 INFO:teuthology.orchestra.run.smithi115.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 680 kB/s | 45 kB 00:00 2024-03-22T10:26:56.298 INFO:teuthology.orchestra.run.smithi115.stdout:(7/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 8.6 MB/s | 7.6 MB 00:00 2024-03-22T10:26:56.331 INFO:teuthology.orchestra.run.smithi115.stdout:(8/30): python3-ceph-common-18.2.0-0.el9.x86_64 685 kB/s | 119 kB 00:00 2024-03-22T10:26:56.365 INFO:teuthology.orchestra.run.smithi115.stdout:(9/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-03-22T10:26:56.407 INFO:teuthology.orchestra.run.smithi115.stdout:(10/30): python3-rados-18.2.0-0.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-03-22T10:26:56.440 INFO:teuthology.orchestra.run.smithi115.stdout:(11/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 3.9 MB/s | 297 kB 00:00 2024-03-22T10:26:56.474 INFO:teuthology.orchestra.run.smithi115.stdout:(12/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 1.5 MB/s | 99 kB 00:00 2024-03-22T10:26:56.666 INFO:teuthology.orchestra.run.smithi115.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 472 kB/s | 106 kB 00:00 2024-03-22T10:26:56.691 INFO:teuthology.orchestra.run.smithi115.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2024-03-22T10:26:56.733 INFO:teuthology.orchestra.run.smithi115.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 665 kB/s | 44 kB 00:00 2024-03-22T10:26:56.809 INFO:teuthology.orchestra.run.smithi115.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 661 kB 00:00 2024-03-22T10:26:56.842 INFO:teuthology.orchestra.run.smithi115.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 282 kB 00:00 2024-03-22T10:26:56.892 INFO:teuthology.orchestra.run.smithi115.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 506 kB/s | 41 kB 00:00 2024-03-22T10:26:57.143 INFO:teuthology.orchestra.run.smithi115.stdout:(19/30): ceph-common-18.2.0-0.el9.x86_64.rpm 11 MB/s | 18 MB 00:01 2024-03-22T10:26:57.451 INFO:teuthology.orchestra.run.smithi115.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 182 kB/s | 111 kB 00:00 2024-03-22T10:26:57.568 INFO:teuthology.orchestra.run.smithi115.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-03-22T10:26:57.623 INFO:teuthology.orchestra.run.smithi115.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 423 kB/s | 309 kB 00:00 2024-03-22T10:26:57.694 INFO:teuthology.orchestra.run.smithi115.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 390 kB/s | 49 kB 00:00 2024-03-22T10:26:57.710 INFO:teuthology.orchestra.run.smithi115.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 782 kB/s | 67 kB 00:00 2024-03-22T10:26:57.811 INFO:teuthology.orchestra.run.smithi115.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-22T10:26:57.986 INFO:teuthology.orchestra.run.smithi115.stdout:(26/30): libarrow-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-03-22T10:26:58.036 INFO:teuthology.orchestra.run.smithi115.stdout:(27/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 838 kB 00:00 2024-03-22T10:26:58.120 INFO:teuthology.orchestra.run.smithi115.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-03-22T10:26:58.212 INFO:teuthology.orchestra.run.smithi115.stdout:(29/30): librados2-18.2.0-0.el9.x86_64.rpm 15 MB/s | 3.3 MB 00:00 2024-03-22T10:26:58.279 INFO:teuthology.orchestra.run.smithi115.stdout:(30/30): librbd1-18.2.0-0.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-03-22T10:26:58.287 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:26:58.288 INFO:teuthology.orchestra.run.smithi115.stdout:Total 18 MB/s | 53 MB 00:02 2024-03-22T10:26:58.289 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:26:58.329 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:26:58.347 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:26:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-03-22T10:26:58.347 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:26:58.347 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:26:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-03-22T10:26:58.347 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-03-22T10:26:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-03-22T10:26:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-03-22T10:26:58.347 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-22T10:26:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:26:58.349 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:26:58.350 INFO:teuthology.orchestra.run.smithi150.stdout:Install 28 Packages 2024-03-22T10:26:58.350 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 2 Packages 2024-03-22T10:26:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:26:58.350 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 53 M 2024-03-22T10:26:58.350 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:26:58.359 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:26:58.359 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:26:58.691 INFO:teuthology.orchestra.run.smithi150.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 19 MB/s | 5.2 MB 00:00 2024-03-22T10:26:58.708 INFO:teuthology.orchestra.run.smithi150.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 1.5 MB/s | 24 kB 00:00 2024-03-22T10:26:58.750 INFO:teuthology.orchestra.run.smithi150.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 15 MB/s | 653 kB 00:00 2024-03-22T10:26:58.792 INFO:teuthology.orchestra.run.smithi150.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 11 MB/s | 474 kB 00:00 2024-03-22T10:26:58.870 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:26:58.871 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:26:58.926 INFO:teuthology.orchestra.run.smithi150.stdout:(5/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 15 MB/s | 7.6 MB 00:00 2024-03-22T10:26:58.942 INFO:teuthology.orchestra.run.smithi150.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 2.9 MB/s | 45 kB 00:00 2024-03-22T10:26:58.993 INFO:teuthology.orchestra.run.smithi150.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 2.4 MB/s | 119 kB 00:00 2024-03-22T10:26:59.026 INFO:teuthology.orchestra.run.smithi150.stdout:(8/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 4.7 MB/s | 155 kB 00:00 2024-03-22T10:26:59.068 INFO:teuthology.orchestra.run.smithi150.stdout:(9/30): python3-rados-18.2.0-0.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-22T10:26:59.110 INFO:teuthology.orchestra.run.smithi150.stdout:(10/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 7.2 MB/s | 297 kB 00:00 2024-03-22T10:26:59.135 INFO:teuthology.orchestra.run.smithi150.stdout:(11/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 4.0 MB/s | 99 kB 00:00 2024-03-22T10:26:59.302 INFO:teuthology.orchestra.run.smithi150.stdout:(12/30): librgw2-18.2.0-0.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-03-22T10:26:59.328 INFO:teuthology.orchestra.run.smithi150.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 554 kB/s | 106 kB 00:00 2024-03-22T10:26:59.353 INFO:teuthology.orchestra.run.smithi150.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2024-03-22T10:26:59.370 INFO:teuthology.orchestra.run.smithi150.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-22T10:26:59.420 INFO:teuthology.orchestra.run.smithi150.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 9.9 MB/s | 661 kB 00:00 2024-03-22T10:26:59.462 INFO:teuthology.orchestra.run.smithi150.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 282 kB 00:00 2024-03-22T10:26:59.479 INFO:teuthology.orchestra.run.smithi150.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 743 kB/s | 41 kB 00:00 2024-03-22T10:26:59.829 INFO:teuthology.orchestra.run.smithi150.stdout:(19/30): ceph-common-18.2.0-0.el9.x86_64.rpm 13 MB/s | 18 MB 00:01 2024-03-22T10:26:59.980 INFO:teuthology.orchestra.run.smithi150.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 214 kB/s | 111 kB 00:00 2024-03-22T10:27:00.080 INFO:teuthology.orchestra.run.smithi150.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-03-22T10:27:00.132 INFO:teuthology.orchestra.run.smithi150.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 474 kB/s | 309 kB 00:00 2024-03-22T10:27:00.214 INFO:teuthology.orchestra.run.smithi150.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 828 kB/s | 67 kB 00:00 2024-03-22T10:27:00.239 INFO:teuthology.orchestra.run.smithi150.stdout:(24/30): liboath-2.6.7-2.el9.x86_64.rpm 307 kB/s | 49 kB 00:00 2024-03-22T10:27:00.373 INFO:teuthology.orchestra.run.smithi150.stdout:(25/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-03-22T10:27:00.403 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:27:00.538 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-22T10:27:00.548 INFO:teuthology.orchestra.run.smithi150.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 623 kB/s | 192 kB 00:00 2024-03-22T10:27:00.587 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-22T10:27:00.673 INFO:teuthology.orchestra.run.smithi150.stdout:(27/30): librados2-18.2.0-0.el9.x86_64.rpm 26 MB/s | 3.3 MB 00:00 2024-03-22T10:27:00.723 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-22T10:27:00.774 INFO:teuthology.orchestra.run.smithi150.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-03-22T10:27:00.890 INFO:teuthology.orchestra.run.smithi150.stdout:(29/30): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-03-22T10:27:00.966 INFO:teuthology.orchestra.run.smithi150.stdout:(30/30): librbd1-18.2.0-0.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-03-22T10:27:00.978 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:00.979 INFO:teuthology.orchestra.run.smithi150.stdout:Total 20 MB/s | 53 MB 00:02 2024-03-22T10:27:00.980 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:27:01.052 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:27:01.052 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:27:01.126 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-22T10:27:01.190 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-03-22T10:27:01.315 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-03-22T10:27:01.411 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-03-22T10:27:01.442 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-03-22T10:27:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-03-22T10:27:01.560 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:27:01.560 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:27:01.568 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-22T10:27:01.642 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-03-22T10:27:01.751 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-03-22T10:27:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-03-22T10:27:01.863 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-03-22T10:27:02.046 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-22T10:27:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-22T10:27:02.145 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-22T10:27:02.187 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-22T10:27:02.227 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-22T10:27:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-22T10:27:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-22T10:27:02.709 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-22T10:27:02.741 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-03-22T10:27:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-03-22T10:27:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-03-22T10:27:02.953 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-22T10:27:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:27:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-22T10:27:03.249 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-22T10:27:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-22T10:27:03.434 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-22T10:27:03.460 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-03-22T10:27:03.554 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-03-22T10:27:03.660 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-03-22T10:27:03.688 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-03-22T10:27:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-22T10:27:03.786 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-03-22T10:27:03.911 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-03-22T10:27:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-03-22T10:27:04.046 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-03-22T10:27:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-03-22T10:27:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-22T10:27:04.253 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-03-22T10:27:04.355 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-03-22T10:27:04.382 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-03-22T10:27:04.483 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-03-22T10:27:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-22T10:27:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-22T10:27:04.793 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-22T10:27:04.834 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-22T10:27:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-22T10:27:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-22T10:27:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-22T10:27:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-22T10:27:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-03-22T10:27:05.510 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-03-22T10:27:05.544 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-03-22T10:27:05.604 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-22T10:27:06.063 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-03-22T10:27:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-22T10:27:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-03-22T10:27:06.151 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-03-22T10:27:06.207 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-03-22T10:27:06.307 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-03-22T10:27:06.343 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-03-22T10:27:06.814 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-03-22T10:27:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-03-22T10:27:07.014 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-03-22T10:27:07.014 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-22T10:27:07.014 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-22T10:27:07.014 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:07.064 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-03-22T10:27:08.763 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-03-22T10:27:08.807 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-03-22T10:27:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-03-22T10:27:09.485 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-03-22T10:27:09.624 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-03-22T10:27:09.624 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-22T10:27:09.624 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-22T10:27:09.624 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:09.661 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-03-22T10:27:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-03-22T10:27:18.095 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-03-22T10:27:18.095 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-03-22T10:27:18.095 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-03-22T10:27:18.095 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-03-22T10:27:18.095 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-03-22T10:27:18.095 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-03-22T10:27:18.095 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-03-22T10:27:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:18.939 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-03-22T10:27:19.008 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-03-22T10:27:19.008 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:19.008 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-22T10:27:19.009 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T10:27:19.009 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T10:27:19.009 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:19.013 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-22T10:27:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-22T10:27:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-22T10:27:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-03-22T10:27:20.469 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-03-22T10:27:20.469 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-03-22T10:27:20.469 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-03-22T10:27:20.469 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-03-22T10:27:20.469 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-03-22T10:27:20.470 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-03-22T10:27:20.470 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-03-22T10:27:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-03-22T10:27:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-22T10:27:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-22T10:27:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-22T10:27:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-22T10:27:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-22T10:27:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-03-22T10:27:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-22T10:27:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-03-22T10:27:21.322 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-03-22T10:27:21.419 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-03-22T10:27:21.419 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:21.419 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-22T10:27:21.419 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T10:27:21.419 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T10:27:21.419 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:21.425 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-22T10:27:21.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-22T10:27:21.542 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.543 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.544 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:21.544 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-3.el9.x86_64 2024-03-22T10:27:21.544 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-22T10:27:21.544 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:21.544 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:27:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-22T10:27:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-22T10:27:21.988 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-test 2024-03-22T10:27:22.822 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:30 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:27:22.987 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:27:22.988 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-03-22T10:27:22.988 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:22.988 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2024-03-22T10:27:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:22.990 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 40 M 2024-03-22T10:27:22.990 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 167 M 2024-03-22T10:27:22.990 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:27:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-22T10:27:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-03-22T10:27:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-22T10:27:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-22T10:27:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-03-22T10:27:23.485 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): libcephsqlite-18.2.0-0.el9.x86_64.rpm 730 kB/s | 161 kB 00:00 2024-03-22T10:27:23.619 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 614 kB/s | 217 kB 00:00 2024-03-22T10:27:23.652 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 300 kB 00:00 2024-03-22T10:27:23.669 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-03-22T10:27:23.959 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-22T10:27:23.959 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:23.959 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-03-22T10:27:23.959 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-22T10:27:23.960 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-22T10:27:23.961 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:23.962 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:27:24.378 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-test 2024-03-22T10:27:24.636 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): ceph-test-18.2.0-0.el9.x86_64.rpm 29 MB/s | 40 MB 00:01 2024-03-22T10:27:24.894 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): jq-1.6-16.el9.x86_64.rpm 151 kB/s | 187 kB 00:01 2024-03-22T10:27:24.901 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:24.902 INFO:teuthology.orchestra.run.smithi115.stdout:Total 21 MB/s | 40 MB 00:01 2024-03-22T10:27:24.943 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:27:25.002 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:27:25.002 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:27:25.214 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:30 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:27:25.385 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:27:25.386 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:25.386 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-03-22T10:27:25.386 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:25.386 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout:Install 6 Packages 2024-03-22T10:27:25.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:25.388 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 40 M 2024-03-22T10:27:25.388 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 167 M 2024-03-22T10:27:25.388 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:27:25.418 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:27:25.419 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:27:25.702 INFO:teuthology.orchestra.run.smithi150.stdout:(1/6): libcephsqlite-18.2.0-0.el9.x86_64.rpm 995 kB/s | 161 kB 00:00 2024-03-22T10:27:25.936 INFO:teuthology.orchestra.run.smithi150.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 549 kB/s | 217 kB 00:00 2024-03-22T10:27:25.961 INFO:teuthology.orchestra.run.smithi150.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-03-22T10:27:25.987 INFO:teuthology.orchestra.run.smithi150.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-22T10:27:26.084 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:27:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-22T10:27:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-22T10:27:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T10:27:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-22T10:27:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-03-22T10:27:26.445 INFO:teuthology.orchestra.run.smithi150.stdout:(5/6): ceph-test-18.2.0-0.el9.x86_64.rpm 44 MB/s | 40 MB 00:00 2024-03-22T10:27:26.711 INFO:teuthology.orchestra.run.smithi150.stdout:(6/6): jq-1.6-16.el9.x86_64.rpm 249 kB/s | 187 kB 00:00 2024-03-22T10:27:26.718 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:26.718 INFO:teuthology.orchestra.run.smithi150.stdout:Total 30 MB/s | 40 MB 00:01 2024-03-22T10:27:26.759 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:27:26.815 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:27:26.815 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:27:27.234 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:27:27.235 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:27:27.857 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:27:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-22T10:27:27.995 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-22T10:27:28.069 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T10:27:28.137 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-22T10:27:28.165 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-03-22T10:27:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-03-22T10:27:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-03-22T10:27:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-03-22T10:27:32.483 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-03-22T10:27:32.788 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-03-22T10:27:32.788 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/6 2024-03-22T10:27:32.788 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/6 2024-03-22T10:27:32.788 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T10:27:32.788 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-22T10:27:32.788 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-22T10:27:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-22T10:27:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:33.162 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:27:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:18.2.0-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-03-22T10:27:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:18.2.0-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-03-22T10:27:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-03-22T10:27:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:33.162 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:27:33.520 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph 2024-03-22T10:27:34.395 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:42 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:27:34.517 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-03-22T10:27:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/6 2024-03-22T10:27:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/6 2024-03-22T10:27:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T10:27:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-22T10:27:34.518 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-22T10:27:34.575 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:27:34.577 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-22T10:27:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-22T10:27:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:34.580 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:27:34.581 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:34.581 INFO:teuthology.orchestra.run.smithi115.stdout:Install 33 Packages 2024-03-22T10:27:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:34.581 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 30 M 2024-03-22T10:27:34.581 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 109 M 2024-03-22T10:27:34.581 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:27:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-22T10:27:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:34.819 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:27:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:18.2.0-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-03-22T10:27:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:18.2.0-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-03-22T10:27:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-03-22T10:27:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:34.819 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout:(1/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 10 MB/s | 2.1 MB 00:00 2024-03-22T10:27:34.941 INFO:teuthology.orchestra.run.smithi115.stdout:(2/33): ceph-18.2.0-0.el9.x86_64.rpm 25 kB/s | 6.4 kB 00:00 2024-03-22T10:27:35.128 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph 2024-03-22T10:27:35.308 INFO:teuthology.orchestra.run.smithi115.stdout:(3/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 2.3 MB/s | 1.4 MB 00:00 2024-03-22T10:27:35.366 INFO:teuthology.orchestra.run.smithi115.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 4.1 MB/s | 240 kB 00:00 2024-03-22T10:27:35.534 INFO:teuthology.orchestra.run.smithi115.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-03-22T10:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout:(6/33): python3-mako-1.1.4-6.el9.noarch.rpm 668 kB/s | 144 kB 00:00 2024-03-22T10:27:35.609 INFO:teuthology.orchestra.run.smithi115.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 423 kB/s | 32 kB 00:00 2024-03-22T10:27:35.634 INFO:teuthology.orchestra.run.smithi115.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-03-22T10:27:35.660 INFO:teuthology.orchestra.run.smithi115.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 783 kB/s | 39 kB 00:00 2024-03-22T10:27:35.718 INFO:teuthology.orchestra.run.smithi115.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2024-03-22T10:27:35.977 INFO:teuthology.orchestra.run.smithi115.stdout:(11/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 17 MB/s | 18 MB 00:01 2024-03-22T10:27:35.985 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:40 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:27:36.027 INFO:teuthology.orchestra.run.smithi115.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 3.1 MB/s | 1.1 MB 00:00 2024-03-22T10:27:36.052 INFO:teuthology.orchestra.run.smithi115.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 311 kB/s | 103 kB 00:00 2024-03-22T10:27:36.078 INFO:teuthology.orchestra.run.smithi115.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 1.3 MB/s | 124 kB 00:00 2024-03-22T10:27:36.111 INFO:teuthology.orchestra.run.smithi115.stdout:(15/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.2 MB/s | 187 kB 00:00 2024-03-22T10:27:36.154 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-03-22T10:27:36.157 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-22T10:27:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-22T10:27:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-22T10:27:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-22T10:27:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-22T10:27:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-22T10:27:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-22T10:27:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-22T10:27:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-22T10:27:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-22T10:27:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-22T10:27:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-22T10:27:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-22T10:27:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-22T10:27:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:36.160 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:27:36.160 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:36.160 INFO:teuthology.orchestra.run.smithi150.stdout:Install 33 Packages 2024-03-22T10:27:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:36.161 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 30 M 2024-03-22T10:27:36.161 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 109 M 2024-03-22T10:27:36.161 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:27:36.212 INFO:teuthology.orchestra.run.smithi115.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 330 kB/s | 43 kB 00:00 2024-03-22T10:27:36.312 INFO:teuthology.orchestra.run.smithi115.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 861 kB/s | 172 kB 00:00 2024-03-22T10:27:36.404 INFO:teuthology.orchestra.run.smithi115.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 117 kB/s | 11 kB 00:00 2024-03-22T10:27:36.421 INFO:teuthology.orchestra.run.smithi150.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 43 kB/s | 6.4 kB 00:00 2024-03-22T10:27:36.437 INFO:teuthology.orchestra.run.smithi115.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-03-22T10:27:36.462 INFO:teuthology.orchestra.run.smithi115.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 303 kB/s | 18 kB 00:00 2024-03-22T10:27:36.474 INFO:teuthology.orchestra.run.smithi150.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 7.1 MB/s | 1.4 MB 00:00 2024-03-22T10:27:36.488 INFO:teuthology.orchestra.run.smithi115.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 465 kB/s | 23 kB 00:00 2024-03-22T10:27:36.514 INFO:teuthology.orchestra.run.smithi115.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 390 kB/s | 19 kB 00:00 2024-03-22T10:27:36.533 INFO:teuthology.orchestra.run.smithi150.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 8.0 MB/s | 2.1 MB 00:00 2024-03-22T10:27:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 395 kB/s | 20 kB 00:00 2024-03-22T10:27:36.566 INFO:teuthology.orchestra.run.smithi150.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 7.1 MB/s | 240 kB 00:00 2024-03-22T10:27:36.572 INFO:teuthology.orchestra.run.smithi115.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 797 kB/s | 46 kB 00:00 2024-03-22T10:27:36.598 INFO:teuthology.orchestra.run.smithi115.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-03-22T10:27:36.648 INFO:teuthology.orchestra.run.smithi115.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 332 kB/s | 16 kB 00:00 2024-03-22T10:27:36.682 INFO:teuthology.orchestra.run.smithi115.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-03-22T10:27:36.707 INFO:teuthology.orchestra.run.smithi115.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-03-22T10:27:36.717 INFO:teuthology.orchestra.run.smithi150.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-03-22T10:27:36.732 INFO:teuthology.orchestra.run.smithi115.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 720 kB/s | 36 kB 00:00 2024-03-22T10:27:36.767 INFO:teuthology.orchestra.run.smithi150.stdout:(6/33): python3-mako-1.1.4-6.el9.noarch.rpm 722 kB/s | 144 kB 00:00 2024-03-22T10:27:36.791 INFO:teuthology.orchestra.run.smithi115.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-22T10:27:36.801 INFO:teuthology.orchestra.run.smithi150.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 380 kB/s | 32 kB 00:00 2024-03-22T10:27:36.825 INFO:teuthology.orchestra.run.smithi115.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.6 MB/s | 427 kB 00:00 2024-03-22T10:27:36.826 INFO:teuthology.orchestra.run.smithi150.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 813 kB/s | 47 kB 00:00 2024-03-22T10:27:36.850 INFO:teuthology.orchestra.run.smithi115.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 345 kB/s | 20 kB 00:00 2024-03-22T10:27:36.851 INFO:teuthology.orchestra.run.smithi150.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 786 kB/s | 39 kB 00:00 2024-03-22T10:27:36.885 INFO:teuthology.orchestra.run.smithi150.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-22T10:27:36.942 INFO:teuthology.orchestra.run.smithi115.stdout:(33/33): python3-requests-2.25.1-8.el9.noarch.r 141 kB/s | 125 kB 00:00 2024-03-22T10:27:36.950 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:36.951 INFO:teuthology.orchestra.run.smithi115.stdout:Total 13 MB/s | 30 MB 00:02 2024-03-22T10:27:36.952 INFO:teuthology.orchestra.run.smithi150.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-03-22T10:27:36.977 INFO:teuthology.orchestra.run.smithi150.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-22T10:27:36.991 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:27:37.003 INFO:teuthology.orchestra.run.smithi150.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.5 MB/s | 124 kB 00:00 2024-03-22T10:27:37.036 INFO:teuthology.orchestra.run.smithi150.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.2 MB/s | 187 kB 00:00 2024-03-22T10:27:37.077 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:27:37.078 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:27:37.153 INFO:teuthology.orchestra.run.smithi150.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 846 kB/s | 125 kB 00:00 2024-03-22T10:27:37.412 INFO:teuthology.orchestra.run.smithi150.stdout:(16/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 19 MB/s | 18 MB 00:00 2024-03-22T10:27:37.530 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:27:37.531 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:27:37.779 INFO:teuthology.orchestra.run.smithi150.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 59 kB/s | 43 kB 00:00 2024-03-22T10:27:37.937 INFO:teuthology.orchestra.run.smithi150.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 69 kB/s | 11 kB 00:00 2024-03-22T10:27:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 106 kB/s | 18 kB 00:00 2024-03-22T10:27:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 173 kB/s | 172 kB 00:00 2024-03-22T10:27:38.280 INFO:teuthology.orchestra.run.smithi150.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 133 kB/s | 23 kB 00:00 2024-03-22T10:27:38.347 INFO:teuthology.orchestra.run.smithi150.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 123 kB/s | 19 kB 00:00 2024-03-22T10:27:38.464 INFO:teuthology.orchestra.run.smithi150.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 364 kB/s | 358 kB 00:00 2024-03-22T10:27:38.489 INFO:teuthology.orchestra.run.smithi150.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 94 kB/s | 20 kB 00:00 2024-03-22T10:27:38.514 INFO:teuthology.orchestra.run.smithi150.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 278 kB/s | 46 kB 00:00 2024-03-22T10:27:38.656 INFO:teuthology.orchestra.run.smithi150.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 116 kB/s | 16 kB 00:00 2024-03-22T10:27:38.690 INFO:teuthology.orchestra.run.smithi150.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-03-22T10:27:38.756 INFO:teuthology.orchestra.run.smithi150.stdout:(28/33): python3-more-itertools-8.12.0-2.el9.no 270 kB/s | 79 kB 00:00 2024-03-22T10:27:38.849 INFO:teuthology.orchestra.run.smithi150.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 226 kB/s | 36 kB 00:00 2024-03-22T10:27:38.866 INFO:teuthology.orchestra.run.smithi150.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 432 kB/s | 90 kB 00:00 2024-03-22T10:27:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:27:38.976 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-22T10:27:39.008 INFO:teuthology.orchestra.run.smithi150.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 141 kB/s | 20 kB 00:00 2024-03-22T10:27:39.027 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-22T10:27:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-22T10:27:39.082 INFO:teuthology.orchestra.run.smithi150.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.8 MB/s | 427 kB 00:00 2024-03-22T10:27:39.108 INFO:teuthology.orchestra.run.smithi150.stdout:(33/33): python3-webob-1.8.7-6.el9.noarch.rpm 655 kB/s | 230 kB 00:00 2024-03-22T10:27:39.121 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:39.121 INFO:teuthology.orchestra.run.smithi150.stdout:Total 10 MB/s | 30 MB 00:02 2024-03-22T10:27:39.162 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:27:39.220 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-22T10:27:39.246 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:27:39.246 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:27:39.326 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-22T10:27:39.385 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-22T10:27:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-22T10:27:39.555 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-22T10:27:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-22T10:27:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-22T10:27:39.686 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:27:39.687 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:27:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-22T10:27:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-22T10:27:40.085 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-22T10:27:40.123 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-22T10:27:40.166 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-22T10:27:40.204 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-22T10:27:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-22T10:27:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T10:27:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-22T10:27:40.432 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-22T10:27:40.496 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-22T10:27:40.536 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-22T10:27:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-22T10:27:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-22T10:27:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-22T10:27:41.006 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-22T10:27:41.101 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:27:41.158 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T10:27:41.170 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-22T10:27:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-22T10:27:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-22T10:27:41.344 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-03-22T10:27:41.382 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-03-22T10:27:41.430 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-22T10:27:41.464 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-03-22T10:27:41.464 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:41.464 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-22T10:27:41.464 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T10:27:41.464 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T10:27:41.464 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-22T10:27:41.592 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-22T10:27:41.689 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-22T10:27:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-22T10:27:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-22T10:27:41.899 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-22T10:27:41.995 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-22T10:27:42.250 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-22T10:27:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-22T10:27:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-22T10:27:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-22T10:27:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-22T10:27:42.544 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-22T10:27:42.579 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T10:27:42.614 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-22T10:27:42.662 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-22T10:27:42.717 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-22T10:27:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-22T10:27:42.887 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-22T10:27:42.970 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-22T10:27:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-22T10:27:43.117 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-03-22T10:27:43.196 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-03-22T10:27:43.196 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:43.196 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-22T10:27:43.196 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T10:27:43.196 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T10:27:43.196 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:43.269 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-22T10:27:43.430 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T10:27:43.615 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-03-22T10:27:43.647 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-03-22T10:27:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-03-22T10:27:43.719 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-03-22T10:27:43.719 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:43.719 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-22T10:27:43.719 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T10:27:43.719 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T10:27:43.719 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:43.738 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-03-22T10:27:43.738 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:43.739 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-22T10:27:43.739 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T10:27:43.739 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T10:27:43.739 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:43.960 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-03-22T10:27:44.026 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-03-22T10:27:44.026 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:44.026 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-22T10:27:44.026 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T10:27:44.026 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T10:27:44.027 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:44.107 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-03-22T10:27:44.211 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-03-22T10:27:44.212 INFO:teuthology.orchestra.run.smithi115.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-22T10:27:44.212 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-03-22T10:27:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-03-22T10:27:45.425 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:45.425 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-22T10:27:45.425 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T10:27:45.425 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T10:27:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:45.923 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-03-22T10:27:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-03-22T10:27:46.010 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:46.011 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-22T10:27:46.011 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T10:27:46.011 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T10:27:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-03-22T10:27:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-03-22T10:27:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-03-22T10:27:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-03-22T10:27:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T10:27:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-22T10:27:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-22T10:27:46.232 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-03-22T10:27:46.299 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-03-22T10:27:46.299 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:46.300 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-22T10:27:46.300 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T10:27:46.300 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T10:27:46.300 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-03-22T10:27:46.504 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-03-22T10:27:46.504 INFO:teuthology.orchestra.run.smithi150.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-22T10:27:46.504 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-22T10:27:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-22T10:27:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-22T10:27:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-22T10:27:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-22T10:27:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-22T10:27:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-22T10:27:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-22T10:27:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-22T10:27:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:47.318 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:27:47.691 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-base 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-22T10:27:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-22T10:27:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-22T10:27:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-22T10:27:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T10:27:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-22T10:27:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-22T10:27:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-22T10:27:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-22T10:27:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-22T10:27:48.523 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:56 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:27:48.618 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:27:48.671 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:27:48.672 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-03-22T10:27:48.673 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:27:48.741 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install cephadm 2024-03-22T10:27:49.534 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:57 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:27:49.569 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-22T10:27:49.569 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:49.569 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:27:49.569 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:49.569 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:49.569 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:49.569 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-03-22T10:27:49.569 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-22T10:27:49.570 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-22T10:27:49.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-22T10:27:49.572 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:49.572 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:27:49.685 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:27:49.685 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 209 k 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 200 k 2024-03-22T10:27:49.686 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:27:49.785 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.1 MB/s | 209 kB 00:00 2024-03-22T10:27:49.786 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:49.786 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.0 MB/s | 209 kB 00:00 2024-03-22T10:27:49.786 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:27:49.793 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:27:49.793 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:27:49.796 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:27:49.796 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:27:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:27:49.972 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-base 2024-03-22T10:27:50.556 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-03-22T10:27:50.686 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-03-22T10:27:50.832 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:55 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:27:50.928 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:27:50.982 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:27:50.984 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-03-22T10:27:50.984 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:27:51.054 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install cephadm 2024-03-22T10:27:51.138 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-03-22T10:27:51.446 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-03-22T10:27:51.446 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:51.446 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:27:51.446 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-03-22T10:27:51.446 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:51.446 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:27:51.753 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-immutable-object-cache 2024-03-22T10:27:51.884 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:56 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:27:52.033 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:27:52.034 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:52.034 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-03-22T10:27:52.034 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:52.034 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:27:52.034 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-03-22T10:27:52.034 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:52.035 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:27:52.035 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:52.035 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-03-22T10:27:52.035 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:52.035 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 209 k 2024-03-22T10:27:52.035 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 200 k 2024-03-22T10:27:52.035 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:27:52.122 INFO:teuthology.orchestra.run.smithi150.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.4 MB/s | 209 kB 00:00 2024-03-22T10:27:52.122 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:52.123 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.3 MB/s | 209 kB 00:00 2024-03-22T10:27:52.123 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:27:52.129 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:27:52.129 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:27:52.132 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:27:52.133 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:27:52.220 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:27:52.616 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:00 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:27:52.778 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-03-22T10:27:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:52.780 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 142 k 2024-03-22T10:27:52.780 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 399 k 2024-03-22T10:27:52.780 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:27:52.852 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-03-22T10:27:52.875 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 1.5 MB/s | 142 kB 00:00 2024-03-22T10:27:52.876 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:52.876 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.4 MB/s | 142 kB 00:00 2024-03-22T10:27:52.877 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:27:52.887 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:27:52.887 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:27:52.952 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:27:52.952 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:27:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-03-22T10:27:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:27:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:27:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:27:53.234 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:53.234 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-22T10:27:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-03-22T10:27:53.786 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-03-22T10:27:53.786 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:53.787 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:27:53.787 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-03-22T10:27:53.787 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:53.787 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:27:54.041 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-immutable-object-cache 2024-03-22T10:27:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:27:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:54.496 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:27:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:54.496 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:27:54.751 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr 2024-03-22T10:27:54.894 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:59 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:27:55.050 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:27:55.051 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-03-22T10:27:55.051 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:55.051 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:27:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-03-22T10:27:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:55.052 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:27:55.052 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:55.052 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-03-22T10:27:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:55.052 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 142 k 2024-03-22T10:27:55.052 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 399 k 2024-03-22T10:27:55.052 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:27:55.127 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 1.9 MB/s | 142 kB 00:00 2024-03-22T10:27:55.128 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:55.128 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.8 MB/s | 142 kB 00:00 2024-03-22T10:27:55.129 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:27:55.137 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:27:55.137 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:27:55.195 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:27:55.195 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:27:55.340 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:27:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:27:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:27:55.473 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:27:55.473 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-22T10:27:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:55.599 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:03 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:27:55.695 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:27:55.749 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:27:55.750 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-03-22T10:27:55.750 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:27:55.815 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-dashboard 2024-03-22T10:27:56.634 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:04 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:27:56.728 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:27:56.728 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:56.728 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:27:56.729 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-03-22T10:27:56.729 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:56.729 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:27:56.790 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:27:56.790 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-22T10:27:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:56.792 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:27:56.792 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:27:56.792 INFO:teuthology.orchestra.run.smithi115.stdout:Install 7 Packages 2024-03-22T10:27:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:56.792 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.0 M 2024-03-22T10:27:56.792 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 8.1 M 2024-03-22T10:27:56.793 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:27:57.031 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr 2024-03-22T10:27:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 144 kB/s | 23 kB 00:00 2024-03-22T10:27:57.097 INFO:teuthology.orchestra.run.smithi115.stdout:(2/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 7.8 MB/s | 1.7 MB 00:00 2024-03-22T10:27:57.122 INFO:teuthology.orchestra.run.smithi115.stdout:(3/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 60 kB/s | 15 kB 00:00 2024-03-22T10:27:57.164 INFO:teuthology.orchestra.run.smithi115.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 72 kB/s | 9.0 kB 00:00 2024-03-22T10:27:57.248 INFO:teuthology.orchestra.run.smithi115.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 272 kB/s | 41 kB 00:00 2024-03-22T10:27:57.273 INFO:teuthology.orchestra.run.smithi115.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 205 kB/s | 31 kB 00:00 2024-03-22T10:27:57.340 INFO:teuthology.orchestra.run.smithi115.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.1 MB/s | 188 kB 00:00 2024-03-22T10:27:57.344 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:27:57.345 INFO:teuthology.orchestra.run.smithi115.stdout:Total 3.6 MB/s | 2.0 MB 00:00 2024-03-22T10:27:57.346 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:27:57.372 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:27:57.372 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:27:57.506 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:27:57.507 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:27:57.869 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:02 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:27:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:27:57.965 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:27:57.976 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-22T10:27:58.019 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:27:58.020 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-03-22T10:27:58.020 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:27:58.037 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-22T10:27:58.082 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-dashboard 2024-03-22T10:27:58.141 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-22T10:27:58.174 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-22T10:27:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-03-22T10:27:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-03-22T10:27:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-03-22T10:27:58.902 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:03 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:27:59.061 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:27:59.062 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:59.062 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-03-22T10:27:59.062 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:59.062 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:27:59.062 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-03-22T10:27:59.062 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout:Install 7 Packages 2024-03-22T10:27:59.063 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:27:59.064 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 2.0 M 2024-03-22T10:27:59.064 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 8.1 M 2024-03-22T10:27:59.064 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:27:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-03-22T10:27:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-03-22T10:27:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-03-22T10:27:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-03-22T10:27:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-22T10:27:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-22T10:27:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-22T10:27:59.298 INFO:teuthology.orchestra.run.smithi150.stdout:(1/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 102 kB/s | 15 kB 00:00 2024-03-22T10:27:59.322 INFO:teuthology.orchestra.run.smithi150.stdout:(2/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 137 kB/s | 23 kB 00:00 2024-03-22T10:27:59.373 INFO:teuthology.orchestra.run.smithi150.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 7.8 MB/s | 1.7 MB 00:00 2024-03-22T10:27:59.532 INFO:teuthology.orchestra.run.smithi150.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 39 kB/s | 9.0 kB 00:00 2024-03-22T10:27:59.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:27:59.539 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:27:59.590 INFO:teuthology.orchestra.run.smithi150.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 153 kB/s | 41 kB 00:00 2024-03-22T10:27:59.640 INFO:teuthology.orchestra.run.smithi150.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 115 kB/s | 31 kB 00:00 2024-03-22T10:27:59.819 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-22T10:28:00.040 INFO:teuthology.orchestra.run.smithi150.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 371 kB/s | 188 kB 00:00 2024-03-22T10:28:00.046 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:00.047 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.0 MB/s | 2.0 MB 00:00 2024-03-22T10:28:00.048 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:00.074 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:00.074 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:00.210 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:00.210 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:00.604 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:00.679 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:08 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:00.709 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-22T10:28:00.767 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-22T10:28:00.835 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:00.837 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:00.837 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-03-22T10:28:00.837 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:00.837 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:28:00.837 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-03-22T10:28:00.837 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-03-22T10:28:00.837 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:28:00.838 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:00.839 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-03-22T10:28:00.839 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:00.839 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 37 M 2024-03-22T10:28:00.839 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 209 M 2024-03-22T10:28:00.839 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:28:00.875 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-22T10:28:00.915 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-22T10:28:00.965 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-03-22T10:28:01.276 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-03-22T10:28:01.320 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-03-22T10:28:01.339 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 145 kB/s | 32 kB 00:00 2024-03-22T10:28:01.373 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 539 kB/s | 17 kB 00:00 2024-03-22T10:28:01.415 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 837 kB/s | 34 kB 00:00 2024-03-22T10:28:01.606 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2024-03-22T10:28:01.790 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): ceph-mgr-diskprediction-local-18.2.0-0. 11 MB/s | 7.4 MB 00:00 2024-03-22T10:28:01.882 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.9 MB/s | 3.0 MB 00:00 2024-03-22T10:28:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-03-22T10:28:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-03-22T10:28:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-03-22T10:28:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-03-22T10:28:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-22T10:28:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-22T10:28:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-22T10:28:01.990 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 4.8 MB 00:00 2024-03-22T10:28:02.024 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.0 MB/s | 217 kB 00:00 2024-03-22T10:28:02.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-22T10:28:02.301 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:02.301 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:02.301 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-03-22T10:28:02.301 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-03-22T10:28:02.301 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-03-22T10:28:02.301 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-22T10:28:02.302 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-22T10:28:02.302 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-22T10:28:02.302 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-22T10:28:02.302 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:02.302 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:02.449 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-03-22T10:28:02.541 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): python3-devel-3.9.17-2.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-03-22T10:28:02.588 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-22T10:28:02.816 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 251 kB/s | 195 kB 00:00 2024-03-22T10:28:03.058 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 755 kB/s | 803 kB 00:01 2024-03-22T10:28:03.063 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:03.064 INFO:teuthology.orchestra.run.smithi115.stdout:Total 17 MB/s | 37 MB 00:02 2024-03-22T10:28:03.184 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:28:03.248 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:28:03.248 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:28:03.410 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:08 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-03-22T10:28:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-22T10:28:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-22T10:28:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-22T10:28:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:28:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:Install 12 Packages 2024-03-22T10:28:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 37 M 2024-03-22T10:28:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 209 M 2024-03-22T10:28:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:28:03.779 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:28:03.780 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:28:03.881 INFO:teuthology.orchestra.run.smithi150.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 191 kB/s | 32 kB 00:00 2024-03-22T10:28:03.922 INFO:teuthology.orchestra.run.smithi150.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 419 kB/s | 17 kB 00:00 2024-03-22T10:28:03.956 INFO:teuthology.orchestra.run.smithi150.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-22T10:28:04.073 INFO:teuthology.orchestra.run.smithi150.stdout:(4/12): ceph-mgr-diskprediction-local-18.2.0-0. 21 MB/s | 7.4 MB 00:00 2024-03-22T10:28:04.215 INFO:teuthology.orchestra.run.smithi150.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.0 MB/s | 3.0 MB 00:00 2024-03-22T10:28:04.432 INFO:teuthology.orchestra.run.smithi150.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 10 MB/s | 4.7 MB 00:00 2024-03-22T10:28:04.474 INFO:teuthology.orchestra.run.smithi150.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 889 kB/s | 217 kB 00:00 2024-03-22T10:28:04.583 INFO:teuthology.orchestra.run.smithi150.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.6 MB/s | 4.8 MB 00:00 2024-03-22T10:28:04.639 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:28:04.703 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-22T10:28:04.739 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-22T10:28:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-22T10:28:04.926 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-22T10:28:04.958 INFO:teuthology.orchestra.run.smithi150.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-03-22T10:28:04.983 INFO:teuthology.orchestra.run.smithi150.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 506 kB/s | 195 kB 00:00 2024-03-22T10:28:05.042 INFO:teuthology.orchestra.run.smithi150.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-03-22T10:28:05.171 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T10:28:05.207 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T10:28:05.312 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-22T10:28:05.958 INFO:teuthology.orchestra.run.smithi150.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 245 kB/s | 245 kB 00:00 2024-03-22T10:28:05.965 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:05.966 INFO:teuthology.orchestra.run.smithi150.stdout:Total 15 MB/s | 37 MB 00:02 2024-03-22T10:28:06.086 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:06.150 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:06.150 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:06.160 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-22T10:28:06.299 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-22T10:28:06.656 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:06.657 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-22T10:28:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-22T10:28:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-22T10:28:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-22T10:28:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-22T10:28:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T10:28:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T10:28:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-22T10:28:08.985 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-22T10:28:09.014 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-03-22T10:28:09.059 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-22T10:28:09.184 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-22T10:28:10.697 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-22T10:28:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-03-22T10:28:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-03-22T10:28:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-22T10:28:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-22T10:28:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-22T10:28:11.756 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T10:28:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T10:28:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-22T10:28:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-22T10:28:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-22T10:28:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-22T10:28:11.757 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-22T10:28:11.833 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-22T10:28:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-22T10:28:12.244 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-22T10:28:12.245 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-22T10:28:12.245 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-22T10:28:12.245 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-22T10:28:12.245 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-22T10:28:12.245 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:12.245 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:12.686 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-rook 2024-03-22T10:28:13.526 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:21 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:13.686 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:13.687 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:13.687 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-03-22T10:28:13.687 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:13.687 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:28:13.687 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-03-22T10:28:13.687 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-22T10:28:13.688 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:13.689 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:28:13.689 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:13.689 INFO:teuthology.orchestra.run.smithi115.stdout:Install 13 Packages 2024-03-22T10:28:13.689 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:13.689 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.0 M 2024-03-22T10:28:13.689 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 26 M 2024-03-22T10:28:13.689 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:28:14.284 INFO:teuthology.orchestra.run.smithi115.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 228 kB/s | 47 kB 00:00 2024-03-22T10:28:14.309 INFO:teuthology.orchestra.run.smithi115.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 105 kB/s | 24 kB 00:00 2024-03-22T10:28:14.334 INFO:teuthology.orchestra.run.smithi115.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 70 kB/s | 18 kB 00:00 2024-03-22T10:28:14.401 INFO:teuthology.orchestra.run.smithi115.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 174 kB 00:00 2024-03-22T10:28:14.427 INFO:teuthology.orchestra.run.smithi115.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 132 kB 00:00 2024-03-22T10:28:14.453 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-22T10:28:14.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-22T10:28:14.455 INFO:teuthology.orchestra.run.smithi115.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.8 MB/s | 211 kB 00:00 2024-03-22T10:28:14.477 INFO:teuthology.orchestra.run.smithi115.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 579 kB/s | 43 kB 00:00 2024-03-22T10:28:14.769 INFO:teuthology.orchestra.run.smithi115.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 47 kB/s | 15 kB 00:00 2024-03-22T10:28:14.794 INFO:teuthology.orchestra.run.smithi115.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 88 kB/s | 32 kB 00:00 2024-03-22T10:28:14.920 INFO:teuthology.orchestra.run.smithi115.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 474 kB/s | 59 kB 00:00 2024-03-22T10:28:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 426 kB/s | 203 kB 00:00 2024-03-22T10:28:14.969 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-22T10:28:14.969 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:14.969 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:14.969 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:14.970 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:15.012 INFO:teuthology.orchestra.run.smithi115.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 985 kB/s | 90 kB 00:00 2024-03-22T10:28:15.120 INFO:teuthology.orchestra.run.smithi115.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.9 MB/s | 1.0 MB 00:00 2024-03-22T10:28:15.130 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:15.131 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2024-03-22T10:28:15.132 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:28:15.172 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:28:15.172 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:28:15.321 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:28:15.321 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:28:15.390 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-rook 2024-03-22T10:28:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:28:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-22T10:28:16.226 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:21 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:16.272 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-22T10:28:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-22T10:28:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-22T10:28:16.384 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:16.386 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-03-22T10:28:16.386 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:16.386 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:28:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-03-22T10:28:16.386 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-03-22T10:28:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-22T10:28:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-22T10:28:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout:Install 13 Packages 2024-03-22T10:28:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:16.388 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 2.0 M 2024-03-22T10:28:16.388 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 26 M 2024-03-22T10:28:16.388 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:28:16.423 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-22T10:28:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-22T10:28:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-22T10:28:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-22T10:28:16.957 INFO:teuthology.orchestra.run.smithi150.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 243 kB/s | 47 kB 00:00 2024-03-22T10:28:16.982 INFO:teuthology.orchestra.run.smithi150.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 111 kB/s | 24 kB 00:00 2024-03-22T10:28:17.007 INFO:teuthology.orchestra.run.smithi150.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 74 kB/s | 18 kB 00:00 2024-03-22T10:28:17.032 INFO:teuthology.orchestra.run.smithi150.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-22T10:28:17.066 INFO:teuthology.orchestra.run.smithi150.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-22T10:28:17.091 INFO:teuthology.orchestra.run.smithi150.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 211 kB 00:00 2024-03-22T10:28:17.141 INFO:teuthology.orchestra.run.smithi150.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 404 kB/s | 43 kB 00:00 2024-03-22T10:28:17.266 INFO:teuthology.orchestra.run.smithi150.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 185 kB/s | 32 kB 00:00 2024-03-22T10:28:17.658 INFO:teuthology.orchestra.run.smithi150.stdout:(9/13): python3-kubernetes-26.1.0-2.el9.noarch. 2.6 MB/s | 1.0 MB 00:00 2024-03-22T10:28:17.674 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-22T10:28:17.717 INFO:teuthology.orchestra.run.smithi150.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-03-22T10:28:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-22T10:28:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-22T10:28:17.784 INFO:teuthology.orchestra.run.smithi150.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-03-22T10:28:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-22T10:28:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-03-22T10:28:18.292 INFO:teuthology.orchestra.run.smithi150.stdout:(12/13): python3-certifi-2021.10.8-2.el9.noarch 12 kB/s | 15 kB 00:01 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-22T10:28:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-22T10:28:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-22T10:28:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-22T10:28:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-22T10:28:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-22T10:28:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-22T10:28:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-22T10:28:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:19.192 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:19.434 INFO:teuthology.orchestra.run.smithi150.stdout:(13/13): python3-google-auth-2.28.2-1.el9.noarc 88 kB/s | 203 kB 00:02 2024-03-22T10:28:19.438 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:19.439 INFO:teuthology.orchestra.run.smithi150.stdout:Total 686 kB/s | 2.0 MB 00:03 2024-03-22T10:28:19.440 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:19.483 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:19.483 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:19.529 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-cephadm 2024-03-22T10:28:19.639 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:19.640 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:20.317 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:20.364 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:28 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:20.524 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:20.525 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:28:20.526 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:20.526 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2024-03-22T10:28:20.526 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:20.526 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 6.8 M 2024-03-22T10:28:20.526 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 33 M 2024-03-22T10:28:20.526 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:28:20.547 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-22T10:28:20.637 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-22T10:28:20.709 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-22T10:28:20.748 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-22T10:28:20.795 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-22T10:28:20.920 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-22T10:28:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-22T10:28:21.082 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 741 kB/s | 127 kB 00:00 2024-03-22T10:28:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-22T10:28:21.265 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-03-22T10:28:21.307 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.4 MB/s | 548 kB 00:00 2024-03-22T10:28:21.349 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 1.2 MB/s | 51 kB 00:00 2024-03-22T10:28:21.374 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 535 kB/s | 58 kB 00:00 2024-03-22T10:28:21.816 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 274 kB/s | 248 kB 00:00 2024-03-22T10:28:21.819 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:21.820 INFO:teuthology.orchestra.run.smithi115.stdout:Total 5.2 MB/s | 6.8 MB 00:01 2024-03-22T10:28:21.861 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:28:21.895 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:28:21.895 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:28:22.035 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:28:22.042 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:28:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-22T10:28:22.128 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-22T10:28:22.161 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-22T10:28:22.214 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-22T10:28:22.238 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-03-22T10:28:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:28:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-22T10:28:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-22T10:28:23.163 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-22T10:28:23.164 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-22T10:28:23.404 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-22T10:28:23.505 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-22T10:28:23.594 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-22T10:28:23.622 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-03-22T10:28:23.630 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-22T10:28:23.630 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-22T10:28:23.631 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-22T10:28:23.632 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-22T10:28:23.632 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:23.632 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:23.946 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-cephadm 2024-03-22T10:28:24.466 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-03-22T10:28:24.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-03-22T10:28:24.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-22T10:28:24.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-22T10:28:24.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-22T10:28:24.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-22T10:28:24.789 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-22T10:28:24.789 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:24.790 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:28:24.790 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-03-22T10:28:24.790 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-22T10:28:24.790 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-22T10:28:24.790 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-22T10:28:24.790 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-22T10:28:24.790 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-22T10:28:24.790 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:24.790 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:24.817 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:29 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:24.975 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:24.976 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-03-22T10:28:24.976 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout:Install 6 Packages 2024-03-22T10:28:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:24.978 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 6.8 M 2024-03-22T10:28:24.978 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 33 M 2024-03-22T10:28:24.978 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:28:25.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-fuse 2024-03-22T10:28:25.653 INFO:teuthology.orchestra.run.smithi150.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 759 kB/s | 127 kB 00:00 2024-03-22T10:28:25.737 INFO:teuthology.orchestra.run.smithi150.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 989 kB/s | 248 kB 00:00 2024-03-22T10:28:25.929 INFO:teuthology.orchestra.run.smithi150.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2024-03-22T10:28:25.947 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:33 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:26.108 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:26.108 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:26.108 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:26.108 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:26.108 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:28:26.109 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-03-22T10:28:26.109 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-03-22T10:28:26.109 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-22T10:28:26.109 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:26.109 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:28:26.109 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:26.109 INFO:teuthology.orchestra.run.smithi115.stdout:Install 2 Packages 2024-03-22T10:28:26.109 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:26.110 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 913 k 2024-03-22T10:28:26.110 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 2.7 M 2024-03-22T10:28:26.110 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:28:26.170 INFO:teuthology.orchestra.run.smithi150.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 135 kB/s | 58 kB 00:00 2024-03-22T10:28:26.234 INFO:teuthology.orchestra.run.smithi115.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 6.6 MB/s | 835 kB 00:00 2024-03-22T10:28:26.237 INFO:teuthology.orchestra.run.smithi150.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 166 kB/s | 51 kB 00:00 2024-03-22T10:28:26.276 INFO:teuthology.orchestra.run.smithi115.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 474 kB/s | 79 kB 00:00 2024-03-22T10:28:26.278 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:26.279 INFO:teuthology.orchestra.run.smithi115.stdout:Total 5.3 MB/s | 913 kB 00:00 2024-03-22T10:28:26.279 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:28:26.320 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:28:26.320 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:28:26.420 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:28:26.420 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:28:26.437 INFO:teuthology.orchestra.run.smithi150.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 702 kB/s | 548 kB 00:00 2024-03-22T10:28:26.440 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:26.441 INFO:teuthology.orchestra.run.smithi150.stdout:Total 4.6 MB/s | 6.8 MB 00:01 2024-03-22T10:28:26.482 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:26.517 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:26.518 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:28:26.659 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:26.659 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-22T10:28:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-03-22T10:28:27.040 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:27.245 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-22T10:28:27.303 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-22T10:28:27.773 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-03-22T10:28:27.773 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-03-22T10:28:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-22T10:28:28.061 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-22T10:28:28.061 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:28.061 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:28:28.061 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-22T10:28:28.062 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:28.062 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:28.086 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-22T10:28:28.195 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-22T10:28:28.235 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-03-22T10:28:28.396 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados-devel 2024-03-22T10:28:29.099 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-03-22T10:28:29.099 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-03-22T10:28:29.099 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-22T10:28:29.099 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-22T10:28:29.099 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-22T10:28:29.099 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-22T10:28:29.249 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:37 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:29.402 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-03-22T10:28:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:29.404 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 126 k 2024-03-22T10:28:29.404 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 456 k 2024-03-22T10:28:29.404 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:28:29.454 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-22T10:28:29.454 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:29.454 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:29.454 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-03-22T10:28:29.454 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-22T10:28:29.454 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-22T10:28:29.455 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-22T10:28:29.455 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-22T10:28:29.455 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-22T10:28:29.455 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:29.455 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:29.499 INFO:teuthology.orchestra.run.smithi115.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.3 MB/s | 126 kB 00:00 2024-03-22T10:28:29.500 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:29.500 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.3 MB/s | 126 kB 00:00 2024-03-22T10:28:29.500 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:28:29.507 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:28:29.507 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:28:29.574 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:28:29.575 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:28:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:28:29.771 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-fuse 2024-03-22T10:28:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:30.619 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:35 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:30.780 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:30.780 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:30.780 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:30.780 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:28:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-03-22T10:28:30.781 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-03-22T10:28:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-22T10:28:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:30.781 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:28:30.781 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:30.781 INFO:teuthology.orchestra.run.smithi150.stdout:Install 2 Packages 2024-03-22T10:28:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:30.782 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 913 k 2024-03-22T10:28:30.782 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 2.7 M 2024-03-22T10:28:30.782 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:28:30.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:30.797 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:30.797 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:28:30.797 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:30.797 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:30.797 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:30.890 INFO:teuthology.orchestra.run.smithi150.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 7.6 MB/s | 835 kB 00:00 2024-03-22T10:28:30.948 INFO:teuthology.orchestra.run.smithi150.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 476 kB/s | 79 kB 00:00 2024-03-22T10:28:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:30.951 INFO:teuthology.orchestra.run.smithi150.stdout:Total 5.2 MB/s | 913 kB 00:00 2024-03-22T10:28:30.952 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:31.009 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:31.109 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:31.109 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:31.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs2 2024-03-22T10:28:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-22T10:28:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-03-22T10:28:31.985 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:39 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:32.086 INFO:teuthology.orchestra.run.smithi115.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:32.142 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:32.144 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-03-22T10:28:32.144 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:32.225 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs-devel 2024-03-22T10:28:32.511 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-03-22T10:28:32.512 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-03-22T10:28:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-22T10:28:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:32.819 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-22T10:28:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:32.819 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:33.020 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:41 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:33.130 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados-devel 2024-03-22T10:28:33.172 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-03-22T10:28:33.173 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:33.174 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 30 k 2024-03-22T10:28:33.174 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 137 k 2024-03-22T10:28:33.174 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:28:33.262 INFO:teuthology.orchestra.run.smithi115.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 348 kB/s | 30 kB 00:00 2024-03-22T10:28:33.262 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:33.263 INFO:teuthology.orchestra.run.smithi115.stdout:Total 342 kB/s | 30 kB 00:00 2024-03-22T10:28:33.263 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:28:33.267 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:28:33.267 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:28:33.304 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:28:33.304 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:28:33.431 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:28:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:33.988 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:38 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:34.054 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:34.141 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:34.142 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 126 k 2024-03-22T10:28:34.143 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 456 k 2024-03-22T10:28:34.143 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:28:34.210 INFO:teuthology.orchestra.run.smithi150.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.8 MB/s | 126 kB 00:00 2024-03-22T10:28:34.211 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:34.211 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.8 MB/s | 126 kB 00:00 2024-03-22T10:28:34.211 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:34.218 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:34.218 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:34.278 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:34.279 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:34.326 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:28:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:34.326 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:34.516 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados2 2024-03-22T10:28:34.525 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:35.099 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:35.336 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:43 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:35.412 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:35.412 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:35.412 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:35.438 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:35.496 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:35.497 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-03-22T10:28:35.498 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:35.573 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librbd1 2024-03-22T10:28:35.712 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs2 2024-03-22T10:28:36.396 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:44 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:36.498 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:36.555 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:36.556 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-03-22T10:28:36.556 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:36.579 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:41 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:36.628 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rados 2024-03-22T10:28:36.681 INFO:teuthology.orchestra.run.smithi150.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:36.736 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:36.737 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-03-22T10:28:36.737 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:36.803 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs-devel 2024-03-22T10:28:37.456 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:45 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:37.555 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:37.610 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:37.611 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-03-22T10:28:37.611 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:37.631 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:42 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:37.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rgw 2024-03-22T10:28:37.783 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:37.784 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 30 k 2024-03-22T10:28:37.785 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 137 k 2024-03-22T10:28:37.785 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:28:37.847 INFO:teuthology.orchestra.run.smithi150.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 493 kB/s | 30 kB 00:00 2024-03-22T10:28:37.848 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:37.848 INFO:teuthology.orchestra.run.smithi150.stdout:Total 480 kB/s | 30 kB 00:00 2024-03-22T10:28:37.848 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:37.852 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:37.852 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:37.894 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:37.895 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:38.167 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:38.480 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:46 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:38.579 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:38.634 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:38.635 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-03-22T10:28:38.635 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:38.701 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-cephfs 2024-03-22T10:28:38.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:38.909 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:38.909 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:38.909 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:38.909 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:38.909 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:39.134 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados2 2024-03-22T10:28:39.502 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:47 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:39.599 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:39.655 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:39.656 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-03-22T10:28:39.656 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:39.721 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rbd 2024-03-22T10:28:39.964 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:44 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:40.066 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:40.125 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:40.126 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-03-22T10:28:40.126 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:40.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librbd1 2024-03-22T10:28:40.578 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:48 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:40.683 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:40.741 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:40.742 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-03-22T10:28:40.743 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:40.811 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-fuse 2024-03-22T10:28:40.999 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:46 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:41.103 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:41.159 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:41.160 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-03-22T10:28:41.160 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:41.228 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rados 2024-03-22T10:28:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:49 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:41.778 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:41.779 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:41.779 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:41.779 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:28:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-03-22T10:28:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:41.779 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:28:41.779 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:41.779 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-03-22T10:28:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:41.780 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 86 k 2024-03-22T10:28:41.780 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 230 k 2024-03-22T10:28:41.780 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:28:41.874 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 919 kB/s | 86 kB 00:00 2024-03-22T10:28:41.874 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:41.875 INFO:teuthology.orchestra.run.smithi115.stdout:Total 901 kB/s | 86 kB 00:00 2024-03-22T10:28:41.875 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:28:41.883 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:28:41.884 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:28:41.950 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:28:41.950 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:28:42.066 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:47 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:42.105 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:28:42.174 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:42.217 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:42.233 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:42.234 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-03-22T10:28:42.235 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:42.308 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rgw 2024-03-22T10:28:42.739 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:43.059 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:43.059 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:43.059 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:28:43.059 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:43.059 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:43.059 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:43.144 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:48 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:43.299 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:43.300 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-03-22T10:28:43.300 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:43.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-mirror 2024-03-22T10:28:43.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-cephfs 2024-03-22T10:28:44.164 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:49 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:44.185 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:52 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:44.260 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:44.315 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:44.316 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-03-22T10:28:44.316 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:44.339 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:44.339 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:44.340 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:44.340 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:28:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-03-22T10:28:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:44.340 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:28:44.340 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:44.340 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-03-22T10:28:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:44.341 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 3.0 M 2024-03-22T10:28:44.341 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 12 M 2024-03-22T10:28:44.341 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:28:44.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rbd 2024-03-22T10:28:44.494 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 20 MB/s | 3.0 MB 00:00 2024-03-22T10:28:44.495 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:44.495 INFO:teuthology.orchestra.run.smithi115.stdout:Total 19 MB/s | 3.0 MB 00:00 2024-03-22T10:28:44.496 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:28:44.511 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:28:44.511 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:28:44.603 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:28:44.603 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:28:45.089 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:28:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:45.189 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:50 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:45.223 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:28:45.223 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-22T10:28:45.223 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T10:28:45.223 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T10:28:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:45.290 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-03-22T10:28:45.346 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:45.347 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-03-22T10:28:45.347 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:45.420 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-fuse 2024-03-22T10:28:46.222 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:51 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:46.383 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:46.384 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:46.384 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:46.384 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:28:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-03-22T10:28:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:46.385 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:28:46.385 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:46.385 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-03-22T10:28:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:46.385 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 86 k 2024-03-22T10:28:46.385 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 230 k 2024-03-22T10:28:46.385 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:28:46.461 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2024-03-22T10:28:46.462 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:46.462 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-03-22T10:28:46.462 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:46.470 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:46.470 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:46.530 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:46.531 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:46.693 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:46.709 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:28:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:46.709 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:46.797 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:47.040 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-nbd 2024-03-22T10:28:47.372 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:47.710 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:47.710 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:47.710 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:47.710 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:47.710 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:47.710 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:47.882 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:55 ago on Fri 22 Mar 2024 10:26:52 AM UTC. 2024-03-22T10:28:48.002 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-mirror 2024-03-22T10:28:48.042 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-22T10:28:48.043 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:48.043 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:48.043 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:48.043 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-22T10:28:48.043 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-03-22T10:28:48.043 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:48.044 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-22T10:28:48.044 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-03-22T10:28:48.044 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-03-22T10:28:48.044 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:48.044 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 169 k 2024-03-22T10:28:48.044 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 490 k 2024-03-22T10:28:48.044 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-22T10:28:48.139 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 1.8 MB/s | 169 kB 00:00 2024-03-22T10:28:48.139 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:48.139 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.7 MB/s | 169 kB 00:00 2024-03-22T10:28:48.140 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-22T10:28:48.149 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-22T10:28:48.149 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-22T10:28:48.212 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-22T10:28:48.213 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-22T10:28:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-22T10:28:48.566 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:48.870 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:53 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:49.030 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-03-22T10:28:49.031 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:49.032 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 3.0 M 2024-03-22T10:28:49.032 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 12 M 2024-03-22T10:28:49.032 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:28:49.127 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-03-22T10:28:49.148 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:49.149 INFO:teuthology.orchestra.run.smithi150.stdout:Total 25 MB/s | 3.0 MB 00:00 2024-03-22T10:28:49.149 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:49.160 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:49.160 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:49.237 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:49.237 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:49.438 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-22T10:28:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:28:49.438 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-22T10:28:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:49.729 DEBUG:teuthology.parallel:result is None 2024-03-22T10:28:49.757 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:49.828 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:49.828 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T10:28:49.828 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-22T10:28:49.828 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T10:28:49.828 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T10:28:49.828 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:51.388 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:51.388 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:51.388 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:51.388 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:51.388 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:51.388 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:51.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-nbd 2024-03-22T10:28:52.548 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:57 ago on Fri 22 Mar 2024 10:26:55 AM UTC. 2024-03-22T10:28:52.713 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-22T10:28:52.713 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:52.713 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-03-22T10:28:52.713 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:52.713 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-22T10:28:52.714 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-03-22T10:28:52.714 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:52.714 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-22T10:28:52.714 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-03-22T10:28:52.714 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-03-22T10:28:52.714 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:52.714 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 169 k 2024-03-22T10:28:52.715 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 490 k 2024-03-22T10:28:52.715 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-22T10:28:52.785 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.4 MB/s | 169 kB 00:00 2024-03-22T10:28:52.786 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:28:52.786 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.3 MB/s | 169 kB 00:00 2024-03-22T10:28:52.786 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-22T10:28:52.795 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-22T10:28:52.795 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-22T10:28:52.856 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-22T10:28:52.856 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-22T10:28:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-22T10:28:53.136 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-03-22T10:28:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:54.167 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-22T10:28:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-03-22T10:28:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:28:54.167 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-22T10:28:54.485 DEBUG:teuthology.parallel:result is None 2024-03-22T10:28:54.485 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-22T10:28:54.485 INFO:teuthology.packaging:ref: None 2024-03-22T10:28:54.485 INFO:teuthology.packaging:tag: v18.2.0 2024-03-22T10:28:54.485 INFO:teuthology.packaging:branch: None 2024-03-22T10:28:54.486 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:28:54.486 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-22T10:28:54.486 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-03-22T10:28:54.709 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T10:28:54.782 INFO:teuthology.orchestra.run.smithi115.stdout:18.2.0-0.el9 2024-03-22T10:28:54.783 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-03-22T10:28:54.783 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-03-22T10:28:54.785 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-22T10:28:54.785 INFO:teuthology.packaging:ref: None 2024-03-22T10:28:54.785 INFO:teuthology.packaging:tag: v18.2.0 2024-03-22T10:28:54.785 INFO:teuthology.packaging:branch: None 2024-03-22T10:28:54.785 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:28:54.785 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-22T10:28:54.785 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-03-22T10:28:55.007 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T10:28:55.070 INFO:teuthology.orchestra.run.smithi150.stdout:18.2.0-0.el9 2024-03-22T10:28:55.070 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-03-22T10:28:55.070 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-03-22T10:28:55.072 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-22T10:28:55.073 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:28:55.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T10:28:55.171 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:28:55.171 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T10:28:55.298 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-22T10:28:55.298 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:28:55.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T10:28:55.397 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T10:28:55.490 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:28:55.490 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T10:28:55.588 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T10:28:55.722 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-22T10:28:55.723 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:28:55.723 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T10:28:55.818 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T10:28:55.946 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:28:55.946 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T10:28:56.046 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T10:28:56.184 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-22T10:28:56.184 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:28:56.185 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T10:28:56.284 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T10:28:56.377 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:28:56.377 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T10:28:56.477 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T10:28:56.566 INFO:teuthology.run_tasks:Running task print... 2024-03-22T10:28:56.577 INFO:teuthology.task.print:**** done install task... 2024-03-22T10:28:56.577 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-22T10:28:56.745 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.0', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-22T10:28:56.745 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.0 2024-03-22T10:28:56.745 INFO:tasks.cephadm:Cluster fsid is fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:28:56.745 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-22T10:28:56.746 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-22T10:28:56.746 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi115': '172.21.15.115', 'mon.smithi150': '172.21.15.150'} 2024-03-22T10:28:56.746 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-22T10:28:56.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2024-03-22T10:28:56.846 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-03-22T10:28:56.946 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-03-22T10:28:56.946 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:28:57.167 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77877/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2145-g3be9a5c9', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2145.g3be9a5c9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 05:51:42.267384', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-main-20240318.032620', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-22T10:28:57.392 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 ea74f2a1f3b9578955c75452d36d95e0cac33c2e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-03-22T10:28:57.394 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/ea74f2a1f3b9578955c75452d36d95e0cac33c2e/centos/9/x86_64/flavors/default/cephadm 2024-03-22T10:28:57.394 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/ea74f2a1f3b9578955c75452d36d95e0cac33c2e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T10:28:57.867 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-r--r--. 1 ubuntu ubuntu 215106 Mar 22 10:28 /home/ubuntu/cephtest/cephadm 2024-03-22T10:28:57.868 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/ea74f2a1f3b9578955c75452d36d95e0cac33c2e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T10:28:58.341 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 215106 Mar 22 10:28 /home/ubuntu/cephtest/cephadm 2024-03-22T10:28:58.342 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T10:28:58.401 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T10:28:58.472 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.0 on all hosts... 2024-03-22T10:28:58.472 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-03-22T10:28:58.479 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-03-22T10:28:59.658 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-03-22T10:28:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-03-22T10:29:24.567 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:29:24.568 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-03-22T10:29:24.568 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-03-22T10:29:24.568 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2024-03-22T10:29:24.568 INFO:teuthology.orchestra.run.smithi115.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-03-22T10:29:24.568 INFO:teuthology.orchestra.run.smithi115.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-03-22T10:29:24.568 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-03-22T10:29:24.568 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:29:24.880 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-03-22T10:29:24.880 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-03-22T10:29:24.880 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-03-22T10:29:24.880 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-03-22T10:29:24.880 INFO:teuthology.orchestra.run.smithi150.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-03-22T10:29:24.880 INFO:teuthology.orchestra.run.smithi150.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-03-22T10:29:24.880 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-03-22T10:29:24.880 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-03-22T10:29:24.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2024-03-22T10:29:25.035 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-03-22T10:29:25.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2024-03-22T10:29:25.239 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-03-22T10:29:25.339 INFO:tasks.cephadm:Writing seed config... 2024-03-22T10:29:25.340 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-22T10:29:25.340 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-22T10:29:25.340 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-22T10:29:25.340 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-22T10:29:25.340 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-22T10:29:25.340 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-22T10:29:25.340 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-22T10:29:25.340 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-22T10:29:25.341 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-22T10:29:25.342 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-22T10:29:25.343 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-22T10:29:25.343 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-22T10:29:25.343 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-22T10:29:25.343 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-22T10:29:25.343 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-22T10:29:25.344 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:29:25.344 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-22T10:29:25.399 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = fd1caff4-e836-11ee-95cd-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-22T10:29:25.399 DEBUG:teuthology.orchestra.run.smithi115:mon.smithi115> sudo journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115.service 2024-03-22T10:29:25.447 INFO:tasks.cephadm:Bootstrapping... 2024-03-22T10:29:25.447 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 -v bootstrap --fsid fd1caff4-e836-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.115 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-22T10:29:25.691 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-22T10:29:25.691 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.0', '-v', 'bootstrap', '--fsid', 'fd1caff4-e836-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.115', '--skip-admin-label'] 2024-03-22T10:29:25.753 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T10:29:25.753 INFO:teuthology.orchestra.run.smithi115.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-22T10:29:25.754 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying podman|docker is present... 2024-03-22T10:29:25.805 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T10:29:25.805 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying lvm2 is present... 2024-03-22T10:29:25.805 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying time synchronization is in place... 2024-03-22T10:29:25.823 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T10:29:25.823 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T10:29:25.842 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T10:29:25.842 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-03-22T10:29:25.865 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout enabled 2024-03-22T10:29:25.884 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout active 2024-03-22T10:29:25.884 INFO:teuthology.orchestra.run.smithi115.stdout:Unit chronyd.service is enabled and running 2024-03-22T10:29:25.885 INFO:teuthology.orchestra.run.smithi115.stdout:Repeating the final host check... 2024-03-22T10:29:25.938 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T10:29:25.938 INFO:teuthology.orchestra.run.smithi115.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-22T10:29:25.938 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl is present 2024-03-22T10:29:25.938 INFO:teuthology.orchestra.run.smithi115.stdout:lvcreate is present 2024-03-22T10:29:25.952 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T10:29:25.952 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T10:29:25.966 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T10:29:25.967 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-03-22T10:29:25.991 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout enabled 2024-03-22T10:29:26.013 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout active 2024-03-22T10:29:26.013 INFO:teuthology.orchestra.run.smithi115.stdout:Unit chronyd.service is enabled and running 2024-03-22T10:29:26.014 INFO:teuthology.orchestra.run.smithi115.stdout:Host looks OK 2024-03-22T10:29:26.014 INFO:teuthology.orchestra.run.smithi115.stdout:Cluster fsid: fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:29:26.014 INFO:teuthology.orchestra.run.smithi115.stdout:Acquiring lock 139898358254272 on /run/cephadm/fd1caff4-e836-11ee-95cd-87774f69a715.lock 2024-03-22T10:29:26.015 INFO:teuthology.orchestra.run.smithi115.stdout:Lock 139898358254272 acquired on /run/cephadm/fd1caff4-e836-11ee-95cd-87774f69a715.lock 2024-03-22T10:29:26.016 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying IP 172.21.15.115 port 3300 ... 2024-03-22T10:29:26.017 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying IP 172.21.15.115 port 6789 ... 2024-03-22T10:29:26.018 INFO:teuthology.orchestra.run.smithi115.stdout:Base mon IP(s) is [172.21.15.115:3300, 172.21.15.115:6789], mon addrv is [v2:172.21.15.115:3300,v1:172.21.15.115:6789] 2024-03-22T10:29:26.032 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.115 metric 100 2024-03-22T10:29:26.033 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.115 metric 100 2024-03-22T10:29:26.045 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-22T10:29:26.045 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-22T10:29:26.045 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2024-03-22T10:29:26.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-22T10:29:26.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-22T10:29:26.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T10:29:26.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-22T10:29:26.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2024-03-22T10:29:26.058 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T10:29:26.059 INFO:teuthology.orchestra.run.smithi115.stdout:Mon IP `172.21.15.115` is in CIDR network `172.21.0.0/20` 2024-03-22T10:29:26.059 INFO:teuthology.orchestra.run.smithi115.stdout:Mon IP `172.21.15.115` is in CIDR network `172.21.0.0/20` 2024-03-22T10:29:26.060 INFO:teuthology.orchestra.run.smithi115.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-22T10:29:26.062 INFO:teuthology.orchestra.run.smithi115.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-22T10:29:26.065 INFO:teuthology.orchestra.run.smithi115.stdout:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-03-22T10:29:27.027 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-03-22T10:29:27.027 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.0... 2024-03-22T10:29:27.027 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Getting image source signatures 2024-03-22T10:29:27.027 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying blob sha256:3bd20aeff60302f668275dc2005d10679ae56492967a3a5a54fd3dde85333aec 2024-03-22T10:29:27.027 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2024-03-22T10:29:27.028 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying config sha256:dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-03-22T10:29:27.028 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-22T10:29:28.272 INFO:teuthology.orchestra.run.smithi115.stdout:ceph: stdout ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-03-22T10:29:28.272 INFO:teuthology.orchestra.run.smithi115.stdout:Ceph version: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-03-22T10:29:28.272 INFO:teuthology.orchestra.run.smithi115.stdout:Extracting ceph user uid/gid from container image... 2024-03-22T10:29:29.347 INFO:teuthology.orchestra.run.smithi115.stdout:stat: stdout 167 167 2024-03-22T10:29:29.347 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial keys... 2024-03-22T10:29:30.416 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQCJXf1lcCEEMBAA92uwLc7UdJb6uCt9Deg5yw== 2024-03-22T10:29:31.558 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQCKXf1leoVXNBAAl2bH0xDVI0VfvJYV1CO11A== 2024-03-22T10:29:32.674 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQCMXf1ltxcMAxAAufmsTilO9AbJe2ttFhpAig== 2024-03-22T10:29:32.678 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial monmap... 2024-03-22T10:29:33.770 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T10:29:33.770 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-22T10:29:33.770 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:29:33.770 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T10:29:33.770 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool for smithi115 [v2:172.21.15.115:3300,v1:172.21.15.115:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T10:29:33.770 INFO:teuthology.orchestra.run.smithi115.stdout:setting min_mon_release = pacific 2024-03-22T10:29:33.771 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: set fsid to fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:29:33.771 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T10:29:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:29:33.771 INFO:teuthology.orchestra.run.smithi115.stdout:Creating mon... 2024-03-22T10:29:34.965 INFO:teuthology.orchestra.run.smithi115.stdout:create mon.smithi115 on 2024-03-22T10:29:35.338 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-22T10:29:35.575 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-22T10:29:35.856 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-fd1caff4-e836-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-fd1caff4-e836-11ee-95cd-87774f69a715.target. 2024-03-22T10:29:35.856 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-fd1caff4-e836-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-fd1caff4-e836-11ee-95cd-87774f69a715.target. 2024-03-22T10:29:36.228 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115 2024-03-22T10:29:36.228 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115.service: Unit ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115.service not loaded. 2024-03-22T10:29:36.509 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fd1caff4-e836-11ee-95cd-87774f69a715.target.wants/ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115.service → /etc/systemd/system/ceph-fd1caff4-e836-11ee-95cd-87774f69a715@.service. 2024-03-22T10:29:37.142 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:36 smithi115 podman[28723]: 2024-03-22 10:29:36.901276458 +0000 UTC m=+0.137965588 container create 58dec76be469ebac159961eba3474e11d7822e7f2f6e2590e2a5902f0252bbb5 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1) 2024-03-22T10:29:37.142 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:37 smithi115 podman[28723]: 2024-03-22 10:29:37.146600776 +0000 UTC m=+0.383289913 container init 58dec76be469ebac159961eba3474e11d7822e7f2f6e2590e2a5902f0252bbb5 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-03-22T10:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T10:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-03-22T10:29:37.258 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T10:29:37.258 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-03-22T10:29:37.258 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-03-22T10:29:37.258 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T10:29:37.259 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mon to start... 2024-03-22T10:29:37.259 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mon... 2024-03-22T10:29:37.399 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:37 smithi115 podman[28723]: 2024-03-22 10:29:37.15710959 +0000 UTC m=+0.393798690 container start 58dec76be469ebac159961eba3474e11d7822e7f2f6e2590e2a5902f0252bbb5 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux ) 2024-03-22T10:29:37.400 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:37 smithi115 bash[28723]: 58dec76be469ebac159961eba3474e11d7822e7f2f6e2590e2a5902f0252bbb5 2024-03-22T10:29:37.400 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:37 smithi115 systemd[1]: Started Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:29:37.400 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:37 smithi115 ceph-mon[28741]: mkfs fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:29:37.400 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:37 smithi115 ceph-mon[28741]: mon.smithi115 is new leader, mons smithi115 in quorum (ranks 0) 2024-03-22T10:29:38.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:38 smithi115 ceph-mon[28741]: mon.smithi115 is new leader, mons smithi115 in quorum (ranks 0) 2024-03-22T10:29:38.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:38 smithi115 ceph-mon[28741]: monmap e1: 1 mons at {smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2024-03-22T10:29:38.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:38 smithi115 ceph-mon[28741]: fsmap 2024-03-22T10:29:38.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:38 smithi115 ceph-mon[28741]: osdmap e1: 0 total, 0 up, 0 in 2024-03-22T10:29:38.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:38 smithi115 ceph-mon[28741]: mgrmap e1: no daemons active 2024-03-22T10:29:38.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:38 smithi115 ceph-mon[28741]: from='client.? 172.21.15.115:0/658176130' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-22T10:29:38.841 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout cluster: 2024-03-22T10:29:38.841 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout id: fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:29:38.841 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-22T10:29:38.841 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:29:38.841 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout services: 2024-03-22T10:29:38.842 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi115 (age 0.677873s) 2024-03-22T10:29:38.842 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-22T10:29:38.842 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-22T10:29:38.842 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:29:38.842 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout data: 2024-03-22T10:29:38.842 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-22T10:29:38.842 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-22T10:29:38.842 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-22T10:29:38.842 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout pgs: 2024-03-22T10:29:38.843 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:29:38.843 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.818+0000 7fecd0c00700 1 Processor -- start 2024-03-22T10:29:38.843 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.818+0000 7fecd0c00700 1 -- start start 2024-03-22T10:29:38.843 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.819+0000 7fecd0c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:38.843 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.819+0000 7fecd0c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc079720 con 0x7feccc07ace0 2024-03-22T10:29:38.843 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.820+0000 7fecc9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:38.843 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.820+0000 7fecc9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc0791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59272/0 (socket says 172.21.15.115:59272) 2024-03-22T10:29:38.844 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.820+0000 7fecc9e00700 1 -- 172.21.15.115:0/732454875 learned_addr learned my addr 172.21.15.115:0/732454875 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:38.844 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.821+0000 7fecc9e00700 1 -- 172.21.15.115:0/732454875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc079860 con 0x7feccc07ace0 2024-03-22T10:29:38.844 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.821+0000 7fecc9e00700 1 --2- 172.21.15.115:0/732454875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc0791e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0098a0 tx=0x7fecbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c28a4f21028348c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:38.844 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.821+0000 7fecc8a00700 1 -- 172.21.15.115:0/732454875 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecbc004180 con 0x7feccc07ace0 2024-03-22T10:29:38.844 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.822+0000 7fecc8a00700 1 -- 172.21.15.115:0/732454875 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc0042e0 con 0x7feccc07ace0 2024-03-22T10:29:38.844 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.822+0000 7fecc8a00700 1 -- 172.21.15.115:0/732454875 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecbc0036a0 con 0x7feccc07ace0 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.822+0000 7fecd0c00700 1 -- 172.21.15.115:0/732454875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 msgr2=0x7feccc0791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.822+0000 7fecd0c00700 1 --2- 172.21.15.115:0/732454875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc0791e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0098a0 tx=0x7fecbc009bb0 comp rx=0 tx=0).stop 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.822+0000 7fecd0c00700 1 -- 172.21.15.115:0/732454875 shutdown_connections 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.822+0000 7fecd0c00700 1 --2- 172.21.15.115:0/732454875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc0791e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.822+0000 7fecd0c00700 1 -- 172.21.15.115:0/732454875 >> 172.21.15.115:0/732454875 conn(0x7feccc100fd0 msgr2=0x7feccc103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.822+0000 7fecd0c00700 1 -- 172.21.15.115:0/732454875 shutdown_connections 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.822+0000 7fecd0c00700 1 -- 172.21.15.115:0/732454875 wait complete. 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.823+0000 7fecd0c00700 1 Processor -- start 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.823+0000 7fecd0c00700 1 -- start start 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.824+0000 7fecd0c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc19b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.824+0000 7fecd0c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc19b620 con 0x7feccc07ace0 2024-03-22T10:29:38.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.824+0000 7fecc9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc19b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.824+0000 7fecc9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc19b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59284/0 (socket says 172.21.15.115:59284) 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.824+0000 7fecc9e00700 1 -- 172.21.15.115:0/658176130 learned_addr learned my addr 172.21.15.115:0/658176130 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.824+0000 7fecc9e00700 1 -- 172.21.15.115:0/658176130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc009550 con 0x7feccc07ace0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.825+0000 7fecc9e00700 1 --2- 172.21.15.115:0/658176130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc19b0e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fecbc004000 tx=0x7fecbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.825+0000 7fecc2a00700 1 -- 172.21.15.115:0/658176130 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecbc003ce0 con 0x7feccc07ace0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.825+0000 7fecc2a00700 1 -- 172.21.15.115:0/658176130 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc020070 con 0x7feccc07ace0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.825+0000 7fecc2a00700 1 -- 172.21.15.115:0/658176130 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecbc011630 con 0x7feccc07ace0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.825+0000 7fecd0c00700 1 -- 172.21.15.115:0/658176130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc19b820 con 0x7feccc07ace0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.825+0000 7fecd0c00700 1 -- 172.21.15.115:0/658176130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc19bcc0 con 0x7feccc07ace0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.826+0000 7fecc2a00700 1 -- 172.21.15.115:0/658176130 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fecbc022030 con 0x7feccc07ace0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.826+0000 7fecc2a00700 1 -- 172.21.15.115:0/658176130 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fecbc025d40 con 0x7feccc07ace0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.826+0000 7fecd0c00700 1 -- 172.21.15.115:0/658176130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feccc194f10 con 0x7feccc07ace0 2024-03-22T10:29:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.830+0000 7fecc2a00700 1 -- 172.21.15.115:0/658176130 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fecbc025480 con 0x7feccc07ace0 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.908+0000 7fecd0c00700 1 -- 172.21.15.115:0/658176130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7feccc062380 con 0x7feccc07ace0 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.909+0000 7fecc2a00700 1 -- 172.21.15.115:0/658176130 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fecbc011bd0 con 0x7feccc07ace0 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.911+0000 7fecd0c00700 1 -- 172.21.15.115:0/658176130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 msgr2=0x7feccc19b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.911+0000 7fecd0c00700 1 --2- 172.21.15.115:0/658176130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc19b0e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fecbc004000 tx=0x7fecbc004750 comp rx=0 tx=0).stop 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.911+0000 7fecd0c00700 1 -- 172.21.15.115:0/658176130 shutdown_connections 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.911+0000 7fecd0c00700 1 --2- 172.21.15.115:0/658176130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feccc07ace0 0x7feccc19b0e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.911+0000 7fecd0c00700 1 -- 172.21.15.115:0/658176130 >> 172.21.15.115:0/658176130 conn(0x7feccc100fd0 msgr2=0x7feccc102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.911+0000 7fecd0c00700 1 -- 172.21.15.115:0/658176130 shutdown_connections 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:37.912+0000 7fecd0c00700 1 -- 172.21.15.115:0/658176130 wait complete. 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:mon is available 2024-03-22T10:29:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:Assimilating anything we can from ceph.conf... 2024-03-22T10:29:39.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:39 smithi115 ceph-mon[28741]: from='client.? 172.21.15.115:0/837610369' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-22T10:29:39.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:39 smithi115 ceph-mon[28741]: from='client.? 172.21.15.115:0/837610369' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-22T10:29:40.200 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:29:40.200 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T10:29:40.200 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout fsid = fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:29:40.204 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.115:3300,v1:172.21.15.115:6789] 2024-03-22T10:29:40.204 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T10:29:40.204 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T10:29:40.204 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T10:29:40.204 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T10:29:40.204 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.442+0000 7f964aa00700 1 Processor -- start 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.442+0000 7f964aa00700 1 -- start start 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.443+0000 7f964aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f9644105dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.443+0000 7f964aa00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9644106300 con 0x7f96441059b0 2024-03-22T10:29:40.205 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.443+0000 7f9643e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f9644105dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.443+0000 7f9643e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f9644105dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59298/0 (socket says 172.21.15.115:59298) 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.444+0000 7f9643e00700 1 -- 172.21.15.115:0/549029804 learned_addr learned my addr 172.21.15.115:0/549029804 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.444+0000 7f9643e00700 1 -- 172.21.15.115:0/549029804 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9644106440 con 0x7f96441059b0 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.444+0000 7f9643e00700 1 --2- 172.21.15.115:0/549029804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f9644105dc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9634009e80 tx=0x7f963400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be642a9c4ac53739 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.444+0000 7f9643400700 1 -- 172.21.15.115:0/549029804 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f963400e070 con 0x7f96441059b0 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.445+0000 7f9643400700 1 -- 172.21.15.115:0/549029804 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634003a20 con 0x7f96441059b0 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.445+0000 7f9643400700 1 -- 172.21.15.115:0/549029804 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9634003bd0 con 0x7f96441059b0 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.445+0000 7f964aa00700 1 -- 172.21.15.115:0/549029804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 msgr2=0x7f9644105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.445+0000 7f964aa00700 1 --2- 172.21.15.115:0/549029804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f9644105dc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9634009e80 tx=0x7f963400b330 comp rx=0 tx=0).stop 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.445+0000 7f964aa00700 1 -- 172.21.15.115:0/549029804 shutdown_connections 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.445+0000 7f964aa00700 1 --2- 172.21.15.115:0/549029804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f9644105dc0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:40.206 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.445+0000 7f964aa00700 1 -- 172.21.15.115:0/549029804 >> 172.21.15.115:0/549029804 conn(0x7f9644101020 msgr2=0x7f9644103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.445+0000 7f964aa00700 1 -- 172.21.15.115:0/549029804 shutdown_connections 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.445+0000 7f964aa00700 1 -- 172.21.15.115:0/549029804 wait complete. 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.446+0000 7f964aa00700 1 Processor -- start 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.446+0000 7f964aa00700 1 -- start start 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.447+0000 7f964aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f964407bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.447+0000 7f964aa00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964407c160 con 0x7f96441059b0 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.447+0000 7f9643e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f964407bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.447+0000 7f9643e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f964407bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59300/0 (socket says 172.21.15.115:59300) 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.447+0000 7f9643e00700 1 -- 172.21.15.115:0/837610369 learned_addr learned my addr 172.21.15.115:0/837610369 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.447+0000 7f9643e00700 1 -- 172.21.15.115:0/837610369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634009b30 con 0x7f96441059b0 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.448+0000 7f9643e00700 1 --2- 172.21.15.115:0/837610369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f964407bc20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f96441066e0 tx=0x7f9634004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.448+0000 7f9641600700 1 -- 172.21.15.115:0/837610369 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9634004240 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.448+0000 7f9641600700 1 -- 172.21.15.115:0/837610369 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96340043e0 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.448+0000 7f964aa00700 1 -- 172.21.15.115:0/837610369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964407a270 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.448+0000 7f9641600700 1 -- 172.21.15.115:0/837610369 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96340046e0 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.448+0000 7f964aa00700 1 -- 172.21.15.115:0/837610369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964407a710 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.449+0000 7f9641600700 1 -- 172.21.15.115:0/837610369 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9634019050 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.449+0000 7f9641600700 1 -- 172.21.15.115:0/837610369 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9634025790 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.449+0000 7f964aa00700 1 -- 172.21.15.115:0/837610369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9644062380 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.452+0000 7f9641600700 1 -- 172.21.15.115:0/837610369 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f963402a060 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.528+0000 7f964aa00700 1 -- 172.21.15.115:0/837610369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f9644192620 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.538+0000 7f9641600700 1 -- 172.21.15.115:0/837610369 <== mon.0 v2:172.21.15.115:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9634012d30 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.538+0000 7f9641600700 1 -- 172.21.15.115:0/837610369 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f9634020030 con 0x7f96441059b0 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.540+0000 7f964aa00700 1 -- 172.21.15.115:0/837610369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 msgr2=0x7f964407bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.540+0000 7f964aa00700 1 --2- 172.21.15.115:0/837610369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f964407bc20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f96441066e0 tx=0x7f9634004ad0 comp rx=0 tx=0).stop 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.541+0000 7f964aa00700 1 -- 172.21.15.115:0/837610369 shutdown_connections 2024-03-22T10:29:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.541+0000 7f964aa00700 1 --2- 172.21.15.115:0/837610369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96441059b0 0x7f964407bc20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:40.212 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.541+0000 7f964aa00700 1 -- 172.21.15.115:0/837610369 >> 172.21.15.115:0/837610369 conn(0x7f9644101020 msgr2=0x7f964418d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:40.212 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.541+0000 7f964aa00700 1 -- 172.21.15.115:0/837610369 shutdown_connections 2024-03-22T10:29:40.212 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:39.541+0000 7f964aa00700 1 -- 172.21.15.115:0/837610369 wait complete. 2024-03-22T10:29:40.212 INFO:teuthology.orchestra.run.smithi115.stdout:Generating new minimal ceph.conf... 2024-03-22T10:29:41.205 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:40 smithi115 ceph-mon[28741]: from='client.? 172.21.15.115:0/459064537' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.776+0000 7efcc2400700 1 Processor -- start 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.776+0000 7efcc2400700 1 -- start start 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.776+0000 7efcc2400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc07fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.776+0000 7efcc2400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc080340 con 0x7efcbc07f9f0 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.777+0000 7efcbb400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc07fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.777+0000 7efcbb400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc07fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59310/0 (socket says 172.21.15.115:59310) 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.777+0000 7efcbb400700 1 -- 172.21.15.115:0/1402875185 learned_addr learned my addr 172.21.15.115:0/1402875185 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.777+0000 7efcbb400700 1 -- 172.21.15.115:0/1402875185 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc080480 con 0x7efcbc07f9f0 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.778+0000 7efcbb400700 1 --2- 172.21.15.115:0/1402875185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc07fe00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efcac009e80 tx=0x7efcac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41aa0c6cff84a876 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:41.480 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.778+0000 7efcbaa00700 1 -- 172.21.15.115:0/1402875185 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcac00e070 con 0x7efcbc07f9f0 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.778+0000 7efcbaa00700 1 -- 172.21.15.115:0/1402875185 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcac003e90 con 0x7efcbc07f9f0 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.778+0000 7efcbaa00700 1 -- 172.21.15.115:0/1402875185 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcac004380 con 0x7efcbc07f9f0 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.778+0000 7efcc2400700 1 -- 172.21.15.115:0/1402875185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 msgr2=0x7efcbc07fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.778+0000 7efcc2400700 1 --2- 172.21.15.115:0/1402875185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc07fe00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efcac009e80 tx=0x7efcac00b330 comp rx=0 tx=0).stop 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.779+0000 7efcc2400700 1 -- 172.21.15.115:0/1402875185 shutdown_connections 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.779+0000 7efcc2400700 1 --2- 172.21.15.115:0/1402875185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc07fe00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.779+0000 7efcc2400700 1 -- 172.21.15.115:0/1402875185 >> 172.21.15.115:0/1402875185 conn(0x7efcbc07afc0 msgr2=0x7efcbc07d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.779+0000 7efcc2400700 1 -- 172.21.15.115:0/1402875185 shutdown_connections 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.779+0000 7efcc2400700 1 -- 172.21.15.115:0/1402875185 wait complete. 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.780+0000 7efcc2400700 1 Processor -- start 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.780+0000 7efcc2400700 1 -- start start 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.780+0000 7efcc2400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:41.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.780+0000 7efcc2400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc1a00f0 con 0x7efcbc07f9f0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.781+0000 7efcbb400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.781+0000 7efcbb400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc19fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59316/0 (socket says 172.21.15.115:59316) 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.781+0000 7efcbb400700 1 -- 172.21.15.115:0/459064537 learned_addr learned my addr 172.21.15.115:0/459064537 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.781+0000 7efcbb400700 1 -- 172.21.15.115:0/459064537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009b30 con 0x7efcbc07f9f0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.781+0000 7efcbb400700 1 --2- 172.21.15.115:0/459064537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc19fbb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efcac000f80 tx=0x7efcac004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.781+0000 7efcb8c00700 1 -- 172.21.15.115:0/459064537 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcac00e040 con 0x7efcbc07f9f0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.781+0000 7efcb8c00700 1 -- 172.21.15.115:0/459064537 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcac003cf0 con 0x7efcbc07f9f0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.782+0000 7efcc2400700 1 -- 172.21.15.115:0/459064537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc1a02f0 con 0x7efcbc07f9f0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.782+0000 7efcb8c00700 1 -- 172.21.15.115:0/459064537 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcac01b930 con 0x7efcbc07f9f0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.782+0000 7efcc2400700 1 -- 172.21.15.115:0/459064537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc1a07f0 con 0x7efcbc07f9f0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.782+0000 7efcb8c00700 1 -- 172.21.15.115:0/459064537 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efcac019040 con 0x7efcbc07f9f0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.782+0000 7efcb8c00700 1 -- 172.21.15.115:0/459064537 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efcac024cc0 con 0x7efcbc07f9f0 2024-03-22T10:29:41.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.783+0000 7efcc2400700 1 -- 172.21.15.115:0/459064537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc062380 con 0x7efcbc07f9f0 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.786+0000 7efcb8c00700 1 -- 172.21.15.115:0/459064537 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7efcac029060 con 0x7efcbc07f9f0 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.862+0000 7efcc2400700 1 -- 172.21.15.115:0/459064537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7efcbc1a3720 con 0x7efcbc07f9f0 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.863+0000 7efcb8c00700 1 -- 172.21.15.115:0/459064537 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7efcac01f070 con 0x7efcbc07f9f0 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.864+0000 7efcc2400700 1 -- 172.21.15.115:0/459064537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 msgr2=0x7efcbc19fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.864+0000 7efcc2400700 1 --2- 172.21.15.115:0/459064537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc19fbb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efcac000f80 tx=0x7efcac004920 comp rx=0 tx=0).stop 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.865+0000 7efcc2400700 1 -- 172.21.15.115:0/459064537 shutdown_connections 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.865+0000 7efcc2400700 1 --2- 172.21.15.115:0/459064537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcbc07f9f0 0x7efcbc19fbb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.865+0000 7efcc2400700 1 -- 172.21.15.115:0/459064537 >> 172.21.15.115:0/459064537 conn(0x7efcbc07afc0 msgr2=0x7efcbc07bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.865+0000 7efcc2400700 1 -- 172.21.15.115:0/459064537 shutdown_connections 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:40.865+0000 7efcc2400700 1 -- 172.21.15.115:0/459064537 wait complete. 2024-03-22T10:29:41.483 INFO:teuthology.orchestra.run.smithi115.stdout:Restarting the monitor... 2024-03-22T10:29:41.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:41 smithi115 systemd[1]: Stopping Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:29:42.231 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:41 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[28737]: 2024-03-22T10:29:41.916+0000 7f681a400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi115 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T10:29:42.231 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:41 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[28737]: 2024-03-22T10:29:41.916+0000 7f681a400700 -1 mon.smithi115@0(leader) e1 *** Got Signal Terminated *** 2024-03-22T10:29:42.231 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:42 smithi115 podman[28983]: 2024-03-22 10:29:42.133421967 +0000 UTC m=+0.322295100 container died 58dec76be469ebac159961eba3474e11d7822e7f2f6e2590e2a5902f0252bbb5 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0) 2024-03-22T10:29:42.805 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:42 smithi115 podman[28983]: 2024-03-22 10:29:42.543454579 +0000 UTC m=+0.732327734 container cleanup 58dec76be469ebac159961eba3474e11d7822e7f2f6e2590e2a5902f0252bbb5 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.29.1) 2024-03-22T10:29:42.806 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:42 smithi115 bash[28983]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115 2024-03-22T10:29:43.173 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:42 smithi115 podman[28997]: 2024-03-22 10:29:42.810899691 +0000 UTC m=+0.662287800 container remove 58dec76be469ebac159961eba3474e11d7822e7f2f6e2590e2a5902f0252bbb5 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-22T10:29:43.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:43 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115.service: Deactivated successfully. 2024-03-22T10:29:43.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:43 smithi115 systemd[1]: Stopped Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:29:43.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:43 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115.service: Consumed 1.697s CPU time. 2024-03-22T10:29:43.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:43 smithi115 systemd[1]: Starting Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:29:43.841 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:43 smithi115 podman[29092]: 2024-03-22 10:29:43.473302817 +0000 UTC m=+0.033612646 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-03-22T10:29:48.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:47 smithi115 podman[29092]: 2024-03-22 10:29:47.652023584 +0000 UTC m=+4.212333450 container create d5ceb65ee6ecd123f66507043c8e07dfa6c7f5dec86dbd2cfc6631c2f6bf7981 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, ceph=True, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2) 2024-03-22T10:29:52.794 INFO:teuthology.orchestra.run.smithi115.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-22T10:29:53.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 podman[29092]: 2024-03-22 10:29:52.708208856 +0000 UTC m=+9.268518729 container init d5ceb65ee6ecd123f66507043c8e07dfa6c7f5dec86dbd2cfc6631c2f6bf7981 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD) 2024-03-22T10:29:53.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 podman[29092]: 2024-03-22 10:29:52.71867187 +0000 UTC m=+9.278981717 container start d5ceb65ee6ecd123f66507043c8e07dfa6c7f5dec86dbd2cfc6631c2f6bf7981 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-03-22T10:29:53.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T10:29:53.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-03-22T10:29:53.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: pidfile_write: ignore empty --pid-file 2024-03-22T10:29:53.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: load: jerasure load: lrc 2024-03-22T10:29:53.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: RocksDB version: 7.9.2 2024-03-22T10:29:53.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Git sha 0 2024-03-22T10:29:53.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Compile date 2023-08-03 19:21:13 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: DB SUMMARY 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: DB Session ID: 87Z0SBJ5JB02PMGXS4P4 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: CURRENT file: CURRENT 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi115/store.db dir, Total Num: 1, files: 000008.sst 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi115/store.db: 000009.log size: 97404 ; 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.error_if_exists: 0 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.create_if_missing: 0 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.paranoid_checks: 1 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T10:29:53.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T10:29:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.env: 0x55ff3ab72720 2024-03-22T10:29:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T10:29:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.info_log: 0x55ff3c2c12c0 2024-03-22T10:29:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T10:29:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.statistics: (nil) 2024-03-22T10:29:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.use_fsync: 0 2024-03-22T10:29:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_log_file_size: 0 2024-03-22T10:29:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.allow_fallocate: 1 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.use_direct_reads: 0 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.db_log_dir: 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.wal_dir: 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T10:29:53.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.write_buffer_manager: 0x55ff3b5323c0 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T10:29:53.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.unordered_write: 0 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.row_cache: None 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.wal_filter: None 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.two_write_queues: 0 2024-03-22T10:29:53.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.wal_compression: 0 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.atomic_flush: 0 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.log_readahead_size: 0 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T10:29:53.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_background_jobs: 2 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_background_compactions: -1 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_subcompactions: 1 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_open_files: -1 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T10:29:53.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_background_flushes: -1 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Compression algorithms supported: 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: kZSTD supported: 0 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: kXpressCompression supported: 0 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: kZlibCompression supported: 1 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: kSnappyCompression supported: 1 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: kLZ4Compression supported: 1 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: kBZip2Compression supported: 0 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T10:29:53.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi115/store.db/MANIFEST-000010 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.merge_operator: 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_filter: None 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ff3c2c13c0) 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: cache_index_and_filter_blocks: 1 2024-03-22T10:29:53.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: pin_top_level_index_and_filter: 1 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: index_type: 0 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: data_block_index_type: 0 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: index_shortening: 1 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: checksum: 4 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: no_block_cache: 0 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache: 0x55ff3b5d3350 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache_name: BinnedLRUCache 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache_options: 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: capacity : 536870912 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: num_shard_bits : 4 2024-03-22T10:29:53.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: strict_capacity_limit : 0 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: high_pri_pool_ratio: 0.000 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache_compressed: (nil) 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: persistent_cache: (nil) 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_size: 4096 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_size_deviation: 10 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_restart_interval: 16 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: index_block_restart_interval: 1 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: metadata_block_size: 4096 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: partition_filters: 0 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: use_delta_encoding: 1 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: filter_policy: bloomfilter 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: whole_key_filtering: 1 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: verify_compression: 0 2024-03-22T10:29:53.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: read_amp_bytes_per_bit: 0 2024-03-22T10:29:53.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: format_version: 5 2024-03-22T10:29:53.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: enable_index_compression: 1 2024-03-22T10:29:53.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_align: 0 2024-03-22T10:29:53.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: max_auto_readahead_size: 262144 2024-03-22T10:29:53.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: prepopulate_block_cache: 0 2024-03-22T10:29:53.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: initial_auto_readahead_size: 8192 2024-03-22T10:29:53.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T10:29:53.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T10:29:53.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T10:29:53.028 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression: NoCompression 2024-03-22T10:29:53.028 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T10:29:53.028 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T10:29:53.028 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.num_levels: 7 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T10:29:53.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T10:29:53.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T10:29:53.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T10:29:53.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.table_properties_collectors: 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.inplace_update_support: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.bloom_locality: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.max_successive_merges: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.ttl: 2592000 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T10:29:53.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.enable_blob_files: false 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.min_blob_size: 0 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi115/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0be229ed-63fe-4e11-89f3-2fbdbcb326ae 2024-03-22T10:29:53.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711103392774687, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711103392776500, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92754, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 90771, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83486, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 269, "num_filter_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711103392, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0be229ed-63fe-4e11-89f3-2fbdbcb326ae", "db_session_id": "87Z0SBJ5JB02PMGXS4P4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711103392776664, "job": 1, "event": "recovery_finished"} 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi115/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ff3b670000 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: DB pointer 0x55ff3b65c000 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** DB Stats ** 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T10:29:53.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** Compaction Stats [default] ** 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: L0 2/0 92.45 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Sum 2/0 92.45 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** Compaction Stats [default] ** 2024-03-22T10:29:53.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 78.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative compaction: 0.00 GB write, 12.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval compaction: 0.00 GB write, 12.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T10:29:53.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Block cache BinnedLRUCache@0x55ff3b5d3350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: starting mon.smithi115 rank 0 at public addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] at bind addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi115 fsid fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mon.smithi115@-1(???) e1 preinit fsid fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mon.smithi115@-1(???).mds e1 new map 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mon.smithi115@-1(???).mds e1 print_map 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: e1 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: legacy client fscid: -1 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: No filesystems configured 2024-03-22T10:29:53.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mon.smithi115@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mon.smithi115@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mon.smithi115@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mon.smithi115@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mon.smithi115@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta expand map: {default=false} 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta from 'false' to 'false' 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta expanded map: {default=false} 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta expand map: {default=info} 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta from 'info' to 'info' 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta expanded map: {default=info} 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta expand map: {default=daemon} 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta from 'daemon' to 'daemon' 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta expanded map: {default=daemon} 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta expand map: {default=debug} 2024-03-22T10:29:53.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta from 'debug' to 'debug' 2024-03-22T10:29:53.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: expand_channel_meta expanded map: {default=debug} 2024-03-22T10:29:53.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 bash[29092]: d5ceb65ee6ecd123f66507043c8e07dfa6c7f5dec86dbd2cfc6631c2f6bf7981 2024-03-22T10:29:53.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 systemd[1]: Started Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:29:53.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mon.smithi115 is new leader, mons smithi115 in quorum (ranks 0) 2024-03-22T10:29:53.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: monmap e1: 1 mons at {smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2024-03-22T10:29:53.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: fsmap 2024-03-22T10:29:53.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: osdmap e1: 0 total, 0 up, 0 in 2024-03-22T10:29:53.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:52 smithi115 ceph-mon[29111]: mgrmap e1: no daemons active 2024-03-22T10:29:54.156 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.368+0000 7ff6a6400700 1 Processor -- start 2024-03-22T10:29:54.156 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.368+0000 7ff6a6400700 1 -- start start 2024-03-22T10:29:54.156 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.368+0000 7ff6a6400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a0105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:54.156 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.368+0000 7ff6a6400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0106320 con 0x7ff6a01059d0 2024-03-22T10:29:54.156 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.369+0000 7ff69f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a0105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:54.156 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.369+0000 7ff69f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a0105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47484/0 (socket says 172.21.15.115:47484) 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.369+0000 7ff69f400700 1 -- 172.21.15.115:0/3561422555 learned_addr learned my addr 172.21.15.115:0/3561422555 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.370+0000 7ff69f400700 1 -- 172.21.15.115:0/3561422555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0106460 con 0x7ff6a01059d0 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.370+0000 7ff69f400700 1 --2- 172.21.15.115:0/3561422555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a0105de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff690009870 tx=0x7ff690009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7199848f65377cc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.370+0000 7ff69e000700 1 -- 172.21.15.115:0/3561422555 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff690005c30 con 0x7ff6a01059d0 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.370+0000 7ff69e000700 1 -- 172.21.15.115:0/3561422555 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff690005d90 con 0x7ff6a01059d0 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.370+0000 7ff69e000700 1 -- 172.21.15.115:0/3561422555 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff690006030 con 0x7ff6a01059d0 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.371+0000 7ff6a6400700 1 -- 172.21.15.115:0/3561422555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 msgr2=0x7ff6a0105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.371+0000 7ff6a6400700 1 --2- 172.21.15.115:0/3561422555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a0105de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff690009870 tx=0x7ff690009b80 comp rx=0 tx=0).stop 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.371+0000 7ff6a6400700 1 -- 172.21.15.115:0/3561422555 shutdown_connections 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.371+0000 7ff6a6400700 1 --2- 172.21.15.115:0/3561422555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a0105de0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.371+0000 7ff6a6400700 1 -- 172.21.15.115:0/3561422555 >> 172.21.15.115:0/3561422555 conn(0x7ff6a0101020 msgr2=0x7ff6a0103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.371+0000 7ff6a6400700 1 -- 172.21.15.115:0/3561422555 shutdown_connections 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.371+0000 7ff6a6400700 1 -- 172.21.15.115:0/3561422555 wait complete. 2024-03-22T10:29:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.372+0000 7ff6a6400700 1 Processor -- start 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.372+0000 7ff6a6400700 1 -- start start 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.373+0000 7ff6a6400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.373+0000 7ff6a6400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0199c00 con 0x7ff6a01059d0 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.373+0000 7ff69f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.373+0000 7ff69f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a01996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47496/0 (socket says 172.21.15.115:47496) 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.373+0000 7ff69f400700 1 -- 172.21.15.115:0/1213240082 learned_addr learned my addr 172.21.15.115:0/1213240082 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.373+0000 7ff69f400700 1 -- 172.21.15.115:0/1213240082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff690009500 con 0x7ff6a01059d0 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.374+0000 7ff69f400700 1 --2- 172.21.15.115:0/1213240082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a01996c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff690006130 tx=0x7ff690003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.374+0000 7ff697e00700 1 -- 172.21.15.115:0/1213240082 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6900043d0 con 0x7ff6a01059d0 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.374+0000 7ff697e00700 1 -- 172.21.15.115:0/1213240082 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff690004530 con 0x7ff6a01059d0 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.374+0000 7ff6a6400700 1 -- 172.21.15.115:0/1213240082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a0199e00 con 0x7ff6a01059d0 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.374+0000 7ff697e00700 1 -- 172.21.15.115:0/1213240082 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff690011c60 con 0x7ff6a01059d0 2024-03-22T10:29:54.158 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.374+0000 7ff6a6400700 1 -- 172.21.15.115:0/1213240082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a019a2a0 con 0x7ff6a01059d0 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.374+0000 7ff697e00700 1 -- 172.21.15.115:0/1213240082 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff690018070 con 0x7ff6a01059d0 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.375+0000 7ff697e00700 1 -- 172.21.15.115:0/1213240082 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff690024960 con 0x7ff6a01059d0 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.375+0000 7ff6a6400700 1 -- 172.21.15.115:0/1213240082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a0109e60 con 0x7ff6a01059d0 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.378+0000 7ff697e00700 1 -- 172.21.15.115:0/1213240082 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff690016070 con 0x7ff6a01059d0 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.454+0000 7ff6a6400700 1 -- 172.21.15.115:0/1213240082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7ff6a0107840 con 0x7ff6a01059d0 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.459+0000 7ff697e00700 1 -- 172.21.15.115:0/1213240082 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff690011520 con 0x7ff6a01059d0 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.461+0000 7ff6a6400700 1 -- 172.21.15.115:0/1213240082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 msgr2=0x7ff6a01996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.461+0000 7ff6a6400700 1 --2- 172.21.15.115:0/1213240082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a01996c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff690006130 tx=0x7ff690003e30 comp rx=0 tx=0).stop 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.461+0000 7ff6a6400700 1 -- 172.21.15.115:0/1213240082 shutdown_connections 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.461+0000 7ff6a6400700 1 --2- 172.21.15.115:0/1213240082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6a01059d0 0x7ff6a01996c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.461+0000 7ff6a6400700 1 -- 172.21.15.115:0/1213240082 >> 172.21.15.115:0/1213240082 conn(0x7ff6a0101020 msgr2=0x7ff6a0101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.461+0000 7ff6a6400700 1 -- 172.21.15.115:0/1213240082 shutdown_connections 2024-03-22T10:29:54.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:53.461+0000 7ff6a6400700 1 -- 172.21.15.115:0/1213240082 wait complete. 2024-03-22T10:29:54.160 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-22T10:29:54.181 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-22T10:29:54.181 INFO:teuthology.orchestra.run.smithi115.stdout:Creating mgr... 2024-03-22T10:29:54.181 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-22T10:29:54.182 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-22T10:29:54.182 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying port 0.0.0.0:8443 ... 2024-03-22T10:29:54.618 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mgr.smithi115.iqzwsz 2024-03-22T10:29:54.618 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mgr.smithi115.iqzwsz.service: Unit ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mgr.smithi115.iqzwsz.service not loaded. 2024-03-22T10:29:54.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:54 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1213240082' entity='client.admin' 2024-03-22T10:29:54.873 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fd1caff4-e836-11ee-95cd-87774f69a715.target.wants/ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mgr.smithi115.iqzwsz.service → /etc/systemd/system/ceph-fd1caff4-e836-11ee-95cd-87774f69a715@.service. 2024-03-22T10:29:55.592 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T10:29:55.592 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-03-22T10:29:55.610 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T10:29:55.610 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-03-22T10:29:55.610 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-03-22T10:29:55.610 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T10:29:55.630 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T10:29:55.630 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-03-22T10:29:55.651 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T10:29:55.652 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-03-22T10:29:55.652 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-03-22T10:29:55.652 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-03-22T10:29:55.652 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr to start... 2024-03-22T10:29:55.652 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr... 2024-03-22T10:29:56.725 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:56 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/688503809' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T10:29:57.036 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:29:57.036 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-03-22T10:29:57.036 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "fd1caff4-e836-11ee-95cd-87774f69a715", 2024-03-22T10:29:57.036 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T10:29:57.036 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T10:29:57.036 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "smithi115" 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-22T10:29:57.037 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T10:29:57.038 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:29:57.039 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T10:29:37.234491+0000", 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:29:57.040 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.269+0000 7f0f28a00700 1 Processor -- start 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.269+0000 7f0f28a00700 1 -- start start 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.269+0000 7f0f28a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f24105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.269+0000 7f0f28a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f24106310 con 0x7f0f241059c0 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.270+0000 7f0f21e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f24105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.270+0000 7f0f21e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f24105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47524/0 (socket says 172.21.15.115:47524) 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.270+0000 7f0f21e00700 1 -- 172.21.15.115:0/1754571373 learned_addr learned my addr 172.21.15.115:0/1754571373 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.270+0000 7f0f21e00700 1 -- 172.21.15.115:0/1754571373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f24106450 con 0x7f0f241059c0 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.270+0000 7f0f21e00700 1 --2- 172.21.15.115:0/1754571373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f24105dd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0f140098a0 tx=0x7f0f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2546b1cb736b30dc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.271+0000 7f0f20a00700 1 -- 172.21.15.115:0/1754571373 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f14004180 con 0x7f0f241059c0 2024-03-22T10:29:57.041 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.271+0000 7f0f20a00700 1 -- 172.21.15.115:0/1754571373 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f140042e0 con 0x7f0f241059c0 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.271+0000 7f0f20a00700 1 -- 172.21.15.115:0/1754571373 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f14011a50 con 0x7f0f241059c0 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.271+0000 7f0f28a00700 1 -- 172.21.15.115:0/1754571373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 msgr2=0x7f0f24105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.271+0000 7f0f28a00700 1 --2- 172.21.15.115:0/1754571373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f24105dd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0f140098a0 tx=0x7f0f14009bb0 comp rx=0 tx=0).stop 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.272+0000 7f0f28a00700 1 -- 172.21.15.115:0/1754571373 shutdown_connections 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.272+0000 7f0f28a00700 1 --2- 172.21.15.115:0/1754571373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f24105dd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.272+0000 7f0f28a00700 1 -- 172.21.15.115:0/1754571373 >> 172.21.15.115:0/1754571373 conn(0x7f0f24100fd0 msgr2=0x7f0f24103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.272+0000 7f0f28a00700 1 -- 172.21.15.115:0/1754571373 shutdown_connections 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.272+0000 7f0f28a00700 1 -- 172.21.15.115:0/1754571373 wait complete. 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.273+0000 7f0f28a00700 1 Processor -- start 2024-03-22T10:29:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.273+0000 7f0f28a00700 1 -- start start 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.273+0000 7f0f28a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f2419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.273+0000 7f0f28a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2419be40 con 0x7f0f241059c0 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.274+0000 7f0f21e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f2419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.274+0000 7f0f21e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f2419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47532/0 (socket says 172.21.15.115:47532) 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.274+0000 7f0f21e00700 1 -- 172.21.15.115:0/688503809 learned_addr learned my addr 172.21.15.115:0/688503809 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.274+0000 7f0f21e00700 1 -- 172.21.15.115:0/688503809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f14009550 con 0x7f0f241059c0 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.274+0000 7f0f21e00700 1 --2- 172.21.15.115:0/688503809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f2419b900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f14003de0 tx=0x7f0f14011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.274+0000 7f0f1aa00700 1 -- 172.21.15.115:0/688503809 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f14003930 con 0x7f0f241059c0 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.275+0000 7f0f28a00700 1 -- 172.21.15.115:0/688503809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2419c040 con 0x7f0f241059c0 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.275+0000 7f0f1aa00700 1 -- 172.21.15.115:0/688503809 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f14003a90 con 0x7f0f241059c0 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.275+0000 7f0f28a00700 1 -- 172.21.15.115:0/688503809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2419c4e0 con 0x7f0f241059c0 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.275+0000 7f0f1aa00700 1 -- 172.21.15.115:0/688503809 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f14019610 con 0x7f0f241059c0 2024-03-22T10:29:57.043 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.275+0000 7f0f1aa00700 1 -- 172.21.15.115:0/688503809 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0f14019a70 con 0x7f0f241059c0 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.275+0000 7f0f1aa00700 1 -- 172.21.15.115:0/688503809 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0f14024df0 con 0x7f0f241059c0 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.276+0000 7f0f28a00700 1 -- 172.21.15.115:0/688503809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f24062380 con 0x7f0f241059c0 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.278+0000 7f0f1aa00700 1 -- 172.21.15.115:0/688503809 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f0f1401abc0 con 0x7f0f241059c0 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.357+0000 7f0f28a00700 1 -- 172.21.15.115:0/688503809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0f241078b0 con 0x7f0f241059c0 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.358+0000 7f0f1aa00700 1 -- 172.21.15.115:0/688503809 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f0f14024410 con 0x7f0f241059c0 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.359+0000 7f0f28a00700 1 -- 172.21.15.115:0/688503809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 msgr2=0x7f0f2419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.359+0000 7f0f28a00700 1 --2- 172.21.15.115:0/688503809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f2419b900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f14003de0 tx=0x7f0f14011ab0 comp rx=0 tx=0).stop 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.359+0000 7f0f28a00700 1 -- 172.21.15.115:0/688503809 shutdown_connections 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.359+0000 7f0f28a00700 1 --2- 172.21.15.115:0/688503809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f241059c0 0x7f0f2419b900 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.359+0000 7f0f28a00700 1 -- 172.21.15.115:0/688503809 >> 172.21.15.115:0/688503809 conn(0x7f0f24100fd0 msgr2=0x7f0f241026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:29:57.044 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.360+0000 7f0f28a00700 1 -- 172.21.15.115:0/688503809 shutdown_connections 2024-03-22T10:29:57.045 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:56.360+0000 7f0f28a00700 1 -- 172.21.15.115:0/688503809 wait complete. 2024-03-22T10:29:57.045 INFO:teuthology.orchestra.run.smithi115.stdout:mgr not available, waiting (1/15)... 2024-03-22T10:30:00.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:29:59 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1496309514' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T10:30:00.461 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:30:00.461 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-03-22T10:30:00.461 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "fd1caff4-e836-11ee-95cd-87774f69a715", 2024-03-22T10:30:00.461 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T10:30:00.461 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T10:30:00.461 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T10:30:00.461 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "smithi115" 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:00.462 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T10:30:00.463 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:00.463 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T10:30:00.463 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T10:30:00.463 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T10:30:00.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T10:30:00.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T10:30:00.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T10:30:00.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:00.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:00.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T10:29:37.234491+0000", 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T10:30:00.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.670+0000 7fb7aae00700 1 Processor -- start 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.670+0000 7fb7aae00700 1 -- start start 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.671+0000 7fb7aae00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a4079180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.671+0000 7fb7aae00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40796c0 con 0x7fb7a407ac80 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.672+0000 7fb7a3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a4079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.672+0000 7fb7a3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a4079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47534/0 (socket says 172.21.15.115:47534) 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.672+0000 7fb7a3e00700 1 -- 172.21.15.115:0/13810848 learned_addr learned my addr 172.21.15.115:0/13810848 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.672+0000 7fb7a3e00700 1 -- 172.21.15.115:0/13810848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a4079800 con 0x7fb7a407ac80 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.672+0000 7fb7a3e00700 1 --2- 172.21.15.115:0/13810848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a4079180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098a0 tx=0x7fb794009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71ef9fba8e2add0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.673+0000 7fb7a2a00700 1 -- 172.21.15.115:0/13810848 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb794004180 con 0x7fb7a407ac80 2024-03-22T10:30:00.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.673+0000 7fb7a2a00700 1 -- 172.21.15.115:0/13810848 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7940042e0 con 0x7fb7a407ac80 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.673+0000 7fb7a2a00700 1 -- 172.21.15.115:0/13810848 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb794011a50 con 0x7fb7a407ac80 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.674+0000 7fb7aae00700 1 -- 172.21.15.115:0/13810848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 msgr2=0x7fb7a4079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.674+0000 7fb7aae00700 1 --2- 172.21.15.115:0/13810848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a4079180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098a0 tx=0x7fb794009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.674+0000 7fb7aae00700 1 -- 172.21.15.115:0/13810848 shutdown_connections 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.674+0000 7fb7aae00700 1 --2- 172.21.15.115:0/13810848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a4079180 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.674+0000 7fb7aae00700 1 -- 172.21.15.115:0/13810848 >> 172.21.15.115:0/13810848 conn(0x7fb7a4100fd0 msgr2=0x7fb7a4103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.674+0000 7fb7aae00700 1 -- 172.21.15.115:0/13810848 shutdown_connections 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.674+0000 7fb7aae00700 1 -- 172.21.15.115:0/13810848 wait complete. 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.675+0000 7fb7aae00700 1 Processor -- start 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.675+0000 7fb7aae00700 1 -- start start 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.675+0000 7fb7aae00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.675+0000 7fb7aae00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a419be80 con 0x7fb7a407ac80 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.676+0000 7fb7a3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:00.475 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.676+0000 7fb7a3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a419b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47550/0 (socket says 172.21.15.115:47550) 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.676+0000 7fb7a3e00700 1 -- 172.21.15.115:0/1496309514 learned_addr learned my addr 172.21.15.115:0/1496309514 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.676+0000 7fb7a3e00700 1 -- 172.21.15.115:0/1496309514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794009550 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.676+0000 7fb7a3e00700 1 --2- 172.21.15.115:0/1496309514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a419b940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb794003de0 tx=0x7fb794011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.677+0000 7fb7a0c00700 1 -- 172.21.15.115:0/1496309514 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb794003930 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.677+0000 7fb7a0c00700 1 -- 172.21.15.115:0/1496309514 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb794003a90 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.677+0000 7fb7a0c00700 1 -- 172.21.15.115:0/1496309514 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb7940196c0 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.677+0000 7fb7aae00700 1 -- 172.21.15.115:0/1496309514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a419c080 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.677+0000 7fb7aae00700 1 -- 172.21.15.115:0/1496309514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a419c520 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.677+0000 7fb7a0c00700 1 -- 172.21.15.115:0/1496309514 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb794019c30 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.677+0000 7fb7a0c00700 1 -- 172.21.15.115:0/1496309514 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb794024ad0 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.678+0000 7fb7aae00700 1 -- 172.21.15.115:0/1496309514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a4062380 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.681+0000 7fb7a0c00700 1 -- 172.21.15.115:0/1496309514 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb79401abc0 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.760+0000 7fb7aae00700 1 -- 172.21.15.115:0/1496309514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb7a419eee0 con 0x7fb7a407ac80 2024-03-22T10:30:00.476 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.760+0000 7fb7a0c00700 1 -- 172.21.15.115:0/1496309514 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fb79401a430 con 0x7fb7a407ac80 2024-03-22T10:30:00.477 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.762+0000 7fb7aae00700 1 -- 172.21.15.115:0/1496309514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 msgr2=0x7fb7a419b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:00.477 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.762+0000 7fb7aae00700 1 --2- 172.21.15.115:0/1496309514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a419b940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb794003de0 tx=0x7fb794011ab0 comp rx=0 tx=0).stop 2024-03-22T10:30:00.477 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.762+0000 7fb7aae00700 1 -- 172.21.15.115:0/1496309514 shutdown_connections 2024-03-22T10:30:00.477 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.762+0000 7fb7aae00700 1 --2- 172.21.15.115:0/1496309514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a407ac80 0x7fb7a419b940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:00.477 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.762+0000 7fb7aae00700 1 -- 172.21.15.115:0/1496309514 >> 172.21.15.115:0/1496309514 conn(0x7fb7a4100fd0 msgr2=0x7fb7a41026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:00.477 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.762+0000 7fb7aae00700 1 -- 172.21.15.115:0/1496309514 shutdown_connections 2024-03-22T10:30:00.477 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:29:59.762+0000 7fb7aae00700 1 -- 172.21.15.115:0/1496309514 wait complete. 2024-03-22T10:30:00.477 INFO:teuthology.orchestra.run.smithi115.stdout:mgr not available, waiting (2/15)... 2024-03-22T10:30:03.808 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:30:03.809 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-03-22T10:30:03.809 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "fd1caff4-e836-11ee-95cd-87774f69a715", 2024-03-22T10:30:03.809 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T10:30:03.809 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T10:30:03.809 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T10:30:03.809 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T10:30:03.809 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:03.809 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "smithi115" 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T10:30:03.810 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T10:30:03.811 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T10:30:03.812 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:03.813 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T10:29:37.234491+0000", 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.077+0000 7fd483200700 1 Processor -- start 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.077+0000 7fd483200700 1 -- start start 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.077+0000 7fd483200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c0791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.077+0000 7fd483200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0796e0 con 0x7fd47c07aca0 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.078+0000 7fd47be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c0791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.078+0000 7fd47be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c0791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58048/0 (socket says 172.21.15.115:58048) 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.078+0000 7fd47be00700 1 -- 172.21.15.115:0/484602667 learned_addr learned my addr 172.21.15.115:0/484602667 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.078+0000 7fd47be00700 1 -- 172.21.15.115:0/484602667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c079820 con 0x7fd47c07aca0 2024-03-22T10:30:03.814 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.078+0000 7fd47be00700 1 --2- 172.21.15.115:0/484602667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c0791a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0098a0 tx=0x7fd46c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccd4634be4584a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd47aa00700 1 -- 172.21.15.115:0/484602667 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd46c004180 con 0x7fd47c07aca0 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd47aa00700 1 -- 172.21.15.115:0/484602667 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd46c0042e0 con 0x7fd47c07aca0 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd47aa00700 1 -- 172.21.15.115:0/484602667 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd46c011a50 con 0x7fd47c07aca0 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd483200700 1 -- 172.21.15.115:0/484602667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 msgr2=0x7fd47c0791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd483200700 1 --2- 172.21.15.115:0/484602667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c0791a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0098a0 tx=0x7fd46c009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd483200700 1 -- 172.21.15.115:0/484602667 shutdown_connections 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd483200700 1 --2- 172.21.15.115:0/484602667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c0791a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd483200700 1 -- 172.21.15.115:0/484602667 >> 172.21.15.115:0/484602667 conn(0x7fd47c100fd0 msgr2=0x7fd47c103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd483200700 1 -- 172.21.15.115:0/484602667 shutdown_connections 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.079+0000 7fd483200700 1 -- 172.21.15.115:0/484602667 wait complete. 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.080+0000 7fd483200700 1 Processor -- start 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.081+0000 7fd483200700 1 -- start start 2024-03-22T10:30:03.815 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.081+0000 7fd483200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:03.816 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.081+0000 7fd483200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c19bec0 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.081+0000 7fd47be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.081+0000 7fd47be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58058/0 (socket says 172.21.15.115:58058) 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.081+0000 7fd47be00700 1 -- 172.21.15.115:0/1721155457 learned_addr learned my addr 172.21.15.115:0/1721155457 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.081+0000 7fd47be00700 1 -- 172.21.15.115:0/1721155457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c009550 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.082+0000 7fd47be00700 1 --2- 172.21.15.115:0/1721155457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c19b980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd46c003de0 tx=0x7fd46c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.082+0000 7fd478c00700 1 -- 172.21.15.115:0/1721155457 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd46c0038b0 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.082+0000 7fd478c00700 1 -- 172.21.15.115:0/1721155457 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd46c003a10 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.082+0000 7fd478c00700 1 -- 172.21.15.115:0/1721155457 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd46c019600 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.082+0000 7fd483200700 1 -- 172.21.15.115:0/1721155457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47c19c0c0 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.082+0000 7fd483200700 1 -- 172.21.15.115:0/1721155457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c19c560 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.083+0000 7fd478c00700 1 -- 172.21.15.115:0/1721155457 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd46c019a60 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.083+0000 7fd478c00700 1 -- 172.21.15.115:0/1721155457 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd46c024e30 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.083+0000 7fd483200700 1 -- 172.21.15.115:0/1721155457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47c10bcb0 con 0x7fd47c07aca0 2024-03-22T10:30:03.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.086+0000 7fd478c00700 1 -- 172.21.15.115:0/1721155457 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fd46c042b00 con 0x7fd47c07aca0 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.163+0000 7fd483200700 1 -- 172.21.15.115:0/1721155457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd47c10bea0 con 0x7fd47c07aca0 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.164+0000 7fd478c00700 1 -- 172.21.15.115:0/1721155457 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7fd46c031030 con 0x7fd47c07aca0 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.165+0000 7fd483200700 1 -- 172.21.15.115:0/1721155457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 msgr2=0x7fd47c19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.165+0000 7fd483200700 1 --2- 172.21.15.115:0/1721155457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c19b980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd46c003de0 tx=0x7fd46c0047c0 comp rx=0 tx=0).stop 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.165+0000 7fd483200700 1 -- 172.21.15.115:0/1721155457 shutdown_connections 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.165+0000 7fd483200700 1 --2- 172.21.15.115:0/1721155457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47c07aca0 0x7fd47c19b980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.165+0000 7fd483200700 1 -- 172.21.15.115:0/1721155457 >> 172.21.15.115:0/1721155457 conn(0x7fd47c100fd0 msgr2=0x7fd47c1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.165+0000 7fd483200700 1 -- 172.21.15.115:0/1721155457 shutdown_connections 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:03.165+0000 7fd483200700 1 -- 172.21.15.115:0/1721155457 wait complete. 2024-03-22T10:30:03.818 INFO:teuthology.orchestra.run.smithi115.stdout:mgr not available, waiting (3/15)... 2024-03-22T10:30:04.137 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:03 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1721155457' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: Activating manager daemon smithi115.iqzwsz 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: mgrmap e2: smithi115.iqzwsz(active, starting, since 0.010445s) 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: Manager daemon smithi115.iqzwsz is now available 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/trash_purge_schedule"}]: dispatch 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:04 smithi115 ceph-mon[29111]: from='mgr.14100 172.21.15.115:0/1341145456' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:06.258 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:06 smithi115 ceph-mon[29111]: mgrmap e3: smithi115.iqzwsz(active, since 1.0215s) 2024-03-22T10:30:07.411 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:07 smithi115 ceph-mon[29111]: mgrmap e4: smithi115.iqzwsz(active, since 2s) 2024-03-22T10:30:07.411 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:07 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2455938593' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T10:30:07.429 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "fd1caff4-e836-11ee-95cd-87774f69a715", 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T10:30:07.430 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "smithi115" 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T10:30:07.431 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T10:30:07.433 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T10:29:37.234491+0000", 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.422+0000 7fb27ba00700 1 Processor -- start 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.422+0000 7fb27ba00700 1 -- start start 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.422+0000 7fb27ba00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27410ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.422+0000 7fb27ba00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27410f1f0 con 0x7fb274080210 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.423+0000 7fb279000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27410ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.423+0000 7fb279000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27410ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58126/0 (socket says 172.21.15.115:58126) 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.423+0000 7fb279000700 1 -- 172.21.15.115:0/3202231479 learned_addr learned my addr 172.21.15.115:0/3202231479 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.423+0000 7fb279000700 1 -- 172.21.15.115:0/3202231479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27410f330 con 0x7fb274080210 2024-03-22T10:30:07.434 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.423+0000 7fb279000700 1 --2- 172.21.15.115:0/3202231479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27410ecb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2680098a0 tx=0x7fb268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4df69e41bc70ea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.424+0000 7fb273400700 1 -- 172.21.15.115:0/3202231479 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb268004180 con 0x7fb274080210 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.424+0000 7fb273400700 1 -- 172.21.15.115:0/3202231479 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2680042e0 con 0x7fb274080210 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.424+0000 7fb273400700 1 -- 172.21.15.115:0/3202231479 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb268011a50 con 0x7fb274080210 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.424+0000 7fb27ba00700 1 -- 172.21.15.115:0/3202231479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 msgr2=0x7fb27410ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.424+0000 7fb27ba00700 1 --2- 172.21.15.115:0/3202231479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27410ecb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2680098a0 tx=0x7fb268009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.424+0000 7fb27ba00700 1 -- 172.21.15.115:0/3202231479 shutdown_connections 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.424+0000 7fb27ba00700 1 --2- 172.21.15.115:0/3202231479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27410ecb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.424+0000 7fb27ba00700 1 -- 172.21.15.115:0/3202231479 >> 172.21.15.115:0/3202231479 conn(0x7fb27407b200 msgr2=0x7fb27407d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.425+0000 7fb27ba00700 1 -- 172.21.15.115:0/3202231479 shutdown_connections 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.425+0000 7fb27ba00700 1 -- 172.21.15.115:0/3202231479 wait complete. 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.426+0000 7fb27ba00700 1 Processor -- start 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.426+0000 7fb27ba00700 1 -- start start 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.426+0000 7fb27ba00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.426+0000 7fb27ba00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2741a02f0 con 0x7fb274080210 2024-03-22T10:30:07.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.426+0000 7fb279000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.426+0000 7fb279000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58130/0 (socket says 172.21.15.115:58130) 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.426+0000 7fb279000700 1 -- 172.21.15.115:0/2455938593 learned_addr learned my addr 172.21.15.115:0/2455938593 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.427+0000 7fb279000700 1 -- 172.21.15.115:0/2455938593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb268009550 con 0x7fb274080210 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.427+0000 7fb279000700 1 --2- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27419fdb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb268003de0 tx=0x7fb268011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.427+0000 7fb271600700 1 -- 172.21.15.115:0/2455938593 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb268003930 con 0x7fb274080210 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.427+0000 7fb271600700 1 -- 172.21.15.115:0/2455938593 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb268003a90 con 0x7fb274080210 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.427+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2741a04f0 con 0x7fb274080210 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.427+0000 7fb271600700 1 -- 172.21.15.115:0/2455938593 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2680196c0 con 0x7fb274080210 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.427+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2741a0990 con 0x7fb274080210 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.428+0000 7fb271600700 1 -- 172.21.15.115:0/2455938593 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7fb26801abc0 con 0x7fb274080210 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.428+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2741100b0 con 0x7fb274080210 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.429+0000 7fb271600700 1 --2- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fb25c0383f0 0x7fb25c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:07.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.429+0000 7fb271600700 1 -- 172.21.15.115:0/2455938593 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb26804bf80 con 0x7fb274080210 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.429+0000 7fb273e00700 1 --2- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fb25c0383f0 0x7fb25c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.430+0000 7fb273e00700 1 --2- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fb25c0383f0 0x7fb25c03a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb260006fd0 tx=0x7fb260006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.435+0000 7fb271600700 1 -- 172.21.15.115:0/2455938593 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb268010790 con 0x7fb274080210 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.731+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb2741102a0 con 0x7fb274080210 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.733+0000 7fb271600700 1 -- 172.21.15.115:0/2455938593 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fb26804bda0 con 0x7fb274080210 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.735+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fb25c0383f0 msgr2=0x7fb25c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.735+0000 7fb27ba00700 1 --2- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fb25c0383f0 0x7fb25c03a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb260006fd0 tx=0x7fb260006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.735+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 msgr2=0x7fb27419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.735+0000 7fb27ba00700 1 --2- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27419fdb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb268003de0 tx=0x7fb268011ab0 comp rx=0 tx=0).stop 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.736+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 shutdown_connections 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.736+0000 7fb27ba00700 1 --2- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fb25c0383f0 0x7fb25c03a8a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.736+0000 7fb27ba00700 1 --2- 172.21.15.115:0/2455938593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb274080210 0x7fb27419fdb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.736+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 >> 172.21.15.115:0/2455938593 conn(0x7fb27407b200 msgr2=0x7fb27407d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.736+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 shutdown_connections 2024-03-22T10:30:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:06.736+0000 7fb27ba00700 1 -- 172.21.15.115:0/2455938593 wait complete. 2024-03-22T10:30:07.438 INFO:teuthology.orchestra.run.smithi115.stdout:mgr is available 2024-03-22T10:30:08.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:30:08.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout fsid = fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T10:30:08.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.029+0000 7f6577800700 1 Processor -- start 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.029+0000 7f6577800700 1 -- start start 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.030+0000 7f6577800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f6570079180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.030+0000 7f6577800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65700796c0 con 0x7f657007ac80 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.030+0000 7f6574e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f6570079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.030+0000 7f6574e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f6570079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58144/0 (socket says 172.21.15.115:58144) 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.030+0000 7f6574e00700 1 -- 172.21.15.115:0/1534416774 learned_addr learned my addr 172.21.15.115:0/1534416774 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.030+0000 7f6574e00700 1 -- 172.21.15.115:0/1534416774 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570079800 con 0x7f657007ac80 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.031+0000 7f6574e00700 1 --2- 172.21.15.115:0/1534416774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f6570079180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e80 tx=0x7f656000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b9a1ece786f45 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:08.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.031+0000 7f656fe00700 1 -- 172.21.15.115:0/1534416774 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f656000e070 con 0x7f657007ac80 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.031+0000 7f656fe00700 1 -- 172.21.15.115:0/1534416774 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6560003e90 con 0x7f657007ac80 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.031+0000 7f656fe00700 1 -- 172.21.15.115:0/1534416774 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6560004380 con 0x7f657007ac80 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.032+0000 7f6577800700 1 -- 172.21.15.115:0/1534416774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 msgr2=0x7f6570079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.032+0000 7f6577800700 1 --2- 172.21.15.115:0/1534416774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f6570079180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e80 tx=0x7f656000b330 comp rx=0 tx=0).stop 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.032+0000 7f6577800700 1 -- 172.21.15.115:0/1534416774 shutdown_connections 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.032+0000 7f6577800700 1 --2- 172.21.15.115:0/1534416774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f6570079180 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.032+0000 7f6577800700 1 -- 172.21.15.115:0/1534416774 >> 172.21.15.115:0/1534416774 conn(0x7f6570100fd0 msgr2=0x7f6570103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.032+0000 7f6577800700 1 -- 172.21.15.115:0/1534416774 shutdown_connections 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.032+0000 7f6577800700 1 -- 172.21.15.115:0/1534416774 wait complete. 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.033+0000 7f6577800700 1 Processor -- start 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.033+0000 7f6577800700 1 -- start start 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.034+0000 7f6577800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f657019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:08.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.034+0000 7f6577800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657019be40 con 0x7f657007ac80 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.034+0000 7f6574e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f657019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.034+0000 7f6574e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f657019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58146/0 (socket says 172.21.15.115:58146) 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.034+0000 7f6574e00700 1 -- 172.21.15.115:0/834346448 learned_addr learned my addr 172.21.15.115:0/834346448 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.034+0000 7f6574e00700 1 -- 172.21.15.115:0/834346448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6560009b30 con 0x7f657007ac80 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.034+0000 7f6574e00700 1 --2- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f657019b900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e50 tx=0x7f6560004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.035+0000 7f656e000700 1 -- 172.21.15.115:0/834346448 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f656000e040 con 0x7f657007ac80 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.035+0000 7f656e000700 1 -- 172.21.15.115:0/834346448 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6560003cf0 con 0x7f657007ac80 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.035+0000 7f656e000700 1 -- 172.21.15.115:0/834346448 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f656001b770 con 0x7f657007ac80 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.035+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657019c040 con 0x7f657007ac80 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.035+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657019c540 con 0x7f657007ac80 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.036+0000 7f656e000700 1 -- 172.21.15.115:0/834346448 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f6560019040 con 0x7f657007ac80 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.036+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657010bc70 con 0x7f657007ac80 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.036+0000 7f656e000700 1 --2- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f65540383a0 0x7f655403a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.036+0000 7f656e000700 1 -- 172.21.15.115:0/834346448 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f656004bbf0 con 0x7f657007ac80 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.037+0000 7f6567e00700 1 --2- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f65540383a0 0x7f655403a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.037+0000 7f6567e00700 1 --2- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f65540383a0 0x7f655403a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f655c006fd0 tx=0x7f655c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.042+0000 7f656e000700 1 -- 172.21.15.115:0/834346448 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6560012c50 con 0x7f657007ac80 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.253+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f657010be90 con 0x7f657007ac80 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.255+0000 7f656e000700 1 -- 172.21.15.115:0/834346448 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f6560029df0 con 0x7f657007ac80 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.257+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f65540383a0 msgr2=0x7f655403a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.257+0000 7f6577800700 1 --2- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f65540383a0 0x7f655403a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f655c006fd0 tx=0x7f655c006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.258+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 msgr2=0x7f657019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.258+0000 7f6577800700 1 --2- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f657019b900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e50 tx=0x7f6560004920 comp rx=0 tx=0).stop 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.258+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 shutdown_connections 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.258+0000 7f6577800700 1 --2- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f65540383a0 0x7f655403a850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.258+0000 7f6577800700 1 --2- 172.21.15.115:0/834346448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f657007ac80 0x7f657019b900 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.258+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 >> 172.21.15.115:0/834346448 conn(0x7f6570100fd0 msgr2=0x7f65701026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.258+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 shutdown_connections 2024-03-22T10:30:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:08.258+0000 7f6577800700 1 -- 172.21.15.115:0/834346448 wait complete. 2024-03-22T10:30:08.838 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling cephadm module... 2024-03-22T10:30:09.099 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:08 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/834346448' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-22T10:30:10.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:09 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1328447422' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-22T10:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.397+0000 7f994d600700 1 Processor -- start 2024-03-22T10:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.397+0000 7f994d600700 1 -- start start 2024-03-22T10:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.397+0000 7f994d600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994810ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.397+0000 7f994d600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994810f1f0 con 0x7f9948080210 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.398+0000 7f9946a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994810ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.398+0000 7f9946a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994810ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58150/0 (socket says 172.21.15.115:58150) 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.398+0000 7f9946a00700 1 -- 172.21.15.115:0/4029938096 learned_addr learned my addr 172.21.15.115:0/4029938096 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.398+0000 7f9946a00700 1 -- 172.21.15.115:0/4029938096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994810f330 con 0x7f9948080210 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.399+0000 7f9946a00700 1 --2- 172.21.15.115:0/4029938096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994810ecb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99380098a0 tx=0x7f9938009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b97c1bc022528 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.399+0000 7f9945600700 1 -- 172.21.15.115:0/4029938096 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9938004180 con 0x7f9948080210 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.399+0000 7f9945600700 1 -- 172.21.15.115:0/4029938096 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99380042e0 con 0x7f9948080210 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.399+0000 7f9945600700 1 -- 172.21.15.115:0/4029938096 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9938011a50 con 0x7f9948080210 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.399+0000 7f994d600700 1 -- 172.21.15.115:0/4029938096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 msgr2=0x7f994810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.399+0000 7f994d600700 1 --2- 172.21.15.115:0/4029938096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994810ecb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99380098a0 tx=0x7f9938009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.400+0000 7f994d600700 1 -- 172.21.15.115:0/4029938096 shutdown_connections 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.400+0000 7f994d600700 1 --2- 172.21.15.115:0/4029938096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994810ecb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.400+0000 7f994d600700 1 -- 172.21.15.115:0/4029938096 >> 172.21.15.115:0/4029938096 conn(0x7f994807b200 msgr2=0x7f994807d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.400+0000 7f994d600700 1 -- 172.21.15.115:0/4029938096 shutdown_connections 2024-03-22T10:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.400+0000 7f994d600700 1 -- 172.21.15.115:0/4029938096 wait complete. 2024-03-22T10:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.401+0000 7f994d600700 1 Processor -- start 2024-03-22T10:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.401+0000 7f994d600700 1 -- start start 2024-03-22T10:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.401+0000 7f994d600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.401+0000 7f994d600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99481a02f0 con 0x7f9948080210 2024-03-22T10:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.401+0000 7f9946a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.402+0000 7f9946a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58164/0 (socket says 172.21.15.115:58164) 2024-03-22T10:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.402+0000 7f9946a00700 1 -- 172.21.15.115:0/1328447422 learned_addr learned my addr 172.21.15.115:0/1328447422 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.402+0000 7f9946a00700 1 -- 172.21.15.115:0/1328447422 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938009550 con 0x7f9948080210 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.402+0000 7f9946a00700 1 --2- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994819fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9938009520 tx=0x7f9938011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.402+0000 7f993f400700 1 -- 172.21.15.115:0/1328447422 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9938003a80 con 0x7f9948080210 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.402+0000 7f993f400700 1 -- 172.21.15.115:0/1328447422 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9938003be0 con 0x7f9948080210 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.402+0000 7f993f400700 1 -- 172.21.15.115:0/1328447422 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99380197b0 con 0x7f9948080210 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.402+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99481a04f0 con 0x7f9948080210 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.403+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99481a0990 con 0x7f9948080210 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.404+0000 7f993f400700 1 -- 172.21.15.115:0/1328447422 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f9938019a20 con 0x7f9948080210 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.404+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99481100b0 con 0x7f9948080210 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.404+0000 7f993f400700 1 --2- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f992c0383f0 0x7f992c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.404+0000 7f993f400700 1 -- 172.21.15.115:0/1328447422 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f993804c4b0 con 0x7f9948080210 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.404+0000 7f9946000700 1 --2- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f992c0383f0 0x7f992c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.405+0000 7f9946000700 1 --2- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f992c0383f0 0x7f992c03a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9934006fd0 tx=0x7f9934006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.409+0000 7f993f400700 1 -- 172.21.15.115:0/1328447422 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9938018760 con 0x7f9948080210 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.672+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f99481a3730 con 0x7f9948080210 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.802+0000 7f993f400700 1 -- 172.21.15.115:0/1328447422 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f993801a950 con 0x7f9948080210 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.805+0000 7f993f400700 1 -- 172.21.15.115:0/1328447422 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f993801ac30 con 0x7f9948080210 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.807+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f992c0383f0 msgr2=0x7f992c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.807+0000 7f994d600700 1 --2- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f992c0383f0 0x7f992c03a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9934006fd0 tx=0x7f9934006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.807+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 msgr2=0x7f994819fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.807+0000 7f994d600700 1 --2- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994819fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9938009520 tx=0x7f9938011ab0 comp rx=0 tx=0).stop 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.807+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 shutdown_connections 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.807+0000 7f994d600700 1 --2- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f992c0383f0 0x7f992c03a8a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.807+0000 7f994d600700 1 --2- 172.21.15.115:0/1328447422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9948080210 0x7f994819fdb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.807+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 >> 172.21.15.115:0/1328447422 conn(0x7f994807b200 msgr2=0x7f994807d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.807+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 shutdown_connections 2024-03-22T10:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:09.808+0000 7f994d600700 1 -- 172.21.15.115:0/1328447422 wait complete. 2024-03-22T10:30:10.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:10 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1328447422' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-22T10:30:10.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:10 smithi115 ceph-mon[29111]: mgrmap e5: smithi115.iqzwsz(active, since 5s) 2024-03-22T10:30:11.897 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:11 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/731075957' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T10:30:12.018 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-03-22T10:30:12.063 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "active_name": "smithi115.iqzwsz", 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.060+0000 7fc5b8a00700 1 Processor -- start 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.060+0000 7fc5b8a00700 1 -- start start 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.061+0000 7fc5b8a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b4105c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.061+0000 7fc5b8a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b4106150 con 0x7fc5b4105800 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.061+0000 7fc5b2800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b4105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.061+0000 7fc5b2800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b4105c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58188/0 (socket says 172.21.15.115:58188) 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.061+0000 7fc5b2800700 1 -- 172.21.15.115:0/161064756 learned_addr learned my addr 172.21.15.115:0/161064756 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.062+0000 7fc5b2800700 1 -- 172.21.15.115:0/161064756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b4106290 con 0x7fc5b4105800 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.062+0000 7fc5b2800700 1 --2- 172.21.15.115:0/161064756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b4105c10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40098a0 tx=0x7fc5a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7d1fa461602b41 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.062+0000 7fc5b1400700 1 -- 172.21.15.115:0/161064756 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5a4004180 con 0x7fc5b4105800 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.062+0000 7fc5b1400700 1 -- 172.21.15.115:0/161064756 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc5a40042e0 con 0x7fc5b4105800 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.062+0000 7fc5b1400700 1 -- 172.21.15.115:0/161064756 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5a4011a50 con 0x7fc5b4105800 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.063+0000 7fc5b8a00700 1 -- 172.21.15.115:0/161064756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 msgr2=0x7fc5b4105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.063+0000 7fc5b8a00700 1 --2- 172.21.15.115:0/161064756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b4105c10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40098a0 tx=0x7fc5a4009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.063+0000 7fc5b8a00700 1 -- 172.21.15.115:0/161064756 shutdown_connections 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.063+0000 7fc5b8a00700 1 --2- 172.21.15.115:0/161064756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b4105c10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.063+0000 7fc5b8a00700 1 -- 172.21.15.115:0/161064756 >> 172.21.15.115:0/161064756 conn(0x7fc5b4100e50 msgr2=0x7fc5b4103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.063+0000 7fc5b8a00700 1 -- 172.21.15.115:0/161064756 shutdown_connections 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.063+0000 7fc5b8a00700 1 -- 172.21.15.115:0/161064756 wait complete. 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.065+0000 7fc5b8a00700 1 Processor -- start 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.065+0000 7fc5b8a00700 1 -- start start 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.065+0000 7fc5b8a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b407ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.065+0000 7fc5b8a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b407b050 con 0x7fc5b4105800 2024-03-22T10:30:12.065 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.066+0000 7fc5b2800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b407ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.066+0000 7fc5b2800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b407ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58204/0 (socket says 172.21.15.115:58204) 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.066+0000 7fc5b2800700 1 -- 172.21.15.115:0/731075957 learned_addr learned my addr 172.21.15.115:0/731075957 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.066+0000 7fc5b2800700 1 -- 172.21.15.115:0/731075957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4009550 con 0x7fc5b4105800 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.066+0000 7fc5b2800700 1 --2- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b407ab10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4003de0 tx=0x7fc5a4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.067+0000 7fc5ab400700 1 -- 172.21.15.115:0/731075957 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5a4003930 con 0x7fc5b4105800 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.067+0000 7fc5ab400700 1 -- 172.21.15.115:0/731075957 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc5a4003a90 con 0x7fc5b4105800 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.067+0000 7fc5ab400700 1 -- 172.21.15.115:0/731075957 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5a4019730 con 0x7fc5b4105800 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.067+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b4079160 con 0x7fc5b4105800 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.067+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b4079600 con 0x7fc5b4105800 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.068+0000 7fc5ab400700 1 -- 172.21.15.115:0/731075957 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fc5a4019960 con 0x7fc5b4105800 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.068+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b4062380 con 0x7fc5b4105800 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.068+0000 7fc5ab400700 1 --2- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fc598038440 0x7fc59803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:12.066 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.068+0000 7fc5ab400700 1 -- 172.21.15.115:0/731075957 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc5a404c1b0 con 0x7fc5b4105800 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.068+0000 7fc5b1e00700 1 -- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fc598038440 msgr2=0x7fc59803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2262961610 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.069+0000 7fc5b1e00700 1 --2- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fc598038440 0x7fc59803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.074+0000 7fc5ab400700 1 -- 172.21.15.115:0/731075957 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc5a407e720 con 0x7fc5b4105800 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.270+0000 7fc5b1e00700 1 -- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fc598038440 msgr2=0x7fc59803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2262961610 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.270+0000 7fc5b1e00700 1 --2- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fc598038440 0x7fc59803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.353+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc5b4107670 con 0x7fc5b4105800 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.354+0000 7fc5ab400700 1 -- 172.21.15.115:0/731075957 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fc5a401abc0 con 0x7fc5b4105800 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.357+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fc598038440 msgr2=0x7fc59803a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.357+0000 7fc5b8a00700 1 --2- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fc598038440 0x7fc59803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.357+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 msgr2=0x7fc5b407ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.357+0000 7fc5b8a00700 1 --2- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b407ab10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4003de0 tx=0x7fc5a4011ab0 comp rx=0 tx=0).stop 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.357+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 shutdown_connections 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.357+0000 7fc5b8a00700 1 --2- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7fc598038440 0x7fc59803a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:12.067 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.357+0000 7fc5b8a00700 1 --2- 172.21.15.115:0/731075957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b4105800 0x7fc5b407ab10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:12.068 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.357+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 >> 172.21.15.115:0/731075957 conn(0x7fc5b4100e50 msgr2=0x7fc5b41024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:12.068 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.358+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 shutdown_connections 2024-03-22T10:30:12.068 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:11.358+0000 7fc5b8a00700 1 -- 172.21.15.115:0/731075957 wait complete. 2024-03-22T10:30:12.068 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for the mgr to restart... 2024-03-22T10:30:12.068 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr epoch 5... 2024-03-22T10:30:19.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: Active manager daemon smithi115.iqzwsz restarted 2024-03-22T10:30:19.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: Activating manager daemon smithi115.iqzwsz 2024-03-22T10:30:19.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: osdmap e2: 0 total, 0 up, 0 in 2024-03-22T10:30:19.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: mgrmap e6: smithi115.iqzwsz(active, starting, since 0.00904201s) 2024-03-22T10:30:19.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:30:19.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: Manager daemon smithi115.iqzwsz is now available 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/trash_purge_schedule"}]: dispatch 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:19.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:18 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:19.952 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:19 smithi115 ceph-mon[29111]: Found migration_current of "None". Setting to last migration. 2024-03-22T10:30:19.952 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:19 smithi115 ceph-mon[29111]: mgrmap e7: smithi115.iqzwsz(active, since 1.00877s) 2024-03-22T10:30:19.952 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:19 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:30:19.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-03-22T10:30:19.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-22T10:30:19.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T10:30:19.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.574+0000 7f3c0ca00700 1 Processor -- start 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.575+0000 7f3c0ca00700 1 -- start start 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.575+0000 7f3c0ca00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c080791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.575+0000 7f3c0ca00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08079720 con 0x7f3c0807ace0 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.576+0000 7f3c06800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c080791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.576+0000 7f3c06800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c080791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44012/0 (socket says 172.21.15.115:44012) 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.576+0000 7f3c06800700 1 -- 172.21.15.115:0/2834870218 learned_addr learned my addr 172.21.15.115:0/2834870218 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.576+0000 7f3c06800700 1 -- 172.21.15.115:0/2834870218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c08079860 con 0x7f3c0807ace0 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.576+0000 7f3c06800700 1 --2- 172.21.15.115:0/2834870218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c080791e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80098a0 tx=0x7f3bf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641a60236ed3620a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.577+0000 7f3c05400700 1 -- 172.21.15.115:0/2834870218 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bf8004180 con 0x7f3c0807ace0 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.577+0000 7f3c05400700 1 -- 172.21.15.115:0/2834870218 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3bf80042e0 con 0x7f3c0807ace0 2024-03-22T10:30:19.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.577+0000 7f3c05400700 1 -- 172.21.15.115:0/2834870218 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bf8011a50 con 0x7f3c0807ace0 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.577+0000 7f3c0ca00700 1 -- 172.21.15.115:0/2834870218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 msgr2=0x7f3c080791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.577+0000 7f3c0ca00700 1 --2- 172.21.15.115:0/2834870218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c080791e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80098a0 tx=0x7f3bf8009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.577+0000 7f3c0ca00700 1 -- 172.21.15.115:0/2834870218 shutdown_connections 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.577+0000 7f3c0ca00700 1 --2- 172.21.15.115:0/2834870218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c080791e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.577+0000 7f3c0ca00700 1 -- 172.21.15.115:0/2834870218 >> 172.21.15.115:0/2834870218 conn(0x7f3c08100e50 msgr2=0x7f3c08103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.578+0000 7f3c0ca00700 1 -- 172.21.15.115:0/2834870218 shutdown_connections 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.578+0000 7f3c0ca00700 1 -- 172.21.15.115:0/2834870218 wait complete. 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.579+0000 7f3c0ca00700 1 Processor -- start 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.579+0000 7f3c0ca00700 1 -- start start 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.580+0000 7f3c0ca00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c0819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.580+0000 7f3c0ca00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0819bcf0 con 0x7f3c0807ace0 2024-03-22T10:30:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.580+0000 7f3c06800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c0819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.580+0000 7f3c06800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c0819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44020/0 (socket says 172.21.15.115:44020) 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.580+0000 7f3c06800700 1 -- 172.21.15.115:0/3992496863 learned_addr learned my addr 172.21.15.115:0/3992496863 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.580+0000 7f3c06800700 1 -- 172.21.15.115:0/3992496863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf8009550 con 0x7f3c0807ace0 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.581+0000 7f3c06800700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c0819b7b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8003de0 tx=0x7f3bf8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.581+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bf8003930 con 0x7f3c0807ace0 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.581+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3bf8003a90 con 0x7f3c0807ace0 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.581+0000 7f3c0ca00700 1 -- 172.21.15.115:0/3992496863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0819bef0 con 0x7f3c0807ace0 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.581+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bf80196c0 con 0x7f3c0807ace0 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.581+0000 7f3c0ca00700 1 -- 172.21.15.115:0/3992496863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0819c390 con 0x7f3c0807ace0 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.582+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f3bf801a430 con 0x7f3c0807ace0 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.583+0000 7f3bff400700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 0x7f3bec03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:19.974 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.583+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 --> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bec03b000 con 0x7f3bec038440 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.583+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3bf804c0a0 con 0x7f3c0807ace0 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.583+0000 7f3c05e00700 1 -- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 msgr2=0x7f3bec03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2262961610 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.583+0000 7f3c05e00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 0x7f3bec03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.784+0000 7f3c05e00700 1 -- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 msgr2=0x7f3bec03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2262961610 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:12.784+0000 7f3c05e00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 0x7f3bec03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:13.185+0000 7f3c05e00700 1 -- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 msgr2=0x7f3bec03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2262961610 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:13.185+0000 7f3c05e00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 0x7f3bec03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:13.986+0000 7f3c05e00700 1 -- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 msgr2=0x7f3bec03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2262961610 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:13.986+0000 7f3c05e00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 0x7f3bec03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:15.588+0000 7f3c05e00700 1 -- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 msgr2=0x7f3bec03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2262961610 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:15.588+0000 7f3c05e00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 0x7f3bec03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-22T10:30:19.975 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:18.288+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mgrmap(e 6) v1 ==== 44851+0+0 (secure 0 0 0) 0x7f3bf801abc0 con 0x7f3c0807ace0 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:18.288+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 msgr2=0x7f3bec03a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:18.288+0000 7f3bff400700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 0x7f3bec03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.288+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7f3bf804ddf0 con 0x7f3c0807ace0 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.288+0000 7f3bff400700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f3bec03bd10 0x7f3bec03e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.288+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bec03b000 con 0x7f3bec03bd10 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.289+0000 7f3c05e00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f3bec03bd10 0x7f3bec03e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.290+0000 7f3c05e00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f3bec03bd10 0x7f3bec03e0f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4003a10 tx=0x7f3bf40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.291+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f3bec03b000 con 0x7f3bec03bd10 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.298+0000 7f3c0ca00700 1 -- 172.21.15.115:0/3992496863 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f3c0819caf0 con 0x7f3bec03bd10 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.298+0000 7f3bff400700 1 -- 172.21.15.115:0/3992496863 <== mgr.14120 v2:172.21.15.115:6800/778669970 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f3c0819caf0 con 0x7f3bec03bd10 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.298+0000 7f3c0ca00700 1 -- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f3bec03bd10 msgr2=0x7f3bec03e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:19.976 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.298+0000 7f3c0ca00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f3bec03bd10 0x7f3bec03e0f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4003a10 tx=0x7f3bf40092b0 comp rx=0 tx=0).stop 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.298+0000 7f3c0ca00700 1 -- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 msgr2=0x7f3c0819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.298+0000 7f3c0ca00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c0819b7b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8003de0 tx=0x7f3bf8011ab0 comp rx=0 tx=0).stop 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.299+0000 7f3c0ca00700 1 -- 172.21.15.115:0/3992496863 shutdown_connections 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.299+0000 7f3c0ca00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f3bec03bd10 0x7f3bec03e0f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.299+0000 7f3c0ca00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c0807ace0 0x7f3c0819b7b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.299+0000 7f3c0ca00700 1 --2- 172.21.15.115:0/3992496863 >> [v2:172.21.15.115:6800/2262961610,v1:172.21.15.115:6801/2262961610] conn(0x7f3bec038440 0x7f3bec03a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.299+0000 7f3c0ca00700 1 -- 172.21.15.115:0/3992496863 >> 172.21.15.115:0/3992496863 conn(0x7f3c08100e50 msgr2=0x7f3c08102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.299+0000 7f3c0ca00700 1 -- 172.21.15.115:0/3992496863 shutdown_connections 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:19.299+0000 7f3c0ca00700 1 -- 172.21.15.115:0/3992496863 wait complete. 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:mgr epoch 5 is available 2024-03-22T10:30:19.977 INFO:teuthology.orchestra.run.smithi115.stdout:Setting orchestrator backend to cephadm... 2024-03-22T10:30:21.094 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:20 smithi115 ceph-mon[29111]: [22/Mar/2024:10:30:19] ENGINE Bus STARTING 2024-03-22T10:30:21.094 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:20 smithi115 ceph-mon[29111]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-22T10:30:21.094 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:20 smithi115 ceph-mon[29111]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-22T10:30:21.094 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:20 smithi115 ceph-mon[29111]: [22/Mar/2024:10:30:19] ENGINE Serving on http://172.21.15.115:8765 2024-03-22T10:30:21.095 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:20 smithi115 ceph-mon[29111]: [22/Mar/2024:10:30:19] ENGINE Serving on https://172.21.15.115:7150 2024-03-22T10:30:21.095 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:20 smithi115 ceph-mon[29111]: [22/Mar/2024:10:30:19] ENGINE Bus STARTED 2024-03-22T10:30:21.095 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:20 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:21.095 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:20 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:30:21.359 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.540+0000 7f620b200700 1 Processor -- start 2024-03-22T10:30:21.359 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.540+0000 7f620b200700 1 -- start start 2024-03-22T10:30:21.359 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.541+0000 7f620b200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f6204104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:21.359 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.541+0000 7f620b200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6204105230 con 0x7f62041048e0 2024-03-22T10:30:21.359 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.541+0000 7f6203e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f6204104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:21.359 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.541+0000 7f6203e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f6204104cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44118/0 (socket says 172.21.15.115:44118) 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.541+0000 7f6203e00700 1 -- 172.21.15.115:0/505570233 learned_addr learned my addr 172.21.15.115:0/505570233 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.542+0000 7f6203e00700 1 -- 172.21.15.115:0/505570233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6204105370 con 0x7f62041048e0 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.542+0000 7f6203e00700 1 --2- 172.21.15.115:0/505570233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f6204104cf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f61f400bb40 tx=0x7f61f400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817a29a6f0a28dfb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.542+0000 7f6202a00700 1 -- 172.21.15.115:0/505570233 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61f400dc00 con 0x7f62041048e0 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.542+0000 7f6202a00700 1 -- 172.21.15.115:0/505570233 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61f40044a0 con 0x7f62041048e0 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.542+0000 7f6202a00700 1 -- 172.21.15.115:0/505570233 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61f4010470 con 0x7f62041048e0 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.543+0000 7f620b200700 1 -- 172.21.15.115:0/505570233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 msgr2=0x7f6204104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.543+0000 7f620b200700 1 --2- 172.21.15.115:0/505570233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f6204104cf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f61f400bb40 tx=0x7f61f400be50 comp rx=0 tx=0).stop 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.543+0000 7f620b200700 1 -- 172.21.15.115:0/505570233 shutdown_connections 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.543+0000 7f620b200700 1 --2- 172.21.15.115:0/505570233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f6204104cf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.543+0000 7f620b200700 1 -- 172.21.15.115:0/505570233 >> 172.21.15.115:0/505570233 conn(0x7f62040fff10 msgr2=0x7f6204102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:21.361 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.543+0000 7f620b200700 1 -- 172.21.15.115:0/505570233 shutdown_connections 2024-03-22T10:30:21.361 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.543+0000 7f620b200700 1 -- 172.21.15.115:0/505570233 wait complete. 2024-03-22T10:30:21.361 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.544+0000 7f620b200700 1 Processor -- start 2024-03-22T10:30:21.361 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.544+0000 7f620b200700 1 -- start start 2024-03-22T10:30:21.361 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.544+0000 7f620b200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f620419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:21.361 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.544+0000 7f620b200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620419be50 con 0x7f62041048e0 2024-03-22T10:30:21.361 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.545+0000 7f6203e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f620419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:21.361 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.545+0000 7f6203e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f620419b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44130/0 (socket says 172.21.15.115:44130) 2024-03-22T10:30:21.362 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.545+0000 7f6203e00700 1 -- 172.21.15.115:0/91434916 learned_addr learned my addr 172.21.15.115:0/91434916 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:21.362 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.545+0000 7f6203e00700 1 -- 172.21.15.115:0/91434916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f400b7f0 con 0x7f62041048e0 2024-03-22T10:30:21.362 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.545+0000 7f6203e00700 1 --2- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f620419b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f61f4003aa0 tx=0x7f61f4003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:21.362 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.546+0000 7f6200c00700 1 -- 172.21.15.115:0/91434916 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61f40041a0 con 0x7f62041048e0 2024-03-22T10:30:21.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.546+0000 7f6200c00700 1 -- 172.21.15.115:0/91434916 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61f4004300 con 0x7f62041048e0 2024-03-22T10:30:21.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.546+0000 7f620b200700 1 -- 172.21.15.115:0/91434916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620419c050 con 0x7f62041048e0 2024-03-22T10:30:21.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.546+0000 7f6200c00700 1 -- 172.21.15.115:0/91434916 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61f4021ac0 con 0x7f62041048e0 2024-03-22T10:30:21.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.546+0000 7f620b200700 1 -- 172.21.15.115:0/91434916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620419c490 con 0x7f62041048e0 2024-03-22T10:30:21.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.547+0000 7f61fa000700 1 -- 172.21.15.115:0/91434916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6204062380 con 0x7f62041048e0 2024-03-22T10:30:21.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.547+0000 7f6200c00700 1 -- 172.21.15.115:0/91434916 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7f61f40106e0 con 0x7f62041048e0 2024-03-22T10:30:21.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.548+0000 7f6200c00700 1 --2- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f61e8038320 0x7f61e803a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:21.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.548+0000 7f6200c00700 1 -- 172.21.15.115:0/91434916 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f61f404c380 con 0x7f62041048e0 2024-03-22T10:30:21.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.548+0000 7f6203400700 1 --2- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f61e8038320 0x7f61e803a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:21.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.549+0000 7f6203400700 1 --2- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f61e8038320 0x7f61e803a7d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61f0006fd0 tx=0x7f61f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:21.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.554+0000 7f6200c00700 1 -- 172.21.15.115:0/91434916 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f61f4006d80 con 0x7f62041048e0 2024-03-22T10:30:21.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.766+0000 7f61fa000700 1 -- 172.21.15.115:0/91434916 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f6204106750 con 0x7f61e8038320 2024-03-22T10:30:21.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.776+0000 7f6200c00700 1 -- 172.21.15.115:0/91434916 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6204106750 con 0x7f61e8038320 2024-03-22T10:30:21.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.778+0000 7f61fa000700 1 -- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f61e8038320 msgr2=0x7f61e803a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:21.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.778+0000 7f61fa000700 1 --2- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f61e8038320 0x7f61e803a7d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61f0006fd0 tx=0x7f61f0006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:21.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.778+0000 7f61fa000700 1 -- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 msgr2=0x7f620419b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:21.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.778+0000 7f61fa000700 1 --2- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f620419b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f61f4003aa0 tx=0x7f61f4003d70 comp rx=0 tx=0).stop 2024-03-22T10:30:21.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.779+0000 7f61fa000700 1 -- 172.21.15.115:0/91434916 shutdown_connections 2024-03-22T10:30:21.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.779+0000 7f61fa000700 1 --2- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f61e8038320 0x7f61e803a7d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:21.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.779+0000 7f61fa000700 1 --2- 172.21.15.115:0/91434916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62041048e0 0x7f620419b910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:21.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.779+0000 7f61fa000700 1 -- 172.21.15.115:0/91434916 >> 172.21.15.115:0/91434916 conn(0x7f62040fff10 msgr2=0x7f6204192760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:21.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.779+0000 7f61fa000700 1 -- 172.21.15.115:0/91434916 shutdown_connections 2024-03-22T10:30:21.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:20.779+0000 7f61fa000700 1 -- 172.21.15.115:0/91434916 wait complete. 2024-03-22T10:30:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:21 smithi115 ceph-mon[29111]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:21 smithi115 ceph-mon[29111]: mgrmap e8: smithi115.iqzwsz(active, since 2s) 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.994+0000 7f6c7a800700 1 Processor -- start 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.994+0000 7f6c7a800700 1 -- start start 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.994+0000 7f6c7a800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7410ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.994+0000 7f6c7a800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7410f1f0 con 0x7f6c74080210 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.995+0000 7f6c73400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7410ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.995+0000 7f6c73400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7410ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60930/0 (socket says 172.21.15.115:60930) 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.995+0000 7f6c73400700 1 -- 172.21.15.115:0/3063549867 learned_addr learned my addr 172.21.15.115:0/3063549867 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.995+0000 7f6c73400700 1 -- 172.21.15.115:0/3063549867 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7410f330 con 0x7f6c74080210 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.996+0000 7f6c73400700 1 --2- 172.21.15.115:0/3063549867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7410ecb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c640098a0 tx=0x7f6c64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d47bc774c2f5962 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.996+0000 7f6c72000700 1 -- 172.21.15.115:0/3063549867 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c64004180 con 0x7f6c74080210 2024-03-22T10:30:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.996+0000 7f6c72000700 1 -- 172.21.15.115:0/3063549867 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c640042e0 con 0x7f6c74080210 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.996+0000 7f6c72000700 1 -- 172.21.15.115:0/3063549867 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c64011a50 con 0x7f6c74080210 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.997+0000 7f6c7a800700 1 -- 172.21.15.115:0/3063549867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 msgr2=0x7f6c7410ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.997+0000 7f6c7a800700 1 --2- 172.21.15.115:0/3063549867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7410ecb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c640098a0 tx=0x7f6c64009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.997+0000 7f6c7a800700 1 -- 172.21.15.115:0/3063549867 shutdown_connections 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.997+0000 7f6c7a800700 1 --2- 172.21.15.115:0/3063549867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7410ecb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.997+0000 7f6c7a800700 1 -- 172.21.15.115:0/3063549867 >> 172.21.15.115:0/3063549867 conn(0x7f6c7407b200 msgr2=0x7f6c7407d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.997+0000 7f6c7a800700 1 -- 172.21.15.115:0/3063549867 shutdown_connections 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.997+0000 7f6c7a800700 1 -- 172.21.15.115:0/3063549867 wait complete. 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.998+0000 7f6c7a800700 1 Processor -- start 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.998+0000 7f6c7a800700 1 -- start start 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.999+0000 7f6c7a800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:22.830 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.999+0000 7f6c7a800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c741a02f0 con 0x7f6c74080210 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.999+0000 7f6c73400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.999+0000 7f6c73400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60944/0 (socket says 172.21.15.115:60944) 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.999+0000 7f6c73400700 1 -- 172.21.15.115:0/7360826 learned_addr learned my addr 172.21.15.115:0/7360826 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:21.999+0000 7f6c73400700 1 -- 172.21.15.115:0/7360826 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64009550 con 0x7f6c74080210 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.000+0000 7f6c73400700 1 --2- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7419fdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6c64003de0 tx=0x7f6c64011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.000+0000 7f6c6be00700 1 -- 172.21.15.115:0/7360826 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c64003930 con 0x7f6c74080210 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.000+0000 7f6c6be00700 1 -- 172.21.15.115:0/7360826 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c64003a90 con 0x7f6c74080210 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.000+0000 7f6c6be00700 1 -- 172.21.15.115:0/7360826 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c640196c0 con 0x7f6c74080210 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.000+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c741a04f0 con 0x7f6c74080210 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.000+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c741a0990 con 0x7f6c74080210 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.001+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c741100b0 con 0x7f6c74080210 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.001+0000 7f6c6be00700 1 -- 172.21.15.115:0/7360826 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f6c6401abc0 con 0x7f6c74080210 2024-03-22T10:30:22.831 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.002+0000 7f6c6be00700 1 --2- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f6c58038440 0x7f6c5803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.002+0000 7f6c6be00700 1 -- 172.21.15.115:0/7360826 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6c6404c4c0 con 0x7f6c74080210 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.002+0000 7f6c72a00700 1 --2- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f6c58038440 0x7f6c5803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.003+0000 7f6c72a00700 1 --2- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f6c58038440 0x7f6c5803a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c60006fd0 tx=0x7f6c60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.007+0000 7f6c6be00700 1 -- 172.21.15.115:0/7360826 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6c64010790 con 0x7f6c74080210 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.222+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c7407e580 con 0x7f6c58038440 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.224+0000 7f6c6be00700 1 -- 172.21.15.115:0/7360826 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f6c7407e580 con 0x7f6c58038440 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.227+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f6c58038440 msgr2=0x7f6c5803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.227+0000 7f6c7a800700 1 --2- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f6c58038440 0x7f6c5803a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c60006fd0 tx=0x7f6c60006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.227+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 msgr2=0x7f6c7419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.227+0000 7f6c7a800700 1 --2- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7419fdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6c64003de0 tx=0x7f6c64011ab0 comp rx=0 tx=0).stop 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.228+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 shutdown_connections 2024-03-22T10:30:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.228+0000 7f6c7a800700 1 --2- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f6c58038440 0x7f6c5803a8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:22.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.228+0000 7f6c7a800700 1 --2- 172.21.15.115:0/7360826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c74080210 0x7f6c7419fdb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:22.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.228+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 >> 172.21.15.115:0/7360826 conn(0x7f6c7407b200 msgr2=0x7f6c7407d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:22.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.228+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 shutdown_connections 2024-03-22T10:30:22.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:22.228+0000 7f6c7a800700 1 -- 172.21.15.115:0/7360826 wait complete. 2024-03-22T10:30:22.833 INFO:teuthology.orchestra.run.smithi115.stdout:Generating ssh key... 2024-03-22T10:30:24.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:23 smithi115 ceph-mon[29111]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:24.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:23 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:24.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:23 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:24.379 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.398+0000 7fb57fa00700 1 Processor -- start 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.398+0000 7fb57fa00700 1 -- start start 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.399+0000 7fb57fa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb578105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.399+0000 7fb57fa00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb578106310 con 0x7fb5781059c0 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.399+0000 7fb57d000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb578105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.399+0000 7fb57d000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb578105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60960/0 (socket says 172.21.15.115:60960) 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.399+0000 7fb57d000700 1 -- 172.21.15.115:0/366507727 learned_addr learned my addr 172.21.15.115:0/366507727 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.400+0000 7fb57d000700 1 -- 172.21.15.115:0/366507727 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578106450 con 0x7fb5781059c0 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.400+0000 7fb57d000700 1 --2- 172.21.15.115:0/366507727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb578105dd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0098a0 tx=0x7fb56c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d2d26efaa100be server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.400+0000 7fb577400700 1 -- 172.21.15.115:0/366507727 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb56c004180 con 0x7fb5781059c0 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.400+0000 7fb577400700 1 -- 172.21.15.115:0/366507727 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb56c0042e0 con 0x7fb5781059c0 2024-03-22T10:30:24.380 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.400+0000 7fb577400700 1 -- 172.21.15.115:0/366507727 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb56c011a50 con 0x7fb5781059c0 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.401+0000 7fb57fa00700 1 -- 172.21.15.115:0/366507727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 msgr2=0x7fb578105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.401+0000 7fb57fa00700 1 --2- 172.21.15.115:0/366507727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb578105dd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0098a0 tx=0x7fb56c009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.401+0000 7fb57fa00700 1 -- 172.21.15.115:0/366507727 shutdown_connections 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.401+0000 7fb57fa00700 1 --2- 172.21.15.115:0/366507727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb578105dd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.401+0000 7fb57fa00700 1 -- 172.21.15.115:0/366507727 >> 172.21.15.115:0/366507727 conn(0x7fb578100fd0 msgr2=0x7fb578103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.401+0000 7fb57fa00700 1 -- 172.21.15.115:0/366507727 shutdown_connections 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.401+0000 7fb57fa00700 1 -- 172.21.15.115:0/366507727 wait complete. 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.402+0000 7fb57fa00700 1 Processor -- start 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.403+0000 7fb57fa00700 1 -- start start 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.403+0000 7fb57fa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb57819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.403+0000 7fb57fa00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57819be40 con 0x7fb5781059c0 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.403+0000 7fb57d000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb57819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.403+0000 7fb57d000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb57819b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60962/0 (socket says 172.21.15.115:60962) 2024-03-22T10:30:24.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.403+0000 7fb57d000700 1 -- 172.21.15.115:0/791944012 learned_addr learned my addr 172.21.15.115:0/791944012 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.403+0000 7fb57d000700 1 -- 172.21.15.115:0/791944012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c009550 con 0x7fb5781059c0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.404+0000 7fb57d000700 1 --2- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb57819b900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb56c003de0 tx=0x7fb56c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.404+0000 7fb575600700 1 -- 172.21.15.115:0/791944012 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb56c003930 con 0x7fb5781059c0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.404+0000 7fb575600700 1 -- 172.21.15.115:0/791944012 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb56c003a90 con 0x7fb5781059c0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.404+0000 7fb575600700 1 -- 172.21.15.115:0/791944012 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb56c019730 con 0x7fb5781059c0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.404+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57819c040 con 0x7fb5781059c0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.404+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57819c4e0 con 0x7fb5781059c0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.405+0000 7fb575600700 1 -- 172.21.15.115:0/791944012 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fb56c019960 con 0x7fb5781059c0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.405+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb578062380 con 0x7fb5781059c0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.406+0000 7fb575600700 1 --2- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fb560038440 0x7fb56003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.406+0000 7fb575600700 1 -- 172.21.15.115:0/791944012 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb56c04c4f0 con 0x7fb5781059c0 2024-03-22T10:30:24.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.406+0000 7fb577e00700 1 --2- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fb560038440 0x7fb56003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.407+0000 7fb577e00700 1 --2- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fb560038440 0x7fb56003a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb564006fd0 tx=0x7fb564006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.411+0000 7fb575600700 1 -- 172.21.15.115:0/791944012 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb56c01abc0 con 0x7fb5781059c0 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.624+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5781032f0 con 0x7fb560038440 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.762+0000 7fb575600700 1 -- 172.21.15.115:0/791944012 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb5781032f0 con 0x7fb560038440 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fb560038440 msgr2=0x7fb56003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 --2- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fb560038440 0x7fb56003a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb564006fd0 tx=0x7fb564006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 msgr2=0x7fb57819b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 --2- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb57819b900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb56c003de0 tx=0x7fb56c011ab0 comp rx=0 tx=0).stop 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 shutdown_connections 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 --2- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fb560038440 0x7fb56003a8f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 --2- 172.21.15.115:0/791944012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781059c0 0x7fb57819b900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 >> 172.21.15.115:0/791944012 conn(0x7fb578100fd0 msgr2=0x7fb5781026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 shutdown_connections 2024-03-22T10:30:24.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:23.765+0000 7fb57fa00700 1 -- 172.21.15.115:0/791944012 wait complete. 2024-03-22T10:30:24.941 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:24 smithi115 ceph-mon[29111]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:24.941 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:24 smithi115 ceph-mon[29111]: Generating ssh key... 2024-03-22T10:30:25.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC++tz/NJA/QWGAKJCrF4ksITVh9LmBsF3Q1pasowSn2lUfT9fp0nfrKLCM/mIKDQ8LLoZApCP+8U5J6rjihA2U55YMjSmtGXp4DzXqtvuCpTljrdSywAkGXAKmxVUxvp3O7/fNgS/elQgXTHRT7nlEMGTXlRd265Hk/QTvgUmtPa/LMVGGTijvGRzAsprwbRDcSoYlikjJN203RgnFyAi0YfmXEr3OHHhdxAuBui74xUGJXZZU2KuMaewqJd8w2cF814KvzDgndLr/7ko0PYjuxXBwlEyZtIfXuVJWgzJEdh03RqKREWOVJoFDdxmQ2/PNN3Qigjhj8+7dTenfEWoxfgdaNZNAE6nStecdcfSwbChvLzvXGgixObkGhTvE+N/5nGMfeKY6TdOCmS9+uMscDbij/AYzgzbSn+wNatXIotJKb6S12PjZrAouYhIs7nXe8SzCU02skFfTwcoM0eqKHWcp3s6m+NbaisSNfbdCCp0meu5q3CHhEY3w99sb1Ps= ceph-fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:30:25.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.947+0000 7f83b2400700 1 Processor -- start 2024-03-22T10:30:25.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.947+0000 7f83b2400700 1 -- start start 2024-03-22T10:30:25.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.948+0000 7f83b2400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac104b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:25.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.948+0000 7f83b2400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac105050 con 0x7f83ac104700 2024-03-22T10:30:25.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.948+0000 7f83b1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:25.854 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.948+0000 7f83b1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac104b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60974/0 (socket says 172.21.15.115:60974) 2024-03-22T10:30:25.854 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.948+0000 7f83b1000700 1 -- 172.21.15.115:0/1895344176 learned_addr learned my addr 172.21.15.115:0/1895344176 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:25.854 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.949+0000 7f83b1000700 1 -- 172.21.15.115:0/1895344176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac105190 con 0x7f83ac104700 2024-03-22T10:30:25.854 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.949+0000 7f83b1000700 1 --2- 172.21.15.115:0/1895344176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac104b10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f83a00098a0 tx=0x7f83a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf03543e8b2e9a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:25.854 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.949+0000 7f83ab400700 1 -- 172.21.15.115:0/1895344176 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83a0004180 con 0x7f83ac104700 2024-03-22T10:30:25.854 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.949+0000 7f83ab400700 1 -- 172.21.15.115:0/1895344176 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f83a00042e0 con 0x7f83ac104700 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.949+0000 7f83ab400700 1 -- 172.21.15.115:0/1895344176 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83a0011a50 con 0x7f83ac104700 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.950+0000 7f83b2400700 1 -- 172.21.15.115:0/1895344176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 msgr2=0x7f83ac104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.950+0000 7f83b2400700 1 --2- 172.21.15.115:0/1895344176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac104b10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f83a00098a0 tx=0x7f83a0009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.950+0000 7f83b2400700 1 -- 172.21.15.115:0/1895344176 shutdown_connections 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.950+0000 7f83b2400700 1 --2- 172.21.15.115:0/1895344176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac104b10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.950+0000 7f83b2400700 1 -- 172.21.15.115:0/1895344176 >> 172.21.15.115:0/1895344176 conn(0x7f83ac0ffd10 msgr2=0x7f83ac102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.950+0000 7f83b2400700 1 -- 172.21.15.115:0/1895344176 shutdown_connections 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.950+0000 7f83b2400700 1 -- 172.21.15.115:0/1895344176 wait complete. 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.951+0000 7f83b2400700 1 Processor -- start 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.951+0000 7f83b2400700 1 -- start start 2024-03-22T10:30:25.859 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.952+0000 7f83b2400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac18f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.952+0000 7f83b2400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac18fde0 con 0x7f83ac104700 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.952+0000 7f83b1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac18f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.952+0000 7f83b1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac18f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60984/0 (socket says 172.21.15.115:60984) 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.952+0000 7f83b1000700 1 -- 172.21.15.115:0/999226761 learned_addr learned my addr 172.21.15.115:0/999226761 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.952+0000 7f83b1000700 1 -- 172.21.15.115:0/999226761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009550 con 0x7f83ac104700 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.953+0000 7f83b1000700 1 --2- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac18f8a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f83a0003f80 tx=0x7f83a0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.953+0000 7f83a9600700 1 -- 172.21.15.115:0/999226761 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83a0003a80 con 0x7f83ac104700 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.953+0000 7f83a9600700 1 -- 172.21.15.115:0/999226761 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f83a0003be0 con 0x7f83ac104700 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.953+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ac18def0 con 0x7f83ac104700 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.953+0000 7f83a9600700 1 -- 172.21.15.115:0/999226761 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83a0019820 con 0x7f83ac104700 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.953+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ac18e390 con 0x7f83ac104700 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.954+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ac062380 con 0x7f83ac104700 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.954+0000 7f83a9600700 1 -- 172.21.15.115:0/999226761 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f83a001a430 con 0x7f83ac104700 2024-03-22T10:30:25.860 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.955+0000 7f83a9600700 1 --2- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f83940383f0 0x7f839403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.955+0000 7f83a9600700 1 -- 172.21.15.115:0/999226761 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f83a004c660 con 0x7f83ac104700 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.955+0000 7f83abe00700 1 --2- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f83940383f0 0x7f839403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.956+0000 7f83abe00700 1 --2- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f83940383f0 0x7f839403a8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8398006fd0 tx=0x7f8398006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:24.960+0000 7f83a9600700 1 -- 172.21.15.115:0/999226761 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f83a001acc0 con 0x7f83ac104700 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.173+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f83ac101f00 con 0x7f83940383f0 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.174+0000 7f83a9600700 1 -- 172.21.15.115:0/999226761 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f83ac101f00 con 0x7f83940383f0 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.177+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f83940383f0 msgr2=0x7f839403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.177+0000 7f83b2400700 1 --2- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f83940383f0 0x7f839403a8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8398006fd0 tx=0x7f8398006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.177+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 msgr2=0x7f83ac18f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.177+0000 7f83b2400700 1 --2- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac18f8a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f83a0003f80 tx=0x7f83a0011ab0 comp rx=0 tx=0).stop 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.177+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 shutdown_connections 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.177+0000 7f83b2400700 1 --2- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f83940383f0 0x7f839403a8a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.177+0000 7f83b2400700 1 --2- 172.21.15.115:0/999226761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83ac104700 0x7f83ac18f8a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.177+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 >> 172.21.15.115:0/999226761 conn(0x7f83ac0ffd10 msgr2=0x7f83ac079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:25.862 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.178+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 shutdown_connections 2024-03-22T10:30:25.862 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:25.178+0000 7f83b2400700 1 -- 172.21.15.115:0/999226761 wait complete. 2024-03-22T10:30:25.862 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-22T10:30:25.862 INFO:teuthology.orchestra.run.smithi115.stdout:Adding key to root@localhost authorized_keys... 2024-03-22T10:30:25.862 INFO:teuthology.orchestra.run.smithi115.stdout:Adding host smithi115... 2024-03-22T10:30:26.155 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:25 smithi115 ceph-mon[29111]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:27.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:26 smithi115 ceph-mon[29111]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "addr": "172.21.15.115", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:29.079 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:28 smithi115 ceph-mon[29111]: Deploying cephadm binary to smithi115 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Added host 'smithi115' with addr '172.21.15.115' 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.474+0000 7f2339e00700 1 Processor -- start 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.474+0000 7f2339e00700 1 -- start start 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.474+0000 7f2339e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233407a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.474+0000 7f2339e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233407a7f0 con 0x7f233407bdb0 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.475+0000 7f2332a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233407a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.475+0000 7f2332a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233407a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60988/0 (socket says 172.21.15.115:60988) 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.475+0000 7f2332a00700 1 -- 172.21.15.115:0/134277227 learned_addr learned my addr 172.21.15.115:0/134277227 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.476+0000 7f2332a00700 1 -- 172.21.15.115:0/134277227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233407a930 con 0x7f233407bdb0 2024-03-22T10:30:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.476+0000 7f2332a00700 1 --2- 172.21.15.115:0/134277227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233407a2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2324009890 tx=0x7f2324009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4dd6b4015edfda server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.476+0000 7f2331600700 1 -- 172.21.15.115:0/134277227 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2324004180 con 0x7f233407bdb0 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.476+0000 7f2331600700 1 -- 172.21.15.115:0/134277227 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f23240042e0 con 0x7f233407bdb0 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.476+0000 7f2331600700 1 -- 172.21.15.115:0/134277227 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2324011a40 con 0x7f233407bdb0 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.477+0000 7f2339e00700 1 -- 172.21.15.115:0/134277227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 msgr2=0x7f233407a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.477+0000 7f2339e00700 1 --2- 172.21.15.115:0/134277227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233407a2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2324009890 tx=0x7f2324009ba0 comp rx=0 tx=0).stop 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.477+0000 7f2339e00700 1 -- 172.21.15.115:0/134277227 shutdown_connections 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.477+0000 7f2339e00700 1 --2- 172.21.15.115:0/134277227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233407a2b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.477+0000 7f2339e00700 1 -- 172.21.15.115:0/134277227 >> 172.21.15.115:0/134277227 conn(0x7f2334101020 msgr2=0x7f2334103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.477+0000 7f2339e00700 1 -- 172.21.15.115:0/134277227 shutdown_connections 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.477+0000 7f2339e00700 1 -- 172.21.15.115:0/134277227 wait complete. 2024-03-22T10:30:31.834 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.478+0000 7f2339e00700 1 Processor -- start 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.479+0000 7f2339e00700 1 -- start start 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.479+0000 7f2339e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.479+0000 7f2339e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233419be90 con 0x7f233407bdb0 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.479+0000 7f2332a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.479+0000 7f2332a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233419b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60998/0 (socket says 172.21.15.115:60998) 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.479+0000 7f2332a00700 1 -- 172.21.15.115:0/3372285316 learned_addr learned my addr 172.21.15.115:0/3372285316 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.479+0000 7f2332a00700 1 -- 172.21.15.115:0/3372285316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23240094f0 con 0x7f233407bdb0 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.480+0000 7f2332a00700 1 --2- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233419b950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2324006fa0 tx=0x7f2324004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.480+0000 7f2338a00700 1 -- 172.21.15.115:0/3372285316 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2324003bb0 con 0x7f233407bdb0 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.480+0000 7f2338a00700 1 -- 172.21.15.115:0/3372285316 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2324003d10 con 0x7f233407bdb0 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.480+0000 7f2338a00700 1 -- 172.21.15.115:0/3372285316 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f23240197e0 con 0x7f233407bdb0 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.480+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233419c090 con 0x7f233407bdb0 2024-03-22T10:30:31.835 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.480+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233419c530 con 0x7f233407bdb0 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.481+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233410bcc0 con 0x7f233407bdb0 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.481+0000 7f2338a00700 1 -- 172.21.15.115:0/3372285316 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f2324019a10 con 0x7f233407bdb0 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.482+0000 7f2338a00700 1 --2- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f23180383f0 0x7f231803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.482+0000 7f2338a00700 1 -- 172.21.15.115:0/3372285316 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f232404c4d0 con 0x7f233407bdb0 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.482+0000 7f2332000700 1 --2- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f23180383f0 0x7f231803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.483+0000 7f2332000700 1 --2- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f23180383f0 0x7f231803a8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2320006fd0 tx=0x7f2320006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.488+0000 7f2338a00700 1 -- 172.21.15.115:0/3372285316 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2324018350 con 0x7f233407bdb0 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:26.705+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi115", "addr": "172.21.15.115", "target": ["mon-mgr", ""]}) v1 -- 0x7f23341032e0 con 0x7f23180383f0 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.183+0000 7f2338a00700 1 -- 172.21.15.115:0/3372285316 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f23341032e0 con 0x7f23180383f0 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.187+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f23180383f0 msgr2=0x7f231803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.187+0000 7f2339e00700 1 --2- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f23180383f0 0x7f231803a8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2320006fd0 tx=0x7f2320006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.187+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 msgr2=0x7f233419b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.187+0000 7f2339e00700 1 --2- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233419b950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2324006fa0 tx=0x7f2324004730 comp rx=0 tx=0).stop 2024-03-22T10:30:31.836 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.188+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 shutdown_connections 2024-03-22T10:30:31.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.188+0000 7f2339e00700 1 --2- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f23180383f0 0x7f231803a8a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:31.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.188+0000 7f2339e00700 1 --2- 172.21.15.115:0/3372285316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f233407bdb0 0x7f233419b950 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:31.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.188+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 >> 172.21.15.115:0/3372285316 conn(0x7f2334101020 msgr2=0x7f2334102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:31.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.188+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 shutdown_connections 2024-03-22T10:30:31.837 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:31.188+0000 7f2339e00700 1 -- 172.21.15.115:0/3372285316 wait complete. 2024-03-22T10:30:31.837 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying mon service with default placement... 2024-03-22T10:30:32.404 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:32 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:32.404 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:32 smithi115 ceph-mon[29111]: Added host smithi115 2024-03-22T10:30:32.404 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:32 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:30:33.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:33 smithi115 ceph-mon[29111]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:33.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:33 smithi115 ceph-mon[29111]: Saving service mon spec with placement count:5 2024-03-22T10:30:33.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:33 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:34.141 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-22T10:30:34.141 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.434+0000 7fc13e200700 1 Processor -- start 2024-03-22T10:30:34.141 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.434+0000 7fc13e200700 1 -- start start 2024-03-22T10:30:34.141 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.434+0000 7fc13e200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc138074490 0x7fc138106370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:34.141 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.434+0000 7fc13e200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1381068b0 con 0x7fc138074490 2024-03-22T10:30:34.141 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.435+0000 7fc137400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc138074490 0x7fc138106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:34.142 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.435+0000 7fc137400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc138074490 0x7fc138106370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40736/0 (socket says 172.21.15.115:40736) 2024-03-22T10:30:34.142 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.435+0000 7fc137400700 1 -- 172.21.15.115:0/2554256696 learned_addr learned my addr 172.21.15.115:0/2554256696 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:34.142 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.435+0000 7fc137400700 1 -- 172.21.15.115:0/2554256696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1381069f0 con 0x7fc138074490 2024-03-22T10:30:34.142 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.436+0000 7fc137400700 1 --2- 172.21.15.115:0/2554256696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc138074490 0x7fc138106370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1280098a0 tx=0x7fc128009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef4b69ccc31e109 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:34.142 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.436+0000 7fc136000700 1 -- 172.21.15.115:0/2554256696 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc128004180 con 0x7fc138074490 2024-03-22T10:30:34.142 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.437+0000 7fc136000700 1 -- 172.21.15.115:0/2554256696 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1280042e0 con 0x7fc138074490 2024-03-22T10:30:34.142 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.437+0000 7fc136000700 1 -- 172.21.15.115:0/2554256696 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc128011a50 con 0x7fc138074490 2024-03-22T10:30:34.142 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.437+0000 7fc13e200700 1 -- 172.21.15.115:0/2554256696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc138074490 msgr2=0x7fc138106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.437+0000 7fc13e200700 1 --2- 172.21.15.115:0/2554256696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc138074490 0x7fc138106370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1280098a0 tx=0x7fc128009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.438+0000 7fc13e200700 1 -- 172.21.15.115:0/2554256696 shutdown_connections 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.438+0000 7fc13e200700 1 --2- 172.21.15.115:0/2554256696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc138074490 0x7fc138106370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.438+0000 7fc13e200700 1 -- 172.21.15.115:0/2554256696 >> 172.21.15.115:0/2554256696 conn(0x7fc1380fff10 msgr2=0x7fc138102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.438+0000 7fc13e200700 1 -- 172.21.15.115:0/2554256696 shutdown_connections 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.438+0000 7fc13e200700 1 -- 172.21.15.115:0/2554256696 wait complete. 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.439+0000 7fc13e200700 1 Processor -- start 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.439+0000 7fc13e200700 1 -- start start 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.439+0000 7fc13e200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13819fcf0 0x7fc1381a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.439+0000 7fc13e200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1381a0640 con 0x7fc13819fcf0 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.440+0000 7fc137400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13819fcf0 0x7fc1381a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.440+0000 7fc137400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13819fcf0 0x7fc1381a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40746/0 (socket says 172.21.15.115:40746) 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.440+0000 7fc137400700 1 -- 172.21.15.115:0/368135671 learned_addr learned my addr 172.21.15.115:0/368135671 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.440+0000 7fc137400700 1 -- 172.21.15.115:0/368135671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc128009550 con 0x7fc13819fcf0 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.440+0000 7fc137400700 1 --2- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13819fcf0 0x7fc1381a0100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc128003de0 tx=0x7fc128011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:34.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.441+0000 7fc12fe00700 1 -- 172.21.15.115:0/368135671 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc128003930 con 0x7fc13819fcf0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.441+0000 7fc12fe00700 1 -- 172.21.15.115:0/368135671 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc128003a90 con 0x7fc13819fcf0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.441+0000 7fc13e200700 1 -- 172.21.15.115:0/368135671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1381a0840 con 0x7fc13819fcf0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.441+0000 7fc12fe00700 1 -- 172.21.15.115:0/368135671 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1280196c0 con 0x7fc13819fcf0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.441+0000 7fc13e200700 1 -- 172.21.15.115:0/368135671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1381a34a0 con 0x7fc13819fcf0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.443+0000 7fc12d600700 1 -- 172.21.15.115:0/368135671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc138074490 con 0x7fc13819fcf0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.443+0000 7fc12fe00700 1 -- 172.21.15.115:0/368135671 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fc12801abc0 con 0x7fc13819fcf0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.443+0000 7fc12fe00700 1 --2- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc11c0383f0 0x7fc11c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.443+0000 7fc12fe00700 1 -- 172.21.15.115:0/368135671 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc12804c3f0 con 0x7fc13819fcf0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.444+0000 7fc136a00700 1 --2- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc11c0383f0 0x7fc11c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.444+0000 7fc136a00700 1 --2- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc11c0383f0 0x7fc11c03a8a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc124006fd0 tx=0x7fc124006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.449+0000 7fc12fe00700 1 -- 172.21.15.115:0/368135671 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc128010560 con 0x7fc13819fcf0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.690+0000 7fc12d600700 1 -- 172.21.15.115:0/368135671 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1380746e0 con 0x7fc11c0383f0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.699+0000 7fc12fe00700 1 -- 172.21.15.115:0/368135671 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc1380746e0 con 0x7fc11c0383f0 2024-03-22T10:30:34.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 -- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc11c0383f0 msgr2=0x7fc11c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 --2- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc11c0383f0 0x7fc11c03a8a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc124006fd0 tx=0x7fc124006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 -- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13819fcf0 msgr2=0x7fc1381a0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 --2- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13819fcf0 0x7fc1381a0100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc128003de0 tx=0x7fc128011ab0 comp rx=0 tx=0).stop 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 -- 172.21.15.115:0/368135671 shutdown_connections 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 --2- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc11c0383f0 0x7fc11c03a8a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 --2- 172.21.15.115:0/368135671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc13819fcf0 0x7fc1381a0100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 -- 172.21.15.115:0/368135671 >> 172.21.15.115:0/368135671 conn(0x7fc1380fff10 msgr2=0x7fc1381015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 -- 172.21.15.115:0/368135671 shutdown_connections 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:32.702+0000 7fc12d600700 1 -- 172.21.15.115:0/368135671 wait complete. 2024-03-22T10:30:34.145 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying mgr service with default placement... 2024-03-22T10:30:35.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:35 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:35.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:35 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:35.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:35 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:35.693 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.738+0000 7f0e36400700 1 Processor -- start 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.738+0000 7f0e36400700 1 -- start start 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.738+0000 7f0e36400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e30105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.738+0000 7f0e36400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30106310 con 0x7f0e301059c0 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.739+0000 7f0e2f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e30105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.739+0000 7f0e2f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e30105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40748/0 (socket says 172.21.15.115:40748) 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.739+0000 7f0e2f400700 1 -- 172.21.15.115:0/3280790781 learned_addr learned my addr 172.21.15.115:0/3280790781 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.739+0000 7f0e2f400700 1 -- 172.21.15.115:0/3280790781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30106450 con 0x7f0e301059c0 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.740+0000 7f0e2f400700 1 --2- 172.21.15.115:0/3280790781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e30105dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e200098a0 tx=0x7f0e20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5ee19edee56d0b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.740+0000 7f0e2e000700 1 -- 172.21.15.115:0/3280790781 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e20004180 con 0x7f0e301059c0 2024-03-22T10:30:35.694 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.740+0000 7f0e2e000700 1 -- 172.21.15.115:0/3280790781 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e200042e0 con 0x7f0e301059c0 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.740+0000 7f0e2e000700 1 -- 172.21.15.115:0/3280790781 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e20011af0 con 0x7f0e301059c0 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.741+0000 7f0e36400700 1 -- 172.21.15.115:0/3280790781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 msgr2=0x7f0e30105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.741+0000 7f0e36400700 1 --2- 172.21.15.115:0/3280790781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e30105dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e200098a0 tx=0x7f0e20009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.741+0000 7f0e36400700 1 -- 172.21.15.115:0/3280790781 shutdown_connections 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.741+0000 7f0e36400700 1 --2- 172.21.15.115:0/3280790781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e30105dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.741+0000 7f0e36400700 1 -- 172.21.15.115:0/3280790781 >> 172.21.15.115:0/3280790781 conn(0x7f0e30100fd0 msgr2=0x7f0e30103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.741+0000 7f0e36400700 1 -- 172.21.15.115:0/3280790781 shutdown_connections 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.741+0000 7f0e36400700 1 -- 172.21.15.115:0/3280790781 wait complete. 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.743+0000 7f0e36400700 1 Processor -- start 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.743+0000 7f0e36400700 1 -- start start 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.743+0000 7f0e36400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e3019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.743+0000 7f0e36400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3019be70 con 0x7f0e301059c0 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.744+0000 7f0e2f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e3019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:35.695 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.744+0000 7f0e2f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e3019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40756/0 (socket says 172.21.15.115:40756) 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.744+0000 7f0e2f400700 1 -- 172.21.15.115:0/3909152858 learned_addr learned my addr 172.21.15.115:0/3909152858 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.744+0000 7f0e2f400700 1 -- 172.21.15.115:0/3909152858 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20009550 con 0x7f0e301059c0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.744+0000 7f0e2f400700 1 --2- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e3019b930 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e20011670 tx=0x7f0e20011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.745+0000 7f0e27e00700 1 -- 172.21.15.115:0/3909152858 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e20003a40 con 0x7f0e301059c0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.745+0000 7f0e27e00700 1 -- 172.21.15.115:0/3909152858 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e20003ba0 con 0x7f0e301059c0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.745+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3019c070 con 0x7f0e301059c0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.745+0000 7f0e27e00700 1 -- 172.21.15.115:0/3909152858 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e20024790 con 0x7f0e301059c0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.746+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3019c510 con 0x7f0e301059c0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.746+0000 7f0e27e00700 1 -- 172.21.15.115:0/3909152858 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0e200193c0 con 0x7f0e301059c0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.747+0000 7f0e27e00700 1 --2- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0e14038440 0x7f0e1403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.747+0000 7f0e27e00700 1 -- 172.21.15.115:0/3909152858 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0e20051580 con 0x7f0e301059c0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.747+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e30062380 con 0x7f0e301059c0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.747+0000 7f0e2ea00700 1 --2- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0e14038440 0x7f0e1403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:35.696 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.748+0000 7f0e2ea00700 1 --2- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0e14038440 0x7f0e1403a8f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c006fd0 tx=0x7f0e1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.755+0000 7f0e27e00700 1 -- 172.21.15.115:0/3909152858 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0e20052060 con 0x7f0e301059c0 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.983+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e301024d0 con 0x7f0e14038440 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.990+0000 7f0e27e00700 1 -- 172.21.15.115:0/3909152858 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0e301024d0 con 0x7f0e14038440 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.992+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0e14038440 msgr2=0x7f0e1403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.992+0000 7f0e36400700 1 --2- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0e14038440 0x7f0e1403a8f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c006fd0 tx=0x7f0e1c006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.992+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 msgr2=0x7f0e3019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.992+0000 7f0e36400700 1 --2- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e3019b930 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e20011670 tx=0x7f0e20011b50 comp rx=0 tx=0).stop 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.992+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 shutdown_connections 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.992+0000 7f0e36400700 1 --2- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0e14038440 0x7f0e1403a8f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.992+0000 7f0e36400700 1 --2- 172.21.15.115:0/3909152858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e301059c0 0x7f0e3019b930 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.992+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 >> 172.21.15.115:0/3909152858 conn(0x7f0e30100fd0 msgr2=0x7f0e30101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.993+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 shutdown_connections 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:34.993+0000 7f0e36400700 1 -- 172.21.15.115:0/3909152858 wait complete. 2024-03-22T10:30:35.697 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying crash service with default placement... 2024-03-22T10:30:36.245 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:36 smithi115 ceph-mon[29111]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:36.246 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:36 smithi115 ceph-mon[29111]: Saving service mgr spec with placement count:2 2024-03-22T10:30:37.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-22T10:30:37.436 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.250+0000 7f527d400700 1 Processor -- start 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.250+0000 7f527d400700 1 -- start start 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.251+0000 7f527d400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527810ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.251+0000 7f527d400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527810f1f0 con 0x7f5278080210 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.252+0000 7f5276800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527810ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.252+0000 7f5276800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527810ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40772/0 (socket says 172.21.15.115:40772) 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.252+0000 7f5276800700 1 -- 172.21.15.115:0/173868926 learned_addr learned my addr 172.21.15.115:0/173868926 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.252+0000 7f5276800700 1 -- 172.21.15.115:0/173868926 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527810f330 con 0x7f5278080210 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.253+0000 7f5276800700 1 --2- 172.21.15.115:0/173868926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527810ecb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52680098a0 tx=0x7f5268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca56325218ade1bd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.253+0000 7f5275e00700 1 -- 172.21.15.115:0/173868926 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5268004180 con 0x7f5278080210 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.253+0000 7f5275e00700 1 -- 172.21.15.115:0/173868926 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f52680042e0 con 0x7f5278080210 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.253+0000 7f5275e00700 1 -- 172.21.15.115:0/173868926 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5268011af0 con 0x7f5278080210 2024-03-22T10:30:37.438 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.254+0000 7f527d400700 1 -- 172.21.15.115:0/173868926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 msgr2=0x7f527810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.254+0000 7f527d400700 1 --2- 172.21.15.115:0/173868926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527810ecb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52680098a0 tx=0x7f5268009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.254+0000 7f527d400700 1 -- 172.21.15.115:0/173868926 shutdown_connections 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.254+0000 7f527d400700 1 --2- 172.21.15.115:0/173868926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527810ecb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.254+0000 7f527d400700 1 -- 172.21.15.115:0/173868926 >> 172.21.15.115:0/173868926 conn(0x7f527807b200 msgr2=0x7f527807d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.254+0000 7f527d400700 1 -- 172.21.15.115:0/173868926 shutdown_connections 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.254+0000 7f527d400700 1 -- 172.21.15.115:0/173868926 wait complete. 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.255+0000 7f527d400700 1 Processor -- start 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.255+0000 7f527d400700 1 -- start start 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.256+0000 7f527d400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.256+0000 7f527d400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52781a0310 con 0x7f5278080210 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.256+0000 7f5276800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.256+0000 7f5276800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527819fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40774/0 (socket says 172.21.15.115:40774) 2024-03-22T10:30:37.439 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.256+0000 7f5276800700 1 -- 172.21.15.115:0/2301904441 learned_addr learned my addr 172.21.15.115:0/2301904441 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.256+0000 7f5276800700 1 -- 172.21.15.115:0/2301904441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268009550 con 0x7f5278080210 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.256+0000 7f5276800700 1 --2- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527819fdd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5268011670 tx=0x7f5268011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.257+0000 7f526f400700 1 -- 172.21.15.115:0/2301904441 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52680039f0 con 0x7f5278080210 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.257+0000 7f526f400700 1 -- 172.21.15.115:0/2301904441 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5268003b50 con 0x7f5278080210 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.257+0000 7f526f400700 1 -- 172.21.15.115:0/2301904441 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52680247c0 con 0x7f5278080210 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.257+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52781a0510 con 0x7f5278080210 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.257+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52781a09b0 con 0x7f5278080210 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.258+0000 7f526f400700 1 -- 172.21.15.115:0/2301904441 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f526801ac00 con 0x7f5278080210 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.258+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52781100b0 con 0x7f5278080210 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.258+0000 7f526f400700 1 --2- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f525c038440 0x7f525c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.259+0000 7f526f400700 1 -- 172.21.15.115:0/2301904441 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f526804c440 con 0x7f5278080210 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.259+0000 7f526fe00700 1 --2- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f525c038440 0x7f525c03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:37.440 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.259+0000 7f526fe00700 1 --2- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f525c038440 0x7f525c03a8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5260006fd0 tx=0x7f5260006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.263+0000 7f526f400700 1 -- 172.21.15.115:0/2301904441 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5268010380 con 0x7f5278080210 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.496+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f527807e580 con 0x7f525c038440 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.506+0000 7f526f400700 1 -- 172.21.15.115:0/2301904441 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f527807e580 con 0x7f525c038440 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.508+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f525c038440 msgr2=0x7f525c03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.508+0000 7f527d400700 1 --2- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f525c038440 0x7f525c03a8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5260006fd0 tx=0x7f5260006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.508+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 msgr2=0x7f527819fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.509+0000 7f527d400700 1 --2- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527819fdd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5268011670 tx=0x7f5268011b50 comp rx=0 tx=0).stop 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.509+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 shutdown_connections 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.509+0000 7f527d400700 1 --2- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f525c038440 0x7f525c03a8f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.509+0000 7f527d400700 1 --2- 172.21.15.115:0/2301904441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5278080210 0x7f527819fdd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.509+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 >> 172.21.15.115:0/2301904441 conn(0x7f527807b200 msgr2=0x7f527807d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.509+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 shutdown_connections 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:36.510+0000 7f527d400700 1 -- 172.21.15.115:0/2301904441 wait complete. 2024-03-22T10:30:37.441 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying ceph-exporter service with default placement... 2024-03-22T10:30:37.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:37 smithi115 ceph-mon[29111]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:37.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:37 smithi115 ceph-mon[29111]: Saving service crash spec with placement * 2024-03-22T10:30:37.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:37 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:37.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:37 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:38.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-03-22T10:30:38.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.034+0000 7f1aca200700 1 Processor -- start 2024-03-22T10:30:38.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.035+0000 7f1aca200700 1 -- start start 2024-03-22T10:30:38.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.035+0000 7f1aca200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac4108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:38.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.035+0000 7f1aca200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40744b0 con 0x7f1ac4106550 2024-03-22T10:30:38.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.035+0000 7f1ac3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac4108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:38.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.036+0000 7f1ac3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac4108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40790/0 (socket says 172.21.15.115:40790) 2024-03-22T10:30:38.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.036+0000 7f1ac3400700 1 -- 172.21.15.115:0/2788107017 learned_addr learned my addr 172.21.15.115:0/2788107017 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:38.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.036+0000 7f1ac3400700 1 -- 172.21.15.115:0/2788107017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac40745f0 con 0x7f1ac4106550 2024-03-22T10:30:38.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.036+0000 7f1ac3400700 1 --2- 172.21.15.115:0/2788107017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac4108930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40098a0 tx=0x7f1ab4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c9055867db3387 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:38.942 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.037+0000 7f1ac2000700 1 -- 172.21.15.115:0/2788107017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ab4004180 con 0x7f1ac4106550 2024-03-22T10:30:38.942 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.037+0000 7f1ac2000700 1 -- 172.21.15.115:0/2788107017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1ab40042e0 con 0x7f1ac4106550 2024-03-22T10:30:38.942 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.037+0000 7f1ac2000700 1 -- 172.21.15.115:0/2788107017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ab4011af0 con 0x7f1ac4106550 2024-03-22T10:30:38.942 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.038+0000 7f1aca200700 1 -- 172.21.15.115:0/2788107017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 msgr2=0x7f1ac4108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:38.942 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.038+0000 7f1aca200700 1 --2- 172.21.15.115:0/2788107017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac4108930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40098a0 tx=0x7f1ab4009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:38.942 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.038+0000 7f1aca200700 1 -- 172.21.15.115:0/2788107017 shutdown_connections 2024-03-22T10:30:38.943 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.038+0000 7f1aca200700 1 --2- 172.21.15.115:0/2788107017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac4108930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:38.943 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.038+0000 7f1aca200700 1 -- 172.21.15.115:0/2788107017 >> 172.21.15.115:0/2788107017 conn(0x7f1ac40fff10 msgr2=0x7f1ac4102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:38.943 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.038+0000 7f1aca200700 1 -- 172.21.15.115:0/2788107017 shutdown_connections 2024-03-22T10:30:38.943 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.038+0000 7f1aca200700 1 -- 172.21.15.115:0/2788107017 wait complete. 2024-03-22T10:30:38.943 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.039+0000 7f1aca200700 1 Processor -- start 2024-03-22T10:30:38.944 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.039+0000 7f1aca200700 1 -- start start 2024-03-22T10:30:38.944 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.039+0000 7f1aca200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:38.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.040+0000 7f1aca200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac419bec0 con 0x7f1ac4106550 2024-03-22T10:30:38.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.040+0000 7f1ac3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:38.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.040+0000 7f1ac3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac419b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40798/0 (socket says 172.21.15.115:40798) 2024-03-22T10:30:38.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.040+0000 7f1ac3400700 1 -- 172.21.15.115:0/3447314115 learned_addr learned my addr 172.21.15.115:0/3447314115 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:38.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.040+0000 7f1ac3400700 1 -- 172.21.15.115:0/3447314115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009550 con 0x7f1ac4106550 2024-03-22T10:30:38.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.040+0000 7f1ac3400700 1 --2- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac419b980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4011670 tx=0x7f1ab4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:38.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.041+0000 7f1abbe00700 1 -- 172.21.15.115:0/3447314115 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ab40039f0 con 0x7f1ac4106550 2024-03-22T10:30:38.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.041+0000 7f1abbe00700 1 -- 172.21.15.115:0/3447314115 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1ab4003b50 con 0x7f1ac4106550 2024-03-22T10:30:38.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.041+0000 7f1abbe00700 1 -- 172.21.15.115:0/3447314115 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ab40247c0 con 0x7f1ac4106550 2024-03-22T10:30:38.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.041+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac419c0c0 con 0x7f1ac4106550 2024-03-22T10:30:38.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.041+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac419c560 con 0x7f1ac4106550 2024-03-22T10:30:38.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.042+0000 7f1abbe00700 1 -- 172.21.15.115:0/3447314115 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f1ab401ac00 con 0x7f1ac4106550 2024-03-22T10:30:38.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.043+0000 7f1abbe00700 1 --2- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f1aa80383f0 0x7f1aa803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:38.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.043+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac4109ad0 con 0x7f1ac4106550 2024-03-22T10:30:38.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.043+0000 7f1abbe00700 1 -- 172.21.15.115:0/3447314115 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1ab404c500 con 0x7f1ac4106550 2024-03-22T10:30:38.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.043+0000 7f1ac2a00700 1 --2- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f1aa80383f0 0x7f1aa803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:38.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.044+0000 7f1ac2a00700 1 --2- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f1aa80383f0 0x7f1aa803a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0006fd0 tx=0x7f1ab0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:38.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.053+0000 7f1abbe00700 1 -- 172.21.15.115:0/3447314115 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1ab401a430 con 0x7f1ac4106550 2024-03-22T10:30:38.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.283+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ac41021f0 con 0x7f1aa80383f0 2024-03-22T10:30:38.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.294+0000 7f1abbe00700 1 -- 172.21.15.115:0/3447314115 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f1ac41021f0 con 0x7f1aa80383f0 2024-03-22T10:30:38.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.297+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f1aa80383f0 msgr2=0x7f1aa803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:38.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.297+0000 7f1aca200700 1 --2- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f1aa80383f0 0x7f1aa803a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0006fd0 tx=0x7f1ab0006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:38.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.297+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 msgr2=0x7f1ac419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:38.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.297+0000 7f1aca200700 1 --2- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac419b980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4011670 tx=0x7f1ab4011b50 comp rx=0 tx=0).stop 2024-03-22T10:30:38.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.298+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 shutdown_connections 2024-03-22T10:30:38.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.298+0000 7f1aca200700 1 --2- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f1aa80383f0 0x7f1aa803a8a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:38.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.298+0000 7f1aca200700 1 --2- 172.21.15.115:0/3447314115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ac4106550 0x7f1ac419b980 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:38.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.298+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 >> 172.21.15.115:0/3447314115 conn(0x7f1ac40fff10 msgr2=0x7f1ac4101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:38.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.298+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 shutdown_connections 2024-03-22T10:30:38.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:38.298+0000 7f1aca200700 1 -- 172.21.15.115:0/3447314115 wait complete. 2024-03-22T10:30:38.950 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying prometheus service with default placement... 2024-03-22T10:30:39.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:39 smithi115 ceph-mon[29111]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:39.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:39 smithi115 ceph-mon[29111]: Saving service ceph-exporter spec with placement * 2024-03-22T10:30:39.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:39 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:39.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:39 smithi115 ceph-mon[29111]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:30:39.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:39 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:40.790 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-22T10:30:40.790 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.572+0000 7f59af600700 1 Processor -- start 2024-03-22T10:30:40.790 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.572+0000 7f59af600700 1 -- start start 2024-03-22T10:30:40.790 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.572+0000 7f59af600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a80791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:40.790 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.572+0000 7f59af600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a80796e0 con 0x7f59a807aca0 2024-03-22T10:30:40.790 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.573+0000 7f59acc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a80791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:40.790 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.573+0000 7f59acc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a80791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40810/0 (socket says 172.21.15.115:40810) 2024-03-22T10:30:40.795 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.573+0000 7f59acc00700 1 -- 172.21.15.115:0/3109969012 learned_addr learned my addr 172.21.15.115:0/3109969012 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.573+0000 7f59acc00700 1 -- 172.21.15.115:0/3109969012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a8079820 con 0x7f59a807aca0 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.573+0000 7f59acc00700 1 --2- 172.21.15.115:0/3109969012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a80791a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f599c0098a0 tx=0x7f599c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b008becc5c2c8e5f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.574+0000 7f59a7400700 1 -- 172.21.15.115:0/3109969012 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f599c004180 con 0x7f59a807aca0 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.574+0000 7f59a7400700 1 -- 172.21.15.115:0/3109969012 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f599c0042e0 con 0x7f59a807aca0 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.574+0000 7f59a7400700 1 -- 172.21.15.115:0/3109969012 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f599c011af0 con 0x7f59a807aca0 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.575+0000 7f59af600700 1 -- 172.21.15.115:0/3109969012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 msgr2=0x7f59a80791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.575+0000 7f59af600700 1 --2- 172.21.15.115:0/3109969012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a80791a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f599c0098a0 tx=0x7f599c009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.575+0000 7f59af600700 1 -- 172.21.15.115:0/3109969012 shutdown_connections 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.575+0000 7f59af600700 1 --2- 172.21.15.115:0/3109969012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a80791a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.575+0000 7f59af600700 1 -- 172.21.15.115:0/3109969012 >> 172.21.15.115:0/3109969012 conn(0x7f59a8100fd0 msgr2=0x7f59a8103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.575+0000 7f59af600700 1 -- 172.21.15.115:0/3109969012 shutdown_connections 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.575+0000 7f59af600700 1 -- 172.21.15.115:0/3109969012 wait complete. 2024-03-22T10:30:40.796 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.576+0000 7f59af600700 1 Processor -- start 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.576+0000 7f59af600700 1 -- start start 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.577+0000 7f59af600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.577+0000 7f59af600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a819bed0 con 0x7f59a807aca0 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.577+0000 7f59acc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.577+0000 7f59acc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40814/0 (socket says 172.21.15.115:40814) 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.577+0000 7f59acc00700 1 -- 172.21.15.115:0/658005891 learned_addr learned my addr 172.21.15.115:0/658005891 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.577+0000 7f59acc00700 1 -- 172.21.15.115:0/658005891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c009550 con 0x7f59a807aca0 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.577+0000 7f59acc00700 1 --2- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a819b990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f599c011670 tx=0x7f599c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.578+0000 7f59a5600700 1 -- 172.21.15.115:0/658005891 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f599c0039f0 con 0x7f59a807aca0 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.578+0000 7f59a5600700 1 -- 172.21.15.115:0/658005891 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f599c003b50 con 0x7f59a807aca0 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.578+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a819c0d0 con 0x7f59a807aca0 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.578+0000 7f59a5600700 1 -- 172.21.15.115:0/658005891 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f599c0247c0 con 0x7f59a807aca0 2024-03-22T10:30:40.797 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.578+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a819c570 con 0x7f59a807aca0 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.579+0000 7f59a5600700 1 -- 172.21.15.115:0/658005891 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f599c02d3f0 con 0x7f59a807aca0 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.579+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5988005320 con 0x7f59a807aca0 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.580+0000 7f59a5600700 1 --2- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f5990038490 0x7f599003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.580+0000 7f59a5600700 1 -- 172.21.15.115:0/658005891 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f599c04c4f0 con 0x7f59a807aca0 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.580+0000 7f59a7e00700 1 --2- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f5990038490 0x7f599003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.581+0000 7f59a7e00700 1 --2- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f5990038490 0x7f599003a940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5994006fd0 tx=0x7f5994006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.585+0000 7f59a5600700 1 -- 172.21.15.115:0/658005891 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f599c010970 con 0x7f59a807aca0 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.822+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f5988000bf0 con 0x7f5990038490 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.830+0000 7f59a5600700 1 -- 172.21.15.115:0/658005891 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f5988000bf0 con 0x7f5990038490 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.833+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f5990038490 msgr2=0x7f599003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.833+0000 7f59af600700 1 --2- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f5990038490 0x7f599003a940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5994006fd0 tx=0x7f5994006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.833+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 msgr2=0x7f59a819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.833+0000 7f59af600700 1 --2- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a819b990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f599c011670 tx=0x7f599c011b50 comp rx=0 tx=0).stop 2024-03-22T10:30:40.798 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.834+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 shutdown_connections 2024-03-22T10:30:40.799 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.834+0000 7f59af600700 1 --2- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f5990038490 0x7f599003a940 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:40.799 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.834+0000 7f59af600700 1 --2- 172.21.15.115:0/658005891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59a807aca0 0x7f59a819b990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:40.799 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.834+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 >> 172.21.15.115:0/658005891 conn(0x7f59a8100fd0 msgr2=0x7f59a81026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:40.799 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.834+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 shutdown_connections 2024-03-22T10:30:40.799 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:39.834+0000 7f59af600700 1 -- 172.21.15.115:0/658005891 wait complete. 2024-03-22T10:30:40.799 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying grafana service with default placement... 2024-03-22T10:30:41.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:40 smithi115 ceph-mon[29111]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:41.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:40 smithi115 ceph-mon[29111]: Saving service prometheus spec with placement count:1 2024-03-22T10:30:41.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:40 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:42.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:41 smithi115 ceph-mon[29111]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:30:42.732 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-22T10:30:42.732 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.803+0000 7f0f15000700 1 Processor -- start 2024-03-22T10:30:42.732 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.803+0000 7f0f15000700 1 -- start start 2024-03-22T10:30:42.732 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.804+0000 7f0f15000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f10105c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:42.732 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.804+0000 7f0f15000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f10106150 con 0x7f0f10105800 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.804+0000 7f0f0f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f10105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.804+0000 7f0f0f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f10105c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35778/0 (socket says 172.21.15.115:35778) 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.804+0000 7f0f0f400700 1 -- 172.21.15.115:0/1183778252 learned_addr learned my addr 172.21.15.115:0/1183778252 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.805+0000 7f0f0f400700 1 -- 172.21.15.115:0/1183778252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f10106290 con 0x7f0f10105800 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.805+0000 7f0f0f400700 1 --2- 172.21.15.115:0/1183778252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f10105c10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0f000098a0 tx=0x7f0f00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29dc5359d857929b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.805+0000 7f0f0ea00700 1 -- 172.21.15.115:0/1183778252 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f00004180 con 0x7f0f10105800 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.805+0000 7f0f0ea00700 1 -- 172.21.15.115:0/1183778252 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f000042e0 con 0x7f0f10105800 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.805+0000 7f0f0ea00700 1 -- 172.21.15.115:0/1183778252 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f00011af0 con 0x7f0f10105800 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.806+0000 7f0f15000700 1 -- 172.21.15.115:0/1183778252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 msgr2=0x7f0f10105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.806+0000 7f0f15000700 1 --2- 172.21.15.115:0/1183778252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f10105c10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0f000098a0 tx=0x7f0f00009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.806+0000 7f0f15000700 1 -- 172.21.15.115:0/1183778252 shutdown_connections 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.806+0000 7f0f15000700 1 --2- 172.21.15.115:0/1183778252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f10105c10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.806+0000 7f0f15000700 1 -- 172.21.15.115:0/1183778252 >> 172.21.15.115:0/1183778252 conn(0x7f0f10100e50 msgr2=0x7f0f10103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:42.733 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.806+0000 7f0f15000700 1 -- 172.21.15.115:0/1183778252 shutdown_connections 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.806+0000 7f0f15000700 1 -- 172.21.15.115:0/1183778252 wait complete. 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.808+0000 7f0f15000700 1 Processor -- start 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.808+0000 7f0f15000700 1 -- start start 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.808+0000 7f0f15000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f1007ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.808+0000 7f0f15000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1007b050 con 0x7f0f10105800 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.809+0000 7f0f0f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f1007ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.809+0000 7f0f0f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f1007ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35786/0 (socket says 172.21.15.115:35786) 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.809+0000 7f0f0f400700 1 -- 172.21.15.115:0/2504410631 learned_addr learned my addr 172.21.15.115:0/2504410631 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.809+0000 7f0f0f400700 1 -- 172.21.15.115:0/2504410631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f00009550 con 0x7f0f10105800 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.809+0000 7f0f0f400700 1 --2- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f1007ab10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f00011670 tx=0x7f0f00011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.809+0000 7f0f0cc00700 1 -- 172.21.15.115:0/2504410631 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f00003a40 con 0x7f0f10105800 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.810+0000 7f0f0cc00700 1 -- 172.21.15.115:0/2504410631 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f00003ba0 con 0x7f0f10105800 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.810+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f10079160 con 0x7f0f10105800 2024-03-22T10:30:42.734 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.810+0000 7f0f0cc00700 1 -- 172.21.15.115:0/2504410631 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f00024790 con 0x7f0f10105800 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.810+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f10079600 con 0x7f0f10105800 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.811+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f10062380 con 0x7f0f10105800 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.811+0000 7f0f0cc00700 1 -- 172.21.15.115:0/2504410631 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0f0001a430 con 0x7f0f10105800 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.811+0000 7f0f0cc00700 1 --2- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0ef4038440 0x7f0ef403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.812+0000 7f0f0cc00700 1 -- 172.21.15.115:0/2504410631 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0f0004c410 con 0x7f0f10105800 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.812+0000 7f0f07e00700 1 --2- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0ef4038440 0x7f0ef403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.812+0000 7f0f07e00700 1 --2- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0ef4038440 0x7f0ef403a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006fd0 tx=0x7f0ef8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:41.816+0000 7f0f0cc00700 1 -- 172.21.15.115:0/2504410631 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0f0001ac50 con 0x7f0f10105800 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.041+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f10103110 con 0x7f0ef4038440 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.052+0000 7f0f0cc00700 1 -- 172.21.15.115:0/2504410631 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f0f10103110 con 0x7f0ef4038440 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.055+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0ef4038440 msgr2=0x7f0ef403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.055+0000 7f0f15000700 1 --2- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0ef4038440 0x7f0ef403a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006fd0 tx=0x7f0ef8006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.055+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 msgr2=0x7f0f1007ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:42.735 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.055+0000 7f0f15000700 1 --2- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f1007ab10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f00011670 tx=0x7f0f00011b50 comp rx=0 tx=0).stop 2024-03-22T10:30:42.736 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.055+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 shutdown_connections 2024-03-22T10:30:42.736 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.055+0000 7f0f15000700 1 --2- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f0ef4038440 0x7f0ef403a8f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:42.736 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.055+0000 7f0f15000700 1 --2- 172.21.15.115:0/2504410631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f10105800 0x7f0f1007ab10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:42.736 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.055+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 >> 172.21.15.115:0/2504410631 conn(0x7f0f10100e50 msgr2=0x7f0f101024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:42.736 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.056+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 shutdown_connections 2024-03-22T10:30:42.736 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:42.056+0000 7f0f15000700 1 -- 172.21.15.115:0/2504410631 wait complete. 2024-03-22T10:30:42.736 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying node-exporter service with default placement... 2024-03-22T10:30:43.331 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:43 smithi115 ceph-mon[29111]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:43.332 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:43 smithi115 ceph-mon[29111]: Saving service grafana spec with placement count:1 2024-03-22T10:30:43.332 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:43 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:43.332 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:43 smithi115 ceph-mon[29111]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.336+0000 7f4824400700 1 Processor -- start 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.336+0000 7f4824400700 1 -- start start 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.337+0000 7f4824400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.337+0000 7f4824400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c108530 con 0x7f481c107be0 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.337+0000 7f4821a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.338+0000 7f4821a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c107ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35788/0 (socket says 172.21.15.115:35788) 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.338+0000 7f4821a00700 1 -- 172.21.15.115:0/600613252 learned_addr learned my addr 172.21.15.115:0/600613252 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.338+0000 7f4821a00700 1 -- 172.21.15.115:0/600613252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c108670 con 0x7f481c107be0 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.338+0000 7f4821a00700 1 --2- 172.21.15.115:0/600613252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c107ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48100098a0 tx=0x7f4810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f37464e2f00dcf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:44.208 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.339+0000 7f481be00700 1 -- 172.21.15.115:0/600613252 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4810004180 con 0x7f481c107be0 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.339+0000 7f481be00700 1 -- 172.21.15.115:0/600613252 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f48100042e0 con 0x7f481c107be0 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.339+0000 7f481be00700 1 -- 172.21.15.115:0/600613252 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4810011af0 con 0x7f481c107be0 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.339+0000 7f4824400700 1 -- 172.21.15.115:0/600613252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 msgr2=0x7f481c107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.339+0000 7f4824400700 1 --2- 172.21.15.115:0/600613252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c107ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48100098a0 tx=0x7f4810009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.340+0000 7f4824400700 1 -- 172.21.15.115:0/600613252 shutdown_connections 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.340+0000 7f4824400700 1 --2- 172.21.15.115:0/600613252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c107ff0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.340+0000 7f4824400700 1 -- 172.21.15.115:0/600613252 >> 172.21.15.115:0/600613252 conn(0x7f481c07b200 msgr2=0x7f481c07b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.340+0000 7f4824400700 1 -- 172.21.15.115:0/600613252 shutdown_connections 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.340+0000 7f4824400700 1 -- 172.21.15.115:0/600613252 wait complete. 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.341+0000 7f4824400700 1 Processor -- start 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.341+0000 7f4824400700 1 -- start start 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.341+0000 7f4824400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.341+0000 7f4824400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c19bef0 con 0x7f481c107be0 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.341+0000 7f4821a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.342+0000 7f4821a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c19b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35804/0 (socket says 172.21.15.115:35804) 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.342+0000 7f4821a00700 1 -- 172.21.15.115:0/2225783833 learned_addr learned my addr 172.21.15.115:0/2225783833 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.342+0000 7f4821a00700 1 -- 172.21.15.115:0/2225783833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810009550 con 0x7f481c107be0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.342+0000 7f4821a00700 1 --2- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c19b9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4810011670 tx=0x7f4810011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.342+0000 7f481a000700 1 -- 172.21.15.115:0/2225783833 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4810003a40 con 0x7f481c107be0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.342+0000 7f481a000700 1 -- 172.21.15.115:0/2225783833 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4810003ba0 con 0x7f481c107be0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.342+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c19c0f0 con 0x7f481c107be0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.343+0000 7f481a000700 1 -- 172.21.15.115:0/2225783833 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4810024820 con 0x7f481c107be0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.343+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c19c590 con 0x7f481c107be0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.344+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c062380 con 0x7f481c107be0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.344+0000 7f481a000700 1 -- 172.21.15.115:0/2225783833 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f481001ac70 con 0x7f481c107be0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.344+0000 7f481a000700 1 --2- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4804038400 0x7f480403a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.345+0000 7f481a000700 1 -- 172.21.15.115:0/2225783833 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f481004c300 con 0x7f481c107be0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.345+0000 7f4821000700 1 --2- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4804038400 0x7f480403a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.345+0000 7f4821000700 1 --2- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4804038400 0x7f480403a8b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4808006fd0 tx=0x7f4808006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:44.210 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.350+0000 7f481a000700 1 -- 172.21.15.115:0/2225783833 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4810018760 con 0x7f481c107be0 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.578+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f481c106c60 con 0x7f4804038400 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.588+0000 7f481a000700 1 -- 172.21.15.115:0/2225783833 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f481c106c60 con 0x7f4804038400 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.591+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4804038400 msgr2=0x7f480403a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.591+0000 7f4824400700 1 --2- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4804038400 0x7f480403a8b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4808006fd0 tx=0x7f4808006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.591+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 msgr2=0x7f481c19b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.591+0000 7f4824400700 1 --2- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c19b9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4810011670 tx=0x7f4810011aa0 comp rx=0 tx=0).stop 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.591+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 shutdown_connections 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.591+0000 7f4824400700 1 --2- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4804038400 0x7f480403a8b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.591+0000 7f4824400700 1 --2- 172.21.15.115:0/2225783833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c107be0 0x7f481c19b9b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.591+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 >> 172.21.15.115:0/2225783833 conn(0x7f481c07b200 msgr2=0x7f481c106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.592+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 shutdown_connections 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:43.592+0000 7f4824400700 1 -- 172.21.15.115:0/2225783833 wait complete. 2024-03-22T10:30:44.211 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying alertmanager service with default placement... 2024-03-22T10:30:44.846 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:44 smithi115 ceph-mon[29111]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:44.846 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:44 smithi115 ceph-mon[29111]: Saving service node-exporter spec with placement * 2024-03-22T10:30:44.846 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:44 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:45.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:45 smithi115 ceph-mon[29111]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:30:45.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:45 smithi115 ceph-mon[29111]: from='mgr.14120 172.21.15.115:0/579963321' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:30:45.756 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-22T10:30:45.756 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.848+0000 7f05bbe00700 1 Processor -- start 2024-03-22T10:30:45.756 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.848+0000 7f05bbe00700 1 -- start start 2024-03-22T10:30:45.756 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.849+0000 7f05bbe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b4105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:45.757 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.849+0000 7f05bbe00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4106320 con 0x7f05b41059d0 2024-03-22T10:30:45.757 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.849+0000 7f05b9400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b4105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:45.757 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.849+0000 7f05b9400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b4105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35806/0 (socket says 172.21.15.115:35806) 2024-03-22T10:30:45.757 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.849+0000 7f05b9400700 1 -- 172.21.15.115:0/3760043061 learned_addr learned my addr 172.21.15.115:0/3760043061 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:45.757 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.850+0000 7f05b9400700 1 -- 172.21.15.115:0/3760043061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4106460 con 0x7f05b41059d0 2024-03-22T10:30:45.757 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.850+0000 7f05b9400700 1 --2- 172.21.15.115:0/3760043061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b4105de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009890 tx=0x7f05a8009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c033c62a0d7308a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:45.762 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.850+0000 7f05b3e00700 1 -- 172.21.15.115:0/3760043061 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a8004180 con 0x7f05b41059d0 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.851+0000 7f05b3e00700 1 -- 172.21.15.115:0/3760043061 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05a80042e0 con 0x7f05b41059d0 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.851+0000 7f05b3e00700 1 -- 172.21.15.115:0/3760043061 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a8011b10 con 0x7f05b41059d0 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.851+0000 7f05bbe00700 1 -- 172.21.15.115:0/3760043061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 msgr2=0x7f05b4105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.851+0000 7f05bbe00700 1 --2- 172.21.15.115:0/3760043061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b4105de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009890 tx=0x7f05a8009ba0 comp rx=0 tx=0).stop 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.851+0000 7f05bbe00700 1 -- 172.21.15.115:0/3760043061 shutdown_connections 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.851+0000 7f05bbe00700 1 --2- 172.21.15.115:0/3760043061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b4105de0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.851+0000 7f05bbe00700 1 -- 172.21.15.115:0/3760043061 >> 172.21.15.115:0/3760043061 conn(0x7f05b4101020 msgr2=0x7f05b4103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.852+0000 7f05bbe00700 1 -- 172.21.15.115:0/3760043061 shutdown_connections 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.852+0000 7f05bbe00700 1 -- 172.21.15.115:0/3760043061 wait complete. 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.853+0000 7f05bbe00700 1 Processor -- start 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.853+0000 7f05bbe00700 1 -- start start 2024-03-22T10:30:45.763 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.853+0000 7f05bbe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.853+0000 7f05bbe00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b419be20 con 0x7f05b41059d0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.853+0000 7f05b9400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.853+0000 7f05b9400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35814/0 (socket says 172.21.15.115:35814) 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.853+0000 7f05b9400700 1 -- 172.21.15.115:0/1489971977 learned_addr learned my addr 172.21.15.115:0/1489971977 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.854+0000 7f05b9400700 1 -- 172.21.15.115:0/1489971977 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80094f0 con 0x7f05b41059d0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.854+0000 7f05b9400700 1 --2- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b419b8e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05a8006fa0 tx=0x7f05a80039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.854+0000 7f05b2000700 1 -- 172.21.15.115:0/1489971977 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a8003fa0 con 0x7f05b41059d0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.854+0000 7f05b2000700 1 -- 172.21.15.115:0/1489971977 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05a8004100 con 0x7f05b41059d0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.854+0000 7f05b2000700 1 -- 172.21.15.115:0/1489971977 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a8019900 con 0x7f05b41059d0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.854+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b419c020 con 0x7f05b41059d0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.855+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b419c4c0 con 0x7f05b41059d0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.856+0000 7f05b2000700 1 -- 172.21.15.115:0/1489971977 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f05a8019b30 con 0x7f05b41059d0 2024-03-22T10:30:45.764 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.856+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b4062380 con 0x7f05b41059d0 2024-03-22T10:30:45.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.856+0000 7f05b2000700 1 --2- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f059c0383f0 0x7f059c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:45.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.856+0000 7f05b2000700 1 -- 172.21.15.115:0/1489971977 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f05a804c4f0 con 0x7f05b41059d0 2024-03-22T10:30:45.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.856+0000 7f05b8a00700 1 --2- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f059c0383f0 0x7f059c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:45.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.857+0000 7f05b8a00700 1 --2- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f059c0383f0 0x7f059c03a8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05a0006fd0 tx=0x7f05a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:45.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:44.861+0000 7f05b2000700 1 -- 172.21.15.115:0/1489971977 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f05a802c7c0 con 0x7f05b41059d0 2024-03-22T10:30:45.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.087+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f05b419f300 con 0x7f059c0383f0 2024-03-22T10:30:45.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.094+0000 7f05b2000700 1 -- 172.21.15.115:0/1489971977 <== mgr.14120 v2:172.21.15.115:6800/778669970 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f05b419f300 con 0x7f059c0383f0 2024-03-22T10:30:45.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.097+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f059c0383f0 msgr2=0x7f059c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:45.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.097+0000 7f05bbe00700 1 --2- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f059c0383f0 0x7f059c03a8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05a0006fd0 tx=0x7f05a0006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:45.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.097+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 msgr2=0x7f05b419b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:45.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.097+0000 7f05bbe00700 1 --2- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b419b8e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05a8006fa0 tx=0x7f05a80039f0 comp rx=0 tx=0).stop 2024-03-22T10:30:45.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.097+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 shutdown_connections 2024-03-22T10:30:45.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.097+0000 7f05bbe00700 1 --2- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f059c0383f0 0x7f059c03a8a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:45.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.097+0000 7f05bbe00700 1 --2- 172.21.15.115:0/1489971977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05b41059d0 0x7f05b419b8e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:45.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.097+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 >> 172.21.15.115:0/1489971977 conn(0x7f05b4101020 msgr2=0x7f05b4101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:45.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.097+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 shutdown_connections 2024-03-22T10:30:45.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:45.098+0000 7f05bbe00700 1 -- 172.21.15.115:0/1489971977 wait complete. 2024-03-22T10:30:46.880 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:46 smithi115 ceph-mon[29111]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:30:46.880 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:46 smithi115 ceph-mon[29111]: Saving service alertmanager spec with placement count:1 2024-03-22T10:30:46.880 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:46 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/911238271' entity='client.admin' 2024-03-22T10:30:47.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.324+0000 7fb008c00700 1 Processor -- start 2024-03-22T10:30:47.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.324+0000 7fb008c00700 1 -- start start 2024-03-22T10:30:47.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.325+0000 7fb008c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004079cd0 0x7fb00410a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:47.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.325+0000 7fb008c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004106350 con 0x7fb004079cd0 2024-03-22T10:30:47.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.325+0000 7fb001e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004079cd0 0x7fb00410a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:47.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.325+0000 7fb001e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004079cd0 0x7fb00410a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35820/0 (socket says 172.21.15.115:35820) 2024-03-22T10:30:47.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.325+0000 7fb001e00700 1 -- 172.21.15.115:0/2021319312 learned_addr learned my addr 172.21.15.115:0/2021319312 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:47.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.326+0000 7fb001e00700 1 -- 172.21.15.115:0/2021319312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00410aea0 con 0x7fb004079cd0 2024-03-22T10:30:47.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.326+0000 7fb001e00700 1 --2- 172.21.15.115:0/2021319312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004079cd0 0x7fb00410a960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faff40098a0 tx=0x7faff4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8edc4c12083821 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:47.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.326+0000 7fb000a00700 1 -- 172.21.15.115:0/2021319312 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faff4004180 con 0x7fb004079cd0 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.326+0000 7fb000a00700 1 -- 172.21.15.115:0/2021319312 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faff40042e0 con 0x7fb004079cd0 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.326+0000 7fb000a00700 1 -- 172.21.15.115:0/2021319312 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faff4011af0 con 0x7fb004079cd0 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.327+0000 7fb008c00700 1 -- 172.21.15.115:0/2021319312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004079cd0 msgr2=0x7fb00410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.327+0000 7fb008c00700 1 --2- 172.21.15.115:0/2021319312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004079cd0 0x7fb00410a960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faff40098a0 tx=0x7faff4009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.327+0000 7fb008c00700 1 -- 172.21.15.115:0/2021319312 shutdown_connections 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.327+0000 7fb008c00700 1 --2- 172.21.15.115:0/2021319312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004079cd0 0x7fb00410a960 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.327+0000 7fb008c00700 1 -- 172.21.15.115:0/2021319312 >> 172.21.15.115:0/2021319312 conn(0x7fb0040ffef0 msgr2=0x7fb004102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.327+0000 7fb008c00700 1 -- 172.21.15.115:0/2021319312 shutdown_connections 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.327+0000 7fb008c00700 1 -- 172.21.15.115:0/2021319312 wait complete. 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.328+0000 7fb008c00700 1 Processor -- start 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.328+0000 7fb008c00700 1 -- start start 2024-03-22T10:30:47.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.329+0000 7fb008c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb00419b940 0x7fb00419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.329+0000 7fb008c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00419c290 con 0x7fb00419b940 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.329+0000 7fb001e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb00419b940 0x7fb00419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.329+0000 7fb001e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb00419b940 0x7fb00419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35832/0 (socket says 172.21.15.115:35832) 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.329+0000 7fb001e00700 1 -- 172.21.15.115:0/911238271 learned_addr learned my addr 172.21.15.115:0/911238271 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.329+0000 7fb001e00700 1 -- 172.21.15.115:0/911238271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4009550 con 0x7fb00419b940 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.330+0000 7fb001e00700 1 --2- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb00419b940 0x7fb00419bd50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faff4011670 tx=0x7faff4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.330+0000 7faffaa00700 1 -- 172.21.15.115:0/911238271 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faff40039f0 con 0x7fb00419b940 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.330+0000 7faffaa00700 1 -- 172.21.15.115:0/911238271 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faff4003b50 con 0x7fb00419b940 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.330+0000 7faffaa00700 1 -- 172.21.15.115:0/911238271 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faff40247c0 con 0x7fb00419b940 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.330+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00419c490 con 0x7fb00419b940 2024-03-22T10:30:47.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.330+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00419f0f0 con 0x7fb00419b940 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.331+0000 7faffaa00700 1 -- 172.21.15.115:0/911238271 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7faff401ac00 con 0x7fb00419b940 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.331+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb004079cd0 con 0x7fb00419b940 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.332+0000 7faffaa00700 1 --2- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fafe80383f0 0x7fafe803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.332+0000 7faffaa00700 1 -- 172.21.15.115:0/911238271 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faff404c490 con 0x7fb00419b940 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.332+0000 7fb001400700 1 --2- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fafe80383f0 0x7fafe803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.333+0000 7fb001400700 1 --2- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fafe80383f0 0x7fafe803a8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faff0006fd0 tx=0x7faff0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.337+0000 7faffaa00700 1 -- 172.21.15.115:0/911238271 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7faff401a430 con 0x7fb00419b940 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.547+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fb004079ef0 con 0x7fb00419b940 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.555+0000 7faffaa00700 1 -- 172.21.15.115:0/911238271 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7faff4018b40 con 0x7fb00419b940 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.557+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fafe80383f0 msgr2=0x7fafe803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:47.190 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.557+0000 7fb008c00700 1 --2- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fafe80383f0 0x7fafe803a8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faff0006fd0 tx=0x7faff0006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:47.191 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.558+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb00419b940 msgr2=0x7fb00419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:47.191 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.558+0000 7fb008c00700 1 --2- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb00419b940 0x7fb00419bd50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faff4011670 tx=0x7faff4011b50 comp rx=0 tx=0).stop 2024-03-22T10:30:47.191 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.558+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 shutdown_connections 2024-03-22T10:30:47.191 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.558+0000 7fb008c00700 1 --2- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fafe80383f0 0x7fafe803a8a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:47.191 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.558+0000 7fb008c00700 1 --2- 172.21.15.115:0/911238271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb00419b940 0x7fb00419bd50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:47.191 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.558+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 >> 172.21.15.115:0/911238271 conn(0x7fb0040ffef0 msgr2=0x7fb004101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:47.191 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.558+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 shutdown_connections 2024-03-22T10:30:47.191 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:46.558+0000 7fb008c00700 1 -- 172.21.15.115:0/911238271 wait complete. 2024-03-22T10:30:47.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:47 smithi115 ceph-mon[29111]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:30:48.658 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.771+0000 7fe58e800700 1 Processor -- start 2024-03-22T10:30:48.658 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.771+0000 7fe58e800700 1 -- start start 2024-03-22T10:30:48.658 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.771+0000 7fe58e800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe588108910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:48.658 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.771+0000 7fe58e800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880744d0 con 0x7fe588106530 2024-03-22T10:30:48.658 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.772+0000 7fe587400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe588108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:48.658 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.772+0000 7fe587400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe588108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35834/0 (socket says 172.21.15.115:35834) 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.772+0000 7fe587400700 1 -- 172.21.15.115:0/1725542601 learned_addr learned my addr 172.21.15.115:0/1725542601 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.772+0000 7fe587400700 1 -- 172.21.15.115:0/1725542601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588074610 con 0x7fe588106530 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.773+0000 7fe587400700 1 --2- 172.21.15.115:0/1725542601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe588108910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5780098a0 tx=0x7fe578009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224b91969559013 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.773+0000 7fe586000700 1 -- 172.21.15.115:0/1725542601 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe578004180 con 0x7fe588106530 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.773+0000 7fe586000700 1 -- 172.21.15.115:0/1725542601 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe5780042e0 con 0x7fe588106530 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.773+0000 7fe586000700 1 -- 172.21.15.115:0/1725542601 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe578011af0 con 0x7fe588106530 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.773+0000 7fe58e800700 1 -- 172.21.15.115:0/1725542601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 msgr2=0x7fe588108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.773+0000 7fe58e800700 1 --2- 172.21.15.115:0/1725542601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe588108910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5780098a0 tx=0x7fe578009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.774+0000 7fe58e800700 1 -- 172.21.15.115:0/1725542601 shutdown_connections 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.774+0000 7fe58e800700 1 --2- 172.21.15.115:0/1725542601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe588108910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.774+0000 7fe58e800700 1 -- 172.21.15.115:0/1725542601 >> 172.21.15.115:0/1725542601 conn(0x7fe5880fff10 msgr2=0x7fe588102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.774+0000 7fe58e800700 1 -- 172.21.15.115:0/1725542601 shutdown_connections 2024-03-22T10:30:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.774+0000 7fe58e800700 1 -- 172.21.15.115:0/1725542601 wait complete. 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.775+0000 7fe58e800700 1 Processor -- start 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.775+0000 7fe58e800700 1 -- start start 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.775+0000 7fe58e800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe58819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.775+0000 7fe58e800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58819be90 con 0x7fe588106530 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.775+0000 7fe587400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe58819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.776+0000 7fe587400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe58819b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35848/0 (socket says 172.21.15.115:35848) 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.776+0000 7fe587400700 1 -- 172.21.15.115:0/2650543170 learned_addr learned my addr 172.21.15.115:0/2650543170 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.776+0000 7fe587400700 1 -- 172.21.15.115:0/2650543170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe578009550 con 0x7fe588106530 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.776+0000 7fe587400700 1 --2- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe58819b950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe578011670 tx=0x7fe578011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.776+0000 7fe57fe00700 1 -- 172.21.15.115:0/2650543170 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5780039c0 con 0x7fe588106530 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.777+0000 7fe57fe00700 1 -- 172.21.15.115:0/2650543170 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe578003b20 con 0x7fe588106530 2024-03-22T10:30:48.660 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.777+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58819c0f0 con 0x7fe588106530 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.777+0000 7fe57fe00700 1 -- 172.21.15.115:0/2650543170 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5780247f0 con 0x7fe588106530 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.777+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58819eef0 con 0x7fe588106530 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.778+0000 7fe57fe00700 1 -- 172.21.15.115:0/2650543170 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fe57801ac70 con 0x7fe588106530 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.778+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe588062380 con 0x7fe588106530 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.778+0000 7fe57fe00700 1 --2- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fe56c038400 0x7fe56c03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.778+0000 7fe57fe00700 1 -- 172.21.15.115:0/2650543170 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe57804c3b0 con 0x7fe588106530 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.779+0000 7fe586a00700 1 --2- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fe56c038400 0x7fe56c03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.779+0000 7fe586a00700 1 --2- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fe56c038400 0x7fe56c03a8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe574006fd0 tx=0x7fe574006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.783+0000 7fe57fe00700 1 -- 172.21.15.115:0/2650543170 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe578018760 con 0x7fe588106530 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:47.997+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe58819f780 con 0x7fe588106530 2024-03-22T10:30:48.661 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.005+0000 7fe57fe00700 1 -- 172.21.15.115:0/2650543170 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe578018350 con 0x7fe588106530 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.008+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fe56c038400 msgr2=0x7fe56c03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.008+0000 7fe58e800700 1 --2- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fe56c038400 0x7fe56c03a8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe574006fd0 tx=0x7fe574006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.008+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 msgr2=0x7fe58819b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.008+0000 7fe58e800700 1 --2- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe58819b950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe578011670 tx=0x7fe578011aa0 comp rx=0 tx=0).stop 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.008+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 shutdown_connections 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.008+0000 7fe58e800700 1 --2- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fe56c038400 0x7fe56c03a8b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.008+0000 7fe58e800700 1 --2- 172.21.15.115:0/2650543170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe588106530 0x7fe58819b950 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.008+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 >> 172.21.15.115:0/2650543170 conn(0x7fe5880fff10 msgr2=0x7fe5881015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.008+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 shutdown_connections 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:48.009+0000 7fe58e800700 1 -- 172.21.15.115:0/2650543170 wait complete. 2024-03-22T10:30:48.662 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling the dashboard module... 2024-03-22T10:30:49.180 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:49 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2650543170' entity='client.admin' 2024-03-22T10:30:49.180 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:49 smithi115 ceph-mon[29111]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:30:50.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:50 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/53365970' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.258+0000 7f60cf200700 1 Processor -- start 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.258+0000 7f60cf200700 1 -- start start 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.259+0000 7f60cf200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c8079cd0 0x7f60c810a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.259+0000 7f60cf200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c8106360 con 0x7f60c8079cd0 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.259+0000 7f60c7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c8079cd0 0x7f60c810a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.259+0000 7f60c7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c8079cd0 0x7f60c810a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35850/0 (socket says 172.21.15.115:35850) 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.259+0000 7f60c7e00700 1 -- 172.21.15.115:0/722098239 learned_addr learned my addr 172.21.15.115:0/722098239 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.260+0000 7f60c7e00700 1 -- 172.21.15.115:0/722098239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c810aeb0 con 0x7f60c8079cd0 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.260+0000 7f60c7e00700 1 --2- 172.21.15.115:0/722098239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c8079cd0 0x7f60c810a970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009e80 tx=0x7f60b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ba61dc25f48aab server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:50.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.260+0000 7f60c7400700 1 -- 172.21.15.115:0/722098239 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60b800e070 con 0x7f60c8079cd0 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.260+0000 7f60c7400700 1 -- 172.21.15.115:0/722098239 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f60b800b8c0 con 0x7f60c8079cd0 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.260+0000 7f60c7400700 1 -- 172.21.15.115:0/722098239 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60b80044d0 con 0x7f60c8079cd0 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.261+0000 7f60cf200700 1 -- 172.21.15.115:0/722098239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c8079cd0 msgr2=0x7f60c810a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.261+0000 7f60cf200700 1 --2- 172.21.15.115:0/722098239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c8079cd0 0x7f60c810a970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009e80 tx=0x7f60b800b330 comp rx=0 tx=0).stop 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.261+0000 7f60cf200700 1 -- 172.21.15.115:0/722098239 shutdown_connections 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.261+0000 7f60cf200700 1 --2- 172.21.15.115:0/722098239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c8079cd0 0x7f60c810a970 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.261+0000 7f60cf200700 1 -- 172.21.15.115:0/722098239 >> 172.21.15.115:0/722098239 conn(0x7f60c80fff00 msgr2=0x7f60c8102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.261+0000 7f60cf200700 1 -- 172.21.15.115:0/722098239 shutdown_connections 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.261+0000 7f60cf200700 1 -- 172.21.15.115:0/722098239 wait complete. 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.262+0000 7f60cf200700 1 Processor -- start 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.263+0000 7f60cf200700 1 -- start start 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.263+0000 7f60cf200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c819b8c0 0x7f60c819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.263+0000 7f60cf200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c819c210 con 0x7f60c819b8c0 2024-03-22T10:30:50.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.263+0000 7f60c7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c819b8c0 0x7f60c819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.263+0000 7f60c7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c819b8c0 0x7f60c819bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35858/0 (socket says 172.21.15.115:35858) 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.263+0000 7f60c7e00700 1 -- 172.21.15.115:0/53365970 learned_addr learned my addr 172.21.15.115:0/53365970 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.264+0000 7f60c7e00700 1 -- 172.21.15.115:0/53365970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8009b30 con 0x7f60c819b8c0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.264+0000 7f60c7e00700 1 --2- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c819b8c0 0x7f60c819bcd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60b8004940 tx=0x7f60b8004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.264+0000 7f60c5600700 1 -- 172.21.15.115:0/53365970 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60b800e040 con 0x7f60c819b8c0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.264+0000 7f60c5600700 1 -- 172.21.15.115:0/53365970 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f60b800bc40 con 0x7f60c819b8c0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.264+0000 7f60c5600700 1 -- 172.21.15.115:0/53365970 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60b8012670 con 0x7f60c819b8c0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.264+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c819c410 con 0x7f60c819b8c0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.265+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c819f070 con 0x7f60c819b8c0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.266+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c8062380 con 0x7f60c819b8c0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.266+0000 7f60c5600700 1 -- 172.21.15.115:0/53365970 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f60b8019040 con 0x7f60c819b8c0 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.266+0000 7f60c5600700 1 --2- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f60ac0383f0 0x7f60ac03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:50.687 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.266+0000 7f60c5600700 1 -- 172.21.15.115:0/53365970 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f60b804bbe0 con 0x7f60c819b8c0 2024-03-22T10:30:50.689 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.267+0000 7f60bf400700 1 --2- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f60ac0383f0 0x7f60ac03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:50.689 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.267+0000 7f60bf400700 1 --2- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f60ac0383f0 0x7f60ac03a8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006fd0 tx=0x7f60b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:50.689 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.272+0000 7f60c5600700 1 -- 172.21.15.115:0/53365970 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f60b801b430 con 0x7f60c819b8c0 2024-03-22T10:30:50.689 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:49.546+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f60c8079cd0 con 0x7f60c819b8c0 2024-03-22T10:30:50.689 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.015+0000 7f60c5600700 1 -- 172.21.15.115:0/53365970 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f60b801bb00 con 0x7f60c819b8c0 2024-03-22T10:30:50.689 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.018+0000 7f60c5600700 1 -- 172.21.15.115:0/53365970 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f60b800bdb0 con 0x7f60c819b8c0 2024-03-22T10:30:50.689 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.021+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f60ac0383f0 msgr2=0x7f60ac03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:50.689 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.021+0000 7f60cf200700 1 --2- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f60ac0383f0 0x7f60ac03a8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006fd0 tx=0x7f60b0006e40 comp rx=0 tx=0).stop 2024-03-22T10:30:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.021+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c819b8c0 msgr2=0x7f60c819bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.021+0000 7f60cf200700 1 --2- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c819b8c0 0x7f60c819bcd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60b8004940 tx=0x7f60b8004a20 comp rx=0 tx=0).stop 2024-03-22T10:30:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.021+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 shutdown_connections 2024-03-22T10:30:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.021+0000 7f60cf200700 1 --2- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f60ac0383f0 0x7f60ac03a8a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.021+0000 7f60cf200700 1 --2- 172.21.15.115:0/53365970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c819b8c0 0x7f60c819bcd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.021+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 >> 172.21.15.115:0/53365970 conn(0x7f60c80fff00 msgr2=0x7f60c8100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.022+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 shutdown_connections 2024-03-22T10:30:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:50.022+0000 7f60cf200700 1 -- 172.21.15.115:0/53365970 wait complete. 2024-03-22T10:30:51.187 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:51 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/53365970' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-22T10:30:51.187 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:51 smithi115 ceph-mon[29111]: mgrmap e9: smithi115.iqzwsz(active, since 31s) 2024-03-22T10:30:52.231 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-03-22T10:30:52.231 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-22T10:30:52.231 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T10:30:52.231 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "active_name": "smithi115.iqzwsz", 2024-03-22T10:30:52.231 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T10:30:52.231 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-03-22T10:30:52.231 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.261+0000 7f4d0ce00700 1 Processor -- start 2024-03-22T10:30:52.231 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.261+0000 7f4d0ce00700 1 -- start start 2024-03-22T10:30:52.232 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.261+0000 7f4d0ce00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d08104b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:52.237 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.261+0000 7f4d0ce00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08105050 con 0x7f4d08104700 2024-03-22T10:30:52.237 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.262+0000 7f4d07400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d08104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:52.237 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.262+0000 7f4d07400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d08104b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35870/0 (socket says 172.21.15.115:35870) 2024-03-22T10:30:52.237 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.262+0000 7f4d07400700 1 -- 172.21.15.115:0/2475590010 learned_addr learned my addr 172.21.15.115:0/2475590010 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:52.237 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.262+0000 7f4d07400700 1 -- 172.21.15.115:0/2475590010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08105190 con 0x7f4d08104700 2024-03-22T10:30:52.237 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.262+0000 7f4d07400700 1 --2- 172.21.15.115:0/2475590010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d08104b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80098a0 tx=0x7f4cf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff007819f39bdd38 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:52.237 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.263+0000 7f4d06000700 1 -- 172.21.15.115:0/2475590010 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cf8004180 con 0x7f4d08104700 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.263+0000 7f4d06000700 1 -- 172.21.15.115:0/2475590010 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4cf80042e0 con 0x7f4d08104700 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.263+0000 7f4d06000700 1 -- 172.21.15.115:0/2475590010 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cf8011af0 con 0x7f4d08104700 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.263+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2475590010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 msgr2=0x7f4d08104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.263+0000 7f4d0ce00700 1 --2- 172.21.15.115:0/2475590010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d08104b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80098a0 tx=0x7f4cf8009bb0 comp rx=0 tx=0).stop 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.264+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2475590010 shutdown_connections 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.264+0000 7f4d0ce00700 1 --2- 172.21.15.115:0/2475590010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d08104b10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.264+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2475590010 >> 172.21.15.115:0/2475590010 conn(0x7f4d080ffd30 msgr2=0x7f4d08102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.264+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2475590010 shutdown_connections 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.264+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2475590010 wait complete. 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.265+0000 7f4d0ce00700 1 Processor -- start 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.265+0000 7f4d0ce00700 1 -- start start 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.265+0000 7f4d0ce00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d0818f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.265+0000 7f4d0ce00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0818fe20 con 0x7f4d08104700 2024-03-22T10:30:52.238 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.265+0000 7f4d07400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d0818f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.265+0000 7f4d07400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d0818f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35886/0 (socket says 172.21.15.115:35886) 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.265+0000 7f4d07400700 1 -- 172.21.15.115:0/2857713352 learned_addr learned my addr 172.21.15.115:0/2857713352 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.266+0000 7f4d07400700 1 -- 172.21.15.115:0/2857713352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf8009550 con 0x7f4d08104700 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.266+0000 7f4d07400700 1 --2- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d0818f8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8011670 tx=0x7f4cf8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.266+0000 7f4cffe00700 1 -- 172.21.15.115:0/2857713352 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cf8003a40 con 0x7f4d08104700 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.266+0000 7f4cffe00700 1 -- 172.21.15.115:0/2857713352 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4cf8003ba0 con 0x7f4d08104700 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.266+0000 7f4cffe00700 1 -- 172.21.15.115:0/2857713352 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cf8024790 con 0x7f4d08104700 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.266+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0818df30 con 0x7f4d08104700 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.266+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0818e3d0 con 0x7f4d08104700 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.267+0000 7f4cffe00700 1 -- 172.21.15.115:0/2857713352 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f4cf801a430 con 0x7f4d08104700 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.268+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d08062380 con 0x7f4d08104700 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.268+0000 7f4cffe00700 1 --2- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4cec0384e0 0x7f4cec03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.268+0000 7f4cffe00700 1 -- 172.21.15.115:0/2857713352 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4cf804c520 con 0x7f4d08104700 2024-03-22T10:30:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.268+0000 7f4d06a00700 1 -- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4cec0384e0 msgr2=0x7f4cec03a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/778669970 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.268+0000 7f4d06a00700 1 --2- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4cec0384e0 0x7f4cec03a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.274+0000 7f4cffe00700 1 -- 172.21.15.115:0/2857713352 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4cf801ac50 con 0x7f4d08104700 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.469+0000 7f4d06a00700 1 -- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4cec0384e0 msgr2=0x7f4cec03a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/778669970 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.469+0000 7f4d06a00700 1 --2- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4cec0384e0 0x7f4cec03a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.545+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f4d081065b0 con 0x7f4d08104700 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.546+0000 7f4cffe00700 1 -- 172.21.15.115:0/2857713352 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f4cf8018b40 con 0x7f4d08104700 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.548+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4cec0384e0 msgr2=0x7f4cec03a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.548+0000 7f4d0ce00700 1 --2- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4cec0384e0 0x7f4cec03a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.548+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 msgr2=0x7f4d0818f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.548+0000 7f4d0ce00700 1 --2- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d0818f8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8011670 tx=0x7f4cf8011b50 comp rx=0 tx=0).stop 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.548+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 shutdown_connections 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.548+0000 7f4d0ce00700 1 --2- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7f4cec0384e0 0x7f4cec03a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.548+0000 7f4d0ce00700 1 --2- 172.21.15.115:0/2857713352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d08104700 0x7f4d0818f8e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.548+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 >> 172.21.15.115:0/2857713352 conn(0x7f4d080ffd30 msgr2=0x7f4d08074050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:30:52.240 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.549+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 shutdown_connections 2024-03-22T10:30:52.241 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:51.549+0000 7f4d0ce00700 1 -- 172.21.15.115:0/2857713352 wait complete. 2024-03-22T10:30:52.241 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for the mgr to restart... 2024-03-22T10:30:52.241 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr epoch 9... 2024-03-22T10:30:52.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:52 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2857713352' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T10:30:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: Active manager daemon smithi115.iqzwsz restarted 2024-03-22T10:30:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: Activating manager daemon smithi115.iqzwsz 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: osdmap e3: 0 total, 0 up, 0 in 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: mgrmap e10: smithi115.iqzwsz(active, starting, since 0.00876542s) 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: Manager daemon smithi115.iqzwsz is now available 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:30:59.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/trash_purge_schedule"}]: dispatch 2024-03-22T10:31:00.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:59 smithi115 ceph-mon[29111]: [22/Mar/2024:10:30:58] ENGINE Bus STARTING 2024-03-22T10:31:00.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:59 smithi115 ceph-mon[29111]: [22/Mar/2024:10:30:58] ENGINE Serving on http://172.21.15.115:8765 2024-03-22T10:31:00.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:59 smithi115 ceph-mon[29111]: [22/Mar/2024:10:30:59] ENGINE Serving on https://172.21.15.115:7150 2024-03-22T10:31:00.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:59 smithi115 ceph-mon[29111]: [22/Mar/2024:10:30:59] ENGINE Bus STARTED 2024-03-22T10:31:00.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:30:59 smithi115 ceph-mon[29111]: mgrmap e11: smithi115.iqzwsz(active, since 1.01264s) 2024-03-22T10:31:00.362 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-03-22T10:31:00.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-22T10:31:00.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T10:31:00.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-03-22T10:31:00.363 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.887+0000 7fc74ce00700 1 Processor -- start 2024-03-22T10:31:00.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.887+0000 7fc74ce00700 1 -- start start 2024-03-22T10:31:00.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.888+0000 7fc74ce00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc748105c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:00.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.888+0000 7fc74ce00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc748106150 con 0x7fc748105800 2024-03-22T10:31:00.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.888+0000 7fc747400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc748105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:00.364 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.888+0000 7fc747400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc748105c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57348/0 (socket says 172.21.15.115:57348) 2024-03-22T10:31:00.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.888+0000 7fc747400700 1 -- 172.21.15.115:0/3286951076 learned_addr learned my addr 172.21.15.115:0/3286951076 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:00.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.889+0000 7fc747400700 1 -- 172.21.15.115:0/3286951076 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc748106290 con 0x7fc748105800 2024-03-22T10:31:00.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.889+0000 7fc747400700 1 --2- 172.21.15.115:0/3286951076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc748105c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7380098a0 tx=0x7fc738009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6356a65e100984 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:00.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.889+0000 7fc746000700 1 -- 172.21.15.115:0/3286951076 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc738004180 con 0x7fc748105800 2024-03-22T10:31:00.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.889+0000 7fc746000700 1 -- 172.21.15.115:0/3286951076 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc7380042e0 con 0x7fc748105800 2024-03-22T10:31:00.365 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.889+0000 7fc746000700 1 -- 172.21.15.115:0/3286951076 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc738011af0 con 0x7fc748105800 2024-03-22T10:31:00.368 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.890+0000 7fc74ce00700 1 -- 172.21.15.115:0/3286951076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 msgr2=0x7fc748105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:00.368 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.890+0000 7fc74ce00700 1 --2- 172.21.15.115:0/3286951076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc748105c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7380098a0 tx=0x7fc738009bb0 comp rx=0 tx=0).stop 2024-03-22T10:31:00.368 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.890+0000 7fc74ce00700 1 -- 172.21.15.115:0/3286951076 shutdown_connections 2024-03-22T10:31:00.368 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.890+0000 7fc74ce00700 1 --2- 172.21.15.115:0/3286951076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc748105c10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:00.368 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.890+0000 7fc74ce00700 1 -- 172.21.15.115:0/3286951076 >> 172.21.15.115:0/3286951076 conn(0x7fc748100e50 msgr2=0x7fc748103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.890+0000 7fc74ce00700 1 -- 172.21.15.115:0/3286951076 shutdown_connections 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.890+0000 7fc74ce00700 1 -- 172.21.15.115:0/3286951076 wait complete. 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.891+0000 7fc74ce00700 1 Processor -- start 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.891+0000 7fc74ce00700 1 -- start start 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.891+0000 7fc74ce00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc74807ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.891+0000 7fc74ce00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74807b050 con 0x7fc748105800 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.892+0000 7fc747400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc74807ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.892+0000 7fc747400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc74807ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57358/0 (socket says 172.21.15.115:57358) 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.892+0000 7fc747400700 1 -- 172.21.15.115:0/1140429552 learned_addr learned my addr 172.21.15.115:0/1140429552 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.892+0000 7fc747400700 1 -- 172.21.15.115:0/1140429552 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc738009550 con 0x7fc748105800 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.892+0000 7fc747400700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc74807ab10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc738011670 tx=0x7fc738011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.893+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc738003a40 con 0x7fc748105800 2024-03-22T10:31:00.369 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.893+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc738003ba0 con 0x7fc748105800 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.893+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc738024790 con 0x7fc748105800 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.893+0000 7fc74ce00700 1 -- 172.21.15.115:0/1140429552 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc748079160 con 0x7fc748105800 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.893+0000 7fc74ce00700 1 -- 172.21.15.115:0/1140429552 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc748079600 con 0x7fc748105800 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.894+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fc73801a430 con 0x7fc748105800 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.894+0000 7fc73fe00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 0x7fc72c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.894+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 --> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc72c03b000 con 0x7fc72c038440 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.894+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc73804c5b0 con 0x7fc748105800 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.894+0000 7fc746a00700 1 -- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 msgr2=0x7fc72c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/778669970 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:52.894+0000 7fc746a00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 0x7fc72c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:53.095+0000 7fc746a00700 1 -- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 msgr2=0x7fc72c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/778669970 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:53.095+0000 7fc746a00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 0x7fc72c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T10:31:00.370 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:53.496+0000 7fc746a00700 1 -- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 msgr2=0x7fc72c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/778669970 2024-03-22T10:31:00.373 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:53.496+0000 7fc746a00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 0x7fc72c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T10:31:00.373 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:54.297+0000 7fc746a00700 1 -- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 msgr2=0x7fc72c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/778669970 2024-03-22T10:31:00.373 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:54.297+0000 7fc746a00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 0x7fc72c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:55.899+0000 7fc746a00700 1 -- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 msgr2=0x7fc72c03a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/778669970 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:55.899+0000 7fc746a00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 0x7fc72c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:58.618+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mgrmap(e 10) v1 ==== 44864+0+0 (secure 0 0 0) 0x7fc73801a730 con 0x7fc748105800 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:58.618+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 msgr2=0x7fc72c03a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:58.618+0000 7fc73fe00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 0x7fc72c03a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.622+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7fc738018350 con 0x7fc748105800 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.623+0000 7fc73fe00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc72c03bea0 0x7fc72c03e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.623+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 --> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc72c03b000 con 0x7fc72c03bea0 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.624+0000 7fc746a00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc72c03bea0 0x7fc72c03e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.626+0000 7fc746a00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc72c03bea0 0x7fc72c03e280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc734003a10 tx=0x7fc7340092b0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.627+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 <== mgr.14164 v2:172.21.15.115:6800/2729066911 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fc72c03b000 con 0x7fc72c03bea0 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.634+0000 7fc74ce00700 1 -- 172.21.15.115:0/1140429552 --> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc74807a6b0 con 0x7fc72c03bea0 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.634+0000 7fc73fe00700 1 -- 172.21.15.115:0/1140429552 <== mgr.14164 v2:172.21.15.115:6800/2729066911 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc74807a6b0 con 0x7fc72c03bea0 2024-03-22T10:31:00.374 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 -- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc72c03bea0 msgr2=0x7fc72c03e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc72c03bea0 0x7fc72c03e280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc734003a10 tx=0x7fc7340092b0 comp rx=0 tx=0).stop 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 -- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 msgr2=0x7fc74807ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc74807ab10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc738011670 tx=0x7fc738011b50 comp rx=0 tx=0).stop 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 -- 172.21.15.115:0/1140429552 shutdown_connections 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc72c03bea0 0x7fc72c03e280 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748105800 0x7fc74807ab10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 --2- 172.21.15.115:0/1140429552 >> [v2:172.21.15.115:6800/778669970,v1:172.21.15.115:6801/778669970] conn(0x7fc72c038440 0x7fc72c03a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 -- 172.21.15.115:0/1140429552 >> 172.21.15.115:0/1140429552 conn(0x7fc748100e50 msgr2=0x7fc7481024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 -- 172.21.15.115:0/1140429552 shutdown_connections 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:30:59.635+0000 7fc74ce00700 1 -- 172.21.15.115:0/1140429552 wait complete. 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:mgr epoch 9 is available 2024-03-22T10:31:00.375 INFO:teuthology.orchestra.run.smithi115.stdout:Generating a dashboard self-signed certificate... 2024-03-22T10:31:01.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:00 smithi115 ceph-mon[29111]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-22T10:31:01.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:00 smithi115 ceph-mon[29111]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-22T10:31:02.187 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:01 smithi115 ceph-mon[29111]: mgrmap e12: smithi115.iqzwsz(active, since 2s) 2024-03-22T10:31:02.187 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:01 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:02.187 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:01 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:02.188 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:01 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:02.497 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.302+0000 7f6b7a400700 1 Processor -- start 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.302+0000 7f6b7a400700 1 -- start start 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.303+0000 7f6b7a400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b74107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.303+0000 7f6b7a400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b74108530 con 0x7f6b74107be0 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.303+0000 7f6b73400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b74107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.303+0000 7f6b73400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b74107ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57418/0 (socket says 172.21.15.115:57418) 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.303+0000 7f6b73400700 1 -- 172.21.15.115:0/2793142607 learned_addr learned my addr 172.21.15.115:0/2793142607 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.304+0000 7f6b73400700 1 -- 172.21.15.115:0/2793142607 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b74108670 con 0x7f6b74107be0 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.304+0000 7f6b73400700 1 --2- 172.21.15.115:0/2793142607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b74107ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b640098a0 tx=0x7f6b64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c773223591d59ac server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.304+0000 7f6b72000700 1 -- 172.21.15.115:0/2793142607 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b64004180 con 0x7f6b74107be0 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.304+0000 7f6b72000700 1 -- 172.21.15.115:0/2793142607 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b640042e0 con 0x7f6b74107be0 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.304+0000 7f6b72000700 1 -- 172.21.15.115:0/2793142607 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b64011af0 con 0x7f6b74107be0 2024-03-22T10:31:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.305+0000 7f6b7a400700 1 -- 172.21.15.115:0/2793142607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 msgr2=0x7f6b74107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.305+0000 7f6b7a400700 1 --2- 172.21.15.115:0/2793142607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b74107ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b640098a0 tx=0x7f6b64009bb0 comp rx=0 tx=0).stop 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.305+0000 7f6b7a400700 1 -- 172.21.15.115:0/2793142607 shutdown_connections 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.305+0000 7f6b7a400700 1 --2- 172.21.15.115:0/2793142607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b74107ff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.305+0000 7f6b7a400700 1 -- 172.21.15.115:0/2793142607 >> 172.21.15.115:0/2793142607 conn(0x7f6b7407b200 msgr2=0x7f6b7407b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.305+0000 7f6b7a400700 1 -- 172.21.15.115:0/2793142607 shutdown_connections 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.305+0000 7f6b7a400700 1 -- 172.21.15.115:0/2793142607 wait complete. 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.307+0000 7f6b7a400700 1 Processor -- start 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.307+0000 7f6b7a400700 1 -- start start 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.307+0000 7f6b7a400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b7419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.307+0000 7f6b7a400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7419bef0 con 0x7f6b74107be0 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.307+0000 7f6b73400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b7419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.307+0000 7f6b73400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b7419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57432/0 (socket says 172.21.15.115:57432) 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.307+0000 7f6b73400700 1 -- 172.21.15.115:0/353970245 learned_addr learned my addr 172.21.15.115:0/353970245 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:02.499 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.308+0000 7f6b73400700 1 -- 172.21.15.115:0/353970245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b64009550 con 0x7f6b74107be0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.308+0000 7f6b73400700 1 --2- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b7419b9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b64011670 tx=0x7f6b64011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.308+0000 7f6b6be00700 1 -- 172.21.15.115:0/353970245 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b64003a40 con 0x7f6b74107be0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.308+0000 7f6b6be00700 1 -- 172.21.15.115:0/353970245 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b64003ba0 con 0x7f6b74107be0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.308+0000 7f6b7a400700 1 -- 172.21.15.115:0/353970245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7419c0f0 con 0x7f6b74107be0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.308+0000 7f6b6be00700 1 -- 172.21.15.115:0/353970245 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b64024790 con 0x7f6b74107be0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.309+0000 7f6b7a400700 1 -- 172.21.15.115:0/353970245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7419c590 con 0x7f6b74107be0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.310+0000 7f6b6be00700 1 -- 172.21.15.115:0/353970245 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f6b6402d3f0 con 0x7f6b74107be0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.310+0000 7f6b69600700 1 -- 172.21.15.115:0/353970245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b74062380 con 0x7f6b74107be0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.310+0000 7f6b6be00700 1 --2- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f6b58038440 0x7f6b5803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.310+0000 7f6b6be00700 1 -- 172.21.15.115:0/353970245 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b6404c750 con 0x7f6b74107be0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.310+0000 7f6b72a00700 1 --2- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f6b58038440 0x7f6b5803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.311+0000 7f6b72a00700 1 --2- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f6b58038440 0x7f6b5803a8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b60006fd0 tx=0x7f6b60006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:02.500 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.316+0000 7f6b6be00700 1 -- 172.21.15.115:0/353970245 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6b64010bd0 con 0x7f6b74107be0 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.539+0000 7f6b69600700 1 -- 172.21.15.115:0/353970245 --> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b74109a90 con 0x7f6b58038440 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.795+0000 7f6b6be00700 1 -- 172.21.15.115:0/353970245 <== mgr.14164 v2:172.21.15.115:6800/2729066911 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f6b74109a90 con 0x7f6b58038440 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 -- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f6b58038440 msgr2=0x7f6b5803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 --2- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f6b58038440 0x7f6b5803a8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b60006fd0 tx=0x7f6b60006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 -- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 msgr2=0x7f6b7419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 --2- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b7419b9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b64011670 tx=0x7f6b64011b50 comp rx=0 tx=0).stop 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 -- 172.21.15.115:0/353970245 shutdown_connections 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 --2- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f6b58038440 0x7f6b5803a8f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 --2- 172.21.15.115:0/353970245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b74107be0 0x7f6b7419b9b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 -- 172.21.15.115:0/353970245 >> 172.21.15.115:0/353970245 conn(0x7f6b7407b200 msgr2=0x7f6b74106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 -- 172.21.15.115:0/353970245 shutdown_connections 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:01.798+0000 7f6b69600700 1 -- 172.21.15.115:0/353970245 wait complete. 2024-03-22T10:31:02.501 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial admin user... 2024-03-22T10:31:03.085 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:02 smithi115 ceph-mon[29111]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:31:04.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:03 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:04.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:03 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:04.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Da01QCPyWVXFbya.5Ie26ujZQz4jFFBuz5bEJAK8eSh1UC4u43jJG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711103463, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-22T10:31:04.953 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.158+0000 7ff815c00700 1 Processor -- start 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.158+0000 7ff815c00700 1 -- start start 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.159+0000 7ff815c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff810105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.159+0000 7ff815c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff810106310 con 0x7ff8101059c0 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.159+0000 7ff80ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff810105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.159+0000 7ff80ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff810105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48658/0 (socket says 172.21.15.115:48658) 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.159+0000 7ff80ea00700 1 -- 172.21.15.115:0/3036056137 learned_addr learned my addr 172.21.15.115:0/3036056137 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.160+0000 7ff80ea00700 1 -- 172.21.15.115:0/3036056137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810106450 con 0x7ff8101059c0 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.160+0000 7ff80ea00700 1 --2- 172.21.15.115:0/3036056137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff810105dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff8000098a0 tx=0x7ff800009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564e96c329defe24 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.160+0000 7ff80d600700 1 -- 172.21.15.115:0/3036056137 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff800004180 con 0x7ff8101059c0 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.161+0000 7ff80d600700 1 -- 172.21.15.115:0/3036056137 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff8000042e0 con 0x7ff8101059c0 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.161+0000 7ff80d600700 1 -- 172.21.15.115:0/3036056137 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff800011af0 con 0x7ff8101059c0 2024-03-22T10:31:04.954 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.161+0000 7ff815c00700 1 -- 172.21.15.115:0/3036056137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 msgr2=0x7ff810105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.161+0000 7ff815c00700 1 --2- 172.21.15.115:0/3036056137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff810105dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff8000098a0 tx=0x7ff800009bb0 comp rx=0 tx=0).stop 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.161+0000 7ff815c00700 1 -- 172.21.15.115:0/3036056137 shutdown_connections 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.161+0000 7ff815c00700 1 --2- 172.21.15.115:0/3036056137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff810105dd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.161+0000 7ff815c00700 1 -- 172.21.15.115:0/3036056137 >> 172.21.15.115:0/3036056137 conn(0x7ff810100fd0 msgr2=0x7ff810103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.162+0000 7ff815c00700 1 -- 172.21.15.115:0/3036056137 shutdown_connections 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.162+0000 7ff815c00700 1 -- 172.21.15.115:0/3036056137 wait complete. 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.163+0000 7ff815c00700 1 Processor -- start 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.163+0000 7ff815c00700 1 -- start start 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.163+0000 7ff815c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff81019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.163+0000 7ff815c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81019beb0 con 0x7ff8101059c0 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.164+0000 7ff80ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff81019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.164+0000 7ff80ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff81019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48672/0 (socket says 172.21.15.115:48672) 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.164+0000 7ff80ea00700 1 -- 172.21.15.115:0/667557506 learned_addr learned my addr 172.21.15.115:0/667557506 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.164+0000 7ff80ea00700 1 -- 172.21.15.115:0/667557506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800009550 con 0x7ff8101059c0 2024-03-22T10:31:04.955 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.164+0000 7ff80ea00700 1 --2- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff81019b970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff800011670 tx=0x7ff800011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.165+0000 7ff807400700 1 -- 172.21.15.115:0/667557506 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff800003a40 con 0x7ff8101059c0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.165+0000 7ff807400700 1 -- 172.21.15.115:0/667557506 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff800003ba0 con 0x7ff8101059c0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.165+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81019c0b0 con 0x7ff8101059c0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.165+0000 7ff807400700 1 -- 172.21.15.115:0/667557506 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff800024790 con 0x7ff8101059c0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.165+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81019c550 con 0x7ff8101059c0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.167+0000 7ff807400700 1 -- 172.21.15.115:0/667557506 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7ff80001a430 con 0x7ff8101059c0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.167+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff810062380 con 0x7ff8101059c0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.167+0000 7ff807400700 1 --2- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7ff7f4038440 0x7ff7f403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.167+0000 7ff807400700 1 -- 172.21.15.115:0/667557506 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff80004c730 con 0x7ff8101059c0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.167+0000 7ff80e000700 1 --2- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7ff7f4038440 0x7ff7f403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.168+0000 7ff80e000700 1 --2- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7ff7f4038440 0x7ff7f403a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc006fd0 tx=0x7ff7fc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.178+0000 7ff807400700 1 -- 172.21.15.115:0/667557506 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff800010380 con 0x7ff8101059c0 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.412+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 --> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff8100621a0 con 0x7ff7f4038440 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.652+0000 7ff807400700 1 -- 172.21.15.115:0/667557506 <== mgr.14164 v2:172.21.15.115:6800/2729066911 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7ff8100621a0 con 0x7ff7f4038440 2024-03-22T10:31:04.956 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.656+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7ff7f4038440 msgr2=0x7ff7f403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:04.957 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.656+0000 7ff815c00700 1 --2- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7ff7f4038440 0x7ff7f403a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc006fd0 tx=0x7ff7fc006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:04.957 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.656+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 msgr2=0x7ff81019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.656+0000 7ff815c00700 1 --2- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff81019b970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff800011670 tx=0x7ff800011b50 comp rx=0 tx=0).stop 2024-03-22T10:31:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.656+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 shutdown_connections 2024-03-22T10:31:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.656+0000 7ff815c00700 1 --2- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7ff7f4038440 0x7ff7f403a8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.656+0000 7ff815c00700 1 --2- 172.21.15.115:0/667557506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8101059c0 0x7ff81019b970 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.656+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 >> 172.21.15.115:0/667557506 conn(0x7ff810100fd0 msgr2=0x7ff8101026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.656+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 shutdown_connections 2024-03-22T10:31:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:03.657+0000 7ff815c00700 1 -- 172.21.15.115:0/667557506 wait complete. 2024-03-22T10:31:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:Fetching dashboard port number... 2024-03-22T10:31:05.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:04 smithi115 ceph-mon[29111]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:31:06.163 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:05 smithi115 ceph-mon[29111]: mgrmap e13: smithi115.iqzwsz(active, since 6s) 2024-03-22T10:31:06.163 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:05 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/951541866' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 8443 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.641+0000 7fe3e9400700 1 Processor -- start 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.641+0000 7fe3e9400700 1 -- start start 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.641+0000 7fe3e9400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e4080210 0x7fe3e410ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.641+0000 7fe3e9400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e410f1f0 con 0x7fe3e4080210 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.642+0000 7fe3e2800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e4080210 0x7fe3e410ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.642+0000 7fe3e2800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e4080210 0x7fe3e410ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48682/0 (socket says 172.21.15.115:48682) 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.642+0000 7fe3e2800700 1 -- 172.21.15.115:0/957886195 learned_addr learned my addr 172.21.15.115:0/957886195 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.642+0000 7fe3e2800700 1 -- 172.21.15.115:0/957886195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e410f330 con 0x7fe3e4080210 2024-03-22T10:31:06.469 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.643+0000 7fe3e2800700 1 --2- 172.21.15.115:0/957886195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e4080210 0x7fe3e410ecb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40098a0 tx=0x7fe3d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227bfb9069920e4b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:06.470 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.643+0000 7fe3e1400700 1 -- 172.21.15.115:0/957886195 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3d4004180 con 0x7fe3e4080210 2024-03-22T10:31:06.470 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.643+0000 7fe3e1400700 1 -- 172.21.15.115:0/957886195 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe3d40042e0 con 0x7fe3e4080210 2024-03-22T10:31:06.470 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.643+0000 7fe3e1400700 1 -- 172.21.15.115:0/957886195 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3d4011af0 con 0x7fe3e4080210 2024-03-22T10:31:06.470 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.644+0000 7fe3e9400700 1 -- 172.21.15.115:0/957886195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e4080210 msgr2=0x7fe3e410ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:06.470 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.644+0000 7fe3e9400700 1 --2- 172.21.15.115:0/957886195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e4080210 0x7fe3e410ecb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40098a0 tx=0x7fe3d4009bb0 comp rx=0 tx=0).stop 2024-03-22T10:31:06.470 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.644+0000 7fe3e9400700 1 -- 172.21.15.115:0/957886195 shutdown_connections 2024-03-22T10:31:06.470 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.644+0000 7fe3e9400700 1 --2- 172.21.15.115:0/957886195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e4080210 0x7fe3e410ecb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:06.470 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.644+0000 7fe3e9400700 1 -- 172.21.15.115:0/957886195 >> 172.21.15.115:0/957886195 conn(0x7fe3e407b200 msgr2=0x7fe3e407d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:06.470 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.644+0000 7fe3e9400700 1 -- 172.21.15.115:0/957886195 shutdown_connections 2024-03-22T10:31:06.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.644+0000 7fe3e9400700 1 -- 172.21.15.115:0/957886195 wait complete. 2024-03-22T10:31:06.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.645+0000 7fe3e9400700 1 Processor -- start 2024-03-22T10:31:06.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.645+0000 7fe3e9400700 1 -- start start 2024-03-22T10:31:06.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.646+0000 7fe3e9400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e419ffc0 0x7fe3e41a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:06.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.646+0000 7fe3e9400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41a0910 con 0x7fe3e419ffc0 2024-03-22T10:31:06.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.646+0000 7fe3e2800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e419ffc0 0x7fe3e41a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:06.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.646+0000 7fe3e2800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e419ffc0 0x7fe3e41a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48686/0 (socket says 172.21.15.115:48686) 2024-03-22T10:31:06.471 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.646+0000 7fe3e2800700 1 -- 172.21.15.115:0/951541866 learned_addr learned my addr 172.21.15.115:0/951541866 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.646+0000 7fe3e2800700 1 -- 172.21.15.115:0/951541866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d4009550 con 0x7fe3e419ffc0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.647+0000 7fe3e2800700 1 --2- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e419ffc0 0x7fe3e41a03d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe3e407fee0 tx=0x7fe3d4011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.647+0000 7fe3db400700 1 -- 172.21.15.115:0/951541866 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3d4011ba0 con 0x7fe3e419ffc0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.647+0000 7fe3db400700 1 -- 172.21.15.115:0/951541866 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe3d401f070 con 0x7fe3e419ffc0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.647+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e41a0b10 con 0x7fe3e419ffc0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.647+0000 7fe3db400700 1 -- 172.21.15.115:0/951541866 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3d401ad90 con 0x7fe3e419ffc0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.647+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e41a3770 con 0x7fe3e419ffc0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.648+0000 7fe3db400700 1 -- 172.21.15.115:0/951541866 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fe3d4021030 con 0x7fe3e419ffc0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.648+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e4199de0 con 0x7fe3e419ffc0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.648+0000 7fe3db400700 1 --2- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fe3c8038530 0x7fe3c803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.648+0000 7fe3db400700 1 -- 172.21.15.115:0/951541866 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe3d404c380 con 0x7fe3e419ffc0 2024-03-22T10:31:06.472 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.649+0000 7fe3e1e00700 1 --2- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fe3c8038530 0x7fe3c803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.649+0000 7fe3e1e00700 1 --2- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fe3c8038530 0x7fe3c803a9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0006fd0 tx=0x7fe3d0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.654+0000 7fe3db400700 1 -- 172.21.15.115:0/951541866 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe3d400b9d0 con 0x7fe3e419ffc0 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.871+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fe3e402cc30 con 0x7fe3e419ffc0 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.871+0000 7fe3db400700 1 -- 172.21.15.115:0/951541866 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fe3d400b3e0 con 0x7fe3e419ffc0 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.874+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fe3c8038530 msgr2=0x7fe3c803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.874+0000 7fe3e9400700 1 --2- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fe3c8038530 0x7fe3c803a9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0006fd0 tx=0x7fe3d0006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.874+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e419ffc0 msgr2=0x7fe3e41a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.874+0000 7fe3e9400700 1 --2- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e419ffc0 0x7fe3e41a03d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe3e407fee0 tx=0x7fe3d4011870 comp rx=0 tx=0).stop 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.874+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 shutdown_connections 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.874+0000 7fe3e9400700 1 --2- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fe3c8038530 0x7fe3c803a9e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:06.473 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.874+0000 7fe3e9400700 1 --2- 172.21.15.115:0/951541866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3e419ffc0 0x7fe3e41a03d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:06.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.874+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 >> 172.21.15.115:0/951541866 conn(0x7fe3e407b200 msgr2=0x7fe3e407da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:06.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.875+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 shutdown_connections 2024-03-22T10:31:06.474 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:05.875+0000 7fe3e9400700 1 -- 172.21.15.115:0/951541866 wait complete. 2024-03-22T10:31:06.507 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T10:31:06.507 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-03-22T10:31:06.532 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T10:31:06.532 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-03-22T10:31:06.532 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-03-22T10:31:06.532 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-22T10:31:06.540 INFO:teuthology.orchestra.run.smithi115.stdout:Ceph Dashboard is now available at: 2024-03-22T10:31:06.540 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:06.540 INFO:teuthology.orchestra.run.smithi115.stdout: URL: https://smithi115.front.sepia.ceph.com:8443/ 2024-03-22T10:31:06.540 INFO:teuthology.orchestra.run.smithi115.stdout: User: admin 2024-03-22T10:31:06.540 INFO:teuthology.orchestra.run.smithi115.stdout: Password: xmkwofr0j3 2024-03-22T10:31:06.540 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:06.541 INFO:teuthology.orchestra.run.smithi115.stdout:Saving cluster configuration to /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config directory 2024-03-22T10:31:06.543 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling autotune for osd_memory_target 2024-03-22T10:31:08.070 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.141+0000 7fd861800700 1 Processor -- start 2024-03-22T10:31:08.070 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.142+0000 7fd861800700 1 -- start start 2024-03-22T10:31:08.070 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.142+0000 7fd861800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:08.070 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.142+0000 7fd861800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c079740 con 0x7fd85c07ad00 2024-03-22T10:31:08.070 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.142+0000 7fd85aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:08.070 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.143+0000 7fd85aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48690/0 (socket says 172.21.15.115:48690) 2024-03-22T10:31:08.070 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.143+0000 7fd85aa00700 1 -- 172.21.15.115:0/282290412 learned_addr learned my addr 172.21.15.115:0/282290412 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.143+0000 7fd85aa00700 1 -- 172.21.15.115:0/282290412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c079880 con 0x7fd85c07ad00 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.143+0000 7fd85aa00700 1 --2- 172.21.15.115:0/282290412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c079200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0098a0 tx=0x7fd84c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6874e5f00375fac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.144+0000 7fd859600700 1 -- 172.21.15.115:0/282290412 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd84c004180 con 0x7fd85c07ad00 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.144+0000 7fd859600700 1 -- 172.21.15.115:0/282290412 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd84c0042e0 con 0x7fd85c07ad00 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.144+0000 7fd859600700 1 -- 172.21.15.115:0/282290412 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd84c011af0 con 0x7fd85c07ad00 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.144+0000 7fd861800700 1 -- 172.21.15.115:0/282290412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 msgr2=0x7fd85c079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.144+0000 7fd861800700 1 --2- 172.21.15.115:0/282290412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c079200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0098a0 tx=0x7fd84c009bb0 comp rx=0 tx=0).stop 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.144+0000 7fd861800700 1 -- 172.21.15.115:0/282290412 shutdown_connections 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.144+0000 7fd861800700 1 --2- 172.21.15.115:0/282290412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c079200 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.144+0000 7fd861800700 1 -- 172.21.15.115:0/282290412 >> 172.21.15.115:0/282290412 conn(0x7fd85c100fd0 msgr2=0x7fd85c103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.145+0000 7fd861800700 1 -- 172.21.15.115:0/282290412 shutdown_connections 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.145+0000 7fd861800700 1 -- 172.21.15.115:0/282290412 wait complete. 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.146+0000 7fd861800700 1 Processor -- start 2024-03-22T10:31:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.146+0000 7fd861800700 1 -- start start 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.146+0000 7fd861800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.146+0000 7fd861800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c19be80 con 0x7fd85c07ad00 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.147+0000 7fd85aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.147+0000 7fd85aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c19b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48696/0 (socket says 172.21.15.115:48696) 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.147+0000 7fd85aa00700 1 -- 172.21.15.115:0/4056509620 learned_addr learned my addr 172.21.15.115:0/4056509620 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.147+0000 7fd85aa00700 1 -- 172.21.15.115:0/4056509620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c009550 con 0x7fd85c07ad00 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.147+0000 7fd85aa00700 1 --2- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c19b940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd84c011670 tx=0x7fd84c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.147+0000 7fd853400700 1 -- 172.21.15.115:0/4056509620 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd84c0039f0 con 0x7fd85c07ad00 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.147+0000 7fd853400700 1 -- 172.21.15.115:0/4056509620 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd84c003b50 con 0x7fd85c07ad00 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.148+0000 7fd853400700 1 -- 172.21.15.115:0/4056509620 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd84c0247c0 con 0x7fd85c07ad00 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.148+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c19c080 con 0x7fd85c07ad00 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.148+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c19c520 con 0x7fd85c07ad00 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.149+0000 7fd853400700 1 -- 172.21.15.115:0/4056509620 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fd84c01ac00 con 0x7fd85c07ad00 2024-03-22T10:31:08.072 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.149+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c10bcb0 con 0x7fd85c07ad00 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.149+0000 7fd853400700 1 --2- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fd840038530 0x7fd84003a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.149+0000 7fd853400700 1 -- 172.21.15.115:0/4056509620 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd84c04c670 con 0x7fd85c07ad00 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.150+0000 7fd85a000700 1 --2- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fd840038530 0x7fd84003a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.150+0000 7fd85a000700 1 --2- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fd840038530 0x7fd84003a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd848006fd0 tx=0x7fd848006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.154+0000 7fd853400700 1 -- 172.21.15.115:0/4056509620 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd84c010380 con 0x7fd85c07ad00 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.368+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fd85c19f360 con 0x7fd85c07ad00 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.369+0000 7fd853400700 1 -- 172.21.15.115:0/4056509620 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fd84c018b40 con 0x7fd85c07ad00 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.372+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fd840038530 msgr2=0x7fd84003a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.372+0000 7fd861800700 1 --2- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fd840038530 0x7fd84003a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd848006fd0 tx=0x7fd848006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.372+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 msgr2=0x7fd85c19b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.372+0000 7fd861800700 1 --2- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c19b940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd84c011670 tx=0x7fd84c011b50 comp rx=0 tx=0).stop 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.372+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 shutdown_connections 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.372+0000 7fd861800700 1 --2- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fd840038530 0x7fd84003a9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:08.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.372+0000 7fd861800700 1 --2- 172.21.15.115:0/4056509620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c07ad00 0x7fd85c19b940 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:08.074 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.372+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 >> 172.21.15.115:0/4056509620 conn(0x7fd85c100fd0 msgr2=0x7fd85c102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:08.074 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.373+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 shutdown_connections 2024-03-22T10:31:08.074 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:07.373+0000 7fd861800700 1 -- 172.21.15.115:0/4056509620 wait complete. 2024-03-22T10:31:09.674 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.628+0000 7fed73200700 1 Processor -- start 2024-03-22T10:31:09.674 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.628+0000 7fed73200700 1 -- start start 2024-03-22T10:31:09.674 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.628+0000 7fed73200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed74107e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:09.674 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.628+0000 7fed73200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed74108340 con 0x7fed741079f0 2024-03-22T10:31:09.674 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.629+0000 7fed71e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed74107e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:09.674 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.629+0000 7fed71e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed74107e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48706/0 (socket says 172.21.15.115:48706) 2024-03-22T10:31:09.674 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.629+0000 7fed71e00700 1 -- 172.21.15.115:0/2272259504 learned_addr learned my addr 172.21.15.115:0/2272259504 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:09.674 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.629+0000 7fed71e00700 1 -- 172.21.15.115:0/2272259504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed74108480 con 0x7fed741079f0 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.630+0000 7fed71e00700 1 --2- 172.21.15.115:0/2272259504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed74107e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed640098a0 tx=0x7fed64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784f494bd96ed79f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.630+0000 7fed70a00700 1 -- 172.21.15.115:0/2272259504 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed64004180 con 0x7fed741079f0 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.630+0000 7fed70a00700 1 -- 172.21.15.115:0/2272259504 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fed640042e0 con 0x7fed741079f0 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.630+0000 7fed70a00700 1 -- 172.21.15.115:0/2272259504 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed64011af0 con 0x7fed741079f0 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.631+0000 7fed73200700 1 -- 172.21.15.115:0/2272259504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 msgr2=0x7fed74107e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.631+0000 7fed73200700 1 --2- 172.21.15.115:0/2272259504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed74107e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed640098a0 tx=0x7fed64009bb0 comp rx=0 tx=0).stop 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.631+0000 7fed73200700 1 -- 172.21.15.115:0/2272259504 shutdown_connections 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.631+0000 7fed73200700 1 --2- 172.21.15.115:0/2272259504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed74107e00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.631+0000 7fed73200700 1 -- 172.21.15.115:0/2272259504 >> 172.21.15.115:0/2272259504 conn(0x7fed7407ae80 msgr2=0x7fed7407b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.631+0000 7fed73200700 1 -- 172.21.15.115:0/2272259504 shutdown_connections 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.631+0000 7fed73200700 1 -- 172.21.15.115:0/2272259504 wait complete. 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.632+0000 7fed73200700 1 Processor -- start 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.632+0000 7fed73200700 1 -- start start 2024-03-22T10:31:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.633+0000 7fed73200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed7419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.633+0000 7fed73200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7419bcb0 con 0x7fed741079f0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.633+0000 7fed71e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed7419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.633+0000 7fed71e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed7419b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48714/0 (socket says 172.21.15.115:48714) 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.633+0000 7fed71e00700 1 -- 172.21.15.115:0/2832504848 learned_addr learned my addr 172.21.15.115:0/2832504848 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.633+0000 7fed71e00700 1 -- 172.21.15.115:0/2832504848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64009550 con 0x7fed741079f0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.634+0000 7fed71e00700 1 --2- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed7419b770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed64011670 tx=0x7fed64011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.634+0000 7fed6aa00700 1 -- 172.21.15.115:0/2832504848 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed64003a40 con 0x7fed741079f0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.634+0000 7fed6aa00700 1 -- 172.21.15.115:0/2832504848 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fed64003ba0 con 0x7fed741079f0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.634+0000 7fed6aa00700 1 -- 172.21.15.115:0/2832504848 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed64024790 con 0x7fed741079f0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.634+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7419beb0 con 0x7fed741079f0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.634+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7419c350 con 0x7fed741079f0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.635+0000 7fed6aa00700 1 -- 172.21.15.115:0/2832504848 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fed6401a430 con 0x7fed741079f0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.635+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed74062380 con 0x7fed741079f0 2024-03-22T10:31:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.636+0000 7fed6aa00700 1 --2- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fed580384e0 0x7fed5803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.636+0000 7fed6aa00700 1 -- 172.21.15.115:0/2832504848 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fed6404c820 con 0x7fed741079f0 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.636+0000 7fed71400700 1 --2- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fed580384e0 0x7fed5803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.637+0000 7fed71400700 1 --2- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fed580384e0 0x7fed5803a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed60006fd0 tx=0x7fed60006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.641+0000 7fed6aa00700 1 -- 172.21.15.115:0/2832504848 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fed6401ac50 con 0x7fed741079f0 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.965+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fed7419f250 con 0x7fed741079f0 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.970+0000 7fed6aa00700 1 -- 172.21.15.115:0/2832504848 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7fed64018b00 con 0x7fed741079f0 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.973+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fed580384e0 msgr2=0x7fed5803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.973+0000 7fed73200700 1 --2- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fed580384e0 0x7fed5803a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed60006fd0 tx=0x7fed60006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.973+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 msgr2=0x7fed7419b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.973+0000 7fed73200700 1 --2- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed7419b770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed64011670 tx=0x7fed64011b50 comp rx=0 tx=0).stop 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.973+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 shutdown_connections 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.973+0000 7fed73200700 1 --2- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fed580384e0 0x7fed5803a990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.973+0000 7fed73200700 1 --2- 172.21.15.115:0/2832504848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed741079f0 0x7fed7419b770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.973+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 >> 172.21.15.115:0/2832504848 conn(0x7fed7407ae80 msgr2=0x7fed741057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.974+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 shutdown_connections 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-03-22T10:31:08.974+0000 7fed73200700 1 -- 172.21.15.115:0/2832504848 wait complete. 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout:Or, if you are only running a single cluster on this host: 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:09.678 INFO:teuthology.orchestra.run.smithi115.stdout: ceph telemetry on 2024-03-22T10:31:09.679 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:09.679 INFO:teuthology.orchestra.run.smithi115.stdout:For more information see: 2024-03-22T10:31:09.679 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:09.679 INFO:teuthology.orchestra.run.smithi115.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-22T10:31:09.679 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:09.679 INFO:teuthology.orchestra.run.smithi115.stdout:Bootstrap complete. 2024-03-22T10:31:09.762 INFO:tasks.cephadm:Fetching config... 2024-03-22T10:31:09.762 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:31:09.762 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-22T10:31:09.807 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-22T10:31:09.807 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:31:09.807 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-22T10:31:09.890 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-22T10:31:09.890 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:31:09.890 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/keyring of=/dev/stdout 2024-03-22T10:31:10.020 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-22T10:31:10.021 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:31:10.021 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-22T10:31:10.075 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-22T10:31:10.076 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC++tz/NJA/QWGAKJCrF4ksITVh9LmBsF3Q1pasowSn2lUfT9fp0nfrKLCM/mIKDQ8LLoZApCP+8U5J6rjihA2U55YMjSmtGXp4DzXqtvuCpTljrdSywAkGXAKmxVUxvp3O7/fNgS/elQgXTHRT7nlEMGTXlRd265Hk/QTvgUmtPa/LMVGGTijvGRzAsprwbRDcSoYlikjJN203RgnFyAi0YfmXEr3OHHhdxAuBui74xUGJXZZU2KuMaewqJd8w2cF814KvzDgndLr/7ko0PYjuxXBwlEyZtIfXuVJWgzJEdh03RqKREWOVJoFDdxmQ2/PNN3Qigjhj8+7dTenfEWoxfgdaNZNAE6nStecdcfSwbChvLzvXGgixObkGhTvE+N/5nGMfeKY6TdOCmS9+uMscDbij/AYzgzbSn+wNatXIotJKb6S12PjZrAouYhIs7nXe8SzCU02skFfTwcoM0eqKHWcp3s6m+NbaisSNfbdCCp0meu5q3CHhEY3w99sb1Ps= ceph-fd1caff4-e836-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T10:31:10.187 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:09 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2832504848' entity='client.admin' 2024-03-22T10:31:10.195 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC++tz/NJA/QWGAKJCrF4ksITVh9LmBsF3Q1pasowSn2lUfT9fp0nfrKLCM/mIKDQ8LLoZApCP+8U5J6rjihA2U55YMjSmtGXp4DzXqtvuCpTljrdSywAkGXAKmxVUxvp3O7/fNgS/elQgXTHRT7nlEMGTXlRd265Hk/QTvgUmtPa/LMVGGTijvGRzAsprwbRDcSoYlikjJN203RgnFyAi0YfmXEr3OHHhdxAuBui74xUGJXZZU2KuMaewqJd8w2cF814KvzDgndLr/7ko0PYjuxXBwlEyZtIfXuVJWgzJEdh03RqKREWOVJoFDdxmQ2/PNN3Qigjhj8+7dTenfEWoxfgdaNZNAE6nStecdcfSwbChvLzvXGgixObkGhTvE+N/5nGMfeKY6TdOCmS9+uMscDbij/AYzgzbSn+wNatXIotJKb6S12PjZrAouYhIs7nXe8SzCU02skFfTwcoM0eqKHWcp3s6m+NbaisSNfbdCCp0meu5q3CHhEY3w99sb1Ps= ceph-fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:31:10.255 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC++tz/NJA/QWGAKJCrF4ksITVh9LmBsF3Q1pasowSn2lUfT9fp0nfrKLCM/mIKDQ8LLoZApCP+8U5J6rjihA2U55YMjSmtGXp4DzXqtvuCpTljrdSywAkGXAKmxVUxvp3O7/fNgS/elQgXTHRT7nlEMGTXlRd265Hk/QTvgUmtPa/LMVGGTijvGRzAsprwbRDcSoYlikjJN203RgnFyAi0YfmXEr3OHHhdxAuBui74xUGJXZZU2KuMaewqJd8w2cF814KvzDgndLr/7ko0PYjuxXBwlEyZtIfXuVJWgzJEdh03RqKREWOVJoFDdxmQ2/PNN3Qigjhj8+7dTenfEWoxfgdaNZNAE6nStecdcfSwbChvLzvXGgixObkGhTvE+N/5nGMfeKY6TdOCmS9+uMscDbij/AYzgzbSn+wNatXIotJKb6S12PjZrAouYhIs7nXe8SzCU02skFfTwcoM0eqKHWcp3s6m+NbaisSNfbdCCp0meu5q3CHhEY3w99sb1Ps= ceph-fd1caff4-e836-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T10:31:10.391 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC++tz/NJA/QWGAKJCrF4ksITVh9LmBsF3Q1pasowSn2lUfT9fp0nfrKLCM/mIKDQ8LLoZApCP+8U5J6rjihA2U55YMjSmtGXp4DzXqtvuCpTljrdSywAkGXAKmxVUxvp3O7/fNgS/elQgXTHRT7nlEMGTXlRd265Hk/QTvgUmtPa/LMVGGTijvGRzAsprwbRDcSoYlikjJN203RgnFyAi0YfmXEr3OHHhdxAuBui74xUGJXZZU2KuMaewqJd8w2cF814KvzDgndLr/7ko0PYjuxXBwlEyZtIfXuVJWgzJEdh03RqKREWOVJoFDdxmQ2/PNN3Qigjhj8+7dTenfEWoxfgdaNZNAE6nStecdcfSwbChvLzvXGgixObkGhTvE+N/5nGMfeKY6TdOCmS9+uMscDbij/AYzgzbSn+wNatXIotJKb6S12PjZrAouYhIs7nXe8SzCU02skFfTwcoM0eqKHWcp3s6m+NbaisSNfbdCCp0meu5q3CHhEY3w99sb1Ps= ceph-fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:31:10.451 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-22T10:31:10.826 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:31:13.389 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:13 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:13.389 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:13 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:13.389 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:13 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:31:13.389 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:13 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:13.389 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:13 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:31:13.390 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:13 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-22T10:31:13.390 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:13 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:31:13.390 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:13 smithi115 ceph-mon[29111]: Deploying daemon ceph-exporter.smithi115 on smithi115 2024-03-22T10:31:13.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.474+0000 7f5cc7400700 1 -- 172.21.15.115:0/912604030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 msgr2=0x7f5cc00731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:13.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.474+0000 7f5cc7400700 1 --2- 172.21.15.115:0/912604030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 0x7f5cc00731e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009910 tx=0x7f5cb4009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:13.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.474+0000 7f5cc7400700 1 -- 172.21.15.115:0/912604030 shutdown_connections 2024-03-22T10:31:13.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.474+0000 7f5cc7400700 1 --2- 172.21.15.115:0/912604030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 0x7f5cc00731e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:13.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.474+0000 7f5cc7400700 1 -- 172.21.15.115:0/912604030 >> 172.21.15.115:0/912604030 conn(0x7f5cc00fb3d0 msgr2=0x7f5cc00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:13.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.475+0000 7f5cc7400700 1 -- 172.21.15.115:0/912604030 shutdown_connections 2024-03-22T10:31:13.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.475+0000 7f5cc7400700 1 -- 172.21.15.115:0/912604030 wait complete. 2024-03-22T10:31:13.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.476+0000 7f5cc7400700 1 Processor -- start 2024-03-22T10:31:13.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.476+0000 7f5cc7400700 1 -- start start 2024-03-22T10:31:13.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.476+0000 7f5cc7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 0x7f5cc019b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:13.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.476+0000 7f5cc7400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc019bad0 con 0x7f5cc0074d80 2024-03-22T10:31:13.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.476+0000 7f5cc4a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 0x7f5cc019b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:13.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.476+0000 7f5cc4a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 0x7f5cc019b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39828/0 (socket says 172.21.15.115:39828) 2024-03-22T10:31:13.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.476+0000 7f5cc4a00700 1 -- 172.21.15.115:0/1745855306 learned_addr learned my addr 172.21.15.115:0/1745855306 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:13.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.477+0000 7f5cc4a00700 1 -- 172.21.15.115:0/1745855306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb40095f0 con 0x7f5cc0074d80 2024-03-22T10:31:13.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.477+0000 7f5cc4a00700 1 --2- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 0x7f5cc019b590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40092e0 tx=0x7f5cb4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:13.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.477+0000 7f5cbd600700 1 -- 172.21.15.115:0/1745855306 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cb401d070 con 0x7f5cc0074d80 2024-03-22T10:31:13.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.478+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc019bcd0 con 0x7f5cc0074d80 2024-03-22T10:31:13.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.478+0000 7f5cbd600700 1 -- 172.21.15.115:0/1745855306 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5cb400be10 con 0x7f5cc0074d80 2024-03-22T10:31:13.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.478+0000 7f5cbd600700 1 -- 172.21.15.115:0/1745855306 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cb400f460 con 0x7f5cc0074d80 2024-03-22T10:31:13.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.478+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc019c170 con 0x7f5cc0074d80 2024-03-22T10:31:13.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.479+0000 7f5cbd600700 1 -- 172.21.15.115:0/1745855306 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f5cb400b960 con 0x7f5cc0074d80 2024-03-22T10:31:13.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.479+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc0195120 con 0x7f5cc0074d80 2024-03-22T10:31:13.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.479+0000 7f5cbd600700 1 --2- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5ca8038490 0x7f5ca803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:13.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.479+0000 7f5cbd600700 1 -- 172.21.15.115:0/1745855306 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5cb404c4b0 con 0x7f5cc0074d80 2024-03-22T10:31:13.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.480+0000 7f5cbfe00700 1 --2- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5ca8038490 0x7f5ca803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:13.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.480+0000 7f5cbfe00700 1 --2- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5ca8038490 0x7f5ca803a940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006fd0 tx=0x7f5cac006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:13.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.489+0000 7f5cbd600700 1 -- 172.21.15.115:0/1745855306 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5cb400f6d0 con 0x7f5cc0074d80 2024-03-22T10:31:13.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.707+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f5cc010c420 con 0x7f5cc0074d80 2024-03-22T10:31:13.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.711+0000 7f5cbd600700 1 -- 172.21.15.115:0/1745855306 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5cb4027020 con 0x7f5cc0074d80 2024-03-22T10:31:13.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.714+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5ca8038490 msgr2=0x7f5ca803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:13.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.714+0000 7f5cc7400700 1 --2- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5ca8038490 0x7f5ca803a940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006fd0 tx=0x7f5cac006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:13.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.714+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 msgr2=0x7f5cc019b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:13.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.714+0000 7f5cc7400700 1 --2- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 0x7f5cc019b590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40092e0 tx=0x7f5cb4009e60 comp rx=0 tx=0).stop 2024-03-22T10:31:13.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.714+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 shutdown_connections 2024-03-22T10:31:13.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.714+0000 7f5cc7400700 1 --2- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5ca8038490 0x7f5ca803a940 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:13.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.714+0000 7f5cc7400700 1 --2- 172.21.15.115:0/1745855306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc0074d80 0x7f5cc019b590 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:13.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.715+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 >> 172.21.15.115:0/1745855306 conn(0x7f5cc00fb3d0 msgr2=0x7f5cc00fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:13.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.715+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 shutdown_connections 2024-03-22T10:31:13.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:13.715+0000 7f5cc7400700 1 -- 172.21.15.115:0/1745855306 wait complete. 2024-03-22T10:31:14.896 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:14 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1745855306' entity='client.admin' 2024-03-22T10:31:14.943 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-22T10:31:14.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-22T10:31:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:31:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.082+0000 7f8401c00700 1 -- 172.21.15.115:0/3252644079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 msgr2=0x7f83fc0fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.082+0000 7f8401c00700 1 --2- 172.21.15.115:0/3252644079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 0x7f83fc0fe500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009910 tx=0x7f83ec009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.082+0000 7f8401c00700 1 -- 172.21.15.115:0/3252644079 shutdown_connections 2024-03-22T10:31:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.082+0000 7f8401c00700 1 --2- 172.21.15.115:0/3252644079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 0x7f83fc0fe500 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.082+0000 7f8401c00700 1 -- 172.21.15.115:0/3252644079 >> 172.21.15.115:0/3252644079 conn(0x7f83fc0f9a60 msgr2=0x7f83fc0fbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.082+0000 7f8401c00700 1 -- 172.21.15.115:0/3252644079 shutdown_connections 2024-03-22T10:31:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.082+0000 7f8401c00700 1 -- 172.21.15.115:0/3252644079 wait complete. 2024-03-22T10:31:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.083+0000 7f8401c00700 1 Processor -- start 2024-03-22T10:31:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.083+0000 7f8401c00700 1 -- start start 2024-03-22T10:31:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.084+0000 7f8401c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 0x7f83fc197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.084+0000 7f8401c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc1976c0 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.084+0000 7f83faa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 0x7f83fc197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.084+0000 7f83faa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 0x7f83fc197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39852/0 (socket says 172.21.15.115:39852) 2024-03-22T10:31:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.084+0000 7f83faa00700 1 -- 172.21.15.115:0/404467871 learned_addr learned my addr 172.21.15.115:0/404467871 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.084+0000 7f83faa00700 1 -- 172.21.15.115:0/404467871 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0095f0 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.085+0000 7f83faa00700 1 --2- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 0x7f83fc197180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005950 tx=0x7f83ec009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:18.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.085+0000 7f83f3400700 1 -- 172.21.15.115:0/404467871 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83ec01d070 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.085+0000 7f83f3400700 1 -- 172.21.15.115:0/404467871 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f83ec00be10 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.085+0000 7f83f3400700 1 -- 172.21.15.115:0/404467871 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83ec00f460 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.085+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc1978c0 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.085+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc197d60 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.086+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc190f10 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.087+0000 7f83f3400700 1 -- 172.21.15.115:0/404467871 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f83ec0227c0 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.087+0000 7f83f3400700 1 --2- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f83e00384e0 0x7f83e003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.087+0000 7f83f3400700 1 -- 172.21.15.115:0/404467871 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f83ec04c420 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.087+0000 7f83fa000700 1 --2- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f83e00384e0 0x7f83e003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.088+0000 7f83fa000700 1 --2- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f83e00384e0 0x7f83e003a990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f83e8006fd0 tx=0x7f83e8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:18.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.093+0000 7f83f3400700 1 -- 172.21.15.115:0/404467871 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f83ec005c00 con 0x7f83fc0fe0f0 2024-03-22T10:31:18.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.312+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 --> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f83fc061220 con 0x7f83e00384e0 2024-03-22T10:31:18.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.320+0000 7f83f3400700 1 -- 172.21.15.115:0/404467871 <== mgr.14164 v2:172.21.15.115:6800/2729066911 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f83fc061220 con 0x7f83e00384e0 2024-03-22T10:31:18.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f83e00384e0 msgr2=0x7f83e003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 --2- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f83e00384e0 0x7f83e003a990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f83e8006fd0 tx=0x7f83e8006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 msgr2=0x7f83fc197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 --2- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 0x7f83fc197180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005950 tx=0x7f83ec009e60 comp rx=0 tx=0).stop 2024-03-22T10:31:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 shutdown_connections 2024-03-22T10:31:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 --2- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f83e00384e0 0x7f83e003a990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 --2- 172.21.15.115:0/404467871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83fc0fe0f0 0x7f83fc197180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 >> 172.21.15.115:0/404467871 conn(0x7f83fc0f9a60 msgr2=0x7f83fc0fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 shutdown_connections 2024-03-22T10:31:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:18.324+0000 7f8401c00700 1 -- 172.21.15.115:0/404467871 wait complete. 2024-03-22T10:31:19.005 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-03-22T10:31:19.005 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:31:19.005 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-03-22T10:31:19.059 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:31:19.060 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:31:19.155 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-03-22T10:31:19.156 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch host add smithi150 2024-03-22T10:31:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:31:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:31:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-22T10:31:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:31:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: Deploying daemon crash.smithi115 on smithi115 2024-03-22T10:31:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:19 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:20.268 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:31:21.945 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:21 smithi115 ceph-mon[29111]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:22.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.520+0000 7f20e3800700 1 -- 172.21.15.115:0/2394889869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 msgr2=0x7f20dc0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:22.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.520+0000 7f20e3800700 1 --2- 172.21.15.115:0/2394889869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 0x7f20dc0731e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009910 tx=0x7f20d0009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.520+0000 7f20e3800700 1 -- 172.21.15.115:0/2394889869 shutdown_connections 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.520+0000 7f20e3800700 1 --2- 172.21.15.115:0/2394889869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 0x7f20dc0731e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.520+0000 7f20e3800700 1 -- 172.21.15.115:0/2394889869 >> 172.21.15.115:0/2394889869 conn(0x7f20dc0fb3d0 msgr2=0x7f20dc0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.520+0000 7f20e3800700 1 -- 172.21.15.115:0/2394889869 shutdown_connections 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.520+0000 7f20e3800700 1 -- 172.21.15.115:0/2394889869 wait complete. 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.522+0000 7f20e3800700 1 Processor -- start 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.522+0000 7f20e3800700 1 -- start start 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.522+0000 7f20e3800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 0x7f20dc197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.522+0000 7f20e3800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc1976b0 con 0x7f20dc074d80 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.522+0000 7f20e0e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 0x7f20dc197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.523+0000 7f20e0e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 0x7f20dc197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50210/0 (socket says 172.21.15.115:50210) 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.523+0000 7f20e0e00700 1 -- 172.21.15.115:0/1762117608 learned_addr learned my addr 172.21.15.115:0/1762117608 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.523+0000 7f20e0e00700 1 -- 172.21.15.115:0/1762117608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d00095f0 con 0x7f20dc074d80 2024-03-22T10:31:22.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.523+0000 7f20e0e00700 1 --2- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 0x7f20dc197170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005b40 tx=0x7f20d0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.524+0000 7f20d9600700 1 -- 172.21.15.115:0/1762117608 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20d001d070 con 0x7f20dc074d80 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.524+0000 7f20d9600700 1 -- 172.21.15.115:0/1762117608 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f20d000be10 con 0x7f20dc074d80 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.524+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc1978b0 con 0x7f20dc074d80 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.524+0000 7f20d9600700 1 -- 172.21.15.115:0/1762117608 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20d000f460 con 0x7f20dc074d80 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.524+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc197d50 con 0x7f20dc074d80 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.525+0000 7f20d9600700 1 -- 172.21.15.115:0/1762117608 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f20d000f5c0 con 0x7f20dc074d80 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.525+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc190d20 con 0x7f20dc074d80 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.526+0000 7f20d9600700 1 --2- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f20c40384e0 0x7f20c403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.526+0000 7f20d9600700 1 -- 172.21.15.115:0/1762117608 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f20d004d450 con 0x7f20dc074d80 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.526+0000 7f20dbe00700 1 --2- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f20c40384e0 0x7f20c403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.527+0000 7f20dbe00700 1 --2- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f20c40384e0 0x7f20c403a990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f20c8006fd0 tx=0x7f20c8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:22.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.531+0000 7f20d9600700 1 -- 172.21.15.115:0/1762117608 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f20d002a360 con 0x7f20dc074d80 2024-03-22T10:31:22.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:22.755+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 --> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) v1 -- 0x7f20dc061190 con 0x7f20c40384e0 2024-03-22T10:31:23.858 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:23 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:23.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:23 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:23.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:23 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:23.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:23 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:23.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:23 smithi115 ceph-mon[29111]: Deploying daemon node-exporter.smithi115 on smithi115 2024-03-22T10:31:23.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:23 smithi115 ceph-mon[29111]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:23.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:23 smithi115 ceph-mon[29111]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:31:25.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:24 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:25.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:24 smithi115 ceph-mon[29111]: Deploying cephadm binary to smithi150 2024-03-22T10:31:26.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:25 smithi115 ceph-mon[29111]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:27.435 INFO:teuthology.orchestra.run.smithi115.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.437+0000 7f20d9600700 1 -- 172.21.15.115:0/1762117608 <== mgr.14164 v2:172.21.15.115:6800/2729066911 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f20dc061190 con 0x7f20c40384e0 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f20c40384e0 msgr2=0x7f20c403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 --2- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f20c40384e0 0x7f20c403a990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f20c8006fd0 tx=0x7f20c8006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 msgr2=0x7f20dc197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 --2- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 0x7f20dc197170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005b40 tx=0x7f20d0004740 comp rx=0 tx=0).stop 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 shutdown_connections 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 --2- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f20c40384e0 0x7f20c403a990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 --2- 172.21.15.115:0/1762117608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc074d80 0x7f20dc197170 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 >> 172.21.15.115:0/1762117608 conn(0x7f20dc0fb3d0 msgr2=0x7f20dc0fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 shutdown_connections 2024-03-22T10:31:27.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:27.441+0000 7f20e3800700 1 -- 172.21.15.115:0/1762117608 wait complete. 2024-03-22T10:31:27.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:27 smithi115 ceph-mon[29111]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:27.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:27 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:27.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:27 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:27.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:27 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:27.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:27 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:27.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:27 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:28.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-22T10:31:28.521 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:31:28.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:28 smithi115 ceph-mon[29111]: Added host smithi150 2024-03-22T10:31:28.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:28 smithi115 ceph-mon[29111]: Deploying daemon alertmanager.smithi115 on smithi115 2024-03-22T10:31:28.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:28 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:29 smithi115 ceph-mon[29111]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.031+0000 7fc958400700 1 -- 172.21.15.115:0/2900308984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 msgr2=0x7fc950073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.031+0000 7fc958400700 1 --2- 172.21.15.115:0/2900308984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 0x7fc950073160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc944009910 tx=0x7fc944009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:31.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.032+0000 7fc958400700 1 -- 172.21.15.115:0/2900308984 shutdown_connections 2024-03-22T10:31:31.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.032+0000 7fc958400700 1 --2- 172.21.15.115:0/2900308984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 0x7fc950073160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:31.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.032+0000 7fc958400700 1 -- 172.21.15.115:0/2900308984 >> 172.21.15.115:0/2900308984 conn(0x7fc9500fb3d0 msgr2=0x7fc9500fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:31.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.032+0000 7fc958400700 1 -- 172.21.15.115:0/2900308984 shutdown_connections 2024-03-22T10:31:31.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.032+0000 7fc958400700 1 -- 172.21.15.115:0/2900308984 wait complete. 2024-03-22T10:31:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.033+0000 7fc958400700 1 Processor -- start 2024-03-22T10:31:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.033+0000 7fc958400700 1 -- start start 2024-03-22T10:31:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.034+0000 7fc958400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 0x7fc950197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.034+0000 7fc958400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501976d0 con 0x7fc950074d00 2024-03-22T10:31:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.034+0000 7fc955a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 0x7fc950197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.034+0000 7fc955a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 0x7fc950197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50226/0 (socket says 172.21.15.115:50226) 2024-03-22T10:31:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.034+0000 7fc955a00700 1 -- 172.21.15.115:0/591904252 learned_addr learned my addr 172.21.15.115:0/591904252 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.035+0000 7fc955a00700 1 -- 172.21.15.115:0/591904252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9440095f0 con 0x7fc950074d00 2024-03-22T10:31:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.035+0000 7fc955a00700 1 --2- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 0x7fc950197190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc9440092e0 tx=0x7fc944009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.035+0000 7fc94e000700 1 -- 172.21.15.115:0/591904252 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc94401d070 con 0x7fc950074d00 2024-03-22T10:31:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.035+0000 7fc94e000700 1 -- 172.21.15.115:0/591904252 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc94400be10 con 0x7fc950074d00 2024-03-22T10:31:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.035+0000 7fc94e000700 1 -- 172.21.15.115:0/591904252 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc94400f460 con 0x7fc950074d00 2024-03-22T10:31:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.035+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9501978d0 con 0x7fc950074d00 2024-03-22T10:31:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.036+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc950197d70 con 0x7fc950074d00 2024-03-22T10:31:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.037+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950190d20 con 0x7fc950074d00 2024-03-22T10:31:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.038+0000 7fc94e000700 1 -- 172.21.15.115:0/591904252 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fc944022470 con 0x7fc950074d00 2024-03-22T10:31:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.038+0000 7fc94e000700 1 --2- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc938038530 0x7fc93803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.038+0000 7fc94e000700 1 -- 172.21.15.115:0/591904252 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc94404c490 con 0x7fc950074d00 2024-03-22T10:31:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.038+0000 7fc955000700 1 --2- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc938038530 0x7fc93803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.039+0000 7fc955000700 1 --2- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc938038530 0x7fc93803a9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc93c006fd0 tx=0x7fc93c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.048+0000 7fc94e000700 1 -- 172.21.15.115:0/591904252 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc94402ba00 con 0x7fc950074d00 2024-03-22T10:31:31.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.332+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 --> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc950061190 con 0x7fc938038530 2024-03-22T10:31:31.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.334+0000 7fc94e000700 1 -- 172.21.15.115:0/591904252 <== mgr.14164 v2:172.21.15.115:6800/2729066911 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7fc950061190 con 0x7fc938038530 2024-03-22T10:31:31.329 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:31:31.329 INFO:teuthology.orchestra.run.smithi115.stdout:[{"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-03-22T10:31:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.337+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc938038530 msgr2=0x7fc93803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.337+0000 7fc958400700 1 --2- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc938038530 0x7fc93803a9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc93c006fd0 tx=0x7fc93c006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.337+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 msgr2=0x7fc950197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.337+0000 7fc958400700 1 --2- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 0x7fc950197190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc9440092e0 tx=0x7fc944009e60 comp rx=0 tx=0).stop 2024-03-22T10:31:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.337+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 shutdown_connections 2024-03-22T10:31:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.337+0000 7fc958400700 1 --2- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc938038530 0x7fc93803a9e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.337+0000 7fc958400700 1 --2- 172.21.15.115:0/591904252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc950074d00 0x7fc950197190 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.337+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 >> 172.21.15.115:0/591904252 conn(0x7fc9500fb3d0 msgr2=0x7fc9500fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:31.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.338+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 shutdown_connections 2024-03-22T10:31:31.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:31.338+0000 7fc958400700 1 -- 172.21.15.115:0/591904252 wait complete. 2024-03-22T10:31:32.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:31 smithi115 ceph-mon[29111]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:33.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:32 smithi115 ceph-mon[29111]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T10:31:33.134 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-22T10:31:33.135 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-22T10:31:33.947 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:33 smithi115 ceph-mon[29111]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:34.443 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:31:36.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:35 smithi115 ceph-mon[29111]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:37.963 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:37 smithi115 ceph-mon[29111]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:38.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.026+0000 7f9057400700 1 -- 172.21.15.115:0/2418543330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 msgr2=0x7f9050100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:38.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.026+0000 7f9057400700 1 --2- 172.21.15.115:0/2418543330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 0x7f9050100250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9044009910 tx=0x7f9044009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:38.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.026+0000 7f9057400700 1 -- 172.21.15.115:0/2418543330 shutdown_connections 2024-03-22T10:31:38.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.026+0000 7f9057400700 1 --2- 172.21.15.115:0/2418543330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 0x7f9050100250 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:38.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.026+0000 7f9057400700 1 -- 172.21.15.115:0/2418543330 >> 172.21.15.115:0/2418543330 conn(0x7f90500fb3d0 msgr2=0x7f90500fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:38.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.026+0000 7f9057400700 1 -- 172.21.15.115:0/2418543330 shutdown_connections 2024-03-22T10:31:38.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.026+0000 7f9057400700 1 -- 172.21.15.115:0/2418543330 wait complete. 2024-03-22T10:31:38.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.027+0000 7f9057400700 1 Processor -- start 2024-03-22T10:31:38.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.027+0000 7f9057400700 1 -- start start 2024-03-22T10:31:38.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.028+0000 7f9057400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 0x7f9050197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:38.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.028+0000 7f9057400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90501976d0 con 0x7f90500ffe40 2024-03-22T10:31:38.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.028+0000 7f9054a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 0x7f9050197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:38.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.028+0000 7f9054a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 0x7f9050197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45424/0 (socket says 172.21.15.115:45424) 2024-03-22T10:31:38.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.028+0000 7f9054a00700 1 -- 172.21.15.115:0/339422584 learned_addr learned my addr 172.21.15.115:0/339422584 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:31:38.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.029+0000 7f9054a00700 1 -- 172.21.15.115:0/339422584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440095f0 con 0x7f90500ffe40 2024-03-22T10:31:38.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.029+0000 7f9054a00700 1 --2- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 0x7f9050197190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90440092e0 tx=0x7f9044009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:38.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.029+0000 7f904d600700 1 -- 172.21.15.115:0/339422584 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f904401d070 con 0x7f90500ffe40 2024-03-22T10:31:38.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.029+0000 7f904d600700 1 -- 172.21.15.115:0/339422584 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f904400be10 con 0x7f90500ffe40 2024-03-22T10:31:38.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.029+0000 7f904d600700 1 -- 172.21.15.115:0/339422584 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f904400f460 con 0x7f90500ffe40 2024-03-22T10:31:38.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.030+0000 7f9057400700 1 -- 172.21.15.115:0/339422584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90501978d0 con 0x7f90500ffe40 2024-03-22T10:31:38.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.030+0000 7f9057400700 1 -- 172.21.15.115:0/339422584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9050197d70 con 0x7f90500ffe40 2024-03-22T10:31:38.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.031+0000 7f9042a00700 1 -- 172.21.15.115:0/339422584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9050105bf0 con 0x7f90500ffe40 2024-03-22T10:31:38.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.032+0000 7f904d600700 1 -- 172.21.15.115:0/339422584 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f9044022470 con 0x7f90500ffe40 2024-03-22T10:31:38.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.032+0000 7f904d600700 1 --2- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9038038490 0x7f903803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:38.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.032+0000 7f904d600700 1 -- 172.21.15.115:0/339422584 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f904404c180 con 0x7f90500ffe40 2024-03-22T10:31:38.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.033+0000 7f904fe00700 1 --2- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9038038490 0x7f903803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:38.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.034+0000 7f904fe00700 1 --2- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9038038490 0x7f903803a940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f903c006fd0 tx=0x7f903c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:38.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.038+0000 7f904d600700 1 -- 172.21.15.115:0/339422584 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f904400b960 con 0x7f90500ffe40 2024-03-22T10:31:38.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.291+0000 7f9042a00700 1 -- 172.21.15.115:0/339422584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f90501008d0 con 0x7f90500ffe40 2024-03-22T10:31:38.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.854+0000 7f904d600700 1 -- 172.21.15.115:0/339422584 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f90501008d0 con 0x7f90500ffe40 2024-03-22T10:31:38.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.858+0000 7f9042a00700 1 -- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9038038490 msgr2=0x7f903803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:38.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.858+0000 7f9042a00700 1 --2- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9038038490 0x7f903803a940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f903c006fd0 tx=0x7f903c006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:38.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.859+0000 7f9042a00700 1 -- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 msgr2=0x7f9050197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:38.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.859+0000 7f9042a00700 1 --2- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 0x7f9050197190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90440092e0 tx=0x7f9044009e60 comp rx=0 tx=0).stop 2024-03-22T10:31:38.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.859+0000 7f9042a00700 1 -- 172.21.15.115:0/339422584 shutdown_connections 2024-03-22T10:31:38.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.859+0000 7f9042a00700 1 --2- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9038038490 0x7f903803a940 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:38.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.859+0000 7f9042a00700 1 --2- 172.21.15.115:0/339422584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90500ffe40 0x7f9050197190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:38.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.859+0000 7f9042a00700 1 -- 172.21.15.115:0/339422584 >> 172.21.15.115:0/339422584 conn(0x7f90500fb3d0 msgr2=0x7f90500fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:38.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.859+0000 7f9042a00700 1 -- 172.21.15.115:0/339422584 shutdown_connections 2024-03-22T10:31:38.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:31:38.859+0000 7f9042a00700 1 -- 172.21.15.115:0/339422584 wait complete. 2024-03-22T10:31:38.855 INFO:teuthology.orchestra.run.smithi115.stderr:adjusted tunables profile to default 2024-03-22T10:31:39.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:38 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/339422584' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-22T10:31:39.775 INFO:tasks.cephadm:Adding mon.smithi115 on smithi115 2024-03-22T10:31:39.775 INFO:tasks.cephadm:Adding mon.smithi150 on smithi150 2024-03-22T10:31:39.775 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch apply mon '2;smithi115:172.21.15.115=smithi115;smithi150:172.21.15.150=smithi150' 2024-03-22T10:31:40.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:40.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/339422584' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-22T10:31:40.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T10:31:40.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:40.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:40.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:40.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:40.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:40.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:40.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T10:31:40.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:39 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:40.149 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:40.246 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:41.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:40 smithi115 ceph-mon[29111]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-22T10:31:41.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:40 smithi115 ceph-mon[29111]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T10:31:41.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:40 smithi115 ceph-mon[29111]: Deploying daemon grafana.smithi115 on smithi115 2024-03-22T10:31:42.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.235+0000 7f3c9a000700 1 -- 172.21.15.150:0/3365279294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 msgr2=0x7f3c94100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:42.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.236+0000 7f3c9a000700 1 --2- 172.21.15.150:0/3365279294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 0x7f3c94100250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009910 tx=0x7f3c84009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:42.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.236+0000 7f3c9a000700 1 -- 172.21.15.150:0/3365279294 shutdown_connections 2024-03-22T10:31:42.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.236+0000 7f3c9a000700 1 --2- 172.21.15.150:0/3365279294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 0x7f3c94100250 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:42.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.236+0000 7f3c9a000700 1 -- 172.21.15.150:0/3365279294 >> 172.21.15.150:0/3365279294 conn(0x7f3c940fb3d0 msgr2=0x7f3c940fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:42.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.236+0000 7f3c9a000700 1 -- 172.21.15.150:0/3365279294 shutdown_connections 2024-03-22T10:31:42.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.236+0000 7f3c9a000700 1 -- 172.21.15.150:0/3365279294 wait complete. 2024-03-22T10:31:42.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.237+0000 7f3c9a000700 1 Processor -- start 2024-03-22T10:31:42.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.238+0000 7f3c9a000700 1 -- start start 2024-03-22T10:31:42.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.238+0000 7f3c9a000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 0x7f3c94074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:42.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.238+0000 7f3c9a000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c94075030 con 0x7f3c940ffe40 2024-03-22T10:31:42.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.239+0000 7f3c93400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 0x7f3c94074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:42.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.239+0000 7f3c93400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 0x7f3c94074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:51994/0 (socket says 172.21.15.150:51994) 2024-03-22T10:31:42.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.239+0000 7f3c93400700 1 -- 172.21.15.150:0/2425494304 learned_addr learned my addr 172.21.15.150:0/2425494304 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:31:42.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.240+0000 7f3c93400700 1 -- 172.21.15.150:0/2425494304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c840095f0 con 0x7f3c940ffe40 2024-03-22T10:31:42.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.240+0000 7f3c93400700 1 --2- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 0x7f3c94074af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c840092e0 tx=0x7f3c84009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:42.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.241+0000 7f3c90c00700 1 -- 172.21.15.150:0/2425494304 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c8401d070 con 0x7f3c940ffe40 2024-03-22T10:31:42.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.241+0000 7f3c90c00700 1 -- 172.21.15.150:0/2425494304 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3c8400be10 con 0x7f3c940ffe40 2024-03-22T10:31:42.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.241+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c94073140 con 0x7f3c940ffe40 2024-03-22T10:31:42.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.241+0000 7f3c90c00700 1 -- 172.21.15.150:0/2425494304 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c8400f460 con 0x7f3c940ffe40 2024-03-22T10:31:42.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.241+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c940735e0 con 0x7f3c940ffe40 2024-03-22T10:31:42.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.242+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c94062380 con 0x7f3c940ffe40 2024-03-22T10:31:42.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.243+0000 7f3c90c00700 1 -- 172.21.15.150:0/2425494304 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f3c84022470 con 0x7f3c940ffe40 2024-03-22T10:31:42.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.243+0000 7f3c90c00700 1 --2- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3c780384e0 0x7f3c7803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:42.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.243+0000 7f3c90c00700 1 -- 172.21.15.150:0/2425494304 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3c8404cc60 con 0x7f3c940ffe40 2024-03-22T10:31:42.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.244+0000 7f3c8be00700 1 --2- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3c780384e0 0x7f3c7803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:42.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.245+0000 7f3c8be00700 1 --2- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3c780384e0 0x7f3c7803a990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c006fd0 tx=0x7f3c7c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:42.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.258+0000 7f3c90c00700 1 -- 172.21.15.150:0/2425494304 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3c84022790 con 0x7f3c940ffe40 2024-03-22T10:31:42.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:41 smithi115 ceph-mon[29111]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:42.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.484+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 --> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi115:172.21.15.115=smithi115;smithi150:172.21.15.150=smithi150", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c94073820 con 0x7f3c780384e0 2024-03-22T10:31:42.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.501+0000 7f3c90c00700 1 -- 172.21.15.150:0/2425494304 <== mgr.14164 v2:172.21.15.115:6800/2729066911 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3c94073820 con 0x7f3c780384e0 2024-03-22T10:31:42.495 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mon update... 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3c780384e0 msgr2=0x7f3c7803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 --2- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3c780384e0 0x7f3c7803a990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c006fd0 tx=0x7f3c7c006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 msgr2=0x7f3c94074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 --2- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 0x7f3c94074af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c840092e0 tx=0x7f3c84009e60 comp rx=0 tx=0).stop 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 shutdown_connections 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 --2- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3c780384e0 0x7f3c7803a990 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 --2- 172.21.15.150:0/2425494304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c940ffe40 0x7f3c94074af0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 >> 172.21.15.150:0/2425494304 conn(0x7f3c940fb3d0 msgr2=0x7f3c940fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 shutdown_connections 2024-03-22T10:31:42.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:42.503+0000 7f3c9a000700 1 -- 172.21.15.150:0/2425494304 wait complete. 2024-03-22T10:31:43.111 DEBUG:teuthology.orchestra.run.smithi150:mon.smithi150> sudo journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi150.service 2024-03-22T10:31:43.119 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:31:43.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:31:43.521 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:43.625 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:43.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:43 smithi115 ceph-mon[29111]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi115:172.21.15.115=smithi115;smithi150:172.21.15.150=smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:31:43.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:43 smithi115 ceph-mon[29111]: Saving service mon spec with placement smithi115:172.21.15.115=smithi115;smithi150:172.21.15.150=smithi150;count:2 2024-03-22T10:31:43.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:43 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:43.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:43 smithi115 ceph-mon[29111]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:45.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:44 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:45.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.425+0000 7f0741000700 1 -- 172.21.15.150:0/3542876340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 msgr2=0x7f073c102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:45.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.425+0000 7f0741000700 1 --2- 172.21.15.150:0/3542876340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 0x7f073c102ac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f072c009910 tx=0x7f072c009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:45.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.425+0000 7f0741000700 1 -- 172.21.15.150:0/3542876340 shutdown_connections 2024-03-22T10:31:45.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.425+0000 7f0741000700 1 --2- 172.21.15.150:0/3542876340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 0x7f073c102ac0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:45.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.425+0000 7f0741000700 1 -- 172.21.15.150:0/3542876340 >> 172.21.15.150:0/3542876340 conn(0x7f073c0fa0e0 msgr2=0x7f073c0fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:45.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.425+0000 7f0741000700 1 -- 172.21.15.150:0/3542876340 shutdown_connections 2024-03-22T10:31:45.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.425+0000 7f0741000700 1 -- 172.21.15.150:0/3542876340 wait complete. 2024-03-22T10:31:45.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.427+0000 7f0741000700 1 Processor -- start 2024-03-22T10:31:45.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.427+0000 7f0741000700 1 -- start start 2024-03-22T10:31:45.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.427+0000 7f0741000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 0x7f073c196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:45.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.427+0000 7f0741000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1974f0 con 0x7f073c1006e0 2024-03-22T10:31:45.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.428+0000 7f073b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 0x7f073c196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:45.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.428+0000 7f073b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 0x7f073c196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:52010/0 (socket says 172.21.15.150:52010) 2024-03-22T10:31:45.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.428+0000 7f073b400700 1 -- 172.21.15.150:0/774036424 learned_addr learned my addr 172.21.15.150:0/774036424 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:31:45.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.428+0000 7f073b400700 1 -- 172.21.15.150:0/774036424 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0095f0 con 0x7f073c1006e0 2024-03-22T10:31:45.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.429+0000 7f073b400700 1 --2- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 0x7f073c196fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f072c0092e0 tx=0x7f072c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:45.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.429+0000 7f0733e00700 1 -- 172.21.15.150:0/774036424 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f072c01d070 con 0x7f073c1006e0 2024-03-22T10:31:45.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.429+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c1976f0 con 0x7f073c1006e0 2024-03-22T10:31:45.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.429+0000 7f0733e00700 1 -- 172.21.15.150:0/774036424 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f072c00be10 con 0x7f073c1006e0 2024-03-22T10:31:45.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.429+0000 7f0733e00700 1 -- 172.21.15.150:0/774036424 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f072c00f460 con 0x7f073c1006e0 2024-03-22T10:31:45.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.430+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c197b90 con 0x7f073c1006e0 2024-03-22T10:31:45.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.431+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c190d30 con 0x7f073c1006e0 2024-03-22T10:31:45.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.431+0000 7f0733e00700 1 -- 172.21.15.150:0/774036424 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f072c022470 con 0x7f073c1006e0 2024-03-22T10:31:45.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.431+0000 7f0733e00700 1 --2- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f07200384e0 0x7f072003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:45.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.432+0000 7f0733e00700 1 -- 172.21.15.150:0/774036424 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f072c04c540 con 0x7f073c1006e0 2024-03-22T10:31:45.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.432+0000 7f073aa00700 1 --2- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f07200384e0 0x7f072003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:45.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.433+0000 7f073aa00700 1 --2- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f07200384e0 0x7f072003a990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0728006fd0 tx=0x7f0728006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:45.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.447+0000 7f0733e00700 1 -- 172.21.15.150:0/774036424 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f072c02ba00 con 0x7f073c1006e0 2024-03-22T10:31:45.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.748+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f073c02cc30 con 0x7f073c1006e0 2024-03-22T10:31:45.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.749+0000 7f0733e00700 1 -- 172.21.15.150:0/774036424 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f072c027070 con 0x7f073c1006e0 2024-03-22T10:31:45.746 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:31:45.746 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f07200384e0 msgr2=0x7f072003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 --2- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f07200384e0 0x7f072003a990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0728006fd0 tx=0x7f0728006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 msgr2=0x7f073c196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 --2- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 0x7f073c196fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f072c0092e0 tx=0x7f072c009e60 comp rx=0 tx=0).stop 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 shutdown_connections 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 --2- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f07200384e0 0x7f072003a990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 --2- 172.21.15.150:0/774036424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f073c1006e0 0x7f073c196fb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 >> 172.21.15.150:0/774036424 conn(0x7f073c0fa0e0 msgr2=0x7f073c0fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 shutdown_connections 2024-03-22T10:31:45.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:45.752+0000 7f0741000700 1 -- 172.21.15.150:0/774036424 wait complete. 2024-03-22T10:31:45.748 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:31:46.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:45 smithi115 ceph-mon[29111]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:47.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:46 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/774036424' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:31:47.424 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:31:47.424 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:31:47.798 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:47.894 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:48.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:47 smithi115 ceph-mon[29111]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:50.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.132+0000 7f5893200700 1 -- 172.21.15.150:0/2544018494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 msgr2=0x7f588c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:50.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.132+0000 7f5893200700 1 --2- 172.21.15.150:0/2544018494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 0x7f588c0731e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f587c009910 tx=0x7f587c009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:50.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.132+0000 7f5893200700 1 -- 172.21.15.150:0/2544018494 shutdown_connections 2024-03-22T10:31:50.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.132+0000 7f5893200700 1 --2- 172.21.15.150:0/2544018494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 0x7f588c0731e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:50.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.132+0000 7f5893200700 1 -- 172.21.15.150:0/2544018494 >> 172.21.15.150:0/2544018494 conn(0x7f588c0fb3d0 msgr2=0x7f588c0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:50.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.132+0000 7f5893200700 1 -- 172.21.15.150:0/2544018494 shutdown_connections 2024-03-22T10:31:50.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.132+0000 7f5893200700 1 -- 172.21.15.150:0/2544018494 wait complete. 2024-03-22T10:31:50.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.134+0000 7f5893200700 1 Processor -- start 2024-03-22T10:31:50.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.134+0000 7f5893200700 1 -- start start 2024-03-22T10:31:50.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.134+0000 7f5893200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 0x7f588c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:50.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.134+0000 7f5893200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1976b0 con 0x7f588c074d80 2024-03-22T10:31:50.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.135+0000 7f588be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 0x7f588c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:50.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.135+0000 7f588be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 0x7f588c197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:52030/0 (socket says 172.21.15.150:52030) 2024-03-22T10:31:50.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.135+0000 7f588be00700 1 -- 172.21.15.150:0/2829916502 learned_addr learned my addr 172.21.15.150:0/2829916502 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:31:50.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.135+0000 7f588be00700 1 -- 172.21.15.150:0/2829916502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c0095f0 con 0x7f588c074d80 2024-03-22T10:31:50.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.135+0000 7f588be00700 1 --2- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 0x7f588c197170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f587c0092e0 tx=0x7f587c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:50.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.136+0000 7f5888c00700 1 -- 172.21.15.150:0/2829916502 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f587c01d070 con 0x7f588c074d80 2024-03-22T10:31:50.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.136+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c1978b0 con 0x7f588c074d80 2024-03-22T10:31:50.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.136+0000 7f5888c00700 1 -- 172.21.15.150:0/2829916502 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f587c00be10 con 0x7f588c074d80 2024-03-22T10:31:50.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.136+0000 7f5888c00700 1 -- 172.21.15.150:0/2829916502 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f587c00f460 con 0x7f588c074d80 2024-03-22T10:31:50.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.136+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c197d50 con 0x7f588c074d80 2024-03-22T10:31:50.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.137+0000 7f5888c00700 1 -- 172.21.15.150:0/2829916502 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f587c022470 con 0x7f588c074d80 2024-03-22T10:31:50.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.137+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c190d20 con 0x7f588c074d80 2024-03-22T10:31:50.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.137+0000 7f5888c00700 1 --2- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5870038490 0x7f587003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:50.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.137+0000 7f5888c00700 1 -- 172.21.15.150:0/2829916502 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f587c04c4a0 con 0x7f588c074d80 2024-03-22T10:31:50.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.138+0000 7f588b400700 1 --2- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5870038490 0x7f587003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:50.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.139+0000 7f588b400700 1 --2- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5870038490 0x7f587003a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5878006fd0 tx=0x7f5878006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.153+0000 7f5888c00700 1 -- 172.21.15.150:0/2829916502 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f587c00b960 con 0x7f588c074d80 2024-03-22T10:31:50.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:49 smithi115 ceph-mon[29111]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:50.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.451+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f588c02cc30 con 0x7f588c074d80 2024-03-22T10:31:50.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.452+0000 7f5888c00700 1 -- 172.21.15.150:0/2829916502 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f587c027070 con 0x7f588c074d80 2024-03-22T10:31:50.447 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:31:50.447 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5870038490 msgr2=0x7f587003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 --2- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5870038490 0x7f587003a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5878006fd0 tx=0x7f5878006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 msgr2=0x7f588c197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 --2- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 0x7f588c197170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f587c0092e0 tx=0x7f587c009e60 comp rx=0 tx=0).stop 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 shutdown_connections 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 --2- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f5870038490 0x7f587003a940 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 --2- 172.21.15.150:0/2829916502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c074d80 0x7f588c197170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 >> 172.21.15.150:0/2829916502 conn(0x7f588c0fb3d0 msgr2=0x7f588c0fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 shutdown_connections 2024-03-22T10:31:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:50.455+0000 7f5893200700 1 -- 172.21.15.150:0/2829916502 wait complete. 2024-03-22T10:31:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:31:51.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:50 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/2829916502' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:31:52.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:51 smithi115 ceph-mon[29111]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:52.505 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:31:52.506 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:31:52.878 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:52.966 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:54.132 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:53 smithi115 ceph-mon[29111]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:55.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.123+0000 7fc1ade00700 1 -- 172.21.15.150:0/2756159463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 msgr2=0x7fc1a8075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:55.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.123+0000 7fc1ade00700 1 --2- 172.21.15.150:0/2756159463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 0x7fc1a8075a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009910 tx=0x7fc19c009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:55.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.123+0000 7fc1ade00700 1 -- 172.21.15.150:0/2756159463 shutdown_connections 2024-03-22T10:31:55.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.123+0000 7fc1ade00700 1 --2- 172.21.15.150:0/2756159463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 0x7fc1a8075a70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:55.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.123+0000 7fc1ade00700 1 -- 172.21.15.150:0/2756159463 >> 172.21.15.150:0/2756159463 conn(0x7fc1a80fd700 msgr2=0x7fc1a80ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:55.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.123+0000 7fc1ade00700 1 -- 172.21.15.150:0/2756159463 shutdown_connections 2024-03-22T10:31:55.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.123+0000 7fc1ade00700 1 -- 172.21.15.150:0/2756159463 wait complete. 2024-03-22T10:31:55.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.124+0000 7fc1ade00700 1 Processor -- start 2024-03-22T10:31:55.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.124+0000 7fc1ade00700 1 -- start start 2024-03-22T10:31:55.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.125+0000 7fc1ade00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 0x7fc1a819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:55.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.125+0000 7fc1ade00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a819b920 con 0x7fc1a8075660 2024-03-22T10:31:55.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.125+0000 7fc1aca00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 0x7fc1a819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:55.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.125+0000 7fc1aca00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 0x7fc1a819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:34194/0 (socket says 172.21.15.150:34194) 2024-03-22T10:31:55.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.125+0000 7fc1aca00700 1 -- 172.21.15.150:0/1132524536 learned_addr learned my addr 172.21.15.150:0/1132524536 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:31:55.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.126+0000 7fc1aca00700 1 -- 172.21.15.150:0/1132524536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0095f0 con 0x7fc1a8075660 2024-03-22T10:31:55.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.126+0000 7fc1aca00700 1 --2- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 0x7fc1a819b3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc19c005b40 tx=0x7fc19c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:55.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.126+0000 7fc1a5600700 1 -- 172.21.15.150:0/1132524536 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc19c01d070 con 0x7fc1a8075660 2024-03-22T10:31:55.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.126+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a819bb20 con 0x7fc1a8075660 2024-03-22T10:31:55.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.127+0000 7fc1a5600700 1 -- 172.21.15.150:0/1132524536 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc19c00be10 con 0x7fc1a8075660 2024-03-22T10:31:55.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.127+0000 7fc1a5600700 1 -- 172.21.15.150:0/1132524536 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc19c00f460 con 0x7fc1a8075660 2024-03-22T10:31:55.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.127+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a819bfc0 con 0x7fc1a8075660 2024-03-22T10:31:55.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.128+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a8194ef0 con 0x7fc1a8075660 2024-03-22T10:31:55.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.128+0000 7fc1a5600700 1 -- 172.21.15.150:0/1132524536 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fc19c0227e0 con 0x7fc1a8075660 2024-03-22T10:31:55.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.129+0000 7fc1a5600700 1 --2- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc1900384e0 0x7fc19003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:55.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.129+0000 7fc1a5600700 1 -- 172.21.15.150:0/1132524536 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc19c04cc10 con 0x7fc1a8075660 2024-03-22T10:31:55.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.130+0000 7fc1a7e00700 1 --2- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc1900384e0 0x7fc19003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:55.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.130+0000 7fc1a7e00700 1 --2- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc1900384e0 0x7fc19003a990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc194006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:55.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.144+0000 7fc1a5600700 1 -- 172.21.15.150:0/1132524536 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc19c02ba00 con 0x7fc1a8075660 2024-03-22T10:31:55.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.445+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc1a802cc30 con 0x7fc1a8075660 2024-03-22T10:31:55.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.446+0000 7fc1a5600700 1 -- 172.21.15.150:0/1132524536 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fc19c027030 con 0x7fc1a8075660 2024-03-22T10:31:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:31:55.440 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:31:55.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.449+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc1900384e0 msgr2=0x7fc19003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:55.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.449+0000 7fc1ade00700 1 --2- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc1900384e0 0x7fc19003a990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc194006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:55.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.449+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 msgr2=0x7fc1a819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:55.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.449+0000 7fc1ade00700 1 --2- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 0x7fc1a819b3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc19c005b40 tx=0x7fc19c004740 comp rx=0 tx=0).stop 2024-03-22T10:31:55.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.449+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 shutdown_connections 2024-03-22T10:31:55.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.449+0000 7fc1ade00700 1 --2- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fc1900384e0 0x7fc19003a990 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:55.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.449+0000 7fc1ade00700 1 --2- 172.21.15.150:0/1132524536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a8075660 0x7fc1a819b3e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:55.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.449+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 >> 172.21.15.150:0/1132524536 conn(0x7fc1a80fd700 msgr2=0x7fc1a80ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:55.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.450+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 shutdown_connections 2024-03-22T10:31:55.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:55.450+0000 7fc1ade00700 1 -- 172.21.15.150:0/1132524536 wait complete. 2024-03-22T10:31:55.444 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:31:56.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:55 smithi115 ceph-mon[29111]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:56.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:55 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/1132524536' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:31:57.140 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:31:57.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:31:57.532 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:57.631 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:31:57.858 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:57 smithi115 ceph-mon[29111]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:31:57.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:57 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:57.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:57 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:57.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:57 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:57.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:57 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:57.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:57 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:57.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:57 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:59.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:58 smithi115 ceph-mon[29111]: Deploying daemon prometheus.smithi115 on smithi115 2024-03-22T10:31:59.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:58 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:31:59.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.515+0000 7fa06be00700 1 -- 172.21.15.150:0/2103395705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 msgr2=0x7fa064102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:59.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.515+0000 7fa06be00700 1 --2- 172.21.15.150:0/2103395705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 0x7fa064102450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa058009910 tx=0x7fa058009c20 comp rx=0 tx=0).stop 2024-03-22T10:31:59.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.515+0000 7fa06be00700 1 -- 172.21.15.150:0/2103395705 shutdown_connections 2024-03-22T10:31:59.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.515+0000 7fa06be00700 1 --2- 172.21.15.150:0/2103395705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 0x7fa064102450 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:59.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.515+0000 7fa06be00700 1 -- 172.21.15.150:0/2103395705 >> 172.21.15.150:0/2103395705 conn(0x7fa0640fd700 msgr2=0x7fa0640ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:59.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.516+0000 7fa06be00700 1 -- 172.21.15.150:0/2103395705 shutdown_connections 2024-03-22T10:31:59.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.516+0000 7fa06be00700 1 -- 172.21.15.150:0/2103395705 wait complete. 2024-03-22T10:31:59.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.517+0000 7fa06be00700 1 Processor -- start 2024-03-22T10:31:59.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.517+0000 7fa06be00700 1 -- start start 2024-03-22T10:31:59.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.518+0000 7fa06be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 0x7fa0641971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:59.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.518+0000 7fa06be00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa064197710 con 0x7fa064102040 2024-03-22T10:31:59.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.518+0000 7fa069400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 0x7fa0641971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:59.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.518+0000 7fa069400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 0x7fa0641971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:34202/0 (socket says 172.21.15.150:34202) 2024-03-22T10:31:59.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.518+0000 7fa069400700 1 -- 172.21.15.150:0/2294619609 learned_addr learned my addr 172.21.15.150:0/2294619609 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:31:59.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.519+0000 7fa069400700 1 -- 172.21.15.150:0/2294619609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580095f0 con 0x7fa064102040 2024-03-22T10:31:59.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.519+0000 7fa069400700 1 --2- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 0x7fa0641971d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa058005b40 tx=0x7fa058004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.519+0000 7fa062000700 1 -- 172.21.15.150:0/2294619609 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa05801d070 con 0x7fa064102040 2024-03-22T10:31:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.520+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa064197910 con 0x7fa064102040 2024-03-22T10:31:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.520+0000 7fa062000700 1 -- 172.21.15.150:0/2294619609 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa05800be10 con 0x7fa064102040 2024-03-22T10:31:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.520+0000 7fa062000700 1 -- 172.21.15.150:0/2294619609 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa05800f460 con 0x7fa064102040 2024-03-22T10:31:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.520+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa064197db0 con 0x7fa064102040 2024-03-22T10:31:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.521+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa064190d20 con 0x7fa064102040 2024-03-22T10:31:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.521+0000 7fa062000700 1 -- 172.21.15.150:0/2294619609 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fa0580227e0 con 0x7fa064102040 2024-03-22T10:31:59.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.521+0000 7fa062000700 1 --2- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fa04c0384e0 0x7fa04c03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:31:59.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.521+0000 7fa062000700 1 -- 172.21.15.150:0/2294619609 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa05804c3e0 con 0x7fa064102040 2024-03-22T10:31:59.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.522+0000 7fa068a00700 1 --2- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fa04c0384e0 0x7fa04c03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:31:59.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.523+0000 7fa068a00700 1 --2- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fa04c0384e0 0x7fa04c03a990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa050006fd0 tx=0x7fa050006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:31:59.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.528+0000 7fa062000700 1 -- 172.21.15.150:0/2294619609 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa05802ba00 con 0x7fa064102040 2024-03-22T10:31:59.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.824+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa06402cc30 con 0x7fa064102040 2024-03-22T10:31:59.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.826+0000 7fa062000700 1 -- 172.21.15.150:0/2294619609 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fa058027020 con 0x7fa064102040 2024-03-22T10:31:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:31:59.819 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:31:59.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.828+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fa04c0384e0 msgr2=0x7fa04c03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:59.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.829+0000 7fa06be00700 1 --2- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fa04c0384e0 0x7fa04c03a990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa050006fd0 tx=0x7fa050006e40 comp rx=0 tx=0).stop 2024-03-22T10:31:59.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.829+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 msgr2=0x7fa0641971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:31:59.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.829+0000 7fa06be00700 1 --2- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 0x7fa0641971d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa058005b40 tx=0x7fa058004740 comp rx=0 tx=0).stop 2024-03-22T10:31:59.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.829+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 shutdown_connections 2024-03-22T10:31:59.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.829+0000 7fa06be00700 1 --2- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fa04c0384e0 0x7fa04c03a990 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:59.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.829+0000 7fa06be00700 1 --2- 172.21.15.150:0/2294619609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa064102040 0x7fa0641971d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:31:59.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.829+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 >> 172.21.15.150:0/2294619609 conn(0x7fa0640fd700 msgr2=0x7fa0640ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:31:59.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.829+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 shutdown_connections 2024-03-22T10:31:59.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:31:59.829+0000 7fa06be00700 1 -- 172.21.15.150:0/2294619609 wait complete. 2024-03-22T10:31:59.824 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:00.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:59 smithi115 ceph-mon[29111]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:00.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:31:59 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/2294619609' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:01.442 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:01.442 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:01.825 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:01.925 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:02.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:01 smithi115 ceph-mon[29111]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:03.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.867+0000 7f94c2200700 1 -- 172.21.15.150:0/448123853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 msgr2=0x7f94bc100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:03.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.867+0000 7f94c2200700 1 --2- 172.21.15.150:0/448123853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 0x7f94bc100250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94ac009910 tx=0x7f94ac009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:03.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.867+0000 7f94c2200700 1 -- 172.21.15.150:0/448123853 shutdown_connections 2024-03-22T10:32:03.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.867+0000 7f94c2200700 1 --2- 172.21.15.150:0/448123853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 0x7f94bc100250 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:03.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.867+0000 7f94c2200700 1 -- 172.21.15.150:0/448123853 >> 172.21.15.150:0/448123853 conn(0x7f94bc0fb3d0 msgr2=0x7f94bc0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:03.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.867+0000 7f94c2200700 1 -- 172.21.15.150:0/448123853 shutdown_connections 2024-03-22T10:32:03.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.867+0000 7f94c2200700 1 -- 172.21.15.150:0/448123853 wait complete. 2024-03-22T10:32:03.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.869+0000 7f94c2200700 1 Processor -- start 2024-03-22T10:32:03.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.869+0000 7f94c2200700 1 -- start start 2024-03-22T10:32:03.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.870+0000 7f94c2200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 0x7f94bc197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:03.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.870+0000 7f94c2200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1976d0 con 0x7f94bc0ffe40 2024-03-22T10:32:03.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.870+0000 7f94bb400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 0x7f94bc197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:03.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.870+0000 7f94bb400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 0x7f94bc197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:55416/0 (socket says 172.21.15.150:55416) 2024-03-22T10:32:03.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.870+0000 7f94bb400700 1 -- 172.21.15.150:0/1612481474 learned_addr learned my addr 172.21.15.150:0/1612481474 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:03.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.870+0000 7f94bb400700 1 -- 172.21.15.150:0/1612481474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0095f0 con 0x7f94bc0ffe40 2024-03-22T10:32:03.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.871+0000 7f94bb400700 1 --2- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 0x7f94bc197190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0092e0 tx=0x7f94ac009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:03.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.871+0000 7f94b3e00700 1 -- 172.21.15.150:0/1612481474 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94ac01d070 con 0x7f94bc0ffe40 2024-03-22T10:32:03.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.871+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc1978d0 con 0x7f94bc0ffe40 2024-03-22T10:32:03.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.871+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc197d70 con 0x7f94bc0ffe40 2024-03-22T10:32:03.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.871+0000 7f94b3e00700 1 -- 172.21.15.150:0/1612481474 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94ac00be10 con 0x7f94bc0ffe40 2024-03-22T10:32:03.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.871+0000 7f94b3e00700 1 -- 172.21.15.150:0/1612481474 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94ac00f460 con 0x7f94bc0ffe40 2024-03-22T10:32:03.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.872+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc190ea0 con 0x7f94bc0ffe40 2024-03-22T10:32:03.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.873+0000 7f94b3e00700 1 -- 172.21.15.150:0/1612481474 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f94ac022470 con 0x7f94bc0ffe40 2024-03-22T10:32:03.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.873+0000 7f94b3e00700 1 --2- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f94a00384e0 0x7f94a003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:03.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.873+0000 7f94b3e00700 1 -- 172.21.15.150:0/1612481474 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f94ac04c230 con 0x7f94bc0ffe40 2024-03-22T10:32:03.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.874+0000 7f94baa00700 1 --2- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f94a00384e0 0x7f94a003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:03.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.874+0000 7f94baa00700 1 --2- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f94a00384e0 0x7f94a003a990 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94a8006fd0 tx=0x7f94a8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:03.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:03.895+0000 7f94b3e00700 1 -- 172.21.15.150:0/1612481474 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f94ac022790 con 0x7f94bc0ffe40 2024-03-22T10:32:04.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.193+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f94bc1008d0 con 0x7f94bc0ffe40 2024-03-22T10:32:04.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.194+0000 7f94b3e00700 1 -- 172.21.15.150:0/1612481474 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f94bc1008d0 con 0x7f94bc0ffe40 2024-03-22T10:32:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:04.188 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.197+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f94a00384e0 msgr2=0x7f94a003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.197+0000 7f94c2200700 1 --2- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f94a00384e0 0x7f94a003a990 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94a8006fd0 tx=0x7f94a8006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.197+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 msgr2=0x7f94bc197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.197+0000 7f94c2200700 1 --2- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 0x7f94bc197190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0092e0 tx=0x7f94ac009e60 comp rx=0 tx=0).stop 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.197+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 shutdown_connections 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.197+0000 7f94c2200700 1 --2- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f94a00384e0 0x7f94a003a990 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.197+0000 7f94c2200700 1 --2- 172.21.15.150:0/1612481474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0ffe40 0x7f94bc197190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.197+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 >> 172.21.15.150:0/1612481474 conn(0x7f94bc0fb3d0 msgr2=0x7f94bc0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.198+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 shutdown_connections 2024-03-22T10:32:04.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:04.198+0000 7f94c2200700 1 -- 172.21.15.150:0/1612481474 wait complete. 2024-03-22T10:32:04.192 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:04.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:03 smithi115 ceph-mon[29111]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:05.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:04 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/1612481474' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:05.860 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:05.860 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:06.249 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:06.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:05 smithi115 ceph-mon[29111]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:06.343 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:08.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:07 smithi115 ceph-mon[29111]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.369+0000 7fddc3a00700 1 -- 172.21.15.150:0/144930614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 msgr2=0x7fddbc073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.369+0000 7fddc3a00700 1 --2- 172.21.15.150:0/144930614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 0x7fddbc073160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009910 tx=0x7fddb0009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.369+0000 7fddc3a00700 1 -- 172.21.15.150:0/144930614 shutdown_connections 2024-03-22T10:32:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.369+0000 7fddc3a00700 1 --2- 172.21.15.150:0/144930614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 0x7fddbc073160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.369+0000 7fddc3a00700 1 -- 172.21.15.150:0/144930614 >> 172.21.15.150:0/144930614 conn(0x7fddbc0fb3d0 msgr2=0x7fddbc0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.369+0000 7fddc3a00700 1 -- 172.21.15.150:0/144930614 shutdown_connections 2024-03-22T10:32:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.369+0000 7fddc3a00700 1 -- 172.21.15.150:0/144930614 wait complete. 2024-03-22T10:32:08.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.371+0000 7fddc3a00700 1 Processor -- start 2024-03-22T10:32:08.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.371+0000 7fddc3a00700 1 -- start start 2024-03-22T10:32:08.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.371+0000 7fddc3a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 0x7fddbc197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:08.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.371+0000 7fddc3a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc1976d0 con 0x7fddbc074d00 2024-03-22T10:32:08.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.372+0000 7fddc1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 0x7fddbc197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:08.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.372+0000 7fddc1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 0x7fddbc197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:55426/0 (socket says 172.21.15.150:55426) 2024-03-22T10:32:08.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.372+0000 7fddc1000700 1 -- 172.21.15.150:0/3669876647 learned_addr learned my addr 172.21.15.150:0/3669876647 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:08.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.372+0000 7fddc1000700 1 -- 172.21.15.150:0/3669876647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb00095f0 con 0x7fddbc074d00 2024-03-22T10:32:08.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.373+0000 7fddc1000700 1 --2- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 0x7fddbc197190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fddb00092e0 tx=0x7fddb0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:08.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.373+0000 7fddb9600700 1 -- 172.21.15.150:0/3669876647 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddb001d070 con 0x7fddbc074d00 2024-03-22T10:32:08.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.373+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc1978d0 con 0x7fddbc074d00 2024-03-22T10:32:08.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.373+0000 7fddb9600700 1 -- 172.21.15.150:0/3669876647 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fddb000be10 con 0x7fddbc074d00 2024-03-22T10:32:08.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.373+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc197d70 con 0x7fddbc074d00 2024-03-22T10:32:08.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.373+0000 7fddb9600700 1 -- 172.21.15.150:0/3669876647 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddb000f460 con 0x7fddbc074d00 2024-03-22T10:32:08.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.375+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddbc190d20 con 0x7fddbc074d00 2024-03-22T10:32:08.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.375+0000 7fddb9600700 1 -- 172.21.15.150:0/3669876647 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fddb0022470 con 0x7fddbc074d00 2024-03-22T10:32:08.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.375+0000 7fddb9600700 1 --2- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fdda4038490 0x7fdda403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:08.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.375+0000 7fddb9600700 1 -- 172.21.15.150:0/3669876647 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fddb004c370 con 0x7fddbc074d00 2024-03-22T10:32:08.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.376+0000 7fddbbe00700 1 --2- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fdda4038490 0x7fdda403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:08.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.377+0000 7fddbbe00700 1 --2- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fdda4038490 0x7fdda403a940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdda8006fd0 tx=0x7fdda8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:08.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.390+0000 7fddb9600700 1 -- 172.21.15.150:0/3669876647 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fddb000b960 con 0x7fddbc074d00 2024-03-22T10:32:08.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.688+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fddbc02cc30 con 0x7fddbc074d00 2024-03-22T10:32:08.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.689+0000 7fddb9600700 1 -- 172.21.15.150:0/3669876647 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fddb0027070 con 0x7fddbc074d00 2024-03-22T10:32:08.683 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:08.683 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:08.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fdda4038490 msgr2=0x7fdda403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 --2- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fdda4038490 0x7fdda403a940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdda8006fd0 tx=0x7fdda8006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 msgr2=0x7fddbc197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 --2- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 0x7fddbc197190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fddb00092e0 tx=0x7fddb0009e60 comp rx=0 tx=0).stop 2024-03-22T10:32:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 shutdown_connections 2024-03-22T10:32:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 --2- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7fdda4038490 0x7fdda403a940 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 --2- 172.21.15.150:0/3669876647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddbc074d00 0x7fddbc197190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 >> 172.21.15.150:0/3669876647 conn(0x7fddbc0fb3d0 msgr2=0x7fddbc0fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 shutdown_connections 2024-03-22T10:32:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:08.692+0000 7fddc3a00700 1 -- 172.21.15.150:0/3669876647 wait complete. 2024-03-22T10:32:08.687 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:09.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:08 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/3669876647' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:10.149 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:09 smithi115 ceph-mon[29111]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:10.368 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:10.368 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:10.749 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:10.855 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:12.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:11 smithi115 ceph-mon[29111]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.756+0000 7f3efcc00700 1 -- 172.21.15.150:0/4120643033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 msgr2=0x7f3ef8100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.756+0000 7f3efcc00700 1 --2- 172.21.15.150:0/4120643033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 0x7f3ef8100250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009910 tx=0x7f3ee8009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.756+0000 7f3efcc00700 1 -- 172.21.15.150:0/4120643033 shutdown_connections 2024-03-22T10:32:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.756+0000 7f3efcc00700 1 --2- 172.21.15.150:0/4120643033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 0x7f3ef8100250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.756+0000 7f3efcc00700 1 -- 172.21.15.150:0/4120643033 >> 172.21.15.150:0/4120643033 conn(0x7f3ef80fb3d0 msgr2=0x7f3ef80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.756+0000 7f3efcc00700 1 -- 172.21.15.150:0/4120643033 shutdown_connections 2024-03-22T10:32:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.756+0000 7f3efcc00700 1 -- 172.21.15.150:0/4120643033 wait complete. 2024-03-22T10:32:12.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.757+0000 7f3efcc00700 1 Processor -- start 2024-03-22T10:32:12.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.757+0000 7f3efcc00700 1 -- start start 2024-03-22T10:32:12.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.758+0000 7f3efcc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 0x7f3ef8194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:12.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.758+0000 7f3efcc00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef81954d0 con 0x7f3ef80ffe40 2024-03-22T10:32:12.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.758+0000 7f3ef5e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 0x7f3ef8194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:12.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.758+0000 7f3ef5e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 0x7f3ef8194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:40364/0 (socket says 172.21.15.150:40364) 2024-03-22T10:32:12.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.758+0000 7f3ef5e00700 1 -- 172.21.15.150:0/1145510884 learned_addr learned my addr 172.21.15.150:0/1145510884 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:12.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.759+0000 7f3ef5e00700 1 -- 172.21.15.150:0/1145510884 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80095f0 con 0x7f3ef80ffe40 2024-03-22T10:32:12.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.759+0000 7f3ef5e00700 1 --2- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 0x7f3ef8194f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80092e0 tx=0x7f3ee8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:12.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.760+0000 7f3eeea00700 1 -- 172.21.15.150:0/1145510884 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee801d070 con 0x7f3ef80ffe40 2024-03-22T10:32:12.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.760+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef81956d0 con 0x7f3ef80ffe40 2024-03-22T10:32:12.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.760+0000 7f3eeea00700 1 -- 172.21.15.150:0/1145510884 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3ee800be10 con 0x7f3ef80ffe40 2024-03-22T10:32:12.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.760+0000 7f3eeea00700 1 -- 172.21.15.150:0/1145510884 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee800f460 con 0x7f3ef80ffe40 2024-03-22T10:32:12.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.760+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef8195b70 con 0x7f3ef80ffe40 2024-03-22T10:32:12.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.761+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef818eaa0 con 0x7f3ef80ffe40 2024-03-22T10:32:12.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.761+0000 7f3eeea00700 1 -- 172.21.15.150:0/1145510884 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f3ee8022470 con 0x7f3ef80ffe40 2024-03-22T10:32:12.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.762+0000 7f3eeea00700 1 --2- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3edc0384e0 0x7f3edc03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:12.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.762+0000 7f3eeea00700 1 -- 172.21.15.150:0/1145510884 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3ee804c2a0 con 0x7f3ef80ffe40 2024-03-22T10:32:12.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.762+0000 7f3ef5400700 1 --2- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3edc0384e0 0x7f3edc03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:12.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.763+0000 7f3ef5400700 1 --2- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3edc0384e0 0x7f3edc03a990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4006fd0 tx=0x7f3ee4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:12.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:12.776+0000 7f3eeea00700 1 -- 172.21.15.150:0/1145510884 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3ee800b960 con 0x7f3ef80ffe40 2024-03-22T10:32:13.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.071+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3ef81008d0 con 0x7f3ef80ffe40 2024-03-22T10:32:13.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.072+0000 7f3eeea00700 1 -- 172.21.15.150:0/1145510884 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f3ef81008d0 con 0x7f3ef80ffe40 2024-03-22T10:32:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:13.066 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:13.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3edc0384e0 msgr2=0x7f3edc03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:13.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 --2- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3edc0384e0 0x7f3edc03a990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4006fd0 tx=0x7f3ee4006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:13.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 msgr2=0x7f3ef8194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:13.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 --2- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 0x7f3ef8194f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80092e0 tx=0x7f3ee8009e60 comp rx=0 tx=0).stop 2024-03-22T10:32:13.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 shutdown_connections 2024-03-22T10:32:13.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 --2- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f3edc0384e0 0x7f3edc03a990 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:13.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 --2- 172.21.15.150:0/1145510884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ef80ffe40 0x7f3ef8194f90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:13.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 >> 172.21.15.150:0/1145510884 conn(0x7f3ef80fb3d0 msgr2=0x7f3ef80fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:13.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 shutdown_connections 2024-03-22T10:32:13.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:13.075+0000 7f3efcc00700 1 -- 172.21.15.150:0/1145510884 wait complete. 2024-03-22T10:32:13.070 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:14.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:13 smithi115 ceph-mon[29111]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:14.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:13 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/1145510884' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:14.732 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:14.732 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:15.205 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:16.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:15 smithi115 ceph-mon[29111]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:16.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:15 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:16.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:15 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:16.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:15 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:16.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:15 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-22T10:32:17.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.106+0000 7f2bff200700 1 -- 172.21.15.150:0/2591130045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 msgr2=0x7f2bf80731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:17.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.106+0000 7f2bff200700 1 --2- 172.21.15.150:0/2591130045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 0x7f2bf80731e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009910 tx=0x7f2be8009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:17.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.106+0000 7f2bff200700 1 -- 172.21.15.150:0/2591130045 shutdown_connections 2024-03-22T10:32:17.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.106+0000 7f2bff200700 1 --2- 172.21.15.150:0/2591130045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 0x7f2bf80731e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:17.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.106+0000 7f2bff200700 1 -- 172.21.15.150:0/2591130045 >> 172.21.15.150:0/2591130045 conn(0x7f2bf80fb3d0 msgr2=0x7f2bf80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:17.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.106+0000 7f2bff200700 1 -- 172.21.15.150:0/2591130045 shutdown_connections 2024-03-22T10:32:17.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.106+0000 7f2bff200700 1 -- 172.21.15.150:0/2591130045 wait complete. 2024-03-22T10:32:17.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.108+0000 7f2bff200700 1 Processor -- start 2024-03-22T10:32:17.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.108+0000 7f2bff200700 1 -- start start 2024-03-22T10:32:17.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.108+0000 7f2bff200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 0x7f2bf8197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:17.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.108+0000 7f2bff200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf81976b0 con 0x7f2bf8074d80 2024-03-22T10:32:17.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.109+0000 7f2bf7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 0x7f2bf8197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:17.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.109+0000 7f2bf7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 0x7f2bf8197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:40380/0 (socket says 172.21.15.150:40380) 2024-03-22T10:32:17.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.109+0000 7f2bf7e00700 1 -- 172.21.15.150:0/503885531 learned_addr learned my addr 172.21.15.150:0/503885531 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:17.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.110+0000 7f2bf7e00700 1 -- 172.21.15.150:0/503885531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80095f0 con 0x7f2bf8074d80 2024-03-22T10:32:17.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.110+0000 7f2bf7e00700 1 --2- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 0x7f2bf8197170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2be8005b40 tx=0x7f2be8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:17.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.110+0000 7f2bf5600700 1 -- 172.21.15.150:0/503885531 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2be801d070 con 0x7f2bf8074d80 2024-03-22T10:32:17.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.110+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf81978b0 con 0x7f2bf8074d80 2024-03-22T10:32:17.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.110+0000 7f2bf5600700 1 -- 172.21.15.150:0/503885531 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2be800be10 con 0x7f2bf8074d80 2024-03-22T10:32:17.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.110+0000 7f2bf5600700 1 -- 172.21.15.150:0/503885531 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2be800f460 con 0x7f2bf8074d80 2024-03-22T10:32:17.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.111+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf8197d50 con 0x7f2bf8074d80 2024-03-22T10:32:17.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.112+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf8190d20 con 0x7f2bf8074d80 2024-03-22T10:32:17.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.112+0000 7f2bf5600700 1 -- 172.21.15.150:0/503885531 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 45155+0+0 (secure 0 0 0) 0x7f2be80227e0 con 0x7f2bf8074d80 2024-03-22T10:32:17.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.112+0000 7f2bf5600700 1 --2- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f2bdc038530 0x7f2bdc03a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:17.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.113+0000 7f2bf5600700 1 -- 172.21.15.150:0/503885531 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2be804c410 con 0x7f2bf8074d80 2024-03-22T10:32:17.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.113+0000 7f2befe00700 1 -- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f2bdc038530 msgr2=0x7f2bdc03a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2729066911 2024-03-22T10:32:17.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.113+0000 7f2befe00700 1 --2- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f2bdc038530 0x7f2bdc03a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:32:17.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.127+0000 7f2bf5600700 1 -- 172.21.15.150:0/503885531 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2be802ba00 con 0x7f2bf8074d80 2024-03-22T10:32:17.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.314+0000 7f2befe00700 1 -- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f2bdc038530 msgr2=0x7f2bdc03a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2729066911 2024-03-22T10:32:17.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.314+0000 7f2befe00700 1 --2- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f2bdc038530 0x7f2bdc03a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T10:32:17.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.425+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2bf802cc30 con 0x7f2bf8074d80 2024-03-22T10:32:17.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.427+0000 7f2bf5600700 1 -- 172.21.15.150:0/503885531 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f2be8027070 con 0x7f2bf8074d80 2024-03-22T10:32:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:17.421 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:17.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.429+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f2bdc038530 msgr2=0x7f2bdc03a9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:32:17.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.429+0000 7f2bff200700 1 --2- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f2bdc038530 0x7f2bdc03a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:17.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.429+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 msgr2=0x7f2bf8197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:17.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.429+0000 7f2bff200700 1 --2- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 0x7f2bf8197170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2be8005b40 tx=0x7f2be8004740 comp rx=0 tx=0).stop 2024-03-22T10:32:17.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.429+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 shutdown_connections 2024-03-22T10:32:17.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.429+0000 7f2bff200700 1 --2- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f2bdc038530 0x7f2bdc03a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:17.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.429+0000 7f2bff200700 1 --2- 172.21.15.150:0/503885531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bf8074d80 0x7f2bf8197170 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:17.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.430+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 >> 172.21.15.150:0/503885531 conn(0x7f2bf80fb3d0 msgr2=0x7f2bf80fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:17.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.430+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 shutdown_connections 2024-03-22T10:32:17.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:17.430+0000 7f2bff200700 1 -- 172.21.15.150:0/503885531 wait complete. 2024-03-22T10:32:17.424 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:17.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:17 smithi115 ceph-mon[29111]: from='mgr.14164 172.21.15.115:0/3733575198' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-22T10:32:17.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:17 smithi115 ceph-mon[29111]: mgrmap e14: smithi115.iqzwsz(active, since 78s) 2024-03-22T10:32:17.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:17 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/503885531' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:19.122 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:19.122 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:19.512 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:19.609 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:21.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.586+0000 7f912f200700 1 -- 172.21.15.150:0/1064445846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 msgr2=0x7f9130102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:21.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.586+0000 7f912f200700 1 --2- 172.21.15.150:0/1064445846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 0x7f9130102ae0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9120009910 tx=0x7f9120009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:21.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.587+0000 7f912f200700 1 -- 172.21.15.150:0/1064445846 shutdown_connections 2024-03-22T10:32:21.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.587+0000 7f912f200700 1 --2- 172.21.15.150:0/1064445846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 0x7f9130102ae0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:21.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.587+0000 7f912f200700 1 -- 172.21.15.150:0/1064445846 >> 172.21.15.150:0/1064445846 conn(0x7f91300fa140 msgr2=0x7f91300fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:21.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.587+0000 7f912f200700 1 -- 172.21.15.150:0/1064445846 shutdown_connections 2024-03-22T10:32:21.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.587+0000 7f912f200700 1 -- 172.21.15.150:0/1064445846 wait complete. 2024-03-22T10:32:21.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.588+0000 7f912f200700 1 Processor -- start 2024-03-22T10:32:21.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.588+0000 7f912f200700 1 -- start start 2024-03-22T10:32:21.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.589+0000 7f912f200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 0x7f9130194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:21.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.589+0000 7f912f200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130195390 con 0x7f9130100700 2024-03-22T10:32:21.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.589+0000 7f912de00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 0x7f9130194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:21.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.589+0000 7f912de00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 0x7f9130194e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:53326/0 (socket says 172.21.15.150:53326) 2024-03-22T10:32:21.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.589+0000 7f912de00700 1 -- 172.21.15.150:0/3967519449 learned_addr learned my addr 172.21.15.150:0/3967519449 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:21.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.590+0000 7f912de00700 1 -- 172.21.15.150:0/3967519449 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91200095f0 con 0x7f9130100700 2024-03-22T10:32:21.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.590+0000 7f912de00700 1 --2- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 0x7f9130194e50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91200092e0 tx=0x7f9120004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:21.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.591+0000 7f9126a00700 1 -- 172.21.15.150:0/3967519449 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f912001d070 con 0x7f9130100700 2024-03-22T10:32:21.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.591+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9130195590 con 0x7f9130100700 2024-03-22T10:32:21.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.591+0000 7f9126a00700 1 -- 172.21.15.150:0/3967519449 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9120005000 con 0x7f9130100700 2024-03-22T10:32:21.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.591+0000 7f9126a00700 1 -- 172.21.15.150:0/3967519449 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9120022e30 con 0x7f9130100700 2024-03-22T10:32:21.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.591+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9130195a30 con 0x7f9130100700 2024-03-22T10:32:21.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.592+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913018e950 con 0x7f9130100700 2024-03-22T10:32:21.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.592+0000 7f9126a00700 1 -- 172.21.15.150:0/3967519449 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 45155+0+0 (secure 0 0 0) 0x7f9120022470 con 0x7f9130100700 2024-03-22T10:32:21.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.593+0000 7f9126a00700 1 --2- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9114038490 0x7f911403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:21.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.593+0000 7f9126a00700 1 -- 172.21.15.150:0/3967519449 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f912004c420 con 0x7f9130100700 2024-03-22T10:32:21.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.593+0000 7f912d400700 1 -- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9114038490 msgr2=0x7f911403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2729066911 2024-03-22T10:32:21.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.593+0000 7f912d400700 1 --2- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9114038490 0x7f911403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:32:21.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.607+0000 7f9126a00700 1 -- 172.21.15.150:0/3967519449 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f912002ba00 con 0x7f9130100700 2024-03-22T10:32:21.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.794+0000 7f912d400700 1 -- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9114038490 msgr2=0x7f911403a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2729066911 2024-03-22T10:32:21.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.794+0000 7f912d400700 1 --2- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9114038490 0x7f911403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T10:32:21.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.906+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f913002cc30 con 0x7f9130100700 2024-03-22T10:32:21.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.907+0000 7f9126a00700 1 -- 172.21.15.150:0/3967519449 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f9120027070 con 0x7f9130100700 2024-03-22T10:32:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:21.901 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:21.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.910+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9114038490 msgr2=0x7f911403a940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:32:21.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.910+0000 7f912f200700 1 --2- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9114038490 0x7f911403a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:21.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.910+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 msgr2=0x7f9130194e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:21.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.910+0000 7f912f200700 1 --2- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 0x7f9130194e50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91200092e0 tx=0x7f9120004690 comp rx=0 tx=0).stop 2024-03-22T10:32:21.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.910+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 shutdown_connections 2024-03-22T10:32:21.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.910+0000 7f912f200700 1 --2- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:6800/2729066911,v1:172.21.15.115:6801/2729066911] conn(0x7f9114038490 0x7f911403a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:21.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.910+0000 7f912f200700 1 --2- 172.21.15.150:0/3967519449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9130100700 0x7f9130194e50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:21.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.910+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 >> 172.21.15.150:0/3967519449 conn(0x7f91300fa140 msgr2=0x7f91300fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:21.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.911+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 shutdown_connections 2024-03-22T10:32:21.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:21.911+0000 7f912f200700 1 -- 172.21.15.150:0/3967519449 wait complete. 2024-03-22T10:32:21.905 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:22.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:21 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/3967519449' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:23.616 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:23.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:23.990 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:24.086 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:25.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: Active manager daemon smithi115.iqzwsz restarted 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: Activating manager daemon smithi115.iqzwsz 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: mgrmap e15: smithi115.iqzwsz(active, starting, since 0.00835483s) 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: Manager daemon smithi115.iqzwsz is now available 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:32:25.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:32:25.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.989+0000 7fcfca200700 1 -- 172.21.15.150:0/4199953617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 msgr2=0x7fcfc4100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:25.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.989+0000 7fcfca200700 1 --2- 172.21.15.150:0/4199953617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 0x7fcfc4100250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009910 tx=0x7fcfb4009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:25.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.989+0000 7fcfca200700 1 -- 172.21.15.150:0/4199953617 shutdown_connections 2024-03-22T10:32:25.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.989+0000 7fcfca200700 1 --2- 172.21.15.150:0/4199953617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 0x7fcfc4100250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:25.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.989+0000 7fcfca200700 1 -- 172.21.15.150:0/4199953617 >> 172.21.15.150:0/4199953617 conn(0x7fcfc40fb3d0 msgr2=0x7fcfc40fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:25.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.989+0000 7fcfca200700 1 -- 172.21.15.150:0/4199953617 shutdown_connections 2024-03-22T10:32:25.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.989+0000 7fcfca200700 1 -- 172.21.15.150:0/4199953617 wait complete. 2024-03-22T10:32:25.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.991+0000 7fcfca200700 1 Processor -- start 2024-03-22T10:32:25.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.991+0000 7fcfca200700 1 -- start start 2024-03-22T10:32:25.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.992+0000 7fcfca200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 0x7fcfc4194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:25.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.992+0000 7fcfca200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc41954d0 con 0x7fcfc40ffe40 2024-03-22T10:32:25.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.992+0000 7fcfc3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 0x7fcfc4194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:25.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.992+0000 7fcfc3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 0x7fcfc4194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:53344/0 (socket says 172.21.15.150:53344) 2024-03-22T10:32:25.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.992+0000 7fcfc3400700 1 -- 172.21.15.150:0/1711177847 learned_addr learned my addr 172.21.15.150:0/1711177847 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:25.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.993+0000 7fcfc3400700 1 -- 172.21.15.150:0/1711177847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40095f0 con 0x7fcfc40ffe40 2024-03-22T10:32:25.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.993+0000 7fcfc3400700 1 --2- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 0x7fcfc4194f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40092e0 tx=0x7fcfb4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:25.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.993+0000 7fcfbbe00700 1 -- 172.21.15.150:0/1711177847 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcfb401d070 con 0x7fcfc40ffe40 2024-03-22T10:32:25.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.993+0000 7fcfbbe00700 1 -- 172.21.15.150:0/1711177847 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcfb400be10 con 0x7fcfc40ffe40 2024-03-22T10:32:25.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.993+0000 7fcfbbe00700 1 -- 172.21.15.150:0/1711177847 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcfb400f460 con 0x7fcfc40ffe40 2024-03-22T10:32:25.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.993+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc41956d0 con 0x7fcfc40ffe40 2024-03-22T10:32:25.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.994+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc4195b70 con 0x7fcfc40ffe40 2024-03-22T10:32:25.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.995+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc418eaa0 con 0x7fcfc40ffe40 2024-03-22T10:32:25.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.995+0000 7fcfbbe00700 1 -- 172.21.15.150:0/1711177847 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 44878+0+0 (secure 0 0 0) 0x7fcfb4022470 con 0x7fcfc40ffe40 2024-03-22T10:32:25.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:25.996+0000 7fcfbbe00700 1 -- 172.21.15.150:0/1711177847 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fcfb404c010 con 0x7fcfc40ffe40 2024-03-22T10:32:25.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.001+0000 7fcfbbe00700 1 -- 172.21.15.150:0/1711177847 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcfb400b960 con 0x7fcfc40ffe40 2024-03-22T10:32:26.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.182+0000 7fcfbbe00700 1 -- 172.21.15.150:0/1711177847 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 16) v1 ==== 45005+0+0 (secure 0 0 0) 0x7fcfb4022790 con 0x7fcfc40ffe40 2024-03-22T10:32:26.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.182+0000 7fcfbbe00700 1 --2- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcfa8039530 0x7fcfa803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:26.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.182+0000 7fcfc2a00700 1 --2- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcfa8039530 0x7fcfa803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:26.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.184+0000 7fcfc2a00700 1 --2- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcfa8039530 0x7fcfa803b910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006fd0 tx=0x7fcfb0006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:26.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.303+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcfc41008d0 con 0x7fcfc40ffe40 2024-03-22T10:32:26.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.304+0000 7fcfbbe00700 1 -- 172.21.15.150:0/1711177847 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fcfc41008d0 con 0x7fcfc40ffe40 2024-03-22T10:32:26.297 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:26.298 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:26.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.306+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcfa8039530 msgr2=0x7fcfa803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:26.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.306+0000 7fcfca200700 1 --2- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcfa8039530 0x7fcfa803b910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006fd0 tx=0x7fcfb0006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:26.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.306+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 msgr2=0x7fcfc4194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:26.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.306+0000 7fcfca200700 1 --2- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 0x7fcfc4194f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40092e0 tx=0x7fcfb4009e60 comp rx=0 tx=0).stop 2024-03-22T10:32:26.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.307+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 shutdown_connections 2024-03-22T10:32:26.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.307+0000 7fcfca200700 1 --2- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcfa8039530 0x7fcfa803b910 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:26.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.307+0000 7fcfca200700 1 --2- 172.21.15.150:0/1711177847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfc40ffe40 0x7fcfc4194f90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:26.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.307+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 >> 172.21.15.150:0/1711177847 conn(0x7fcfc40fb3d0 msgr2=0x7fcfc40fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:26.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.307+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 shutdown_connections 2024-03-22T10:32:26.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:26.307+0000 7fcfca200700 1 -- 172.21.15.150:0/1711177847 wait complete. 2024-03-22T10:32:26.301 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:26.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:26.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:32:26.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:32:26.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/trash_purge_schedule"}]: dispatch 2024-03-22T10:32:26.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:26 smithi115 ceph-mon[29111]: [22/Mar/2024:10:32:25] ENGINE Bus STARTING 2024-03-22T10:32:26.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:26.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:26 smithi115 ceph-mon[29111]: mgrmap e16: smithi115.iqzwsz(active, since 1.0094s) 2024-03-22T10:32:27.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:27 smithi115 ceph-mon[29111]: [22/Mar/2024:10:32:25] ENGINE Serving on http://172.21.15.115:8765 2024-03-22T10:32:27.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:27 smithi115 ceph-mon[29111]: [22/Mar/2024:10:32:25] ENGINE Serving on https://172.21.15.115:7150 2024-03-22T10:32:27.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:27 smithi115 ceph-mon[29111]: [22/Mar/2024:10:32:25] ENGINE Bus STARTED 2024-03-22T10:32:27.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:27 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/1711177847' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:27.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:28.053 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:28.054 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:28.409 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:28.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:28 smithi115 ceph-mon[29111]: mgrmap e17: smithi115.iqzwsz(active, since 2s) 2024-03-22T10:32:28.517 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:30.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:31.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.395+0000 7f023f200700 1 -- 172.21.15.150:0/609778307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 msgr2=0x7f0240102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:31.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.395+0000 7f023f200700 1 --2- 172.21.15.150:0/609778307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 0x7f0240102280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f023000b0a0 tx=0x7f023000b3b0 comp rx=0 tx=0).stop 2024-03-22T10:32:31.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.395+0000 7f023f200700 1 -- 172.21.15.150:0/609778307 shutdown_connections 2024-03-22T10:32:31.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.395+0000 7f023f200700 1 --2- 172.21.15.150:0/609778307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 0x7f0240102280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:31.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.395+0000 7f023f200700 1 -- 172.21.15.150:0/609778307 >> 172.21.15.150:0/609778307 conn(0x7f02400fd530 msgr2=0x7f02400ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:31.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.395+0000 7f023f200700 1 -- 172.21.15.150:0/609778307 shutdown_connections 2024-03-22T10:32:31.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.395+0000 7f023f200700 1 -- 172.21.15.150:0/609778307 wait complete. 2024-03-22T10:32:31.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.397+0000 7f023f200700 1 Processor -- start 2024-03-22T10:32:31.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.397+0000 7f023f200700 1 -- start start 2024-03-22T10:32:31.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.398+0000 7f023f200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 0x7f0240196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:31.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.398+0000 7f023f200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0240197520 con 0x7f0240101e70 2024-03-22T10:32:31.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.398+0000 7f023de00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 0x7f0240196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.398+0000 7f023de00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 0x7f0240196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:58428/0 (socket says 172.21.15.150:58428) 2024-03-22T10:32:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.398+0000 7f023de00700 1 -- 172.21.15.150:0/3472623315 learned_addr learned my addr 172.21.15.150:0/3472623315 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.399+0000 7f023de00700 1 -- 172.21.15.150:0/3472623315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230009d00 con 0x7f0240101e70 2024-03-22T10:32:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.399+0000 7f023de00700 1 --2- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 0x7f0240196fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0230009fd0 tx=0x7f023000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.399+0000 7f0237400700 1 -- 172.21.15.150:0/3472623315 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f023000e040 con 0x7f0240101e70 2024-03-22T10:32:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.399+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0240197720 con 0x7f0240101e70 2024-03-22T10:32:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.399+0000 7f0237400700 1 -- 172.21.15.150:0/3472623315 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f023000bdc0 con 0x7f0240101e70 2024-03-22T10:32:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.399+0000 7f0237400700 1 -- 172.21.15.150:0/3472623315 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f023001da40 con 0x7f0240101e70 2024-03-22T10:32:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.399+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0240197b40 con 0x7f0240101e70 2024-03-22T10:32:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.400+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0240190ad0 con 0x7f0240101e70 2024-03-22T10:32:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.401+0000 7f0237400700 1 -- 172.21.15.150:0/3472623315 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f023001d480 con 0x7f0240101e70 2024-03-22T10:32:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.401+0000 7f0237400700 1 --2- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f02240385d0 0x7f022403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:31.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.401+0000 7f0237400700 1 -- 172.21.15.150:0/3472623315 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f023004c950 con 0x7f0240101e70 2024-03-22T10:32:31.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.402+0000 7f0235400700 1 --2- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f02240385d0 0x7f022403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:31.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.402+0000 7f0235400700 1 --2- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f02240385d0 0x7f022403aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0228006fd0 tx=0x7f0228006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:31.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.417+0000 7f0237400700 1 -- 172.21.15.150:0/3472623315 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f023001c3e0 con 0x7f0240101e70 2024-03-22T10:32:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.717+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0240062380 con 0x7f0240101e70 2024-03-22T10:32:31.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.718+0000 7f0237400700 1 -- 172.21.15.150:0/3472623315 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f023001c3e0 con 0x7f0240101e70 2024-03-22T10:32:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:31.713 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:31.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f02240385d0 msgr2=0x7f022403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:31.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 --2- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f02240385d0 0x7f022403aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0228006fd0 tx=0x7f0228006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:31.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 msgr2=0x7f0240196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:31.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 --2- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 0x7f0240196fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0230009fd0 tx=0x7f023000bb10 comp rx=0 tx=0).stop 2024-03-22T10:32:31.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 shutdown_connections 2024-03-22T10:32:31.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 --2- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f02240385d0 0x7f022403aa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:31.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 --2- 172.21.15.150:0/3472623315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0240101e70 0x7f0240196fe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:31.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 >> 172.21.15.150:0/3472623315 conn(0x7f02400fd530 msgr2=0x7f02400ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:31.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 shutdown_connections 2024-03-22T10:32:31.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:31.721+0000 7f023f200700 1 -- 172.21.15.150:0/3472623315 wait complete. 2024-03-22T10:32:31.717 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:32.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:31 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:32.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:31 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:32.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:31 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/3472623315' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:33.533 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:33.534 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:33.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:33 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:33.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:33 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:33.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:33 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:32:34.366 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:34.472 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T10:32:35.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:35.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:35.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:35.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:35.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:32:35.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:32:35.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:32:35.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:35 smithi115 ceph-mon[29111]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-22T10:32:35.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:35 smithi115 ceph-mon[29111]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-22T10:32:36.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.528+0000 7ff5c9000700 1 -- 172.21.15.150:0/662095923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 msgr2=0x7ff5c4102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:36.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.528+0000 7ff5c9000700 1 --2- 172.21.15.150:0/662095923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 0x7ff5c4102ca0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009910 tx=0x7ff5b4009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:36.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.528+0000 7ff5c9000700 1 -- 172.21.15.150:0/662095923 shutdown_connections 2024-03-22T10:32:36.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.528+0000 7ff5c9000700 1 --2- 172.21.15.150:0/662095923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 0x7ff5c4102ca0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:36.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.528+0000 7ff5c9000700 1 -- 172.21.15.150:0/662095923 >> 172.21.15.150:0/662095923 conn(0x7ff5c40fa2e0 msgr2=0x7ff5c40fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:36.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.528+0000 7ff5c9000700 1 -- 172.21.15.150:0/662095923 shutdown_connections 2024-03-22T10:32:36.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.528+0000 7ff5c9000700 1 -- 172.21.15.150:0/662095923 wait complete. 2024-03-22T10:32:36.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.529+0000 7ff5c9000700 1 Processor -- start 2024-03-22T10:32:36.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.529+0000 7ff5c9000700 1 -- start start 2024-03-22T10:32:36.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.530+0000 7ff5c9000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 0x7ff5c4197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:36.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.530+0000 7ff5c9000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c4197670 con 0x7ff5c41008c0 2024-03-22T10:32:36.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.530+0000 7ff5c1e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 0x7ff5c4197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:36.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.530+0000 7ff5c1e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 0x7ff5c4197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:58442/0 (socket says 172.21.15.150:58442) 2024-03-22T10:32:36.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.530+0000 7ff5c1e00700 1 -- 172.21.15.150:0/3530373788 learned_addr learned my addr 172.21.15.150:0/3530373788 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:36.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.531+0000 7ff5c1e00700 1 -- 172.21.15.150:0/3530373788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b40095f0 con 0x7ff5c41008c0 2024-03-22T10:32:36.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.531+0000 7ff5c1e00700 1 --2- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 0x7ff5c4197130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005b40 tx=0x7ff5b4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:36.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.532+0000 7ff5baa00700 1 -- 172.21.15.150:0/3530373788 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5b401d070 con 0x7ff5c41008c0 2024-03-22T10:32:36.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.532+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c4197870 con 0x7ff5c41008c0 2024-03-22T10:32:36.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.532+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c4197d10 con 0x7ff5c41008c0 2024-03-22T10:32:36.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.532+0000 7ff5baa00700 1 -- 172.21.15.150:0/3530373788 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff5b4005c00 con 0x7ff5c41008c0 2024-03-22T10:32:36.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.532+0000 7ff5baa00700 1 -- 172.21.15.150:0/3530373788 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5b400fc30 con 0x7ff5c41008c0 2024-03-22T10:32:36.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.533+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a4005320 con 0x7ff5c41008c0 2024-03-22T10:32:36.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.534+0000 7ff5baa00700 1 -- 172.21.15.150:0/3530373788 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7ff5b400f5f0 con 0x7ff5c41008c0 2024-03-22T10:32:36.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.534+0000 7ff5baa00700 1 --2- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff5a8038580 0x7ff5a803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:36.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.534+0000 7ff5baa00700 1 -- 172.21.15.150:0/3530373788 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff5b404c520 con 0x7ff5c41008c0 2024-03-22T10:32:36.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.535+0000 7ff5c1400700 1 --2- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff5a8038580 0x7ff5a803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:36.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.536+0000 7ff5c1400700 1 --2- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff5a8038580 0x7ff5a803aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006fd0 tx=0x7ff5b0006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:36.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.550+0000 7ff5baa00700 1 -- 172.21.15.150:0/3530373788 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff5b400b960 con 0x7ff5c41008c0 2024-03-22T10:32:36.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:36 smithi115 ceph-mon[29111]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:32:36.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:36 smithi115 ceph-mon[29111]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:32:36.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.849+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff5a4005190 con 0x7ff5c41008c0 2024-03-22T10:32:36.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.849+0000 7ff5baa00700 1 -- 172.21.15.150:0/3530373788 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7ff5b4027020 con 0x7ff5c41008c0 2024-03-22T10:32:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:36.844 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff5a8038580 msgr2=0x7ff5a803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 --2- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff5a8038580 0x7ff5a803aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006fd0 tx=0x7ff5b0006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 msgr2=0x7ff5c4197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 --2- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 0x7ff5c4197130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005b40 tx=0x7ff5b4004740 comp rx=0 tx=0).stop 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 shutdown_connections 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 --2- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff5a8038580 0x7ff5a803aa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 --2- 172.21.15.150:0/3530373788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c41008c0 0x7ff5c4197130 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 >> 172.21.15.150:0/3530373788 conn(0x7ff5c40fa2e0 msgr2=0x7ff5c40fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 shutdown_connections 2024-03-22T10:32:36.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:36.852+0000 7ff5c9000700 1 -- 172.21.15.150:0/3530373788 wait complete. 2024-03-22T10:32:36.848 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:37.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:32:37.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:32:37.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:32:37.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:32:37.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/3530373788' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:37.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:37.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:37.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:37.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:37.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:37.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:32:37.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-22T10:32:37.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:37 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:32:38.606 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:38.606 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:38.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:38 smithi115 ceph-mon[29111]: Deploying daemon ceph-exporter.smithi150 on smithi150 2024-03-22T10:32:39.347 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:32:40.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:32:41.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.689+0000 7f9c5a200700 1 -- 172.21.15.150:0/1187962403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 msgr2=0x7f9c540734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:41.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.689+0000 7f9c5a200700 1 --2- 172.21.15.150:0/1187962403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 0x7f9c540734c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009910 tx=0x7f9c44009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:41.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.689+0000 7f9c5a200700 1 -- 172.21.15.150:0/1187962403 shutdown_connections 2024-03-22T10:32:41.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.689+0000 7f9c5a200700 1 --2- 172.21.15.150:0/1187962403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 0x7f9c540734c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:41.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.689+0000 7f9c5a200700 1 -- 172.21.15.150:0/1187962403 >> 172.21.15.150:0/1187962403 conn(0x7f9c540fb990 msgr2=0x7f9c540fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:41.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.689+0000 7f9c5a200700 1 -- 172.21.15.150:0/1187962403 shutdown_connections 2024-03-22T10:32:41.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.690+0000 7f9c5a200700 1 -- 172.21.15.150:0/1187962403 wait complete. 2024-03-22T10:32:41.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.690+0000 7f9c5a200700 1 Processor -- start 2024-03-22T10:32:41.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.691+0000 7f9c5a200700 1 -- start start 2024-03-22T10:32:41.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.691+0000 7f9c5a200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 0x7f9c541974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:41.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.691+0000 7f9c5a200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54197aa0 con 0x7f9c540730f0 2024-03-22T10:32:41.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.692+0000 7f9c53400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 0x7f9c541974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:41.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.692+0000 7f9c53400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 0x7f9c541974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:35042/0 (socket says 172.21.15.150:35042) 2024-03-22T10:32:41.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.692+0000 7f9c53400700 1 -- 172.21.15.150:0/1579243246 learned_addr learned my addr 172.21.15.150:0/1579243246 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:41.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.692+0000 7f9c53400700 1 -- 172.21.15.150:0/1579243246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c440095f0 con 0x7f9c540730f0 2024-03-22T10:32:41.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.693+0000 7f9c53400700 1 --2- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 0x7f9c541974d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c440092e0 tx=0x7f9c44004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:41.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.693+0000 7f9c4be00700 1 -- 172.21.15.150:0/1579243246 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c4401d070 con 0x7f9c540730f0 2024-03-22T10:32:41.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.693+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c54197ca0 con 0x7f9c540730f0 2024-03-22T10:32:41.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.693+0000 7f9c4be00700 1 -- 172.21.15.150:0/1579243246 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9c4400be10 con 0x7f9c540730f0 2024-03-22T10:32:41.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.693+0000 7f9c4be00700 1 -- 172.21.15.150:0/1579243246 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c44022e10 con 0x7f9c540730f0 2024-03-22T10:32:41.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.694+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c541980c0 con 0x7f9c540730f0 2024-03-22T10:32:41.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.695+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c54062380 con 0x7f9c540730f0 2024-03-22T10:32:41.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.695+0000 7f9c4be00700 1 -- 172.21.15.150:0/1579243246 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f9c44022470 con 0x7f9c540730f0 2024-03-22T10:32:41.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.696+0000 7f9c4be00700 1 --2- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9c38038530 0x7f9c3803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:41.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.696+0000 7f9c4be00700 1 -- 172.21.15.150:0/1579243246 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f9c4404c540 con 0x7f9c540730f0 2024-03-22T10:32:41.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.696+0000 7f9c52a00700 1 --2- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9c38038530 0x7f9c3803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:41.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.697+0000 7f9c52a00700 1 --2- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9c38038530 0x7f9c3803a9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9c40006fd0 tx=0x7f9c40006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:41.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:41.713+0000 7f9c4be00700 1 -- 172.21.15.150:0/1579243246 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9c4402a540 con 0x7f9c540730f0 2024-03-22T10:32:42.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.017+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9c54109460 con 0x7f9c540730f0 2024-03-22T10:32:42.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.018+0000 7f9c4be00700 1 -- 172.21.15.150:0/1579243246 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f9c44027030 con 0x7f9c540730f0 2024-03-22T10:32:42.013 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:42.013 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:42.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9c38038530 msgr2=0x7f9c3803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:42.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 --2- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9c38038530 0x7f9c3803a9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9c40006fd0 tx=0x7f9c40006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:42.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 msgr2=0x7f9c541974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:42.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 --2- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 0x7f9c541974d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c440092e0 tx=0x7f9c44004c80 comp rx=0 tx=0).stop 2024-03-22T10:32:42.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 shutdown_connections 2024-03-22T10:32:42.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 --2- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9c38038530 0x7f9c3803a9e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:42.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 --2- 172.21.15.150:0/1579243246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540730f0 0x7f9c541974d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:42.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 >> 172.21.15.150:0/1579243246 conn(0x7f9c540fb990 msgr2=0x7f9c54106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:42.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 shutdown_connections 2024-03-22T10:32:42.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:42.022+0000 7f9c5a200700 1 -- 172.21.15.150:0/1579243246 wait complete. 2024-03-22T10:32:42.017 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:43.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:42 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:43.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:42 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:43.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:42 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:43.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:42 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:43.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:42 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi150", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:32:43.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:42 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi150", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-22T10:32:43.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:42 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:32:43.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:42 smithi115 ceph-mon[29111]: Deploying daemon crash.smithi150 on smithi150 2024-03-22T10:32:43.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:42 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/1579243246' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:43.792 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:43.792 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:44.319 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:32:46.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:46 smithi115 ceph-mon[29111]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:46.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:46.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:46.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:46.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.691+0000 7f330a600700 1 -- 172.21.15.150:0/1298805064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 msgr2=0x7f33040ff0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:46.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.691+0000 7f330a600700 1 --2- 172.21.15.150:0/1298805064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 0x7f33040ff0f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009910 tx=0x7f32f4009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:46.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.691+0000 7f330a600700 1 -- 172.21.15.150:0/1298805064 shutdown_connections 2024-03-22T10:32:46.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.691+0000 7f330a600700 1 --2- 172.21.15.150:0/1298805064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 0x7f33040ff0f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:46.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.691+0000 7f330a600700 1 -- 172.21.15.150:0/1298805064 >> 172.21.15.150:0/1298805064 conn(0x7f3304074af0 msgr2=0x7f3304074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:46.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.691+0000 7f330a600700 1 -- 172.21.15.150:0/1298805064 shutdown_connections 2024-03-22T10:32:46.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.691+0000 7f330a600700 1 -- 172.21.15.150:0/1298805064 wait complete. 2024-03-22T10:32:46.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.693+0000 7f330a600700 1 Processor -- start 2024-03-22T10:32:46.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.693+0000 7f330a600700 1 -- start start 2024-03-22T10:32:46.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.694+0000 7f330a600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 0x7f330419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:46.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.694+0000 7f330a600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330419bea0 con 0x7f33040fed20 2024-03-22T10:32:46.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.694+0000 7f3303400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 0x7f330419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:46.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.694+0000 7f3303400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 0x7f330419b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:35064/0 (socket says 172.21.15.150:35064) 2024-03-22T10:32:46.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.694+0000 7f3303400700 1 -- 172.21.15.150:0/587951481 learned_addr learned my addr 172.21.15.150:0/587951481 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:46.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.695+0000 7f3303400700 1 -- 172.21.15.150:0/587951481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f40095f0 con 0x7f33040fed20 2024-03-22T10:32:46.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.695+0000 7f3303400700 1 --2- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 0x7f330419b960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f32f4005b40 tx=0x7f32f4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:46.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.696+0000 7f32fbe00700 1 -- 172.21.15.150:0/587951481 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32f401d070 con 0x7f33040fed20 2024-03-22T10:32:46.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.696+0000 7f32fbe00700 1 -- 172.21.15.150:0/587951481 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32f40044d0 con 0x7f33040fed20 2024-03-22T10:32:46.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.696+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330419c0a0 con 0x7f33040fed20 2024-03-22T10:32:46.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.696+0000 7f32fbe00700 1 -- 172.21.15.150:0/587951481 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32f4003930 con 0x7f33040fed20 2024-03-22T10:32:46.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.696+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330419c460 con 0x7f33040fed20 2024-03-22T10:32:46.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.697+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3304062380 con 0x7f33040fed20 2024-03-22T10:32:46.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.698+0000 7f32fbe00700 1 -- 172.21.15.150:0/587951481 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f32f400b960 con 0x7f33040fed20 2024-03-22T10:32:46.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.698+0000 7f32fbe00700 1 --2- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f32e8038580 0x7f32e803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:46.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.698+0000 7f32fbe00700 1 -- 172.21.15.150:0/587951481 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f32f404c410 con 0x7f33040fed20 2024-03-22T10:32:46.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.699+0000 7f3302a00700 1 --2- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f32e8038580 0x7f32e803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:46.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.700+0000 7f3302a00700 1 --2- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f32e8038580 0x7f32e803aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f32f0006fd0 tx=0x7f32f0006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:46.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:46.713+0000 7f32fbe00700 1 -- 172.21.15.150:0/587951481 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f32f402a540 con 0x7f33040fed20 2024-03-22T10:32:47.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.039+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3304102f40 con 0x7f33040fed20 2024-03-22T10:32:47.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.040+0000 7f32fbe00700 1 -- 172.21.15.150:0/587951481 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f32f402a360 con 0x7f33040fed20 2024-03-22T10:32:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:47.034 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:47.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.043+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f32e8038580 msgr2=0x7f32e803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:47.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.043+0000 7f330a600700 1 --2- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f32e8038580 0x7f32e803aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f32f0006fd0 tx=0x7f32f0006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:47.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.043+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 msgr2=0x7f330419b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:47.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.043+0000 7f330a600700 1 --2- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 0x7f330419b960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f32f4005b40 tx=0x7f32f4004c80 comp rx=0 tx=0).stop 2024-03-22T10:32:47.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.043+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 shutdown_connections 2024-03-22T10:32:47.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.043+0000 7f330a600700 1 --2- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f32e8038580 0x7f32e803aa30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:47.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.043+0000 7f330a600700 1 --2- 172.21.15.150:0/587951481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33040fed20 0x7f330419b960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:47.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.043+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 >> 172.21.15.150:0/587951481 conn(0x7f3304074af0 msgr2=0x7f330410e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:47.037 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.044+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 shutdown_connections 2024-03-22T10:32:47.038 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:47.044+0000 7f330a600700 1 -- 172.21.15.150:0/587951481 wait complete. 2024-03-22T10:32:47.039 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:47.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:47 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:47.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:47 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:47.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:47 smithi115 ceph-mon[29111]: Deploying daemon node-exporter.smithi150 on smithi150 2024-03-22T10:32:47.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:47 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/587951481' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:48.561 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:48 smithi115 ceph-mon[29111]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:48.729 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:48.729 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:49.120 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:32:50.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:50 smithi115 ceph-mon[29111]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:50.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:50 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:52.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.303+0000 7f196dc00700 1 -- 172.21.15.150:0/3828943804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 msgr2=0x7f1968100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:52.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.303+0000 7f196dc00700 1 --2- 172.21.15.150:0/3828943804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 0x7f1968100480 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f195800b0a0 tx=0x7f195800b3b0 comp rx=0 tx=0).stop 2024-03-22T10:32:52.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.303+0000 7f196dc00700 1 -- 172.21.15.150:0/3828943804 shutdown_connections 2024-03-22T10:32:52.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.303+0000 7f196dc00700 1 --2- 172.21.15.150:0/3828943804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 0x7f1968100480 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:52.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.303+0000 7f196dc00700 1 -- 172.21.15.150:0/3828943804 >> 172.21.15.150:0/3828943804 conn(0x7f19680fb990 msgr2=0x7f19680fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:52.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.303+0000 7f196dc00700 1 -- 172.21.15.150:0/3828943804 shutdown_connections 2024-03-22T10:32:52.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.303+0000 7f196dc00700 1 -- 172.21.15.150:0/3828943804 wait complete. 2024-03-22T10:32:52.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.304+0000 7f196dc00700 1 Processor -- start 2024-03-22T10:32:52.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.304+0000 7f196dc00700 1 -- start start 2024-03-22T10:32:52.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.305+0000 7f196dc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 0x7f19681953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:52.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.305+0000 7f196dc00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19681958e0 con 0x7f19681000b0 2024-03-22T10:32:52.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.305+0000 7f1966a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 0x7f19681953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:52.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.306+0000 7f1966a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 0x7f19681953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:52306/0 (socket says 172.21.15.150:52306) 2024-03-22T10:32:52.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.306+0000 7f1966a00700 1 -- 172.21.15.150:0/3004026625 learned_addr learned my addr 172.21.15.150:0/3004026625 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:52.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.306+0000 7f1966a00700 1 -- 172.21.15.150:0/3004026625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958009d00 con 0x7f19681000b0 2024-03-22T10:32:52.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.307+0000 7f1966a00700 1 --2- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 0x7f19681953a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1958009fd0 tx=0x7f1958003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:52.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.307+0000 7f195fe00700 1 -- 172.21.15.150:0/3004026625 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f195800e040 con 0x7f19681000b0 2024-03-22T10:32:52.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.308+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1968195ae0 con 0x7f19681000b0 2024-03-22T10:32:52.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.308+0000 7f195fe00700 1 -- 172.21.15.150:0/3004026625 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f195800bdd0 con 0x7f19681000b0 2024-03-22T10:32:52.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.308+0000 7f195fe00700 1 -- 172.21.15.150:0/3004026625 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f195801d9a0 con 0x7f19681000b0 2024-03-22T10:32:52.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.308+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1968195f60 con 0x7f19681000b0 2024-03-22T10:32:52.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.309+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1968062380 con 0x7f19681000b0 2024-03-22T10:32:52.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.310+0000 7f195fe00700 1 -- 172.21.15.150:0/3004026625 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f1958004250 con 0x7f19681000b0 2024-03-22T10:32:52.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.310+0000 7f195fe00700 1 --2- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f194c0385d0 0x7f194c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:52.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.310+0000 7f195fe00700 1 -- 172.21.15.150:0/3004026625 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f195804c4e0 con 0x7f19681000b0 2024-03-22T10:32:52.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.311+0000 7f195e000700 1 --2- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f194c0385d0 0x7f194c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:52.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.312+0000 7f195e000700 1 --2- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f194c0385d0 0x7f194c03aa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1950006fd0 tx=0x7f1950006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:52.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.326+0000 7f195fe00700 1 -- 172.21.15.150:0/3004026625 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1958011960 con 0x7f19681000b0 2024-03-22T10:32:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.620+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1968107e20 con 0x7f19681000b0 2024-03-22T10:32:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.621+0000 7f195fe00700 1 -- 172.21.15.150:0/3004026625 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f1958011390 con 0x7f19681000b0 2024-03-22T10:32:52.615 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:52.615 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.624+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f194c0385d0 msgr2=0x7f194c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.624+0000 7f196dc00700 1 --2- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f194c0385d0 0x7f194c03aa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1950006fd0 tx=0x7f1950006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.624+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 msgr2=0x7f19681953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.624+0000 7f196dc00700 1 --2- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 0x7f19681953a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1958009fd0 tx=0x7f1958003ce0 comp rx=0 tx=0).stop 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.624+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 shutdown_connections 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.624+0000 7f196dc00700 1 --2- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f194c0385d0 0x7f194c03aa80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.624+0000 7f196dc00700 1 --2- 172.21.15.150:0/3004026625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19681000b0 0x7f19681953a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.624+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 >> 172.21.15.150:0/3004026625 conn(0x7f19680fb990 msgr2=0x7f1968104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.624+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 shutdown_connections 2024-03-22T10:32:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:52.625+0000 7f196dc00700 1 -- 172.21.15.150:0/3004026625 wait complete. 2024-03-22T10:32:52.619 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:52.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:52 smithi115 ceph-mon[29111]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:52.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:52 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:52.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:52 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:52.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:52 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:52.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:52 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:52.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:52 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:32:53.578 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T10:32:53.578 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:32:53.578 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:32:53.578 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:53 smithi115 ceph-mon[29111]: Deploying daemon mgr.smithi150.fmoumv on smithi150 2024-03-22T10:32:53.578 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:53 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/3004026625' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:54.329 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:54.329 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:32:54.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:54 smithi115 ceph-mon[29111]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:55.084 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:32:55.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:32:55.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:56.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:56 smithi115 ceph-mon[29111]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:57.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.699+0000 7fa037e00700 1 -- 172.21.15.150:0/1696535891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 msgr2=0x7fa0300734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:57.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.699+0000 7fa037e00700 1 --2- 172.21.15.150:0/1696535891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 0x7fa0300734c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa024009910 tx=0x7fa024009c20 comp rx=0 tx=0).stop 2024-03-22T10:32:57.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.699+0000 7fa037e00700 1 -- 172.21.15.150:0/1696535891 shutdown_connections 2024-03-22T10:32:57.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.699+0000 7fa037e00700 1 --2- 172.21.15.150:0/1696535891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 0x7fa0300734c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:57.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.699+0000 7fa037e00700 1 -- 172.21.15.150:0/1696535891 >> 172.21.15.150:0/1696535891 conn(0x7fa0300fb990 msgr2=0x7fa0300fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:57.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.699+0000 7fa037e00700 1 -- 172.21.15.150:0/1696535891 shutdown_connections 2024-03-22T10:32:57.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.700+0000 7fa037e00700 1 -- 172.21.15.150:0/1696535891 wait complete. 2024-03-22T10:32:57.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.700+0000 7fa037e00700 1 Processor -- start 2024-03-22T10:32:57.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.701+0000 7fa037e00700 1 -- start start 2024-03-22T10:32:57.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.701+0000 7fa037e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 0x7fa0301a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:57.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.701+0000 7fa037e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301a1be0 con 0x7fa0300730f0 2024-03-22T10:32:57.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.702+0000 7fa035400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 0x7fa0301a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:57.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.702+0000 7fa035400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 0x7fa0301a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.150:52340/0 (socket says 172.21.15.150:52340) 2024-03-22T10:32:57.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.702+0000 7fa035400700 1 -- 172.21.15.150:0/1617331294 learned_addr learned my addr 172.21.15.150:0/1617331294 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:32:57.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.702+0000 7fa035400700 1 -- 172.21.15.150:0/1617331294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240095f0 con 0x7fa0300730f0 2024-03-22T10:32:57.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.703+0000 7fa035400700 1 --2- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 0x7fa0301a16a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0240092e0 tx=0x7fa024004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:57.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.703+0000 7fa02e000700 1 -- 172.21.15.150:0/1617331294 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa02401d070 con 0x7fa0300730f0 2024-03-22T10:32:57.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.703+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0301a1de0 con 0x7fa0300730f0 2024-03-22T10:32:57.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.704+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0301a2200 con 0x7fa0300730f0 2024-03-22T10:32:57.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.704+0000 7fa02e000700 1 -- 172.21.15.150:0/1617331294 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa02400be10 con 0x7fa0300730f0 2024-03-22T10:32:57.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.704+0000 7fa02e000700 1 -- 172.21.15.150:0/1617331294 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa02400fdf0 con 0x7fa0300730f0 2024-03-22T10:32:57.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.705+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa010005320 con 0x7fa0300730f0 2024-03-22T10:32:57.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.705+0000 7fa02e000700 1 -- 172.21.15.150:0/1617331294 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7fa02400f6b0 con 0x7fa0300730f0 2024-03-22T10:32:57.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.706+0000 7fa02e000700 1 --2- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa018038530 0x7fa01803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:32:57.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.706+0000 7fa02e000700 1 -- 172.21.15.150:0/1617331294 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa02404c570 con 0x7fa0300730f0 2024-03-22T10:32:57.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.707+0000 7fa034a00700 1 --2- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa018038530 0x7fa01803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:32:57.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.708+0000 7fa034a00700 1 --2- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa018038530 0x7fa01803a9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa01c006fd0 tx=0x7fa01c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:32:57.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:57.722+0000 7fa02e000700 1 -- 172.21.15.150:0/1617331294 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa0240215b0 con 0x7fa0300730f0 2024-03-22T10:32:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.024+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa010005190 con 0x7fa0300730f0 2024-03-22T10:32:58.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.026+0000 7fa02e000700 1 -- 172.21.15.150:0/1617331294 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fa024027020 con 0x7fa0300730f0 2024-03-22T10:32:58.019 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:32:58.020 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:29:33.145545Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T10:32:58.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa018038530 msgr2=0x7fa01803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:58.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 --2- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa018038530 0x7fa01803a9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa01c006fd0 tx=0x7fa01c006e40 comp rx=0 tx=0).stop 2024-03-22T10:32:58.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 msgr2=0x7fa0301a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:32:58.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 --2- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 0x7fa0301a16a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0240092e0 tx=0x7fa024004c80 comp rx=0 tx=0).stop 2024-03-22T10:32:58.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 shutdown_connections 2024-03-22T10:32:58.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 --2- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa018038530 0x7fa01803a9e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:58.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 --2- 172.21.15.150:0/1617331294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0300730f0 0x7fa0301a16a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:32:58.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 >> 172.21.15.150:0/1617331294 conn(0x7fa0300fb990 msgr2=0x7fa0300fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:32:58.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 shutdown_connections 2024-03-22T10:32:58.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:32:58.029+0000 7fa037e00700 1 -- 172.21.15.150:0/1617331294 wait complete. 2024-03-22T10:32:58.023 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-03-22T10:32:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:58 smithi115 ceph-mon[29111]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:32:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:32:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:32:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:32:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:58 smithi115 ceph-mon[29111]: Deploying daemon mon.smithi150 on smithi150 2024-03-22T10:32:59.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:32:58 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/1617331294' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:32:59.775 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T10:32:59.775 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T10:33:01.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:00 smithi115 ceph-mon[29111]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:01.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:01.447 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi150/config 2024-03-22T10:33:03.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:02 smithi115 ceph-mon[29111]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:04.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:03 smithi115 ceph-mon[29111]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:09.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:09 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:09.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: mon.smithi115 calling monitor election 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: mon.smithi150 calling monitor election 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/2419303946' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/crt"}]: dispatch 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: mon.smithi115 is new leader, mons smithi115,smithi150 in quorum (ranks 0,1) 2024-03-22T10:33:09.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: monmap e2: 2 mons at {smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],smithi150=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: fsmap 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: mgrmap e17: smithi115.iqzwsz(active, since 43s) 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: Standby manager daemon smithi150.fmoumv started 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/2419303946' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: overall HEALTH_OK 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/2419303946' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/key"}]: dispatch 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/2419303946' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:09.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:09 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:10.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.098+0000 7fefb9600700 1 -- 172.21.15.150:0/2872084724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef98006220 msgr2=0x7fef98008680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:33:10.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.098+0000 7fefb9600700 1 --2- 172.21.15.150:0/2872084724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef98006220 0x7fef98008680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fefb40741b0 tx=0x7fefa400fe90 comp rx=0 tx=0).stop 2024-03-22T10:33:10.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.098+0000 7fefb9600700 1 -- 172.21.15.150:0/2872084724 shutdown_connections 2024-03-22T10:33:10.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.098+0000 7fefb9600700 1 --2- 172.21.15.150:0/2872084724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef98006220 0x7fef98008680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:10.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.098+0000 7fefb9600700 1 -- 172.21.15.150:0/2872084724 >> 172.21.15.150:0/2872084724 conn(0x7fefb40fba50 msgr2=0x7fefb40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:33:10.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.099+0000 7fefb9600700 1 -- 172.21.15.150:0/2872084724 shutdown_connections 2024-03-22T10:33:10.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.099+0000 7fefb9600700 1 -- 172.21.15.150:0/2872084724 wait complete. 2024-03-22T10:33:10.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.100+0000 7fefb9600700 1 Processor -- start 2024-03-22T10:33:10.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.101+0000 7fefb9600700 1 -- start start 2024-03-22T10:33:10.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.101+0000 7fefb9600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 0x7fefb41a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:33:10.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.101+0000 7fefb9600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefb41a1f20 0x7fefb419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:33:10.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.101+0000 7fefb9600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb41a24e0 con 0x7fefb41a1f20 2024-03-22T10:33:10.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.101+0000 7fefb9600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb419bfa0 con 0x7fef98006220 2024-03-22T10:33:10.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.102+0000 7fefb2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 0x7fefb41a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:33:10.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.102+0000 7fefb2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 0x7fefb41a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39700/0 (socket says 172.21.15.150:39700) 2024-03-22T10:33:10.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.102+0000 7fefb2a00700 1 -- 172.21.15.150:0/3636548508 learned_addr learned my addr 172.21.15.150:0/3636548508 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:33:10.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.102+0000 7fefb2a00700 1 -- 172.21.15.150:0/3636548508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 msgr2=0x7fefb41a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T10:33:10.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.102+0000 7fefb2a00700 1 -- 172.21.15.150:0/3636548508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 msgr2=0x7fefb41a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T10:33:10.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.102+0000 7fefb2a00700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 0x7fefb41a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T10:33:10.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.102+0000 7fefb2000700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefb41a1f20 0x7fefb419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:33:10.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.102+0000 7fefb2a00700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 0x7fefb41a19e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:33:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.103+0000 7fefb2000700 1 -- 172.21.15.150:0/3636548508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 msgr2=0x7fefb41a19e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:33:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.103+0000 7fefb2000700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 0x7fefb41a19e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.103+0000 7fefb2000700 1 -- 172.21.15.150:0/3636548508 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa400f8e0 con 0x7fefb41a1f20 2024-03-22T10:33:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.103+0000 7fefb2000700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefb41a1f20 0x7fefb419ba60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fefa000b4d0 tx=0x7fefa000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:33:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.103+0000 7fefab400700 1 -- 172.21.15.150:0/3636548508 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefa0011800 con 0x7fefb41a1f20 2024-03-22T10:33:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.104+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb419c280 con 0x7fefb41a1f20 2024-03-22T10:33:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.104+0000 7fefab400700 1 -- 172.21.15.150:0/3636548508 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fefa0011e10 con 0x7fefb41a1f20 2024-03-22T10:33:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.104+0000 7fefab400700 1 -- 172.21.15.150:0/3636548508 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefa00193b0 con 0x7fefb41a1f20 2024-03-22T10:33:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.104+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb419c7d0 con 0x7fefb41a1f20 2024-03-22T10:33:10.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.105+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb4062380 con 0x7fefb41a1f20 2024-03-22T10:33:10.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.106+0000 7fefab400700 1 -- 172.21.15.150:0/3636548508 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7fefa001a800 con 0x7fefb41a1f20 2024-03-22T10:33:10.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.107+0000 7fefab400700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fef9406c750 0x7fef9406ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:33:10.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.107+0000 7fefab400700 1 -- 172.21.15.150:0/3636548508 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fefa008a830 con 0x7fefb41a1f20 2024-03-22T10:33:10.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.108+0000 7fefb2a00700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fef9406c750 0x7fef9406ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:33:10.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.109+0000 7fefb2a00700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fef9406c750 0x7fef9406ec00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005ed0 tx=0x7fefa4005da0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:33:10.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.121+0000 7fefab400700 1 -- 172.21.15.150:0/3636548508 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fefa0059ba0 con 0x7fefb41a1f20 2024-03-22T10:33:10.303 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:10.304 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:10 smithi150 ceph-mon[34518]: mgrmap e18: smithi115.iqzwsz(active, since 43s), standbys: smithi150.fmoumv 2024-03-22T10:33:10.304 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi150.fmoumv", "id": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:33:10.304 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:10.304 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:10 smithi150 ceph-mon[34518]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:10.304 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:33:10.304 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:33:10.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.430+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fefb419cf70 con 0x7fefb41a1f20 2024-03-22T10:33:10.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.432+0000 7fefab400700 1 -- 172.21.15.150:0/3636548508 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7fefa0015090 con 0x7fefb41a1f20 2024-03-22T10:33:10.425 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-22T10:33:10.425 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":2,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","modified":"2024-03-22T10:33:03.944793Z","created":"2024-03-22T10:29:33.145545Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi150","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-22T10:33:10.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.434+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fef9406c750 msgr2=0x7fef9406ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:33:10.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.434+0000 7fefb9600700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fef9406c750 0x7fef9406ec00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005ed0 tx=0x7fefa4005da0 comp rx=0 tx=0).stop 2024-03-22T10:33:10.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.434+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefb41a1f20 msgr2=0x7fefb419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:33:10.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.434+0000 7fefb9600700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefb41a1f20 0x7fefb419ba60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fefa000b4d0 tx=0x7fefa000b890 comp rx=0 tx=0).stop 2024-03-22T10:33:10.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.435+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 shutdown_connections 2024-03-22T10:33:10.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.435+0000 7fefb9600700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fef9406c750 0x7fef9406ec00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:10.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.435+0000 7fefb9600700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef98006220 0x7fefb41a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:10.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.435+0000 7fefb9600700 1 --2- 172.21.15.150:0/3636548508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefb41a1f20 0x7fefb419ba60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:10.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.435+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 >> 172.21.15.150:0/3636548508 conn(0x7fefb40fba50 msgr2=0x7fefb40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:33:10.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.435+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 shutdown_connections 2024-03-22T10:33:10.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:33:10.435+0000 7fefb9600700 1 -- 172.21.15.150:0/3636548508 wait complete. 2024-03-22T10:33:10.429 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 2 2024-03-22T10:33:10.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:10.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:10 smithi115 ceph-mon[29111]: mgrmap e18: smithi115.iqzwsz(active, since 43s), standbys: smithi150.fmoumv 2024-03-22T10:33:10.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi150.fmoumv", "id": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:33:10.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:10.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:10 smithi115 ceph-mon[29111]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:10.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:33:10.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:33:11.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:11 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:33:11.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:11 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:11.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:11 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/3636548508' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:33:11.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:11 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:33:11.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:11 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:11.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:11 smithi150 ceph-mon[34518]: from='client.? 172.21.15.150:0/3636548508' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T10:33:11.559 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-22T10:33:11.560 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-22T10:33:11.973 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:33:12.062 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:12 smithi150 ceph-mon[34518]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:12.325 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:12 smithi115 ceph-mon[29111]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:13.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:13 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:13.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:13 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:13.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:13 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:13.600 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:13 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:33:13.600 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:13 smithi115 ceph-mon[29111]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-22T10:33:13.600 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:13 smithi115 ceph-mon[29111]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-22T10:33:13.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:13 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:13.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:13 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:13.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:13 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:13.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:13 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:33:13.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:13 smithi150 ceph-mon[34518]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-22T10:33:13.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:13 smithi150 ceph-mon[34518]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-22T10:33:14.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.105+0000 7f3b22000700 1 -- 172.21.15.115:0/2438313732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 msgr2=0x7f3b1c1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:33:14.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.105+0000 7f3b22000700 1 --2- 172.21.15.115:0/2438313732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 0x7f3b1c1013d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009910 tx=0x7f3b0c009c20 comp rx=0 tx=0).stop 2024-03-22T10:33:14.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.105+0000 7f3b22000700 1 -- 172.21.15.115:0/2438313732 shutdown_connections 2024-03-22T10:33:14.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.105+0000 7f3b22000700 1 --2- 172.21.15.115:0/2438313732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 0x7f3b1c1013d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:14.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.105+0000 7f3b22000700 1 -- 172.21.15.115:0/2438313732 >> 172.21.15.115:0/2438313732 conn(0x7f3b1c0fa860 msgr2=0x7f3b1c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:33:14.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.105+0000 7f3b22000700 1 -- 172.21.15.115:0/2438313732 shutdown_connections 2024-03-22T10:33:14.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.105+0000 7f3b22000700 1 -- 172.21.15.115:0/2438313732 wait complete. 2024-03-22T10:33:14.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.106+0000 7f3b22000700 1 Processor -- start 2024-03-22T10:33:14.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.106+0000 7f3b22000700 1 -- start start 2024-03-22T10:33:14.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.107+0000 7f3b22000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 0x7f3b1c0fec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:33:14.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.107+0000 7f3b22000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c0ff1a0 0x7f3b1c0ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:33:14.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.107+0000 7f3b22000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1a16e0 con 0x7f3b1c101000 2024-03-22T10:33:14.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.107+0000 7f3b22000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1a1820 con 0x7f3b1c0ff1a0 2024-03-22T10:33:14.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.107+0000 7f3b1b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 0x7f3b1c0fec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.107+0000 7f3b1b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 0x7f3b1c0fec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44766/0 (socket says 172.21.15.115:44766) 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.107+0000 7f3b1b400700 1 -- 172.21.15.115:0/76431789 learned_addr learned my addr 172.21.15.115:0/76431789 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.107+0000 7f3b1b400700 1 -- 172.21.15.115:0/76431789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c0ff1a0 msgr2=0x7f3b1c0ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.107+0000 7f3b1aa00700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c0ff1a0 0x7f3b1c0ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.108+0000 7f3b1b400700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c0ff1a0 0x7f3b1c0ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.108+0000 7f3b1b400700 1 -- 172.21.15.115:0/76431789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c0095f0 con 0x7f3b1c101000 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.108+0000 7f3b1aa00700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c0ff1a0 0x7f3b1c0ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.108+0000 7f3b1b400700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 0x7f3b1c0fec60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c006010 tx=0x7f3b0c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.108+0000 7f3b13e00700 1 -- 172.21.15.115:0/76431789 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b0c01d070 con 0x7f3b1c101000 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.108+0000 7f3b13e00700 1 -- 172.21.15.115:0/76431789 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3b0c005140 con 0x7f3b1c101000 2024-03-22T10:33:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.108+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c1a1960 con 0x7f3b1c101000 2024-03-22T10:33:14.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.108+0000 7f3b13e00700 1 -- 172.21.15.115:0/76431789 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b0c017440 con 0x7f3b1c101000 2024-03-22T10:33:14.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.108+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c1a1e50 con 0x7f3b1c101000 2024-03-22T10:33:14.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.110+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c107650 con 0x7f3b1c101000 2024-03-22T10:33:14.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.111+0000 7f3b13e00700 1 -- 172.21.15.115:0/76431789 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f3b0c0175a0 con 0x7f3b1c101000 2024-03-22T10:33:14.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.111+0000 7f3b13e00700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f3b0006c760 0x7f3b0006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:33:14.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.112+0000 7f3b13e00700 1 -- 172.21.15.115:0/76431789 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3b0c08bb60 con 0x7f3b1c101000 2024-03-22T10:33:14.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.112+0000 7f3b1aa00700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f3b0006c760 0x7f3b0006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:33:14.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.112+0000 7f3b1aa00700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f3b0006c760 0x7f3b0006ec10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b080079b0 tx=0x7f3b08008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:33:14.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.121+0000 7f3b13e00700 1 -- 172.21.15.115:0/76431789 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3b0c057280 con 0x7f3b1c101000 2024-03-22T10:33:14.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.355+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3b1c062380 con 0x7f3b1c101000 2024-03-22T10:33:14.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.355+0000 7f3b13e00700 1 -- 172.21.15.115:0/76431789 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7f3b0c026090 con 0x7f3b1c101000 2024-03-22T10:33:14.349 INFO:teuthology.orchestra.run.smithi115.stdout:# minimal ceph.conf for fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:33:14.350 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-03-22T10:33:14.350 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:33:14.350 INFO:teuthology.orchestra.run.smithi115.stdout: mon_host = [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.358+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f3b0006c760 msgr2=0x7f3b0006ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.358+0000 7f3b22000700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f3b0006c760 0x7f3b0006ec10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b080079b0 tx=0x7f3b08008040 comp rx=0 tx=0).stop 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.358+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 msgr2=0x7f3b1c0fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.358+0000 7f3b22000700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 0x7f3b1c0fec60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c006010 tx=0x7f3b0c00bda0 comp rx=0 tx=0).stop 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.358+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 shutdown_connections 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.358+0000 7f3b22000700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f3b0006c760 0x7f3b0006ec10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.358+0000 7f3b22000700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b1c101000 0x7f3b1c0fec60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.358+0000 7f3b22000700 1 --2- 172.21.15.115:0/76431789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c0ff1a0 0x7f3b1c0ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.358+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 >> 172.21.15.115:0/76431789 conn(0x7f3b1c0fa860 msgr2=0x7f3b1c104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.359+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 shutdown_connections 2024-03-22T10:33:14.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:14.359+0000 7f3b22000700 1 -- 172.21.15.115:0/76431789 wait complete. 2024-03-22T10:33:15.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:15.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:14 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/76431789' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:15.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:14 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/76431789' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:15.197 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-22T10:33:15.198 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:33:15.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T10:33:15.281 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:33:15.281 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:33:15.411 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:33:15.411 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T10:33:15.510 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:33:15.510 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:33:15.653 INFO:tasks.cephadm:Deploying OSDs... 2024-03-22T10:33:15.653 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:33:15.653 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T10:33:15.698 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T10:33:15.698 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-03-22T10:33:15.785 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T10:33:15.785 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T10:33:15.785 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-03-22T10:33:15.785 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:33:15.785 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T10:33:15.785 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-03-22 10:32:14.581986469 +0000 2024-03-22T10:33:15.786 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-03-22 10:25:52.593419022 +0000 2024-03-22T10:33:15.786 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-03-22 10:25:52.593419022 +0000 2024-03-22T10:33:15.786 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-03-22 10:25:52.593419022 +0000 2024-03-22T10:33:15.786 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T10:33:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-03-22T10:33:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-03-22T10:33:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000292843 s, 1.7 MB/s 2024-03-22T10:33:15.873 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T10:33:15.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:15 smithi150 ceph-mon[34518]: Reconfiguring mon.smithi115 (unknown last config time)... 2024-03-22T10:33:15.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:15 smithi150 ceph-mon[34518]: Reconfiguring daemon mon.smithi115 on smithi115 2024-03-22T10:33:15.918 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-03-22T10:33:15.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:15 smithi115 ceph-mon[29111]: Reconfiguring mon.smithi115 (unknown last config time)... 2024-03-22T10:33:15.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:15 smithi115 ceph-mon[29111]: Reconfiguring daemon mon.smithi115 on smithi115 2024-03-22T10:33:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T10:33:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T10:33:16.009 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-22T10:33:16.009 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:33:16.009 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T10:33:16.009 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-03-22 10:32:14.581986469 +0000 2024-03-22T10:33:16.009 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-03-22 10:25:53.158408333 +0000 2024-03-22T10:33:16.009 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-03-22 10:25:53.158408333 +0000 2024-03-22T10:33:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-03-22 10:25:53.158408333 +0000 2024-03-22T10:33:16.009 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T10:33:16.141 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-03-22T10:33:16.141 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-03-22T10:33:16.141 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000319485 s, 1.6 MB/s 2024-03-22T10:33:16.145 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T10:33:16.204 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-03-22T10:33:16.250 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T10:33:16.250 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T10:33:16.250 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-22T10:33:16.250 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:33:16.250 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T10:33:16.251 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-03-22 10:32:14.582986450 +0000 2024-03-22T10:33:16.251 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-03-22 10:25:53.722397663 +0000 2024-03-22T10:33:16.251 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-03-22 10:25:53.722397663 +0000 2024-03-22T10:33:16.251 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-03-22 10:25:53.722397663 +0000 2024-03-22T10:33:16.251 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T10:33:16.382 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-03-22T10:33:16.382 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-03-22T10:33:16.382 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000321904 s, 1.6 MB/s 2024-03-22T10:33:16.385 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T10:33:16.439 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-03-22T10:33:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T10:33:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T10:33:16.530 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-22T10:33:16.530 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:33:16.530 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T10:33:16.530 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-03-22 10:32:14.583986431 +0000 2024-03-22T10:33:16.530 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-03-22 10:25:54.287386974 +0000 2024-03-22T10:33:16.530 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-03-22 10:25:54.287386974 +0000 2024-03-22T10:33:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-03-22 10:25:54.287386974 +0000 2024-03-22T10:33:16.531 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T10:33:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-03-22T10:33:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-03-22T10:33:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.00026562 s, 1.9 MB/s 2024-03-22T10:33:16.651 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T10:33:16.703 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:33:16.704 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T10:33:16.763 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T10:33:16.763 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-03-22T10:33:16.814 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T10:33:16.814 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T10:33:16.814 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-22T10:33:16.815 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:33:16.815 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T10:33:16.815 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-03-22 10:33:01.981306136 +0000 2024-03-22T10:33:16.815 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-03-22 10:25:52.603077688 +0000 2024-03-22T10:33:16.815 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-03-22 10:25:52.603077688 +0000 2024-03-22T10:33:16.815 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-03-22 10:25:52.603077688 +0000 2024-03-22T10:33:16.815 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T10:33:16.900 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-03-22T10:33:16.900 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-03-22T10:33:16.900 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000465947 s, 1.1 MB/s 2024-03-22T10:33:16.904 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T10:33:16.961 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-03-22T10:33:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T10:33:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T10:33:17.007 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-22T10:33:17.007 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:33:17.007 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T10:33:17.008 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-03-22 10:33:01.982306116 +0000 2024-03-22T10:33:17.008 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-03-22 10:25:53.165066669 +0000 2024-03-22T10:33:17.008 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-03-22 10:25:53.165066669 +0000 2024-03-22T10:33:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-03-22 10:25:53.165066669 +0000 2024-03-22T10:33:17.008 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T10:33:17.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:16 smithi115 ceph-mon[29111]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:17.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:17.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:17.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.iqzwsz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:33:17.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:33:17.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:17.081 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:16 smithi150 ceph-mon[34518]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:17.081 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:17.081 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:17.081 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.iqzwsz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:33:17.081 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:33:17.081 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:17.092 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-03-22T10:33:17.092 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-03-22T10:33:17.092 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000453892 s, 1.1 MB/s 2024-03-22T10:33:17.096 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T10:33:17.153 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-03-22T10:33:17.204 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T10:33:17.205 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T10:33:17.205 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-22T10:33:17.205 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:33:17.205 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T10:33:17.205 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-03-22 10:33:01.982306116 +0000 2024-03-22T10:33:17.205 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-03-22 10:25:53.826053710 +0000 2024-03-22T10:33:17.205 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-03-22 10:25:53.826053710 +0000 2024-03-22T10:33:17.205 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-03-22 10:25:53.826053710 +0000 2024-03-22T10:33:17.205 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T10:33:17.292 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-03-22T10:33:17.292 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-03-22T10:33:17.292 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00038668 s, 1.3 MB/s 2024-03-22T10:33:17.295 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T10:33:17.348 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-03-22T10:33:17.397 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T10:33:17.397 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T10:33:17.397 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-22T10:33:17.397 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:33:17.397 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T10:33:17.397 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-03-22 10:33:01.983306096 +0000 2024-03-22T10:33:17.397 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-03-22 10:25:54.402042416 +0000 2024-03-22T10:33:17.397 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-03-22 10:25:54.402042416 +0000 2024-03-22T10:33:17.397 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-03-22 10:25:54.402042416 +0000 2024-03-22T10:33:17.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T10:33:17.481 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-03-22T10:33:17.481 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-03-22T10:33:17.481 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000378616 s, 1.4 MB/s 2024-03-22T10:33:17.485 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T10:33:17.539 INFO:tasks.cephadm:Deploying osd.0 on smithi115 with /dev/vg_nvme/lv_4... 2024-03-22T10:33:17.539 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T10:33:18.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:17 smithi115 ceph-mon[29111]: Reconfiguring mgr.smithi115.iqzwsz (unknown last config time)... 2024-03-22T10:33:18.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:17 smithi115 ceph-mon[29111]: Reconfiguring daemon mgr.smithi115.iqzwsz on smithi115 2024-03-22T10:33:18.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:17 smithi150 ceph-mon[34518]: Reconfiguring mgr.smithi115.iqzwsz (unknown last config time)... 2024-03-22T10:33:18.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:17 smithi150 ceph-mon[34518]: Reconfiguring daemon mgr.smithi115.iqzwsz on smithi115 2024-03-22T10:33:18.311 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:33:19.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:18 smithi115 ceph-mon[29111]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:19.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:19.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:19.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:33:19.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:19.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:18 smithi150 ceph-mon[34518]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:19.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:19.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:19.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:33:19.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:20.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:19 smithi115 ceph-mon[29111]: Reconfiguring ceph-exporter.smithi115 (monmap changed)... 2024-03-22T10:33:20.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:19 smithi115 ceph-mon[29111]: Reconfiguring daemon ceph-exporter.smithi115 on smithi115 2024-03-22T10:33:20.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:19 smithi115 ceph-mon[29111]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:20.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:19 smithi150 ceph-mon[34518]: Reconfiguring ceph-exporter.smithi115 (monmap changed)... 2024-03-22T10:33:20.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:19 smithi150 ceph-mon[34518]: Reconfiguring daemon ceph-exporter.smithi115 on smithi115 2024-03-22T10:33:20.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:19 smithi150 ceph-mon[34518]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:21.076 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T10:33:21.076 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T10:33:21.076 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-03-22T10:33:21.076 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-03-22T10:33:21.076 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036516 s, 287 MB/s 2024-03-22T10:33:21.076 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-03-22T10:33:22.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2024-03-22T10:33:22.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:22 smithi150 ceph-mon[34518]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:22.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:22 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:22.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:22 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:22.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:22 smithi150 ceph-mon[34518]: Reconfiguring crash.smithi115 (monmap changed)... 2024-03-22T10:33:22.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:22 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:33:22.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:22 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:22.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:22 smithi150 ceph-mon[34518]: Reconfiguring daemon crash.smithi115 on smithi115 2024-03-22T10:33:23.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:22 smithi115 ceph-mon[29111]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:23.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:22 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:23.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:22 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:23.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:22 smithi115 ceph-mon[29111]: Reconfiguring crash.smithi115 (monmap changed)... 2024-03-22T10:33:23.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:22 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:33:23.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:22 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:23.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:22 smithi115 ceph-mon[29111]: Reconfiguring daemon crash.smithi115 on smithi115 2024-03-22T10:33:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:33:24.831 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:24 smithi115 ceph-mon[29111]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:24.831 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:24 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:24.831 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:24 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:24.831 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:24 smithi115 ceph-mon[29111]: Reconfiguring alertmanager.smithi115 (dependencies changed)... 2024-03-22T10:33:24.831 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:24 smithi115 ceph-mon[29111]: Reconfiguring daemon alertmanager.smithi115 on smithi115 2024-03-22T10:33:24.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:24 smithi150 ceph-mon[34518]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:24.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:24 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:24.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:24 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:24.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:24 smithi150 ceph-mon[34518]: Reconfiguring alertmanager.smithi115 (dependencies changed)... 2024-03-22T10:33:24.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:24 smithi150 ceph-mon[34518]: Reconfiguring daemon alertmanager.smithi115 on smithi115 2024-03-22T10:33:25.731 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:33:25.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:33:26.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.091+0000 7f0487e00700 1 -- 172.21.15.115:0/808676352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 msgr2=0x7f0480100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.091+0000 7f0487e00700 1 --2- 172.21.15.115:0/808676352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 0x7f0480100560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0474009910 tx=0x7f0474009c20 comp rx=0 tx=0).stop 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.091+0000 7f0487e00700 1 -- 172.21.15.115:0/808676352 shutdown_connections 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.091+0000 7f0487e00700 1 --2- 172.21.15.115:0/808676352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0480101350 0x7f04801017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.091+0000 7f0487e00700 1 --2- 172.21.15.115:0/808676352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 0x7f0480100560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.091+0000 7f0487e00700 1 -- 172.21.15.115:0/808676352 >> 172.21.15.115:0/808676352 conn(0x7f04800fb6e0 msgr2=0x7f04800fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.091+0000 7f0487e00700 1 -- 172.21.15.115:0/808676352 shutdown_connections 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.091+0000 7f0487e00700 1 -- 172.21.15.115:0/808676352 wait complete. 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.093+0000 7f0487e00700 1 Processor -- start 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.093+0000 7f0487e00700 1 -- start start 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.093+0000 7f0487e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 0x7f0480197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:33:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.093+0000 7f0487e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0480101350 0x7f04801980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.093+0000 7f0487e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04801986e0 con 0x7f0480100150 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.093+0000 7f0487e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0480198820 con 0x7f0480101350 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.093+0000 7f0485400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 0x7f0480197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.093+0000 7f0485400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 0x7f0480197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42894/0 (socket says 172.21.15.115:42894) 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.093+0000 7f0485400700 1 -- 172.21.15.115:0/1758762277 learned_addr learned my addr 172.21.15.115:0/1758762277 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.094+0000 7f0484a00700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0480101350 0x7f04801980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.094+0000 7f0485400700 1 -- 172.21.15.115:0/1758762277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0480101350 msgr2=0x7f04801980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.094+0000 7f0485400700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0480101350 0x7f04801980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.094+0000 7f0485400700 1 -- 172.21.15.115:0/1758762277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04740095f0 con 0x7f0480100150 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.094+0000 7f0485400700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 0x7f0480197b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f047400bda0 tx=0x7f047400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.095+0000 7f047e000700 1 -- 172.21.15.115:0/1758762277 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f047401d070 con 0x7f0480100150 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.095+0000 7f047e000700 1 -- 172.21.15.115:0/1758762277 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f047400f460 con 0x7f0480100150 2024-03-22T10:33:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.095+0000 7f047e000700 1 -- 172.21.15.115:0/1758762277 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0474021640 con 0x7f0480100150 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.095+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048019d270 con 0x7f0480100150 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.095+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048019d7e0 con 0x7f0480100150 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.096+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0480105430 con 0x7f0480100150 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.097+0000 7f047e000700 1 -- 172.21.15.115:0/1758762277 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f047400f5d0 con 0x7f0480100150 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.098+0000 7f047e000700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f046806c610 0x7f046806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.098+0000 7f047e000700 1 -- 172.21.15.115:0/1758762277 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f047408baa0 con 0x7f0480100150 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.098+0000 7f0484a00700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f046806c610 0x7f046806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.098+0000 7f0484a00700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f046806c610 0x7f046806eac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f046c005950 tx=0x7f046c0058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.103+0000 7f047e000700 1 -- 172.21.15.115:0/1758762277 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f047405acd0 con 0x7f0480100150 2024-03-22T10:33:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:33:26.330+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0480061190 con 0x7f046806c610 2024-03-22T10:33:26.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:26 smithi150 ceph-mon[34518]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:26.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:33:26.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:33:26.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:26.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:26 smithi115 ceph-mon[29111]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:26.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:33:26.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:33:26.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:27.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:27 smithi115 ceph-mon[29111]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:33:27.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:27 smithi150 ceph-mon[34518]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:33:28.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:28 smithi115 ceph-mon[29111]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:28.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:28 smithi150 ceph-mon[34518]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:30.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:30 smithi150 ceph-mon[34518]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:31.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:30 smithi115 ceph-mon[29111]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:32.876 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:32 smithi115 ceph-mon[29111]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:32.876 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:32.876 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:32.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:32 smithi150 ceph-mon[34518]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:32.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:32.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:33.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:33 smithi115 ceph-mon[29111]: Reconfiguring grafana.smithi115 (dependencies changed)... 2024-03-22T10:33:33.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:33 smithi115 ceph-mon[29111]: Reconfiguring daemon grafana.smithi115 on smithi115 2024-03-22T10:33:33.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:33 smithi150 ceph-mon[34518]: Reconfiguring grafana.smithi115 (dependencies changed)... 2024-03-22T10:33:33.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:33 smithi150 ceph-mon[34518]: Reconfiguring daemon grafana.smithi115 on smithi115 2024-03-22T10:33:34.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:34 smithi150 ceph-mon[34518]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:34.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:34 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/864680773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a34c0f70-4e9d-4044-b7b4-b6099b74ba6c"}]: dispatch 2024-03-22T10:33:34.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:34 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/864680773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a34c0f70-4e9d-4044-b7b4-b6099b74ba6c"}]': finished 2024-03-22T10:33:34.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:34 smithi150 ceph-mon[34518]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T10:33:34.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:34 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:33:35.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:34 smithi115 ceph-mon[29111]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:35.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:34 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/864680773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a34c0f70-4e9d-4044-b7b4-b6099b74ba6c"}]: dispatch 2024-03-22T10:33:35.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:34 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/864680773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a34c0f70-4e9d-4044-b7b4-b6099b74ba6c"}]': finished 2024-03-22T10:33:35.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:34 smithi115 ceph-mon[29111]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T10:33:35.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:34 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:33:36.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:35 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/4232158937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:33:36.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:35 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/4232158937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:33:37.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:36 smithi115 ceph-mon[29111]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:37.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:36 smithi150 ceph-mon[34518]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:38.709 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:38 smithi115 ceph-mon[29111]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:38 smithi150 ceph-mon[34518]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:40.953 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:40 smithi150 ceph-mon[34518]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:40.954 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:40 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:33:41.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:40 smithi115 ceph-mon[29111]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:41.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:33:42.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:42 smithi150 ceph-mon[34518]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:43.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:42 smithi115 ceph-mon[29111]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:44.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:43 smithi150 ceph-mon[34518]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:44.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:43 smithi115 ceph-mon[29111]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:46.135 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:45 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:46.136 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:45 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:46.136 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:45 smithi115 ceph-mon[29111]: Reconfiguring prometheus.smithi115 (dependencies changed)... 2024-03-22T10:33:46.136 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:45 smithi115 ceph-mon[29111]: Reconfiguring daemon prometheus.smithi115 on smithi115 2024-03-22T10:33:46.136 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:45 smithi115 ceph-mon[29111]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:46.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:45 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:46.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:45 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:46.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:45 smithi150 ceph-mon[34518]: Reconfiguring prometheus.smithi115 (dependencies changed)... 2024-03-22T10:33:46.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:45 smithi150 ceph-mon[34518]: Reconfiguring daemon prometheus.smithi115 on smithi115 2024-03-22T10:33:46.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:45 smithi150 ceph-mon[34518]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:46.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T10:33:46.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:46.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:46 smithi115 ceph-mon[29111]: Deploying daemon osd.0 on smithi115 2024-03-22T10:33:47.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:46 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T10:33:47.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:46 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:47.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:46 smithi150 ceph-mon[34518]: Deploying daemon osd.0 on smithi115 2024-03-22T10:33:48.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:47 smithi150 ceph-mon[34518]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:48.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:47 smithi115 ceph-mon[29111]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:50.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:50 smithi115 ceph-mon[29111]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:50.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:50 smithi150 ceph-mon[34518]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:52.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:52 smithi115 ceph-mon[29111]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:52.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:52 smithi150 ceph-mon[34518]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:53.602 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:53.603 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:53.603 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:53 smithi150 ceph-mon[34518]: Reconfiguring ceph-exporter.smithi150 (monmap changed)... 2024-03-22T10:33:53.603 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:33:53.603 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:53.603 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:53 smithi150 ceph-mon[34518]: Reconfiguring daemon ceph-exporter.smithi150 on smithi150 2024-03-22T10:33:53.758 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:53.758 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:53.759 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:53 smithi115 ceph-mon[29111]: Reconfiguring ceph-exporter.smithi150 (monmap changed)... 2024-03-22T10:33:53.759 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:33:53.759 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:53.759 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:53 smithi115 ceph-mon[29111]: Reconfiguring daemon ceph-exporter.smithi150 on smithi150 2024-03-22T10:33:54.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:54 smithi115 ceph-mon[29111]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:54.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:54 smithi150 ceph-mon[34518]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:55.518 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:55.518 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:55.518 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:55 smithi150 ceph-mon[34518]: Reconfiguring crash.smithi150 (monmap changed)... 2024-03-22T10:33:55.518 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi150", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:33:55.518 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:55.518 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:55 smithi150 ceph-mon[34518]: Reconfiguring daemon crash.smithi150 on smithi150 2024-03-22T10:33:55.519 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:33:55.568 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:55.568 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:55.568 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:55 smithi115 ceph-mon[29111]: Reconfiguring crash.smithi150 (monmap changed)... 2024-03-22T10:33:55.569 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi150", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:33:55.569 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:55.569 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:55 smithi115 ceph-mon[29111]: Reconfiguring daemon crash.smithi150 on smithi150 2024-03-22T10:33:55.569 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:33:56.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:56 smithi115 ceph-mon[29111]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:56.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:56 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:56.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:56 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:56.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:56 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:33:56.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:56 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:33:56.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:56 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:56.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:56 smithi150 ceph-mon[34518]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:56.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:56 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:56.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:56 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:56.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:56 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:33:56.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:56 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:33:56.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:56 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:57.540 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:57 smithi150 ceph-mon[34518]: Reconfiguring mgr.smithi150.fmoumv (monmap changed)... 2024-03-22T10:33:57.540 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:57 smithi150 ceph-mon[34518]: Reconfiguring daemon mgr.smithi150.fmoumv on smithi150 2024-03-22T10:33:57.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:57 smithi115 ceph-mon[29111]: Reconfiguring mgr.smithi150.fmoumv (monmap changed)... 2024-03-22T10:33:57.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:57 smithi115 ceph-mon[29111]: Reconfiguring daemon mgr.smithi150.fmoumv on smithi150 2024-03-22T10:33:58.752 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:58 smithi115 ceph-mon[29111]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:58.752 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:58.752 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:58.752 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:33:58.752 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:33:58.752 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:58.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:58 smithi150 ceph-mon[34518]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:33:58.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:58 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:58.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:58 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:33:58.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:58 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:33:58.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:58 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:33:58.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:58 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:33:59.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:59 smithi150 ceph-mon[34518]: Reconfiguring mon.smithi150 (monmap changed)... 2024-03-22T10:33:59.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:33:59 smithi150 ceph-mon[34518]: Reconfiguring daemon mon.smithi150 on smithi150 2024-03-22T10:33:59.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:59 smithi115 ceph-mon[29111]: Reconfiguring mon.smithi150 (monmap changed)... 2024-03-22T10:33:59.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:33:59 smithi115 ceph-mon[29111]: Reconfiguring daemon mon.smithi150 on smithi150 2024-03-22T10:34:00.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:34:00.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi115.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi115.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:00 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T10:34:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi115.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T10:34:00.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:34:00.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi115.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T10:34:00.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:00.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:00 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:34:01.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:01 smithi115 ceph-mon[29111]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T10:34:01.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:01 smithi115 ceph-mon[29111]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T10:34:01.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:01 smithi115 ceph-mon[29111]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T10:34:01.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:01 smithi115 ceph-mon[29111]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi115.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T10:34:01.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:01 smithi115 ceph-mon[29111]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:34:01.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:01 smithi115 ceph-mon[29111]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi115.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T10:34:01.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:01 smithi150 ceph-mon[34518]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T10:34:01.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:01 smithi150 ceph-mon[34518]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T10:34:01.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:01 smithi150 ceph-mon[34518]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T10:34:01.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:01 smithi150 ceph-mon[34518]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi115.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T10:34:01.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:01 smithi150 ceph-mon[34518]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:34:01.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:01 smithi150 ceph-mon[34518]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi115.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T10:34:02.876 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:02 smithi115 ceph-mon[29111]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:34:02.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:02 smithi150 ceph-mon[34518]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:34:03.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.845+0000 7f047e000700 1 -- 172.21.15.115:0/1758762277 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0480061190 con 0x7f046806c610 2024-03-22T10:34:03.840 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 0 on host 'smithi115' 2024-03-22T10:34:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.848+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f046806c610 msgr2=0x7f046806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:34:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.848+0000 7f0487e00700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f046806c610 0x7f046806eac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f046c005950 tx=0x7f046c0058e0 comp rx=0 tx=0).stop 2024-03-22T10:34:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.848+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 msgr2=0x7f0480197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:34:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.848+0000 7f0487e00700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 0x7f0480197b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f047400bda0 tx=0x7f047400be80 comp rx=0 tx=0).stop 2024-03-22T10:34:03.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.848+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 shutdown_connections 2024-03-22T10:34:03.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.848+0000 7f0487e00700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f046806c610 0x7f046806eac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:03.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.848+0000 7f0487e00700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480100150 0x7f0480197b80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:03.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.848+0000 7f0487e00700 1 --2- 172.21.15.115:0/1758762277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0480101350 0x7f04801980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:03.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.848+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 >> 172.21.15.115:0/1758762277 conn(0x7f04800fb6e0 msgr2=0x7f0480102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:34:03.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.849+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 shutdown_connections 2024-03-22T10:34:03.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:03.849+0000 7f0487e00700 1 -- 172.21.15.115:0/1758762277 wait complete. 2024-03-22T10:34:04.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:03 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:04.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:03 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:04.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:03 smithi115 ceph-mon[29111]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:34:04.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:03 smithi115 ceph-mon[29111]: from='osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T10:34:04.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:03 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:04.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:03 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:04.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:03 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:04.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:03 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:04.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:03 smithi150 ceph-mon[34518]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:34:04.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:03 smithi150 ceph-mon[34518]: from='osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T10:34:04.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:03 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:04.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:03 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:04.614 DEBUG:teuthology.orchestra.run.smithi115:osd.0> sudo journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.0.service 2024-03-22T10:34:04.622 INFO:tasks.cephadm:Deploying osd.1 on smithi115 with /dev/vg_nvme/lv_3... 2024-03-22T10:34:04.622 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T10:34:05.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:04 smithi115 ceph-mon[29111]: from='osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T10:34:05.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:04 smithi115 ceph-mon[29111]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T10:34:05.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:04 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:05.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:04 smithi115 ceph-mon[29111]: from='osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:34:05.263 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:34:04 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[43926]: 2024-03-22T10:34:04.966+0000 7f7a1d000700 -1 osd.0 0 waiting for initial osdmap 2024-03-22T10:34:05.263 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:34:04 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[43926]: 2024-03-22T10:34:04.972+0000 7f7a17400700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T10:34:05.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:04 smithi150 ceph-mon[34518]: from='osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T10:34:05.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:04 smithi150 ceph-mon[34518]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T10:34:05.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:04 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:05.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:04 smithi150 ceph-mon[34518]: from='osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:34:05.456 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:34:06.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:05 smithi115 ceph-mon[29111]: from='osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-22T10:34:06.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:05 smithi115 ceph-mon[29111]: osdmap e8: 1 total, 0 up, 1 in 2024-03-22T10:34:06.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:06.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:06.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:05 smithi115 ceph-mon[29111]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:34:06.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:05 smithi150 ceph-mon[34518]: from='osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-22T10:34:06.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:05 smithi150 ceph-mon[34518]: osdmap e8: 1 total, 0 up, 1 in 2024-03-22T10:34:06.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:06.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:06.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:05 smithi150 ceph-mon[34518]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T10:34:07.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:06 smithi150 ceph-mon[34518]: purged_snaps scrub starts 2024-03-22T10:34:07.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:06 smithi150 ceph-mon[34518]: purged_snaps scrub ok 2024-03-22T10:34:07.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:06 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:07.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:06 smithi150 ceph-mon[34518]: osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] boot 2024-03-22T10:34:07.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:06 smithi150 ceph-mon[34518]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T10:34:07.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:06 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:07.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:06 smithi115 ceph-mon[29111]: purged_snaps scrub starts 2024-03-22T10:34:07.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:06 smithi115 ceph-mon[29111]: purged_snaps scrub ok 2024-03-22T10:34:07.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:06 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:07.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:06 smithi115 ceph-mon[29111]: osd.0 [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] boot 2024-03-22T10:34:07.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:06 smithi115 ceph-mon[29111]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T10:34:07.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:06 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:34:08.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:07 smithi115 ceph-mon[29111]: osdmap e10: 1 total, 1 up, 1 in 2024-03-22T10:34:08.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:07 smithi115 ceph-mon[29111]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:08.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:07 smithi150 ceph-mon[34518]: osdmap e10: 1 total, 1 up, 1 in 2024-03-22T10:34:08.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:07 smithi150 ceph-mon[34518]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T10:34:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T10:34:08.949 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-03-22T10:34:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-03-22T10:34:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0353207 s, 297 MB/s 2024-03-22T10:34:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-03-22T10:34:09.794 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2024-03-22T10:34:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:34:10.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:10 smithi115 ceph-mon[29111]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:10.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:10.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:10.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:34:10.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:10 smithi150 ceph-mon[34518]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:10.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:10.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:10.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:34:12.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.338+0000 7f72d0800700 1 -- 172.21.15.115:0/484946619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 msgr2=0x7f72c8100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:34:12.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.338+0000 7f72d0800700 1 --2- 172.21.15.115:0/484946619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 0x7f72c8100560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009910 tx=0x7f72bc009c20 comp rx=0 tx=0).stop 2024-03-22T10:34:12.333 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:12 smithi115 ceph-mon[29111]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:12.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.338+0000 7f72d0800700 1 -- 172.21.15.115:0/484946619 shutdown_connections 2024-03-22T10:34:12.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.338+0000 7f72d0800700 1 --2- 172.21.15.115:0/484946619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72c8101350 0x7f72c81017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:12.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.338+0000 7f72d0800700 1 --2- 172.21.15.115:0/484946619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 0x7f72c8100560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:12.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.338+0000 7f72d0800700 1 -- 172.21.15.115:0/484946619 >> 172.21.15.115:0/484946619 conn(0x7f72c80fb6e0 msgr2=0x7f72c80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:34:12.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.338+0000 7f72d0800700 1 -- 172.21.15.115:0/484946619 shutdown_connections 2024-03-22T10:34:12.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.339+0000 7f72d0800700 1 -- 172.21.15.115:0/484946619 wait complete. 2024-03-22T10:34:12.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.340+0000 7f72d0800700 1 Processor -- start 2024-03-22T10:34:12.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.340+0000 7f72d0800700 1 -- start start 2024-03-22T10:34:12.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.340+0000 7f72d0800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 0x7f72c81959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:34:12.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.340+0000 7f72d0800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72c8101350 0x7f72c8195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:34:12.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.340+0000 7f72d0800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c8196500 con 0x7f72c8100150 2024-03-22T10:34:12.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.340+0000 7f72d0800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c8196640 con 0x7f72c8101350 2024-03-22T10:34:12.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.341+0000 7f72cde00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 0x7f72c81959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:34:12.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.341+0000 7f72cde00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 0x7f72c81959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47478/0 (socket says 172.21.15.115:47478) 2024-03-22T10:34:12.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.341+0000 7f72cde00700 1 -- 172.21.15.115:0/2229647347 learned_addr learned my addr 172.21.15.115:0/2229647347 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:34:12.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.341+0000 7f72cd400700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72c8101350 0x7f72c8195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:34:12.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.341+0000 7f72cde00700 1 -- 172.21.15.115:0/2229647347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72c8101350 msgr2=0x7f72c8195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:34:12.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.341+0000 7f72cde00700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72c8101350 0x7f72c8195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:12.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.341+0000 7f72cde00700 1 -- 172.21.15.115:0/2229647347 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0095f0 con 0x7f72c8100150 2024-03-22T10:34:12.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.342+0000 7f72cde00700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 0x7f72c81959a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005b40 tx=0x7f72bc00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:34:12.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.342+0000 7f72c2a00700 1 -- 172.21.15.115:0/2229647347 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72bc01d070 con 0x7f72c8100150 2024-03-22T10:34:12.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.342+0000 7f72c2a00700 1 -- 172.21.15.115:0/2229647347 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f72bc005dc0 con 0x7f72c8100150 2024-03-22T10:34:12.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.342+0000 7f72c2a00700 1 -- 172.21.15.115:0/2229647347 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72bc0226a0 con 0x7f72c8100150 2024-03-22T10:34:12.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.342+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c819b090 con 0x7f72c8100150 2024-03-22T10:34:12.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.342+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c819b580 con 0x7f72c8100150 2024-03-22T10:34:12.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.343+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c8108ca0 con 0x7f72c8100150 2024-03-22T10:34:12.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.344+0000 7f72c2a00700 1 -- 172.21.15.115:0/2229647347 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f72bc003680 con 0x7f72c8100150 2024-03-22T10:34:12.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.345+0000 7f72c2a00700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f72b006c610 0x7f72b006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:34:12.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.345+0000 7f72c2a00700 1 -- 172.21.15.115:0/2229647347 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f72bc08bf80 con 0x7f72c8100150 2024-03-22T10:34:12.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.345+0000 7f72cd400700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f72b006c610 0x7f72b006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:34:12.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.346+0000 7f72cd400700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f72b006c610 0x7f72b006eac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009bf0 tx=0x7f72b8009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:34:12.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.350+0000 7f72c2a00700 1 -- 172.21.15.115:0/2229647347 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f72bc05af50 con 0x7f72c8100150 2024-03-22T10:34:12.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:12.585+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f72c8105ca0 con 0x7f72b006c610 2024-03-22T10:34:12.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:12 smithi150 ceph-mon[34518]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:13.403 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:13 smithi115 ceph-mon[29111]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:34:13.403 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:13 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:34:13.403 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:13 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:34:13.404 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:13 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:13.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:13 smithi150 ceph-mon[34518]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:34:13.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:13 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:34:13.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:13 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:34:13.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:13 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:14.521 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:14 smithi115 ceph-mon[29111]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:14.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:14 smithi150 ceph-mon[34518]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:16.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:16.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: Detected new or changed devices on smithi115 2024-03-22T10:34:16.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:16.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:16.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:34:16.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: Adjusting osd_memory_target on smithi115 to 11843M 2024-03-22T10:34:16.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:16.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:16.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:34:16.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:16.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:34:16.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:16.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: Detected new or changed devices on smithi115 2024-03-22T10:34:16.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:16.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:16.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:34:16.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: Adjusting osd_memory_target on smithi115 to 11843M 2024-03-22T10:34:16.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:16.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:16.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:34:16.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:16.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:34:17.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:17 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1382461794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1037db4c-35bc-48da-8950-460c772d50b3"}]: dispatch 2024-03-22T10:34:17.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:17 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1382461794' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1037db4c-35bc-48da-8950-460c772d50b3"}]': finished 2024-03-22T10:34:17.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:17 smithi150 ceph-mon[34518]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T10:34:17.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:17 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:17.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:17 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1382461794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1037db4c-35bc-48da-8950-460c772d50b3"}]: dispatch 2024-03-22T10:34:17.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:17 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1382461794' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1037db4c-35bc-48da-8950-460c772d50b3"}]': finished 2024-03-22T10:34:17.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:17 smithi115 ceph-mon[29111]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T10:34:17.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:17 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:18.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:18 smithi150 ceph-mon[34518]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:18.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:18 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/4008484476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:34:18.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:18 smithi115 ceph-mon[29111]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:18.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:18 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/4008484476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:34:19.728 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:19 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:19.728 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:19 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:19.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:19 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:19.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:19 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:20.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:20 smithi115 ceph-mon[29111]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:20.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:20 smithi150 ceph-mon[34518]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:22.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:22 smithi150 ceph-mon[34518]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:22.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:22 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:22.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:22 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:22.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:22 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:22.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:22 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:34:22.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:22 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:22.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:22 smithi115 ceph-mon[29111]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:22.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:22 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:22.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:22 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:22.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:22 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:22.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:22 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:34:22.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:22 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:24.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:23 smithi150 ceph-mon[34518]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:24.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:23 smithi115 ceph-mon[29111]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:25.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:34:25.754 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:34:26.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:26 smithi115 ceph-mon[29111]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:26.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T10:34:26.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:26 smithi150 ceph-mon[34518]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T10:34:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:27.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:27 smithi115 ceph-mon[29111]: Deploying daemon osd.1 on smithi115 2024-03-22T10:34:27.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:27 smithi150 ceph-mon[34518]: Deploying daemon osd.1 on smithi115 2024-03-22T10:34:28.618 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:28 smithi115 ceph-mon[29111]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:28.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:28 smithi150 ceph-mon[34518]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:30.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:30 smithi115 ceph-mon[29111]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:30.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:30 smithi150 ceph-mon[34518]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:32.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:32 smithi150 ceph-mon[34518]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:32.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:32 smithi115 ceph-mon[29111]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:34.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:33 smithi115 ceph-mon[29111]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:34.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:33 smithi150 ceph-mon[34518]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:35.438 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:35.438 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:34:35.438 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:35.600 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:35.600 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:34:35.600 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:36.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:36 smithi115 ceph-mon[29111]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:36.590 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:36 smithi150 ceph-mon[34518]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:38.167 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:38 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:38.167 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:38 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:38.168 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:38 smithi115 ceph-mon[29111]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:38.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:38 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:38.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:38 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:38.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:38 smithi150 ceph-mon[34518]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.119+0000 7f72c2a00700 1 -- 172.21.15.115:0/2229647347 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f72c8105ca0 con 0x7f72b006c610 2024-03-22T10:34:39.114 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 1 on host 'smithi115' 2024-03-22T10:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.123+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f72b006c610 msgr2=0x7f72b006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.123+0000 7f72d0800700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f72b006c610 0x7f72b006eac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009bf0 tx=0x7f72b8009260 comp rx=0 tx=0).stop 2024-03-22T10:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.124+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 msgr2=0x7f72c81959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.124+0000 7f72d0800700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 0x7f72c81959a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005b40 tx=0x7f72bc00fab0 comp rx=0 tx=0).stop 2024-03-22T10:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.124+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 shutdown_connections 2024-03-22T10:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.124+0000 7f72d0800700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f72b006c610 0x7f72b006eac0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.124+0000 7f72d0800700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72c8100150 0x7f72c81959a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.124+0000 7f72d0800700 1 --2- 172.21.15.115:0/2229647347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72c8101350 0x7f72c8195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.124+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 >> 172.21.15.115:0/2229647347 conn(0x7f72c80fb6e0 msgr2=0x7f72c8104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:34:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.124+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 shutdown_connections 2024-03-22T10:34:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:39.124+0000 7f72d0800700 1 -- 172.21.15.115:0/2229647347 wait complete. 2024-03-22T10:34:39.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:39 smithi150 ceph-mon[34518]: from='osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T10:34:39.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:39 smithi115 ceph-mon[29111]: from='osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T10:34:40.251 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:40 smithi115 ceph-mon[29111]: from='osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T10:34:40.251 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:40 smithi115 ceph-mon[29111]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T10:34:40.251 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:40.252 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:40 smithi115 ceph-mon[29111]: from='osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:34:40.252 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:40.252 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:40.252 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:40 smithi115 ceph-mon[29111]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:40.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:40 smithi150 ceph-mon[34518]: from='osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T10:34:40.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:40 smithi150 ceph-mon[34518]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T10:34:40.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:40 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:40.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:40 smithi150 ceph-mon[34518]: from='osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:34:40.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:40 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:40.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:40 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:40.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:40 smithi150 ceph-mon[34518]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:40.581 DEBUG:teuthology.orchestra.run.smithi115:osd.1> sudo journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.1.service 2024-03-22T10:34:40.587 INFO:tasks.cephadm:Deploying osd.2 on smithi115 with /dev/vg_nvme/lv_2... 2024-03-22T10:34:40.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T10:34:41.054 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:34:41.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:41 smithi115 ceph-mon[29111]: from='osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-22T10:34:41.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:41 smithi115 ceph-mon[29111]: osdmap e13: 2 total, 1 up, 2 in 2024-03-22T10:34:41.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:41.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:41.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:34:41.328 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:41 smithi150 ceph-mon[34518]: from='osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-22T10:34:41.328 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:41 smithi150 ceph-mon[34518]: osdmap e13: 2 total, 1 up, 2 in 2024-03-22T10:34:41.328 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:41.328 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:41.328 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:34:42.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:42 smithi115 ceph-mon[29111]: purged_snaps scrub starts 2024-03-22T10:34:42.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:42 smithi115 ceph-mon[29111]: purged_snaps scrub ok 2024-03-22T10:34:42.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:42 smithi115 ceph-mon[29111]: osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] boot 2024-03-22T10:34:42.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:42 smithi115 ceph-mon[29111]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T10:34:42.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:42 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:42.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:42 smithi115 ceph-mon[29111]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:42.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:42 smithi150 ceph-mon[34518]: purged_snaps scrub starts 2024-03-22T10:34:42.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:42 smithi150 ceph-mon[34518]: purged_snaps scrub ok 2024-03-22T10:34:42.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:42 smithi150 ceph-mon[34518]: osd.1 [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] boot 2024-03-22T10:34:42.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:42 smithi150 ceph-mon[34518]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T10:34:42.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:42 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:34:42.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:42 smithi150 ceph-mon[34518]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-22T10:34:43.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:43 smithi150 ceph-mon[34518]: osdmap e15: 2 total, 2 up, 2 in 2024-03-22T10:34:43.474 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:43 smithi115 ceph-mon[29111]: osdmap e15: 2 total, 2 up, 2 in 2024-03-22T10:34:44.334 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:44 smithi115 ceph-mon[29111]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:44.334 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:44 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:44.334 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:44 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:44.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:44 smithi150 ceph-mon[34518]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:44.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:44 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:44.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:44 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:44.457 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T10:34:44.457 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T10:34:44.457 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-03-22T10:34:44.457 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-03-22T10:34:44.457 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0357161 s, 294 MB/s 2024-03-22T10:34:44.457 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-03-22T10:34:45.223 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2024-03-22T10:34:46.461 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:46 smithi115 ceph-mon[29111]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:46.480 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:34:46.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:46 smithi150 ceph-mon[34518]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:48.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:48 smithi150 ceph-mon[34518]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:48.743 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:48 smithi115 ceph-mon[29111]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:49.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.386+0000 7f6d53c00700 1 -- 172.21.15.115:0/2867107438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 msgr2=0x7f6d4c1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:34:49.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.386+0000 7f6d53c00700 1 --2- 172.21.15.115:0/2867107438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 0x7f6d4c1017a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009910 tx=0x7f6d38009c20 comp rx=0 tx=0).stop 2024-03-22T10:34:49.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.386+0000 7f6d53c00700 1 -- 172.21.15.115:0/2867107438 shutdown_connections 2024-03-22T10:34:49.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.386+0000 7f6d53c00700 1 --2- 172.21.15.115:0/2867107438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 0x7f6d4c1017a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:49.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.386+0000 7f6d53c00700 1 --2- 172.21.15.115:0/2867107438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d4c100150 0x7f6d4c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:49.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.386+0000 7f6d53c00700 1 -- 172.21.15.115:0/2867107438 >> 172.21.15.115:0/2867107438 conn(0x7f6d4c0fb6e0 msgr2=0x7f6d4c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:34:49.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.386+0000 7f6d53c00700 1 -- 172.21.15.115:0/2867107438 shutdown_connections 2024-03-22T10:34:49.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.387+0000 7f6d53c00700 1 -- 172.21.15.115:0/2867107438 wait complete. 2024-03-22T10:34:49.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.389+0000 7f6d53c00700 1 Processor -- start 2024-03-22T10:34:49.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.389+0000 7f6d53c00700 1 -- start start 2024-03-22T10:34:49.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.389+0000 7f6d53c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d4c100150 0x7f6d4c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:34:49.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.389+0000 7f6d53c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 0x7f6d4c073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:34:49.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.389+0000 7f6d53c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c073680 con 0x7f6d4c101350 2024-03-22T10:34:49.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.389+0000 7f6d53c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0737c0 con 0x7f6d4c100150 2024-03-22T10:34:49.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.390+0000 7f6d4be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 0x7f6d4c073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:34:49.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.390+0000 7f6d4be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 0x7f6d4c073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48876/0 (socket says 172.21.15.115:48876) 2024-03-22T10:34:49.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.390+0000 7f6d4be00700 1 -- 172.21.15.115:0/886746849 learned_addr learned my addr 172.21.15.115:0/886746849 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:34:49.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.390+0000 7f6d51200700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d4c100150 0x7f6d4c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:34:49.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.390+0000 7f6d4be00700 1 -- 172.21.15.115:0/886746849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d4c100150 msgr2=0x7f6d4c074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:34:49.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.390+0000 7f6d4be00700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d4c100150 0x7f6d4c074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:34:49.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.390+0000 7f6d4be00700 1 -- 172.21.15.115:0/886746849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380095f0 con 0x7f6d4c101350 2024-03-22T10:34:49.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.391+0000 7f6d4be00700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 0x7f6d4c073140 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800bd70 tx=0x7f6d3800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:34:49.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.391+0000 7f6d49600700 1 -- 172.21.15.115:0/886746849 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d3801d070 con 0x7f6d4c101350 2024-03-22T10:34:49.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.391+0000 7f6d49600700 1 -- 172.21.15.115:0/886746849 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6d38005dc0 con 0x7f6d4c101350 2024-03-22T10:34:49.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.391+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c073a40 con 0x7f6d4c101350 2024-03-22T10:34:49.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.391+0000 7f6d49600700 1 -- 172.21.15.115:0/886746849 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d38021410 con 0x7f6d4c101350 2024-03-22T10:34:49.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.391+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c073f30 con 0x7f6d4c101350 2024-03-22T10:34:49.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.393+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c108ca0 con 0x7f6d4c101350 2024-03-22T10:34:49.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.394+0000 7f6d49600700 1 -- 172.21.15.115:0/886746849 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f6d3800f7d0 con 0x7f6d4c101350 2024-03-22T10:34:49.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.395+0000 7f6d49600700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d3406c600 0x7f6d3406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:34:49.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.395+0000 7f6d49600700 1 -- 172.21.15.115:0/886746849 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f6d3808bef0 con 0x7f6d4c101350 2024-03-22T10:34:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.395+0000 7f6d51200700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d3406c600 0x7f6d3406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:34:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.396+0000 7f6d51200700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d3406c600 0x7f6d3406eab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c1011b0 tx=0x7f6d4000b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:34:49.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.404+0000 7f6d49600700 1 -- 172.21.15.115:0/886746849 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6d3805ad30 con 0x7f6d4c101350 2024-03-22T10:34:49.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:34:49.631+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d4c105ca0 con 0x7f6d3406c600 2024-03-22T10:34:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: Detected new or changed devices on smithi115 2024-03-22T10:34:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:34:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: Adjusting osd_memory_target on smithi115 to 5921M 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:34:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: Detected new or changed devices on smithi115 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: Adjusting osd_memory_target on smithi115 to 5921M 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:34:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:34:50.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:34:50.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:52.320 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:52 smithi150 ceph-mon[34518]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:52.321 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:52 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:52.321 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:52 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:52.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:52 smithi115 ceph-mon[29111]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:52.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:52 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:52.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:52 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:54.214 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:53 smithi115 ceph-mon[29111]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:54.215 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:53 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/4136314071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e685a22-30ee-4404-bba5-ff944af9b858"}]: dispatch 2024-03-22T10:34:54.215 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:53 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/4136314071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e685a22-30ee-4404-bba5-ff944af9b858"}]': finished 2024-03-22T10:34:54.215 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:53 smithi115 ceph-mon[29111]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T10:34:54.215 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:34:54.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:53 smithi150 ceph-mon[34518]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:54.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:53 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/4136314071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e685a22-30ee-4404-bba5-ff944af9b858"}]: dispatch 2024-03-22T10:34:54.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:53 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/4136314071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e685a22-30ee-4404-bba5-ff944af9b858"}]': finished 2024-03-22T10:34:54.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:53 smithi150 ceph-mon[34518]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T10:34:54.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:34:55.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:54 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1928472666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:34:55.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:54 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1928472666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:34:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:55 smithi115 ceph-mon[29111]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:34:56.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:55 smithi150 ceph-mon[34518]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:56.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:34:57.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:57.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:57.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:57.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:34:57.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:57.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:57.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:57.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:34:57.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:34:57.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:34:58.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:34:58 smithi115 ceph-mon[29111]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:34:58.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:34:58 smithi150 ceph-mon[34518]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:00.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:00 smithi115 ceph-mon[29111]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:00.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:00 smithi150 ceph-mon[34518]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:02.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:02 smithi150 ceph-mon[34518]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:02.468 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:02 smithi115 ceph-mon[29111]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:03.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:03 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T10:35:03.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:03 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:03.655 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:03 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T10:35:03.655 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:03 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:04.531 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:04 smithi115 ceph-mon[29111]: Deploying daemon osd.2 on smithi115 2024-03-22T10:35:04.531 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:04 smithi115 ceph-mon[29111]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:04 smithi150 ceph-mon[34518]: Deploying daemon osd.2 on smithi115 2024-03-22T10:35:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:04 smithi150 ceph-mon[34518]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:06.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:06 smithi150 ceph-mon[34518]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:06.750 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:06 smithi115 ceph-mon[29111]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:08.388 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:08 smithi115 ceph-mon[29111]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:08.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:08 smithi150 ceph-mon[34518]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:10.559 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:10 smithi115 ceph-mon[29111]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:10.559 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:35:10.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:10 smithi150 ceph-mon[34518]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:10.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:35:11.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:11 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:11.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:11 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:11.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:11 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:35:11.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:11 smithi115 ceph-mon[29111]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:12.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:11 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:12.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:11 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:12.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:11 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:35:12.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:11 smithi150 ceph-mon[34518]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:14.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:13 smithi115 ceph-mon[29111]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:14.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:13 smithi150 ceph-mon[34518]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:15.205 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 2 on host 'smithi115' 2024-03-22T10:35:15.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.211+0000 7f6d49600700 1 -- 172.21.15.115:0/886746849 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6d4c105ca0 con 0x7f6d3406c600 2024-03-22T10:35:15.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.213+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d3406c600 msgr2=0x7f6d3406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:35:15.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.213+0000 7f6d53c00700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d3406c600 0x7f6d3406eab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c1011b0 tx=0x7f6d4000b410 comp rx=0 tx=0).stop 2024-03-22T10:35:15.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.213+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 msgr2=0x7f6d4c073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:35:15.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.213+0000 7f6d53c00700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 0x7f6d4c073140 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800bd70 tx=0x7f6d3800bda0 comp rx=0 tx=0).stop 2024-03-22T10:35:15.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.214+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 shutdown_connections 2024-03-22T10:35:15.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.214+0000 7f6d53c00700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d3406c600 0x7f6d3406eab0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:15.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.214+0000 7f6d53c00700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d4c100150 0x7f6d4c074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:15.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.214+0000 7f6d53c00700 1 --2- 172.21.15.115:0/886746849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d4c101350 0x7f6d4c073140 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:15.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.214+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 >> 172.21.15.115:0/886746849 conn(0x7f6d4c0fb6e0 msgr2=0x7f6d4c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:35:15.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.214+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 shutdown_connections 2024-03-22T10:35:15.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:35:15.214+0000 7f6d53c00700 1 -- 172.21.15.115:0/886746849 wait complete. 2024-03-22T10:35:15.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:15 smithi150 ceph-mon[34518]: from='osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T10:35:15.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:15 smithi115 ceph-mon[29111]: from='osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T10:35:16.219 DEBUG:teuthology.orchestra.run.smithi115:osd.2> sudo journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.2.service 2024-03-22T10:35:16.224 INFO:tasks.cephadm:Deploying osd.3 on smithi150 with /dev/vg_nvme/lv_4... 2024-03-22T10:35:16.224 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T10:35:16.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:16 smithi150 ceph-mon[34518]: from='osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T10:35:16.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:16 smithi150 ceph-mon[34518]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T10:35:16.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:35:16.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:16 smithi150 ceph-mon[34518]: from='osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:35:16.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:16.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:16.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:16 smithi150 ceph-mon[34518]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:16.478 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:16 smithi115 ceph-mon[29111]: from='osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T10:35:16.478 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:16 smithi115 ceph-mon[29111]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T10:35:16.478 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:35:16.478 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:16 smithi115 ceph-mon[29111]: from='osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:35:16.478 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:16.478 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:16.478 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:16 smithi115 ceph-mon[29111]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-22T10:35:16.613 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi150/config 2024-03-22T10:35:17.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:17 smithi150 ceph-mon[34518]: from='osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-22T10:35:17.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:17 smithi150 ceph-mon[34518]: osdmap e18: 3 total, 2 up, 3 in 2024-03-22T10:35:17.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:17 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:35:17.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:17 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:35:17.492 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:17 smithi115 ceph-mon[29111]: from='osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-22T10:35:17.492 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:17 smithi115 ceph-mon[29111]: osdmap e18: 3 total, 2 up, 3 in 2024-03-22T10:35:17.492 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:17 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:35:17.492 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:17 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:35:18.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:18 smithi150 ceph-mon[34518]: purged_snaps scrub starts 2024-03-22T10:35:18.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:18 smithi150 ceph-mon[34518]: purged_snaps scrub ok 2024-03-22T10:35:18.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:18 smithi150 ceph-mon[34518]: osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] boot 2024-03-22T10:35:18.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:18 smithi150 ceph-mon[34518]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T10:35:18.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:35:18.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:18 smithi150 ceph-mon[34518]: pgmap v93: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:18.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-22T10:35:18.419 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:18 smithi115 ceph-mon[29111]: purged_snaps scrub starts 2024-03-22T10:35:18.419 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:18 smithi115 ceph-mon[29111]: purged_snaps scrub ok 2024-03-22T10:35:18.419 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:18 smithi115 ceph-mon[29111]: osd.2 [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] boot 2024-03-22T10:35:18.419 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:18 smithi115 ceph-mon[29111]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T10:35:18.419 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:35:18.419 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:18 smithi115 ceph-mon[29111]: pgmap v93: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:18.419 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-22T10:35:19.171 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T10:35:19.171 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T10:35:19.171 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-03-22T10:35:19.171 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-03-22T10:35:19.171 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0363287 s, 289 MB/s 2024-03-22T10:35:19.171 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 2024-03-22T10:35:19.171 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-03-22T10:35:19.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:19 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-22T10:35:19.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:19 smithi150 ceph-mon[34518]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T10:35:19.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:19 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T10:35:19.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:19 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:19.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:19 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:19 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-22T10:35:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:19 smithi115 ceph-mon[29111]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T10:35:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:19 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T10:35:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:19 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:19 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:19.898 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-03-22T10:35:20.307 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi150/config 2024-03-22T10:35:20.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:20 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T10:35:20.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:20 smithi150 ceph-mon[34518]: osdmap e21: 3 total, 3 up, 3 in 2024-03-22T10:35:20.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:20 smithi150 ceph-mon[34518]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:20.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:20 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T10:35:20.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:20 smithi115 ceph-mon[29111]: osdmap e21: 3 total, 3 up, 3 in 2024-03-22T10:35:20.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:20 smithi115 ceph-mon[29111]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:20.519 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:35:20 smithi115 sudo[56855]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T10:35:20.519 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:35:20 smithi115 sudo[56855]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:20.519 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:35:20 smithi115 sudo[56855]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:20.519 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:35:20 smithi115 sudo[56855]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:21.012 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:35:20 smithi115 sudo[56941]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T10:35:21.013 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:35:20 smithi115 sudo[56941]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:21.013 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:35:20 smithi115 sudo[56941]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:21.013 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:35:20 smithi115 sudo[56941]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:21.342 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:21 smithi115 ceph-mon[29111]: osdmap e22: 3 total, 3 up, 3 in 2024-03-22T10:35:21.342 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:21 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:21.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:21 smithi150 ceph-mon[34518]: osdmap e22: 3 total, 3 up, 3 in 2024-03-22T10:35:21.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:21 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:21.598 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:35:21 smithi115 sudo[56962]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T10:35:21.598 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:35:21 smithi115 sudo[56962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:21.598 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:35:21 smithi115 sudo[56962]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:21.598 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:35:21 smithi115 sudo[56962]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:22.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:22 smithi115 ceph-mon[29111]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:22.262 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:35:21 smithi115 sudo[56968]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T10:35:22.262 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:35:21 smithi115 sudo[56968]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:22.262 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:35:21 smithi115 sudo[56968]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:22.262 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:35:21 smithi115 sudo[56968]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:22.318 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:22 smithi150 ceph-mon[34518]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:22.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.495+0000 7fa2cc000700 1 -- 172.21.15.150:0/3928340945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 msgr2=0x7fa2c40731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:35:22.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.495+0000 7fa2cc000700 1 --2- 172.21.15.150:0/3928340945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 0x7fa2c40731e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800b0a0 tx=0x7fa2b800b3b0 comp rx=0 tx=0).stop 2024-03-22T10:35:22.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.496+0000 7fa2cc000700 1 -- 172.21.15.150:0/3928340945 shutdown_connections 2024-03-22T10:35:22.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.496+0000 7fa2cc000700 1 --2- 172.21.15.150:0/3928340945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c40737b0 0x7fa2c4073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:22.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.496+0000 7fa2cc000700 1 --2- 172.21.15.150:0/3928340945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 0x7fa2c40731e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:22.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.496+0000 7fa2cc000700 1 -- 172.21.15.150:0/3928340945 >> 172.21.15.150:0/3928340945 conn(0x7fa2c40fb6e0 msgr2=0x7fa2c40fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:35:22.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.496+0000 7fa2cc000700 1 -- 172.21.15.150:0/3928340945 shutdown_connections 2024-03-22T10:35:22.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.496+0000 7fa2cc000700 1 -- 172.21.15.150:0/3928340945 wait complete. 2024-03-22T10:35:22.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.497+0000 7fa2cc000700 1 Processor -- start 2024-03-22T10:35:22.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.497+0000 7fa2cc000700 1 -- start start 2024-03-22T10:35:22.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.498+0000 7fa2cc000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c40737b0 0x7fa2c4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:35:22.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.498+0000 7fa2cc000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 0x7fa2c41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:35:22.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.498+0000 7fa2cc000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41986e0 con 0x7fa2c40737b0 2024-03-22T10:35:22.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.498+0000 7fa2cc000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4198820 con 0x7fa2c4074d80 2024-03-22T10:35:22.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.498+0000 7fa2c8c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 0x7fa2c41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:35:22.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.498+0000 7fa2c8c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 0x7fa2c41980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46598/0 (socket says 172.21.15.150:46598) 2024-03-22T10:35:22.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.498+0000 7fa2c8c00700 1 -- 172.21.15.150:0/3041363009 learned_addr learned my addr 172.21.15.150:0/3041363009 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:35:22.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.498+0000 7fa2c9600700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c40737b0 0x7fa2c4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.499+0000 7fa2c8c00700 1 -- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c40737b0 msgr2=0x7fa2c4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.499+0000 7fa2c8c00700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c40737b0 0x7fa2c4197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.499+0000 7fa2c8c00700 1 -- 172.21.15.150:0/3041363009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8009d00 con 0x7fa2c4074d80 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.499+0000 7fa2c8c00700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 0x7fa2c41980c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000d8d0 tx=0x7fa2b000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.499+0000 7fa2c2000700 1 -- 172.21.15.150:0/3041363009 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2b00098a0 con 0x7fa2c4074d80 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.499+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c419d2d0 con 0x7fa2c4074d80 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.499+0000 7fa2c2000700 1 -- 172.21.15.150:0/3041363009 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa2b0010460 con 0x7fa2c4074d80 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.499+0000 7fa2c2000700 1 -- 172.21.15.150:0/3041363009 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2b000f5f0 con 0x7fa2c4074d80 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.499+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c419d8a0 con 0x7fa2c4074d80 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.501+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c410ae90 con 0x7fa2c4074d80 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.501+0000 7fa2c2000700 1 -- 172.21.15.150:0/3041363009 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa2b0009a00 con 0x7fa2c4074d80 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.502+0000 7fa2c2000700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa2ac06c680 0x7fa2ac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:35:22.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.502+0000 7fa2c2000700 1 -- 172.21.15.150:0/3041363009 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3232+0+0 (secure 0 0 0) 0x7fa2b008af20 con 0x7fa2c4074d80 2024-03-22T10:35:22.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.503+0000 7fa2c9600700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa2ac06c680 0x7fa2ac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:35:22.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.503+0000 7fa2c9600700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa2ac06c680 0x7fa2ac06eb30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8000f80 tx=0x7fa2b800b7b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:35:22.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.508+0000 7fa2c2000700 1 -- 172.21.15.150:0/3041363009 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa2b00598d0 con 0x7fa2c4074d80 2024-03-22T10:35:22.632 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:35:22 smithi115 sudo[56992]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T10:35:22.632 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:35:22 smithi115 sudo[56992]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:22.632 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:35:22 smithi115 sudo[56992]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:22.632 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:35:22 smithi115 sudo[56992]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:22.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:22.729+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2c4107650 con 0x7fa2ac06c680 2024-03-22T10:35:23.322 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:35:23 smithi115 sudo[57009]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T10:35:23.322 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:35:23 smithi115 sudo[57009]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:23.322 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:35:23 smithi115 sudo[57009]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:23.322 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:35:23 smithi115 sudo[57009]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:24.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:23 smithi115 sudo[57358]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T10:35:24.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:23 smithi115 sudo[57358]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:24.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:23 smithi115 sudo[57358]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:24.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:23 smithi115 sudo[57358]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:24.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:24 smithi115 sudo[57361]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T10:35:24.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:24 smithi115 sudo[57361]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:24.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:24 smithi115 sudo[57361]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:24.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:24 smithi115 sudo[57361]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:24.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:24 smithi115 ceph-mon[29111]: mgrmap e19: smithi115.iqzwsz(active, since 2m), standbys: smithi150.fmoumv 2024-03-22T10:35:24.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:24 smithi115 ceph-mon[29111]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:35:24.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:24 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:35:24.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:24 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:35:24.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:24 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:24.591 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:24 smithi150 ceph-mon[34518]: mgrmap e19: smithi115.iqzwsz(active, since 2m), standbys: smithi150.fmoumv 2024-03-22T10:35:24.591 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:24 smithi150 ceph-mon[34518]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:35:24.591 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:24 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:35:24.591 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:24 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:35:24.591 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:24 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:24.884 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:24 smithi150 sudo[38276]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T10:35:24.884 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:24 smithi150 sudo[38276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:24.884 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:24 smithi150 sudo[38276]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:24.885 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:24 smithi150 sudo[38276]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:25.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 sudo[38293]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T10:35:25.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 sudo[38293]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T10:35:25.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 sudo[38293]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-22T10:35:25.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 sudo[38293]: pam_unix(sudo:session): session closed for user root 2024-03-22T10:35:25.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 ceph-mon[34518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T10:35:25.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 ceph-mon[34518]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:25.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 ceph-mon[34518]: Detected new or changed devices on smithi115 2024-03-22T10:35:25.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 ceph-mon[34518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T10:35:25.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:35:25.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:35:25.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:25.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:35:25.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:35:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:25 smithi115 ceph-mon[29111]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T10:35:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:25 smithi115 ceph-mon[29111]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:25 smithi115 ceph-mon[29111]: Detected new or changed devices on smithi115 2024-03-22T10:35:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:25 smithi115 ceph-mon[29111]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T10:35:25.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:35:25.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:35:25.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:25.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:35:25.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:35:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T10:35:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:35:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T10:35:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:35:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:35:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:35:26.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: Adjusting osd_memory_target on smithi115 to 3947M 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='client.? 172.21.15.150:0/1012496908' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1273664e-16c8-46c1-bfb4-c51e3e84ea6b"}]: dispatch 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1273664e-16c8-46c1-bfb4-c51e3e84ea6b"}]: dispatch 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1273664e-16c8-46c1-bfb4-c51e3e84ea6b"}]': finished 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: osdmap e23: 4 total, 3 up, 4 in 2024-03-22T10:35:26.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:26 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: Adjusting osd_memory_target on smithi115 to 3947M 2024-03-22T10:35:26.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/1012496908' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1273664e-16c8-46c1-bfb4-c51e3e84ea6b"}]: dispatch 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1273664e-16c8-46c1-bfb4-c51e3e84ea6b"}]: dispatch 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1273664e-16c8-46c1-bfb4-c51e3e84ea6b"}]': finished 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: osdmap e23: 4 total, 3 up, 4 in 2024-03-22T10:35:26.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:26 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:27.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:27 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/519385966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:35:27.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:27 smithi150 ceph-mon[34518]: from='client.? 172.21.15.150:0/519385966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:35:28.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:28 smithi115 ceph-mon[29111]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-22T10:35:28.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:28 smithi150 ceph-mon[34518]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-22T10:35:30.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:30 smithi115 ceph-mon[29111]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-22T10:35:30.886 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:30 smithi150 ceph-mon[34518]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-22T10:35:32.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:32 smithi150 ceph-mon[34518]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-22T10:35:32.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:32.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:32.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:32.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:35:32.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:32.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:32 smithi115 ceph-mon[29111]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-22T10:35:32.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:32.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:32.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:32.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:35:32.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:34.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:33 smithi150 ceph-mon[34518]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-22T10:35:34.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:33 smithi115 ceph-mon[29111]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-22T10:35:35.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T10:35:35.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:35.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T10:35:35.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:36.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:36 smithi150 ceph-mon[34518]: Deploying daemon osd.3 on smithi150 2024-03-22T10:35:36.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:36 smithi150 ceph-mon[34518]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-22T10:35:36.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:36 smithi115 ceph-mon[29111]: Deploying daemon osd.3 on smithi150 2024-03-22T10:35:36.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:36 smithi115 ceph-mon[29111]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-22T10:35:38.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:38 smithi150 ceph-mon[34518]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 71 KiB/s wr, 5 op/s 2024-03-22T10:35:38.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:38 smithi115 ceph-mon[29111]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 71 KiB/s wr, 5 op/s 2024-03-22T10:35:40.480 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:40 smithi150 ceph-mon[34518]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:40.481 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:40 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:35:40.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:40 smithi115 ceph-mon[29111]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:40.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:35:42.635 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:42 smithi150 ceph-mon[34518]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:42.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:42 smithi115 ceph-mon[29111]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:44.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:44.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:44.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:35:44.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:43 smithi115 ceph-mon[29111]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:44.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:44.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:44.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:35:44.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:43 smithi150 ceph-mon[34518]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:46.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:46 smithi150 ceph-mon[34518]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:46 smithi115 ceph-mon[29111]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:47.554 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:47 smithi150 ceph-mon[34518]: from='osd.3 [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T10:35:47.554 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:47 smithi150 ceph-mon[34518]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T10:35:47.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.678+0000 7fa2c2000700 1 -- 172.21.15.150:0/3041363009 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa2c4107650 con 0x7fa2ac06c680 2024-03-22T10:35:47.672 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 3 on host 'smithi150' 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa2ac06c680 msgr2=0x7fa2ac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa2ac06c680 0x7fa2ac06eb30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8000f80 tx=0x7fa2b800b7b0 comp rx=0 tx=0).stop 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 msgr2=0x7fa2c41980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 0x7fa2c41980c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000d8d0 tx=0x7fa2b000dbe0 comp rx=0 tx=0).stop 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 shutdown_connections 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa2ac06c680 0x7fa2ac06eb30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c40737b0 0x7fa2c4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 --2- 172.21.15.150:0/3041363009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c4074d80 0x7fa2c41980c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 >> 172.21.15.150:0/3041363009 conn(0x7fa2c40fb6e0 msgr2=0x7fa2c4105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:35:47.675 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 shutdown_connections 2024-03-22T10:35:47.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:47.682+0000 7fa2cc000700 1 -- 172.21.15.150:0/3041363009 wait complete. 2024-03-22T10:35:47.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:47 smithi115 ceph-mon[29111]: from='osd.3 [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T10:35:47.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:47 smithi115 ceph-mon[29111]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T10:35:48.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:48.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T10:35:48.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: osdmap e24: 4 total, 3 up, 4 in 2024-03-22T10:35:48.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:48.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: from='osd.3 [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:35:48.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:35:48.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:48.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:48.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:48.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:48 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:48.520 DEBUG:teuthology.orchestra.run.smithi150:osd.3> sudo journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.3.service 2024-03-22T10:35:48.526 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_3... 2024-03-22T10:35:48.526 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T10:35:48.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:48.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T10:35:48.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: osdmap e24: 4 total, 3 up, 4 in 2024-03-22T10:35:48.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:48.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: from='osd.3 [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:35:48.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:35:48.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:48.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:48.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:48.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:48 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:49.621 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:49 smithi150 ceph-mon[34518]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-22T10:35:49.621 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:49 smithi150 ceph-mon[34518]: osdmap e25: 4 total, 3 up, 4 in 2024-03-22T10:35:49.621 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:49.621 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:49 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:49.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:49 smithi115 ceph-mon[29111]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-22T10:35:49.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:49 smithi115 ceph-mon[29111]: osdmap e25: 4 total, 3 up, 4 in 2024-03-22T10:35:49.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:49.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:49 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:49.951 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi150/config 2024-03-22T10:35:50.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:50 smithi115 ceph-mon[29111]: purged_snaps scrub starts 2024-03-22T10:35:50.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:50 smithi115 ceph-mon[29111]: purged_snaps scrub ok 2024-03-22T10:35:50.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:50 smithi115 ceph-mon[29111]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:50.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:50 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:50.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:50 smithi115 ceph-mon[29111]: osd.3 [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] boot 2024-03-22T10:35:50.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:50 smithi115 ceph-mon[29111]: osdmap e26: 4 total, 4 up, 4 in 2024-03-22T10:35:50.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:50 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:50.556 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:50 smithi150 ceph-mon[34518]: purged_snaps scrub starts 2024-03-22T10:35:50.556 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:50 smithi150 ceph-mon[34518]: purged_snaps scrub ok 2024-03-22T10:35:50.556 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:50 smithi150 ceph-mon[34518]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-22T10:35:50.556 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:50 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:50.556 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:50 smithi150 ceph-mon[34518]: osd.3 [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] boot 2024-03-22T10:35:50.556 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:50 smithi150 ceph-mon[34518]: osdmap e26: 4 total, 4 up, 4 in 2024-03-22T10:35:50.556 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:50 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:35:51.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:51 smithi150 ceph-mon[34518]: osdmap e27: 4 total, 4 up, 4 in 2024-03-22T10:35:51.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:51 smithi150 ceph-mon[34518]: osdmap e28: 4 total, 4 up, 4 in 2024-03-22T10:35:51.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:51 smithi115 ceph-mon[29111]: osdmap e27: 4 total, 4 up, 4 in 2024-03-22T10:35:51.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:51 smithi115 ceph-mon[29111]: osdmap e28: 4 total, 4 up, 4 in 2024-03-22T10:35:52.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:52 smithi150 ceph-mon[34518]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:35:52.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:52 smithi115 ceph-mon[29111]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:35:53.613 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T10:35:53.614 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T10:35:53.614 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-03-22T10:35:53.614 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-03-22T10:35:53.614 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0340069 s, 308 MB/s 2024-03-22T10:35:53.614 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-03-22T10:35:53.614 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: osdmap e29: 4 total, 4 up, 4 in 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:35:53.615 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:53.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: osdmap e29: 4 total, 4 up, 4 in 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:35:53.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:35:54.408 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-03-22T10:35:54.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:54 smithi150 ceph-mon[34518]: Detected new or changed devices on smithi150 2024-03-22T10:35:54.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:54 smithi150 ceph-mon[34518]: Adjusting osd_memory_target on smithi150 to 14915M 2024-03-22T10:35:54.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:54 smithi150 ceph-mon[34518]: pgmap v121: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T10:35:54.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:54 smithi150 ceph-mon[34518]: osdmap e30: 4 total, 4 up, 4 in 2024-03-22T10:35:54.704 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:54 smithi115 ceph-mon[29111]: Detected new or changed devices on smithi150 2024-03-22T10:35:54.704 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:54 smithi115 ceph-mon[29111]: Adjusting osd_memory_target on smithi150 to 14915M 2024-03-22T10:35:54.704 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:54 smithi115 ceph-mon[29111]: pgmap v121: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T10:35:54.704 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:54 smithi115 ceph-mon[29111]: osdmap e30: 4 total, 4 up, 4 in 2024-03-22T10:35:54.814 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi150/config 2024-03-22T10:35:55.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:35:55.641 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:35:56.582 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:56 smithi150 ceph-mon[34518]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-22T10:35:56.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.756+0000 7f6fd4600700 1 -- 172.21.15.150:0/69064023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc103560 msgr2=0x7f6fcc105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:35:56.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.756+0000 7f6fd4600700 1 --2- 172.21.15.150:0/69064023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc103560 0x7f6fcc105940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009910 tx=0x7f6fb8009c20 comp rx=0 tx=0).stop 2024-03-22T10:35:56.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.756+0000 7f6fd4600700 1 -- 172.21.15.150:0/69064023 shutdown_connections 2024-03-22T10:35:56.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.756+0000 7f6fd4600700 1 --2- 172.21.15.150:0/69064023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc103560 0x7f6fcc105940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:56.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.756+0000 7f6fd4600700 1 --2- 172.21.15.150:0/69064023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fcc100c40 0x7f6fcc103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:56.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.756+0000 7f6fd4600700 1 -- 172.21.15.150:0/69064023 >> 172.21.15.150:0/69064023 conn(0x7f6fcc0fa5e0 msgr2=0x7f6fcc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:35:56.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.756+0000 7f6fd4600700 1 -- 172.21.15.150:0/69064023 shutdown_connections 2024-03-22T10:35:56.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.756+0000 7f6fd4600700 1 -- 172.21.15.150:0/69064023 wait complete. 2024-03-22T10:35:56.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.758+0000 7f6fd4600700 1 Processor -- start 2024-03-22T10:35:56.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.758+0000 7f6fd4600700 1 -- start start 2024-03-22T10:35:56.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.758+0000 7f6fd4600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc100c40 0x7f6fcc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:35:56.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.758+0000 7f6fd4600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fcc103560 0x7f6fcc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:35:56.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.758+0000 7f6fd4600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc1985d0 con 0x7f6fcc103560 2024-03-22T10:35:56.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.758+0000 7f6fd4600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc198710 con 0x7f6fcc100c40 2024-03-22T10:35:56.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.759+0000 7f6fd1c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc100c40 0x7f6fcc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:35:56.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.759+0000 7f6fd1c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc100c40 0x7f6fcc197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48552/0 (socket says 172.21.15.150:48552) 2024-03-22T10:35:56.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.759+0000 7f6fd1c00700 1 -- 172.21.15.150:0/1658986259 learned_addr learned my addr 172.21.15.150:0/1658986259 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:35:56.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.759+0000 7f6fd1200700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fcc103560 0x7f6fcc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:35:56.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.759+0000 7f6fd1c00700 1 -- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fcc103560 msgr2=0x7f6fcc198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:35:56.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.759+0000 7f6fd1c00700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fcc103560 0x7f6fcc198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:35:56.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.759+0000 7f6fd1c00700 1 -- 172.21.15.150:0/1658986259 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb80095f0 con 0x7f6fcc100c40 2024-03-22T10:35:56.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.759+0000 7f6fd1c00700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc100c40 0x7f6fcc197b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000c930 tx=0x7f6fc000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:35:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.760+0000 7f6fca000700 1 -- 172.21.15.150:0/1658986259 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc0007ad0 con 0x7f6fcc100c40 2024-03-22T10:35:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.760+0000 7f6fca000700 1 -- 172.21.15.150:0/1658986259 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6fc0007c30 con 0x7f6fcc100c40 2024-03-22T10:35:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.760+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc19d1d0 con 0x7f6fcc100c40 2024-03-22T10:35:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.760+0000 7f6fca000700 1 -- 172.21.15.150:0/1658986259 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc0018630 con 0x7f6fcc100c40 2024-03-22T10:35:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.760+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc19d6f0 con 0x7f6fcc100c40 2024-03-22T10:35:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.761+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc0fc1c0 con 0x7f6fcc100c40 2024-03-22T10:35:56.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.762+0000 7f6fca000700 1 -- 172.21.15.150:0/1658986259 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f6fc000f450 con 0x7f6fcc100c40 2024-03-22T10:35:56.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.763+0000 7f6fca000700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6fb406c680 0x7f6fb406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:35:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.763+0000 7f6fca000700 1 -- 172.21.15.150:0/1658986259 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f6fc008b480 con 0x7f6fcc100c40 2024-03-22T10:35:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.764+0000 7f6fd1200700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6fb406c680 0x7f6fb406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:35:56.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.765+0000 7f6fd1200700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6fb406c680 0x7f6fb406eb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8006010 tx=0x7f6fb800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:35:56.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:56 smithi115 ceph-mon[29111]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-22T10:35:56.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.768+0000 7f6fca000700 1 -- 172.21.15.150:0/1658986259 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6fc0059bf0 con 0x7f6fcc100c40 2024-03-22T10:35:56.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:35:56.993+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fcc061190 con 0x7f6fb406c680 2024-03-22T10:35:57.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:35:57.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:35:57.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:57.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:35:57.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:35:57.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:35:58.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:58 smithi150 ceph-mon[34518]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:35:58.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:35:58 smithi150 ceph-mon[34518]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-22T10:35:58.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:58 smithi115 ceph-mon[29111]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:35:58.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:35:58 smithi115 ceph-mon[29111]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-22T10:36:00.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:00 smithi150 ceph-mon[34518]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-22T10:36:00.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:00 smithi115 ceph-mon[29111]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-22T10:36:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:01 smithi150 ceph-mon[34518]: from='client.? 172.21.15.150:0/2048615003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574fd0d5-c58a-4e16-961d-b384e52d82b2"}]: dispatch 2024-03-22T10:36:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:01 smithi150 ceph-mon[34518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574fd0d5-c58a-4e16-961d-b384e52d82b2"}]: dispatch 2024-03-22T10:36:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:01 smithi150 ceph-mon[34518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "574fd0d5-c58a-4e16-961d-b384e52d82b2"}]': finished 2024-03-22T10:36:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:01 smithi150 ceph-mon[34518]: osdmap e31: 5 total, 4 up, 5 in 2024-03-22T10:36:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:01 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:01.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:01 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/2048615003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574fd0d5-c58a-4e16-961d-b384e52d82b2"}]: dispatch 2024-03-22T10:36:01.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:01 smithi115 ceph-mon[29111]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574fd0d5-c58a-4e16-961d-b384e52d82b2"}]: dispatch 2024-03-22T10:36:01.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:01 smithi115 ceph-mon[29111]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "574fd0d5-c58a-4e16-961d-b384e52d82b2"}]': finished 2024-03-22T10:36:01.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:01 smithi115 ceph-mon[29111]: osdmap e31: 5 total, 4 up, 5 in 2024-03-22T10:36:01.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:01 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:02.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:02 smithi150 ceph-mon[34518]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:02.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:02 smithi150 ceph-mon[34518]: from='client.? 172.21.15.150:0/1060983716' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:36:02.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:02 smithi115 ceph-mon[29111]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:02.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:02 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/1060983716' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:36:04.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:03 smithi115 ceph-mon[29111]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:04.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:03 smithi150 ceph-mon[34518]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:06.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:06 smithi150 ceph-mon[34518]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:06.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:06 smithi115 ceph-mon[29111]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:08.495 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:08 smithi150 ceph-mon[34518]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:08.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:08 smithi115 ceph-mon[29111]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:10.559 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:10 smithi115 ceph-mon[29111]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:10.559 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T10:36:10.559 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:10.559 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:10 smithi115 ceph-mon[29111]: Deploying daemon osd.4 on smithi150 2024-03-22T10:36:10.559 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:36:10.626 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:10 smithi150 ceph-mon[34518]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:10.626 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T10:36:10.626 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:10.626 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:10 smithi150 ceph-mon[34518]: Deploying daemon osd.4 on smithi150 2024-03-22T10:36:10.627 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:36:12.500 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:12 smithi150 ceph-mon[34518]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:12.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:12 smithi115 ceph-mon[29111]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:14.027 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:13 smithi150 ceph-mon[34518]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:14.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:13 smithi115 ceph-mon[29111]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:16.588 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:16 smithi150 ceph-mon[34518]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:16.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:16 smithi115 ceph-mon[29111]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:18.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:18 smithi150 ceph-mon[34518]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:18.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:18.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:18.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:36:18.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:18 smithi115 ceph-mon[29111]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:18.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:18.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:18.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:36:20.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:20 smithi150 ceph-mon[34518]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:20.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:20 smithi115 ceph-mon[29111]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:22.571 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:22 smithi150 ceph-mon[34518]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:22.572 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:22 smithi150 ceph-mon[34518]: from='osd.4 [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T10:36:22.572 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:22 smithi150 ceph-mon[34518]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T10:36:22.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:22 smithi115 ceph-mon[29111]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:22.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:22 smithi115 ceph-mon[29111]: from='osd.4 [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T10:36:22.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:22 smithi115 ceph-mon[29111]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T10:36:22.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.003+0000 7f6fca000700 1 -- 172.21.15.150:0/1658986259 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6fcc061190 con 0x7f6fb406c680 2024-03-22T10:36:22.996 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-03-22T10:36:22.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6fb406c680 msgr2=0x7f6fb406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6fb406c680 0x7f6fb406eb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8006010 tx=0x7f6fb800b540 comp rx=0 tx=0).stop 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc100c40 msgr2=0x7f6fcc197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc100c40 0x7f6fcc197b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000c930 tx=0x7f6fc000ccf0 comp rx=0 tx=0).stop 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 shutdown_connections 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6fb406c680 0x7f6fb406eb30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fcc100c40 0x7f6fcc197b00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 --2- 172.21.15.150:0/1658986259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fcc103560 0x7f6fcc198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 >> 172.21.15.150:0/1658986259 conn(0x7f6fcc0fa5e0 msgr2=0x7f6fcc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 shutdown_connections 2024-03-22T10:36:23.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:23.007+0000 7f6fd4600700 1 -- 172.21.15.150:0/1658986259 wait complete. 2024-03-22T10:36:23.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: osdmap e32: 5 total, 4 up, 5 in 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='osd.4 [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: osdmap e33: 5 total, 4 up, 5 in 2024-03-22T10:36:23.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:23 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:23.749 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.4.service 2024-03-22T10:36:23.756 INFO:tasks.cephadm:Deploying osd.5 on smithi150 with /dev/vg_nvme/lv_2... 2024-03-22T10:36:23.756 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T10:36:23.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T10:36:23.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: osdmap e32: 5 total, 4 up, 5 in 2024-03-22T10:36:23.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='osd.4 [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:36:23.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:23.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:36:23.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:23.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:23.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:23.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:23.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-22T10:36:23.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: osdmap e33: 5 total, 4 up, 5 in 2024-03-22T10:36:23.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:23 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:24.196 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi150/config 2024-03-22T10:36:24.624 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:24 smithi150 ceph-mon[34518]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:24.624 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:24 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:24.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:24 smithi115 ceph-mon[29111]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-22T10:36:24.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:24 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:25.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:25 smithi150 ceph-mon[34518]: purged_snaps scrub starts 2024-03-22T10:36:25.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:25 smithi150 ceph-mon[34518]: purged_snaps scrub ok 2024-03-22T10:36:25.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:25 smithi150 ceph-mon[34518]: osd.4 [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] boot 2024-03-22T10:36:25.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:25 smithi150 ceph-mon[34518]: osdmap e34: 5 total, 5 up, 5 in 2024-03-22T10:36:25.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:25.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:36:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:25 smithi115 ceph-mon[29111]: purged_snaps scrub starts 2024-03-22T10:36:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:25 smithi115 ceph-mon[29111]: purged_snaps scrub ok 2024-03-22T10:36:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:25 smithi115 ceph-mon[29111]: osd.4 [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] boot 2024-03-22T10:36:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:25 smithi115 ceph-mon[29111]: osdmap e34: 5 total, 5 up, 5 in 2024-03-22T10:36:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:36:25.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:36:26.619 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:26 smithi150 ceph-mon[34518]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:26.619 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:26 smithi150 ceph-mon[34518]: osdmap e35: 5 total, 5 up, 5 in 2024-03-22T10:36:26.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:26 smithi115 ceph-mon[29111]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:26.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:26 smithi115 ceph-mon[29111]: osdmap e35: 5 total, 5 up, 5 in 2024-03-22T10:36:27.382 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T10:36:27.382 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T10:36:27.382 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-03-22T10:36:27.382 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-03-22T10:36:27.382 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0349173 s, 300 MB/s 2024-03-22T10:36:27.382 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 2024-03-22T10:36:27.382 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-03-22T10:36:28.452 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-03-22T10:36:28.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:28 smithi150 ceph-mon[34518]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:28.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:28 smithi115 ceph-mon[29111]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:29.264 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi150/config 2024-03-22T10:36:30.500 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: Detected new or changed devices on smithi150 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: Adjusting osd_memory_target on smithi150 to 7457M 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:30.501 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:30 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: Detected new or changed devices on smithi150 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: Adjusting osd_memory_target on smithi150 to 7457M 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:30.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:30 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:36:31.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.248+0000 7f5777800700 1 -- 172.21.15.150:0/43736123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 msgr2=0x7f5770073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:36:31.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.248+0000 7f5777800700 1 --2- 172.21.15.150:0/43736123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 0x7f5770073c20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5758009910 tx=0x7f5758009c20 comp rx=0 tx=0).stop 2024-03-22T10:36:31.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.248+0000 7f5777800700 1 -- 172.21.15.150:0/43736123 shutdown_connections 2024-03-22T10:36:31.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.248+0000 7f5777800700 1 --2- 172.21.15.150:0/43736123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 0x7f5770073c20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:36:31.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.248+0000 7f5777800700 1 --2- 172.21.15.150:0/43736123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5770074d80 0x7f57700731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:36:31.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.248+0000 7f5777800700 1 -- 172.21.15.150:0/43736123 >> 172.21.15.150:0/43736123 conn(0x7f57700fb6e0 msgr2=0x7f57700fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:36:31.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.248+0000 7f5777800700 1 -- 172.21.15.150:0/43736123 shutdown_connections 2024-03-22T10:36:31.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.248+0000 7f5777800700 1 -- 172.21.15.150:0/43736123 wait complete. 2024-03-22T10:36:31.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.249+0000 7f5777800700 1 Processor -- start 2024-03-22T10:36:31.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.250+0000 7f5777800700 1 -- start start 2024-03-22T10:36:31.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.250+0000 7f5777800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 0x7f5770197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:36:31.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.250+0000 7f5777800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5770074d80 0x7f5770198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:36:31.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.250+0000 7f5777800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770198670 con 0x7f5770074d80 2024-03-22T10:36:31.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.250+0000 7f5777800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57701987b0 con 0x7f57700737b0 2024-03-22T10:36:31.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.250+0000 7f5774e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 0x7f5770197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:36:31.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.250+0000 7f5774e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 0x7f5770197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37416/0 (socket says 172.21.15.150:37416) 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.250+0000 7f5774e00700 1 -- 172.21.15.150:0/1621185631 learned_addr learned my addr 172.21.15.150:0/1621185631 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f5767e00700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5770074d80 0x7f5770198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f5774e00700 1 -- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5770074d80 msgr2=0x7f5770198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f5774e00700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5770074d80 0x7f5770198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f5774e00700 1 -- 172.21.15.150:0/1621185631 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57580095f0 con 0x7f57700737b0 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f5774e00700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 0x7f5770197b10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f576000c8a0 tx=0x7f576000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f576e000700 1 -- 172.21.15.150:0/1621185631 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f576000cea0 con 0x7f57700737b0 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f576e000700 1 -- 172.21.15.150:0/1621185631 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5760004830 con 0x7f57700737b0 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f576e000700 1 -- 172.21.15.150:0/1621185631 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57600055b0 con 0x7f57700737b0 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577019d260 con 0x7f57700737b0 2024-03-22T10:36:31.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.251+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577019d7b0 con 0x7f57700737b0 2024-03-22T10:36:31.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.253+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577010ae90 con 0x7f57700737b0 2024-03-22T10:36:31.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.254+0000 7f576e000700 1 -- 172.21.15.150:0/1621185631 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f5760007ad0 con 0x7f57700737b0 2024-03-22T10:36:31.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.255+0000 7f576e000700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f575406c680 0x7f575406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:36:31.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.255+0000 7f576e000700 1 -- 172.21.15.150:0/1621185631 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f576008b0e0 con 0x7f57700737b0 2024-03-22T10:36:31.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.255+0000 7f5767e00700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f575406c680 0x7f575406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:36:31.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.256+0000 7f5767e00700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f575406c680 0x7f575406eb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5758006010 tx=0x7f575800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:36:31.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.259+0000 7f576e000700 1 -- 172.21.15.150:0/1621185631 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5760059720 con 0x7f57700737b0 2024-03-22T10:36:31.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:31 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:31.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:31 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:36:31.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:31 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:31.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:31.483+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5770107650 con 0x7f575406c680 2024-03-22T10:36:31.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:31 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:31.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:31 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:36:31.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:31 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:32.532 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:32 smithi150 ceph-mon[34518]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:32.533 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:32 smithi150 ceph-mon[34518]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:36:32.533 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:36:32.533 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:36:32.533 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:32 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:32.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:32 smithi115 ceph-mon[29111]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:32.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:32 smithi115 ceph-mon[29111]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:36:32.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T10:36:32.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T10:36:32.764 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:32 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:34.116 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:33 smithi150 ceph-mon[34518]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:34.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:33 smithi115 ceph-mon[29111]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:35.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:35 smithi150 ceph-mon[34518]: from='client.? 172.21.15.150:0/542062562' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc37270e-923d-45cf-accd-d4119dcc1a7c"}]: dispatch 2024-03-22T10:36:35.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:35 smithi150 ceph-mon[34518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc37270e-923d-45cf-accd-d4119dcc1a7c"}]: dispatch 2024-03-22T10:36:35.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:35 smithi150 ceph-mon[34518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc37270e-923d-45cf-accd-d4119dcc1a7c"}]': finished 2024-03-22T10:36:35.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:35 smithi150 ceph-mon[34518]: osdmap e36: 6 total, 5 up, 6 in 2024-03-22T10:36:35.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:36:35.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:35 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/542062562' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc37270e-923d-45cf-accd-d4119dcc1a7c"}]: dispatch 2024-03-22T10:36:35.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:35 smithi115 ceph-mon[29111]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc37270e-923d-45cf-accd-d4119dcc1a7c"}]: dispatch 2024-03-22T10:36:35.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:35 smithi115 ceph-mon[29111]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc37270e-923d-45cf-accd-d4119dcc1a7c"}]': finished 2024-03-22T10:36:35.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:35 smithi115 ceph-mon[29111]: osdmap e36: 6 total, 5 up, 6 in 2024-03-22T10:36:35.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:36:36.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:36 smithi150 ceph-mon[34518]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:36.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:36 smithi150 ceph-mon[34518]: from='client.? 172.21.15.150:0/3348916143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:36:36.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:36 smithi115 ceph-mon[29111]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:36.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:36 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/3348916143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T10:36:38.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:38 smithi150 ceph-mon[34518]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:38.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:38 smithi115 ceph-mon[29111]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:40.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:40 smithi150 ceph-mon[34518]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:40.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:40 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:36:40.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:40 smithi115 ceph-mon[29111]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:40.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:36:42.566 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:42 smithi150 ceph-mon[34518]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:42.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:42 smithi115 ceph-mon[29111]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:44.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:43 smithi115 ceph-mon[29111]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:44.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:43 smithi150 ceph-mon[34518]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:45.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:44 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T10:36:45.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:44 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:45.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:44 smithi115 ceph-mon[29111]: Deploying daemon osd.5 on smithi150 2024-03-22T10:36:45.369 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:44 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T10:36:45.369 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:44 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:36:45.369 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:44 smithi150 ceph-mon[34518]: Deploying daemon osd.5 on smithi150 2024-03-22T10:36:46.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:45 smithi115 ceph-mon[29111]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:46.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:45 smithi150 ceph-mon[34518]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:48.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:48 smithi150 ceph-mon[34518]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:48.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:48 smithi115 ceph-mon[29111]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:50.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:50 smithi150 ceph-mon[34518]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:50.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:50 smithi115 ceph-mon[29111]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:52.376 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:52 smithi150 ceph-mon[34518]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:52.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:52 smithi115 ceph-mon[29111]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:53.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:53.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:36:53.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:53 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:53.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:53.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:36:53.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:53 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:54.790 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:54 smithi150 ceph-mon[34518]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:55.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:54 smithi115 ceph-mon[29111]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:55.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:36:56.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:36:56.840 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:56 smithi150 ceph-mon[34518]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:56.840 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:56 smithi150 ceph-mon[34518]: from='osd.5 [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T10:36:56.841 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:56 smithi150 ceph-mon[34518]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T10:36:57.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:56 smithi115 ceph-mon[29111]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:57.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:56 smithi115 ceph-mon[29111]: from='osd.5 [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T10:36:57.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:56 smithi115 ceph-mon[29111]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T10:36:57.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.393+0000 7f576e000700 1 -- 172.21.15.150:0/1621185631 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5770107650 con 0x7f575406c680 2024-03-22T10:36:57.387 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 5 on host 'smithi150' 2024-03-22T10:36:57.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f575406c680 msgr2=0x7f575406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:36:57.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f575406c680 0x7f575406eb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5758006010 tx=0x7f575800b540 comp rx=0 tx=0).stop 2024-03-22T10:36:57.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 msgr2=0x7f5770197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:36:57.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 0x7f5770197b10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f576000c8a0 tx=0x7f576000cc60 comp rx=0 tx=0).stop 2024-03-22T10:36:57.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 shutdown_connections 2024-03-22T10:36:57.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f575406c680 0x7f575406eb30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:36:57.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57700737b0 0x7f5770197b10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:36:57.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 --2- 172.21.15.150:0/1621185631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5770074d80 0x7f5770198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:36:57.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 >> 172.21.15.150:0/1621185631 conn(0x7f57700fb6e0 msgr2=0x7f5770105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:36:57.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.396+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 shutdown_connections 2024-03-22T10:36:57.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:36:57.397+0000 7f5777800700 1 -- 172.21.15.150:0/1621185631 wait complete. 2024-03-22T10:36:57.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:57 smithi150 ceph-mon[34518]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T10:36:57.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:57 smithi150 ceph-mon[34518]: osdmap e37: 6 total, 5 up, 6 in 2024-03-22T10:36:57.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:36:57.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:57 smithi150 ceph-mon[34518]: from='osd.5 [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:36:57.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:57 smithi150 ceph-mon[34518]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:36:57.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:57.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:57 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:58.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:57 smithi115 ceph-mon[29111]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T10:36:58.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:57 smithi115 ceph-mon[29111]: osdmap e37: 6 total, 5 up, 6 in 2024-03-22T10:36:58.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:36:58.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:57 smithi115 ceph-mon[29111]: from='osd.5 [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:36:58.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:57 smithi115 ceph-mon[29111]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:36:58.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:58.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:57 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:58.247 DEBUG:teuthology.orchestra.run.smithi150:osd.5> sudo journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.5.service 2024-03-22T10:36:58.252 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-22T10:36:58.252 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-22T10:36:58.673 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:36:58.822 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:58 smithi150 ceph-mon[34518]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:58.823 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:58 smithi150 ceph-mon[34518]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-22T10:36:58.823 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:58 smithi150 ceph-mon[34518]: osdmap e38: 6 total, 5 up, 6 in 2024-03-22T10:36:58.823 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:58 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:36:58.823 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:58 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:36:58.823 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:58 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:58.823 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:58 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:58.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:58 smithi115 ceph-mon[29111]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-22T10:36:58.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:58 smithi115 ceph-mon[29111]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-22T10:36:58.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:58 smithi115 ceph-mon[29111]: osdmap e38: 6 total, 5 up, 6 in 2024-03-22T10:36:58.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:36:58.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:36:58.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:58.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:58 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:36:59.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:59 smithi115 ceph-mon[29111]: purged_snaps scrub starts 2024-03-22T10:36:59.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:59 smithi115 ceph-mon[29111]: purged_snaps scrub ok 2024-03-22T10:36:59.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:59 smithi115 ceph-mon[29111]: osd.5 [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] boot 2024-03-22T10:36:59.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:59 smithi115 ceph-mon[29111]: osdmap e39: 6 total, 6 up, 6 in 2024-03-22T10:36:59.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:36:59 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:36:59.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:59 smithi150 ceph-mon[34518]: purged_snaps scrub starts 2024-03-22T10:36:59.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:59 smithi150 ceph-mon[34518]: purged_snaps scrub ok 2024-03-22T10:36:59.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:59 smithi150 ceph-mon[34518]: osd.5 [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] boot 2024-03-22T10:36:59.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:59 smithi150 ceph-mon[34518]: osdmap e39: 6 total, 6 up, 6 in 2024-03-22T10:36:59.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:36:59 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:37:00.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.731+0000 7fcafe400700 1 -- 172.21.15.115:0/1856040979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf81013f0 msgr2=0x7fcaf8101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:00.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.731+0000 7fcafe400700 1 --2- 172.21.15.115:0/1856040979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf81013f0 0x7fcaf8101840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009910 tx=0x7fcae0009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:00.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.731+0000 7fcafe400700 1 -- 172.21.15.115:0/1856040979 shutdown_connections 2024-03-22T10:37:00.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.731+0000 7fcafe400700 1 --2- 172.21.15.115:0/1856040979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf81013f0 0x7fcaf8101840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:00.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.731+0000 7fcafe400700 1 --2- 172.21.15.115:0/1856040979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf80fe410 0x7fcaf80fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:00.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.731+0000 7fcafe400700 1 -- 172.21.15.115:0/1856040979 >> 172.21.15.115:0/1856040979 conn(0x7fcaf80f9d80 msgr2=0x7fcaf80fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:00.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.731+0000 7fcafe400700 1 -- 172.21.15.115:0/1856040979 shutdown_connections 2024-03-22T10:37:00.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.731+0000 7fcafe400700 1 -- 172.21.15.115:0/1856040979 wait complete. 2024-03-22T10:37:00.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.732+0000 7fcafe400700 1 Processor -- start 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.732+0000 7fcafe400700 1 -- start start 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.732+0000 7fcafe400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf80fe410 0x7fcaf8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.732+0000 7fcafe400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81013f0 0x7fcaf81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.732+0000 7fcafe400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81986c0 con 0x7fcaf80fe410 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.732+0000 7fcafe400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf8198800 con 0x7fcaf81013f0 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.733+0000 7fcaf7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf80fe410 0x7fcaf8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.733+0000 7fcaf7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf80fe410 0x7fcaf8197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50500/0 (socket says 172.21.15.115:50500) 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.733+0000 7fcaf7400700 1 -- 172.21.15.115:0/930438651 learned_addr learned my addr 172.21.15.115:0/930438651 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.733+0000 7fcaeea00700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81013f0 0x7fcaf81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.733+0000 7fcaf7400700 1 -- 172.21.15.115:0/930438651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81013f0 msgr2=0x7fcaf81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.733+0000 7fcaf7400700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81013f0 0x7fcaf81980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.733+0000 7fcaf7400700 1 -- 172.21.15.115:0/930438651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00095f0 con 0x7fcaf80fe410 2024-03-22T10:37:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.733+0000 7fcaf7400700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf80fe410 0x7fcaf8197b60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcae8007e90 tx=0x7fcae800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:00.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.734+0000 7fcaf4c00700 1 -- 172.21.15.115:0/930438651 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae8017070 con 0x7fcaf80fe410 2024-03-22T10:37:00.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.734+0000 7fcaf4c00700 1 -- 172.21.15.115:0/930438651 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcae800f040 con 0x7fcaf80fe410 2024-03-22T10:37:00.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.734+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf819d2b0 con 0x7fcaf80fe410 2024-03-22T10:37:00.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.734+0000 7fcaf4c00700 1 -- 172.21.15.115:0/930438651 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae8004880 con 0x7fcaf80fe410 2024-03-22T10:37:00.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.734+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf819d800 con 0x7fcaf80fe410 2024-03-22T10:37:00.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.735+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf81054d0 con 0x7fcaf80fe410 2024-03-22T10:37:00.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.736+0000 7fcaf4c00700 1 -- 172.21.15.115:0/930438651 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fcae80040b0 con 0x7fcaf80fe410 2024-03-22T10:37:00.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.737+0000 7fcaf4c00700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcadc06c620 0x7fcadc06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:00.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.737+0000 7fcaf4c00700 1 -- 172.21.15.115:0/930438651 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fcae808afc0 con 0x7fcaf80fe410 2024-03-22T10:37:00.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.737+0000 7fcaeea00700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcadc06c620 0x7fcadc06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:00.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.738+0000 7fcaeea00700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcadc06c620 0x7fcadc06ead0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcae0006010 tx=0x7fcae000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:00.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.742+0000 7fcaf4c00700 1 -- 172.21.15.115:0/930438651 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcae8059470 con 0x7fcaf80fe410 2024-03-22T10:37:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:00 smithi150 ceph-mon[34518]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:00.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:00 smithi150 ceph-mon[34518]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T10:37:00.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.954+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fcaf802cc30 con 0x7fcaf80fe410 2024-03-22T10:37:00.948 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:00 smithi115 ceph-mon[29111]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:00.948 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:00 smithi115 ceph-mon[29111]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T10:37:00.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.955+0000 7fcaf4c00700 1 -- 172.21.15.115:0/930438651 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7fcae8059000 con 0x7fcaf80fe410 2024-03-22T10:37:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:37:00.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.958+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcadc06c620 msgr2=0x7fcadc06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:00.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.958+0000 7fcafe400700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcadc06c620 0x7fcadc06ead0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcae0006010 tx=0x7fcae000b540 comp rx=0 tx=0).stop 2024-03-22T10:37:00.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.958+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf80fe410 msgr2=0x7fcaf8197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:00.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.958+0000 7fcafe400700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf80fe410 0x7fcaf8197b60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcae8007e90 tx=0x7fcae800d3b0 comp rx=0 tx=0).stop 2024-03-22T10:37:00.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.958+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 shutdown_connections 2024-03-22T10:37:00.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.958+0000 7fcafe400700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fcadc06c620 0x7fcadc06ead0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:00.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.958+0000 7fcafe400700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf80fe410 0x7fcaf8197b60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:00.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.958+0000 7fcafe400700 1 --2- 172.21.15.115:0/930438651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81013f0 0x7fcaf81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:00.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.958+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 >> 172.21.15.115:0/930438651 conn(0x7fcaf80f9d80 msgr2=0x7fcaf8102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:00.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.959+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 shutdown_connections 2024-03-22T10:37:00.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:00.959+0000 7fcafe400700 1 -- 172.21.15.115:0/930438651 wait complete. 2024-03-22T10:37:01.592 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1711103818,"num_in_osds":6,"osd_in_since":1711103795,"num_remapped_pgs":0} 2024-03-22T10:37:01.592 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T10:37:01.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:01 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/930438651' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T10:37:01.965 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:01 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/930438651' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T10:37:02.004 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:02.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:02.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:02.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:02.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:37:02.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:02 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:03.142 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:37:03.143 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:37:03.143 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:37:03.143 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:02 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:37:03.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:03 smithi115 ceph-mon[29111]: Detected new or changed devices on smithi150 2024-03-22T10:37:03.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:03 smithi115 ceph-mon[29111]: Adjusting osd_memory_target on smithi150 to 4971M 2024-03-22T10:37:03.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.985+0000 7f7453c00700 1 -- 172.21.15.115:0/3568724888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10c010 msgr2=0x7f744c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:03.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.985+0000 7f7453c00700 1 --2- 172.21.15.115:0/3568724888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10c010 0x7f744c106b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7438009910 tx=0x7f7438009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:03.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.985+0000 7f7453c00700 1 -- 172.21.15.115:0/3568724888 shutdown_connections 2024-03-22T10:37:03.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.985+0000 7f7453c00700 1 --2- 172.21.15.115:0/3568724888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10c010 0x7f744c106b80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:03.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.985+0000 7f7453c00700 1 --2- 172.21.15.115:0/3568724888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f744c10b700 0x7f744c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:03.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.985+0000 7f7453c00700 1 -- 172.21.15.115:0/3568724888 >> 172.21.15.115:0/3568724888 conn(0x7f744c076000 msgr2=0x7f744c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:03.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.985+0000 7f7453c00700 1 -- 172.21.15.115:0/3568724888 shutdown_connections 2024-03-22T10:37:03.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.985+0000 7f7453c00700 1 -- 172.21.15.115:0/3568724888 wait complete. 2024-03-22T10:37:03.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.986+0000 7f7453c00700 1 Processor -- start 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.987+0000 7f7453c00700 1 -- start start 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.987+0000 7f7453c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10b700 0x7f744c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.987+0000 7f7453c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f744c10c010 0x7f744c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.987+0000 7f7453c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c19cf20 con 0x7f744c10b700 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.987+0000 7f7453c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1a0cb0 con 0x7f744c10c010 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.987+0000 7f7451200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10b700 0x7f744c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.987+0000 7f7451200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10b700 0x7f744c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45632/0 (socket says 172.21.15.115:45632) 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.987+0000 7f7451200700 1 -- 172.21.15.115:0/299653893 learned_addr learned my addr 172.21.15.115:0/299653893 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f744be00700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f744c10c010 0x7f744c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f7451200700 1 -- 172.21.15.115:0/299653893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f744c10c010 msgr2=0x7f744c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f7451200700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f744c10c010 0x7f744c19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:03.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f7451200700 1 -- 172.21.15.115:0/299653893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380095f0 con 0x7f744c10b700 2024-03-22T10:37:03.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f7451200700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10b700 0x7f744c19c300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f744000c8f0 tx=0x7f744000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:03.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f7449600700 1 -- 172.21.15.115:0/299653893 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7440007a70 con 0x7f744c10b700 2024-03-22T10:37:03.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f7449600700 1 -- 172.21.15.115:0/299653893 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7440004ce0 con 0x7f744c10b700 2024-03-22T10:37:03.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744c1a0f90 con 0x7f744c10b700 2024-03-22T10:37:03.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f7449600700 1 -- 172.21.15.115:0/299653893 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7440005610 con 0x7f744c10b700 2024-03-22T10:37:03.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.988+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c1a14e0 con 0x7f744c10b700 2024-03-22T10:37:03.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.989+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744c04ea50 con 0x7f744c10b700 2024-03-22T10:37:03.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.990+0000 7f7449600700 1 -- 172.21.15.115:0/299653893 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7440004750 con 0x7f744c10b700 2024-03-22T10:37:03.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.992+0000 7f7449600700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f743406c680 0x7f743406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:03.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.992+0000 7f7449600700 1 -- 172.21.15.115:0/299653893 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7440013070 con 0x7f744c10b700 2024-03-22T10:37:03.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.992+0000 7f744be00700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f743406c680 0x7f743406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:03.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.993+0000 7f744be00700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f743406c680 0x7f743406eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7438006010 tx=0x7f743800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:03.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:03.997+0000 7f7449600700 1 -- 172.21.15.115:0/299653893 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7440055e80 con 0x7f744c10b700 2024-03-22T10:37:04.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:03 smithi150 ceph-mon[34518]: Detected new or changed devices on smithi150 2024-03-22T10:37:04.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:03 smithi150 ceph-mon[34518]: Adjusting osd_memory_target on smithi150 to 4971M 2024-03-22T10:37:04.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.211+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f744c19d660 con 0x7f744c10b700 2024-03-22T10:37:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.213+0000 7f7449600700 1 -- 172.21.15.115:0/299653893 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11349 (secure 0 0 0) 0x7f74400594a0 con 0x7f744c10b700 2024-03-22T10:37:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:37:04.206 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","created":"2024-03-22T10:29:37.233959+0000","modified":"2024-03-22T10:36:59.634944+0000","last_up_change":"2024-03-22T10:36:58.621026+0000","last_in_change":"2024-03-22T10:36:35.075110+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T10:35:17.870309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a34c0f70-4e9d-4044-b7b4-b6099b74ba6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6803","nonce":764585829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6805","nonce":764585829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6809","nonce":764585829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6807","nonce":764585829}]},"public_addr":"172.21.15.115:6803/764585829","cluster_addr":"172.21.15.115:6805/764585829","heartbeat_back_addr":"172.21.15.115:6809/764585829","heartbeat_front_addr":"172.21.15.115:6807/764585829","state":["exists","up"]},{"osd":1,"uuid":"1037db4c-35bc-48da-8950-460c772d50b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6811","nonce":586118879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6813","nonce":586118879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6817","nonce":586118879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6815","nonce":586118879}]},"public_addr":"172.21.15.115:6811/586118879","cluster_addr":"172.21.15.115:6813/586118879","heartbeat_back_addr":"172.21.15.115:6817/586118879","heartbeat_front_addr":"172.21.15.115:6815/586118879","state":["exists","up"]},{"osd":2,"uuid":"4e685a22-30ee-4404-bba5-ff944af9b858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6819","nonce":285410766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6821","nonce":285410766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6825","nonce":285410766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6823","nonce":285410766}]},"public_addr":"172.21.15.115:6819/285410766","cluster_addr":"172.21.15.115:6821/285410766","heartbeat_back_addr":"172.21.15.115:6825/285410766","heartbeat_front_addr":"172.21.15.115:6823/285410766","state":["exists","up"]},{"osd":3,"uuid":"1273664e-16c8-46c1-bfb4-c51e3e84ea6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6801","nonce":3913650733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6803","nonce":3913650733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6807","nonce":3913650733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6805","nonce":3913650733}]},"public_addr":"172.21.15.150:6801/3913650733","cluster_addr":"172.21.15.150:6803/3913650733","heartbeat_back_addr":"172.21.15.150:6807/3913650733","heartbeat_front_addr":"172.21.15.150:6805/3913650733","state":["exists","up"]},{"osd":4,"uuid":"574fd0d5-c58a-4e16-961d-b384e52d82b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6809","nonce":792436265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6811","nonce":792436265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6815","nonce":792436265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6813","nonce":792436265}]},"public_addr":"172.21.15.150:6809/792436265","cluster_addr":"172.21.15.150:6811/792436265","heartbeat_back_addr":"172.21.15.150:6815/792436265","heartbeat_front_addr":"172.21.15.150:6813/792436265","state":["exists","up"]},{"osd":5,"uuid":"cc37270e-923d-45cf-accd-d4119dcc1a7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6817","nonce":2178668917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6819","nonce":2178668917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6823","nonce":2178668917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6821","nonce":2178668917}]},"public_addr":"172.21.15.150:6817/2178668917","cluster_addr":"172.21.15.150:6819/2178668917","heartbeat_back_addr":"172.21.15.150:6823/2178668917","heartbeat_front_addr":"172.21.15.150:6821/2178668917","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:04.542374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:39.181161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:16.028358+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:47.869896+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:22.969676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:57.504061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6801/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/901503025":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/2275951342":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/1384534494":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/450820820":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3451764091":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6801/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/400653309":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3025276768":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/3641576098":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3264237187":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/778669970":"2024-03-23T10:30:58.613511+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T10:37:04.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.217+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f743406c680 msgr2=0x7f743406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.217+0000 7f7453c00700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f743406c680 0x7f743406eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7438006010 tx=0x7f743800b540 comp rx=0 tx=0).stop 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.217+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10b700 msgr2=0x7f744c19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.217+0000 7f7453c00700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10b700 0x7f744c19c300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f744000c8f0 tx=0x7f744000ccb0 comp rx=0 tx=0).stop 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.217+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 shutdown_connections 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.217+0000 7f7453c00700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f743406c680 0x7f743406eb30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.217+0000 7f7453c00700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c10b700 0x7f744c19c300 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.217+0000 7f7453c00700 1 --2- 172.21.15.115:0/299653893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f744c10c010 0x7f744c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.217+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 >> 172.21.15.115:0/299653893 conn(0x7f744c076000 msgr2=0x7f744c077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.218+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 shutdown_connections 2024-03-22T10:37:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:04.218+0000 7f7453c00700 1 -- 172.21.15.115:0/299653893 wait complete. 2024-03-22T10:37:04.840 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:04 smithi115 ceph-mon[29111]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:04.840 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:04 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/299653893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:37:04.878 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T10:35:17.870309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T10:37:04.879 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-22T10:37:05.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:04 smithi150 ceph-mon[34518]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:05.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:04 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/299653893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:37:05.295 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:06.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:06 smithi115 ceph-mon[29111]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:07.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:06 smithi150 ceph-mon[34518]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:07.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.442+0000 7f054d600700 1 -- 172.21.15.115:0/3343350367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0548100d50 msgr2=0x7f0548104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:07.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.442+0000 7f054d600700 1 --2- 172.21.15.115:0/3343350367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0548100d50 0x7f0548104d20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0534009910 tx=0x7f0534009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:07.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.442+0000 7f054d600700 1 -- 172.21.15.115:0/3343350367 shutdown_connections 2024-03-22T10:37:07.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.442+0000 7f054d600700 1 --2- 172.21.15.115:0/3343350367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0548100d50 0x7f0548104d20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:07.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.442+0000 7f054d600700 1 --2- 172.21.15.115:0/3343350367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05481003b0 0x7f0548100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:07.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.442+0000 7f054d600700 1 -- 172.21.15.115:0/3343350367 >> 172.21.15.115:0/3343350367 conn(0x7f05480fbc30 msgr2=0x7f05480fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:07.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.442+0000 7f054d600700 1 -- 172.21.15.115:0/3343350367 shutdown_connections 2024-03-22T10:37:07.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.442+0000 7f054d600700 1 -- 172.21.15.115:0/3343350367 wait complete. 2024-03-22T10:37:07.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.444+0000 7f054d600700 1 Processor -- start 2024-03-22T10:37:07.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.444+0000 7f054d600700 1 -- start start 2024-03-22T10:37:07.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.444+0000 7f054d600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05481003b0 0x7f0548195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:07.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.444+0000 7f054d600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0548100d50 0x7f0548196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:07.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.444+0000 7f054d600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548196950 con 0x7f05481003b0 2024-03-22T10:37:07.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.444+0000 7f054d600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548196a90 con 0x7f0548100d50 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.445+0000 7f0546a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05481003b0 0x7f0548195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.445+0000 7f0546a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05481003b0 0x7f0548195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45642/0 (socket says 172.21.15.115:45642) 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.445+0000 7f0546a00700 1 -- 172.21.15.115:0/2425046731 learned_addr learned my addr 172.21.15.115:0/2425046731 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.445+0000 7f0546000700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0548100d50 0x7f0548196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.445+0000 7f0546a00700 1 -- 172.21.15.115:0/2425046731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0548100d50 msgr2=0x7f0548196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.445+0000 7f0546a00700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0548100d50 0x7f0548196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.445+0000 7f0546a00700 1 -- 172.21.15.115:0/2425046731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340095f0 con 0x7f05481003b0 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.445+0000 7f0546a00700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05481003b0 0x7f0548195dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f053800c930 tx=0x7f053800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.446+0000 7f053f400700 1 -- 172.21.15.115:0/2425046731 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0538007ad0 con 0x7f05481003b0 2024-03-22T10:37:07.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.446+0000 7f053f400700 1 -- 172.21.15.115:0/2425046731 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0538007c30 con 0x7f05481003b0 2024-03-22T10:37:07.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.446+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054819a8e0 con 0x7f05481003b0 2024-03-22T10:37:07.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.446+0000 7f053f400700 1 -- 172.21.15.115:0/2425046731 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0538018630 con 0x7f05481003b0 2024-03-22T10:37:07.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.446+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054819ae30 con 0x7f05481003b0 2024-03-22T10:37:07.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.447+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0548108720 con 0x7f05481003b0 2024-03-22T10:37:07.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.448+0000 7f053f400700 1 -- 172.21.15.115:0/2425046731 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f053800f450 con 0x7f05481003b0 2024-03-22T10:37:07.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.449+0000 7f053f400700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f052c06c680 0x7f052c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:07.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.449+0000 7f053f400700 1 -- 172.21.15.115:0/2425046731 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f053808b7e0 con 0x7f05481003b0 2024-03-22T10:37:07.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.449+0000 7f0546000700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f052c06c680 0x7f052c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:07.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.450+0000 7f0546000700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f052c06c680 0x7f052c06eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f05481973e0 tx=0x7f053401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:07.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.454+0000 7f053f400700 1 -- 172.21.15.115:0/2425046731 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0538056160 con 0x7f05481003b0 2024-03-22T10:37:07.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.668+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f054802ccf0 con 0x7f05481003b0 2024-03-22T10:37:07.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.669+0000 7f053f400700 1 -- 172.21.15.115:0/2425046731 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0538059780 con 0x7f05481003b0 2024-03-22T10:37:07.662 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-03-22T10:37:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.672+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f052c06c680 msgr2=0x7f052c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.672+0000 7f054d600700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f052c06c680 0x7f052c06eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f05481973e0 tx=0x7f053401a040 comp rx=0 tx=0).stop 2024-03-22T10:37:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.672+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05481003b0 msgr2=0x7f0548195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.672+0000 7f054d600700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05481003b0 0x7f0548195dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f053800c930 tx=0x7f053800ccf0 comp rx=0 tx=0).stop 2024-03-22T10:37:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.673+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 shutdown_connections 2024-03-22T10:37:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.673+0000 7f054d600700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f052c06c680 0x7f052c06eb30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.673+0000 7f054d600700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05481003b0 0x7f0548195dc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.673+0000 7f054d600700 1 --2- 172.21.15.115:0/2425046731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0548100d50 0x7f0548196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.673+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 >> 172.21.15.115:0/2425046731 conn(0x7f05480fbc30 msgr2=0x7f05480fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.673+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 shutdown_connections 2024-03-22T10:37:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:07.673+0000 7f054d600700 1 -- 172.21.15.115:0/2425046731 wait complete. 2024-03-22T10:37:07.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:07 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2425046731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:37:08.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:07 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2425046731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:37:08.314 INFO:tasks.cephadm:Setting up client nodes... 2024-03-22T10:37:08.315 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-22T10:37:08.728 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:09.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:08 smithi115 ceph-mon[29111]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:09.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:08 smithi150 ceph-mon[34518]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.648+0000 7f22a3000700 1 -- 172.21.15.115:0/625572558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1065b0 msgr2=0x7f229c1069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.648+0000 7f22a3000700 1 --2- 172.21.15.115:0/625572558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1065b0 0x7f229c1069c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2290009910 tx=0x7f2290009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.648+0000 7f22a3000700 1 -- 172.21.15.115:0/625572558 shutdown_connections 2024-03-22T10:37:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.648+0000 7f22a3000700 1 --2- 172.21.15.115:0/625572558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f229c1077b0 0x7f229c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.648+0000 7f22a3000700 1 --2- 172.21.15.115:0/625572558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1065b0 0x7f229c1069c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.648+0000 7f22a3000700 1 -- 172.21.15.115:0/625572558 >> 172.21.15.115:0/625572558 conn(0x7f229c075740 msgr2=0x7f229c077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.648+0000 7f22a3000700 1 -- 172.21.15.115:0/625572558 shutdown_connections 2024-03-22T10:37:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.648+0000 7f22a3000700 1 -- 172.21.15.115:0/625572558 wait complete. 2024-03-22T10:37:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.649+0000 7f22a3000700 1 Processor -- start 2024-03-22T10:37:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.650+0000 7f22a3000700 1 -- start start 2024-03-22T10:37:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.650+0000 7f22a3000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f229c1065b0 0x7f229c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.650+0000 7f22a3000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1077b0 0x7f229c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.650+0000 7f22a3000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229c19c920 con 0x7f229c1077b0 2024-03-22T10:37:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.650+0000 7f22a3000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229c19ca60 con 0x7f229c1065b0 2024-03-22T10:37:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.650+0000 7f22a1200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1077b0 0x7f229c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.650+0000 7f22a1200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1077b0 0x7f229c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45656/0 (socket says 172.21.15.115:45656) 2024-03-22T10:37:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.650+0000 7f22a1200700 1 -- 172.21.15.115:0/1573569659 learned_addr learned my addr 172.21.15.115:0/1573569659 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f22a1c00700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f229c1065b0 0x7f229c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f22a1200700 1 -- 172.21.15.115:0/1573569659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f229c1065b0 msgr2=0x7f229c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f22a1200700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f229c1065b0 0x7f229c19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f22a1200700 1 -- 172.21.15.115:0/1573569659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900095f0 con 0x7f229c1077b0 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f22a1200700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1077b0 0x7f229c19c300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f228800d8d0 tx=0x7f228800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f229a000700 1 -- 172.21.15.115:0/1573569659 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2288009940 con 0x7f229c1077b0 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f229c1a1510 con 0x7f229c1077b0 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f229a000700 1 -- 172.21.15.115:0/1573569659 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2288010460 con 0x7f229c1077b0 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f229a000700 1 -- 172.21.15.115:0/1573569659 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f228800f5f0 con 0x7f229c1077b0 2024-03-22T10:37:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.651+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229c1a1a60 con 0x7f229c1077b0 2024-03-22T10:37:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.652+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229c10f100 con 0x7f229c1077b0 2024-03-22T10:37:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.653+0000 7f229a000700 1 -- 172.21.15.115:0/1573569659 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f22880105d0 con 0x7f229c1077b0 2024-03-22T10:37:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.654+0000 7f229a000700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f228406c680 0x7f228406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.654+0000 7f229a000700 1 -- 172.21.15.115:0/1573569659 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f228808b4f0 con 0x7f229c1077b0 2024-03-22T10:37:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.655+0000 7f22a1c00700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f228406c680 0x7f228406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.655+0000 7f22a1c00700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f228406c680 0x7f228406eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2290000c00 tx=0x7f2290005fb0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:10.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.659+0000 7f229a000700 1 -- 172.21.15.115:0/1573569659 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2288059980 con 0x7f229c1077b0 2024-03-22T10:37:10.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:10 smithi115 ceph-mon[29111]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:10.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:37:10.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.956+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f229c1a2190 con 0x7f229c1077b0 2024-03-22T10:37:10.952 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:10 smithi150 ceph-mon[34518]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:10.952 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:37:10.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.965+0000 7f229a000700 1 -- 172.21.15.115:0/1573569659 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2288059510 con 0x7f229c1077b0 2024-03-22T10:37:10.957 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-03-22T10:37:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBWX/1lRqQZORAAQGk/Hf0z0sxLPcekk9vUbQ== 2024-03-22T10:37:10.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f228406c680 msgr2=0x7f228406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:10.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f228406c680 0x7f228406eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2290000c00 tx=0x7f2290005fb0 comp rx=0 tx=0).stop 2024-03-22T10:37:10.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1077b0 msgr2=0x7f229c19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:10.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1077b0 0x7f229c19c300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f228800d8d0 tx=0x7f228800dc90 comp rx=0 tx=0).stop 2024-03-22T10:37:10.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 shutdown_connections 2024-03-22T10:37:10.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f228406c680 0x7f228406eb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:10.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f229c1065b0 0x7f229c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:10.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 --2- 172.21.15.115:0/1573569659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f229c1077b0 0x7f229c19c300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:10.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 >> 172.21.15.115:0/1573569659 conn(0x7f229c075740 msgr2=0x7f229c10a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:10.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 shutdown_connections 2024-03-22T10:37:10.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:10.968+0000 7f22a3000700 1 -- 172.21.15.115:0/1573569659 wait complete. 2024-03-22T10:37:11.690 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:37:11.690 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-22T10:37:11.690 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-22T10:37:11.851 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-22T10:37:11.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:11 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1573569659' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T10:37:11.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:11 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1573569659' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T10:37:12.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:11 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1573569659' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T10:37:12.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:11 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1573569659' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T10:37:12.227 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi150/config 2024-03-22T10:37:13.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:12 smithi115 ceph-mon[29111]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:13.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:12 smithi150 ceph-mon[34518]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:14.216 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:13 smithi150 ceph-mon[34518]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:14.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:13 smithi115 ceph-mon[29111]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:14.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.404+0000 7f23e0a00700 1 -- 172.21.15.150:0/8006964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc103340 msgr2=0x7f23dc105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:14.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.404+0000 7f23e0a00700 1 --2- 172.21.15.150:0/8006964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc103340 0x7f23dc105720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009910 tx=0x7f23c4009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:14.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.404+0000 7f23e0a00700 1 -- 172.21.15.150:0/8006964 shutdown_connections 2024-03-22T10:37:14.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.404+0000 7f23e0a00700 1 --2- 172.21.15.150:0/8006964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc103340 0x7f23dc105720 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:14.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.404+0000 7f23e0a00700 1 --2- 172.21.15.150:0/8006964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23dc100a20 0x7f23dc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:14.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.404+0000 7f23e0a00700 1 -- 172.21.15.150:0/8006964 >> 172.21.15.150:0/8006964 conn(0x7f23dc0fa440 msgr2=0x7f23dc0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:14.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.405+0000 7f23e0a00700 1 -- 172.21.15.150:0/8006964 shutdown_connections 2024-03-22T10:37:14.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.405+0000 7f23e0a00700 1 -- 172.21.15.150:0/8006964 wait complete. 2024-03-22T10:37:14.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.406+0000 7f23e0a00700 1 Processor -- start 2024-03-22T10:37:14.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.407+0000 7f23e0a00700 1 -- start start 2024-03-22T10:37:14.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.407+0000 7f23e0a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc100a20 0x7f23dc100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:14.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.407+0000 7f23e0a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23dc103340 0x7f23dc0fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.407+0000 7f23e0a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0fef10 con 0x7f23dc103340 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.407+0000 7f23e0a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0ff050 con 0x7f23dc100a20 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.407+0000 7f23db400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc100a20 0x7f23dc100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.408+0000 7f23db400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc100a20 0x7f23dc100380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44554/0 (socket says 172.21.15.150:44554) 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.408+0000 7f23db400700 1 -- 172.21.15.150:0/2269827541 learned_addr learned my addr 172.21.15.150:0/2269827541 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.408+0000 7f23d3e00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23dc103340 0x7f23dc0fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.408+0000 7f23db400700 1 -- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23dc103340 msgr2=0x7f23dc0fe9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.408+0000 7f23db400700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23dc103340 0x7f23dc0fe9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.408+0000 7f23db400700 1 -- 172.21.15.150:0/2269827541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c40095f0 con 0x7f23dc100a20 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.408+0000 7f23d3e00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23dc103340 0x7f23dc0fe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:37:14.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.408+0000 7f23db400700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc100a20 0x7f23dc100380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00d8d0 tx=0x7f23cc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:14.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.409+0000 7f23d8c00700 1 -- 172.21.15.150:0/2269827541 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23cc009940 con 0x7f23dc100a20 2024-03-22T10:37:14.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.409+0000 7f23d8c00700 1 -- 172.21.15.150:0/2269827541 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f23cc010460 con 0x7f23dc100a20 2024-03-22T10:37:14.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.409+0000 7f23d8c00700 1 -- 172.21.15.150:0/2269827541 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23cc00f5f0 con 0x7f23dc100a20 2024-03-22T10:37:14.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.409+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc0ff330 con 0x7f23dc100a20 2024-03-22T10:37:14.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.409+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc0ff830 con 0x7f23dc100a20 2024-03-22T10:37:14.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.410+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc18f9b0 con 0x7f23dc100a20 2024-03-22T10:37:14.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.411+0000 7f23d8c00700 1 -- 172.21.15.150:0/2269827541 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f23cc0105d0 con 0x7f23dc100a20 2024-03-22T10:37:14.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.412+0000 7f23d8c00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f23c006c680 0x7f23c006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:14.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.412+0000 7f23d8c00700 1 -- 172.21.15.150:0/2269827541 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f23cc08b540 con 0x7f23dc100a20 2024-03-22T10:37:14.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.413+0000 7f23d3e00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f23c006c680 0x7f23c006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:14.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.414+0000 7f23d3e00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f23c006c680 0x7f23c006eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f23c400b5c0 tx=0x7f23c4005e20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:14.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.416+0000 7f23d8c00700 1 -- 172.21.15.150:0/2269827541 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f23cc0599f0 con 0x7f23dc100a20 2024-03-22T10:37:14.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.714+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f23dc02cc30 con 0x7f23dc100a20 2024-03-22T10:37:14.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.725+0000 7f23d8c00700 1 -- 172.21.15.150:0/2269827541 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f23cc059580 con 0x7f23dc100a20 2024-03-22T10:37:14.718 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2024-03-22T10:37:14.718 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBaX/1l56K8KhAAqx4K1uW3Rf12tDqK+5CHBg== 2024-03-22T10:37:14.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f23c006c680 msgr2=0x7f23c006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:14.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f23c006c680 0x7f23c006eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f23c400b5c0 tx=0x7f23c4005e20 comp rx=0 tx=0).stop 2024-03-22T10:37:14.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc100a20 msgr2=0x7f23dc100380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:14.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc100a20 0x7f23dc100380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00d8d0 tx=0x7f23cc00dc90 comp rx=0 tx=0).stop 2024-03-22T10:37:14.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 shutdown_connections 2024-03-22T10:37:14.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f23c006c680 0x7f23c006eb30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:14.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23dc100a20 0x7f23dc100380 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:14.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 --2- 172.21.15.150:0/2269827541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23dc103340 0x7f23dc0fe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:14.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 >> 172.21.15.150:0/2269827541 conn(0x7f23dc0fa440 msgr2=0x7f23dc0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:14.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 shutdown_connections 2024-03-22T10:37:14.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:37:14.729+0000 7f23e0a00700 1 -- 172.21.15.150:0/2269827541 wait complete. 2024-03-22T10:37:15.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:14 smithi115 ceph-mon[29111]: from='client.? 172.21.15.150:0/2269827541' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T10:37:15.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:14 smithi115 ceph-mon[29111]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T10:37:15.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:14 smithi115 ceph-mon[29111]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T10:37:15.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:14 smithi150 ceph-mon[34518]: from='client.? 172.21.15.150:0/2269827541' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T10:37:15.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:14 smithi150 ceph-mon[34518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T10:37:15.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:14 smithi150 ceph-mon[34518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T10:37:15.550 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:37:15.550 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-22T10:37:15.550 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-22T10:37:15.745 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-22T10:37:15.746 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-22T10:37:15.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-22T10:37:16.144 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:16.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:15 smithi150 ceph-mon[34518]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:15 smithi115 ceph-mon[29111]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.208+0000 7f51d7800700 1 -- 172.21.15.115:0/2840268967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 msgr2=0x7f51d01007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.208+0000 7f51d7800700 1 --2- 172.21.15.115:0/2840268967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 0x7f51d01007a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009910 tx=0x7f51c4009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.208+0000 7f51d7800700 1 -- 172.21.15.115:0/2840268967 shutdown_connections 2024-03-22T10:37:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.208+0000 7f51d7800700 1 --2- 172.21.15.115:0/2840268967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d0100d70 0x7f51d0104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.208+0000 7f51d7800700 1 --2- 172.21.15.115:0/2840268967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 0x7f51d01007a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.208+0000 7f51d7800700 1 -- 172.21.15.115:0/2840268967 >> 172.21.15.115:0/2840268967 conn(0x7f51d00fbc90 msgr2=0x7f51d00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.208+0000 7f51d7800700 1 -- 172.21.15.115:0/2840268967 shutdown_connections 2024-03-22T10:37:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.208+0000 7f51d7800700 1 -- 172.21.15.115:0/2840268967 wait complete. 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.209+0000 7f51d7800700 1 Processor -- start 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.210+0000 7f51d7800700 1 -- start start 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.210+0000 7f51d7800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 0x7f51d0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.210+0000 7f51d7800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d0100d70 0x7f51d0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.210+0000 7f51d7800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0198b70 con 0x7f51d01003d0 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.210+0000 7f51d7800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d019c900 con 0x7f51d0100d70 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.210+0000 7f51d4e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 0x7f51d0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.210+0000 7f51d4e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 0x7f51d0197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37122/0 (socket says 172.21.15.115:37122) 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.210+0000 7f51d4e00700 1 -- 172.21.15.115:0/4274719006 learned_addr learned my addr 172.21.15.115:0/4274719006 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.211+0000 7f51d4e00700 1 -- 172.21.15.115:0/4274719006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d0100d70 msgr2=0x7f51d0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.211+0000 7f51cfe00700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d0100d70 0x7f51d0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.211+0000 7f51d4e00700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d0100d70 0x7f51d0198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.211+0000 7f51d4e00700 1 -- 172.21.15.115:0/4274719006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40095f0 con 0x7f51d01003d0 2024-03-22T10:37:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.211+0000 7f51d4e00700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 0x7f51d0197f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51c4006010 tx=0x7f51c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.211+0000 7f51cd600700 1 -- 172.21.15.115:0/4274719006 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c401d070 con 0x7f51d01003d0 2024-03-22T10:37:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.211+0000 7f51cd600700 1 -- 172.21.15.115:0/4274719006 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51c400f460 con 0x7f51d01003d0 2024-03-22T10:37:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.211+0000 7f51cd600700 1 -- 172.21.15.115:0/4274719006 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c4017630 con 0x7f51d01003d0 2024-03-22T10:37:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.211+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d019cb80 con 0x7f51d01003d0 2024-03-22T10:37:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.212+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d019d070 con 0x7f51d01003d0 2024-03-22T10:37:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.213+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d01044c0 con 0x7f51d01003d0 2024-03-22T10:37:18.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.214+0000 7f51cd600700 1 -- 172.21.15.115:0/4274719006 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f51c400f5d0 con 0x7f51d01003d0 2024-03-22T10:37:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.215+0000 7f51cd600700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f51b806c680 0x7f51b806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.215+0000 7f51cd600700 1 -- 172.21.15.115:0/4274719006 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f51c408cb30 con 0x7f51d01003d0 2024-03-22T10:37:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.215+0000 7f51cfe00700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f51b806c680 0x7f51b806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.216+0000 7f51cfe00700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f51b806c680 0x7f51b806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51d0199570 tx=0x7f51bc005cf0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:18.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.220+0000 7f51cd600700 1 -- 172.21.15.115:0/4274719006 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f51c4057530 con 0x7f51d01003d0 2024-03-22T10:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.494+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f51d004ea50 con 0x7f51d01003d0 2024-03-22T10:37:18.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.502+0000 7f51cd600700 1 -- 172.21.15.115:0/4274719006 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+172863 (secure 0 0 0) 0x7f51c405ab50 con 0x7f51d01003d0 2024-03-22T10:37:18.497 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:37:18.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f51b806c680 msgr2=0x7f51b806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:18.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f51b806c680 0x7f51b806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51d0199570 tx=0x7f51bc005cf0 comp rx=0 tx=0).stop 2024-03-22T10:37:18.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 msgr2=0x7f51d0197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:18.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 0x7f51d0197f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51c4006010 tx=0x7f51c400be20 comp rx=0 tx=0).stop 2024-03-22T10:37:18.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 shutdown_connections 2024-03-22T10:37:18.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f51b806c680 0x7f51b806eb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:18.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51d01003d0 0x7f51d0197f50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:18.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 --2- 172.21.15.115:0/4274719006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d0100d70 0x7f51d0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:18.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 >> 172.21.15.115:0/4274719006 conn(0x7f51d00fbc90 msgr2=0x7f51d00fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:18.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 shutdown_connections 2024-03-22T10:37:18.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:18.506+0000 7f51d7800700 1 -- 172.21.15.115:0/4274719006 wait complete. 2024-03-22T10:37:18.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:18 smithi150 ceph-mon[34518]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:18.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:18 smithi115 ceph-mon[29111]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:19.267 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":19,"active_gid":14213,"active_name":"smithi115.iqzwsz","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1251137983},{"type":"v1","addr":"172.21.15.115:6801","nonce":1251137983}]},"active_addr":"172.21.15.115:6801/1251137983","active_change":"2024-03-22T10:32:25.174761+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14238,"name":"smithi150.fmoumv","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.115:8443/","prometheus":"http://172.21.15.115:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":3152688785}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":3333444965}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":617745307}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":1202069415}]}]} 2024-03-22T10:37:19.271 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-22T10:37:19.271 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-22T10:37:19.272 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T10:37:19.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:19 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/4274719006' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T10:37:19.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:19 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/4274719006' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T10:37:19.689 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:20.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:20 smithi115 ceph-mon[29111]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:20.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:20 smithi150 ceph-mon[34518]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:21.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.715+0000 7f1c9d200700 1 -- 172.21.15.115:0/2468327287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102fd0 msgr2=0x7f1c98106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:21.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.715+0000 7f1c9d200700 1 --2- 172.21.15.115:0/2468327287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102fd0 0x7f1c98106fa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009910 tx=0x7f1c80009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:21.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.716+0000 7f1c9d200700 1 -- 172.21.15.115:0/2468327287 shutdown_connections 2024-03-22T10:37:21.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.716+0000 7f1c9d200700 1 --2- 172.21.15.115:0/2468327287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102fd0 0x7f1c98106fa0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:21.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.716+0000 7f1c9d200700 1 --2- 172.21.15.115:0/2468327287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c98102630 0x7f1c98102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:21.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.716+0000 7f1c9d200700 1 -- 172.21.15.115:0/2468327287 >> 172.21.15.115:0/2468327287 conn(0x7f1c980fdef0 msgr2=0x7f1c98100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:21.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.716+0000 7f1c9d200700 1 -- 172.21.15.115:0/2468327287 shutdown_connections 2024-03-22T10:37:21.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.716+0000 7f1c9d200700 1 -- 172.21.15.115:0/2468327287 wait complete. 2024-03-22T10:37:21.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.717+0000 7f1c9d200700 1 Processor -- start 2024-03-22T10:37:21.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.717+0000 7f1c9d200700 1 -- start start 2024-03-22T10:37:21.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.717+0000 7f1c9d200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102630 0x7f1c981980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:21.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.717+0000 7f1c9d200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c98102fd0 0x7f1c981985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:21.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.717+0000 7f1c9d200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98198cc0 con 0x7f1c98102630 2024-03-22T10:37:21.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.717+0000 7f1c9d200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9819ca50 con 0x7f1c98102fd0 2024-03-22T10:37:21.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.718+0000 7f1c95e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102630 0x7f1c981980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:21.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.718+0000 7f1c95e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102630 0x7f1c981980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37132/0 (socket says 172.21.15.115:37132) 2024-03-22T10:37:21.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.718+0000 7f1c95e00700 1 -- 172.21.15.115:0/3671362291 learned_addr learned my addr 172.21.15.115:0/3671362291 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:21.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.718+0000 7f1c8d400700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c98102fd0 0x7f1c981985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:21.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.718+0000 7f1c95e00700 1 -- 172.21.15.115:0/3671362291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c98102fd0 msgr2=0x7f1c981985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:21.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.718+0000 7f1c95e00700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c98102fd0 0x7f1c981985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:21.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.718+0000 7f1c95e00700 1 -- 172.21.15.115:0/3671362291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800095f0 con 0x7f1c98102630 2024-03-22T10:37:21.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.719+0000 7f1c95e00700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102630 0x7f1c981980a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800b4b0 tx=0x7f1c8800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:21.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.719+0000 7f1c8f400700 1 -- 172.21.15.115:0/3671362291 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c88011800 con 0x7f1c98102630 2024-03-22T10:37:21.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.719+0000 7f1c8f400700 1 -- 172.21.15.115:0/3671362291 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1c88011e10 con 0x7f1c98102630 2024-03-22T10:37:21.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.719+0000 7f1c8f400700 1 -- 172.21.15.115:0/3671362291 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c8800f2e0 con 0x7f1c98102630 2024-03-22T10:37:21.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.719+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9819cd30 con 0x7f1c98102630 2024-03-22T10:37:21.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.719+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9819d280 con 0x7f1c98102630 2024-03-22T10:37:21.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.720+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9810a8d0 con 0x7f1c98102630 2024-03-22T10:37:21.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.721+0000 7f1c8f400700 1 -- 172.21.15.115:0/3671362291 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f1c88010380 con 0x7f1c98102630 2024-03-22T10:37:21.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.722+0000 7f1c8f400700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1c7806c680 0x7f1c7806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:21.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.722+0000 7f1c8f400700 1 -- 172.21.15.115:0/3671362291 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1c8808b090 con 0x7f1c98102630 2024-03-22T10:37:21.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.722+0000 7f1c8d400700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1c7806c680 0x7f1c7806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:21.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.723+0000 7f1c8d400700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1c7806c680 0x7f1c7806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c981996c0 tx=0x7f1c8001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:21.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.726+0000 7f1c8f400700 1 -- 172.21.15.115:0/3671362291 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1c88055be0 con 0x7f1c98102630 2024-03-22T10:37:21.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.942+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c9804ea50 con 0x7f1c98102630 2024-03-22T10:37:21.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.944+0000 7f1c8f400700 1 -- 172.21.15.115:0/3671362291 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11349 (secure 0 0 0) 0x7f1c88059200 con 0x7f1c98102630 2024-03-22T10:37:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:37:21.936 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","created":"2024-03-22T10:29:37.233959+0000","modified":"2024-03-22T10:36:59.634944+0000","last_up_change":"2024-03-22T10:36:58.621026+0000","last_in_change":"2024-03-22T10:36:35.075110+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T10:35:17.870309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a34c0f70-4e9d-4044-b7b4-b6099b74ba6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6803","nonce":764585829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6805","nonce":764585829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6809","nonce":764585829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6807","nonce":764585829}]},"public_addr":"172.21.15.115:6803/764585829","cluster_addr":"172.21.15.115:6805/764585829","heartbeat_back_addr":"172.21.15.115:6809/764585829","heartbeat_front_addr":"172.21.15.115:6807/764585829","state":["exists","up"]},{"osd":1,"uuid":"1037db4c-35bc-48da-8950-460c772d50b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6811","nonce":586118879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6813","nonce":586118879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6817","nonce":586118879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6815","nonce":586118879}]},"public_addr":"172.21.15.115:6811/586118879","cluster_addr":"172.21.15.115:6813/586118879","heartbeat_back_addr":"172.21.15.115:6817/586118879","heartbeat_front_addr":"172.21.15.115:6815/586118879","state":["exists","up"]},{"osd":2,"uuid":"4e685a22-30ee-4404-bba5-ff944af9b858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6819","nonce":285410766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6821","nonce":285410766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6825","nonce":285410766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6823","nonce":285410766}]},"public_addr":"172.21.15.115:6819/285410766","cluster_addr":"172.21.15.115:6821/285410766","heartbeat_back_addr":"172.21.15.115:6825/285410766","heartbeat_front_addr":"172.21.15.115:6823/285410766","state":["exists","up"]},{"osd":3,"uuid":"1273664e-16c8-46c1-bfb4-c51e3e84ea6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6801","nonce":3913650733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6803","nonce":3913650733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6807","nonce":3913650733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6805","nonce":3913650733}]},"public_addr":"172.21.15.150:6801/3913650733","cluster_addr":"172.21.15.150:6803/3913650733","heartbeat_back_addr":"172.21.15.150:6807/3913650733","heartbeat_front_addr":"172.21.15.150:6805/3913650733","state":["exists","up"]},{"osd":4,"uuid":"574fd0d5-c58a-4e16-961d-b384e52d82b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6809","nonce":792436265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6811","nonce":792436265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6815","nonce":792436265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6813","nonce":792436265}]},"public_addr":"172.21.15.150:6809/792436265","cluster_addr":"172.21.15.150:6811/792436265","heartbeat_back_addr":"172.21.15.150:6815/792436265","heartbeat_front_addr":"172.21.15.150:6813/792436265","state":["exists","up"]},{"osd":5,"uuid":"cc37270e-923d-45cf-accd-d4119dcc1a7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6817","nonce":2178668917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6819","nonce":2178668917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6823","nonce":2178668917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6821","nonce":2178668917}]},"public_addr":"172.21.15.150:6817/2178668917","cluster_addr":"172.21.15.150:6819/2178668917","heartbeat_back_addr":"172.21.15.150:6823/2178668917","heartbeat_front_addr":"172.21.15.150:6821/2178668917","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:04.542374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:39.181161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:16.028358+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:47.869896+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:22.969676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:57.504061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6801/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/901503025":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/2275951342":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/1384534494":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/450820820":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3451764091":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6801/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/400653309":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3025276768":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/3641576098":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3264237187":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/778669970":"2024-03-23T10:30:58.613511+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T10:37:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.946+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1c7806c680 msgr2=0x7f1c7806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.946+0000 7f1c9d200700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1c7806c680 0x7f1c7806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c981996c0 tx=0x7f1c8001a040 comp rx=0 tx=0).stop 2024-03-22T10:37:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.946+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102630 msgr2=0x7f1c981980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.946+0000 7f1c9d200700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102630 0x7f1c981980a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800b4b0 tx=0x7f1c8800b870 comp rx=0 tx=0).stop 2024-03-22T10:37:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.947+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 shutdown_connections 2024-03-22T10:37:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.947+0000 7f1c9d200700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1c7806c680 0x7f1c7806eb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.947+0000 7f1c9d200700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c98102630 0x7f1c981980a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.947+0000 7f1c9d200700 1 --2- 172.21.15.115:0/3671362291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c98102fd0 0x7f1c981985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.947+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 >> 172.21.15.115:0/3671362291 conn(0x7f1c980fdef0 msgr2=0x7f1c980ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.947+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 shutdown_connections 2024-03-22T10:37:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:21.947+0000 7f1c9d200700 1 -- 172.21.15.115:0/3671362291 wait complete. 2024-03-22T10:37:22.575 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:22 smithi115 ceph-mon[29111]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:22.575 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:22 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3671362291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:37:22.639 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-22T10:37:22.639 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T10:37:22.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:22 smithi150 ceph-mon[34518]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:22.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:22 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3671362291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:37:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:24.189 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:23 smithi115 ceph-mon[29111]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:24.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:23 smithi150 ceph-mon[34518]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:24.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.979+0000 7fb8c7e00700 1 -- 172.21.15.115:0/3418721194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c0073a00 msgr2=0x7fb8c010c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:24.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.979+0000 7fb8c7e00700 1 --2- 172.21.15.115:0/3418721194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c0073a00 0x7fb8c010c440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009910 tx=0x7fb8ac009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:24.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.979+0000 7fb8c7e00700 1 -- 172.21.15.115:0/3418721194 shutdown_connections 2024-03-22T10:37:24.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.979+0000 7fb8c7e00700 1 --2- 172.21.15.115:0/3418721194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c0073a00 0x7fb8c010c440 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:24.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.979+0000 7fb8c7e00700 1 --2- 172.21.15.115:0/3418721194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8c00730f0 0x7fb8c00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:24.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.979+0000 7fb8c7e00700 1 -- 172.21.15.115:0/3418721194 >> 172.21.15.115:0/3418721194 conn(0x7fb8c00fbc20 msgr2=0x7fb8c00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:24.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.979+0000 7fb8c7e00700 1 -- 172.21.15.115:0/3418721194 shutdown_connections 2024-03-22T10:37:24.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.979+0000 7fb8c7e00700 1 -- 172.21.15.115:0/3418721194 wait complete. 2024-03-22T10:37:24.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.980+0000 7fb8c7e00700 1 Processor -- start 2024-03-22T10:37:24.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.980+0000 7fb8c7e00700 1 -- start start 2024-03-22T10:37:24.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c00730f0 0x7fb8c0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:24.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8c0073a00 0x7fb8c01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:24.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c7e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c019c7e0 con 0x7fb8c00730f0 2024-03-22T10:37:24.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c7e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c019c950 con 0x7fb8c0073a00 2024-03-22T10:37:24.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c5400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c00730f0 0x7fb8c0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c5400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c00730f0 0x7fb8c0197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60192/0 (socket says 172.21.15.115:60192) 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c5400700 1 -- 172.21.15.115:0/1570466546 learned_addr learned my addr 172.21.15.115:0/1570466546 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c5400700 1 -- 172.21.15.115:0/1570466546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8c0073a00 msgr2=0x7fb8c01984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c5400700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8c0073a00 0x7fb8c01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.981+0000 7fb8c5400700 1 -- 172.21.15.115:0/1570466546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac0095f0 con 0x7fb8c00730f0 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.982+0000 7fb8c5400700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c00730f0 0x7fb8c0197e30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400c8a0 tx=0x7fb8b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.982+0000 7fb8be000700 1 -- 172.21.15.115:0/1570466546 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b400cea0 con 0x7fb8c00730f0 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.982+0000 7fb8be000700 1 -- 172.21.15.115:0/1570466546 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb8b4004830 con 0x7fb8c00730f0 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.982+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c019cc30 con 0x7fb8c00730f0 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.982+0000 7fb8be000700 1 -- 172.21.15.115:0/1570466546 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b40055b0 con 0x7fb8c00730f0 2024-03-22T10:37:24.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.982+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c019d180 con 0x7fb8c00730f0 2024-03-22T10:37:24.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.983+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c0109b40 con 0x7fb8c00730f0 2024-03-22T10:37:24.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.984+0000 7fb8be000700 1 -- 172.21.15.115:0/1570466546 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb8b4007ad0 con 0x7fb8c00730f0 2024-03-22T10:37:24.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.985+0000 7fb8be000700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb8a806c680 0x7fb8a806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:24.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.985+0000 7fb8be000700 1 -- 172.21.15.115:0/1570466546 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb8b408b290 con 0x7fb8c00730f0 2024-03-22T10:37:24.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.985+0000 7fb8c4a00700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb8a806c680 0x7fb8a806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:24.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.986+0000 7fb8c4a00700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb8a806c680 0x7fb8a806eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00b5c0 tx=0x7fb8ac005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:24.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:24.990+0000 7fb8be000700 1 -- 172.21.15.115:0/1570466546 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb8b4055c90 con 0x7fb8c00730f0 2024-03-22T10:37:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.203+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8c004ea50 con 0x7fb8c00730f0 2024-03-22T10:37:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:37:25.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.204+0000 7fb8be000700 1 -- 172.21.15.115:0/1570466546 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11349 (secure 0 0 0) 0x7fb8b40592b0 con 0x7fb8c00730f0 2024-03-22T10:37:25.197 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","created":"2024-03-22T10:29:37.233959+0000","modified":"2024-03-22T10:36:59.634944+0000","last_up_change":"2024-03-22T10:36:58.621026+0000","last_in_change":"2024-03-22T10:36:35.075110+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T10:35:17.870309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a34c0f70-4e9d-4044-b7b4-b6099b74ba6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6803","nonce":764585829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6805","nonce":764585829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6809","nonce":764585829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6807","nonce":764585829}]},"public_addr":"172.21.15.115:6803/764585829","cluster_addr":"172.21.15.115:6805/764585829","heartbeat_back_addr":"172.21.15.115:6809/764585829","heartbeat_front_addr":"172.21.15.115:6807/764585829","state":["exists","up"]},{"osd":1,"uuid":"1037db4c-35bc-48da-8950-460c772d50b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6811","nonce":586118879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6813","nonce":586118879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6817","nonce":586118879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6815","nonce":586118879}]},"public_addr":"172.21.15.115:6811/586118879","cluster_addr":"172.21.15.115:6813/586118879","heartbeat_back_addr":"172.21.15.115:6817/586118879","heartbeat_front_addr":"172.21.15.115:6815/586118879","state":["exists","up"]},{"osd":2,"uuid":"4e685a22-30ee-4404-bba5-ff944af9b858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6819","nonce":285410766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6821","nonce":285410766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6825","nonce":285410766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6823","nonce":285410766}]},"public_addr":"172.21.15.115:6819/285410766","cluster_addr":"172.21.15.115:6821/285410766","heartbeat_back_addr":"172.21.15.115:6825/285410766","heartbeat_front_addr":"172.21.15.115:6823/285410766","state":["exists","up"]},{"osd":3,"uuid":"1273664e-16c8-46c1-bfb4-c51e3e84ea6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6801","nonce":3913650733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6803","nonce":3913650733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6807","nonce":3913650733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6805","nonce":3913650733}]},"public_addr":"172.21.15.150:6801/3913650733","cluster_addr":"172.21.15.150:6803/3913650733","heartbeat_back_addr":"172.21.15.150:6807/3913650733","heartbeat_front_addr":"172.21.15.150:6805/3913650733","state":["exists","up"]},{"osd":4,"uuid":"574fd0d5-c58a-4e16-961d-b384e52d82b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6809","nonce":792436265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6811","nonce":792436265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6815","nonce":792436265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6813","nonce":792436265}]},"public_addr":"172.21.15.150:6809/792436265","cluster_addr":"172.21.15.150:6811/792436265","heartbeat_back_addr":"172.21.15.150:6815/792436265","heartbeat_front_addr":"172.21.15.150:6813/792436265","state":["exists","up"]},{"osd":5,"uuid":"cc37270e-923d-45cf-accd-d4119dcc1a7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6817","nonce":2178668917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6819","nonce":2178668917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6823","nonce":2178668917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6821","nonce":2178668917}]},"public_addr":"172.21.15.150:6817/2178668917","cluster_addr":"172.21.15.150:6819/2178668917","heartbeat_back_addr":"172.21.15.150:6823/2178668917","heartbeat_front_addr":"172.21.15.150:6821/2178668917","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:04.542374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:39.181161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:16.028358+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:47.869896+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:22.969676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:57.504061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6801/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/901503025":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/2275951342":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/1384534494":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/450820820":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3451764091":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6801/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/400653309":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3025276768":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/3641576098":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3264237187":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/778669970":"2024-03-23T10:30:58.613511+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T10:37:25.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb8a806c680 msgr2=0x7fb8a806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:25.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb8a806c680 0x7fb8a806eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00b5c0 tx=0x7fb8ac005960 comp rx=0 tx=0).stop 2024-03-22T10:37:25.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c00730f0 msgr2=0x7fb8c0197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:25.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c00730f0 0x7fb8c0197e30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400c8a0 tx=0x7fb8b400cc60 comp rx=0 tx=0).stop 2024-03-22T10:37:25.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 shutdown_connections 2024-03-22T10:37:25.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb8a806c680 0x7fb8a806eb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:25.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8c00730f0 0x7fb8c0197e30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:25.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 --2- 172.21.15.115:0/1570466546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8c0073a00 0x7fb8c01984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:25.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 >> 172.21.15.115:0/1570466546 conn(0x7fb8c00fbc20 msgr2=0x7fb8c0106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:25.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.207+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 shutdown_connections 2024-03-22T10:37:25.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:25.208+0000 7fb8c7e00700 1 -- 172.21.15.115:0/1570466546 wait complete. 2024-03-22T10:37:25.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:25 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1570466546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:37:25.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:37:25.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:25 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1570466546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:37:25.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:37:25.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-22T10:37:25.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-22T10:37:25.969 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-22T10:37:25.969 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-22T10:37:25.969 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-22T10:37:25.970 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-22T10:37:26.502 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:26.503 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:26.583 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:26.599 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:26.600 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:26.608 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:26.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:26 smithi150 ceph-mon[34518]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:26.665 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:26 smithi115 ceph-mon[29111]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:28.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:28 smithi150 ceph-mon[34518]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:28.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:28 smithi115 ceph-mon[29111]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:30.355 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:30 smithi115 ceph-mon[29111]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:30.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:30 smithi150 ceph-mon[34518]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.300+0000 7f722ae00700 1 -- 172.21.15.115:0/2962365695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72240ffbb0 msgr2=0x7f72240fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.300+0000 7f722ae00700 1 --2- 172.21.15.115:0/2962365695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72240ffbb0 0x7f72240fff80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7214009940 tx=0x7f7214009c50 comp rx=0 tx=0).stop 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.301+0000 7f722ae00700 1 -- 172.21.15.115:0/2962365695 shutdown_connections 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.301+0000 7f722ae00700 1 --2- 172.21.15.115:0/2962365695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72241004c0 0x7f722410c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.301+0000 7f722ae00700 1 --2- 172.21.15.115:0/2962365695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72240ffbb0 0x7f72240fff80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.301+0000 7f722ae00700 1 -- 172.21.15.115:0/2962365695 >> 172.21.15.115:0/2962365695 conn(0x7f7224074f50 msgr2=0x7f7224075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.301+0000 7f722ae00700 1 -- 172.21.15.115:0/2962365695 shutdown_connections 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.301+0000 7f722ae00700 1 -- 172.21.15.115:0/2962365695 wait complete. 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.302+0000 7f722ae00700 1 Processor -- start 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.302+0000 7f722ae00700 1 -- start start 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.302+0000 7f722ae00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72240ffbb0 0x7f7224197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.302+0000 7f722ae00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72241004c0 0x7f72241984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.302+0000 7f722ae00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7224198b80 con 0x7f72241004c0 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.302+0000 7f722ae00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722419c910 con 0x7f72240ffbb0 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.303+0000 7f7223400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72241004c0 0x7f72241984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.303+0000 7f7223400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72241004c0 0x7f72241984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38078/0 (socket says 172.21.15.115:38078) 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.303+0000 7f7223400700 1 -- 172.21.15.115:0/2353446019 learned_addr learned my addr 172.21.15.115:0/2353446019 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.303+0000 7f7223e00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72240ffbb0 0x7f7224197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.303+0000 7f7223400700 1 -- 172.21.15.115:0/2353446019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72240ffbb0 msgr2=0x7f7224197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.303+0000 7f7223400700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72240ffbb0 0x7f7224197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.303+0000 7f7223400700 1 -- 172.21.15.115:0/2353446019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140095f0 con 0x7f72241004c0 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.304+0000 7f7223400700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72241004c0 0x7f72241984a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f721000b4b0 tx=0x7f721000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.304+0000 7f7220c00700 1 -- 172.21.15.115:0/2353446019 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7210011800 con 0x7f72241004c0 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.304+0000 7f7220c00700 1 -- 172.21.15.115:0/2353446019 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7210011e10 con 0x7f72241004c0 2024-03-22T10:37:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.304+0000 7f7220c00700 1 -- 172.21.15.115:0/2353446019 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f721000f2e0 con 0x7f72241004c0 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.304+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722419cbf0 con 0x7f72241004c0 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.304+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722419d0f0 con 0x7f72241004c0 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.305+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f722410f810 con 0x7f72241004c0 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.307+0000 7f7220c00700 1 -- 172.21.15.115:0/2353446019 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7210010380 con 0x7f72241004c0 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.309+0000 7f7220c00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f720806c680 0x7f720806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.309+0000 7f7220c00700 1 -- 172.21.15.115:0/2353446019 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f721008b7a0 con 0x7f72241004c0 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.309+0000 7f7223e00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f720806c680 0x7f720806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.309+0000 7f7220c00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] conn(0x7f7208072350 0x7f7208074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.309+0000 7f7220c00700 1 -- 172.21.15.115:0/2353446019 --> [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7208074e10 con 0x7f7208072350 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.309+0000 7f7220c00700 1 -- 172.21.15.115:0/2353446019 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f721008bc00 con 0x7f72241004c0 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.310+0000 7f7228e00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] conn(0x7f7208072350 0x7f7208074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.310+0000 7f7223e00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f720806c680 0x7f720806eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7214006010 tx=0x7f7214005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.310+0000 7f7228e00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] conn(0x7f7208072350 0x7f7208074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.311+0000 7f7220c00700 1 -- 172.21.15.115:0/2353446019 <== osd.2 v2:172.21.15.115:6818/285410766 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7208074e10 con 0x7f7208072350 2024-03-22T10:37:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.339+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 --> [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f722404ea50 con 0x7f7208072350 2024-03-22T10:37:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f7220c00700 1 -- 172.21.15.115:0/2353446019 <== osd.2 v2:172.21.15.115:6818/285410766 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f722404ea50 con 0x7f7208072350 2024-03-22T10:37:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] conn(0x7f7208072350 msgr2=0x7f7208074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] conn(0x7f7208072350 0x7f7208074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f720806c680 msgr2=0x7f720806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f720806c680 0x7f720806eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7214006010 tx=0x7f7214005a90 comp rx=0 tx=0).stop 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72241004c0 msgr2=0x7f72241984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72241004c0 0x7f72241984a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f721000b4b0 tx=0x7f721000b870 comp rx=0 tx=0).stop 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 shutdown_connections 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6818/285410766,v1:172.21.15.115:6819/285410766] conn(0x7f7208072350 0x7f7208074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f720806c680 0x7f720806eb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.340+0000 7f722ae00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72240ffbb0 0x7f7224197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.341+0000 7f722ae00700 1 --2- 172.21.15.115:0/2353446019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72241004c0 0x7f72241984a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.341+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 >> 172.21.15.115:0/2353446019 conn(0x7f7224074f50 msgr2=0x7f72240fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.341+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 shutdown_connections 2024-03-22T10:37:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.341+0000 7f722ae00700 1 -- 172.21.15.115:0/2353446019 wait complete. 2024-03-22T10:37:32.638 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:32 smithi115 ceph-mon[29111]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:32.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:32 smithi150 ceph-mon[34518]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:33.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.414+0000 7fa382800700 1 -- 172.21.15.115:0/3876723882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c1003b0 msgr2=0x7fa37c100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.414+0000 7fa382800700 1 --2- 172.21.15.115:0/3876723882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c1003b0 0x7fa37c100780 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009910 tx=0x7fa36c009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.414+0000 7fa382800700 1 -- 172.21.15.115:0/3876723882 shutdown_connections 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.414+0000 7fa382800700 1 --2- 172.21.15.115:0/3876723882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c100d50 0x7fa37c104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.414+0000 7fa382800700 1 --2- 172.21.15.115:0/3876723882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c1003b0 0x7fa37c100780 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.414+0000 7fa382800700 1 -- 172.21.15.115:0/3876723882 >> 172.21.15.115:0/3876723882 conn(0x7fa37c0fbc30 msgr2=0x7fa37c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.414+0000 7fa382800700 1 -- 172.21.15.115:0/3876723882 shutdown_connections 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.414+0000 7fa382800700 1 -- 172.21.15.115:0/3876723882 wait complete. 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.415+0000 7fa382800700 1 Processor -- start 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.415+0000 7fa382800700 1 -- start start 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.416+0000 7fa382800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c1003b0 0x7fa37c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.416+0000 7fa382800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c100d50 0x7fa37c196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.416+0000 7fa382800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c196930 con 0x7fa37c100d50 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.416+0000 7fa382800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c19a6c0 con 0x7fa37c1003b0 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.416+0000 7fa37aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c100d50 0x7fa37c196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.416+0000 7fa37aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c100d50 0x7fa37c196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38092/0 (socket says 172.21.15.115:38092) 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.416+0000 7fa37aa00700 1 -- 172.21.15.115:0/3501585302 learned_addr learned my addr 172.21.15.115:0/3501585302 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.416+0000 7fa37b400700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c1003b0 0x7fa37c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.417+0000 7fa37aa00700 1 -- 172.21.15.115:0/3501585302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c1003b0 msgr2=0x7fa37c195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.417+0000 7fa37aa00700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c1003b0 0x7fa37c195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.417+0000 7fa37aa00700 1 -- 172.21.15.115:0/3501585302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0095f0 con 0x7fa37c100d50 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.417+0000 7fa37aa00700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c100d50 0x7fa37c196250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c8f0 tx=0x7fa36800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.417+0000 7fa373e00700 1 -- 172.21.15.115:0/3501585302 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa368007a50 con 0x7fa37c100d50 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.417+0000 7fa373e00700 1 -- 172.21.15.115:0/3501585302 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa368007bb0 con 0x7fa37c100d50 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.417+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c19a9a0 con 0x7fa37c100d50 2024-03-22T10:37:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.417+0000 7fa373e00700 1 -- 172.21.15.115:0/3501585302 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa368018630 con 0x7fa37c100d50 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.417+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c19aef0 con 0x7fa37c100d50 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.418+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa37c10d590 con 0x7fa37c100d50 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.419+0000 7fa373e00700 1 -- 172.21.15.115:0/3501585302 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa368007d20 con 0x7fa37c100d50 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.420+0000 7fa373e00700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa36006c680 0x7fa36006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.420+0000 7fa373e00700 1 -- 172.21.15.115:0/3501585302 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa36808b9d0 con 0x7fa37c100d50 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.421+0000 7fa37b400700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa36006c680 0x7fa36006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.421+0000 7fa373e00700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] conn(0x7fa360072350 0x7fa360074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.421+0000 7fa373e00700 1 -- 172.21.15.115:0/3501585302 --> [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa360074e10 con 0x7fa360072350 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.421+0000 7fa373e00700 1 -- 172.21.15.115:0/3501585302 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa36808be30 con 0x7fa37c100d50 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.421+0000 7fa37be00700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] conn(0x7fa360072350 0x7fa360074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.421+0000 7fa37b400700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa36006c680 0x7fa36006eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00b5c0 tx=0x7fa36c01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.422+0000 7fa37be00700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] conn(0x7fa360072350 0x7fa360074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.422+0000 7fa373e00700 1 -- 172.21.15.115:0/3501585302 <== osd.0 v2:172.21.15.115:6802/764585829 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa360074e10 con 0x7fa360072350 2024-03-22T10:37:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.439+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 --> [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa37c04ea50 con 0x7fa360072350 2024-03-22T10:37:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa373e00700 1 -- 172.21.15.115:0/3501585302 <== osd.0 v2:172.21.15.115:6802/764585829 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa37c04ea50 con 0x7fa360072350 2024-03-22T10:37:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] conn(0x7fa360072350 msgr2=0x7fa360074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] conn(0x7fa360072350 0x7fa360074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa36006c680 msgr2=0x7fa36006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa36006c680 0x7fa36006eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00b5c0 tx=0x7fa36c01a040 comp rx=0 tx=0).stop 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c100d50 msgr2=0x7fa37c196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c100d50 0x7fa37c196250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c8f0 tx=0x7fa36800ccb0 comp rx=0 tx=0).stop 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 shutdown_connections 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa36006c680 0x7fa36006eb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:6802/764585829,v1:172.21.15.115:6803/764585829] conn(0x7fa360072350 0x7fa360074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c1003b0 0x7fa37c195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 --2- 172.21.15.115:0/3501585302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c100d50 0x7fa37c196250 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.440+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 >> 172.21.15.115:0/3501585302 conn(0x7fa37c0fbc30 msgr2=0x7fa37c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.441+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 shutdown_connections 2024-03-22T10:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:32.441+0000 7fa382800700 1 -- 172.21.15.115:0/3501585302 wait complete. 2024-03-22T10:37:34.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:33 smithi150 ceph-mon[34518]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:34.203 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:33 smithi115 ceph-mon[29111]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:34.243 INFO:teuthology.orchestra.run.smithi115.stdout:81604378654 2024-03-22T10:37:34.243 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-22T10:37:34.617 INFO:teuthology.orchestra.run.smithi115.stdout:38654705709 2024-03-22T10:37:34.618 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-22T10:37:35.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.203+0000 7fdd2ac00700 1 -- 172.21.15.115:0/673273574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd241003d0 msgr2=0x7fdd241007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:35.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.203+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/673273574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd241003d0 0x7fdd241007a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009910 tx=0x7fdd14009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.203+0000 7fdd2ac00700 1 -- 172.21.15.115:0/673273574 shutdown_connections 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.203+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/673273574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd24100d70 0x7fdd24104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.203+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/673273574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd241003d0 0x7fdd241007a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.203+0000 7fdd2ac00700 1 -- 172.21.15.115:0/673273574 >> 172.21.15.115:0/673273574 conn(0x7fdd240fbc90 msgr2=0x7fdd240fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.204+0000 7fdd2ac00700 1 -- 172.21.15.115:0/673273574 shutdown_connections 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.204+0000 7fdd2ac00700 1 -- 172.21.15.115:0/673273574 wait complete. 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.205+0000 7fdd2ac00700 1 Processor -- start 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.205+0000 7fdd2ac00700 1 -- start start 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.205+0000 7fdd2ac00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd241003d0 0x7fdd24195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.205+0000 7fdd2ac00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd24100d70 0x7fdd24196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.205+0000 7fdd2ac00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd241969f0 con 0x7fdd24100d70 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.205+0000 7fdd2ac00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2419a780 con 0x7fdd241003d0 2024-03-22T10:37:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.206+0000 7fdd23400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd24100d70 0x7fdd24196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.206+0000 7fdd23e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd241003d0 0x7fdd24195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.206+0000 7fdd23400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd24100d70 0x7fdd24196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38116/0 (socket says 172.21.15.115:38116) 2024-03-22T10:37:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.206+0000 7fdd23400700 1 -- 172.21.15.115:0/1017030472 learned_addr learned my addr 172.21.15.115:0/1017030472 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.206+0000 7fdd23400700 1 -- 172.21.15.115:0/1017030472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd241003d0 msgr2=0x7fdd24195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.206+0000 7fdd23400700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd241003d0 0x7fdd24195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.206+0000 7fdd23400700 1 -- 172.21.15.115:0/1017030472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd140095f0 con 0x7fdd24100d70 2024-03-22T10:37:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.206+0000 7fdd23e00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd241003d0 0x7fdd24195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.206+0000 7fdd23400700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd24100d70 0x7fdd24196310 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000b480 tx=0x7fdd1000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.207+0000 7fdd20c00700 1 -- 172.21.15.115:0/1017030472 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd100118c0 con 0x7fdd24100d70 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.207+0000 7fdd20c00700 1 -- 172.21.15.115:0/1017030472 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdd10010380 con 0x7fdd24100d70 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.207+0000 7fdd20c00700 1 -- 172.21.15.115:0/1017030472 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd1000f4f0 con 0x7fdd24100d70 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.207+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2419aa60 con 0x7fdd24100d70 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.207+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2419afb0 con 0x7fdd24100d70 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.208+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd2410d660 con 0x7fdd24100d70 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.209+0000 7fdd20c00700 1 -- 172.21.15.115:0/1017030472 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fdd100104f0 con 0x7fdd24100d70 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.210+0000 7fdd20c00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdd0806c680 0x7fdd0806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.210+0000 7fdd20c00700 1 -- 172.21.15.115:0/1017030472 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdd1008b160 con 0x7fdd24100d70 2024-03-22T10:37:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.210+0000 7fdd23e00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdd0806c680 0x7fdd0806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.210+0000 7fdd20c00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] conn(0x7fdd08072350 0x7fdd08074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.210+0000 7fdd20c00700 1 -- 172.21.15.115:0/1017030472 --> [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd08074e10 con 0x7fdd08072350 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.210+0000 7fdd20c00700 1 -- 172.21.15.115:0/1017030472 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd1008d790 con 0x7fdd24100d70 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.211+0000 7fdd28c00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] conn(0x7fdd08072350 0x7fdd08074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.211+0000 7fdd23e00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdd0806c680 0x7fdd0806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd14006010 tx=0x7fdd1401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.211+0000 7fdd28c00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] conn(0x7fdd08072350 0x7fdd08074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.212+0000 7fdd20c00700 1 -- 172.21.15.115:0/1017030472 <== osd.1 v2:172.21.15.115:6810/586118879 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdd08074e10 con 0x7fdd08072350 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.232+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 --> [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdd2404ea50 con 0x7fdd08072350 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd20c00700 1 -- 172.21.15.115:0/1017030472 <== osd.1 v2:172.21.15.115:6810/586118879 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdd2404ea50 con 0x7fdd08072350 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] conn(0x7fdd08072350 msgr2=0x7fdd08074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] conn(0x7fdd08072350 0x7fdd08074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdd0806c680 msgr2=0x7fdd0806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdd0806c680 0x7fdd0806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd14006010 tx=0x7fdd1401a040 comp rx=0 tx=0).stop 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd24100d70 msgr2=0x7fdd24196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd24100d70 0x7fdd24196310 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000b480 tx=0x7fdd1000b840 comp rx=0 tx=0).stop 2024-03-22T10:37:35.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 shutdown_connections 2024-03-22T10:37:35.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdd0806c680 0x7fdd0806eb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:35.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd241003d0 0x7fdd24195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:35.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:6810/586118879,v1:172.21.15.115:6811/586118879] conn(0x7fdd08072350 0x7fdd08074760 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:35.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.233+0000 7fdd2ac00700 1 --2- 172.21.15.115:0/1017030472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd24100d70 0x7fdd24196310 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:35.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.234+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 >> 172.21.15.115:0/1017030472 conn(0x7fdd240fbc90 msgr2=0x7fdd240fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:35.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.234+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 shutdown_connections 2024-03-22T10:37:35.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:35.234+0000 7fdd2ac00700 1 -- 172.21.15.115:0/1017030472 wait complete. 2024-03-22T10:37:36.319 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:36 smithi115 ceph-mon[29111]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:36.322 INFO:teuthology.orchestra.run.smithi115.stdout:60129542180 2024-03-22T10:37:36.322 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-22T10:37:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.523+0000 7f60c9800700 1 -- 172.21.15.115:0/3998389158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 msgr2=0x7f60c41016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.523+0000 7f60c9800700 1 --2- 172.21.15.115:0/3998389158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 0x7f60c41016f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009910 tx=0x7f60b4009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.523+0000 7f60c9800700 1 -- 172.21.15.115:0/3998389158 shutdown_connections 2024-03-22T10:37:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.523+0000 7f60c9800700 1 --2- 172.21.15.115:0/3998389158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60c4101c30 0x7f60c410a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.523+0000 7f60c9800700 1 --2- 172.21.15.115:0/3998389158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 0x7f60c41016f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.523+0000 7f60c9800700 1 -- 172.21.15.115:0/3998389158 >> 172.21.15.115:0/3998389158 conn(0x7f60c40fab80 msgr2=0x7f60c40fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.523+0000 7f60c9800700 1 -- 172.21.15.115:0/3998389158 shutdown_connections 2024-03-22T10:37:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.523+0000 7f60c9800700 1 -- 172.21.15.115:0/3998389158 wait complete. 2024-03-22T10:37:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.524+0000 7f60c9800700 1 Processor -- start 2024-03-22T10:37:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.524+0000 7f60c9800700 1 -- start start 2024-03-22T10:37:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.525+0000 7f60c9800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 0x7f60c40fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.525+0000 7f60c9800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60c4101c30 0x7f60c40ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.525+0000 7f60c9800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c40ffa00 con 0x7f60c4101320 2024-03-22T10:37:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.525+0000 7f60c9800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c40ffb40 con 0x7f60c4101c30 2024-03-22T10:37:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.525+0000 7f60c2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 0x7f60c40fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.525+0000 7f60c2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 0x7f60c40fef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38122/0 (socket says 172.21.15.115:38122) 2024-03-22T10:37:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.525+0000 7f60c2a00700 1 -- 172.21.15.115:0/3704198094 learned_addr learned my addr 172.21.15.115:0/3704198094 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.526+0000 7f60c2000700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60c4101c30 0x7f60c40ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.526+0000 7f60c2a00700 1 -- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60c4101c30 msgr2=0x7f60c40ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.526+0000 7f60c2a00700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60c4101c30 0x7f60c40ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.526+0000 7f60c2a00700 1 -- 172.21.15.115:0/3704198094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b40095f0 con 0x7f60c4101320 2024-03-22T10:37:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.526+0000 7f60c2a00700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 0x7f60c40fef80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60b4006010 tx=0x7f60b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.526+0000 7f60bb400700 1 -- 172.21.15.115:0/3704198094 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60b401d070 con 0x7f60c4101320 2024-03-22T10:37:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.526+0000 7f60bb400700 1 -- 172.21.15.115:0/3704198094 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f60b400f460 con 0x7f60c4101320 2024-03-22T10:37:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.526+0000 7f60bb400700 1 -- 172.21.15.115:0/3704198094 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60b4017630 con 0x7f60c4101320 2024-03-22T10:37:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.527+0000 7f60c9800700 1 -- 172.21.15.115:0/3704198094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c40ffdc0 con 0x7f60c4101320 2024-03-22T10:37:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.527+0000 7f60c9800700 1 -- 172.21.15.115:0/3704198094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c41a2200 con 0x7f60c4101320 2024-03-22T10:37:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.528+0000 7f60b8c00700 1 -- 172.21.15.115:0/3704198094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f609c000fc0 con 0x7f60c4101320 2024-03-22T10:37:36.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.529+0000 7f60bb400700 1 -- 172.21.15.115:0/3704198094 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f60b400f5d0 con 0x7f60c4101320 2024-03-22T10:37:36.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.530+0000 7f60bb400700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f60a806c680 0x7f60a806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:36.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.531+0000 7f60bb400700 1 -- 172.21.15.115:0/3704198094 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f60b408cb70 con 0x7f60c4101320 2024-03-22T10:37:36.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.531+0000 7f60bb400700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] conn(0x7f60a8072350 0x7f60a8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:36.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.531+0000 7f60bb400700 1 -- 172.21.15.115:0/3704198094 --> [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60a8074e10 con 0x7f60a8072350 2024-03-22T10:37:36.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.531+0000 7f60bb400700 1 -- 172.21.15.115:0/3704198094 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b40912a0 con 0x7f60c4101320 2024-03-22T10:37:36.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.531+0000 7f60c2000700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f60a806c680 0x7f60a806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:36.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.532+0000 7f60c3400700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] conn(0x7f60a8072350 0x7f60a8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:36.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.532+0000 7f60c2000700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f60a806c680 0x7f60a806eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60b0005ea0 tx=0x7f60b0005e30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:36.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.533+0000 7f60c3400700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] conn(0x7f60a8072350 0x7f60a8074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:36.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.535+0000 7f60bb400700 1 -- 172.21.15.115:0/3704198094 <== osd.3 v2:172.21.15.150:6800/3913650733 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f60a8074e10 con 0x7f60a8072350 2024-03-22T10:37:36.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.568+0000 7f60b8c00700 1 -- 172.21.15.115:0/3704198094 --> [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f609c002cb0 con 0x7f60a8072350 2024-03-22T10:37:36.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.569+0000 7f60bb400700 1 -- 172.21.15.115:0/3704198094 <== osd.3 v2:172.21.15.150:6800/3913650733 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f609c002cb0 con 0x7f60a8072350 2024-03-22T10:37:36.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 -- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] conn(0x7f60a8072350 msgr2=0x7f60a8074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:36.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] conn(0x7f60a8072350 0x7f60a8074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 -- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f60a806c680 msgr2=0x7f60a806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f60a806c680 0x7f60a806eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60b0005ea0 tx=0x7f60b0005e30 comp rx=0 tx=0).stop 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 -- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 msgr2=0x7f60c40fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 0x7f60c40fef80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60b4006010 tx=0x7f60b400be20 comp rx=0 tx=0).stop 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 -- 172.21.15.115:0/3704198094 shutdown_connections 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f60a806c680 0x7f60a806eb30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60c4101320 0x7f60c40fef80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:6800/3913650733,v1:172.21.15.150:6801/3913650733] conn(0x7f60a8072350 0x7f60a8074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 --2- 172.21.15.115:0/3704198094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60c4101c30 0x7f60c40ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 -- 172.21.15.115:0/3704198094 >> 172.21.15.115:0/3704198094 conn(0x7f60c40fab80 msgr2=0x7f60c4104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 -- 172.21.15.115:0/3704198094 shutdown_connections 2024-03-22T10:37:36.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:36.570+0000 7f60b8c00700 1 -- 172.21.15.115:0/3704198094 wait complete. 2024-03-22T10:37:36.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:36 smithi150 ceph-mon[34518]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.204+0000 7ff761c00700 1 -- 172.21.15.115:0/3244511530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 msgr2=0x7ff75c100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.204+0000 7ff761c00700 1 --2- 172.21.15.115:0/3244511530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 0x7ff75c100780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009910 tx=0x7ff74c009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.204+0000 7ff761c00700 1 -- 172.21.15.115:0/3244511530 shutdown_connections 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.204+0000 7ff761c00700 1 --2- 172.21.15.115:0/3244511530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff75c100d50 0x7ff75c104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.204+0000 7ff761c00700 1 --2- 172.21.15.115:0/3244511530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 0x7ff75c100780 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.204+0000 7ff761c00700 1 -- 172.21.15.115:0/3244511530 >> 172.21.15.115:0/3244511530 conn(0x7ff75c0fbc30 msgr2=0x7ff75c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.204+0000 7ff761c00700 1 -- 172.21.15.115:0/3244511530 shutdown_connections 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.204+0000 7ff761c00700 1 -- 172.21.15.115:0/3244511530 wait complete. 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.205+0000 7ff761c00700 1 Processor -- start 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.205+0000 7ff761c00700 1 -- start start 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff761c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 0x7ff75c195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff761c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff75c100d50 0x7ff75c196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff761c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c196950 con 0x7ff75c1003b0 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff761c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c19a6e0 con 0x7ff75c100d50 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff75aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 0x7ff75c195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff75aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 0x7ff75c195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38132/0 (socket says 172.21.15.115:38132) 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff75aa00700 1 -- 172.21.15.115:0/1411047906 learned_addr learned my addr 172.21.15.115:0/1411047906 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff75a000700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff75c100d50 0x7ff75c196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff75aa00700 1 -- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff75c100d50 msgr2=0x7ff75c196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff75aa00700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff75c100d50 0x7ff75c196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff75aa00700 1 -- 172.21.15.115:0/1411047906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c0095f0 con 0x7ff75c1003b0 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.206+0000 7ff75a000700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff75c100d50 0x7ff75c196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:37:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.207+0000 7ff75aa00700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 0x7ff75c195d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00b5c0 tx=0x7ff74c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.207+0000 7ff753400700 1 -- 172.21.15.115:0/1411047906 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c01d070 con 0x7ff75c1003b0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.207+0000 7ff753400700 1 -- 172.21.15.115:0/1411047906 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff74c022470 con 0x7ff75c1003b0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.207+0000 7ff753400700 1 -- 172.21.15.115:0/1411047906 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c00f670 con 0x7ff75c1003b0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.207+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c19a960 con 0x7ff75c1003b0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.207+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c19ae00 con 0x7ff75c1003b0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.208+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff75c10d5b0 con 0x7ff75c1003b0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.209+0000 7ff753400700 1 -- 172.21.15.115:0/1411047906 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7ff74c004c30 con 0x7ff75c1003b0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.210+0000 7ff753400700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff74006c680 0x7ff74006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.210+0000 7ff753400700 1 -- 172.21.15.115:0/1411047906 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff74c08cb10 con 0x7ff75c1003b0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.210+0000 7ff75a000700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff74006c680 0x7ff74006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.211+0000 7ff753400700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] conn(0x7ff740072350 0x7ff740074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.211+0000 7ff753400700 1 -- 172.21.15.115:0/1411047906 --> [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff740074e10 con 0x7ff740072350 2024-03-22T10:37:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.211+0000 7ff753400700 1 -- 172.21.15.115:0/1411047906 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff74c0912a0 con 0x7ff75c1003b0 2024-03-22T10:37:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.211+0000 7ff75a000700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff74006c680 0x7ff74006eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff748005fd0 tx=0x7ff748005dc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.211+0000 7ff75b400700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] conn(0x7ff740072350 0x7ff740074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.212+0000 7ff75b400700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] conn(0x7ff740072350 0x7ff740074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.214+0000 7ff753400700 1 -- 172.21.15.115:0/1411047906 <== osd.5 v2:172.21.15.150:6816/2178668917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff740074e10 con 0x7ff740072350 2024-03-22T10:37:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.231+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 --> [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff75c04ea50 con 0x7ff740072350 2024-03-22T10:37:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.232+0000 7ff753400700 1 -- 172.21.15.115:0/1411047906 <== osd.5 v2:172.21.15.150:6816/2178668917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff75c04ea50 con 0x7ff740072350 2024-03-22T10:37:37.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] conn(0x7ff740072350 msgr2=0x7ff740074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:37.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] conn(0x7ff740072350 0x7ff740074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff74006c680 msgr2=0x7ff74006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff74006c680 0x7ff74006eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff748005fd0 tx=0x7ff748005dc0 comp rx=0 tx=0).stop 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 msgr2=0x7ff75c195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 0x7ff75c195d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00b5c0 tx=0x7ff74c00be00 comp rx=0 tx=0).stop 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 shutdown_connections 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:6816/2178668917,v1:172.21.15.150:6817/2178668917] conn(0x7ff740072350 0x7ff740074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff74006c680 0x7ff74006eb30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff75c1003b0 0x7ff75c195d30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 --2- 172.21.15.115:0/1411047906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff75c100d50 0x7ff75c196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 >> 172.21.15.115:0/1411047906 conn(0x7ff75c0fbc30 msgr2=0x7ff75c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 shutdown_connections 2024-03-22T10:37:37.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:37.233+0000 7ff761c00700 1 -- 172.21.15.115:0/1411047906 wait complete. 2024-03-22T10:37:37.651 INFO:teuthology.orchestra.run.smithi115.stdout:111669149720 2024-03-22T10:37:37.652 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-22T10:37:38.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.384+0000 7f68e5c00700 1 -- 172.21.15.115:0/1019376696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e00730f0 msgr2=0x7f68e00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:38.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.384+0000 7f68e5c00700 1 --2- 172.21.15.115:0/1019376696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e00730f0 0x7f68e00734c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009910 tx=0x7f68d0009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:38.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.384+0000 7f68e5c00700 1 -- 172.21.15.115:0/1019376696 shutdown_connections 2024-03-22T10:37:38.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.384+0000 7f68e5c00700 1 --2- 172.21.15.115:0/1019376696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e0073a00 0x7f68e0110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:38.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.384+0000 7f68e5c00700 1 --2- 172.21.15.115:0/1019376696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e00730f0 0x7f68e00734c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:38.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.384+0000 7f68e5c00700 1 -- 172.21.15.115:0/1019376696 >> 172.21.15.115:0/1019376696 conn(0x7f68e00fbc30 msgr2=0x7f68e00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:38.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.384+0000 7f68e5c00700 1 -- 172.21.15.115:0/1019376696 shutdown_connections 2024-03-22T10:37:38.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.385+0000 7f68e5c00700 1 -- 172.21.15.115:0/1019376696 wait complete. 2024-03-22T10:37:38.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.386+0000 7f68e5c00700 1 Processor -- start 2024-03-22T10:37:38.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.387+0000 7f68e5c00700 1 -- start start 2024-03-22T10:37:38.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.387+0000 7f68e5c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e00730f0 0x7f68e019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:38.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.387+0000 7f68e5c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e0073a00 0x7f68e019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:38.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.387+0000 7f68e5c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e019cfa0 con 0x7f68e0073a00 2024-03-22T10:37:38.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.387+0000 7f68e5c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e01a0d30 con 0x7f68e00730f0 2024-03-22T10:37:38.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.387+0000 7f68de000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e0073a00 0x7f68e019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:38.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.388+0000 7f68de000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e0073a00 0x7f68e019c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38156/0 (socket says 172.21.15.115:38156) 2024-03-22T10:37:38.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.388+0000 7f68de000700 1 -- 172.21.15.115:0/2284027361 learned_addr learned my addr 172.21.15.115:0/2284027361 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:38.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.388+0000 7f68dea00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e00730f0 0x7f68e019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:38.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.388+0000 7f68de000700 1 -- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e00730f0 msgr2=0x7f68e019c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:38.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.388+0000 7f68de000700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e00730f0 0x7f68e019c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:38.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.388+0000 7f68de000700 1 -- 172.21.15.115:0/2284027361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d00095f0 con 0x7f68e0073a00 2024-03-22T10:37:38.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.389+0000 7f68de000700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e0073a00 0x7f68e019c8c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b4b0 tx=0x7f68cc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:38.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.389+0000 7f68d7400700 1 -- 172.21.15.115:0/2284027361 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68cc011800 con 0x7f68e0073a00 2024-03-22T10:37:38.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.389+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e01a1010 con 0x7f68e0073a00 2024-03-22T10:37:38.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.389+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e01a1560 con 0x7f68e0073a00 2024-03-22T10:37:38.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.389+0000 7f68d7400700 1 -- 172.21.15.115:0/2284027361 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f68cc011e10 con 0x7f68e0073a00 2024-03-22T10:37:38.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.389+0000 7f68d7400700 1 -- 172.21.15.115:0/2284027361 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68cc00f2e0 con 0x7f68e0073a00 2024-03-22T10:37:38.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.390+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68c0000ff0 con 0x7f68e0073a00 2024-03-22T10:37:38.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.392+0000 7f68d7400700 1 -- 172.21.15.115:0/2284027361 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f68cc011960 con 0x7f68e0073a00 2024-03-22T10:37:38.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.393+0000 7f68d7400700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f68c406c680 0x7f68c406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:38.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.393+0000 7f68d7400700 1 -- 172.21.15.115:0/2284027361 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f68cc08b5f0 con 0x7f68e0073a00 2024-03-22T10:37:38.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.394+0000 7f68d7400700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] conn(0x7f68c4072350 0x7f68c4074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:38.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.394+0000 7f68d7400700 1 -- 172.21.15.115:0/2284027361 --> [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68c4074e10 con 0x7f68c4072350 2024-03-22T10:37:38.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.394+0000 7f68d7400700 1 -- 172.21.15.115:0/2284027361 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68cc08b9b0 con 0x7f68e0073a00 2024-03-22T10:37:38.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.394+0000 7f68dea00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f68c406c680 0x7f68c406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:38.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.395+0000 7f68df400700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] conn(0x7f68c4072350 0x7f68c4074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:38.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.395+0000 7f68dea00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f68c406c680 0x7f68c406eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68d0006010 tx=0x7f68d001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:38.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.396+0000 7f68df400700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] conn(0x7f68c4072350 0x7f68c4074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:38.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.398+0000 7f68d7400700 1 -- 172.21.15.115:0/2284027361 <== osd.4 v2:172.21.15.150:6808/792436265 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f68c4074e10 con 0x7f68c4072350 2024-03-22T10:37:38.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.416+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 --> [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f68c0002ce0 con 0x7f68c4072350 2024-03-22T10:37:38.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.417+0000 7f68d7400700 1 -- 172.21.15.115:0/2284027361 <== osd.4 v2:172.21.15.150:6808/792436265 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f68c0002ce0 con 0x7f68c4072350 2024-03-22T10:37:38.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] conn(0x7f68c4072350 msgr2=0x7f68c4074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:38.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] conn(0x7f68c4072350 0x7f68c4074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f68c406c680 msgr2=0x7f68c406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f68c406c680 0x7f68c406eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68d0006010 tx=0x7f68d001a040 comp rx=0 tx=0).stop 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e0073a00 msgr2=0x7f68e019c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e0073a00 0x7f68e019c8c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b4b0 tx=0x7f68cc00b7c0 comp rx=0 tx=0).stop 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 shutdown_connections 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:6808/792436265,v1:172.21.15.150:6809/792436265] conn(0x7f68c4072350 0x7f68c4074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f68c406c680 0x7f68c406eb30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e00730f0 0x7f68e019c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 --2- 172.21.15.115:0/2284027361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68e0073a00 0x7f68e019c8c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 >> 172.21.15.115:0/2284027361 conn(0x7f68e00fbc30 msgr2=0x7f68e0102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.418+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 shutdown_connections 2024-03-22T10:37:38.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:38.419+0000 7f68e5c00700 1 -- 172.21.15.115:0/2284027361 wait complete. 2024-03-22T10:37:38.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:38 smithi115 ceph-mon[29111]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:38.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:38 smithi150 ceph-mon[34518]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:39.164 INFO:teuthology.orchestra.run.smithi115.stdout:146028888081 2024-03-22T10:37:39.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-22T10:37:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:39.611 INFO:teuthology.orchestra.run.smithi115.stdout:167503724554 2024-03-22T10:37:39.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-22T10:37:39.686 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:39.690 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:39.690 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:39.695 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:40.441 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:40 smithi115 ceph-mon[29111]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:40.441 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:37:40.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:40 smithi150 ceph-mon[34518]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:40.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:40 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:37:42.604 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:42 smithi115 ceph-mon[29111]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:42.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:42 smithi150 ceph-mon[34518]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:42.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.659+0000 7fb175000700 1 -- 172.21.15.115:0/2380038214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102dd0 msgr2=0x7fb170106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:42.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.659+0000 7fb175000700 1 --2- 172.21.15.115:0/2380038214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102dd0 0x7fb170106da0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb158009910 tx=0x7fb158009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:42.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.659+0000 7fb175000700 1 -- 172.21.15.115:0/2380038214 shutdown_connections 2024-03-22T10:37:42.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.659+0000 7fb175000700 1 --2- 172.21.15.115:0/2380038214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102dd0 0x7fb170106da0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:42.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.659+0000 7fb175000700 1 --2- 172.21.15.115:0/2380038214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170102430 0x7fb170102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:42.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.659+0000 7fb175000700 1 -- 172.21.15.115:0/2380038214 >> 172.21.15.115:0/2380038214 conn(0x7fb1700fdd10 msgr2=0x7fb170100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:42.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.659+0000 7fb175000700 1 -- 172.21.15.115:0/2380038214 shutdown_connections 2024-03-22T10:37:42.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.659+0000 7fb175000700 1 -- 172.21.15.115:0/2380038214 wait complete. 2024-03-22T10:37:42.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.660+0000 7fb175000700 1 Processor -- start 2024-03-22T10:37:42.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.661+0000 7fb175000700 1 -- start start 2024-03-22T10:37:42.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.661+0000 7fb175000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102430 0x7fb170197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:42.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.661+0000 7fb175000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170102dd0 0x7fb170198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:42.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.661+0000 7fb175000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170198a20 con 0x7fb170102430 2024-03-22T10:37:42.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.661+0000 7fb175000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17019c7b0 con 0x7fb170102dd0 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.661+0000 7fb16f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102430 0x7fb170197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.661+0000 7fb16f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102430 0x7fb170197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34470/0 (socket says 172.21.15.115:34470) 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.661+0000 7fb16f400700 1 -- 172.21.15.115:0/1795189446 learned_addr learned my addr 172.21.15.115:0/1795189446 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.662+0000 7fb167e00700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170102dd0 0x7fb170198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.662+0000 7fb16f400700 1 -- 172.21.15.115:0/1795189446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170102dd0 msgr2=0x7fb170198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.662+0000 7fb16f400700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170102dd0 0x7fb170198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.662+0000 7fb16f400700 1 -- 172.21.15.115:0/1795189446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1580095f0 con 0x7fb170102430 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.662+0000 7fb16f400700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102430 0x7fb170197e00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb16000b4b0 tx=0x7fb16000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.662+0000 7fb16cc00700 1 -- 172.21.15.115:0/1795189446 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb160011800 con 0x7fb170102430 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.663+0000 7fb16cc00700 1 -- 172.21.15.115:0/1795189446 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb160011e10 con 0x7fb170102430 2024-03-22T10:37:42.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.663+0000 7fb16cc00700 1 -- 172.21.15.115:0/1795189446 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb16000f2e0 con 0x7fb170102430 2024-03-22T10:37:42.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.663+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17019ca90 con 0x7fb170102430 2024-03-22T10:37:42.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.663+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17019cfe0 con 0x7fb170102430 2024-03-22T10:37:42.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.664+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17010a6d0 con 0x7fb170102430 2024-03-22T10:37:42.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.665+0000 7fb16cc00700 1 -- 172.21.15.115:0/1795189446 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb160010380 con 0x7fb170102430 2024-03-22T10:37:42.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.666+0000 7fb16cc00700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb15406c680 0x7fb15406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:42.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.667+0000 7fb16cc00700 1 -- 172.21.15.115:0/1795189446 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb16008b130 con 0x7fb170102430 2024-03-22T10:37:42.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.667+0000 7fb167e00700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb15406c680 0x7fb15406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:42.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.668+0000 7fb167e00700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb15406c680 0x7fb15406eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb170199420 tx=0x7fb15801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:42.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.671+0000 7fb16cc00700 1 -- 172.21.15.115:0/1795189446 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb160055be0 con 0x7fb170102430 2024-03-22T10:37:42.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.893+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb17004f2a0 con 0x7fb170102430 2024-03-22T10:37:42.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.894+0000 7fb16cc00700 1 -- 172.21.15.115:0/1795189446 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb160059200 con 0x7fb170102430 2024-03-22T10:37:42.887 INFO:teuthology.orchestra.run.smithi115.stdout:81604378655 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.897+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb15406c680 msgr2=0x7fb15406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.897+0000 7fb175000700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb15406c680 0x7fb15406eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb170199420 tx=0x7fb15801a040 comp rx=0 tx=0).stop 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.897+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102430 msgr2=0x7fb170197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.897+0000 7fb175000700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102430 0x7fb170197e00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb16000b4b0 tx=0x7fb16000b870 comp rx=0 tx=0).stop 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.897+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 shutdown_connections 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.897+0000 7fb175000700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb15406c680 0x7fb15406eb30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.897+0000 7fb175000700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb170102430 0x7fb170197e00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.897+0000 7fb175000700 1 --2- 172.21.15.115:0/1795189446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170102dd0 0x7fb170198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.897+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 >> 172.21.15.115:0/1795189446 conn(0x7fb1700fdd10 msgr2=0x7fb1700ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.898+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 shutdown_connections 2024-03-22T10:37:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:42.898+0000 7fb175000700 1 -- 172.21.15.115:0/1795189446 wait complete. 2024-03-22T10:37:43.494 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:43 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1795189446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T10:37:43.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:43 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1795189446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T10:37:44.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:44 smithi150 ceph-mon[34518]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:44.674 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:44 smithi115 ceph-mon[29111]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:44.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.950+0000 7f64c2200700 1 -- 172.21.15.115:0/2503362193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 msgr2=0x7f64bc104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:44.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.950+0000 7f64c2200700 1 --2- 172.21.15.115:0/2503362193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 0x7f64bc104d20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009910 tx=0x7f64a8009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:44.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.950+0000 7f64c2200700 1 -- 172.21.15.115:0/2503362193 shutdown_connections 2024-03-22T10:37:44.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.950+0000 7f64c2200700 1 --2- 172.21.15.115:0/2503362193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 0x7f64bc104d20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:44.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.950+0000 7f64c2200700 1 --2- 172.21.15.115:0/2503362193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64bc1003b0 0x7f64bc100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:44.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.950+0000 7f64c2200700 1 -- 172.21.15.115:0/2503362193 >> 172.21.15.115:0/2503362193 conn(0x7f64bc0fbc30 msgr2=0x7f64bc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:44.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.950+0000 7f64c2200700 1 -- 172.21.15.115:0/2503362193 shutdown_connections 2024-03-22T10:37:44.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.950+0000 7f64c2200700 1 -- 172.21.15.115:0/2503362193 wait complete. 2024-03-22T10:37:44.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.951+0000 7f64c2200700 1 Processor -- start 2024-03-22T10:37:44.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.952+0000 7f64c2200700 1 -- start start 2024-03-22T10:37:44.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.952+0000 7f64c2200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64bc1003b0 0x7f64bc073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:44.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.952+0000 7f64c2200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 0x7f64bc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:44.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.952+0000 7f64c2200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc073ad0 con 0x7f64bc100d50 2024-03-22T10:37:44.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.952+0000 7f64c2200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc073c10 con 0x7f64bc1003b0 2024-03-22T10:37:44.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.952+0000 7f64baa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 0x7f64bc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:44.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.952+0000 7f64baa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 0x7f64bc073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34488/0 (socket says 172.21.15.115:34488) 2024-03-22T10:37:44.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.952+0000 7f64baa00700 1 -- 172.21.15.115:0/3898050100 learned_addr learned my addr 172.21.15.115:0/3898050100 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:44.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.953+0000 7f64bb400700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64bc1003b0 0x7f64bc073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:44.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.953+0000 7f64baa00700 1 -- 172.21.15.115:0/3898050100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64bc1003b0 msgr2=0x7f64bc073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:44.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.953+0000 7f64baa00700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64bc1003b0 0x7f64bc073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:44.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.953+0000 7f64baa00700 1 -- 172.21.15.115:0/3898050100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a80095f0 con 0x7f64bc100d50 2024-03-22T10:37:44.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.953+0000 7f64baa00700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 0x7f64bc073590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64a800bda0 tx=0x7f64a800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:44.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.954+0000 7f64b3e00700 1 -- 172.21.15.115:0/3898050100 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64a801d070 con 0x7f64bc100d50 2024-03-22T10:37:44.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.954+0000 7f64b3e00700 1 -- 172.21.15.115:0/3898050100 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f64a8022470 con 0x7f64bc100d50 2024-03-22T10:37:44.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.954+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64bc073e90 con 0x7f64bc100d50 2024-03-22T10:37:44.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.954+0000 7f64b3e00700 1 -- 172.21.15.115:0/3898050100 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64a800f690 con 0x7f64bc100d50 2024-03-22T10:37:44.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.954+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64bc1a2270 con 0x7f64bc100d50 2024-03-22T10:37:44.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.955+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64bc108720 con 0x7f64bc100d50 2024-03-22T10:37:44.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.956+0000 7f64b3e00700 1 -- 172.21.15.115:0/3898050100 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f64a8022ae0 con 0x7f64bc100d50 2024-03-22T10:37:44.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.957+0000 7f64b3e00700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f64a006c680 0x7f64a006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:44.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.957+0000 7f64b3e00700 1 -- 172.21.15.115:0/3898050100 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f64a808ca00 con 0x7f64bc100d50 2024-03-22T10:37:44.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.957+0000 7f64bb400700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f64a006c680 0x7f64a006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:44.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.958+0000 7f64bb400700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f64a006c680 0x7f64a006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00ba60 tx=0x7f64ac005d50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:44.962+0000 7f64b3e00700 1 -- 172.21.15.115:0/3898050100 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f64a8057550 con 0x7f64bc100d50 2024-03-22T10:37:45.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.211+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f64bc108100 con 0x7f64bc100d50 2024-03-22T10:37:45.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.212+0000 7f64b3e00700 1 -- 172.21.15.115:0/3898050100 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64a8027020 con 0x7f64bc100d50 2024-03-22T10:37:45.206 INFO:teuthology.orchestra.run.smithi115.stdout:146028888082 2024-03-22T10:37:45.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f64a006c680 msgr2=0x7f64a006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:45.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f64a006c680 0x7f64a006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00ba60 tx=0x7f64ac005d50 comp rx=0 tx=0).stop 2024-03-22T10:37:45.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 msgr2=0x7f64bc073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:45.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 0x7f64bc073590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64a800bda0 tx=0x7f64a800be80 comp rx=0 tx=0).stop 2024-03-22T10:37:45.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 shutdown_connections 2024-03-22T10:37:45.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f64a006c680 0x7f64a006eb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:45.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64bc1003b0 0x7f64bc073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:45.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 --2- 172.21.15.115:0/3898050100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64bc100d50 0x7f64bc073590 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:45.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 >> 172.21.15.115:0/3898050100 conn(0x7f64bc0fbc30 msgr2=0x7f64bc0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:45.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 shutdown_connections 2024-03-22T10:37:45.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:45.215+0000 7f64c2200700 1 -- 172.21.15.115:0/3898050100 wait complete. 2024-03-22T10:37:45.466 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:45 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3898050100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T10:37:45.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:45 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3898050100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T10:37:46.647 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:46 smithi115 ceph-mon[29111]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:46.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:46 smithi150 ceph-mon[34518]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:47.413 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378654 got 81604378655 for osd.2 2024-03-22T10:37:47.413 DEBUG:teuthology.parallel:result is None 2024-03-22T10:37:48.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.037+0000 7f165a600700 1 -- 172.21.15.115:0/4062371664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 msgr2=0x7f1654103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.037+0000 7f165a600700 1 --2- 172.21.15.115:0/4062371664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 0x7f1654103ce0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1640009910 tx=0x7f1640009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.037+0000 7f165a600700 1 -- 172.21.15.115:0/4062371664 shutdown_connections 2024-03-22T10:37:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.037+0000 7f165a600700 1 --2- 172.21.15.115:0/4062371664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 0x7f1654103ce0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.037+0000 7f165a600700 1 --2- 172.21.15.115:0/4062371664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16540ff390 0x7f16540ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.037+0000 7f165a600700 1 -- 172.21.15.115:0/4062371664 >> 172.21.15.115:0/4062371664 conn(0x7f1654074f50 msgr2=0x7f1654075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.037+0000 7f165a600700 1 -- 172.21.15.115:0/4062371664 shutdown_connections 2024-03-22T10:37:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.037+0000 7f165a600700 1 -- 172.21.15.115:0/4062371664 wait complete. 2024-03-22T10:37:48.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.038+0000 7f165a600700 1 Processor -- start 2024-03-22T10:37:48.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.038+0000 7f165a600700 1 -- start start 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f165a600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16540ff390 0x7f1654197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f165a600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 0x7f1654198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f165a600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1654198bf0 con 0x7f16540ffd30 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f165a600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165419c980 con 0x7f16540ff390 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f1652a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 0x7f1654198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f1652a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 0x7f1654198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34512/0 (socket says 172.21.15.115:34512) 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f1652a00700 1 -- 172.21.15.115:0/1125648818 learned_addr learned my addr 172.21.15.115:0/1125648818 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f1653400700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16540ff390 0x7f1654197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f1652a00700 1 -- 172.21.15.115:0/1125648818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16540ff390 msgr2=0x7f1654197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f1652a00700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16540ff390 0x7f1654197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f1652a00700 1 -- 172.21.15.115:0/1125648818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16400095f0 con 0x7f16540ffd30 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.039+0000 7f1653400700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16540ff390 0x7f1654197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:37:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.040+0000 7f1652a00700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 0x7f1654198510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1640005f50 tx=0x7f164000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:48.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.040+0000 7f164be00700 1 -- 172.21.15.115:0/1125648818 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f164001d070 con 0x7f16540ffd30 2024-03-22T10:37:48.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.040+0000 7f164be00700 1 -- 172.21.15.115:0/1125648818 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f164000f460 con 0x7f16540ffd30 2024-03-22T10:37:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.040+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165419cc00 con 0x7f16540ffd30 2024-03-22T10:37:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.040+0000 7f164be00700 1 -- 172.21.15.115:0/1125648818 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1640004fd0 con 0x7f16540ffd30 2024-03-22T10:37:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.040+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165419d0f0 con 0x7f16540ffd30 2024-03-22T10:37:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.041+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165404ea50 con 0x7f16540ffd30 2024-03-22T10:37:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.042+0000 7f164be00700 1 -- 172.21.15.115:0/1125648818 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f164000f5d0 con 0x7f16540ffd30 2024-03-22T10:37:48.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.043+0000 7f164be00700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f163806c680 0x7f163806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:48.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.043+0000 7f164be00700 1 -- 172.21.15.115:0/1125648818 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f164008c850 con 0x7f16540ffd30 2024-03-22T10:37:48.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.043+0000 7f1653400700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f163806c680 0x7f163806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.044+0000 7f1653400700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f163806c680 0x7f163806eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1644005fd0 tx=0x7f1644005dc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:48.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.048+0000 7f164be00700 1 -- 172.21.15.115:0/1125648818 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1640057250 con 0x7f16540ffd30 2024-03-22T10:37:48.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.278+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f165419d5b0 con 0x7f16540ffd30 2024-03-22T10:37:48.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.278+0000 7f164be00700 1 -- 172.21.15.115:0/1125648818 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1640026090 con 0x7f16540ffd30 2024-03-22T10:37:48.272 INFO:teuthology.orchestra.run.smithi115.stdout:38654705712 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.281+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f163806c680 msgr2=0x7f163806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.281+0000 7f165a600700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f163806c680 0x7f163806eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1644005fd0 tx=0x7f1644005dc0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.281+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 msgr2=0x7f1654198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.281+0000 7f165a600700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 0x7f1654198510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1640005f50 tx=0x7f164000be20 comp rx=0 tx=0).stop 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.281+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 shutdown_connections 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.281+0000 7f165a600700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f163806c680 0x7f163806eb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.281+0000 7f165a600700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16540ff390 0x7f1654197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.281+0000 7f165a600700 1 --2- 172.21.15.115:0/1125648818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16540ffd30 0x7f1654198510 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.282+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 >> 172.21.15.115:0/1125648818 conn(0x7f1654074f50 msgr2=0x7f1654068cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.282+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 shutdown_connections 2024-03-22T10:37:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.282+0000 7f165a600700 1 -- 172.21.15.115:0/1125648818 wait complete. 2024-03-22T10:37:48.545 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:48 smithi115 ceph-mon[29111]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:48.545 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:48 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1125648818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T10:37:48.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:48 smithi150 ceph-mon[34518]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:48.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:48 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1125648818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T10:37:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.704+0000 7f9bbb400700 1 -- 172.21.15.115:0/3239326201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb4100e40 msgr2=0x7f9bb4104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.704+0000 7f9bbb400700 1 --2- 172.21.15.115:0/3239326201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb4100e40 0x7f9bb4104e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009910 tx=0x7f9ba0009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.704+0000 7f9bbb400700 1 -- 172.21.15.115:0/3239326201 shutdown_connections 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.704+0000 7f9bbb400700 1 --2- 172.21.15.115:0/3239326201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb4100e40 0x7f9bb4104e10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.704+0000 7f9bbb400700 1 --2- 172.21.15.115:0/3239326201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bb41004a0 0x7f9bb4100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.704+0000 7f9bbb400700 1 -- 172.21.15.115:0/3239326201 >> 172.21.15.115:0/3239326201 conn(0x7f9bb40fbd60 msgr2=0x7f9bb40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.704+0000 7f9bbb400700 1 -- 172.21.15.115:0/3239326201 shutdown_connections 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.704+0000 7f9bbb400700 1 -- 172.21.15.115:0/3239326201 wait complete. 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.705+0000 7f9bbb400700 1 Processor -- start 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.705+0000 7f9bbb400700 1 -- start start 2024-03-22T10:37:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bbb400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb41004a0 0x7f9bb4197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bbb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bb4100e40 0x7f9bb4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bbb400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb4198b30 con 0x7f9bb41004a0 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bbb400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb419c8c0 con 0x7f9bb4100e40 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bb8a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb41004a0 0x7f9bb4197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bb8a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb41004a0 0x7f9bb4197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34530/0 (socket says 172.21.15.115:34530) 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bb8a00700 1 -- 172.21.15.115:0/2770699130 learned_addr learned my addr 172.21.15.115:0/2770699130 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bb8a00700 1 -- 172.21.15.115:0/2770699130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bb4100e40 msgr2=0x7f9bb4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bb8a00700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bb4100e40 0x7f9bb4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.706+0000 7f9bb8a00700 1 -- 172.21.15.115:0/2770699130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba00095f0 con 0x7f9bb41004a0 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.707+0000 7f9bb8a00700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb41004a0 0x7f9bb4197f10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800c8f0 tx=0x7f9ba800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.707+0000 7f9bb1600700 1 -- 172.21.15.115:0/2770699130 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ba8007a70 con 0x7f9bb41004a0 2024-03-22T10:37:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.707+0000 7f9bb1600700 1 -- 172.21.15.115:0/2770699130 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9ba8004ce0 con 0x7f9bb41004a0 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.707+0000 7f9bb1600700 1 -- 172.21.15.115:0/2770699130 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ba8005610 con 0x7f9bb41004a0 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.707+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb419cba0 con 0x7f9bb41004a0 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.707+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb419d0f0 con 0x7f9bb41004a0 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.708+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb4104ca0 con 0x7f9bb41004a0 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.710+0000 7f9bb1600700 1 -- 172.21.15.115:0/2770699130 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f9ba8004750 con 0x7f9bb41004a0 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.711+0000 7f9bb1600700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b9c06c680 0x7f9b9c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.711+0000 7f9bb1600700 1 -- 172.21.15.115:0/2770699130 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9ba8013070 con 0x7f9bb41004a0 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.711+0000 7f9bb3e00700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b9c06c680 0x7f9b9c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.712+0000 7f9bb3e00700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b9c06c680 0x7f9b9c06eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000b5c0 tx=0x7f9ba0005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.716+0000 7f9bb1600700 1 -- 172.21.15.115:0/2770699130 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9ba8055e80 con 0x7f9bb41004a0 2024-03-22T10:37:48.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.814+0000 7f3679800700 1 -- 172.21.15.115:0/460069757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100430 msgr2=0x7f3674100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.814+0000 7f3679800700 1 --2- 172.21.15.115:0/460069757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100430 0x7f3674100800 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3664009910 tx=0x7f3664009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:48.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.814+0000 7f3679800700 1 -- 172.21.15.115:0/460069757 shutdown_connections 2024-03-22T10:37:48.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.814+0000 7f3679800700 1 --2- 172.21.15.115:0/460069757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3674100dd0 0x7f3674104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.814+0000 7f3679800700 1 --2- 172.21.15.115:0/460069757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100430 0x7f3674100800 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.814+0000 7f3679800700 1 -- 172.21.15.115:0/460069757 >> 172.21.15.115:0/460069757 conn(0x7f36740fbcf0 msgr2=0x7f36740fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.814+0000 7f3679800700 1 -- 172.21.15.115:0/460069757 shutdown_connections 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.814+0000 7f3679800700 1 -- 172.21.15.115:0/460069757 wait complete. 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.815+0000 7f3679800700 1 Processor -- start 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.815+0000 7f3679800700 1 -- start start 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.815+0000 7f3679800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3674100430 0x7f3674197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.815+0000 7f3679800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100dd0 0x7f36741984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.815+0000 7f3679800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674198b90 con 0x7f3674100dd0 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.815+0000 7f3679800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367419c8b0 con 0x7f3674100430 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.816+0000 7f3672000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100dd0 0x7f36741984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.816+0000 7f3672000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100dd0 0x7f36741984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34552/0 (socket says 172.21.15.115:34552) 2024-03-22T10:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.816+0000 7f3672000700 1 -- 172.21.15.115:0/2988655427 learned_addr learned my addr 172.21.15.115:0/2988655427 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.816+0000 7f3672a00700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3674100430 0x7f3674197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.816+0000 7f3672000700 1 -- 172.21.15.115:0/2988655427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3674100430 msgr2=0x7f3674197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.816+0000 7f3672000700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3674100430 0x7f3674197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.816+0000 7f3672000700 1 -- 172.21.15.115:0/2988655427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36640095f0 con 0x7f3674100dd0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.817+0000 7f3672000700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100dd0 0x7f36741984b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f366000c8f0 tx=0x7f366000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.817+0000 7f366b400700 1 -- 172.21.15.115:0/2988655427 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3660007a50 con 0x7f3674100dd0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.817+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367419cb90 con 0x7f3674100dd0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.817+0000 7f366b400700 1 -- 172.21.15.115:0/2988655427 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3660007bb0 con 0x7f3674100dd0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.817+0000 7f366b400700 1 -- 172.21.15.115:0/2988655427 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3660018630 con 0x7f3674100dd0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.817+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367419d0e0 con 0x7f3674100dd0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.820+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3674104520 con 0x7f3674100dd0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.821+0000 7f366b400700 1 -- 172.21.15.115:0/2988655427 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f3660007d20 con 0x7f3674100dd0 2024-03-22T10:37:48.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.822+0000 7f366b400700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f365806c680 0x7f365806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:48.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.822+0000 7f366b400700 1 -- 172.21.15.115:0/2988655427 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f366008b560 con 0x7f3674100dd0 2024-03-22T10:37:48.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.822+0000 7f3672a00700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f365806c680 0x7f365806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:48.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.823+0000 7f3672a00700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f365806c680 0x7f365806eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f366400b5c0 tx=0x7f366401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:48.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.826+0000 7f366b400700 1 -- 172.21.15.115:0/2988655427 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3660055f20 con 0x7f3674100dd0 2024-03-22T10:37:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.935+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9bb4199270 con 0x7f9bb41004a0 2024-03-22T10:37:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.936+0000 7f9bb1600700 1 -- 172.21.15.115:0/2770699130 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9bb4199270 con 0x7f9bb41004a0 2024-03-22T10:37:48.930 INFO:teuthology.orchestra.run.smithi115.stdout:111669149722 2024-03-22T10:37:48.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b9c06c680 msgr2=0x7f9b9c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b9c06c680 0x7f9b9c06eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000b5c0 tx=0x7f9ba0005fd0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb41004a0 msgr2=0x7f9bb4197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:48.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb41004a0 0x7f9bb4197f10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800c8f0 tx=0x7f9ba800ccb0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 shutdown_connections 2024-03-22T10:37:48.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b9c06c680 0x7f9b9c06eb30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bb41004a0 0x7f9bb4197f10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 --2- 172.21.15.115:0/2770699130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9bb4100e40 0x7f9bb4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:48.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 >> 172.21.15.115:0/2770699130 conn(0x7f9bb40fbd60 msgr2=0x7f9bb4101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:48.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 shutdown_connections 2024-03-22T10:37:48.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:48.939+0000 7f9bbb400700 1 -- 172.21.15.115:0/2770699130 wait complete. 2024-03-22T10:37:49.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.059+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f367404ea50 con 0x7f3674100dd0 2024-03-22T10:37:49.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.059+0000 7f366b400700 1 -- 172.21.15.115:0/2988655427 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3660059540 con 0x7f3674100dd0 2024-03-22T10:37:49.053 INFO:teuthology.orchestra.run.smithi115.stdout:60129542183 2024-03-22T10:37:49.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.062+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f365806c680 msgr2=0x7f365806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:49.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.062+0000 7f3679800700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f365806c680 0x7f365806eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f366400b5c0 tx=0x7f366401a040 comp rx=0 tx=0).stop 2024-03-22T10:37:49.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.062+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100dd0 msgr2=0x7f36741984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:49.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.062+0000 7f3679800700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100dd0 0x7f36741984b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f366000c8f0 tx=0x7f366000ccb0 comp rx=0 tx=0).stop 2024-03-22T10:37:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.063+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 shutdown_connections 2024-03-22T10:37:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.063+0000 7f3679800700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f365806c680 0x7f365806eb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.063+0000 7f3679800700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3674100430 0x7f3674197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.063+0000 7f3679800700 1 --2- 172.21.15.115:0/2988655427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3674100dd0 0x7f36741984b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.063+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 >> 172.21.15.115:0/2988655427 conn(0x7f36740fbcf0 msgr2=0x7f36740fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.063+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 shutdown_connections 2024-03-22T10:37:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:49.063+0000 7f3679800700 1 -- 172.21.15.115:0/2988655427 wait complete. 2024-03-22T10:37:49.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:49 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2770699130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T10:37:49.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:49 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2988655427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T10:37:49.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:49 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2770699130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T10:37:49.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:49 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2988655427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T10:37:50.252 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888081 got 146028888082 for osd.4 2024-03-22T10:37:50.252 DEBUG:teuthology.parallel:result is None 2024-03-22T10:37:50.502 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:50 smithi115 ceph-mon[29111]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:50.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:50 smithi150 ceph-mon[34518]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:50.943 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705709 got 38654705712 for osd.0 2024-03-22T10:37:50.943 DEBUG:teuthology.parallel:result is None 2024-03-22T10:37:51.702 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542180 got 60129542183 for osd.1 2024-03-22T10:37:51.702 DEBUG:teuthology.parallel:result is None 2024-03-22T10:37:51.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.892+0000 7fa6f0800700 1 -- 172.21.15.115:0/1256779899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e81003b0 msgr2=0x7fa6e8100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:51.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.892+0000 7fa6f0800700 1 --2- 172.21.15.115:0/1256779899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e81003b0 0x7fa6e8100780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009910 tx=0x7fa6dc009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:51.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.892+0000 7fa6f0800700 1 -- 172.21.15.115:0/1256779899 shutdown_connections 2024-03-22T10:37:51.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.892+0000 7fa6f0800700 1 --2- 172.21.15.115:0/1256779899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e8100d50 0x7fa6e8104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:51.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.892+0000 7fa6f0800700 1 --2- 172.21.15.115:0/1256779899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e81003b0 0x7fa6e8100780 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:51.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.892+0000 7fa6f0800700 1 -- 172.21.15.115:0/1256779899 >> 172.21.15.115:0/1256779899 conn(0x7fa6e80fbc30 msgr2=0x7fa6e80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:51.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.892+0000 7fa6f0800700 1 -- 172.21.15.115:0/1256779899 shutdown_connections 2024-03-22T10:37:51.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.892+0000 7fa6f0800700 1 -- 172.21.15.115:0/1256779899 wait complete. 2024-03-22T10:37:51.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.894+0000 7fa6f0800700 1 Processor -- start 2024-03-22T10:37:51.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.894+0000 7fa6f0800700 1 -- start start 2024-03-22T10:37:51.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.894+0000 7fa6f0800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e81003b0 0x7fa6e8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:51.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.894+0000 7fa6f0800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e8100d50 0x7fa6e8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:51.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.894+0000 7fa6f0800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e8073ad0 con 0x7fa6e8100d50 2024-03-22T10:37:51.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.894+0000 7fa6f0800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e8073c10 con 0x7fa6e81003b0 2024-03-22T10:37:51.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.895+0000 7fa6ed400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e8100d50 0x7fa6e8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:51.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.895+0000 7fa6ed400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e8100d50 0x7fa6e8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49018/0 (socket says 172.21.15.115:49018) 2024-03-22T10:37:51.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.895+0000 7fa6ed400700 1 -- 172.21.15.115:0/2149471529 learned_addr learned my addr 172.21.15.115:0/2149471529 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.895+0000 7fa6ede00700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e81003b0 0x7fa6e8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.895+0000 7fa6ed400700 1 -- 172.21.15.115:0/2149471529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e81003b0 msgr2=0x7fa6e8073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.895+0000 7fa6ed400700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e81003b0 0x7fa6e8073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.895+0000 7fa6ed400700 1 -- 172.21.15.115:0/2149471529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc0095f0 con 0x7fa6e8100d50 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.895+0000 7fa6ed400700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e8100d50 0x7fa6e8073590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c960 tx=0x7fa6d800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.896+0000 7fa6e2a00700 1 -- 172.21.15.115:0/2149471529 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d8007a70 con 0x7fa6e8100d50 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.896+0000 7fa6e2a00700 1 -- 172.21.15.115:0/2149471529 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa6d8007bd0 con 0x7fa6e8100d50 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.896+0000 7fa6e2a00700 1 -- 172.21.15.115:0/2149471529 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d8018630 con 0x7fa6e8100d50 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.896+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e81a1ee0 con 0x7fa6e8100d50 2024-03-22T10:37:51.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.896+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e81a2240 con 0x7fa6e8100d50 2024-03-22T10:37:51.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.897+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e81086c0 con 0x7fa6e8100d50 2024-03-22T10:37:51.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.898+0000 7fa6e2a00700 1 -- 172.21.15.115:0/2149471529 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa6d800fa90 con 0x7fa6e8100d50 2024-03-22T10:37:51.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.899+0000 7fa6e2a00700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa6d006c680 0x7fa6d006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:51.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.899+0000 7fa6e2a00700 1 -- 172.21.15.115:0/2149471529 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa6d808b540 con 0x7fa6e8100d50 2024-03-22T10:37:51.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.899+0000 7fa6ede00700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa6d006c680 0x7fa6d006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:51.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.900+0000 7fa6ede00700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa6d006c680 0x7fa6d006eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc006010 tx=0x7fa6dc01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:51.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:51.903+0000 7fa6e2a00700 1 -- 172.21.15.115:0/2149471529 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa6d80599d0 con 0x7fa6e8100d50 2024-03-22T10:37:52.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.134+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa6e8073dd0 con 0x7fa6e8100d50 2024-03-22T10:37:52.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.135+0000 7fa6e2a00700 1 -- 172.21.15.115:0/2149471529 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6d8059560 con 0x7fa6e8100d50 2024-03-22T10:37:52.128 INFO:teuthology.orchestra.run.smithi115.stdout:167503724557 2024-03-22T10:37:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa6d006c680 msgr2=0x7fa6d006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa6d006c680 0x7fa6d006eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc006010 tx=0x7fa6dc01a040 comp rx=0 tx=0).stop 2024-03-22T10:37:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e8100d50 msgr2=0x7fa6e8073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e8100d50 0x7fa6e8073590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c960 tx=0x7fa6d800cd20 comp rx=0 tx=0).stop 2024-03-22T10:37:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 shutdown_connections 2024-03-22T10:37:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa6d006c680 0x7fa6d006eb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e81003b0 0x7fa6e8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 --2- 172.21.15.115:0/2149471529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e8100d50 0x7fa6e8073590 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 >> 172.21.15.115:0/2149471529 conn(0x7fa6e80fbc30 msgr2=0x7fa6e80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 shutdown_connections 2024-03-22T10:37:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:52.138+0000 7fa6f0800700 1 -- 172.21.15.115:0/2149471529 wait complete. 2024-03-22T10:37:52.417 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:52 smithi115 ceph-mon[29111]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:52.417 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:52 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2149471529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T10:37:52.420 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149720 got 111669149722 for osd.3 2024-03-22T10:37:52.420 DEBUG:teuthology.parallel:result is None 2024-03-22T10:37:52.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:52 smithi150 ceph-mon[34518]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:52.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:52 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2149471529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T10:37:53.088 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724557 for osd.5 2024-03-22T10:37:53.088 DEBUG:teuthology.parallel:result is None 2024-03-22T10:37:53.088 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-22T10:37:53.088 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T10:37:53.488 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:54.063 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:53 smithi115 ceph-mon[29111]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:54.246 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:53 smithi150 ceph-mon[34518]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.419+0000 7f7ea6000700 1 -- 172.21.15.115:0/2272969663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102fd0 msgr2=0x7f7ea0106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.419+0000 7f7ea6000700 1 --2- 172.21.15.115:0/2272969663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102fd0 0x7f7ea0106fa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009910 tx=0x7f7e88009c20 comp rx=0 tx=0).stop 2024-03-22T10:37:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.419+0000 7f7ea6000700 1 -- 172.21.15.115:0/2272969663 shutdown_connections 2024-03-22T10:37:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.419+0000 7f7ea6000700 1 --2- 172.21.15.115:0/2272969663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102fd0 0x7f7ea0106fa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.419+0000 7f7ea6000700 1 --2- 172.21.15.115:0/2272969663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea0102630 0x7f7ea0102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.419+0000 7f7ea6000700 1 -- 172.21.15.115:0/2272969663 >> 172.21.15.115:0/2272969663 conn(0x7f7ea00fdef0 msgr2=0x7f7ea0100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.420+0000 7f7ea6000700 1 -- 172.21.15.115:0/2272969663 shutdown_connections 2024-03-22T10:37:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.420+0000 7f7ea6000700 1 -- 172.21.15.115:0/2272969663 wait complete. 2024-03-22T10:37:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.421+0000 7f7ea6000700 1 Processor -- start 2024-03-22T10:37:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.421+0000 7f7ea6000700 1 -- start start 2024-03-22T10:37:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.421+0000 7f7ea6000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102630 0x7f7ea0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.421+0000 7f7ea6000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea0102fd0 0x7f7ea01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.421+0000 7f7ea6000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea0198ba0 con 0x7f7ea0102630 2024-03-22T10:37:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.421+0000 7f7ea6000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea019c930 con 0x7f7ea0102fd0 2024-03-22T10:37:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.421+0000 7f7e9f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102630 0x7f7ea0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.422+0000 7f7e9f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102630 0x7f7ea0197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49042/0 (socket says 172.21.15.115:49042) 2024-03-22T10:37:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.422+0000 7f7e9f400700 1 -- 172.21.15.115:0/84697724 learned_addr learned my addr 172.21.15.115:0/84697724 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.422+0000 7f7e97e00700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea0102fd0 0x7f7ea01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.422+0000 7f7e9f400700 1 -- 172.21.15.115:0/84697724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea0102fd0 msgr2=0x7f7ea01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.422+0000 7f7e9f400700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea0102fd0 0x7f7ea01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.422+0000 7f7e9f400700 1 -- 172.21.15.115:0/84697724 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880095f0 con 0x7f7ea0102630 2024-03-22T10:37:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.422+0000 7f7e97e00700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea0102fd0 0x7f7ea01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:37:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.422+0000 7f7e9f400700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102630 0x7f7ea0197f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000c8f0 tx=0x7f7e9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:55.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.422+0000 7f7e9cc00700 1 -- 172.21.15.115:0/84697724 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e90007a70 con 0x7f7ea0102630 2024-03-22T10:37:55.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.423+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea019cc10 con 0x7f7ea0102630 2024-03-22T10:37:55.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.423+0000 7f7e9cc00700 1 -- 172.21.15.115:0/84697724 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7e90004ce0 con 0x7f7ea0102630 2024-03-22T10:37:55.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.423+0000 7f7e9cc00700 1 -- 172.21.15.115:0/84697724 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e90005610 con 0x7f7ea0102630 2024-03-22T10:37:55.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.423+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea019d160 con 0x7f7ea0102630 2024-03-22T10:37:55.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.424+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea010a8b0 con 0x7f7ea0102630 2024-03-22T10:37:55.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.425+0000 7f7e9cc00700 1 -- 172.21.15.115:0/84697724 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7e90004750 con 0x7f7ea0102630 2024-03-22T10:37:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.426+0000 7f7e9cc00700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7e8006c680 0x7f7e8006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:55.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.426+0000 7f7e9cc00700 1 -- 172.21.15.115:0/84697724 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7e90013070 con 0x7f7ea0102630 2024-03-22T10:37:55.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.426+0000 7f7e97e00700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7e8006c680 0x7f7e8006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:55.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.426+0000 7f7e97e00700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7e8006c680 0x7f7e8006eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006010 tx=0x7f7e88005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:55.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.430+0000 7f7e9cc00700 1 -- 172.21.15.115:0/84697724 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7e90055ed0 con 0x7f7ea0102630 2024-03-22T10:37:55.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:37:55.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:37:55.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.659+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ea019d4d0 con 0x7f7e8006c680 2024-03-22T10:37:55.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.661+0000 7f7e9cc00700 1 -- 172.21.15.115:0/84697724 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27073 (secure 0 0 0) 0x7f7ea019d4d0 con 0x7f7e8006c680 2024-03-22T10:37:55.654 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.664+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7e8006c680 msgr2=0x7f7e8006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.664+0000 7f7ea6000700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7e8006c680 0x7f7e8006eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006010 tx=0x7f7e88005960 comp rx=0 tx=0).stop 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.664+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102630 msgr2=0x7f7ea0197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.664+0000 7f7ea6000700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102630 0x7f7ea0197f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000c8f0 tx=0x7f7e9000ccb0 comp rx=0 tx=0).stop 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.665+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 shutdown_connections 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.665+0000 7f7ea6000700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7e8006c680 0x7f7e8006eb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.665+0000 7f7ea6000700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ea0102630 0x7f7ea0197f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.665+0000 7f7ea6000700 1 --2- 172.21.15.115:0/84697724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea0102fd0 0x7f7ea01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.665+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 >> 172.21.15.115:0/84697724 conn(0x7f7ea00fdef0 msgr2=0x7f7ea0106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.665+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 shutdown_connections 2024-03-22T10:37:55.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:55.665+0000 7f7ea6000700 1 -- 172.21.15.115:0/84697724 wait complete. 2024-03-22T10:37:55.658 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-03-22T10:37:56.456 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-03-22T10:37:55.268023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165588,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305324,"statfs":{"total":575970213888,"available":575800651776,"internally_reserved":0,"allocated":3969024,"data_stored":2609166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003165"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-22T10:36:58.630484+0000","last_change":"2024-03-22T10:36:24.329186+0000","last_active":"2024-03-22T10:36:58.630484+0000","last_peered":"2024-03-22T10:36:58.630484+0000","last_clean":"2024-03-22T10:36:58.630484+0000","last_became_active":"2024-03-22T10:36:24.328400+0000","last_became_peered":"2024-03-22T10:36:24.328400+0000","last_unstale":"2024-03-22T10:36:58.630484+0000","last_undegraded":"2024-03-22T10:36:58.630484+0000","last_fullsized":"2024-03-22T10:36:58.630484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T10:35:18.097390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T10:35:18.097390+0000","last_clean_scrub_stamp":"2024-03-22T10:35:18.097390+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:38:08.328340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 10:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri Mar 22 10:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.472}]},{"osd":2,"last update":"Fri Mar 22 10:37:25 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.323},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.2290000000000001}]},{"osd":3,"last update":"Fri Mar 22 10:37:25 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.417}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149723,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 10:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.089}]},{"osd":1,"last update":"Fri Mar 22 10:36:50 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.026},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":1.127}]},{"osd":2,"last update":"Fri Mar 22 10:36:50 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.79000000000000004}]},{"osd":4,"last update":"Fri Mar 22 10:37:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.1890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]},{"osd":2,"up_from":19,"seq":81604378658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 10:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78700000000000003,"15min":0.78000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.878,"15min":0.86099999999999999},"min":{"1min":0.59099999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.93600000000000005}]},{"osd":1,"last update":"Fri Mar 22 10:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.84899999999999998,"15min":0.83899999999999997},"min":{"1min":0.51800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.93300000000000005,"15min":0.92300000000000004},"min":{"1min":0.53600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.0740000000000001}]},{"osd":3,"last update":"Fri Mar 22 10:36:52 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.1299999999999999}]},{"osd":4,"last update":"Fri Mar 22 10:37:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]}]},{"osd":0,"up_from":9,"seq":38654705713,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 10:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.71299999999999997,"15min":0.66100000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.76100000000000001,"15min":0.69699999999999995},"min":{"1min":0.40100000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Mar 22 10:37:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.434,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.47899999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.93200000000000005}]},{"osd":3,"last update":"Fri Mar 22 10:36:50 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.3200000000000001}]},{"osd":4,"last update":"Fri Mar 22 10:37:27 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.246}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 10:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77900000000000003,"15min":0.78700000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.268}]},{"osd":2,"last update":"Fri Mar 22 10:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.91300000000000003,"15min":0.90500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.0720000000000001}]},{"osd":3,"last update":"Fri Mar 22 10:37:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.109,"15min":1.103},"min":{"1min":0.80700000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.538},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.115,"15min":1.105},"min":{"1min":0.90200000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.3260000000000001}]},{"osd":4,"last update":"Fri Mar 22 10:37:26 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.1839999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T10:37:56.457 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T10:37:56.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:56 smithi150 ceph-mon[34518]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:56.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:56 smithi150 ceph-mon[34518]: from='client.14414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T10:37:56.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:56 smithi115 ceph-mon[29111]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:56.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:56 smithi115 ceph-mon[29111]: from='client.14414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T10:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:37:58.576 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:37:58 smithi115 ceph-mon[29111]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:58.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:37:58 smithi150 ceph-mon[34518]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:37:58.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.855+0000 7f7c6a400700 1 -- 172.21.15.115:0/3705388153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640ff7b0 msgr2=0x7f7c6410c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.855+0000 7f7c6a400700 1 --2- 172.21.15.115:0/3705388153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640ff7b0 0x7f7c6410c510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009940 tx=0x7f7c50009c50 comp rx=0 tx=0).stop 2024-03-22T10:37:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.855+0000 7f7c6a400700 1 -- 172.21.15.115:0/3705388153 shutdown_connections 2024-03-22T10:37:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.855+0000 7f7c6a400700 1 --2- 172.21.15.115:0/3705388153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640ff7b0 0x7f7c6410c510 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.855+0000 7f7c6a400700 1 --2- 172.21.15.115:0/3705388153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c640feea0 0x7f7c640ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.855+0000 7f7c6a400700 1 -- 172.21.15.115:0/3705388153 >> 172.21.15.115:0/3705388153 conn(0x7f7c640fab50 msgr2=0x7f7c640fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.855+0000 7f7c6a400700 1 -- 172.21.15.115:0/3705388153 shutdown_connections 2024-03-22T10:37:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.855+0000 7f7c6a400700 1 -- 172.21.15.115:0/3705388153 wait complete. 2024-03-22T10:37:58.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.857+0000 7f7c6a400700 1 Processor -- start 2024-03-22T10:37:58.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.857+0000 7f7c6a400700 1 -- start start 2024-03-22T10:37:58.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.858+0000 7f7c6a400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640feea0 0x7f7c64197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:58.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.858+0000 7f7c6a400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c640ff7b0 0x7f7c64198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:58.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.858+0000 7f7c6a400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c64198b10 con 0x7f7c640feea0 2024-03-22T10:37:58.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.858+0000 7f7c6a400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6419c8a0 con 0x7f7c640ff7b0 2024-03-22T10:37:58.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.858+0000 7f7c63400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640feea0 0x7f7c64197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:58.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.859+0000 7f7c63400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640feea0 0x7f7c64197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49070/0 (socket says 172.21.15.115:49070) 2024-03-22T10:37:58.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.859+0000 7f7c63400700 1 -- 172.21.15.115:0/1918917371 learned_addr learned my addr 172.21.15.115:0/1918917371 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.859+0000 7f7c63400700 1 -- 172.21.15.115:0/1918917371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c640ff7b0 msgr2=0x7f7c64198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.859+0000 7f7c62a00700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c640ff7b0 0x7f7c64198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.859+0000 7f7c63400700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c640ff7b0 0x7f7c64198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.859+0000 7f7c63400700 1 -- 172.21.15.115:0/1918917371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500095f0 con 0x7f7c640feea0 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.859+0000 7f7c63400700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640feea0 0x7f7c64197ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400d8d0 tx=0x7f7c5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.859+0000 7f7c5be00700 1 -- 172.21.15.115:0/1918917371 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c54009940 con 0x7f7c640feea0 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.860+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6419cb80 con 0x7f7c640feea0 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.860+0000 7f7c5be00700 1 -- 172.21.15.115:0/1918917371 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7c54010460 con 0x7f7c640feea0 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.860+0000 7f7c5be00700 1 -- 172.21.15.115:0/1918917371 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c5400f5f0 con 0x7f7c640feea0 2024-03-22T10:37:58.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.860+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6419d0d0 con 0x7f7c640feea0 2024-03-22T10:37:58.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.861+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c64109c80 con 0x7f7c640feea0 2024-03-22T10:37:58.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.862+0000 7f7c5be00700 1 -- 172.21.15.115:0/1918917371 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7c540105d0 con 0x7f7c640feea0 2024-03-22T10:37:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.863+0000 7f7c5be00700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7c4806c680 0x7f7c4806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:37:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.863+0000 7f7c5be00700 1 -- 172.21.15.115:0/1918917371 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7c5408b540 con 0x7f7c640feea0 2024-03-22T10:37:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.863+0000 7f7c62a00700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7c4806c680 0x7f7c4806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:37:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.864+0000 7f7c62a00700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7c4806c680 0x7f7c4806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000b580 tx=0x7f7c50005fb0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:37:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:58.868+0000 7f7c5be00700 1 -- 172.21.15.115:0/1918917371 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7c54055f60 con 0x7f7c640feea0 2024-03-22T10:37:59.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.077+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c6419d450 con 0x7f7c4806c680 2024-03-22T10:37:59.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.079+0000 7f7c5be00700 1 -- 172.21.15.115:0/1918917371 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27304 (secure 0 0 0) 0x7f7c6419d450 con 0x7f7c4806c680 2024-03-22T10:37:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:37:59.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7c4806c680 msgr2=0x7f7c4806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7c4806c680 0x7f7c4806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000b580 tx=0x7f7c50005fb0 comp rx=0 tx=0).stop 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640feea0 msgr2=0x7f7c64197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640feea0 0x7f7c64197ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400d8d0 tx=0x7f7c5400dc90 comp rx=0 tx=0).stop 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 shutdown_connections 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7c4806c680 0x7f7c4806eb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c640feea0 0x7f7c64197ef0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 --2- 172.21.15.115:0/1918917371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c640ff7b0 0x7f7c64198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 >> 172.21.15.115:0/1918917371 conn(0x7f7c640fab50 msgr2=0x7f7c640fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.082+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 shutdown_connections 2024-03-22T10:37:59.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:37:59.083+0000 7f7c6a400700 1 -- 172.21.15.115:0/1918917371 wait complete. 2024-03-22T10:37:59.076 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-03-22T10:37:59.770 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-03-22T10:37:57.268654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165588,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305324,"statfs":{"total":575970213888,"available":575800651776,"internally_reserved":0,"allocated":3969024,"data_stored":2609166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003365"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-22T10:36:58.630484+0000","last_change":"2024-03-22T10:36:24.329186+0000","last_active":"2024-03-22T10:36:58.630484+0000","last_peered":"2024-03-22T10:36:58.630484+0000","last_clean":"2024-03-22T10:36:58.630484+0000","last_became_active":"2024-03-22T10:36:24.328400+0000","last_became_peered":"2024-03-22T10:36:24.328400+0000","last_unstale":"2024-03-22T10:36:58.630484+0000","last_undegraded":"2024-03-22T10:36:58.630484+0000","last_fullsized":"2024-03-22T10:36:58.630484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T10:35:18.097390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T10:35:18.097390+0000","last_clean_scrub_stamp":"2024-03-22T10:35:18.097390+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:38:08.328340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 10:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri Mar 22 10:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.472}]},{"osd":2,"last update":"Fri Mar 22 10:37:25 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.323},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.2290000000000001}]},{"osd":3,"last update":"Fri Mar 22 10:37:25 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.417}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149724,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 10:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.98999999999999999,"15min":0.996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.056,"5min":0.98399999999999999,"15min":0.97199999999999998},"min":{"1min":0.69799999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.2629999999999999}]},{"osd":1,"last update":"Fri Mar 22 10:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.026,"15min":1.03},"min":{"1min":0.73199999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.292,"5min":1.417,"15min":1.417},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0169999999999999,"15min":1.0229999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.2769999999999999,"5min":1.371,"15min":1.371},"last":0.92900000000000005}]},{"osd":2,"last update":"Fri Mar 22 10:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0369999999999999,"15min":1.0449999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3899999999999999,"5min":1.431,"15min":1.431},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.147,"15min":1.1519999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.0840000000000001}]},{"osd":4,"last update":"Fri Mar 22 10:37:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.2170000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 10:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78700000000000003,"15min":0.78000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.878,"15min":0.86099999999999999},"min":{"1min":0.59099999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.78600000000000003}]},{"osd":1,"last update":"Fri Mar 22 10:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.84899999999999998,"15min":0.83899999999999997},"min":{"1min":0.51800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.93300000000000005,"15min":0.92300000000000004},"min":{"1min":0.53600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.86199999999999999}]},{"osd":3,"last update":"Fri Mar 22 10:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1399999999999999,"15min":1.1379999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.198,"15min":1.2},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8720000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2350000000000001}]},{"osd":4,"last update":"Fri Mar 22 10:37:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.109},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 10:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.71299999999999997,"15min":0.66100000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.76100000000000001,"15min":0.69699999999999995},"min":{"1min":0.40100000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.069}]},{"osd":2,"last update":"Fri Mar 22 10:37:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.434,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.995},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.47899999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":1.395}]},{"osd":3,"last update":"Fri Mar 22 10:37:52 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.1879999999999999,"15min":1.1819999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.611},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2070000000000001,"15min":1.214},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.659,"5min":1.696,"15min":1.696},"last":1.3560000000000001}]},{"osd":4,"last update":"Fri Mar 22 10:37:27 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.506},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.1719999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 10:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77900000000000003,"15min":0.78700000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.268}]},{"osd":2,"last update":"Fri Mar 22 10:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.91300000000000003,"15min":0.90500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.0720000000000001}]},{"osd":3,"last update":"Fri Mar 22 10:37:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.109,"15min":1.103},"min":{"1min":0.80700000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.538},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.115,"15min":1.105},"min":{"1min":0.90200000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.3260000000000001}]},{"osd":4,"last update":"Fri Mar 22 10:37:26 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.1839999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T10:37:59.771 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-22T10:37:59.772 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-22T10:37:59.772 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-22T10:37:59.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-22T10:38:00.183 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:00.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:00 smithi115 ceph-mon[29111]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T10:38:00.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:00 smithi115 ceph-mon[29111]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:00.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:00 smithi150 ceph-mon[34518]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T10:38:00.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:00 smithi150 ceph-mon[34518]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:02.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.280+0000 7ff9e9e00700 1 -- 172.21.15.115:0/2599183727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e4073a00 msgr2=0x7ff9e410c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:02.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.280+0000 7ff9e9e00700 1 --2- 172.21.15.115:0/2599183727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e4073a00 0x7ff9e410c450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009910 tx=0x7ff9d0009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:02.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.280+0000 7ff9e9e00700 1 -- 172.21.15.115:0/2599183727 shutdown_connections 2024-03-22T10:38:02.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.280+0000 7ff9e9e00700 1 --2- 172.21.15.115:0/2599183727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e4073a00 0x7ff9e410c450 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:02.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.280+0000 7ff9e9e00700 1 --2- 172.21.15.115:0/2599183727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e40730f0 0x7ff9e40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:02.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.280+0000 7ff9e9e00700 1 -- 172.21.15.115:0/2599183727 >> 172.21.15.115:0/2599183727 conn(0x7ff9e40fbc30 msgr2=0x7ff9e40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:02.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.280+0000 7ff9e9e00700 1 -- 172.21.15.115:0/2599183727 shutdown_connections 2024-03-22T10:38:02.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.280+0000 7ff9e9e00700 1 -- 172.21.15.115:0/2599183727 wait complete. 2024-03-22T10:38:02.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.282+0000 7ff9e9e00700 1 Processor -- start 2024-03-22T10:38:02.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.282+0000 7ff9e9e00700 1 -- start start 2024-03-22T10:38:02.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.282+0000 7ff9e9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e40730f0 0x7ff9e4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:02.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.282+0000 7ff9e9e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e4073a00 0x7ff9e4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:02.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.282+0000 7ff9e9e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e419c7c0 con 0x7ff9e40730f0 2024-03-22T10:38:02.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.283+0000 7ff9e9e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e419c930 con 0x7ff9e4073a00 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.283+0000 7ff9e2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e40730f0 0x7ff9e4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.283+0000 7ff9e2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e40730f0 0x7ff9e4197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51736/0 (socket says 172.21.15.115:51736) 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.283+0000 7ff9e2a00700 1 -- 172.21.15.115:0/991065842 learned_addr learned my addr 172.21.15.115:0/991065842 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.283+0000 7ff9e2000700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e4073a00 0x7ff9e4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.283+0000 7ff9e2a00700 1 -- 172.21.15.115:0/991065842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e4073a00 msgr2=0x7ff9e4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.283+0000 7ff9e2a00700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e4073a00 0x7ff9e4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.283+0000 7ff9e2a00700 1 -- 172.21.15.115:0/991065842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d00095f0 con 0x7ff9e40730f0 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.283+0000 7ff9e2000700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e4073a00 0x7ff9e4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.284+0000 7ff9e2a00700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e40730f0 0x7ff9e4197e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c8a0 tx=0x7ff9d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.284+0000 7ff9e8a00700 1 -- 172.21.15.115:0/991065842 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9d400cea0 con 0x7ff9e40730f0 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.284+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e419cc10 con 0x7ff9e40730f0 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.284+0000 7ff9e8a00700 1 -- 172.21.15.115:0/991065842 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff9d4004830 con 0x7ff9e40730f0 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.284+0000 7ff9e8a00700 1 -- 172.21.15.115:0/991065842 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9d40055b0 con 0x7ff9e40730f0 2024-03-22T10:38:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.284+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e419d160 con 0x7ff9e40730f0 2024-03-22T10:38:02.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.286+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e4109b50 con 0x7ff9e40730f0 2024-03-22T10:38:02.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.287+0000 7ff9e8a00700 1 -- 172.21.15.115:0/991065842 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7ff9d4007ad0 con 0x7ff9e40730f0 2024-03-22T10:38:02.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.287+0000 7ff9e8a00700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff9c806c680 0x7ff9c806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:02.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.288+0000 7ff9e8a00700 1 -- 172.21.15.115:0/991065842 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff9d408b340 con 0x7ff9e40730f0 2024-03-22T10:38:02.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.288+0000 7ff9e2000700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff9c806c680 0x7ff9c806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:02.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.289+0000 7ff9e2000700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff9c806c680 0x7ff9c806eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006010 tx=0x7ff9d000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:02.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.292+0000 7ff9e8a00700 1 -- 172.21.15.115:0/991065842 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff9d4055cc0 con 0x7ff9e40730f0 2024-03-22T10:38:02.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:02 smithi115 ceph-mon[29111]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:02.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.558+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff9e41992b0 con 0x7ff9e40730f0 2024-03-22T10:38:02.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.559+0000 7ff9e8a00700 1 -- 172.21.15.115:0/991065842 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff9d40592e0 con 0x7ff9e40730f0 2024-03-22T10:38:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:38:02.551 INFO:teuthology.orchestra.run.smithi115.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-22T10:38:02.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff9c806c680 msgr2=0x7ff9c806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:02.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff9c806c680 0x7ff9c806eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006010 tx=0x7ff9d000b540 comp rx=0 tx=0).stop 2024-03-22T10:38:02.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e40730f0 msgr2=0x7ff9e4197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:02.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e40730f0 0x7ff9e4197e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c8a0 tx=0x7ff9d400cc60 comp rx=0 tx=0).stop 2024-03-22T10:38:02.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 shutdown_connections 2024-03-22T10:38:02.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff9c806c680 0x7ff9c806eb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:02.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e40730f0 0x7ff9e4197e10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:02.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 --2- 172.21.15.115:0/991065842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e4073a00 0x7ff9e4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:02.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 >> 172.21.15.115:0/991065842 conn(0x7ff9e40fbc30 msgr2=0x7ff9e4106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:02.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 shutdown_connections 2024-03-22T10:38:02.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:02.562+0000 7ff9e9e00700 1 -- 172.21.15.115:0/991065842 wait complete. 2024-03-22T10:38:02.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:02 smithi150 ceph-mon[34518]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:03.305 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-22T10:38:03.305 INFO:tasks.cephadm:Setup complete, yielding 2024-03-22T10:38:03.305 INFO:teuthology.run_tasks:Running task print... 2024-03-22T10:38:03.313 INFO:teuthology.task.print:**** done end installing v18.2.0 cephadm ... 2024-03-22T10:38:03.313 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T10:38:03.330 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:38:03.330 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-22T10:38:03.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:03 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/991065842' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T10:38:03.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:03 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:03.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:03 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/991065842' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T10:38:03.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:03 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:03.707 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:04.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:04 smithi115 ceph-mon[29111]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:04.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:04 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:04.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:04 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:38:04.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:04 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:04 smithi150 ceph-mon[34518]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:04 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:04 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:38:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:04 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:05.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.780+0000 7f7092e00700 1 -- 172.21.15.115:0/4156742370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 msgr2=0x7f708c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:05.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.780+0000 7f7092e00700 1 --2- 172.21.15.115:0/4156742370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 0x7f708c106bb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f707c009910 tx=0x7f707c009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:05.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.780+0000 7f7092e00700 1 -- 172.21.15.115:0/4156742370 shutdown_connections 2024-03-22T10:38:05.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.780+0000 7f7092e00700 1 --2- 172.21.15.115:0/4156742370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c1079a0 0x7f708c107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:05.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.780+0000 7f7092e00700 1 --2- 172.21.15.115:0/4156742370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 0x7f708c106bb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:05.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.780+0000 7f7092e00700 1 -- 172.21.15.115:0/4156742370 >> 172.21.15.115:0/4156742370 conn(0x7f708c075910 msgr2=0x7f708c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:05.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.780+0000 7f7092e00700 1 -- 172.21.15.115:0/4156742370 shutdown_connections 2024-03-22T10:38:05.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.781+0000 7f7092e00700 1 -- 172.21.15.115:0/4156742370 wait complete. 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.782+0000 7f7092e00700 1 Processor -- start 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.782+0000 7f7092e00700 1 -- start start 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.782+0000 7f7092e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 0x7f708c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.782+0000 7f7092e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c1079a0 0x7f708c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.782+0000 7f7092e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c19cb10 con 0x7f708c1067a0 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.782+0000 7f7092e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c19cc50 con 0x7f708c1079a0 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.782+0000 7f708be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 0x7f708c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.782+0000 7f708be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 0x7f708c19bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51764/0 (socket says 172.21.15.115:51764) 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.782+0000 7f708be00700 1 -- 172.21.15.115:0/3074157602 learned_addr learned my addr 172.21.15.115:0/3074157602 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.783+0000 7f708b400700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c1079a0 0x7f708c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:05.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.783+0000 7f708be00700 1 -- 172.21.15.115:0/3074157602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c1079a0 msgr2=0x7f708c19c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:05.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.783+0000 7f708be00700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c1079a0 0x7f708c19c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:05.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.783+0000 7f708be00700 1 -- 172.21.15.115:0/3074157602 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0095f0 con 0x7f708c1067a0 2024-03-22T10:38:05.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.783+0000 7f708be00700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 0x7f708c19bfb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f707c00bda0 tx=0x7f707c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:05.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.783+0000 7f7088c00700 1 -- 172.21.15.115:0/3074157602 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707c01d070 con 0x7f708c1067a0 2024-03-22T10:38:05.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.783+0000 7f7088c00700 1 -- 172.21.15.115:0/3074157602 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f707c00f460 con 0x7f708c1067a0 2024-03-22T10:38:05.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.783+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c1a16a0 con 0x7f708c1067a0 2024-03-22T10:38:05.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.783+0000 7f7088c00700 1 -- 172.21.15.115:0/3074157602 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707c021620 con 0x7f708c1067a0 2024-03-22T10:38:05.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.784+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c1a1b90 con 0x7f708c1067a0 2024-03-22T10:38:05.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.785+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c10f2f0 con 0x7f708c1067a0 2024-03-22T10:38:05.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.786+0000 7f7088c00700 1 -- 172.21.15.115:0/3074157602 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f707c00f5d0 con 0x7f708c1067a0 2024-03-22T10:38:05.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.786+0000 7f7088c00700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f707006c680 0x7f707006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:05.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.787+0000 7f7088c00700 1 -- 172.21.15.115:0/3074157602 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f707c08c880 con 0x7f708c1067a0 2024-03-22T10:38:05.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.787+0000 7f708b400700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f707006c680 0x7f707006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:05.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.788+0000 7f708b400700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f707006c680 0x7f707006eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7078005950 tx=0x7f70780058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:05.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:05.792+0000 7f7088c00700 1 -- 172.21.15.115:0/3074157602 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f707c05ad30 con 0x7f708c1067a0 2024-03-22T10:38:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.007+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f708c10f4b0 con 0x7f708c1067a0 2024-03-22T10:38:06.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.014+0000 7f7088c00700 1 -- 172.21.15.115:0/3074157602 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f707c05a8c0 con 0x7f708c1067a0 2024-03-22T10:38:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.017+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f707006c680 msgr2=0x7f707006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.017+0000 7f7092e00700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f707006c680 0x7f707006eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7078005950 tx=0x7f70780058e0 comp rx=0 tx=0).stop 2024-03-22T10:38:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.017+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 msgr2=0x7f708c19bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.017+0000 7f7092e00700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 0x7f708c19bfb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f707c00bda0 tx=0x7f707c00be80 comp rx=0 tx=0).stop 2024-03-22T10:38:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.018+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 shutdown_connections 2024-03-22T10:38:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.018+0000 7f7092e00700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f707006c680 0x7f707006eb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.018+0000 7f7092e00700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c1067a0 0x7f708c19bfb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.018+0000 7f7092e00700 1 --2- 172.21.15.115:0/3074157602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c1079a0 0x7f708c19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.018+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 >> 172.21.15.115:0/3074157602 conn(0x7f708c075910 msgr2=0x7f708c10abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.018+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 shutdown_connections 2024-03-22T10:38:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:06.018+0000 7f7092e00700 1 -- 172.21.15.115:0/3074157602 wait complete. 2024-03-22T10:38:06.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:06 smithi150 ceph-mon[34518]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:06.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:06 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3074157602' entity='client.admin' 2024-03-22T10:38:06.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:06 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:06.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:06 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:06.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:06 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:38:06.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:06 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:06.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:06 smithi115 ceph-mon[29111]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:06.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:06 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3074157602' entity='client.admin' 2024-03-22T10:38:06.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:06 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:06.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:06 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:06.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:06 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:38:06.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:06 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:06.760 INFO:teuthology.run_tasks:Running task print... 2024-03-22T10:38:06.769 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-22T10:38:06.769 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T10:38:06.784 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:38:06.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-22T10:38:07.219 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:08.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:08 smithi150 ceph-mon[34518]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:08.725 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:08 smithi115 ceph-mon[29111]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:09.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.212+0000 7f7051e00700 1 -- 172.21.15.115:0/1109527941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 msgr2=0x7f704c1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:09.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.212+0000 7f7051e00700 1 --2- 172.21.15.115:0/1109527941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 0x7f704c1017a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7034009910 tx=0x7f7034009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:09.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.212+0000 7f7051e00700 1 -- 172.21.15.115:0/1109527941 shutdown_connections 2024-03-22T10:38:09.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.212+0000 7f7051e00700 1 --2- 172.21.15.115:0/1109527941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 0x7f704c1017a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:09.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.212+0000 7f7051e00700 1 --2- 172.21.15.115:0/1109527941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f704c100150 0x7f704c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:09.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.212+0000 7f7051e00700 1 -- 172.21.15.115:0/1109527941 >> 172.21.15.115:0/1109527941 conn(0x7f704c0fb6e0 msgr2=0x7f704c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:09.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.212+0000 7f7051e00700 1 -- 172.21.15.115:0/1109527941 shutdown_connections 2024-03-22T10:38:09.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.212+0000 7f7051e00700 1 -- 172.21.15.115:0/1109527941 wait complete. 2024-03-22T10:38:09.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.213+0000 7f7051e00700 1 Processor -- start 2024-03-22T10:38:09.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.213+0000 7f7051e00700 1 -- start start 2024-03-22T10:38:09.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.214+0000 7f7051e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f704c100150 0x7f704c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:09.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.214+0000 7f7051e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 0x7f704c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:09.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.214+0000 7f7051e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c1986d0 con 0x7f704c101350 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.214+0000 7f7051e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c198810 con 0x7f704c100150 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.214+0000 7f7042000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 0x7f704c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.214+0000 7f7042000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 0x7f704c1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51778/0 (socket says 172.21.15.115:51778) 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.214+0000 7f7042000700 1 -- 172.21.15.115:0/3790786047 learned_addr learned my addr 172.21.15.115:0/3790786047 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.214+0000 7f704aa00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f704c100150 0x7f704c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f7042000700 1 -- 172.21.15.115:0/3790786047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f704c100150 msgr2=0x7f704c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f7042000700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f704c100150 0x7f704c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f7042000700 1 -- 172.21.15.115:0/3790786047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70340095f0 con 0x7f704c101350 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f704aa00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f704c100150 0x7f704c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f7042000700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 0x7f704c1980b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f703400bd70 tx=0x7f703400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f7050a00700 1 -- 172.21.15.115:0/3790786047 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f703401d070 con 0x7f704c101350 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f7050a00700 1 -- 172.21.15.115:0/3790786047 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7034022470 con 0x7f704c101350 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f7050a00700 1 -- 172.21.15.115:0/3790786047 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f703400f670 con 0x7f704c101350 2024-03-22T10:38:09.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c19d260 con 0x7f704c101350 2024-03-22T10:38:09.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.215+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c19d750 con 0x7f704c101350 2024-03-22T10:38:09.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.217+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c105430 con 0x7f704c101350 2024-03-22T10:38:09.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.217+0000 7f7050a00700 1 -- 172.21.15.115:0/3790786047 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7034022a50 con 0x7f704c101350 2024-03-22T10:38:09.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.218+0000 7f7050a00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f703006c680 0x7f703006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:09.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.218+0000 7f7050a00700 1 -- 172.21.15.115:0/3790786047 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f703408cbe0 con 0x7f704c101350 2024-03-22T10:38:09.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.219+0000 7f704aa00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f703006c680 0x7f703006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:09.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.219+0000 7f704aa00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f703006c680 0x7f703006eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f704c1011b0 tx=0x7f703c005f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:09.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.223+0000 7f7050a00700 1 -- 172.21.15.115:0/3790786047 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f703405b120 con 0x7f704c101350 2024-03-22T10:38:09.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.455+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f704c061190 con 0x7f703006c680 2024-03-22T10:38:09.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.456+0000 7f7050a00700 1 -- 172.21.15.115:0/3790786047 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f704c061190 con 0x7f703006c680 2024-03-22T10:38:09.449 INFO:teuthology.orchestra.run.smithi115.stdout:Backend: cephadm 2024-03-22T10:38:09.449 INFO:teuthology.orchestra.run.smithi115.stdout:Available: Yes 2024-03-22T10:38:09.449 INFO:teuthology.orchestra.run.smithi115.stdout:Paused: No 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.460+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f703006c680 msgr2=0x7f703006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.460+0000 7f7051e00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f703006c680 0x7f703006eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f704c1011b0 tx=0x7f703c005f90 comp rx=0 tx=0).stop 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.460+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 msgr2=0x7f704c1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.460+0000 7f7051e00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 0x7f704c1980b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f703400bd70 tx=0x7f703400be50 comp rx=0 tx=0).stop 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.461+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 shutdown_connections 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.461+0000 7f7051e00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f703006c680 0x7f703006eb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.461+0000 7f7051e00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f704c100150 0x7f704c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.461+0000 7f7051e00700 1 --2- 172.21.15.115:0/3790786047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f704c101350 0x7f704c1980b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.461+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 >> 172.21.15.115:0/3790786047 conn(0x7f704c0fb6e0 msgr2=0x7f704c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:09.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.461+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 shutdown_connections 2024-03-22T10:38:09.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:09.461+0000 7f7051e00700 1 -- 172.21.15.115:0/3790786047 wait complete. 2024-03-22T10:38:10.188 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-22T10:38:10.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:10 smithi115 ceph-mon[29111]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:10.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:10 smithi115 ceph-mon[29111]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:10.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:38:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:10.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:10 smithi150 ceph-mon[34518]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:10.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:10 smithi150 ceph-mon[34518]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:10.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:38:12.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:12 smithi115 ceph-mon[29111]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:12.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:12 smithi150 ceph-mon[34518]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.698+0000 7f176a000700 1 -- 172.21.15.115:0/2209386645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764101370 msgr2=0x7f17641017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.698+0000 7f176a000700 1 --2- 172.21.15.115:0/2209386645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764101370 0x7f17641017c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1750009910 tx=0x7f1750009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.698+0000 7f176a000700 1 -- 172.21.15.115:0/2209386645 shutdown_connections 2024-03-22T10:38:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.698+0000 7f176a000700 1 --2- 172.21.15.115:0/2209386645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764101370 0x7f17641017c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.698+0000 7f176a000700 1 --2- 172.21.15.115:0/2209386645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1764100170 0x7f1764100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.698+0000 7f176a000700 1 -- 172.21.15.115:0/2209386645 >> 172.21.15.115:0/2209386645 conn(0x7f17640fb6e0 msgr2=0x7f17640fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.698+0000 7f176a000700 1 -- 172.21.15.115:0/2209386645 shutdown_connections 2024-03-22T10:38:12.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.698+0000 7f176a000700 1 -- 172.21.15.115:0/2209386645 wait complete. 2024-03-22T10:38:12.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.699+0000 7f176a000700 1 Processor -- start 2024-03-22T10:38:12.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.699+0000 7f176a000700 1 -- start start 2024-03-22T10:38:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f176a000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764100170 0x7f1764197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f176a000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1764101370 0x7f17641980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f176a000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764198660 con 0x7f1764100170 2024-03-22T10:38:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f176a000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17641987a0 con 0x7f1764101370 2024-03-22T10:38:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f1763400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764100170 0x7f1764197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f1763400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764100170 0x7f1764197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34492/0 (socket says 172.21.15.115:34492) 2024-03-22T10:38:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f1763400700 1 -- 172.21.15.115:0/1360837472 learned_addr learned my addr 172.21.15.115:0/1360837472 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f1762a00700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1764101370 0x7f17641980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:12.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f1763400700 1 -- 172.21.15.115:0/1360837472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1764101370 msgr2=0x7f17641980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:12.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f1763400700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1764101370 0x7f17641980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:12.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.700+0000 7f1763400700 1 -- 172.21.15.115:0/1360837472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17500095f0 con 0x7f1764100170 2024-03-22T10:38:12.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.701+0000 7f1763400700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764100170 0x7f1764197b90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f175400c910 tx=0x7f175400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:12.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.701+0000 7f175be00700 1 -- 172.21.15.115:0/1360837472 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17540041f0 con 0x7f1764100170 2024-03-22T10:38:12.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.701+0000 7f175be00700 1 -- 172.21.15.115:0/1360837472 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1754007db0 con 0x7f1764100170 2024-03-22T10:38:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.701+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176419d260 con 0x7f1764100170 2024-03-22T10:38:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.701+0000 7f175be00700 1 -- 172.21.15.115:0/1360837472 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1754003d60 con 0x7f1764100170 2024-03-22T10:38:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.701+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176419d780 con 0x7f1764100170 2024-03-22T10:38:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.702+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1764105450 con 0x7f1764100170 2024-03-22T10:38:12.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.703+0000 7f175be00700 1 -- 172.21.15.115:0/1360837472 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f1754007900 con 0x7f1764100170 2024-03-22T10:38:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.704+0000 7f175be00700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f174806c680 0x7f174806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.704+0000 7f175be00700 1 -- 172.21.15.115:0/1360837472 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f175408b240 con 0x7f1764100170 2024-03-22T10:38:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.704+0000 7f1762a00700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f174806c680 0x7f174806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:12.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.705+0000 7f1762a00700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f174806c680 0x7f174806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1750006010 tx=0x7f175000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:12.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.709+0000 7f175be00700 1 -- 172.21.15.115:0/1360837472 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f17540596f0 con 0x7f1764100170 2024-03-22T10:38:12.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.931+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1764061190 con 0x7f174806c680 2024-03-22T10:38:12.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.957+0000 7f175be00700 1 -- 172.21.15.115:0/1360837472 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7f1764061190 con 0x7f174806c680 2024-03-22T10:38:12.950 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:38:12.950 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (4m) 2m ago 6m 15.0M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:38:12.950 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (6m) 2m ago 6m 7096k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:38:12.950 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (5m) 75s ago 5m 6983k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:38:12.950 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (6m) 2m ago 6m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:38:12.950 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (5m) 75s ago 5m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:38:12.950 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (4m) 2m ago 6m 72.1M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:38:12.950 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:9283,8765,8443 running (8m) 2m ago 8m 480M - 18.2.0 dc2bc1663786 932346df0e0f 2024-03-22T10:38:12.950 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (5m) 75s ago 5m 436M - 18.2.0 dc2bc1663786 3884fe56fbc9 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (8m) 2m ago 8m 43.2M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (5m) 75s ago 5m 32.9M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (6m) 2m ago 6m 7726k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (5m) 75s ago 5m 7247k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (4m) 2m ago 4m 38.1M 3947M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (3m) 2m ago 3m 39.9M 3947M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (3m) 2m ago 3m 40.2M 3947M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (2m) 75s ago 2m 40.1M 4971M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (115s) 75s ago 114s 37.7M 4971M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (80s) 75s ago 80s 12.4M 4971M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:38:12.951 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (4m) 2m ago 5m 42.2M - 2.43.0 a07b618ecd1d 9fcacfb2c3cb 2024-03-22T10:38:12.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.960+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f174806c680 msgr2=0x7f174806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:12.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.960+0000 7f176a000700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f174806c680 0x7f174806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1750006010 tx=0x7f175000b540 comp rx=0 tx=0).stop 2024-03-22T10:38:12.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.960+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764100170 msgr2=0x7f1764197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:12.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.960+0000 7f176a000700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764100170 0x7f1764197b90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f175400c910 tx=0x7f175400ccd0 comp rx=0 tx=0).stop 2024-03-22T10:38:12.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.961+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 shutdown_connections 2024-03-22T10:38:12.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.961+0000 7f176a000700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f174806c680 0x7f174806eb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:12.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.961+0000 7f176a000700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1764100170 0x7f1764197b90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:12.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.961+0000 7f176a000700 1 --2- 172.21.15.115:0/1360837472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1764101370 0x7f17641980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:12.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.961+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 >> 172.21.15.115:0/1360837472 conn(0x7f17640fb6e0 msgr2=0x7f1764102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:12.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.961+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 shutdown_connections 2024-03-22T10:38:12.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:12.961+0000 7f176a000700 1 -- 172.21.15.115:0/1360837472 wait complete. 2024-03-22T10:38:13.863 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-22T10:38:14.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:13 smithi115 ceph-mon[29111]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:14.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:13 smithi115 ceph-mon[29111]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:14.284 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:14.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:13 smithi150 ceph-mon[34518]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:14.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:13 smithi150 ceph-mon[34518]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:16.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.401+0000 7f58bf600700 1 -- 172.21.15.115:0/3759748276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 msgr2=0x7f58b8105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:16.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.401+0000 7f58bf600700 1 --2- 172.21.15.115:0/3759748276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 0x7f58b8105960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009910 tx=0x7f58a4009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:16.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.401+0000 7f58bf600700 1 -- 172.21.15.115:0/3759748276 shutdown_connections 2024-03-22T10:38:16.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.401+0000 7f58bf600700 1 --2- 172.21.15.115:0/3759748276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 0x7f58b8105960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:16.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.401+0000 7f58bf600700 1 --2- 172.21.15.115:0/3759748276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58b8100c60 0x7f58b8103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:16.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.401+0000 7f58bf600700 1 -- 172.21.15.115:0/3759748276 >> 172.21.15.115:0/3759748276 conn(0x7f58b80fa5e0 msgr2=0x7f58b80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:16.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.401+0000 7f58bf600700 1 -- 172.21.15.115:0/3759748276 shutdown_connections 2024-03-22T10:38:16.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.401+0000 7f58bf600700 1 -- 172.21.15.115:0/3759748276 wait complete. 2024-03-22T10:38:16.396 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:16 smithi115 ceph-mon[29111]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:16.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.403+0000 7f58bf600700 1 Processor -- start 2024-03-22T10:38:16.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.403+0000 7f58bf600700 1 -- start start 2024-03-22T10:38:16.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.403+0000 7f58bf600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58b8100c60 0x7f58b8071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:16.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.403+0000 7f58bf600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 0x7f58b8072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:16.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.403+0000 7f58bf600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8072840 con 0x7f58b8103580 2024-03-22T10:38:16.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.403+0000 7f58bf600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81a3480 con 0x7f58b8100c60 2024-03-22T10:38:16.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.404+0000 7f58b7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 0x7f58b8072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:16.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.404+0000 7f58b7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 0x7f58b8072220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34508/0 (socket says 172.21.15.115:34508) 2024-03-22T10:38:16.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.404+0000 7f58b7e00700 1 -- 172.21.15.115:0/889827328 learned_addr learned my addr 172.21.15.115:0/889827328 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:16.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.404+0000 7f58bcc00700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58b8100c60 0x7f58b8071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:16.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.404+0000 7f58b7e00700 1 -- 172.21.15.115:0/889827328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58b8100c60 msgr2=0x7f58b8071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:16.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.404+0000 7f58b7e00700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58b8100c60 0x7f58b8071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:16.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.404+0000 7f58b7e00700 1 -- 172.21.15.115:0/889827328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40095f0 con 0x7f58b8103580 2024-03-22T10:38:16.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.404+0000 7f58bcc00700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58b8100c60 0x7f58b8071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:38:16.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.405+0000 7f58b7e00700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 0x7f58b8072220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f58a4005b40 tx=0x7f58a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.405+0000 7f58b5600700 1 -- 172.21.15.115:0/889827328 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a401d070 con 0x7f58b8103580 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.405+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b80fed10 con 0x7f58b8103580 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.405+0000 7f58b5600700 1 -- 172.21.15.115:0/889827328 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f58a4022470 con 0x7f58b8103580 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.405+0000 7f58b5600700 1 -- 172.21.15.115:0/889827328 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a400f690 con 0x7f58b8103580 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.405+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b80ff280 con 0x7f58b8103580 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.407+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b804ea50 con 0x7f58b8103580 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.409+0000 7f58b5600700 1 -- 172.21.15.115:0/889827328 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f58a40225e0 con 0x7f58b8103580 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.409+0000 7f58b5600700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f58a006c680 0x7f58a006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.410+0000 7f58b5600700 1 -- 172.21.15.115:0/889827328 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f58a408d420 con 0x7f58b8103580 2024-03-22T10:38:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.410+0000 7f58bcc00700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f58a006c680 0x7f58a006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:16.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.410+0000 7f58bcc00700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f58a006c680 0x7f58a006eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0099d0 tx=0x7f58ac008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:16.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.414+0000 7f58b5600700 1 -- 172.21.15.115:0/889827328 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f58a405b9f0 con 0x7f58b8103580 2024-03-22T10:38:16.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.634+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f58b806b960 con 0x7f58a006c680 2024-03-22T10:38:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.640+0000 7f58b5600700 1 -- 172.21.15.115:0/889827328 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7f58b806b960 con 0x7f58a006c680 2024-03-22T10:38:16.632 INFO:teuthology.orchestra.run.smithi115.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-22T10:38:16.632 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-03-22T10:38:16.632 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter 2/2 2m ago 7m * 2024-03-22T10:38:16.632 INFO:teuthology.orchestra.run.smithi115.stdout:crash 2/2 2m ago 7m * 2024-03-22T10:38:16.632 INFO:teuthology.orchestra.run.smithi115.stdout:grafana ?:3000 1/1 2m ago 7m count:1 2024-03-22T10:38:16.633 INFO:teuthology.orchestra.run.smithi115.stdout:mgr 2/2 2m ago 7m count:2 2024-03-22T10:38:16.633 INFO:teuthology.orchestra.run.smithi115.stdout:mon 2/2 2m ago 6m smithi115:172.21.15.115=smithi115;smithi150:172.21.15.150=smithi150;count:2 2024-03-22T10:38:16.633 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-03-22T10:38:16.633 INFO:teuthology.orchestra.run.smithi115.stdout:osd 6 2m ago - 2024-03-22T10:38:16.633 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus ?:9095 1/1 2m ago 7m count:1 2024-03-22T10:38:16.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f58a006c680 msgr2=0x7f58a006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:16.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f58a006c680 0x7f58a006eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0099d0 tx=0x7f58ac008040 comp rx=0 tx=0).stop 2024-03-22T10:38:16.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 msgr2=0x7f58b8072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:16.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 0x7f58b8072220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f58a4005b40 tx=0x7f58a400be20 comp rx=0 tx=0).stop 2024-03-22T10:38:16.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 shutdown_connections 2024-03-22T10:38:16.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f58a006c680 0x7f58a006eb30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:16.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58b8100c60 0x7f58b8071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:16.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 --2- 172.21.15.115:0/889827328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58b8103580 0x7f58b8072220 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:16.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 >> 172.21.15.115:0/889827328 conn(0x7f58b80fa5e0 msgr2=0x7f58b80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:16.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 shutdown_connections 2024-03-22T10:38:16.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:16.643+0000 7f58bf600700 1 -- 172.21.15.115:0/889827328 wait complete. 2024-03-22T10:38:16.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:16 smithi150 ceph-mon[34518]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:17.432 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-22T10:38:17.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:17 smithi150 ceph-mon[34518]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:17.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:17 smithi115 ceph-mon[29111]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:18.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:18 smithi150 ceph-mon[34518]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:18.743 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:18 smithi115 ceph-mon[29111]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:19.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.876+0000 7f765e200700 1 -- 172.21.15.115:0/2880147285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 msgr2=0x7f7658106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:19.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.876+0000 7f765e200700 1 --2- 172.21.15.115:0/2880147285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 0x7f7658106bb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7648009910 tx=0x7f7648009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:19.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.876+0000 7f765e200700 1 -- 172.21.15.115:0/2880147285 shutdown_connections 2024-03-22T10:38:19.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.876+0000 7f765e200700 1 --2- 172.21.15.115:0/2880147285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76581079a0 0x7f7658107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:19.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.876+0000 7f765e200700 1 --2- 172.21.15.115:0/2880147285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 0x7f7658106bb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.876+0000 7f765e200700 1 -- 172.21.15.115:0/2880147285 >> 172.21.15.115:0/2880147285 conn(0x7f7658075910 msgr2=0x7f7658077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.877+0000 7f765e200700 1 -- 172.21.15.115:0/2880147285 shutdown_connections 2024-03-22T10:38:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.877+0000 7f765e200700 1 -- 172.21.15.115:0/2880147285 wait complete. 2024-03-22T10:38:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.878+0000 7f765e200700 1 Processor -- start 2024-03-22T10:38:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.878+0000 7f765e200700 1 -- start start 2024-03-22T10:38:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.878+0000 7f765e200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 0x7f765819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.878+0000 7f765e200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76581079a0 0x7f765819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.878+0000 7f765e200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765819cb10 con 0x7f76581067a0 2024-03-22T10:38:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.878+0000 7f765e200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765819cc50 con 0x7f76581079a0 2024-03-22T10:38:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.879+0000 7f7657400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 0x7f765819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.879+0000 7f7657400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 0x7f765819bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34532/0 (socket says 172.21.15.115:34532) 2024-03-22T10:38:19.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.879+0000 7f7657400700 1 -- 172.21.15.115:0/4117789938 learned_addr learned my addr 172.21.15.115:0/4117789938 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:19.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.879+0000 7f7656a00700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76581079a0 0x7f765819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:19.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.879+0000 7f7657400700 1 -- 172.21.15.115:0/4117789938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76581079a0 msgr2=0x7f765819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:19.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.879+0000 7f7657400700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76581079a0 0x7f765819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:19.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.879+0000 7f7657400700 1 -- 172.21.15.115:0/4117789938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480095f0 con 0x7f76581067a0 2024-03-22T10:38:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.880+0000 7f7657400700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 0x7f765819bfb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f764800bda0 tx=0x7f764800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.880+0000 7f764fe00700 1 -- 172.21.15.115:0/4117789938 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764801d070 con 0x7f76581067a0 2024-03-22T10:38:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.880+0000 7f764fe00700 1 -- 172.21.15.115:0/4117789938 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7648004c50 con 0x7f76581067a0 2024-03-22T10:38:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.880+0000 7f764fe00700 1 -- 172.21.15.115:0/4117789938 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764800f690 con 0x7f76581067a0 2024-03-22T10:38:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.880+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76581a16a0 con 0x7f76581067a0 2024-03-22T10:38:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.880+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76581a1b90 con 0x7f76581067a0 2024-03-22T10:38:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.881+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765810f2f0 con 0x7f76581067a0 2024-03-22T10:38:19.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.882+0000 7f764fe00700 1 -- 172.21.15.115:0/4117789938 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f76480229b0 con 0x7f76581067a0 2024-03-22T10:38:19.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.883+0000 7f764fe00700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f763c06c680 0x7f763c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:19.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.883+0000 7f764fe00700 1 -- 172.21.15.115:0/4117789938 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f764808cc70 con 0x7f76581067a0 2024-03-22T10:38:19.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.883+0000 7f7656a00700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f763c06c680 0x7f763c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:19.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.884+0000 7f7656a00700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f763c06c680 0x7f763c06eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7644005fd0 tx=0x7f764400a560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:19.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:19.888+0000 7f764fe00700 1 -- 172.21.15.115:0/4117789938 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f764805b1d0 con 0x7f76581067a0 2024-03-22T10:38:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.112+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f765810c2f0 con 0x7f763c06c680 2024-03-22T10:38:20.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.115+0000 7f764fe00700 1 -- 172.21.15.115:0/4117789938 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f765810c2f0 con 0x7f763c06c680 2024-03-22T10:38:20.107 INFO:teuthology.orchestra.run.smithi115.stdout:HOST ADDR LABELS STATUS 2024-03-22T10:38:20.107 INFO:teuthology.orchestra.run.smithi115.stdout:smithi115 172.21.15.115 2024-03-22T10:38:20.107 INFO:teuthology.orchestra.run.smithi115.stdout:smithi150 172.21.15.150 2024-03-22T10:38:20.107 INFO:teuthology.orchestra.run.smithi115.stdout:2 hosts in cluster 2024-03-22T10:38:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.117+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f763c06c680 msgr2=0x7f763c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.117+0000 7f765e200700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f763c06c680 0x7f763c06eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7644005fd0 tx=0x7f764400a560 comp rx=0 tx=0).stop 2024-03-22T10:38:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.117+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 msgr2=0x7f765819bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.117+0000 7f765e200700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 0x7f765819bfb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f764800bda0 tx=0x7f764800be80 comp rx=0 tx=0).stop 2024-03-22T10:38:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.118+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 shutdown_connections 2024-03-22T10:38:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.118+0000 7f765e200700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f763c06c680 0x7f763c06eb30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.118+0000 7f765e200700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76581067a0 0x7f765819bfb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.118+0000 7f765e200700 1 --2- 172.21.15.115:0/4117789938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76581079a0 0x7f765819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.118+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 >> 172.21.15.115:0/4117789938 conn(0x7f7658075910 msgr2=0x7f765810abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:20.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.118+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 shutdown_connections 2024-03-22T10:38:20.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:20.118+0000 7f765e200700 1 -- 172.21.15.115:0/4117789938 wait complete. 2024-03-22T10:38:20.455 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:20 smithi115 ceph-mon[29111]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:20.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:20 smithi150 ceph-mon[34518]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:20.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-22T10:38:21.310 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:21.609 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:21 smithi115 ceph-mon[29111]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:21.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:21 smithi150 ceph-mon[34518]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:22.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:22 smithi150 ceph-mon[34518]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:22.757 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:22 smithi115 ceph-mon[29111]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:23.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.566+0000 7fb53de00700 1 -- 172.21.15.115:0/868276390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381021c0 msgr2=0x7fb5381025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:23.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.566+0000 7fb53de00700 1 --2- 172.21.15.115:0/868276390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381021c0 0x7fb5381025d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009910 tx=0x7fb52c009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:23.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.566+0000 7fb53de00700 1 -- 172.21.15.115:0/868276390 shutdown_connections 2024-03-22T10:38:23.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.566+0000 7fb53de00700 1 --2- 172.21.15.115:0/868276390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5381033c0 0x7fb538103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:23.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.566+0000 7fb53de00700 1 --2- 172.21.15.115:0/868276390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381021c0 0x7fb5381025d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:23.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.566+0000 7fb53de00700 1 -- 172.21.15.115:0/868276390 >> 172.21.15.115:0/868276390 conn(0x7fb5380fd8d0 msgr2=0x7fb5380ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:23.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.566+0000 7fb53de00700 1 -- 172.21.15.115:0/868276390 shutdown_connections 2024-03-22T10:38:23.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.566+0000 7fb53de00700 1 -- 172.21.15.115:0/868276390 wait complete. 2024-03-22T10:38:23.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.567+0000 7fb53de00700 1 Processor -- start 2024-03-22T10:38:23.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.568+0000 7fb53de00700 1 -- start start 2024-03-22T10:38:23.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.568+0000 7fb53de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5381021c0 0x7fb5381979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:23.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.568+0000 7fb53de00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381033c0 0x7fb538197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:23.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.568+0000 7fb53de00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb538198520 con 0x7fb5381033c0 2024-03-22T10:38:23.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.568+0000 7fb53de00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb538198660 con 0x7fb5381021c0 2024-03-22T10:38:23.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.568+0000 7fb537e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381033c0 0x7fb538197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:23.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.568+0000 7fb537e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381033c0 0x7fb538197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42354/0 (socket says 172.21.15.115:42354) 2024-03-22T10:38:23.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.568+0000 7fb537e00700 1 -- 172.21.15.115:0/3574883831 learned_addr learned my addr 172.21.15.115:0/3574883831 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:23.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.569+0000 7fb53ca00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5381021c0 0x7fb5381979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.569+0000 7fb537e00700 1 -- 172.21.15.115:0/3574883831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5381021c0 msgr2=0x7fb5381979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.569+0000 7fb537e00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5381021c0 0x7fb5381979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.569+0000 7fb537e00700 1 -- 172.21.15.115:0/3574883831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0095f0 con 0x7fb5381033c0 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.569+0000 7fb537e00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381033c0 0x7fb538197f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb52400d8d0 tx=0x7fb52400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.569+0000 7fb535600700 1 -- 172.21.15.115:0/3574883831 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb524009940 con 0x7fb5381033c0 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.569+0000 7fb535600700 1 -- 172.21.15.115:0/3574883831 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb524010460 con 0x7fb5381033c0 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.569+0000 7fb535600700 1 -- 172.21.15.115:0/3574883831 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb52400f5f0 con 0x7fb5381033c0 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.569+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53819d110 con 0x7fb5381033c0 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.570+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53819d660 con 0x7fb5381033c0 2024-03-22T10:38:23.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.571+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53810ad10 con 0x7fb5381033c0 2024-03-22T10:38:23.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.572+0000 7fb535600700 1 -- 172.21.15.115:0/3574883831 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb5240105d0 con 0x7fb5381033c0 2024-03-22T10:38:23.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.572+0000 7fb535600700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb52006c680 0x7fb52006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:23.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.573+0000 7fb535600700 1 -- 172.21.15.115:0/3574883831 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb52408b510 con 0x7fb5381033c0 2024-03-22T10:38:23.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.573+0000 7fb53ca00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb52006c680 0x7fb52006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:23.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.573+0000 7fb53ca00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb52006c680 0x7fb52006eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0059d0 tx=0x7fb52c005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:23.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.577+0000 7fb535600700 1 -- 172.21.15.115:0/3574883831 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb5240599a0 con 0x7fb5381033c0 2024-03-22T10:38:23.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.802+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb538107d10 con 0x7fb52006c680 2024-03-22T10:38:23.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.805+0000 7fb535600700 1 -- 172.21.15.115:0/3574883831 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fb538107d10 con 0x7fb52006c680 2024-03-22T10:38:23.797 INFO:teuthology.orchestra.run.smithi115.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-22T10:38:23.797 INFO:teuthology.orchestra.run.smithi115.stdout:smithi115 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UU400BGN 372G No 2m ago LVM detected, locked 2024-03-22T10:38:23.798 INFO:teuthology.orchestra.run.smithi115.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 81s ago LVM detected, locked 2024-03-22T10:38:23.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb52006c680 msgr2=0x7fb52006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:23.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb52006c680 0x7fb52006eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0059d0 tx=0x7fb52c005960 comp rx=0 tx=0).stop 2024-03-22T10:38:23.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381033c0 msgr2=0x7fb538197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:23.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381033c0 0x7fb538197f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb52400d8d0 tx=0x7fb52400dc90 comp rx=0 tx=0).stop 2024-03-22T10:38:23.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 shutdown_connections 2024-03-22T10:38:23.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb52006c680 0x7fb52006eb30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:23.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5381021c0 0x7fb5381979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:23.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 --2- 172.21.15.115:0/3574883831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5381033c0 0x7fb538197f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:23.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 >> 172.21.15.115:0/3574883831 conn(0x7fb5380fd8d0 msgr2=0x7fb5381065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:23.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.808+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 shutdown_connections 2024-03-22T10:38:23.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:23.809+0000 7fb53de00700 1 -- 172.21.15.115:0/3574883831 wait complete. 2024-03-22T10:38:24.152 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:23 smithi115 ceph-mon[29111]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:24.152 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:23 smithi115 ceph-mon[29111]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:24.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:23 smithi150 ceph-mon[34518]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:24.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:23 smithi150 ceph-mon[34518]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:24.603 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T10:38:24.613 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:38:24.614 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-22T10:38:25.038 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:25.493 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:38:25.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:38:26.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:26 smithi150 ceph-mon[34518]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:26.728 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:26 smithi115 ceph-mon[29111]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:27.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.230+0000 7f784e400700 1 -- 172.21.15.115:0/3377290973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481035a0 msgr2=0x7f78481039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:27.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.230+0000 7f784e400700 1 --2- 172.21.15.115:0/3377290973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481035a0 0x7f78481039f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7834009910 tx=0x7f7834009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:27.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.230+0000 7f784e400700 1 -- 172.21.15.115:0/3377290973 shutdown_connections 2024-03-22T10:38:27.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.230+0000 7f784e400700 1 --2- 172.21.15.115:0/3377290973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481035a0 0x7f78481039f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:27.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.230+0000 7f784e400700 1 --2- 172.21.15.115:0/3377290973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78481023a0 0x7f78481027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:27.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.230+0000 7f784e400700 1 -- 172.21.15.115:0/3377290973 >> 172.21.15.115:0/3377290973 conn(0x7f78480fd990 msgr2=0x7f78480ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:27.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.230+0000 7f784e400700 1 -- 172.21.15.115:0/3377290973 shutdown_connections 2024-03-22T10:38:27.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.230+0000 7f784e400700 1 -- 172.21.15.115:0/3377290973 wait complete. 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.232+0000 7f784e400700 1 Processor -- start 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.232+0000 7f784e400700 1 -- start start 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.232+0000 7f784e400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481023a0 0x7f7848197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.232+0000 7f784e400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78481035a0 0x7f7848198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.232+0000 7f784e400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78481987a0 con 0x7f78481023a0 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.232+0000 7f784e400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784819d1b0 con 0x7f78481035a0 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.233+0000 7f7847400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481023a0 0x7f7848197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.233+0000 7f7847400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481023a0 0x7f7848197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42374/0 (socket says 172.21.15.115:42374) 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.233+0000 7f7847400700 1 -- 172.21.15.115:0/3769262634 learned_addr learned my addr 172.21.15.115:0/3769262634 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.233+0000 7f7846a00700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78481035a0 0x7f7848198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.233+0000 7f7847400700 1 -- 172.21.15.115:0/3769262634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78481035a0 msgr2=0x7f7848198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:27.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.233+0000 7f7847400700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78481035a0 0x7f7848198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:27.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.233+0000 7f7847400700 1 -- 172.21.15.115:0/3769262634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78340095f0 con 0x7f78481023a0 2024-03-22T10:38:27.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.233+0000 7f7847400700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481023a0 0x7f7848197c40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f783800b480 tx=0x7f783800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:27.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.234+0000 7f783fe00700 1 -- 172.21.15.115:0/3769262634 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78380118c0 con 0x7f78481023a0 2024-03-22T10:38:27.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.234+0000 7f783fe00700 1 -- 172.21.15.115:0/3769262634 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7838010380 con 0x7f78481023a0 2024-03-22T10:38:27.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.234+0000 7f783fe00700 1 -- 172.21.15.115:0/3769262634 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f783800f4f0 con 0x7f78481023a0 2024-03-22T10:38:27.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.234+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784819d3b0 con 0x7f78481023a0 2024-03-22T10:38:27.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.234+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784819d950 con 0x7f78481023a0 2024-03-22T10:38:27.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.235+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784810af90 con 0x7f78481023a0 2024-03-22T10:38:27.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.236+0000 7f783fe00700 1 -- 172.21.15.115:0/3769262634 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f78380104f0 con 0x7f78481023a0 2024-03-22T10:38:27.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.237+0000 7f783fe00700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f782c06c680 0x7f782c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:27.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.237+0000 7f783fe00700 1 -- 172.21.15.115:0/3769262634 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f783808b0d0 con 0x7f78481023a0 2024-03-22T10:38:27.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.237+0000 7f7846a00700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f782c06c680 0x7f782c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:27.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.238+0000 7f7846a00700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f782c06c680 0x7f782c06eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7834006010 tx=0x7f783400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:27.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.241+0000 7f783fe00700 1 -- 172.21.15.115:0/3769262634 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7838055b70 con 0x7f78481023a0 2024-03-22T10:38:27.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:27.486+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7848107ef0 con 0x7f782c06c680 2024-03-22T10:38:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:28 smithi150 ceph-mon[34518]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:28 smithi150 ceph-mon[34518]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:28 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-22T10:38:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:28 smithi115 ceph-mon[29111]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:28 smithi115 ceph-mon[29111]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:38:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:28 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-22T10:38:29.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.398+0000 7f783fe00700 1 -- 172.21.15.115:0/3769262634 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7848107ef0 con 0x7f782c06c680 2024-03-22T10:38:29.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.401+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f782c06c680 msgr2=0x7f782c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:29.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.401+0000 7f784e400700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f782c06c680 0x7f782c06eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7834006010 tx=0x7f783400b540 comp rx=0 tx=0).stop 2024-03-22T10:38:29.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.401+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481023a0 msgr2=0x7f7848197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.401+0000 7f784e400700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481023a0 0x7f7848197c40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f783800b480 tx=0x7f783800b840 comp rx=0 tx=0).stop 2024-03-22T10:38:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.401+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 shutdown_connections 2024-03-22T10:38:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.401+0000 7f784e400700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f782c06c680 0x7f782c06eb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.401+0000 7f784e400700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78481023a0 0x7f7848197c40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.401+0000 7f784e400700 1 --2- 172.21.15.115:0/3769262634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78481035a0 0x7f7848198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.401+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 >> 172.21.15.115:0/3769262634 conn(0x7f78480fd990 msgr2=0x7f78481067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.402+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 shutdown_connections 2024-03-22T10:38:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:29.402+0000 7f784e400700 1 -- 172.21.15.115:0/3769262634 wait complete. 2024-03-22T10:38:29.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[29107]: 2024-03-22T10:38:29.358+0000 7fc4ae600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:38:30.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-22T10:38:30.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: osdmap e41: 6 total, 6 up, 6 in 2024-03-22T10:38:30.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: pgmap v211: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: osdmap e42: 6 total, 6 up, 6 in 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: osdmap e43: 6 total, 6 up, 6 in 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: fsmap cephfs:0 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: Saving service mds.cephfs spec with placement count:4 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:30.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:30.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:38:30.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:30.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.ggxfzi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:38:30.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.ggxfzi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T10:38:30.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:30.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:29 smithi115 ceph-mon[29111]: Deploying daemon mds.cephfs.smithi115.ggxfzi on smithi115 2024-03-22T10:38:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-22T10:38:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: osdmap e41: 6 total, 6 up, 6 in 2024-03-22T10:38:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-22T10:38:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: pgmap v211: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: osdmap e42: 6 total, 6 up, 6 in 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: osdmap e43: 6 total, 6 up, 6 in 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: fsmap cephfs:0 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: Saving service mds.cephfs spec with placement count:4 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.ggxfzi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.ggxfzi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T10:38:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:30.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:29 smithi150 ceph-mon[34518]: Deploying daemon mds.cephfs.smithi115.ggxfzi on smithi115 2024-03-22T10:38:30.223 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T10:38:31.625 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:31 smithi115 ceph-mon[29111]: osdmap e44: 6 total, 6 up, 6 in 2024-03-22T10:38:31.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:31 smithi150 ceph-mon[34518]: osdmap e44: 6 total, 6 up, 6 in 2024-03-22T10:38:31.662 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:32.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:32 smithi115 ceph-mon[29111]: pgmap v215: 65 pgs: 10 active+clean, 14 creating+peering, 41 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:32.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:32 smithi115 ceph-mon[29111]: osdmap e45: 6 total, 6 up, 6 in 2024-03-22T10:38:32.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:32 smithi150 ceph-mon[34518]: pgmap v215: 65 pgs: 10 active+clean, 14 creating+peering, 41 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:32.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:32 smithi150 ceph-mon[34518]: osdmap e45: 6 total, 6 up, 6 in 2024-03-22T10:38:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.144+0000 7f2227200700 1 -- 172.21.15.115:0/1032524753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220075700 msgr2=0x7f2220075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.144+0000 7f2227200700 1 --2- 172.21.15.115:0/1032524753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220075700 0x7f2220075b10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2210009910 tx=0x7f2210009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.144+0000 7f2227200700 1 -- 172.21.15.115:0/1032524753 shutdown_connections 2024-03-22T10:38:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.144+0000 7f2227200700 1 --2- 172.21.15.115:0/1032524753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2220076950 0x7f2220076dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.144+0000 7f2227200700 1 --2- 172.21.15.115:0/1032524753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220075700 0x7f2220075b10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.144+0000 7f2227200700 1 -- 172.21.15.115:0/1032524753 >> 172.21.15.115:0/1032524753 conn(0x7f22200fd930 msgr2=0x7f22200ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.145+0000 7f2227200700 1 -- 172.21.15.115:0/1032524753 shutdown_connections 2024-03-22T10:38:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.145+0000 7f2227200700 1 -- 172.21.15.115:0/1032524753 wait complete. 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.146+0000 7f2227200700 1 Processor -- start 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.146+0000 7f2227200700 1 -- start start 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.146+0000 7f2227200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2220075700 0x7f222019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.146+0000 7f2227200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220076950 0x7f222019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.146+0000 7f2227200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019caa0 con 0x7f2220076950 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.146+0000 7f2227200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019cbe0 con 0x7f2220075700 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.147+0000 7f221f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220076950 0x7f222019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.147+0000 7f221f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220076950 0x7f222019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48060/0 (socket says 172.21.15.115:48060) 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.147+0000 7f221f400700 1 -- 172.21.15.115:0/2769219471 learned_addr learned my addr 172.21.15.115:0/2769219471 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.147+0000 7f221fe00700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2220075700 0x7f222019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.147+0000 7f221f400700 1 -- 172.21.15.115:0/2769219471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2220075700 msgr2=0x7f222019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.147+0000 7f221f400700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2220075700 0x7f222019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.147+0000 7f221f400700 1 -- 172.21.15.115:0/2769219471 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100095f0 con 0x7f2220076950 2024-03-22T10:38:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.147+0000 7f221f400700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220076950 0x7f222019c480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f220c00d8d0 tx=0x7f220c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.148+0000 7f221cc00700 1 -- 172.21.15.115:0/2769219471 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f220c0098a0 con 0x7f2220076950 2024-03-22T10:38:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.148+0000 7f221cc00700 1 -- 172.21.15.115:0/2769219471 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f220c010460 con 0x7f2220076950 2024-03-22T10:38:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.148+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22201a1690 con 0x7f2220076950 2024-03-22T10:38:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.148+0000 7f221cc00700 1 -- 172.21.15.115:0/2769219471 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f220c00f5f0 con 0x7f2220076950 2024-03-22T10:38:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.148+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22201a1be0 con 0x7f2220076950 2024-03-22T10:38:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.149+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222010f2d0 con 0x7f2220076950 2024-03-22T10:38:34.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.151+0000 7f221cc00700 1 -- 172.21.15.115:0/2769219471 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f220c009a00 con 0x7f2220076950 2024-03-22T10:38:34.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.152+0000 7f221cc00700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f220406c680 0x7f220406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:34.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.152+0000 7f221cc00700 1 -- 172.21.15.115:0/2769219471 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f220c08b6b0 con 0x7f2220076950 2024-03-22T10:38:34.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.153+0000 7f221fe00700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f220406c680 0x7f220406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.153+0000 7f221fe00700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f220406c680 0x7f220406eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f221000b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:34.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.157+0000 7f221cc00700 1 -- 172.21.15.115:0/2769219471 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f220c0598a0 con 0x7f2220076950 2024-03-22T10:38:34.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:34 smithi115 ceph-mon[29111]: pgmap v217: 65 pgs: 40 active+clean, 14 creating+peering, 11 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:34.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:34 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:34.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:34 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:34 smithi150 ceph-mon[34518]: pgmap v217: 65 pgs: 40 active+clean, 14 creating+peering, 11 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:38:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:34 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:34 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.409+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f22201a2270 con 0x7f2220076950 2024-03-22T10:38:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.409+0000 7f221cc00700 1 -- 172.21.15.115:0/2769219471 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1266 (secure 0 0 0) 0x7f220c059430 con 0x7f2220076950 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:e4 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 4 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:38:34.404 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:38:34.022432+0000 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 1 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:in 0 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454} 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:38:34.405 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:38:34.406 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:38:34.406 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:38:34.406 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data disabled 2024-03-22T10:38:34.406 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:38:34.406 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:38:34.406 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 0 2024-03-22T10:38:34.406 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:creating seq 1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:38:34.406 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:38:34.406 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f220406c680 msgr2=0x7f220406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f220406c680 0x7f220406eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f221000b560 comp rx=0 tx=0).stop 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220076950 msgr2=0x7f222019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220076950 0x7f222019c480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f220c00d8d0 tx=0x7f220c00dbe0 comp rx=0 tx=0).stop 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 shutdown_connections 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f220406c680 0x7f220406eb30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2220075700 0x7f222019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 --2- 172.21.15.115:0/2769219471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2220076950 0x7f222019c480 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 >> 172.21.15.115:0/2769219471 conn(0x7f22200fd930 msgr2=0x7f222010c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:34.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 shutdown_connections 2024-03-22T10:38:34.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:34.412+0000 7f2227200700 1 -- 172.21.15.115:0/2769219471 wait complete. 2024-03-22T10:38:34.408 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 4 2024-03-22T10:38:35.129 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] up:boot 2024-03-22T10:38:35.129 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: daemon mds.cephfs.smithi115.ggxfzi assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: Cluster is now healthy 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: fsmap cephfs:0 1 up:standby 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:creating} 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.zvyrzc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.zvyrzc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: Deploying daemon mds.cephfs.smithi150.zvyrzc on smithi150 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: daemon mds.cephfs.smithi115.ggxfzi is now active in filesystem cephfs as rank 0 2024-03-22T10:38:35.130 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:35 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2769219471' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:38:35.210 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] up:boot 2024-03-22T10:38:35.210 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: daemon mds.cephfs.smithi115.ggxfzi assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-22T10:38:35.210 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T10:38:35.210 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T10:38:35.210 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: Cluster is now healthy 2024-03-22T10:38:35.210 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: fsmap cephfs:0 1 up:standby 2024-03-22T10:38:35.210 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:38:35.210 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:creating} 2024-03-22T10:38:35.211 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:35.211 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.zvyrzc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:38:35.211 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.zvyrzc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T10:38:35.211 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:35.211 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: Deploying daemon mds.cephfs.smithi150.zvyrzc on smithi150 2024-03-22T10:38:35.211 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: daemon mds.cephfs.smithi115.ggxfzi is now active in filesystem cephfs as rank 0 2024-03-22T10:38:35.211 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:35 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2769219471' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:38:35.269 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T10:38:35.279 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:38:35.279 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-03-22T10:38:35.691 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:36.146 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:36 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] up:active 2024-03-22T10:38:36.146 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:36 smithi115 ceph-mon[29111]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:active} 2024-03-22T10:38:36.147 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:36 smithi115 ceph-mon[29111]: pgmap v218: 65 pgs: 59 active+clean, 6 creating+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 172 B/s wr, 0 op/s 2024-03-22T10:38:36.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:36 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] up:active 2024-03-22T10:38:36.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:36 smithi150 ceph-mon[34518]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:active} 2024-03-22T10:38:36.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:36 smithi150 ceph-mon[34518]: pgmap v218: 65 pgs: 59 active+clean, 6 creating+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 172 B/s wr, 0 op/s 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.638+0000 7f19df200700 1 -- 172.21.15.115:0/3106794563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 msgr2=0x7f19d8107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.638+0000 7f19df200700 1 --2- 172.21.15.115:0/3106794563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 0x7f19d8107df0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009910 tx=0x7f19c4009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.638+0000 7f19df200700 1 -- 172.21.15.115:0/3106794563 shutdown_connections 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.638+0000 7f19df200700 1 --2- 172.21.15.115:0/3106794563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 0x7f19d8107df0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.638+0000 7f19df200700 1 --2- 172.21.15.115:0/3106794563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19d81067a0 0x7f19d8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.638+0000 7f19df200700 1 -- 172.21.15.115:0/3106794563 >> 172.21.15.115:0/3106794563 conn(0x7f19d8075910 msgr2=0x7f19d8077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.638+0000 7f19df200700 1 -- 172.21.15.115:0/3106794563 shutdown_connections 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.638+0000 7f19df200700 1 -- 172.21.15.115:0/3106794563 wait complete. 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.639+0000 7f19df200700 1 Processor -- start 2024-03-22T10:38:37.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.639+0000 7f19df200700 1 -- start start 2024-03-22T10:38:37.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19df200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19d81067a0 0x7f19d819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:37.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19df200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 0x7f19d819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:37.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19df200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d819cb00 con 0x7f19d81079a0 2024-03-22T10:38:37.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19df200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d819cc40 con 0x7f19d81067a0 2024-03-22T10:38:37.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19d7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 0x7f19d819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:37.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19d7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 0x7f19d819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48084/0 (socket says 172.21.15.115:48084) 2024-03-22T10:38:37.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19d7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19d81067a0 0x7f19d819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:37.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19d7400700 1 -- 172.21.15.115:0/1507809666 learned_addr learned my addr 172.21.15.115:0/1507809666 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:37.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19d7400700 1 -- 172.21.15.115:0/1507809666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19d81067a0 msgr2=0x7f19d819bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:37.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19d7400700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19d81067a0 0x7f19d819bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:37.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.640+0000 7f19d7400700 1 -- 172.21.15.115:0/1507809666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c40095f0 con 0x7f19d81079a0 2024-03-22T10:38:37.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.641+0000 7f19d7400700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 0x7f19d819c4e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f19c4005b40 tx=0x7f19c400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:37.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.641+0000 7f19d4c00700 1 -- 172.21.15.115:0/1507809666 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c401d070 con 0x7f19d81079a0 2024-03-22T10:38:37.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.641+0000 7f19d4c00700 1 -- 172.21.15.115:0/1507809666 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f19c400f460 con 0x7f19d81079a0 2024-03-22T10:38:37.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.641+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d81a1690 con 0x7f19d81079a0 2024-03-22T10:38:37.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.641+0000 7f19d4c00700 1 -- 172.21.15.115:0/1507809666 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c4021620 con 0x7f19d81079a0 2024-03-22T10:38:37.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.641+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d81a1b80 con 0x7f19d81079a0 2024-03-22T10:38:37.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.642+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d810f2f0 con 0x7f19d81079a0 2024-03-22T10:38:37.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.643+0000 7f19d4c00700 1 -- 172.21.15.115:0/1507809666 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f19c400fa40 con 0x7f19d81079a0 2024-03-22T10:38:37.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.644+0000 7f19d4c00700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f19bc06c680 0x7f19bc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:37.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.644+0000 7f19d4c00700 1 -- 172.21.15.115:0/1507809666 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f19c408cbd0 con 0x7f19d81079a0 2024-03-22T10:38:37.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.644+0000 7f19d7e00700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f19bc06c680 0x7f19bc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:37.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.645+0000 7f19d7e00700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f19bc06c680 0x7f19bc06eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19d8107800 tx=0x7f19c800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:37.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.649+0000 7f19d4c00700 1 -- 172.21.15.115:0/1507809666 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f19c405aca0 con 0x7f19d81079a0 2024-03-22T10:38:37.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:37.925+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f19d81a2280 con 0x7f19d81079a0 2024-03-22T10:38:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.011+0000 7f19d4c00700 1 -- 172.21.15.115:0/1507809666 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f19c405a830 con 0x7f19d81079a0 2024-03-22T10:38:38.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f19bc06c680 msgr2=0x7f19bc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:38.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f19bc06c680 0x7f19bc06eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19d8107800 tx=0x7f19c800b410 comp rx=0 tx=0).stop 2024-03-22T10:38:38.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 msgr2=0x7f19d819c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:38.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 0x7f19d819c4e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f19c4005b40 tx=0x7f19c400bd70 comp rx=0 tx=0).stop 2024-03-22T10:38:38.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 shutdown_connections 2024-03-22T10:38:38.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f19bc06c680 0x7f19bc06eb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:38.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19d81067a0 0x7f19d819bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:38.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 --2- 172.21.15.115:0/1507809666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19d81079a0 0x7f19d819c4e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:38.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 >> 172.21.15.115:0/1507809666 conn(0x7f19d8075910 msgr2=0x7f19d810abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:38.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.016+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 shutdown_connections 2024-03-22T10:38:38.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:38.017+0000 7f19df200700 1 -- 172.21.15.115:0/1507809666 wait complete. 2024-03-22T10:38:38.871 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T10:38:38.881 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:38:38.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-03-22T10:38:38.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: pgmap v219: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s wr, 5 op/s 2024-03-22T10:38:38.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.wjotmn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.wjotmn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: Deploying daemon mds.cephfs.smithi115.wjotmn on smithi115 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1507809666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: daemon mds.cephfs.smithi150.zvyrzc assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T10:38:38.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] up:boot 2024-03-22T10:38:38.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1507809666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-22T10:38:38.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:creating} 2024-03-22T10:38:38.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:38:38.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:38 smithi115 ceph-mon[29111]: daemon mds.cephfs.smithi150.zvyrzc is now active in filesystem cephfs as rank 1 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: pgmap v219: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s wr, 5 op/s 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.wjotmn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.wjotmn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: Deploying daemon mds.cephfs.smithi115.wjotmn on smithi115 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1507809666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: daemon mds.cephfs.smithi150.zvyrzc assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T10:38:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T10:38:39.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] up:boot 2024-03-22T10:38:39.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1507809666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-22T10:38:39.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:creating} 2024-03-22T10:38:39.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:38:39.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:38 smithi150 ceph-mon[34518]: daemon mds.cephfs.smithi150.zvyrzc is now active in filesystem cephfs as rank 1 2024-03-22T10:38:40.249 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:40.273 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:40 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] up:active 2024-03-22T10:38:40.273 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:40 smithi115 ceph-mon[29111]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2024-03-22T10:38:40.273 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:40 smithi115 ceph-mon[29111]: pgmap v220: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 6 op/s 2024-03-22T10:38:40.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:40 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] up:active 2024-03-22T10:38:40.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:40 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2024-03-22T10:38:40.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:40 smithi150 ceph-mon[34518]: pgmap v220: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 6 op/s 2024-03-22T10:38:41.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:41.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:38:41.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:41.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:38:42.309 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:42 smithi115 ceph-mon[29111]: pgmap v221: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-03-22T10:38:42.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.621+0000 7fab35200700 1 -- 172.21.15.115:0/906796905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 msgr2=0x7fab301017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:42.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.621+0000 7fab35200700 1 --2- 172.21.15.115:0/906796905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 0x7fab301017a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009910 tx=0x7fab1c009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:42.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.621+0000 7fab35200700 1 -- 172.21.15.115:0/906796905 shutdown_connections 2024-03-22T10:38:42.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.621+0000 7fab35200700 1 --2- 172.21.15.115:0/906796905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 0x7fab301017a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:42.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.621+0000 7fab35200700 1 --2- 172.21.15.115:0/906796905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab30100150 0x7fab30100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:42.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.621+0000 7fab35200700 1 -- 172.21.15.115:0/906796905 >> 172.21.15.115:0/906796905 conn(0x7fab300fb6e0 msgr2=0x7fab300fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:42.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.621+0000 7fab35200700 1 -- 172.21.15.115:0/906796905 shutdown_connections 2024-03-22T10:38:42.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.621+0000 7fab35200700 1 -- 172.21.15.115:0/906796905 wait complete. 2024-03-22T10:38:42.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.622+0000 7fab35200700 1 Processor -- start 2024-03-22T10:38:42.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.622+0000 7fab35200700 1 -- start start 2024-03-22T10:38:42.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.622+0000 7fab35200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab30100150 0x7fab30197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:42.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.622+0000 7fab35200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 0x7fab301980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:42.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.622+0000 7fab35200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab301986d0 con 0x7fab30101350 2024-03-22T10:38:42.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.622+0000 7fab35200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab30198810 con 0x7fab30100150 2024-03-22T10:38:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.623+0000 7fab2d400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 0x7fab301980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.623+0000 7fab2d400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 0x7fab301980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59186/0 (socket says 172.21.15.115:59186) 2024-03-22T10:38:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.623+0000 7fab2d400700 1 -- 172.21.15.115:0/25621016 learned_addr learned my addr 172.21.15.115:0/25621016 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.623+0000 7fab2de00700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab30100150 0x7fab30197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.623+0000 7fab2d400700 1 -- 172.21.15.115:0/25621016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab30100150 msgr2=0x7fab30197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.623+0000 7fab2d400700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab30100150 0x7fab30197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.623+0000 7fab2d400700 1 -- 172.21.15.115:0/25621016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0095f0 con 0x7fab30101350 2024-03-22T10:38:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.623+0000 7fab2d400700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 0x7fab301980b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005b40 tx=0x7fab1c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:42.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.624+0000 7fab26a00700 1 -- 172.21.15.115:0/25621016 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab1c01d070 con 0x7fab30101350 2024-03-22T10:38:42.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.624+0000 7fab26a00700 1 -- 172.21.15.115:0/25621016 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fab1c00f460 con 0x7fab30101350 2024-03-22T10:38:42.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.624+0000 7fab26a00700 1 -- 172.21.15.115:0/25621016 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab1c021620 con 0x7fab30101350 2024-03-22T10:38:42.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.624+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3019d260 con 0x7fab30101350 2024-03-22T10:38:42.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.624+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3019d750 con 0x7fab30101350 2024-03-22T10:38:42.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.625+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab30105430 con 0x7fab30101350 2024-03-22T10:38:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.627+0000 7fab26a00700 1 -- 172.21.15.115:0/25621016 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fab1c00fa40 con 0x7fab30101350 2024-03-22T10:38:42.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.627+0000 7fab26a00700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fab1406c680 0x7fab1406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:42.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.628+0000 7fab26a00700 1 -- 172.21.15.115:0/25621016 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fab1c08cb70 con 0x7fab30101350 2024-03-22T10:38:42.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.628+0000 7fab2de00700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fab1406c680 0x7fab1406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:42.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.628+0000 7fab2de00700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fab1406c680 0x7fab1406eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fab20009550 tx=0x7fab200091b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:42.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.633+0000 7fab26a00700 1 -- 172.21.15.115:0/25621016 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fab1c05acc0 con 0x7fab30101350 2024-03-22T10:38:42.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:42 smithi150 ceph-mon[34518]: pgmap v221: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-03-22T10:38:42.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:42.898+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fab3002d0c0 con 0x7fab30101350 2024-03-22T10:38:43.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-22T10:38:43.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: Cluster is now healthy 2024-03-22T10:38:43.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:boot 2024-03-22T10:38:43.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 1 up:standby 2024-03-22T10:38:43.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:38:43.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:43.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:43.526 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:43.526 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.slzwwb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:38:43.526 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.slzwwb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T10:38:43.526 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:43.526 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: Deploying daemon mds.cephfs.smithi150.slzwwb on smithi150 2024-03-22T10:38:43.526 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:43 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/25621016' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-22T10:38:43.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.540+0000 7fab26a00700 1 -- 172.21.15.115:0/25621016 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab1c05a850 con 0x7fab30101350 2024-03-22T10:38:43.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.543+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fab1406c680 msgr2=0x7fab1406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:43.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.543+0000 7fab35200700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fab1406c680 0x7fab1406eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fab20009550 tx=0x7fab200091b0 comp rx=0 tx=0).stop 2024-03-22T10:38:43.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.543+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 msgr2=0x7fab301980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:43.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.543+0000 7fab35200700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 0x7fab301980b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005b40 tx=0x7fab1c00bd70 comp rx=0 tx=0).stop 2024-03-22T10:38:43.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.544+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 shutdown_connections 2024-03-22T10:38:43.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.544+0000 7fab35200700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fab1406c680 0x7fab1406eb30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:43.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.544+0000 7fab35200700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab30100150 0x7fab30197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:43.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.544+0000 7fab35200700 1 --2- 172.21.15.115:0/25621016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab30101350 0x7fab301980b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:43.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.544+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 >> 172.21.15.115:0/25621016 conn(0x7fab300fb6e0 msgr2=0x7fab30102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:43.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.544+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 shutdown_connections 2024-03-22T10:38:43.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:43.544+0000 7fab35200700 1 -- 172.21.15.115:0/25621016 wait complete. 2024-03-22T10:38:43.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-22T10:38:43.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: Cluster is now healthy 2024-03-22T10:38:43.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:boot 2024-03-22T10:38:43.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 1 up:standby 2024-03-22T10:38:43.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:38:43.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:43.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:43.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:43.550 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.slzwwb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:38:43.550 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.slzwwb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T10:38:43.550 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:43.550 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: Deploying daemon mds.cephfs.smithi150.slzwwb on smithi150 2024-03-22T10:38:43.550 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:43 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/25621016' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-22T10:38:44.302 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T10:38:44.313 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:38:44.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-03-22T10:38:44.714 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:44.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:44 smithi115 ceph-mon[29111]: pgmap v222: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-22T10:38:44.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:44 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/25621016' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-22T10:38:44.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:44 smithi115 ceph-mon[29111]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 1 up:standby 2024-03-22T10:38:44.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:44 smithi150 ceph-mon[34518]: pgmap v222: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-22T10:38:44.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:44 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/25621016' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-22T10:38:44.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:44 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 1 up:standby 2024-03-22T10:38:46.426 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:46 smithi115 ceph-mon[29111]: pgmap v223: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-03-22T10:38:46.426 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.426 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.426 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.426 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.426 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.426 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:46 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:46 smithi150 ceph-mon[34518]: pgmap v223: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-03-22T10:38:46.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:46 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:46 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:46 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:46 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:46 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:46 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:46.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.905+0000 7f62b4c00700 1 -- 172.21.15.115:0/3578773453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 msgr2=0x7f62b0073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.905+0000 7f62b4c00700 1 --2- 172.21.15.115:0/3578773453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 0x7f62b0073bc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f629c009910 tx=0x7f629c009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.905+0000 7f62b4c00700 1 -- 172.21.15.115:0/3578773453 shutdown_connections 2024-03-22T10:38:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.905+0000 7f62b4c00700 1 --2- 172.21.15.115:0/3578773453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 0x7f62b0073bc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.905+0000 7f62b4c00700 1 --2- 172.21.15.115:0/3578773453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b0074d20 0x7f62b0073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.905+0000 7f62b4c00700 1 -- 172.21.15.115:0/3578773453 >> 172.21.15.115:0/3578773453 conn(0x7f62b00fb6e0 msgr2=0x7f62b00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.905+0000 7f62b4c00700 1 -- 172.21.15.115:0/3578773453 shutdown_connections 2024-03-22T10:38:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.905+0000 7f62b4c00700 1 -- 172.21.15.115:0/3578773453 wait complete. 2024-03-22T10:38:46.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.906+0000 7f62b4c00700 1 Processor -- start 2024-03-22T10:38:46.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.906+0000 7f62b4c00700 1 -- start start 2024-03-22T10:38:46.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62b4c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 0x7f62b0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:46.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62b4c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b0074d20 0x7f62b0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:46.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62b4c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b01986b0 con 0x7f62b0073750 2024-03-22T10:38:46.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62b4c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b01987f0 con 0x7f62b0074d20 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62ade00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 0x7f62b0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62ade00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 0x7f62b0197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59206/0 (socket says 172.21.15.115:59206) 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62ade00700 1 -- 172.21.15.115:0/3758699054 learned_addr learned my addr 172.21.15.115:0/3758699054 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62ad400700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b0074d20 0x7f62b0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62ade00700 1 -- 172.21.15.115:0/3758699054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b0074d20 msgr2=0x7f62b0198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62ade00700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b0074d20 0x7f62b0198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.907+0000 7f62ade00700 1 -- 172.21.15.115:0/3758699054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0095f0 con 0x7f62b0073750 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.908+0000 7f62ade00700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 0x7f62b0197b50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f62a000d8d0 tx=0x7f62a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.908+0000 7f62a6a00700 1 -- 172.21.15.115:0/3758699054 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a0009940 con 0x7f62b0073750 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.908+0000 7f62a6a00700 1 -- 172.21.15.115:0/3758699054 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f62a0010460 con 0x7f62b0073750 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.908+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b019d2a0 con 0x7f62b0073750 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.908+0000 7f62a6a00700 1 -- 172.21.15.115:0/3758699054 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a000f5f0 con 0x7f62b0073750 2024-03-22T10:38:46.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.908+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b019d7f0 con 0x7f62b0073750 2024-03-22T10:38:46.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.909+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b010ae90 con 0x7f62b0073750 2024-03-22T10:38:46.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.910+0000 7f62a6a00700 1 -- 172.21.15.115:0/3758699054 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f62a0009aa0 con 0x7f62b0073750 2024-03-22T10:38:46.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.911+0000 7f62a6a00700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f629406c680 0x7f629406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:46.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.911+0000 7f62a6a00700 1 -- 172.21.15.115:0/3758699054 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f62a008b8c0 con 0x7f62b0073750 2024-03-22T10:38:46.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.912+0000 7f62ad400700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f629406c680 0x7f629406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:46.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.912+0000 7f62ad400700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f629406c680 0x7f629406eb30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f629c006010 tx=0x7f629c00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:46.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:46.916+0000 7f62a6a00700 1 -- 172.21.15.115:0/3758699054 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f62a0059a30 con 0x7f62b0073750 2024-03-22T10:38:47.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.189+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f62b010b080 con 0x7f62b0073750 2024-03-22T10:38:47.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:47 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:47.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:47 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:boot 2024-03-22T10:38:47.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:47 smithi115 ceph-mon[29111]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:38:47.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:47 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:38:47.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:47 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3758699054' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T10:38:47.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.558+0000 7f62a6a00700 1 -- 172.21.15.115:0/3758699054 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v11) v1 ==== 168+0+0 (secure 0 0 0) 0x7f62b010b080 con 0x7f62b0073750 2024-03-22T10:38:47.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.562+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f629406c680 msgr2=0x7f629406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:47.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.562+0000 7f62b4c00700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f629406c680 0x7f629406eb30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f629c006010 tx=0x7f629c00b540 comp rx=0 tx=0).stop 2024-03-22T10:38:47.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.562+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 msgr2=0x7f62b0197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:47.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.562+0000 7f62b4c00700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 0x7f62b0197b50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f62a000d8d0 tx=0x7f62a000dc90 comp rx=0 tx=0).stop 2024-03-22T10:38:47.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.563+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 shutdown_connections 2024-03-22T10:38:47.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.563+0000 7f62b4c00700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f629406c680 0x7f629406eb30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:47.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.563+0000 7f62b4c00700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b0073750 0x7f62b0197b50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:47.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.563+0000 7f62b4c00700 1 --2- 172.21.15.115:0/3758699054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b0074d20 0x7f62b0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:47.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.563+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 >> 172.21.15.115:0/3758699054 conn(0x7f62b00fb6e0 msgr2=0x7f62b0105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:47.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.563+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 shutdown_connections 2024-03-22T10:38:47.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:47.563+0000 7f62b4c00700 1 -- 172.21.15.115:0/3758699054 wait complete. 2024-03-22T10:38:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:inline data enabled 2024-03-22T10:38:47.620 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:47 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:47.620 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:47 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:boot 2024-03-22T10:38:47.620 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:47 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:38:47.620 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:47 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:38:47.620 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:47 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3758699054' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-22T10:38:48.614 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:48 smithi115 ceph-mon[29111]: pgmap v224: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-03-22T10:38:48.614 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:48 smithi115 ceph-mon[29111]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-03-22T10:38:48.614 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:48 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3758699054' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-03-22T10:38:48.614 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:48 smithi115 ceph-mon[29111]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:38:48.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:48 smithi150 ceph-mon[34518]: pgmap v224: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-03-22T10:38:48.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:48 smithi150 ceph-mon[34518]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-03-22T10:38:48.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:48 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3758699054' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-03-22T10:38:48.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:48 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:38:48.787 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T10:38:48.799 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:38:48.799 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T10:38:49.705 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:50.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:50 smithi150 ceph-mon[34518]: pgmap v225: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 3 op/s 2024-03-22T10:38:50.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:50 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:50.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:50 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:50.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:50 smithi115 ceph-mon[29111]: pgmap v225: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 3 op/s 2024-03-22T10:38:50.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:50 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:50.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:50 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:51.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:51 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:51.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:51 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:standby 2024-03-22T10:38:51.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:51 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] up:active 2024-03-22T10:38:51.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:51 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] up:active 2024-03-22T10:38:51.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:51 smithi150 ceph-mon[34518]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:standby 2024-03-22T10:38:51.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:51 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:38:51.750 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:51 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:51.750 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:51 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:standby 2024-03-22T10:38:51.750 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:51 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] up:active 2024-03-22T10:38:51.751 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:51 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] up:active 2024-03-22T10:38:51.751 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:51 smithi115 ceph-mon[29111]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:standby 2024-03-22T10:38:51.751 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:51 smithi115 ceph-mon[29111]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:38:52.634 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:52 smithi115 ceph-mon[29111]: pgmap v226: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-03-22T10:38:52.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:52 smithi150 ceph-mon[34518]: pgmap v226: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-03-22T10:38:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.795+0000 7f041a800700 1 -- 172.21.15.115:0/1109290088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414101350 msgr2=0x7f04141017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.795+0000 7f041a800700 1 --2- 172.21.15.115:0/1109290088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414101350 0x7f04141017a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0400009910 tx=0x7f0400009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.795+0000 7f041a800700 1 -- 172.21.15.115:0/1109290088 shutdown_connections 2024-03-22T10:38:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.795+0000 7f041a800700 1 --2- 172.21.15.115:0/1109290088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414101350 0x7f04141017a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.795+0000 7f041a800700 1 --2- 172.21.15.115:0/1109290088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0414100150 0x7f0414100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.795+0000 7f041a800700 1 -- 172.21.15.115:0/1109290088 >> 172.21.15.115:0/1109290088 conn(0x7f04140fb6e0 msgr2=0x7f04140fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.795+0000 7f041a800700 1 -- 172.21.15.115:0/1109290088 shutdown_connections 2024-03-22T10:38:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.795+0000 7f041a800700 1 -- 172.21.15.115:0/1109290088 wait complete. 2024-03-22T10:38:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.796+0000 7f041a800700 1 Processor -- start 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.796+0000 7f041a800700 1 -- start start 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f041a800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414100150 0x7f0414197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f041a800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0414101350 0x7f04141980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f041a800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04141986d0 con 0x7f0414100150 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f041a800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0414198810 con 0x7f0414101350 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f0413400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414100150 0x7f0414197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f0413400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414100150 0x7f0414197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42512/0 (socket says 172.21.15.115:42512) 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f0413400700 1 -- 172.21.15.115:0/1803426969 learned_addr learned my addr 172.21.15.115:0/1803426969 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f0412a00700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0414101350 0x7f04141980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f0413400700 1 -- 172.21.15.115:0/1803426969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0414101350 msgr2=0x7f04141980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.797+0000 7f0413400700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0414101350 0x7f04141980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:53.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.798+0000 7f0413400700 1 -- 172.21.15.115:0/1803426969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04000095f0 con 0x7f0414100150 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.798+0000 7f0413400700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414100150 0x7f0414197b70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f040400d900 tx=0x7f040400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.798+0000 7f040be00700 1 -- 172.21.15.115:0/1803426969 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04040041f0 con 0x7f0414100150 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.798+0000 7f040be00700 1 -- 172.21.15.115:0/1803426969 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0404004350 con 0x7f0414100150 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.798+0000 7f040be00700 1 -- 172.21.15.115:0/1803426969 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0404003da0 con 0x7f0414100150 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.798+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041419d2c0 con 0x7f0414100150 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.798+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041419d810 con 0x7f0414100150 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.799+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0414105430 con 0x7f0414100150 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.800+0000 7f040be00700 1 -- 172.21.15.115:0/1803426969 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f0404009730 con 0x7f0414100150 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.801+0000 7f040be00700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f03f806c680 0x7f03f806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.801+0000 7f040be00700 1 -- 172.21.15.115:0/1803426969 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f040408ba00 con 0x7f0414100150 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.802+0000 7f0412a00700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f03f806c680 0x7f03f806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:53.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.802+0000 7f0412a00700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f03f806c680 0x7f03f806eb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0400006010 tx=0x7f040000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:53.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:52.806+0000 7f040be00700 1 -- 172.21.15.115:0/1803426969 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0404059b50 con 0x7f0414100150 2024-03-22T10:38:53.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.072+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0414061a00 con 0x7f0414100150 2024-03-22T10:38:53.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.073+0000 7f040be00700 1 -- 172.21.15.115:0/1803426969 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1839 (secure 0 0 0) 0x7f04040596e0 con 0x7f0414100150 2024-03-22T10:38:53.068 INFO:teuthology.orchestra.run.smithi115.stdout:e12 2024-03-22T10:38:53.068 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:38:53.068 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:38:53.068 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 12 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:38:50.921733+0000 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:38:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:38:53.070 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:38:53.071 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:38:53.071 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:38:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:38:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:38:53.071 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:38:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:38:53.071 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:38:53.071 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f03f806c680 msgr2=0x7f03f806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f03f806c680 0x7f03f806eb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0400006010 tx=0x7f040000b540 comp rx=0 tx=0).stop 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414100150 msgr2=0x7f0414197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414100150 0x7f0414197b70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f040400d900 tx=0x7f040400dcc0 comp rx=0 tx=0).stop 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 shutdown_connections 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f03f806c680 0x7f03f806eb30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0414100150 0x7f0414197b70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 --2- 172.21.15.115:0/1803426969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0414101350 0x7f04141980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 >> 172.21.15.115:0/1803426969 conn(0x7f04140fb6e0 msgr2=0x7f0414102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 shutdown_connections 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:53.076+0000 7f041a800700 1 -- 172.21.15.115:0/1803426969 wait complete. 2024-03-22T10:38:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 12 2024-03-22T10:38:54.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:54 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1803426969' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:38:54.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:54 smithi115 ceph-mon[29111]: pgmap v227: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 1 op/s 2024-03-22T10:38:54.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:54 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1803426969' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:38:54.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:54 smithi150 ceph-mon[34518]: pgmap v227: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 1 op/s 2024-03-22T10:38:54.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-22T10:38:54.873 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:38:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:38:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:55 smithi115 ceph-mon[29111]: pgmap v228: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-03-22T10:38:56.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:56.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:56.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:56.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:38:56.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:38:56.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:56.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:56.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:38:56.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:55 smithi150 ceph-mon[34518]: pgmap v228: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-03-22T10:38:56.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:38:56.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.793+0000 7f4e55c00700 1 -- 172.21.15.115:0/3071447860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 msgr2=0x7f4e50107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:56.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.793+0000 7f4e55c00700 1 --2- 172.21.15.115:0/3071447860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 0x7f4e50107df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009910 tx=0x7f4e38009c20 comp rx=0 tx=0).stop 2024-03-22T10:38:56.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.793+0000 7f4e55c00700 1 -- 172.21.15.115:0/3071447860 shutdown_connections 2024-03-22T10:38:56.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.793+0000 7f4e55c00700 1 --2- 172.21.15.115:0/3071447860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 0x7f4e50107df0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:56.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.793+0000 7f4e55c00700 1 --2- 172.21.15.115:0/3071447860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e501067a0 0x7f4e50106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:56.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.793+0000 7f4e55c00700 1 -- 172.21.15.115:0/3071447860 >> 172.21.15.115:0/3071447860 conn(0x7f4e50075910 msgr2=0x7f4e50077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:56.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.793+0000 7f4e55c00700 1 -- 172.21.15.115:0/3071447860 shutdown_connections 2024-03-22T10:38:56.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.793+0000 7f4e55c00700 1 -- 172.21.15.115:0/3071447860 wait complete. 2024-03-22T10:38:56.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.794+0000 7f4e55c00700 1 Processor -- start 2024-03-22T10:38:56.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.794+0000 7f4e55c00700 1 -- start start 2024-03-22T10:38:56.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.795+0000 7f4e55c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e501067a0 0x7f4e5019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:56.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.795+0000 7f4e55c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 0x7f4e5019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:56.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.795+0000 7f4e55c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5019cb70 con 0x7f4e501079a0 2024-03-22T10:38:56.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.795+0000 7f4e55c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5019ccb0 con 0x7f4e501067a0 2024-03-22T10:38:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.795+0000 7f4e47e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 0x7f4e5019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.795+0000 7f4e47e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 0x7f4e5019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42526/0 (socket says 172.21.15.115:42526) 2024-03-22T10:38:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.795+0000 7f4e47e00700 1 -- 172.21.15.115:0/1178725353 learned_addr learned my addr 172.21.15.115:0/1178725353 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:38:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.796+0000 7f4e4ea00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e501067a0 0x7f4e5019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.796+0000 7f4e47e00700 1 -- 172.21.15.115:0/1178725353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e501067a0 msgr2=0x7f4e5019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.796+0000 7f4e47e00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e501067a0 0x7f4e5019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.796+0000 7f4e47e00700 1 -- 172.21.15.115:0/1178725353 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380095f0 con 0x7f4e501079a0 2024-03-22T10:38:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.796+0000 7f4e47e00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 0x7f4e5019c550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800b5c0 tx=0x7f4e3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:56.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.796+0000 7f4e47400700 1 -- 172.21.15.115:0/1178725353 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e3801d070 con 0x7f4e501079a0 2024-03-22T10:38:56.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.796+0000 7f4e47400700 1 -- 172.21.15.115:0/1178725353 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4e38022470 con 0x7f4e501079a0 2024-03-22T10:38:56.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.796+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e501a1700 con 0x7f4e501079a0 2024-03-22T10:38:56.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.796+0000 7f4e47400700 1 -- 172.21.15.115:0/1178725353 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e3800f690 con 0x7f4e501079a0 2024-03-22T10:38:56.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.797+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e501a1c70 con 0x7f4e501079a0 2024-03-22T10:38:56.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.798+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5010f390 con 0x7f4e501079a0 2024-03-22T10:38:56.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.799+0000 7f4e47400700 1 -- 172.21.15.115:0/1178725353 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f4e38004b70 con 0x7f4e501079a0 2024-03-22T10:38:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.799+0000 7f4e47400700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4e3406c680 0x7f4e3406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:38:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.800+0000 7f4e47400700 1 -- 172.21.15.115:0/1178725353 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4e3808d070 con 0x7f4e501079a0 2024-03-22T10:38:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.800+0000 7f4e4ea00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4e3406c680 0x7f4e3406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:38:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.800+0000 7f4e4ea00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4e3406c680 0x7f4e3406eb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e40006fd0 tx=0x7f4e40008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:38:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:56.804+0000 7f4e47400700 1 -- 172.21.15.115:0/1178725353 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4e3805b1f0 con 0x7f4e501079a0 2024-03-22T10:38:57.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.070+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e5010f580 con 0x7f4e501079a0 2024-03-22T10:38:57.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.071+0000 7f4e47400700 1 -- 172.21.15.115:0/1178725353 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4764 (secure 0 0 0) 0x7f4e3805ad80 con 0x7f4e501079a0 2024-03-22T10:38:57.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.073+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4e3406c680 msgr2=0x7f4e3406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:57.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.073+0000 7f4e55c00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4e3406c680 0x7f4e3406eb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e40006fd0 tx=0x7f4e40008040 comp rx=0 tx=0).stop 2024-03-22T10:38:57.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.073+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 msgr2=0x7f4e5019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:38:57.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.073+0000 7f4e55c00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 0x7f4e5019c550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800b5c0 tx=0x7f4e3800be20 comp rx=0 tx=0).stop 2024-03-22T10:38:57.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.074+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 shutdown_connections 2024-03-22T10:38:57.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.074+0000 7f4e55c00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4e3406c680 0x7f4e3406eb30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:57.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.074+0000 7f4e55c00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e501067a0 0x7f4e5019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:57.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.074+0000 7f4e55c00700 1 --2- 172.21.15.115:0/1178725353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e501079a0 0x7f4e5019c550 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:38:57.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.074+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 >> 172.21.15.115:0/1178725353 conn(0x7f4e50075910 msgr2=0x7f4e5010abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:38:57.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.074+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 shutdown_connections 2024-03-22T10:38:57.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:38:57.074+0000 7f4e55c00700 1 -- 172.21.15.115:0/1178725353 wait complete. 2024-03-22T10:38:57.067 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 12 2024-03-22T10:38:57.082 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:38:57.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:57 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1178725353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T10:38:57.639 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:57 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1178725353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T10:38:58.098 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-22T10:38:58.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:58 smithi115 ceph-mon[29111]: pgmap v229: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-03-22T10:38:58.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:58 smithi150 ceph-mon[34518]: pgmap v229: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-03-22T10:38:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:38:59.935 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:59 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:38:59.935 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:38:59 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:00.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:59 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:00.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:38:59 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:00.945 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:00 smithi115 ceph-mon[29111]: pgmap v230: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:01.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:00 smithi150 ceph-mon[34518]: pgmap v230: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:02.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.354+0000 7fedcac00700 1 -- 172.21.15.115:0/3207205488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 msgr2=0x7fedc4107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.354+0000 7fedcac00700 1 --2- 172.21.15.115:0/3207205488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 0x7fedc4107df0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009910 tx=0x7fedb0009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.354+0000 7fedcac00700 1 -- 172.21.15.115:0/3207205488 shutdown_connections 2024-03-22T10:39:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.354+0000 7fedcac00700 1 --2- 172.21.15.115:0/3207205488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 0x7fedc4107df0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.354+0000 7fedcac00700 1 --2- 172.21.15.115:0/3207205488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc41067a0 0x7fedc4106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.354+0000 7fedcac00700 1 -- 172.21.15.115:0/3207205488 >> 172.21.15.115:0/3207205488 conn(0x7fedc4075910 msgr2=0x7fedc4077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.354+0000 7fedcac00700 1 -- 172.21.15.115:0/3207205488 shutdown_connections 2024-03-22T10:39:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.354+0000 7fedcac00700 1 -- 172.21.15.115:0/3207205488 wait complete. 2024-03-22T10:39:02.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.355+0000 7fedcac00700 1 Processor -- start 2024-03-22T10:39:02.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.355+0000 7fedcac00700 1 -- start start 2024-03-22T10:39:02.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedcac00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc41067a0 0x7fedc419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:02.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedcac00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 0x7fedc419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedcac00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc419ca70 con 0x7fedc41079a0 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedcac00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc419cbb0 con 0x7fedc41067a0 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedc3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 0x7fedc419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedc3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 0x7fedc419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60908/0 (socket says 172.21.15.115:60908) 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedc3400700 1 -- 172.21.15.115:0/83010781 learned_addr learned my addr 172.21.15.115:0/83010781 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedc3e00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc41067a0 0x7fedc419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedc3400700 1 -- 172.21.15.115:0/83010781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc41067a0 msgr2=0x7fedc419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedc3400700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc41067a0 0x7fedc419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedc3400700 1 -- 172.21.15.115:0/83010781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb00095f0 con 0x7fedc41079a0 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.356+0000 7fedc3e00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc41067a0 0x7fedc419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:39:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.357+0000 7fedc3400700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 0x7fedc419c4e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fedb0005b40 tx=0x7fedb000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:02.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.357+0000 7fedc0c00700 1 -- 172.21.15.115:0/83010781 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedb001d070 con 0x7fedc41079a0 2024-03-22T10:39:02.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.357+0000 7fedc0c00700 1 -- 172.21.15.115:0/83010781 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fedb0022470 con 0x7fedc41079a0 2024-03-22T10:39:02.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.357+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc41a1610 con 0x7fedc41079a0 2024-03-22T10:39:02.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.357+0000 7fedc0c00700 1 -- 172.21.15.115:0/83010781 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedb000f690 con 0x7fedc41079a0 2024-03-22T10:39:02.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.357+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc41a1ad0 con 0x7fedc41079a0 2024-03-22T10:39:02.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.358+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc410f2f0 con 0x7fedc41079a0 2024-03-22T10:39:02.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.359+0000 7fedc0c00700 1 -- 172.21.15.115:0/83010781 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fedb00225e0 con 0x7fedc41079a0 2024-03-22T10:39:02.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.360+0000 7fedc0c00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7feda806c680 0x7feda806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:02.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.360+0000 7fedc0c00700 1 -- 172.21.15.115:0/83010781 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fedb008d090 con 0x7fedc41079a0 2024-03-22T10:39:02.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.360+0000 7fedc3e00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7feda806c680 0x7feda806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:02.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.361+0000 7fedc3e00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7feda806c680 0x7feda806eb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fedb4005fd0 tx=0x7fedb4005e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:02.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.365+0000 7fedc0c00700 1 -- 172.21.15.115:0/83010781 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fedb005b260 con 0x7fedc41079a0 2024-03-22T10:39:02.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.675+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fedc410f4b0 con 0x7fedc41079a0 2024-03-22T10:39:02.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.675+0000 7fedc0c00700 1 -- 172.21.15.115:0/83010781 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v12) v1 ==== 78+0+83 (secure 0 0 0) 0x7fedb005adf0 con 0x7fedc41079a0 2024-03-22T10:39:02.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.678+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7feda806c680 msgr2=0x7feda806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:02.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.678+0000 7fedcac00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7feda806c680 0x7feda806eb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fedb4005fd0 tx=0x7fedb4005e40 comp rx=0 tx=0).stop 2024-03-22T10:39:02.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.678+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 msgr2=0x7fedc419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:02.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.678+0000 7fedcac00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 0x7fedc419c4e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fedb0005b40 tx=0x7fedb000be20 comp rx=0 tx=0).stop 2024-03-22T10:39:02.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.678+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 shutdown_connections 2024-03-22T10:39:02.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.678+0000 7fedcac00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7feda806c680 0x7feda806eb30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:02.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.678+0000 7fedcac00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc41067a0 0x7fedc419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:02.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.678+0000 7fedcac00700 1 --2- 172.21.15.115:0/83010781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc41079a0 0x7fedc419c4e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:02.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.678+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 >> 172.21.15.115:0/83010781 conn(0x7fedc4075910 msgr2=0x7fedc410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:02.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.679+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 shutdown_connections 2024-03-22T10:39:02.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:02.679+0000 7fedcac00700 1 -- 172.21.15.115:0/83010781 wait complete. 2024-03-22T10:39:02.685 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:39:03.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:02 smithi115 ceph-mon[29111]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:03.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:02 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/83010781' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-22T10:39:03.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:02 smithi150 ceph-mon[34518]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:03.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:02 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/83010781' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-22T10:39:03.799 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-22T10:39:03.813 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T10:39:04.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.033+0000 7fb5bd600640 1 -- 172.21.15.115:0/3251315894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 msgr2=0x7fb5b80ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.033+0000 7fb5bd600640 1 --2- 172.21.15.115:0/3251315894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 0x7fb5b80ffa50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009880 tx=0x7fb5a802bcf0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.034+0000 7fb5bd600640 1 -- 172.21.15.115:0/3251315894 shutdown_connections 2024-03-22T10:39:04.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.034+0000 7fb5bd600640 1 --2- 172.21.15.115:0/3251315894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b8100830 0x7fb5b8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.034+0000 7fb5bd600640 1 --2- 172.21.15.115:0/3251315894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 0x7fb5b80ffa50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.034+0000 7fb5bd600640 1 -- 172.21.15.115:0/3251315894 >> 172.21.15.115:0/3251315894 conn(0x7fb5b80fabc0 msgr2=0x7fb5b80fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:04.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.034+0000 7fb5bd600640 1 -- 172.21.15.115:0/3251315894 shutdown_connections 2024-03-22T10:39:04.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.034+0000 7fb5bd600640 1 -- 172.21.15.115:0/3251315894 wait complete. 2024-03-22T10:39:04.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.035+0000 7fb5bd600640 1 Processor -- start 2024-03-22T10:39:04.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.035+0000 7fb5bd600640 1 -- start start 2024-03-22T10:39:04.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.035+0000 7fb5bd600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 0x7fb5b8197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.035+0000 7fb5bd600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b8100830 0x7fb5b81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.035+0000 7fb5bd600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b8197b00 con 0x7fb5b80ff640 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.035+0000 7fb5bd600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b8197c70 con 0x7fb5b8100830 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.035+0000 7fb5b6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 0x7fb5b8197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.035+0000 7fb5b6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 0x7fb5b8197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60928/0 (socket says 172.21.15.115:60928) 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.035+0000 7fb5b6a00640 1 -- 172.21.15.115:0/2908691972 learned_addr learned my addr 172.21.15.115:0/2908691972 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.036+0000 7fb5b6000640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b8100830 0x7fb5b81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.036+0000 7fb5b6a00640 1 -- 172.21.15.115:0/2908691972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b8100830 msgr2=0x7fb5b81975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.036+0000 7fb5b6a00640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b8100830 0x7fb5b81975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.036+0000 7fb5b6a00640 1 -- 172.21.15.115:0/2908691972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a8009560 con 0x7fb5b80ff640 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.036+0000 7fb5b6a00640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 0x7fb5b8197080 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8036040 tx=0x7fb5a8004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.036+0000 7fb5af400640 1 -- 172.21.15.115:0/2908691972 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a8005000 con 0x7fb5b80ff640 2024-03-22T10:39:04.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.037+0000 7fb5af400640 1 -- 172.21.15.115:0/2908691972 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb5a8005610 con 0x7fb5b80ff640 2024-03-22T10:39:04.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.037+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b819c6e0 con 0x7fb5b80ff640 2024-03-22T10:39:04.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.037+0000 7fb5af400640 1 -- 172.21.15.115:0/2908691972 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a8003ea0 con 0x7fb5b80ff640 2024-03-22T10:39:04.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.037+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b819cc10 con 0x7fb5b80ff640 2024-03-22T10:39:04.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.038+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b8104880 con 0x7fb5b80ff640 2024-03-22T10:39:04.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.039+0000 7fb5af400640 1 -- 172.21.15.115:0/2908691972 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb5a80035f0 con 0x7fb5b80ff640 2024-03-22T10:39:04.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.040+0000 7fb5af400640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb59c06c6b0 0x7fb59c06eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:04.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.040+0000 7fb5af400640 1 -- 172.21.15.115:0/2908691972 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb5a80adf60 con 0x7fb5b80ff640 2024-03-22T10:39:04.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.040+0000 7fb5b6000640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb59c06c6b0 0x7fb59c06eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:04.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.041+0000 7fb5b6000640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb59c06c6b0 0x7fb59c06eb60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4005e10 tx=0x7fb5a4005da0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:04.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.047+0000 7fb5af400640 1 -- 172.21.15.115:0/2908691972 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb5a80785a0 con 0x7fb5b80ff640 2024-03-22T10:39:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.259+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5b8108200 con 0x7fb5b80ff640 2024-03-22T10:39:04.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.260+0000 7fb5af400640 1 -- 172.21.15.115:0/2908691972 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15012 (secure 0 0 0) 0x7fb5a807bbc0 con 0x7fb5b80ff640 2024-03-22T10:39:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:39:04.252 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":45,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","created":"2024-03-22T10:29:37.233959+0000","modified":"2024-03-22T10:38:31.373877+0000","last_up_change":"2024-03-22T10:36:58.621026+0000","last_in_change":"2024-03-22T10:36:35.075110+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T10:35:17.870309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T10:38:27.996809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T10:38:28.878933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a34c0f70-4e9d-4044-b7b4-b6099b74ba6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6803","nonce":764585829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6805","nonce":764585829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6809","nonce":764585829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":764585829},{"type":"v1","addr":"172.21.15.115:6807","nonce":764585829}]},"public_addr":"172.21.15.115:6803/764585829","cluster_addr":"172.21.15.115:6805/764585829","heartbeat_back_addr":"172.21.15.115:6809/764585829","heartbeat_front_addr":"172.21.15.115:6807/764585829","state":["exists","up"]},{"osd":1,"uuid":"1037db4c-35bc-48da-8950-460c772d50b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6811","nonce":586118879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6813","nonce":586118879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6817","nonce":586118879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":586118879},{"type":"v1","addr":"172.21.15.115:6815","nonce":586118879}]},"public_addr":"172.21.15.115:6811/586118879","cluster_addr":"172.21.15.115:6813/586118879","heartbeat_back_addr":"172.21.15.115:6817/586118879","heartbeat_front_addr":"172.21.15.115:6815/586118879","state":["exists","up"]},{"osd":2,"uuid":"4e685a22-30ee-4404-bba5-ff944af9b858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6819","nonce":285410766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6821","nonce":285410766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6825","nonce":285410766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":285410766},{"type":"v1","addr":"172.21.15.115:6823","nonce":285410766}]},"public_addr":"172.21.15.115:6819/285410766","cluster_addr":"172.21.15.115:6821/285410766","heartbeat_back_addr":"172.21.15.115:6825/285410766","heartbeat_front_addr":"172.21.15.115:6823/285410766","state":["exists","up"]},{"osd":3,"uuid":"1273664e-16c8-46c1-bfb4-c51e3e84ea6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6801","nonce":3913650733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6803","nonce":3913650733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6807","nonce":3913650733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3913650733},{"type":"v1","addr":"172.21.15.150:6805","nonce":3913650733}]},"public_addr":"172.21.15.150:6801/3913650733","cluster_addr":"172.21.15.150:6803/3913650733","heartbeat_back_addr":"172.21.15.150:6807/3913650733","heartbeat_front_addr":"172.21.15.150:6805/3913650733","state":["exists","up"]},{"osd":4,"uuid":"574fd0d5-c58a-4e16-961d-b384e52d82b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6809","nonce":792436265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6811","nonce":792436265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6815","nonce":792436265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":792436265},{"type":"v1","addr":"172.21.15.150:6813","nonce":792436265}]},"public_addr":"172.21.15.150:6809/792436265","cluster_addr":"172.21.15.150:6811/792436265","heartbeat_back_addr":"172.21.15.150:6815/792436265","heartbeat_front_addr":"172.21.15.150:6813/792436265","state":["exists","up"]},{"osd":5,"uuid":"cc37270e-923d-45cf-accd-d4119dcc1a7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6817","nonce":2178668917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6819","nonce":2178668917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6823","nonce":2178668917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2178668917},{"type":"v1","addr":"172.21.15.150:6821","nonce":2178668917}]},"public_addr":"172.21.15.150:6817/2178668917","cluster_addr":"172.21.15.150:6819/2178668917","heartbeat_back_addr":"172.21.15.150:6823/2178668917","heartbeat_front_addr":"172.21.15.150:6821/2178668917","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:04.542374+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:39.181161+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:16.028358+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:47.869896+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:22.969676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:57.504061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6801/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/901503025":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/2275951342":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/1384534494":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/450820820":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3451764091":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6801/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/400653309":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3025276768":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/3641576098":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3264237187":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6800/778669970":"2024-03-23T10:30:58.613511+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb59c06c6b0 msgr2=0x7fb59c06eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb59c06c6b0 0x7fb59c06eb60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4005e10 tx=0x7fb5a4005da0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 msgr2=0x7fb5b8197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 0x7fb5b8197080 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8036040 tx=0x7fb5a8004f00 comp rx=0 tx=0).stop 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 shutdown_connections 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb59c06c6b0 0x7fb59c06eb60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b8100830 0x7fb5b81975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 --2- 172.21.15.115:0/2908691972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5b80ff640 0x7fb5b8197080 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 >> 172.21.15.115:0/2908691972 conn(0x7fb5b80fabc0 msgr2=0x7fb5b8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 shutdown_connections 2024-03-22T10:39:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.263+0000 7fb5bd600640 1 -- 172.21.15.115:0/2908691972 wait complete. 2024-03-22T10:39:04.271 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:04 smithi115 ceph-mon[29111]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:04.273 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T10:35:17.870309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T10:38:27.996809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T10:38:28.878933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T10:39:04.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T10:39:04.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:04 smithi150 ceph-mon[34518]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:04.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.483+0000 7f24e5a00640 1 -- 172.21.15.115:0/26585107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e00ff0c0 msgr2=0x7f24e00ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.483+0000 7f24e5a00640 1 --2- 172.21.15.115:0/26585107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e00ff0c0 0x7f24e00ff530 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009880 tx=0x7f24c802bce0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.483+0000 7f24e5a00640 1 -- 172.21.15.115:0/26585107 shutdown_connections 2024-03-22T10:39:04.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.483+0000 7f24e5a00640 1 --2- 172.21.15.115:0/26585107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e00ff0c0 0x7f24e00ff530 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.483+0000 7f24e5a00640 1 --2- 172.21.15.115:0/26585107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e0100b90 0x7f24e0100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.483+0000 7f24e5a00640 1 -- 172.21.15.115:0/26585107 >> 172.21.15.115:0/26585107 conn(0x7f24e00faa10 msgr2=0x7f24e00fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:04.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.484+0000 7f24e5a00640 1 -- 172.21.15.115:0/26585107 shutdown_connections 2024-03-22T10:39:04.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.484+0000 7f24e5a00640 1 -- 172.21.15.115:0/26585107 wait complete. 2024-03-22T10:39:04.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.485+0000 7f24e5a00640 1 Processor -- start 2024-03-22T10:39:04.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.485+0000 7f24e5a00640 1 -- start start 2024-03-22T10:39:04.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.485+0000 7f24e5a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e00ff0c0 0x7f24e01970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.485+0000 7f24e5a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e0100b90 0x7f24e0197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.485+0000 7f24e5a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0197c00 con 0x7f24e0100b90 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.485+0000 7f24e5a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0197d70 con 0x7f24e00ff0c0 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.485+0000 7f24df400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e0100b90 0x7f24e0197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.485+0000 7f24df400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e0100b90 0x7f24e0197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60940/0 (socket says 172.21.15.115:60940) 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.485+0000 7f24df400640 1 -- 172.21.15.115:0/1732358957 learned_addr learned my addr 172.21.15.115:0/1732358957 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24dfe00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e00ff0c0 0x7f24e01970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24df400640 1 -- 172.21.15.115:0/1732358957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e00ff0c0 msgr2=0x7f24e01970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24df400640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e00ff0c0 0x7f24e01970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24df400640 1 -- 172.21.15.115:0/1732358957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8009560 con 0x7f24e0100b90 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24dfe00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e00ff0c0 0x7f24e01970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24df400640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e0100b90 0x7f24e0197630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009230 tx=0x7f24c8004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24dcc00640 1 -- 172.21.15.115:0/1732358957 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c8004520 con 0x7f24e0100b90 2024-03-22T10:39:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e019c7a0 con 0x7f24e0100b90 2024-03-22T10:39:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24dcc00640 1 -- 172.21.15.115:0/1732358957 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f24c80054b0 con 0x7f24e0100b90 2024-03-22T10:39:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.486+0000 7f24dcc00640 1 -- 172.21.15.115:0/1732358957 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c8034670 con 0x7f24e0100b90 2024-03-22T10:39:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.487+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e019cca0 con 0x7f24e0100b90 2024-03-22T10:39:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.488+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e01046f0 con 0x7f24e0100b90 2024-03-22T10:39:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.489+0000 7f24dcc00640 1 -- 172.21.15.115:0/1732358957 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f24c80038b0 con 0x7f24e0100b90 2024-03-22T10:39:04.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.489+0000 7f24dcc00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f24c406c5f0 0x7f24c406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:04.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.490+0000 7f24dcc00640 1 -- 172.21.15.115:0/1732358957 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f24c80adec0 con 0x7f24e0100b90 2024-03-22T10:39:04.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.490+0000 7f24dfe00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f24c406c5f0 0x7f24c406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:04.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.491+0000 7f24dfe00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f24c406c5f0 0x7f24c406eaa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f24e0100500 tx=0x7f24d0009400 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:04.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.494+0000 7f24dcc00640 1 -- 172.21.15.115:0/1732358957 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f24c8078500 con 0x7f24e0100b90 2024-03-22T10:39:04.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.726+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24e002c390 con 0x7f24e0100b90 2024-03-22T10:39:04.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.726+0000 7f24dcc00640 1 -- 172.21.15.115:0/1732358957 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24c8041020 con 0x7f24e0100b90 2024-03-22T10:39:04.718 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f24c406c5f0 msgr2=0x7f24c406eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f24c406c5f0 0x7f24c406eaa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f24e0100500 tx=0x7f24d0009400 comp rx=0 tx=0).stop 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e0100b90 msgr2=0x7f24e0197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e0100b90 0x7f24e0197630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009230 tx=0x7f24c8004e30 comp rx=0 tx=0).stop 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 shutdown_connections 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f24c406c5f0 0x7f24c406eaa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e0100b90 0x7f24e0197630 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 --2- 172.21.15.115:0/1732358957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24e00ff0c0 0x7f24e01970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 >> 172.21.15.115:0/1732358957 conn(0x7f24e00faa10 msgr2=0x7f24e01018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 shutdown_connections 2024-03-22T10:39:04.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.730+0000 7f24e5a00640 1 -- 172.21.15.115:0/1732358957 wait complete. 2024-03-22T10:39:04.739 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T10:39:04.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.935+0000 7f2113000640 1 -- 172.21.15.115:0/782805676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1000d0 msgr2=0x7f210c1024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.935+0000 7f2113000640 1 --2- 172.21.15.115:0/782805676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1000d0 0x7f210c1024b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0098b0 tx=0x7f20fc02bcd0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.935+0000 7f2113000640 1 -- 172.21.15.115:0/782805676 shutdown_connections 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.935+0000 7f2113000640 1 --2- 172.21.15.115:0/782805676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c1029f0 0x7f210c104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.935+0000 7f2113000640 1 --2- 172.21.15.115:0/782805676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1000d0 0x7f210c1024b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.935+0000 7f2113000640 1 -- 172.21.15.115:0/782805676 >> 172.21.15.115:0/782805676 conn(0x7f210c0f9ad0 msgr2=0x7f210c0fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.936+0000 7f2113000640 1 -- 172.21.15.115:0/782805676 shutdown_connections 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.936+0000 7f2113000640 1 -- 172.21.15.115:0/782805676 wait complete. 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.937+0000 7f2113000640 1 Processor -- start 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.937+0000 7f2113000640 1 -- start start 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.938+0000 7f2113000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c1000d0 0x7f210c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.938+0000 7f2113000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1029f0 0x7f210c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:04.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.938+0000 7f2113000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c197d90 con 0x7f210c1029f0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.938+0000 7f2113000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c197f00 con 0x7f210c1000d0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.938+0000 7f210b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1029f0 0x7f210c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.938+0000 7f210b400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1029f0 0x7f210c1977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60964/0 (socket says 172.21.15.115:60964) 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.938+0000 7f210b400640 1 -- 172.21.15.115:0/444983529 learned_addr learned my addr 172.21.15.115:0/444983529 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.938+0000 7f210be00640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c1000d0 0x7f210c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.939+0000 7f210b400640 1 -- 172.21.15.115:0/444983529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c1000d0 msgr2=0x7f210c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.939+0000 7f210b400640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c1000d0 0x7f210c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.939+0000 7f210b400640 1 -- 172.21.15.115:0/444983529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc009560 con 0x7f210c1029f0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.939+0000 7f210b400640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1029f0 0x7f210c1977c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d6c0 tx=0x7f20f800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.939+0000 7f2108c00640 1 -- 172.21.15.115:0/444983529 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20f80051f0 con 0x7f210c1029f0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.939+0000 7f2108c00640 1 -- 172.21.15.115:0/444983529 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f20f8009cd0 con 0x7f210c1029f0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.939+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c19c990 con 0x7f210c1029f0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.939+0000 7f2108c00640 1 -- 172.21.15.115:0/444983529 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20f8004e40 con 0x7f210c1029f0 2024-03-22T10:39:04.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.940+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c19cef0 con 0x7f210c1029f0 2024-03-22T10:39:04.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.941+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c1083b0 con 0x7f210c1029f0 2024-03-22T10:39:04.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.943+0000 7f2108c00640 1 -- 172.21.15.115:0/444983529 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f20f8005350 con 0x7f210c1029f0 2024-03-22T10:39:04.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.945+0000 7f2108c00640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f20f006c5f0 0x7f20f006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:04.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.945+0000 7f2108c00640 1 -- 172.21.15.115:0/444983529 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f20f808b1d0 con 0x7f210c1029f0 2024-03-22T10:39:04.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.945+0000 7f210be00640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f20f006c5f0 0x7f20f006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:04.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.946+0000 7f210be00640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f20f006c5f0 0x7f20f006eaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000c00 tx=0x7f20fc02df90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:04.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:04.950+0000 7f2108c00640 1 -- 172.21.15.115:0/444983529 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f20f80559e0 con 0x7f210c1029f0 2024-03-22T10:39:05.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.151+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f210c19d330 con 0x7f210c1029f0 2024-03-22T10:39:05.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.152+0000 7f2108c00640 1 -- 172.21.15.115:0/444983529 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f20f8059000 con 0x7f210c1029f0 2024-03-22T10:39:05.143 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-03-22T10:39:05.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.154+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f20f006c5f0 msgr2=0x7f20f006eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.154+0000 7f2113000640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f20f006c5f0 0x7f20f006eaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000c00 tx=0x7f20fc02df90 comp rx=0 tx=0).stop 2024-03-22T10:39:05.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.154+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1029f0 msgr2=0x7f210c1977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.154+0000 7f2113000640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1029f0 0x7f210c1977c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d6c0 tx=0x7f20f800db80 comp rx=0 tx=0).stop 2024-03-22T10:39:05.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.154+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 shutdown_connections 2024-03-22T10:39:05.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.154+0000 7f2113000640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f20f006c5f0 0x7f20f006eaa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.154+0000 7f2113000640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f210c1029f0 0x7f210c1977c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.154+0000 7f2113000640 1 --2- 172.21.15.115:0/444983529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c1000d0 0x7f210c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.154+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 >> 172.21.15.115:0/444983529 conn(0x7f210c0f9ad0 msgr2=0x7f210c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:05.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.155+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 shutdown_connections 2024-03-22T10:39:05.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.155+0000 7f2113000640 1 -- 172.21.15.115:0/444983529 wait complete. 2024-03-22T10:39:05.164 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T10:39:05.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.364+0000 7fa528e00640 1 -- 172.21.15.115:0/2616465501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 msgr2=0x7fa524100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.364+0000 7fa528e00640 1 --2- 172.21.15.115:0/2616465501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 0x7fa524100cb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa510009880 tx=0x7fa51002bce0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.365+0000 7fa528e00640 1 -- 172.21.15.115:0/2616465501 shutdown_connections 2024-03-22T10:39:05.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.365+0000 7fa528e00640 1 --2- 172.21.15.115:0/2616465501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 0x7fa524100cb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.365+0000 7fa528e00640 1 --2- 172.21.15.115:0/2616465501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ff650 0x7fa5240ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.365+0000 7fa528e00640 1 -- 172.21.15.115:0/2616465501 >> 172.21.15.115:0/2616465501 conn(0x7fa5240fabd0 msgr2=0x7fa5240fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:05.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.365+0000 7fa528e00640 1 -- 172.21.15.115:0/2616465501 shutdown_connections 2024-03-22T10:39:05.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.365+0000 7fa528e00640 1 -- 172.21.15.115:0/2616465501 wait complete. 2024-03-22T10:39:05.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.366+0000 7fa528e00640 1 Processor -- start 2024-03-22T10:39:05.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.366+0000 7fa528e00640 1 -- start start 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.366+0000 7fa528e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ff650 0x7fa524197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.366+0000 7fa528e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 0x7fa524197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.366+0000 7fa528e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524197c50 con 0x7fa524100840 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.366+0000 7fa528e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524197dc0 con 0x7fa5240ff650 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.366+0000 7fa521400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 0x7fa524197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.366+0000 7fa521400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 0x7fa524197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60984/0 (socket says 172.21.15.115:60984) 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.366+0000 7fa521400640 1 -- 172.21.15.115:0/2827848395 learned_addr learned my addr 172.21.15.115:0/2827848395 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.367+0000 7fa521e00640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ff650 0x7fa524197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.367+0000 7fa521400640 1 -- 172.21.15.115:0/2827848395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ff650 msgr2=0x7fa524197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.367+0000 7fa521400640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ff650 0x7fa524197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.367+0000 7fa521400640 1 -- 172.21.15.115:0/2827848395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa510009560 con 0x7fa524100840 2024-03-22T10:39:05.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.367+0000 7fa521400640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 0x7fa524197680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa510000c00 tx=0x7fa510004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:05.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.367+0000 7fa51aa00640 1 -- 172.21.15.115:0/2827848395 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5100044a0 con 0x7fa524100840 2024-03-22T10:39:05.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.367+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52419c7f0 con 0x7fa524100840 2024-03-22T10:39:05.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.367+0000 7fa51aa00640 1 -- 172.21.15.115:0/2827848395 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa5100054b0 con 0x7fa524100840 2024-03-22T10:39:05.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.367+0000 7fa51aa00640 1 -- 172.21.15.115:0/2827848395 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa510034670 con 0x7fa524100840 2024-03-22T10:39:05.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.368+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52419ccf0 con 0x7fa524100840 2024-03-22T10:39:05.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.369+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa524104890 con 0x7fa524100840 2024-03-22T10:39:05.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.369+0000 7fa51aa00640 1 -- 172.21.15.115:0/2827848395 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa510005000 con 0x7fa524100840 2024-03-22T10:39:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.370+0000 7fa51aa00640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa50806c5f0 0x7fa50806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.370+0000 7fa51aa00640 1 -- 172.21.15.115:0/2827848395 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa5100adee0 con 0x7fa524100840 2024-03-22T10:39:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.371+0000 7fa521e00640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa50806c5f0 0x7fa50806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:05.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.371+0000 7fa521e00640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa50806c5f0 0x7fa50806eaa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa514005fd0 tx=0x7fa514005850 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:05.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.374+0000 7fa51aa00640 1 -- 172.21.15.115:0/2827848395 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa5100785c0 con 0x7fa524100840 2024-03-22T10:39:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:05 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2908691972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:39:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:05 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1732358957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:39:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:39:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:39:05.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:05 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2908691972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:39:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:05 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1732358957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:39:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:39:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:39:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:05.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.555+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa52402c270 con 0x7fa524100840 2024-03-22T10:39:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.555+0000 7fa51aa00640 1 -- 172.21.15.115:0/2827848395 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa510041020 con 0x7fa524100840 2024-03-22T10:39:05.547 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-03-22T10:39:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa50806c5f0 msgr2=0x7fa50806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa50806c5f0 0x7fa50806eaa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa514005fd0 tx=0x7fa514005850 comp rx=0 tx=0).stop 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 msgr2=0x7fa524197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 0x7fa524197680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa510000c00 tx=0x7fa510004f00 comp rx=0 tx=0).stop 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 shutdown_connections 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa50806c5f0 0x7fa50806eaa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa524100840 0x7fa524197680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 --2- 172.21.15.115:0/2827848395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5240ff650 0x7fa524197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 >> 172.21.15.115:0/2827848395 conn(0x7fa5240fabd0 msgr2=0x7fa524101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 shutdown_connections 2024-03-22T10:39:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.558+0000 7fa528e00640 1 -- 172.21.15.115:0/2827848395 wait complete. 2024-03-22T10:39:05.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T10:39:05.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.773+0000 7f0600a00640 1 -- 172.21.15.115:0/866062034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc101860 msgr2=0x7f05fc101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.773+0000 7f0600a00640 1 --2- 172.21.15.115:0/866062034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc101860 0x7f05fc101c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009880 tx=0x7f05ec02bcf0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.773+0000 7f0600a00640 1 -- 172.21.15.115:0/866062034 shutdown_connections 2024-03-22T10:39:05.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.773+0000 7f0600a00640 1 --2- 172.21.15.115:0/866062034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc102a50 0x7f05fc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.773+0000 7f0600a00640 1 --2- 172.21.15.115:0/866062034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc101860 0x7f05fc101c70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.773+0000 7f0600a00640 1 -- 172.21.15.115:0/866062034 >> 172.21.15.115:0/866062034 conn(0x7f05fc0fce10 msgr2=0x7f05fc0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:05.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.774+0000 7f0600a00640 1 -- 172.21.15.115:0/866062034 shutdown_connections 2024-03-22T10:39:05.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.774+0000 7f0600a00640 1 -- 172.21.15.115:0/866062034 wait complete. 2024-03-22T10:39:05.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.774+0000 7f0600a00640 1 Processor -- start 2024-03-22T10:39:05.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f0600a00640 1 -- start start 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f0600a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc101860 0x7f05fc197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f0600a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc102a50 0x7f05fc1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f0600a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc197b80 con 0x7f05fc102a50 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f0600a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc197cf0 con 0x7f05fc101860 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f05f9400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc102a50 0x7f05fc1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f05f9400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc102a50 0x7f05fc1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60992/0 (socket says 172.21.15.115:60992) 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f05f9400640 1 -- 172.21.15.115:0/4192357457 learned_addr learned my addr 172.21.15.115:0/4192357457 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f05f9e00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc101860 0x7f05fc197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f05f9400640 1 -- 172.21.15.115:0/4192357457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc101860 msgr2=0x7f05fc197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f05f9400640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc101860 0x7f05fc197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f05f9400640 1 -- 172.21.15.115:0/4192357457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec009560 con 0x7f05fc102a50 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.775+0000 7f05f9e00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc101860 0x7f05fc197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:39:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.776+0000 7f05f9400640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc102a50 0x7f05fc1975b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05e800d8d0 tx=0x7f05e800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:05.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.776+0000 7f05f2a00640 1 -- 172.21.15.115:0/4192357457 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e800bea0 con 0x7f05fc102a50 2024-03-22T10:39:05.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.776+0000 7f05f2a00640 1 -- 172.21.15.115:0/4192357457 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05e8009c00 con 0x7f05fc102a50 2024-03-22T10:39:05.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.776+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc19c780 con 0x7f05fc102a50 2024-03-22T10:39:05.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.776+0000 7f05f2a00640 1 -- 172.21.15.115:0/4192357457 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e800f710 con 0x7f05fc102a50 2024-03-22T10:39:05.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.776+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc19cce0 con 0x7f05fc102a50 2024-03-22T10:39:05.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.777+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc10a570 con 0x7f05fc102a50 2024-03-22T10:39:05.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.778+0000 7f05f2a00640 1 -- 172.21.15.115:0/4192357457 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f05e800f990 con 0x7f05fc102a50 2024-03-22T10:39:05.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.779+0000 7f05f2a00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f05e006c6b0 0x7f05e006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:05.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.779+0000 7f05f2a00640 1 -- 172.21.15.115:0/4192357457 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f05e808b730 con 0x7f05fc102a50 2024-03-22T10:39:05.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.780+0000 7f05f9e00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f05e006c6b0 0x7f05e006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:05.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.780+0000 7f05f9e00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f05e006c6b0 0x7f05e006eb60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05ec036040 tx=0x7f05ec005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:05.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.784+0000 7f05f2a00640 1 -- 172.21.15.115:0/4192357457 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f05e8059800 con 0x7f05fc102a50 2024-03-22T10:39:05.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.995+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05fc19d110 con 0x7f05fc102a50 2024-03-22T10:39:05.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.995+0000 7f05f2a00640 1 -- 172.21.15.115:0/4192357457 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4764 (secure 0 0 0) 0x7f05e8020090 con 0x7f05fc102a50 2024-03-22T10:39:05.987 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:39:05.987 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:38:50.921733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14454,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14454":{"gid":14454,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6827/3803356871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3803356871},{"type":"v1","addr":"172.21.15.115:6827","nonce":3803356871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi150.zvyrzc","rank":1,"incarnation":6,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6825/1103557222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1103557222},{"type":"v1","addr":"172.21.15.150:6825","nonce":1103557222}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T10:39:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 12 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f05e006c6b0 msgr2=0x7f05e006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f05e006c6b0 0x7f05e006eb60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05ec036040 tx=0x7f05ec005fd0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc102a50 msgr2=0x7f05fc1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc102a50 0x7f05fc1975b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05e800d8d0 tx=0x7f05e800dd90 comp rx=0 tx=0).stop 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 shutdown_connections 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f05e006c6b0 0x7f05e006eb60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05fc102a50 0x7f05fc1975b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 --2- 172.21.15.115:0/4192357457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc101860 0x7f05fc197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 >> 172.21.15.115:0/4192357457 conn(0x7f05fc0fce10 msgr2=0x7f05fc105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 shutdown_connections 2024-03-22T10:39:05.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:05.998+0000 7f0600a00640 1 -- 172.21.15.115:0/4192357457 wait complete. 2024-03-22T10:39:06.006 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 12, 'max_mds': 2, 'flags': 18} 2024-03-22T10:39:06.006 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-22T10:39:06.033 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-22T10:39:06.033 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-03-22T10:39:06.033 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-22T10:39:06.033 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T10:39:06.034 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:06.034 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-22T10:39:06.034 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T10:39:06.034 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:06.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:06.034 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2024-03-22T10:39:06.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:06.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link delete ceph-brx 2024-03-22T10:39:06.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:06 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/444983529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T10:39:06.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:06 smithi150 ceph-mon[34518]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:06.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:06 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2827848395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T10:39:06.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:06 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/4192357457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T10:39:06.176 INFO:teuthology.orchestra.run.smithi115.stderr:Cannot find device "ceph-brx" 2024-03-22T10:39:06.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T10:39:06.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:06.180 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-03-22T10:39:06.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:06.240 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-03-22T10:39:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:06 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/444983529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T10:39:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:06 smithi115 ceph-mon[29111]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:06.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:06 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2827848395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T10:39:06.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:06 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/4192357457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T10:39:06.338 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-03-22T10:39:06.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T10:39:06.343 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-22T10:39:06.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-22T10:39:06.736 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:39:08.605 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:08 smithi115 ceph-mon[29111]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:08.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:08 smithi150 ceph-mon[34518]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.916+0000 7fa996a00700 1 -- 172.21.15.115:0/600431416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102600 msgr2=0x7fa9901029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.916+0000 7fa996a00700 1 --2- 172.21.15.115:0/600431416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102600 0x7fa9901029d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa980009910 tx=0x7fa980009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.916+0000 7fa996a00700 1 -- 172.21.15.115:0/600431416 shutdown_connections 2024-03-22T10:39:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.916+0000 7fa996a00700 1 --2- 172.21.15.115:0/600431416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa990102fa0 0x7fa990106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.916+0000 7fa996a00700 1 --2- 172.21.15.115:0/600431416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102600 0x7fa9901029d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.916+0000 7fa996a00700 1 -- 172.21.15.115:0/600431416 >> 172.21.15.115:0/600431416 conn(0x7fa9900fde60 msgr2=0x7fa990100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.917+0000 7fa996a00700 1 -- 172.21.15.115:0/600431416 shutdown_connections 2024-03-22T10:39:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.917+0000 7fa996a00700 1 -- 172.21.15.115:0/600431416 wait complete. 2024-03-22T10:39:08.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.918+0000 7fa996a00700 1 Processor -- start 2024-03-22T10:39:08.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.918+0000 7fa996a00700 1 -- start start 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.918+0000 7fa996a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa990102600 0x7fa990075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.918+0000 7fa996a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102fa0 0x7fa9900757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.918+0000 7fa996a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900793f0 con 0x7fa990102fa0 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.918+0000 7fa996a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa990075ce0 con 0x7fa990102600 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.919+0000 7fa987400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102fa0 0x7fa9900757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.919+0000 7fa987400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102fa0 0x7fa9900757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32778/0 (socket says 172.21.15.115:32778) 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.919+0000 7fa987400700 1 -- 172.21.15.115:0/923685483 learned_addr learned my addr 172.21.15.115:0/923685483 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.919+0000 7fa98fe00700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa990102600 0x7fa990075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.919+0000 7fa987400700 1 -- 172.21.15.115:0/923685483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa990102600 msgr2=0x7fa990075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.919+0000 7fa987400700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa990102600 0x7fa990075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:08.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.919+0000 7fa987400700 1 -- 172.21.15.115:0/923685483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800095f0 con 0x7fa990102fa0 2024-03-22T10:39:08.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.919+0000 7fa987400700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102fa0 0x7fa9900757a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa97800b480 tx=0x7fa97800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:08.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.920+0000 7fa98d600700 1 -- 172.21.15.115:0/923685483 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9780118c0 con 0x7fa990102fa0 2024-03-22T10:39:08.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.920+0000 7fa98d600700 1 -- 172.21.15.115:0/923685483 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa978010380 con 0x7fa990102fa0 2024-03-22T10:39:08.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.920+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa990075fc0 con 0x7fa990102fa0 2024-03-22T10:39:08.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.920+0000 7fa98d600700 1 -- 172.21.15.115:0/923685483 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa97800f4f0 con 0x7fa990102fa0 2024-03-22T10:39:08.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.920+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9901a67a0 con 0x7fa990102fa0 2024-03-22T10:39:08.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.921+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99004ea50 con 0x7fa990102fa0 2024-03-22T10:39:08.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.922+0000 7fa98d600700 1 -- 172.21.15.115:0/923685483 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa9780104f0 con 0x7fa990102fa0 2024-03-22T10:39:08.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.923+0000 7fa98d600700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa97406c6d0 0x7fa97406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:08.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.924+0000 7fa98d600700 1 -- 172.21.15.115:0/923685483 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa97808b560 con 0x7fa990102fa0 2024-03-22T10:39:08.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.924+0000 7fa98fe00700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa97406c6d0 0x7fa97406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:08.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.924+0000 7fa98fe00700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa97406c6d0 0x7fa97406eb80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa980006010 tx=0x7fa98001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:08.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:08.928+0000 7fa98d600700 1 -- 172.21.15.115:0/923685483 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa978055ba0 con 0x7fa990102fa0 2024-03-22T10:39:09.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.188+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa990076790 con 0x7fa990102fa0 2024-03-22T10:39:09.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.189+0000 7fa98d600700 1 -- 172.21.15.115:0/923685483 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+83 (secure 0 0 0) 0x7fa9780591c0 con 0x7fa990102fa0 2024-03-22T10:39:09.181 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-22T10:39:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.193+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa97406c6d0 msgr2=0x7fa97406eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.193+0000 7fa996a00700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa97406c6d0 0x7fa97406eb80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa980006010 tx=0x7fa98001a040 comp rx=0 tx=0).stop 2024-03-22T10:39:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.193+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102fa0 msgr2=0x7fa9900757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.193+0000 7fa996a00700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102fa0 0x7fa9900757a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa97800b480 tx=0x7fa97800b840 comp rx=0 tx=0).stop 2024-03-22T10:39:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.193+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 shutdown_connections 2024-03-22T10:39:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.194+0000 7fa996a00700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa97406c6d0 0x7fa97406eb80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.194+0000 7fa996a00700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa990102600 0x7fa990075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.194+0000 7fa996a00700 1 --2- 172.21.15.115:0/923685483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102fa0 0x7fa9900757a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.194+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 >> 172.21.15.115:0/923685483 conn(0x7fa9900fde60 msgr2=0x7fa9900ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.194+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 shutdown_connections 2024-03-22T10:39:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:09.194+0000 7fa996a00700 1 -- 172.21.15.115:0/923685483 wait complete. 2024-03-22T10:39:09.502 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:09 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/923685483' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T10:39:09.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:09 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/923685483' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T10:39:09.919 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T10:39:09.919 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T10:39:09.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2024-03-22T10:39:09.919 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T10:39:09.919 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:09.919 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T10:39:09.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T10:39:09.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T10:39:09.919 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T10:39:09.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:09.920 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T10:39:09.976 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2024-03-22T10:39:09.977 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2024-03-22T10:39:09.977 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T10:39:09.977 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2024-03-22T10:39:09.977 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T10:39:09.977 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2024-03-22T10:39:09.977 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-22T10:39:09.977 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 42048sec preferred_lft 42048sec 2024-03-22T10:39:09.977 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2024-03-22T10:39:09.977 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2024-03-22T10:39:09.978 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T10:39:09.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:09.979 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2024-03-22T10:39:09.979 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-03-22T10:39:09.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add name ceph-brx type bridge 2024-03-22T10:39:09.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr flush dev ceph-brx 2024-03-22T10:39:09.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set ceph-brx up 2024-03-22T10:39:09.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T10:39:09.979 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-03-22T10:39:10.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:10.420 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T10:39:10.497 INFO:teuthology.orchestra.run.smithi115.stdout:1 2024-03-22T10:39:10.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:10.501 DEBUG:teuthology.orchestra.run.smithi115:> route 2024-03-22T10:39:10.564 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2024-03-22T10:39:10.564 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T10:39:10.565 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T10:39:10.565 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T10:39:10.565 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T10:39:10.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:10.565 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2024-03-22T10:39:10.565 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-03-22T10:39:10.565 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T10:39:10.565 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T10:39:10.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T10:39:10.566 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-03-22T10:39:10.644 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:10 smithi115 ceph-mon[29111]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:10.644 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:39:10.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:10 smithi150 ceph-mon[34518]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:10.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:39:11.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:11.279 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2024-03-22T10:39:11.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:11.333 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list-id 2024-03-22T10:39:11.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:11.382 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2024-03-22T10:39:11.382 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-03-22T10:39:11.382 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T10:39:11.382 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-22T10:39:11.383 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-03-22T10:39:11.526 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-22T10:39:11.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:11.526 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2024-03-22T10:39:11.526 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-03-22T10:39:11.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-22T10:39:11.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T10:39:11.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-22T10:39:11.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-22T10:39:11.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-22T10:39:11.527 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-03-22T10:39:11.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:11.806 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2024-03-22T10:39:11.806 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-03-22T10:39:11.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2024-03-22T10:39:11.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T10:39:11.806 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-03-22T10:39:11.968 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-22T10:39:11.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T10:39:11.968 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:12.015 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T10:39:12.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T10:39:12.015 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:12.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo modprobe fuse 2024-03-22T10:39:12.201 DEBUG:teuthology.orchestra.run.smithi115:> cat /proc/self/mounts | awk '{print $2}' 2024-03-22T10:39:12.254 INFO:teuthology.orchestra.run.smithi115.stdout:/proc 2024-03-22T10:39:12.254 INFO:teuthology.orchestra.run.smithi115.stdout:/sys 2024-03-22T10:39:12.254 INFO:teuthology.orchestra.run.smithi115.stdout:/dev 2024-03-22T10:39:12.254 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/security 2024-03-22T10:39:12.254 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/shm 2024-03-22T10:39:12.254 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/pts 2024-03-22T10:39:12.254 INFO:teuthology.orchestra.run.smithi115.stdout:/run 2024-03-22T10:39:12.254 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/cgroup 2024-03-22T10:39:12.254 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/pstore 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/bpf 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/ 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/selinux 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/mqueue 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/hugepages 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/debug 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/tracing 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/fuse/connections 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/config 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-sysctl.service 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-22T10:39:12.255 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/nfs/rpc_pipefs 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/1000 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/ceph 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/0835d2c75ed813818391515d5840de66fb0a8dc4258516238924e9bf5fa9c32c/merged 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/0d4598106aef96a39352c5419a1623953337d63a6ef42b1177f3389d62c675f1/merged 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/0 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f961da32546dab0501b4a00a3b16594b21ed8eaaf7276272a9c64de50f9b8b68/merged 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6e36c67d01c5ee2e18de6c624e7ebf8dd6aa76c598810526df16337743131751/merged 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/a6af226d62bd343b95ca57ea144e77bc8c0c8d2dee7b9dbdbaf8ac0f5e7e0e57/merged 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6cb3a30f956de80ca9f19c46e347163e9b441bae53fe238994ea29e878857917/merged 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/c1628ac66793cb6cf17ba61ce058a813f849a62fc4362a966fa08d1c0af1d888/merged 2024-03-22T10:39:12.256 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f60b8787200ef5ef96c4c5e81ec963479b265f82e4dfe55d137f38862e5b79f3/merged 2024-03-22T10:39:12.257 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/434f641603eb7430c2f938e45db17c8ca9a6567d7f0496919c506a42621e17ec/merged 2024-03-22T10:39:12.257 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/ac3eb9a94f56cc19e94993fcd29f3f8e3f725a26c7922de810618841175300c3/merged 2024-03-22T10:39:12.257 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f5d941035808ec176659f6e72267baba2263141b259f25d8167b6b2e3787af87/merged 2024-03-22T10:39:12.257 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/15dac0888aa7446037f99ff1fa6fb9429f6ea17e4dba7bb1572d9173570cf859/merged 2024-03-22T10:39:12.257 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/a0828b6b9336a20aaa75d0f2418a690345237901d62d5b184cfaa482f25f59a0/merged 2024-03-22T10:39:12.257 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns 2024-03-22T10:39:12.257 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T10:39:12.257 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T10:39:12.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:12.258 DEBUG:teuthology.orchestra.run.smithi115:> ls /sys/fs/fuse/connections 2024-03-22T10:39:12.303 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-22T10:39:12.303 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-22T10:39:12.351 DEBUG:teuthology.orchestra.run.smithi115:> sudo modprobe fuse 2024-03-22T10:39:12.432 DEBUG:teuthology.orchestra.run.smithi115:> cat /proc/self/mounts | awk '{print $2}' 2024-03-22T10:39:12.454 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:12 smithi115 ceph-mon[29111]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:12.476 INFO:teuthology.orchestra.run.smithi115.stdout:/proc 2024-03-22T10:39:12.476 INFO:teuthology.orchestra.run.smithi115.stdout:/sys 2024-03-22T10:39:12.476 INFO:teuthology.orchestra.run.smithi115.stdout:/dev 2024-03-22T10:39:12.476 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/security 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/shm 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/pts 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/run 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/cgroup 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/pstore 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/bpf 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/ 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/selinux 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/mqueue 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/hugepages 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/debug 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/tracing 2024-03-22T10:39:12.477 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/fuse/connections 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/config 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-sysctl.service 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/nfs/rpc_pipefs 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/1000 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/ceph 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/0835d2c75ed813818391515d5840de66fb0a8dc4258516238924e9bf5fa9c32c/merged 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/0d4598106aef96a39352c5419a1623953337d63a6ef42b1177f3389d62c675f1/merged 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/0 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:12.478 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f961da32546dab0501b4a00a3b16594b21ed8eaaf7276272a9c64de50f9b8b68/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6e36c67d01c5ee2e18de6c624e7ebf8dd6aa76c598810526df16337743131751/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/a6af226d62bd343b95ca57ea144e77bc8c0c8d2dee7b9dbdbaf8ac0f5e7e0e57/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6cb3a30f956de80ca9f19c46e347163e9b441bae53fe238994ea29e878857917/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/c1628ac66793cb6cf17ba61ce058a813f849a62fc4362a966fa08d1c0af1d888/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f60b8787200ef5ef96c4c5e81ec963479b265f82e4dfe55d137f38862e5b79f3/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/434f641603eb7430c2f938e45db17c8ca9a6567d7f0496919c506a42621e17ec/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/ac3eb9a94f56cc19e94993fcd29f3f8e3f725a26c7922de810618841175300c3/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f5d941035808ec176659f6e72267baba2263141b259f25d8167b6b2e3787af87/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/15dac0888aa7446037f99ff1fa6fb9429f6ea17e4dba7bb1572d9173570cf859/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/a0828b6b9336a20aaa75d0f2418a690345237901d62d5b184cfaa482f25f59a0/merged 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T10:39:12.479 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T10:39:12.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:12.480 DEBUG:teuthology.orchestra.run.smithi115:> ls /sys/fs/fuse/connections 2024-03-22T10:39:12.560 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi115.stderr:2024-03-22T10:39:12.568+0000 7f6bc0441480 -1 init, newargv = 0x55560de618a0 newargc=15 2024-03-22T10:39:12.560 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi115.stderr:ceph-fuse[66966]: starting ceph client 2024-03-22T10:39:12.579 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi115.stderr:ceph-fuse[66966]: starting fuse 2024-03-22T10:39:12.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:12 smithi150 ceph-mon[34518]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:13.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo modprobe fuse 2024-03-22T10:39:13.761 DEBUG:teuthology.orchestra.run.smithi115:> cat /proc/self/mounts | awk '{print $2}' 2024-03-22T10:39:13.815 INFO:teuthology.orchestra.run.smithi115.stdout:/proc 2024-03-22T10:39:13.815 INFO:teuthology.orchestra.run.smithi115.stdout:/sys 2024-03-22T10:39:13.815 INFO:teuthology.orchestra.run.smithi115.stdout:/dev 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/security 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/shm 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/pts 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/run 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/cgroup 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/pstore 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/bpf 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/ 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/selinux 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/mqueue 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/hugepages 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/debug 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/tracing 2024-03-22T10:39:13.816 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/fuse/connections 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/config 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-sysctl.service 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/nfs/rpc_pipefs 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/1000 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/ceph 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/0835d2c75ed813818391515d5840de66fb0a8dc4258516238924e9bf5fa9c32c/merged 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/0d4598106aef96a39352c5419a1623953337d63a6ef42b1177f3389d62c675f1/merged 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/0 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f961da32546dab0501b4a00a3b16594b21ed8eaaf7276272a9c64de50f9b8b68/merged 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6e36c67d01c5ee2e18de6c624e7ebf8dd6aa76c598810526df16337743131751/merged 2024-03-22T10:39:13.817 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/a6af226d62bd343b95ca57ea144e77bc8c0c8d2dee7b9dbdbaf8ac0f5e7e0e57/merged 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6cb3a30f956de80ca9f19c46e347163e9b441bae53fe238994ea29e878857917/merged 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/c1628ac66793cb6cf17ba61ce058a813f849a62fc4362a966fa08d1c0af1d888/merged 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f60b8787200ef5ef96c4c5e81ec963479b265f82e4dfe55d137f38862e5b79f3/merged 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/434f641603eb7430c2f938e45db17c8ca9a6567d7f0496919c506a42621e17ec/merged 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/ac3eb9a94f56cc19e94993fcd29f3f8e3f725a26c7922de810618841175300c3/merged 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f5d941035808ec176659f6e72267baba2263141b259f25d8167b6b2e3787af87/merged 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/15dac0888aa7446037f99ff1fa6fb9429f6ea17e4dba7bb1572d9173570cf859/merged 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/a0828b6b9336a20aaa75d0f2418a690345237901d62d5b184cfaa482f25f59a0/merged 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T10:39:13.818 INFO:teuthology.orchestra.run.smithi115.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:13.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:13.819 DEBUG:teuthology.orchestra.run.smithi115:> ls /sys/fs/fuse/connections 2024-03-22T10:39:13.870 INFO:teuthology.orchestra.run.smithi115.stdout:158 2024-03-22T10:39:13.870 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [158] 2024-03-22T10:39:13.870 DEBUG:teuthology.orchestra.run.smithi115:> sudo stdin-killer -- python3 -c ' 2024-03-22T10:39:13.870 DEBUG:teuthology.orchestra.run.smithi115:> import glob 2024-03-22T10:39:13.870 DEBUG:teuthology.orchestra.run.smithi115:> import re 2024-03-22T10:39:13.870 DEBUG:teuthology.orchestra.run.smithi115:> import os 2024-03-22T10:39:13.870 DEBUG:teuthology.orchestra.run.smithi115:> import subprocess 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> def _find_admin_socket(client_name): 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> files = glob.glob(asok_path) 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> # Given a non-glob path, it better be there 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> if "*" not in asok_path: 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> assert(len(files) == 1) 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> return files[0] 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> for f in files: 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> contents = proc_f.read() 2024-03-22T10:39:13.871 DEBUG:teuthology.orchestra.run.smithi115:> if mountpoint in contents: 2024-03-22T10:39:13.872 DEBUG:teuthology.orchestra.run.smithi115:> return f 2024-03-22T10:39:13.872 DEBUG:teuthology.orchestra.run.smithi115:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-22T10:39:13.872 DEBUG:teuthology.orchestra.run.smithi115:> 2024-03-22T10:39:13.872 DEBUG:teuthology.orchestra.run.smithi115:> print(_find_admin_socket("client.0")) 2024-03-22T10:39:13.872 DEBUG:teuthology.orchestra.run.smithi115:> ' 2024-03-22T10:39:14.044 INFO:teuthology.orchestra.run.smithi115.stdout:/var/run/ceph/ceph-client.0.66966.asok 2024-03-22T10:39:14.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T10:39:14.062 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.66966.asok 2024-03-22T10:39:14.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:14.062 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.66966.asok status 2024-03-22T10:39:14.293 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:39:14.293 INFO:teuthology.orchestra.run.smithi115.stdout: "metadata": { 2024-03-22T10:39:14.293 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-03-22T10:39:14.293 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-03-22T10:39:14.293 INFO:teuthology.orchestra.run.smithi115.stdout: "entity_id": "0", 2024-03-22T10:39:14.293 INFO:teuthology.orchestra.run.smithi115.stdout: "hostname": "smithi115", 2024-03-22T10:39:14.293 INFO:teuthology.orchestra.run.smithi115.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-22T10:39:14.293 INFO:teuthology.orchestra.run.smithi115.stdout: "pid": "66966", 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "root": "/" 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "dentry_count": 0, 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "dentry_pinned_count": 0, 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 14516, 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "inst": { 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "name": { 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "client", 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 14516 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": { 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "192.168.0.1:0", 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 629163332 2024-03-22T10:39:14.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": { 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "192.168.0.1:0", 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 629163332 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "inst_str": "client.14516 192.168.0.1:0/629163332", 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "addr_str": "192.168.0.1:0/629163332", 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "inode_count": 1, 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_epoch": 12, 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_epoch": 45, 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_epoch_barrier": 0, 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "blocklisted": false, 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs" 2024-03-22T10:39:14.295 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:39:14.305 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:14 smithi115 ceph-mon[29111]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:14.309 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-22T10:39:14.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:14 smithi150 ceph-mon[34518]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:14.696 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:39:16.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:16 smithi150 ceph-mon[34518]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T10:39:16.669 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:16 smithi115 ceph-mon[29111]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T10:39:16.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.845+0000 7fbfa1600700 1 -- 172.21.15.115:0/3957077516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102fd0 msgr2=0x7fbf9c106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:16.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.845+0000 7fbfa1600700 1 --2- 172.21.15.115:0/3957077516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102fd0 0x7fbf9c106fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009910 tx=0x7fbf84009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:16.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.845+0000 7fbfa1600700 1 -- 172.21.15.115:0/3957077516 shutdown_connections 2024-03-22T10:39:16.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.845+0000 7fbfa1600700 1 --2- 172.21.15.115:0/3957077516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102fd0 0x7fbf9c106fa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:16.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.845+0000 7fbfa1600700 1 --2- 172.21.15.115:0/3957077516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf9c102630 0x7fbf9c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:16.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.845+0000 7fbfa1600700 1 -- 172.21.15.115:0/3957077516 >> 172.21.15.115:0/3957077516 conn(0x7fbf9c0fdef0 msgr2=0x7fbf9c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:16.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.846+0000 7fbfa1600700 1 -- 172.21.15.115:0/3957077516 shutdown_connections 2024-03-22T10:39:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.846+0000 7fbfa1600700 1 -- 172.21.15.115:0/3957077516 wait complete. 2024-03-22T10:39:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.847+0000 7fbfa1600700 1 Processor -- start 2024-03-22T10:39:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.847+0000 7fbfa1600700 1 -- start start 2024-03-22T10:39:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.847+0000 7fbfa1600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102630 0x7fbf9c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.847+0000 7fbfa1600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf9c102fd0 0x7fbf9c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.847+0000 7fbfa1600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c198c00 con 0x7fbf9c102630 2024-03-22T10:39:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.847+0000 7fbfa1600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c19c990 con 0x7fbf9c102fd0 2024-03-22T10:39:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.847+0000 7fbf9aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102630 0x7fbf9c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.848+0000 7fbf9aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102630 0x7fbf9c197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59980/0 (socket says 172.21.15.115:59980) 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.848+0000 7fbf9aa00700 1 -- 172.21.15.115:0/3018246310 learned_addr learned my addr 172.21.15.115:0/3018246310 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.848+0000 7fbf93e00700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf9c102fd0 0x7fbf9c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.848+0000 7fbf9aa00700 1 -- 172.21.15.115:0/3018246310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf9c102fd0 msgr2=0x7fbf9c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.848+0000 7fbf9aa00700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf9c102fd0 0x7fbf9c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.848+0000 7fbf9aa00700 1 -- 172.21.15.115:0/3018246310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf840095f0 con 0x7fbf9c102630 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.848+0000 7fbf9aa00700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102630 0x7fbf9c197fe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00c8a0 tx=0x7fbf8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.848+0000 7fbf93400700 1 -- 172.21.15.115:0/3018246310 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf8c00cea0 con 0x7fbf9c102630 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.849+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c19cc70 con 0x7fbf9c102630 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.849+0000 7fbf93400700 1 -- 172.21.15.115:0/3018246310 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c004830 con 0x7fbf9c102630 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.849+0000 7fbf93400700 1 -- 172.21.15.115:0/3018246310 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf8c0055b0 con 0x7fbf9c102630 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.849+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c19d1c0 con 0x7fbf9c102630 2024-03-22T10:39:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.850+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c04ea50 con 0x7fbf9c102630 2024-03-22T10:39:16.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.851+0000 7fbf93400700 1 -- 172.21.15.115:0/3018246310 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fbf8c007ad0 con 0x7fbf9c102630 2024-03-22T10:39:16.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.852+0000 7fbf93400700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbf8006c680 0x7fbf8006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:16.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.852+0000 7fbf93400700 1 -- 172.21.15.115:0/3018246310 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbf8c08b5e0 con 0x7fbf9c102630 2024-03-22T10:39:16.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.852+0000 7fbf93e00700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbf8006c680 0x7fbf8006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:16.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.853+0000 7fbf93e00700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbf8006c680 0x7fbf8006eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c199600 tx=0x7fbf8401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:16.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:16.856+0000 7fbf93400700 1 -- 172.21.15.115:0/3018246310 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbf8c055cc0 con 0x7fbf9c102630 2024-03-22T10:39:17.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.119+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbf9c19d670 con 0x7fbf9c102630 2024-03-22T10:39:17.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.120+0000 7fbf93400700 1 -- 172.21.15.115:0/3018246310 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+83 (secure 0 0 0) 0x7fbf8c0592e0 con 0x7fbf9c102630 2024-03-22T10:39:17.112 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-22T10:39:17.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbf8006c680 msgr2=0x7fbf8006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:17.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbf8006c680 0x7fbf8006eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c199600 tx=0x7fbf8401a040 comp rx=0 tx=0).stop 2024-03-22T10:39:17.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102630 msgr2=0x7fbf9c197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:17.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102630 0x7fbf9c197fe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00c8a0 tx=0x7fbf8c00cc60 comp rx=0 tx=0).stop 2024-03-22T10:39:17.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 shutdown_connections 2024-03-22T10:39:17.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbf8006c680 0x7fbf8006eb30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:17.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf9c102630 0x7fbf9c197fe0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:17.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 --2- 172.21.15.115:0/3018246310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf9c102fd0 0x7fbf9c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:17.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 >> 172.21.15.115:0/3018246310 conn(0x7fbf9c0fdef0 msgr2=0x7fbf9c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:17.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 shutdown_connections 2024-03-22T10:39:17.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:17.123+0000 7fbfa1600700 1 -- 172.21.15.115:0/3018246310 wait complete. 2024-03-22T10:39:17.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:17 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3018246310' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T10:39:17.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:17 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3018246310' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T10:39:17.802 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T10:39:17.802 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T10:39:17.802 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-03-22T10:39:17.802 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-22T10:39:17.802 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:17.802 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T10:39:17.802 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T10:39:17.802 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T10:39:17.803 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-22T10:39:17.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:17.803 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-03-22T10:39:17.860 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T10:39:17.860 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T10:39:17.860 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T10:39:17.860 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-22T10:39:17.861 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42054sec preferred_lft 42054sec 2024-03-22T10:39:17.862 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2024-03-22T10:39:17.862 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-03-22T10:39:17.862 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T10:39:17.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:17.863 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-03-22T10:39:17.863 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-03-22T10:39:17.863 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2024-03-22T10:39:17.863 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2024-03-22T10:39:17.863 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2024-03-22T10:39:17.863 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T10:39:17.863 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-03-22T10:39:18.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:18.304 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T10:39:18.383 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-03-22T10:39:18.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:18.388 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-03-22T10:39:18.446 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-03-22T10:39:18.446 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T10:39:18.446 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T10:39:18.446 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T10:39:18.446 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T10:39:18.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:18.447 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-03-22T10:39:18.447 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-03-22T10:39:18.447 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T10:39:18.447 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T10:39:18.447 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T10:39:18.447 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-03-22T10:39:18.528 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:18 smithi150 ceph-mon[34518]: pgmap v239: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T10:39:18.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:18 smithi115 ceph-mon[29111]: pgmap v239: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T10:39:19.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:19.187 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-03-22T10:39:19.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:19.244 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-03-22T10:39:19.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:19.295 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-03-22T10:39:19.295 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-03-22T10:39:19.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T10:39:19.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-22T10:39:19.296 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-03-22T10:39:19.428 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-22T10:39:19.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:19.429 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-03-22T10:39:19.429 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-03-22T10:39:19.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-22T10:39:19.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T10:39:19.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-22T10:39:19.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-22T10:39:19.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-22T10:39:19.429 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-03-22T10:39:19.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:19.720 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-03-22T10:39:19.720 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-03-22T10:39:19.720 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-03-22T10:39:19.720 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T10:39:19.720 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-03-22T10:39:19.889 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-22T10:39:19.889 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T10:39:19.889 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:19.936 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-22T10:39:19.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T10:39:19.937 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:19.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2024-03-22T10:39:20.124 DEBUG:teuthology.orchestra.run.smithi150:> cat /proc/self/mounts | awk '{print $2}' 2024-03-22T10:39:20.178 INFO:teuthology.orchestra.run.smithi150.stdout:/proc 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/sys 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/dev 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/security 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/shm 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/pts 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/run 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/cgroup 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/pstore 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/bpf 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/ 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/selinux 2024-03-22T10:39:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/hugepages 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/mqueue 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/debug 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/tracing 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/fuse/connections 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/config 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/run/credentials/systemd-sysctl.service 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/nfs/rpc_pipefs 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/run/user/1000 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/ceph 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/run/user/0 2024-03-22T10:39:20.180 INFO:teuthology.orchestra.run.smithi150.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/f01f737689007bea06c4b80d2b3102fa4beee41059081decea70f095d8f32b8a/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/879944ce4889367eeffe9a2ae95078275997b597f5b73165a4d0317f55ad10b6/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/16f785ece15c257ff71a609602437b816b1348be2dcb1357db30b452ecc9c69d/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/43c93cd9901a29e39fc740f2060f296005890a9945a66a86d33c2de9edd57134/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/fec1da1163e3935951df3782fa87654bff5ed1d9558099dc078a96587cccdf1d/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/320031f26edc36ad6bc39d44a45f8b3f250ebae5f968538aa9c0d2d5abed3e25/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/dfe1e605579f7994aa7d9a01b375a6f251d4ec70d3d7c868bd89ac0f13916d52/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/ff4f0dccf1486f3f356ea61b4db8d4850e3238eaa409056436eeaf87d5304d84/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/01ad10d64b9e7bd738de84acde081ff407ee386c0c133486aabb14415f90962d/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/1f22f8832fe07263f9750e1b5b535036d837f361db56ea30af86843d853de3de/merged 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/run/netns 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T10:39:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T10:39:20.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:20.183 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2024-03-22T10:39:20.225 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-22T10:39:20.225 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-22T10:39:20.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2024-03-22T10:39:20.353 DEBUG:teuthology.orchestra.run.smithi150:> cat /proc/self/mounts | awk '{print $2}' 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/proc 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/sys 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/dev 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/security 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/shm 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/pts 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/run 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/cgroup 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/pstore 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/bpf 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/ 2024-03-22T10:39:20.397 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/selinux 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/hugepages 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/mqueue 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/debug 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/tracing 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/fuse/connections 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/config 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/run/credentials/systemd-sysctl.service 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/nfs/rpc_pipefs 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/run/user/1000 2024-03-22T10:39:20.398 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/ceph 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/run/user/0 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/f01f737689007bea06c4b80d2b3102fa4beee41059081decea70f095d8f32b8a/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/879944ce4889367eeffe9a2ae95078275997b597f5b73165a4d0317f55ad10b6/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/16f785ece15c257ff71a609602437b816b1348be2dcb1357db30b452ecc9c69d/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/43c93cd9901a29e39fc740f2060f296005890a9945a66a86d33c2de9edd57134/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/fec1da1163e3935951df3782fa87654bff5ed1d9558099dc078a96587cccdf1d/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/320031f26edc36ad6bc39d44a45f8b3f250ebae5f968538aa9c0d2d5abed3e25/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/dfe1e605579f7994aa7d9a01b375a6f251d4ec70d3d7c868bd89ac0f13916d52/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/ff4f0dccf1486f3f356ea61b4db8d4850e3238eaa409056436eeaf87d5304d84/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/01ad10d64b9e7bd738de84acde081ff407ee386c0c133486aabb14415f90962d/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/1f22f8832fe07263f9750e1b5b535036d837f361db56ea30af86843d853de3de/merged 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/run/netns 2024-03-22T10:39:20.399 INFO:teuthology.orchestra.run.smithi150.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T10:39:20.400 INFO:teuthology.orchestra.run.smithi150.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T10:39:20.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:20.400 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2024-03-22T10:39:20.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi150.stderr:2024-03-22T10:39:20.504+0000 7f0b95434480 -1 init, newargv = 0x5586784a68c0 newargc=15 2024-03-22T10:39:20.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi150.stderr:ceph-fuse[54329]: starting ceph client 2024-03-22T10:39:20.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi150.stderr:ceph-fuse[54329]: starting fuse 2024-03-22T10:39:20.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:20 smithi150 ceph-mon[34518]: pgmap v240: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T10:39:20.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:20 smithi115 ceph-mon[29111]: pgmap v240: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T10:39:21.590 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2024-03-22T10:39:21.693 DEBUG:teuthology.orchestra.run.smithi150:> cat /proc/self/mounts | awk '{print $2}' 2024-03-22T10:39:21.749 INFO:teuthology.orchestra.run.smithi150.stdout:/proc 2024-03-22T10:39:21.749 INFO:teuthology.orchestra.run.smithi150.stdout:/sys 2024-03-22T10:39:21.749 INFO:teuthology.orchestra.run.smithi150.stdout:/dev 2024-03-22T10:39:21.749 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/security 2024-03-22T10:39:21.749 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/shm 2024-03-22T10:39:21.749 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/pts 2024-03-22T10:39:21.749 INFO:teuthology.orchestra.run.smithi150.stdout:/run 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/cgroup 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/pstore 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/bpf 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/ 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/selinux 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/hugepages 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/dev/mqueue 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/debug 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/tracing 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/fs/fuse/connections 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/sys/kernel/config 2024-03-22T10:39:21.750 INFO:teuthology.orchestra.run.smithi150.stdout:/run/credentials/systemd-sysctl.service 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/nfs/rpc_pipefs 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/run/user/1000 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/ceph 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/run/user/0 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/f01f737689007bea06c4b80d2b3102fa4beee41059081decea70f095d8f32b8a/merged 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/879944ce4889367eeffe9a2ae95078275997b597f5b73165a4d0317f55ad10b6/merged 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/16f785ece15c257ff71a609602437b816b1348be2dcb1357db30b452ecc9c69d/merged 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/43c93cd9901a29e39fc740f2060f296005890a9945a66a86d33c2de9edd57134/merged 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/fec1da1163e3935951df3782fa87654bff5ed1d9558099dc078a96587cccdf1d/merged 2024-03-22T10:39:21.751 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/320031f26edc36ad6bc39d44a45f8b3f250ebae5f968538aa9c0d2d5abed3e25/merged 2024-03-22T10:39:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/dfe1e605579f7994aa7d9a01b375a6f251d4ec70d3d7c868bd89ac0f13916d52/merged 2024-03-22T10:39:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/ff4f0dccf1486f3f356ea61b4db8d4850e3238eaa409056436eeaf87d5304d84/merged 2024-03-22T10:39:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/01ad10d64b9e7bd738de84acde081ff407ee386c0c133486aabb14415f90962d/merged 2024-03-22T10:39:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:/var/lib/containers/storage/overlay/1f22f8832fe07263f9750e1b5b535036d837f361db56ea30af86843d853de3de/merged 2024-03-22T10:39:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:/run/netns 2024-03-22T10:39:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T10:39:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T10:39:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:21.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:21.753 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2024-03-22T10:39:21.804 INFO:teuthology.orchestra.run.smithi150.stdout:125 2024-03-22T10:39:21.805 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [125] 2024-03-22T10:39:21.805 DEBUG:teuthology.orchestra.run.smithi150:> sudo stdin-killer -- python3 -c ' 2024-03-22T10:39:21.805 DEBUG:teuthology.orchestra.run.smithi150:> import glob 2024-03-22T10:39:21.805 DEBUG:teuthology.orchestra.run.smithi150:> import re 2024-03-22T10:39:21.805 DEBUG:teuthology.orchestra.run.smithi150:> import os 2024-03-22T10:39:21.805 DEBUG:teuthology.orchestra.run.smithi150:> import subprocess 2024-03-22T10:39:21.805 DEBUG:teuthology.orchestra.run.smithi150:> 2024-03-22T10:39:21.805 DEBUG:teuthology.orchestra.run.smithi150:> def _find_admin_socket(client_name): 2024-03-22T10:39:21.805 DEBUG:teuthology.orchestra.run.smithi150:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-22T10:39:21.805 DEBUG:teuthology.orchestra.run.smithi150:> files = glob.glob(asok_path) 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> # Given a non-glob path, it better be there 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> if "*" not in asok_path: 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> assert(len(files) == 1) 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> return files[0] 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> for f in files: 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> contents = proc_f.read() 2024-03-22T10:39:21.806 DEBUG:teuthology.orchestra.run.smithi150:> if mountpoint in contents: 2024-03-22T10:39:21.807 DEBUG:teuthology.orchestra.run.smithi150:> return f 2024-03-22T10:39:21.807 DEBUG:teuthology.orchestra.run.smithi150:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-22T10:39:21.807 DEBUG:teuthology.orchestra.run.smithi150:> 2024-03-22T10:39:21.807 DEBUG:teuthology.orchestra.run.smithi150:> print(_find_admin_socket("client.1")) 2024-03-22T10:39:21.807 DEBUG:teuthology.orchestra.run.smithi150:> ' 2024-03-22T10:39:21.984 INFO:teuthology.orchestra.run.smithi150.stdout:/var/run/ceph/ceph-client.1.54329.asok 2024-03-22T10:39:21.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-22T10:39:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T10:39:22.002 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.54329.asok 2024-03-22T10:39:22.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:22.002 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.54329.asok status 2024-03-22T10:39:22.234 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-03-22T10:39:22.234 INFO:teuthology.orchestra.run.smithi150.stdout: "metadata": { 2024-03-22T10:39:22.234 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-03-22T10:39:22.234 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-03-22T10:39:22.234 INFO:teuthology.orchestra.run.smithi150.stdout: "entity_id": "1", 2024-03-22T10:39:22.234 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": "smithi150", 2024-03-22T10:39:22.234 INFO:teuthology.orchestra.run.smithi150.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-22T10:39:22.234 INFO:teuthology.orchestra.run.smithi150.stdout: "pid": "54329", 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "root": "/" 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_count": 0, 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_pinned_count": 0, 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24289, 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "inst": { 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "name": { 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "client", 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "num": 24289 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.1:0", 2024-03-22T10:39:22.235 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3857176443 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.1:0", 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3857176443 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "inst_str": "client.24289 192.168.0.1:0/3857176443", 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "addr_str": "192.168.0.1:0/3857176443", 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "inode_count": 1, 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_epoch": 12, 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch": 45, 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch_barrier": 0, 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "blocklisted": false, 2024-03-22T10:39:22.236 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs" 2024-03-22T10:39:22.237 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-03-22T10:39:22.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:22.250 DEBUG:teuthology.orchestra.run.smithi115:> stat --file-system '--printf=%T 2024-03-22T10:39:22.250 DEBUG:teuthology.orchestra.run.smithi115:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:22.310 INFO:teuthology.orchestra.run.smithi115.stdout:fuseblk 2024-03-22T10:39:22.310 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:22.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:22.311 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:22.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:22.419 DEBUG:teuthology.orchestra.run.smithi150:> stat --file-system '--printf=%T 2024-03-22T10:39:22.419 DEBUG:teuthology.orchestra.run.smithi150:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:22.475 INFO:teuthology.orchestra.run.smithi150.stdout:fuseblk 2024-03-22T10:39:22.475 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:22.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:39:22.476 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:22.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:22 smithi150 ceph-mon[34518]: pgmap v241: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T10:39:22.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:22 smithi115 ceph-mon[29111]: pgmap v241: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T10:39:23.830 INFO:teuthology.run_tasks:Running task print... 2024-03-22T10:39:23.840 INFO:teuthology.task.print:**** done client 2024-03-22T10:39:23.840 INFO:teuthology.run_tasks:Running task parallel... 2024-03-22T10:39:23.852 INFO:teuthology.task.parallel:starting parallel... 2024-03-22T10:39:23.852 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-22T10:39:23.852 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T10:39:23.853 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:39:23.853 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-03-22T10:39:23.853 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-22T10:39:23.854 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-22T10:39:23.857 INFO:tasks.workunit:Pulling workunits from ref 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:39:23.857 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-22T10:39:23.857 INFO:tasks.workunit:timeout=3h 2024-03-22T10:39:23.857 INFO:tasks.workunit:cleanup=True 2024-03-22T10:39:23.858 DEBUG:teuthology.orchestra.run.smithi115:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:23.914 INFO:teuthology.orchestra.run.smithi115.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:23.914 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-22T10:39:23.914 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 9eh/158d Inode: 1 Links: 2 2024-03-22T10:39:23.914 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:39:23.914 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-22T10:39:23.914 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-22T10:39:23.914 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-03-22 10:38:34.035183644 +0000 2024-03-22T10:39:23.914 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-03-22 10:39:22.416432212 +0000 2024-03-22T10:39:23.915 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-03-22T10:39:23.915 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-22T10:39:23.915 DEBUG:teuthology.orchestra.run.smithi115:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-22T10:39:24.066 DEBUG:teuthology.orchestra.run.smithi150:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:24.125 INFO:teuthology.orchestra.run.smithi150.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:24.125 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-22T10:39:24.125 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-03-22T10:39:24.125 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T10:39:24.125 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-22T10:39:24.125 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-22T10:39:24.125 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-03-22 10:39:24.058054351 +0000 2024-03-22T10:39:24.125 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-03-22 10:39:24.058054351 +0000 2024-03-22T10:39:24.125 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-03-22T10:39:24.125 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-22T10:39:24.126 DEBUG:teuthology.orchestra.run.smithi150:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-22T10:39:24.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:24 smithi150 ceph-mon[34518]: pgmap v242: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-22T10:39:24.224 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:39:24.224 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:39:24.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:24 smithi115 ceph-mon[29111]: pgmap v242: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-22T10:39:24.265 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:39:24.269 INFO:tasks.workunit.client.0.smithi115.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-22T10:39:24.322 INFO:tasks.workunit.client.1.smithi150.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-22T10:39:25.441 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:39:25.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:39:26.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.296+0000 7fa987800700 1 -- 172.21.15.115:0/4109601922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 msgr2=0x7fa9801039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:26.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.296+0000 7fa987800700 1 --2- 172.21.15.115:0/4109601922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 0x7fa9801039f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009910 tx=0x7fa96c009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:26.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.296+0000 7fa987800700 1 -- 172.21.15.115:0/4109601922 shutdown_connections 2024-03-22T10:39:26.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.296+0000 7fa987800700 1 --2- 172.21.15.115:0/4109601922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 0x7fa9801039f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:26.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.296+0000 7fa987800700 1 --2- 172.21.15.115:0/4109601922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9801023a0 0x7fa9801027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:26.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.296+0000 7fa987800700 1 -- 172.21.15.115:0/4109601922 >> 172.21.15.115:0/4109601922 conn(0x7fa9800fd990 msgr2=0x7fa9800ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:26.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.296+0000 7fa987800700 1 -- 172.21.15.115:0/4109601922 shutdown_connections 2024-03-22T10:39:26.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.296+0000 7fa987800700 1 -- 172.21.15.115:0/4109601922 wait complete. 2024-03-22T10:39:26.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.298+0000 7fa987800700 1 Processor -- start 2024-03-22T10:39:26.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.298+0000 7fa987800700 1 -- start start 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.298+0000 7fa987800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9801023a0 0x7fa980197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.298+0000 7fa987800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 0x7fa9801980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.298+0000 7fa987800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa980198670 con 0x7fa9801035a0 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.298+0000 7fa987800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9801987b0 con 0x7fa9801023a0 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.299+0000 7fa97fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 0x7fa9801980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.299+0000 7fa97fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 0x7fa9801980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35888/0 (socket says 172.21.15.115:35888) 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.299+0000 7fa97fe00700 1 -- 172.21.15.115:0/3123459330 learned_addr learned my addr 172.21.15.115:0/3123459330 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.299+0000 7fa984e00700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9801023a0 0x7fa980197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.299+0000 7fa97fe00700 1 -- 172.21.15.115:0/3123459330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9801023a0 msgr2=0x7fa980197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.299+0000 7fa97fe00700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9801023a0 0x7fa980197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.299+0000 7fa97fe00700 1 -- 172.21.15.115:0/3123459330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0095f0 con 0x7fa9801035a0 2024-03-22T10:39:26.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.300+0000 7fa97fe00700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 0x7fa9801980e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa96c005b40 tx=0x7fa96c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:26.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.300+0000 7fa97d600700 1 -- 172.21.15.115:0/3123459330 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa96c01d070 con 0x7fa9801035a0 2024-03-22T10:39:26.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.300+0000 7fa97d600700 1 -- 172.21.15.115:0/3123459330 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa96c022470 con 0x7fa9801035a0 2024-03-22T10:39:26.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.300+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98019d210 con 0x7fa9801035a0 2024-03-22T10:39:26.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.300+0000 7fa97d600700 1 -- 172.21.15.115:0/3123459330 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa96c00f690 con 0x7fa9801035a0 2024-03-22T10:39:26.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.300+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98019d6d0 con 0x7fa9801035a0 2024-03-22T10:39:26.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.301+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98010aef0 con 0x7fa9801035a0 2024-03-22T10:39:26.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.302+0000 7fa97d600700 1 -- 172.21.15.115:0/3123459330 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa96c0225e0 con 0x7fa9801035a0 2024-03-22T10:39:26.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.303+0000 7fa97d600700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa96806c680 0x7fa96806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:26.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.303+0000 7fa97d600700 1 -- 172.21.15.115:0/3123459330 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa96c08d160 con 0x7fa9801035a0 2024-03-22T10:39:26.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.304+0000 7fa984e00700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa96806c680 0x7fa96806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:26.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.304+0000 7fa984e00700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa96806c680 0x7fa96806eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa974005fd0 tx=0x7fa974005e50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:26.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.308+0000 7fa97d600700 1 -- 172.21.15.115:0/3123459330 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa96c05b2b0 con 0x7fa9801035a0 2024-03-22T10:39:26.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:26 smithi115 ceph-mon[29111]: pgmap v243: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-22T10:39:26.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.528+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fa98019de60 con 0x7fa9801035a0 2024-03-22T10:39:26.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.537+0000 7fa97d600700 1 -- 172.21.15.115:0/3123459330 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v22)=0 v22) v1 ==== 126+0+0 (secure 0 0 0) 0x7fa96c0270c0 con 0x7fa9801035a0 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.539+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa96806c680 msgr2=0x7fa96806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.539+0000 7fa987800700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa96806c680 0x7fa96806eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa974005fd0 tx=0x7fa974005e50 comp rx=0 tx=0).stop 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.539+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 msgr2=0x7fa9801980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.539+0000 7fa987800700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 0x7fa9801980e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa96c005b40 tx=0x7fa96c00be20 comp rx=0 tx=0).stop 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.540+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 shutdown_connections 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.540+0000 7fa987800700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fa96806c680 0x7fa96806eb30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.540+0000 7fa987800700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9801023a0 0x7fa980197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.540+0000 7fa987800700 1 --2- 172.21.15.115:0/3123459330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9801035a0 0x7fa9801980e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.540+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 >> 172.21.15.115:0/3123459330 conn(0x7fa9800fd990 msgr2=0x7fa9801067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:26.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.540+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 shutdown_connections 2024-03-22T10:39:26.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:26.540+0000 7fa987800700 1 -- 172.21.15.115:0/3123459330 wait complete. 2024-03-22T10:39:26.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:26 smithi150 ceph-mon[34518]: pgmap v243: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-22T10:39:27.330 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T10:39:27.331 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:39:27.331 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-22T10:39:27.801 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:39:27.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:27 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3123459330' entity='client.admin' 2024-03-22T10:39:27.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:27 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:39:27.945 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:27 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3123459330' entity='client.admin' 2024-03-22T10:39:27.946 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:39:28.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:28 smithi115 ceph-mon[29111]: pgmap v244: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-22T10:39:28.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:28 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:39:28.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:28 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:39:28.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:28 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:28.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:28 smithi150 ceph-mon[34518]: pgmap v244: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-22T10:39:28.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:28 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:39:28.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:28 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:39:28.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:28 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:30.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.053+0000 7fc1e9c00700 1 -- 172.21.15.115:0/128106058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 msgr2=0x7fc1e41003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:30.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.053+0000 7fc1e9c00700 1 --2- 172.21.15.115:0/128106058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 0x7fc1e41003d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009910 tx=0x7fc1d4009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:30.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.053+0000 7fc1e9c00700 1 -- 172.21.15.115:0/128106058 shutdown_connections 2024-03-22T10:39:30.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.053+0000 7fc1e9c00700 1 --2- 172.21.15.115:0/128106058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1e41011c0 0x7fc1e4101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:30.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.053+0000 7fc1e9c00700 1 --2- 172.21.15.115:0/128106058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 0x7fc1e41003d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:30.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.053+0000 7fc1e9c00700 1 -- 172.21.15.115:0/128106058 >> 172.21.15.115:0/128106058 conn(0x7fc1e40fb570 msgr2=0x7fc1e40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:30.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.053+0000 7fc1e9c00700 1 -- 172.21.15.115:0/128106058 shutdown_connections 2024-03-22T10:39:30.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.053+0000 7fc1e9c00700 1 -- 172.21.15.115:0/128106058 wait complete. 2024-03-22T10:39:30.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.054+0000 7fc1e9c00700 1 Processor -- start 2024-03-22T10:39:30.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.054+0000 7fc1e9c00700 1 -- start start 2024-03-22T10:39:30.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.055+0000 7fc1e9c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 0x7fc1e4195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:30.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.055+0000 7fc1e9c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1e41011c0 0x7fc1e4195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.055+0000 7fc1e9c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41962f0 con 0x7fc1e40fffc0 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.055+0000 7fc1e9c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e4196430 con 0x7fc1e41011c0 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.055+0000 7fc1e3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 0x7fc1e4195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.055+0000 7fc1e3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 0x7fc1e4195790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35902/0 (socket says 172.21.15.115:35902) 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.055+0000 7fc1e3e00700 1 -- 172.21.15.115:0/3595257845 learned_addr learned my addr 172.21.15.115:0/3595257845 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.055+0000 7fc1e3e00700 1 -- 172.21.15.115:0/3595257845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1e41011c0 msgr2=0x7fc1e4195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.055+0000 7fc1e3400700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1e41011c0 0x7fc1e4195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.056+0000 7fc1e3e00700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1e41011c0 0x7fc1e4195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.056+0000 7fc1e3e00700 1 -- 172.21.15.115:0/3595257845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d40095f0 con 0x7fc1e40fffc0 2024-03-22T10:39:30.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.056+0000 7fc1e3e00700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 0x7fc1e4195790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400bd70 tx=0x7fc1d400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:30.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.056+0000 7fc1e0c00700 1 -- 172.21.15.115:0/3595257845 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d401d070 con 0x7fc1e40fffc0 2024-03-22T10:39:30.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.056+0000 7fc1e0c00700 1 -- 172.21.15.115:0/3595257845 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc1d400f460 con 0x7fc1e40fffc0 2024-03-22T10:39:30.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.056+0000 7fc1e0c00700 1 -- 172.21.15.115:0/3595257845 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d4021620 con 0x7fc1e40fffc0 2024-03-22T10:39:30.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.056+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e419ae80 con 0x7fc1e40fffc0 2024-03-22T10:39:30.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.056+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e419b2f0 con 0x7fc1e40fffc0 2024-03-22T10:39:30.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.057+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e4108b10 con 0x7fc1e40fffc0 2024-03-22T10:39:30.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.058+0000 7fc1e0c00700 1 -- 172.21.15.115:0/3595257845 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fc1d400fa40 con 0x7fc1e40fffc0 2024-03-22T10:39:30.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.059+0000 7fc1e0c00700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc1c806c680 0x7fc1c806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:30.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.059+0000 7fc1e0c00700 1 -- 172.21.15.115:0/3595257845 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc1d408cbe0 con 0x7fc1e40fffc0 2024-03-22T10:39:30.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.060+0000 7fc1e3400700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc1c806c680 0x7fc1c806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:30.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.060+0000 7fc1e3400700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc1c806c680 0x7fc1c806eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0005950 tx=0x7fc1d00058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:30.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.064+0000 7fc1e0c00700 1 -- 172.21.15.115:0/3595257845 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc1d405ad30 con 0x7fc1e40fffc0 2024-03-22T10:39:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.281+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fc1e4108d00 con 0x7fc1e40fffc0 2024-03-22T10:39:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.281+0000 7fc1e0c00700 1 -- 172.21.15.115:0/3595257845 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v22)=0 v22) v1 ==== 155+0+0 (secure 0 0 0) 0x7fc1d405a8c0 con 0x7fc1e40fffc0 2024-03-22T10:39:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.284+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc1c806c680 msgr2=0x7fc1c806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.284+0000 7fc1e9c00700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc1c806c680 0x7fc1c806eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0005950 tx=0x7fc1d00058e0 comp rx=0 tx=0).stop 2024-03-22T10:39:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.284+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 msgr2=0x7fc1e4195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.284+0000 7fc1e9c00700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 0x7fc1e4195790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400bd70 tx=0x7fc1d400be50 comp rx=0 tx=0).stop 2024-03-22T10:39:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.284+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 shutdown_connections 2024-03-22T10:39:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.284+0000 7fc1e9c00700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc1c806c680 0x7fc1c806eb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.284+0000 7fc1e9c00700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1e40fffc0 0x7fc1e4195790 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.284+0000 7fc1e9c00700 1 --2- 172.21.15.115:0/3595257845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1e41011c0 0x7fc1e4195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.284+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 >> 172.21.15.115:0/3595257845 conn(0x7fc1e40fb570 msgr2=0x7fc1e41043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.285+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 shutdown_connections 2024-03-22T10:39:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:30.285+0000 7fc1e9c00700 1 -- 172.21.15.115:0/3595257845 wait complete. 2024-03-22T10:39:30.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:30 smithi150 ceph-mon[34518]: pgmap v245: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T10:39:30.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:30 smithi115 ceph-mon[29111]: pgmap v245: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T10:39:30.990 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-22T10:39:31.397 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:39:32.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:32 smithi150 ceph-mon[34518]: pgmap v246: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T10:39:32.980 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:32 smithi115 ceph-mon[29111]: pgmap v246: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T10:39:33.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.452+0000 7f6d63a00700 1 -- 172.21.15.115:0/443860220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c101370 msgr2=0x7f6d5c1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:33.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.452+0000 7f6d63a00700 1 --2- 172.21.15.115:0/443860220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c101370 0x7f6d5c1017c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009910 tx=0x7f6d48009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:33.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.452+0000 7f6d63a00700 1 -- 172.21.15.115:0/443860220 shutdown_connections 2024-03-22T10:39:33.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.452+0000 7f6d63a00700 1 --2- 172.21.15.115:0/443860220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c101370 0x7f6d5c1017c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:33.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.452+0000 7f6d63a00700 1 --2- 172.21.15.115:0/443860220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5c100170 0x7f6d5c100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:33.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.452+0000 7f6d63a00700 1 -- 172.21.15.115:0/443860220 >> 172.21.15.115:0/443860220 conn(0x7f6d5c0fb6e0 msgr2=0x7f6d5c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:33.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.452+0000 7f6d63a00700 1 -- 172.21.15.115:0/443860220 shutdown_connections 2024-03-22T10:39:33.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.452+0000 7f6d63a00700 1 -- 172.21.15.115:0/443860220 wait complete. 2024-03-22T10:39:33.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.453+0000 7f6d63a00700 1 Processor -- start 2024-03-22T10:39:33.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.453+0000 7f6d63a00700 1 -- start start 2024-03-22T10:39:33.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d63a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c100170 0x7f6d5c195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:33.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d63a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5c101370 0x7f6d5c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:33.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d63a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c1964e0 con 0x7f6d5c100170 2024-03-22T10:39:33.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d63a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c196620 con 0x7f6d5c101370 2024-03-22T10:39:33.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d61000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c100170 0x7f6d5c195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:33.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d61000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c100170 0x7f6d5c195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60270/0 (socket says 172.21.15.115:60270) 2024-03-22T10:39:33.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d61000700 1 -- 172.21.15.115:0/1690183481 learned_addr learned my addr 172.21.15.115:0/1690183481 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d5be00700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5c101370 0x7f6d5c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d61000700 1 -- 172.21.15.115:0/1690183481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5c101370 msgr2=0x7f6d5c195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d61000700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5c101370 0x7f6d5c195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.454+0000 7f6d61000700 1 -- 172.21.15.115:0/1690183481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d480095f0 con 0x7f6d5c100170 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.455+0000 7f6d61000700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c100170 0x7f6d5c195980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c8f0 tx=0x7f6d5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.455+0000 7f6d59600700 1 -- 172.21.15.115:0/1690183481 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d50007a70 con 0x7f6d5c100170 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.455+0000 7f6d59600700 1 -- 172.21.15.115:0/1690183481 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6d50007bd0 con 0x7f6d5c100170 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.455+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c19b0d0 con 0x7f6d5c100170 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.455+0000 7f6d59600700 1 -- 172.21.15.115:0/1690183481 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d50018630 con 0x7f6d5c100170 2024-03-22T10:39:33.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.455+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c19b620 con 0x7f6d5c100170 2024-03-22T10:39:33.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.456+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c108cc0 con 0x7f6d5c100170 2024-03-22T10:39:33.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.457+0000 7f6d59600700 1 -- 172.21.15.115:0/1690183481 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f6d5000f450 con 0x7f6d5c100170 2024-03-22T10:39:33.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.458+0000 7f6d59600700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d4406c680 0x7f6d4406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:33.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.458+0000 7f6d59600700 1 -- 172.21.15.115:0/1690183481 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6d5008b890 con 0x7f6d5c100170 2024-03-22T10:39:33.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.458+0000 7f6d5be00700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d4406c680 0x7f6d4406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:33.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.459+0000 7f6d5be00700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d4406c680 0x7f6d4406eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6d48006010 tx=0x7f6d4800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:33.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.463+0000 7f6d59600700 1 -- 172.21.15.115:0/1690183481 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6d50059ab0 con 0x7f6d5c100170 2024-03-22T10:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.675+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f6d5c19bcc0 con 0x7f6d5c100170 2024-03-22T10:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.676+0000 7f6d59600700 1 -- 172.21.15.115:0/1690183481 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v22)=0 v22) v1 ==== 163+0+0 (secure 0 0 0) 0x7f6d50059640 con 0x7f6d5c100170 2024-03-22T10:39:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d4406c680 msgr2=0x7f6d4406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d4406c680 0x7f6d4406eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6d48006010 tx=0x7f6d4800b540 comp rx=0 tx=0).stop 2024-03-22T10:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c100170 msgr2=0x7f6d5c195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c100170 0x7f6d5c195980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c8f0 tx=0x7f6d5000ccb0 comp rx=0 tx=0).stop 2024-03-22T10:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 shutdown_connections 2024-03-22T10:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f6d4406c680 0x7f6d4406eb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:33.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c100170 0x7f6d5c195980 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:33.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 --2- 172.21.15.115:0/1690183481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5c101370 0x7f6d5c195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:33.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 >> 172.21.15.115:0/1690183481 conn(0x7f6d5c0fb6e0 msgr2=0x7f6d5c1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:33.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 shutdown_connections 2024-03-22T10:39:33.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:33.679+0000 7f6d63a00700 1 -- 172.21.15.115:0/1690183481 wait complete. 2024-03-22T10:39:34.312 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:34 smithi115 ceph-mon[29111]: pgmap v247: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 1 op/s 2024-03-22T10:39:34.373 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-22T10:39:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:34 smithi150 ceph-mon[34518]: pgmap v247: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 1 op/s 2024-03-22T10:39:34.771 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:39:36.394 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:36 smithi115 ceph-mon[29111]: pgmap v248: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T10:39:36.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:36 smithi150 ceph-mon[34518]: pgmap v248: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T10:39:36.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.897+0000 7f438ea00700 1 -- 172.21.15.115:0/2609291375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 msgr2=0x7f4388100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:36.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.897+0000 7f438ea00700 1 --2- 172.21.15.115:0/2609291375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 0x7f4388100560 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4378009910 tx=0x7f4378009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:36.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.897+0000 7f438ea00700 1 -- 172.21.15.115:0/2609291375 shutdown_connections 2024-03-22T10:39:36.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.897+0000 7f438ea00700 1 --2- 172.21.15.115:0/2609291375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4388101350 0x7f43881017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:36.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.897+0000 7f438ea00700 1 --2- 172.21.15.115:0/2609291375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 0x7f4388100560 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:36.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.897+0000 7f438ea00700 1 -- 172.21.15.115:0/2609291375 >> 172.21.15.115:0/2609291375 conn(0x7f43880fb6e0 msgr2=0x7f43880fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.897+0000 7f438ea00700 1 -- 172.21.15.115:0/2609291375 shutdown_connections 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.897+0000 7f438ea00700 1 -- 172.21.15.115:0/2609291375 wait complete. 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.898+0000 7f438ea00700 1 Processor -- start 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.899+0000 7f438ea00700 1 -- start start 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.899+0000 7f438ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 0x7f4388197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.899+0000 7f438ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4388101350 0x7f4388198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.899+0000 7f438ea00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43881987b0 con 0x7f4388100150 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.899+0000 7f438ea00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438819d1c0 con 0x7f4388101350 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.899+0000 7f4387e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 0x7f4388197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.899+0000 7f4387e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 0x7f4388197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60290/0 (socket says 172.21.15.115:60290) 2024-03-22T10:39:36.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.899+0000 7f4387e00700 1 -- 172.21.15.115:0/3856746896 learned_addr learned my addr 172.21.15.115:0/3856746896 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:36.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.900+0000 7f437f400700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4388101350 0x7f4388198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.900+0000 7f4387e00700 1 -- 172.21.15.115:0/3856746896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4388101350 msgr2=0x7f4388198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.900+0000 7f4387e00700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4388101350 0x7f4388198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.900+0000 7f4387e00700 1 -- 172.21.15.115:0/3856746896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780095f0 con 0x7f4388100150 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.900+0000 7f4387e00700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 0x7f4388197c50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f437800bd70 tx=0x7f437800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.900+0000 7f4385600700 1 -- 172.21.15.115:0/3856746896 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f437801d070 con 0x7f4388100150 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.900+0000 7f4385600700 1 -- 172.21.15.115:0/3856746896 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f437800f460 con 0x7f4388100150 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.900+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438819d360 con 0x7f4388100150 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.900+0000 7f4385600700 1 -- 172.21.15.115:0/3856746896 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4378021640 con 0x7f4388100150 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.901+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438819d8a0 con 0x7f4388100150 2024-03-22T10:39:36.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.902+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4388105430 con 0x7f4388100150 2024-03-22T10:39:36.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.903+0000 7f4385600700 1 -- 172.21.15.115:0/3856746896 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f437800fab0 con 0x7f4388100150 2024-03-22T10:39:36.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.904+0000 7f4385600700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f436c06c680 0x7f436c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:36.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.904+0000 7f4385600700 1 -- 172.21.15.115:0/3856746896 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f437808cce0 con 0x7f4388100150 2024-03-22T10:39:36.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.904+0000 7f437f400700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f436c06c680 0x7f436c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:36.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.905+0000 7f437f400700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f436c06c680 0x7f436c06eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4370006fd0 tx=0x7f4370008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:36.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:36.908+0000 7f4385600700 1 -- 172.21.15.115:0/3856746896 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f43780573a0 con 0x7f4388100150 2024-03-22T10:39:37.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.122+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f438802cc30 con 0x7f4388100150 2024-03-22T10:39:37.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.122+0000 7f4385600700 1 -- 172.21.15.115:0/3856746896 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v22)=0 v22) v1 ==== 135+0+0 (secure 0 0 0) 0x7f43780260c0 con 0x7f4388100150 2024-03-22T10:39:37.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f436c06c680 msgr2=0x7f436c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:37.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f436c06c680 0x7f436c06eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4370006fd0 tx=0x7f4370008040 comp rx=0 tx=0).stop 2024-03-22T10:39:37.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 msgr2=0x7f4388197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:37.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 0x7f4388197c50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f437800bd70 tx=0x7f437800be50 comp rx=0 tx=0).stop 2024-03-22T10:39:37.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 shutdown_connections 2024-03-22T10:39:37.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f436c06c680 0x7f436c06eb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:37.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4388100150 0x7f4388197c50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:37.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 --2- 172.21.15.115:0/3856746896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4388101350 0x7f4388198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:37.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 >> 172.21.15.115:0/3856746896 conn(0x7f43880fb6e0 msgr2=0x7f4388102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:37.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 shutdown_connections 2024-03-22T10:39:37.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:37.125+0000 7f438ea00700 1 -- 172.21.15.115:0/3856746896 wait complete. 2024-03-22T10:39:37.843 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-22T10:39:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:39:38.611 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:38 smithi115 ceph-mon[29111]: pgmap v249: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-03-22T10:39:38.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:38 smithi150 ceph-mon[34518]: pgmap v249: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-03-22T10:39:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.377+0000 7f57a8a00700 1 -- 172.21.15.115:0/3245143519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4101190 msgr2=0x7f57a41015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.377+0000 7f57a8a00700 1 --2- 172.21.15.115:0/3245143519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4101190 0x7f57a41015e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f578c009910 tx=0x7f578c009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.377+0000 7f57a8a00700 1 -- 172.21.15.115:0/3245143519 shutdown_connections 2024-03-22T10:39:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.377+0000 7f57a8a00700 1 --2- 172.21.15.115:0/3245143519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4101190 0x7f57a41015e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.377+0000 7f57a8a00700 1 --2- 172.21.15.115:0/3245143519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57a40fff90 0x7f57a41003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.377+0000 7f57a8a00700 1 -- 172.21.15.115:0/3245143519 >> 172.21.15.115:0/3245143519 conn(0x7f57a40fb500 msgr2=0x7f57a40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.377+0000 7f57a8a00700 1 -- 172.21.15.115:0/3245143519 shutdown_connections 2024-03-22T10:39:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.378+0000 7f57a8a00700 1 -- 172.21.15.115:0/3245143519 wait complete. 2024-03-22T10:39:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.379+0000 7f57a8a00700 1 Processor -- start 2024-03-22T10:39:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.379+0000 7f57a8a00700 1 -- start start 2024-03-22T10:39:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.379+0000 7f57a8a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a40fff90 0x7f57a41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.379+0000 7f57a8a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57a4101190 0x7f57a4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.379+0000 7f57a8a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4198510 con 0x7f57a40fff90 2024-03-22T10:39:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.379+0000 7f57a8a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4198650 con 0x7f57a4101190 2024-03-22T10:39:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.379+0000 7f57a3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a40fff90 0x7f57a41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.380+0000 7f57a3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a40fff90 0x7f57a41979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60310/0 (socket says 172.21.15.115:60310) 2024-03-22T10:39:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.380+0000 7f57a3400700 1 -- 172.21.15.115:0/3719244740 learned_addr learned my addr 172.21.15.115:0/3719244740 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:40.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.380+0000 7f579be00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57a4101190 0x7f57a4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:40.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.380+0000 7f57a3400700 1 -- 172.21.15.115:0/3719244740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57a4101190 msgr2=0x7f57a4197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:40.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.380+0000 7f57a3400700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57a4101190 0x7f57a4197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:40.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.380+0000 7f57a3400700 1 -- 172.21.15.115:0/3719244740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0095f0 con 0x7f57a40fff90 2024-03-22T10:39:40.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.380+0000 7f579be00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57a4101190 0x7f57a4197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:39:40.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.380+0000 7f57a3400700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a40fff90 0x7f57a41979b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f579400c8f0 tx=0x7f579400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:40.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.381+0000 7f57a0c00700 1 -- 172.21.15.115:0/3719244740 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5794007a70 con 0x7f57a40fff90 2024-03-22T10:39:40.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.381+0000 7f57a0c00700 1 -- 172.21.15.115:0/3719244740 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5794007bd0 con 0x7f57a40fff90 2024-03-22T10:39:40.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.381+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a419d100 con 0x7f57a40fff90 2024-03-22T10:39:40.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.381+0000 7f57a0c00700 1 -- 172.21.15.115:0/3719244740 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5794018630 con 0x7f57a40fff90 2024-03-22T10:39:40.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.381+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a419d650 con 0x7f57a40fff90 2024-03-22T10:39:40.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.382+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a4105270 con 0x7f57a40fff90 2024-03-22T10:39:40.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.383+0000 7f57a0c00700 1 -- 172.21.15.115:0/3719244740 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f579400f450 con 0x7f57a40fff90 2024-03-22T10:39:40.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.384+0000 7f57a0c00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f578806c680 0x7f578806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:40.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.384+0000 7f57a0c00700 1 -- 172.21.15.115:0/3719244740 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f579408b3a0 con 0x7f57a40fff90 2024-03-22T10:39:40.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.384+0000 7f579be00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f578806c680 0x7f578806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:40.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.385+0000 7f579be00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f578806c680 0x7f578806eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f578c006010 tx=0x7f578c00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:40.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.389+0000 7f57a0c00700 1 -- 172.21.15.115:0/3719244740 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5794059630 con 0x7f57a40fff90 2024-03-22T10:39:40.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:40 smithi115 ceph-mon[29111]: pgmap v250: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T10:39:40.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:39:40.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.628+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "target": ["mon-mgr", ""]}) v1 -- 0x7f57a4061230 con 0x7f578806c680 2024-03-22T10:39:40.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.641+0000 7f57a0c00700 1 -- 172.21.15.115:0/3719244740 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f57a4061230 con 0x7f578806c680 2024-03-22T10:39:40.634 INFO:teuthology.orchestra.run.smithi115.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:39:40.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.643+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f578806c680 msgr2=0x7f578806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.643+0000 7f57a8a00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f578806c680 0x7f578806eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f578c006010 tx=0x7f578c00b540 comp rx=0 tx=0).stop 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.643+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a40fff90 msgr2=0x7f57a41979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.643+0000 7f57a8a00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a40fff90 0x7f57a41979b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f579400c8f0 tx=0x7f579400ccb0 comp rx=0 tx=0).stop 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.643+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 shutdown_connections 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.643+0000 7f57a8a00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f578806c680 0x7f578806eb30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.643+0000 7f57a8a00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a40fff90 0x7f57a41979b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.644+0000 7f57a8a00700 1 --2- 172.21.15.115:0/3719244740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57a4101190 0x7f57a4197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.644+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 >> 172.21.15.115:0/3719244740 conn(0x7f57a40fb500 msgr2=0x7f57a41023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.644+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 shutdown_connections 2024-03-22T10:39:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:40.644+0000 7f57a8a00700 1 -- 172.21.15.115:0/3719244740 wait complete. 2024-03-22T10:39:40.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:40 smithi150 ceph-mon[34518]: pgmap v250: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T10:39:40.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:40 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:39:41.504 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T10:39:41.505 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:39:41.505 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-22T10:39:41.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:41 smithi115 ceph-mon[29111]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:41.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:41 smithi115 ceph-mon[29111]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:39:41.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:41.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:39:41.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:39:41.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:39:41.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:41 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:41.814 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:41 smithi150 ceph-mon[34518]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:41.814 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:41 smithi150 ceph-mon[34518]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:39:41.814 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:41.814 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:39:41.814 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:39:41.815 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:39:41.815 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:41 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:39:41.911 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:39:42.939 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:42 smithi115 ceph-mon[29111]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:39:42.939 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:42 smithi115 ceph-mon[29111]: pgmap v251: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T10:39:43.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:42 smithi150 ceph-mon[34518]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-22T10:39:43.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:42 smithi150 ceph-mon[34518]: pgmap v251: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T10:39:44.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.074+0000 7f7460400700 1 -- 172.21.15.115:0/3228597307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 msgr2=0x7f7458102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:44.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.074+0000 7f7460400700 1 --2- 172.21.15.115:0/3228597307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 0x7f7458102fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f744c009910 tx=0x7f744c009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:44.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.074+0000 7f7460400700 1 -- 172.21.15.115:0/3228597307 shutdown_connections 2024-03-22T10:39:44.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.074+0000 7f7460400700 1 --2- 172.21.15.115:0/3228597307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7458103500 0x7f74581058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.074+0000 7f7460400700 1 --2- 172.21.15.115:0/3228597307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 0x7f7458102fc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.074+0000 7f7460400700 1 -- 172.21.15.115:0/3228597307 >> 172.21.15.115:0/3228597307 conn(0x7f74580fa5e0 msgr2=0x7f74580fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:44.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.074+0000 7f7460400700 1 -- 172.21.15.115:0/3228597307 shutdown_connections 2024-03-22T10:39:44.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.074+0000 7f7460400700 1 -- 172.21.15.115:0/3228597307 wait complete. 2024-03-22T10:39:44.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.076+0000 7f7460400700 1 Processor -- start 2024-03-22T10:39:44.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.076+0000 7f7460400700 1 -- start start 2024-03-22T10:39:44.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.076+0000 7f7460400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 0x7f7458100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:44.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.076+0000 7f7460400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7458103500 0x7f74580feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:44.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.076+0000 7f7460400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580ff0d0 con 0x7f7458100be0 2024-03-22T10:39:44.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.076+0000 7f7460400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580ff210 con 0x7f7458103500 2024-03-22T10:39:44.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.077+0000 7f745da00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 0x7f7458100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:44.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.077+0000 7f745da00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 0x7f7458100540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44778/0 (socket says 172.21.15.115:44778) 2024-03-22T10:39:44.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.077+0000 7f745da00700 1 -- 172.21.15.115:0/1750794236 learned_addr learned my addr 172.21.15.115:0/1750794236 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:44.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.077+0000 7f745d000700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7458103500 0x7f74580feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:44.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.077+0000 7f745da00700 1 -- 172.21.15.115:0/1750794236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7458103500 msgr2=0x7f74580feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:44.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.077+0000 7f745da00700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7458103500 0x7f74580feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.077+0000 7f745da00700 1 -- 172.21.15.115:0/1750794236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c0095f0 con 0x7f7458100be0 2024-03-22T10:39:44.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.077+0000 7f745da00700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 0x7f7458100540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f744c00b5c0 tx=0x7f744c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:44.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.078+0000 7f7456000700 1 -- 172.21.15.115:0/1750794236 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744c01d070 con 0x7f7458100be0 2024-03-22T10:39:44.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.078+0000 7f7456000700 1 -- 172.21.15.115:0/1750794236 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f744c00f460 con 0x7f7458100be0 2024-03-22T10:39:44.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.078+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74580ff490 con 0x7f7458100be0 2024-03-22T10:39:44.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.078+0000 7f7456000700 1 -- 172.21.15.115:0/1750794236 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744c021640 con 0x7f7458100be0 2024-03-22T10:39:44.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.078+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74580ff980 con 0x7f7458100be0 2024-03-22T10:39:44.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.080+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745818fb70 con 0x7f7458100be0 2024-03-22T10:39:44.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.080+0000 7f7456000700 1 -- 172.21.15.115:0/1750794236 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f744c00f5d0 con 0x7f7458100be0 2024-03-22T10:39:44.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.082+0000 7f7456000700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f744006c680 0x7f744006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:44.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.082+0000 7f7456000700 1 -- 172.21.15.115:0/1750794236 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f744c08cc80 con 0x7f7458100be0 2024-03-22T10:39:44.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.082+0000 7f745d000700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f744006c680 0x7f744006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:44.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.083+0000 7f745d000700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f744006c680 0x7f744006eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7444005950 tx=0x7f74440058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:44.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.090+0000 7f7456000700 1 -- 172.21.15.115:0/1750794236 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f744c05add0 con 0x7f7458100be0 2024-03-22T10:39:44.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:44 smithi115 ceph-mon[29111]: pgmap v252: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T10:39:44.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.386+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7458061190 con 0x7f744006c680 2024-03-22T10:39:44.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.387+0000 7f7456000700 1 -- 172.21.15.115:0/1750794236 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f7458061190 con 0x7f744006c680 2024-03-22T10:39:44.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f744006c680 msgr2=0x7f744006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:44.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f744006c680 0x7f744006eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7444005950 tx=0x7f74440058e0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 msgr2=0x7f7458100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:44.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 0x7f7458100540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f744c00b5c0 tx=0x7f744c00be20 comp rx=0 tx=0).stop 2024-03-22T10:39:44.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 shutdown_connections 2024-03-22T10:39:44.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f744006c680 0x7f744006eb30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458100be0 0x7f7458100540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 --2- 172.21.15.115:0/1750794236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7458103500 0x7f74580feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 >> 172.21.15.115:0/1750794236 conn(0x7f74580fa5e0 msgr2=0x7f74580fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:44.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.390+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 shutdown_connections 2024-03-22T10:39:44.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.391+0000 7f7460400700 1 -- 172.21.15.115:0/1750794236 wait complete. 2024-03-22T10:39:44.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:44 smithi150 ceph-mon[34518]: pgmap v252: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T10:39:44.401 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:39:44.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.563+0000 7f7fa9e00700 1 -- 172.21.15.115:0/1079692349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa41011b0 msgr2=0x7f7fa4101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:44.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.563+0000 7f7fa9e00700 1 --2- 172.21.15.115:0/1079692349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa41011b0 0x7f7fa4101600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009910 tx=0x7f7f90009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:44.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.563+0000 7f7fa9e00700 1 -- 172.21.15.115:0/1079692349 shutdown_connections 2024-03-22T10:39:44.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.563+0000 7f7fa9e00700 1 --2- 172.21.15.115:0/1079692349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa41011b0 0x7f7fa4101600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.563+0000 7f7fa9e00700 1 --2- 172.21.15.115:0/1079692349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa40fffb0 0x7f7fa41003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.563+0000 7f7fa9e00700 1 -- 172.21.15.115:0/1079692349 >> 172.21.15.115:0/1079692349 conn(0x7f7fa40fb540 msgr2=0x7f7fa40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:44.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.564+0000 7f7fa9e00700 1 -- 172.21.15.115:0/1079692349 shutdown_connections 2024-03-22T10:39:44.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.564+0000 7f7fa9e00700 1 -- 172.21.15.115:0/1079692349 wait complete. 2024-03-22T10:39:44.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.565+0000 7f7fa9e00700 1 Processor -- start 2024-03-22T10:39:44.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.565+0000 7f7fa9e00700 1 -- start start 2024-03-22T10:39:44.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.566+0000 7f7fa9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40fffb0 0x7f7fa4074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:44.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.566+0000 7f7fa9e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa41011b0 0x7f7fa4073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:44.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.566+0000 7f7fa9e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4199410 con 0x7f7fa40fffb0 2024-03-22T10:39:44.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.566+0000 7f7fa9e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4199550 con 0x7f7fa41011b0 2024-03-22T10:39:44.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.566+0000 7f7fa8a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40fffb0 0x7f7fa4074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:44.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.566+0000 7f7fa8a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40fffb0 0x7f7fa4074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44788/0 (socket says 172.21.15.115:44788) 2024-03-22T10:39:44.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.566+0000 7f7fa8a00700 1 -- 172.21.15.115:0/2084865818 learned_addr learned my addr 172.21.15.115:0/2084865818 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:44.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.566+0000 7f7fa3e00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa41011b0 0x7f7fa4073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:44.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.567+0000 7f7fa8a00700 1 -- 172.21.15.115:0/2084865818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa41011b0 msgr2=0x7f7fa4073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:44.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.567+0000 7f7fa8a00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa41011b0 0x7f7fa4073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.567+0000 7f7fa8a00700 1 -- 172.21.15.115:0/2084865818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f900095f0 con 0x7f7fa40fffb0 2024-03-22T10:39:44.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.567+0000 7f7fa3e00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa41011b0 0x7f7fa4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:39:44.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.567+0000 7f7fa8a00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40fffb0 0x7f7fa4074af0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800e8f0 tx=0x7f7f9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:44.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.567+0000 7f7fa1600700 1 -- 172.21.15.115:0/2084865818 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f98009790 con 0x7f7fa40fffb0 2024-03-22T10:39:44.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.567+0000 7f7fa1600700 1 -- 172.21.15.115:0/2084865818 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f980044d0 con 0x7f7fa40fffb0 2024-03-22T10:39:44.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.567+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa4073680 con 0x7f7fa40fffb0 2024-03-22T10:39:44.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.568+0000 7f7fa1600700 1 -- 172.21.15.115:0/2084865818 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f98003da0 con 0x7f7fa40fffb0 2024-03-22T10:39:44.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.568+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa4073ba0 con 0x7f7fa40fffb0 2024-03-22T10:39:44.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.569+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa4073e70 con 0x7f7fa40fffb0 2024-03-22T10:39:44.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.570+0000 7f7fa1600700 1 -- 172.21.15.115:0/2084865818 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7f9800cba0 con 0x7f7fa40fffb0 2024-03-22T10:39:44.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.572+0000 7f7fa1600700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7f8c06c680 0x7f7f8c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:44.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.572+0000 7f7fa1600700 1 -- 172.21.15.115:0/2084865818 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7f98014070 con 0x7f7fa40fffb0 2024-03-22T10:39:44.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.572+0000 7f7fa3e00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7f8c06c680 0x7f7f8c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:44.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.573+0000 7f7fa3e00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7f8c06c680 0x7f7f8c06eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009fd0 tx=0x7f7f90005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:44.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.576+0000 7f7fa1600700 1 -- 172.21.15.115:0/2084865818 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7f98059ef0 con 0x7f7fa40fffb0 2024-03-22T10:39:44.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.857+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fa4105b00 con 0x7f7f8c06c680 2024-03-22T10:39:44.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.859+0000 7f7fa1600700 1 -- 172.21.15.115:0/2084865818 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f7fa4105b00 con 0x7f7f8c06c680 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7f8c06c680 msgr2=0x7f7f8c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7f8c06c680 0x7f7f8c06eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009fd0 tx=0x7f7f90005a90 comp rx=0 tx=0).stop 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40fffb0 msgr2=0x7f7fa4074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40fffb0 0x7f7fa4074af0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800e8f0 tx=0x7f7f9800ecb0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 shutdown_connections 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f7f8c06c680 0x7f7f8c06eb30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40fffb0 0x7f7fa4074af0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 --2- 172.21.15.115:0/2084865818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa41011b0 0x7f7fa4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 >> 172.21.15.115:0/2084865818 conn(0x7f7fa40fb540 msgr2=0x7f7fa41043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 shutdown_connections 2024-03-22T10:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:44.862+0000 7f7fa9e00700 1 -- 172.21.15.115:0/2084865818 wait complete. 2024-03-22T10:39:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.048+0000 7f9ba3800700 1 -- 172.21.15.115:0/1387382424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c100150 msgr2=0x7f9b9c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.048+0000 7f9ba3800700 1 --2- 172.21.15.115:0/1387382424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c100150 0x7f9b9c100560 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009910 tx=0x7f9b90009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.049+0000 7f9ba3800700 1 -- 172.21.15.115:0/1387382424 shutdown_connections 2024-03-22T10:39:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.049+0000 7f9ba3800700 1 --2- 172.21.15.115:0/1387382424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c101350 0x7f9b9c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.049+0000 7f9ba3800700 1 --2- 172.21.15.115:0/1387382424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c100150 0x7f9b9c100560 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.049+0000 7f9ba3800700 1 -- 172.21.15.115:0/1387382424 >> 172.21.15.115:0/1387382424 conn(0x7f9b9c0fb6e0 msgr2=0x7f9b9c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.049+0000 7f9ba3800700 1 -- 172.21.15.115:0/1387382424 shutdown_connections 2024-03-22T10:39:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.049+0000 7f9ba3800700 1 -- 172.21.15.115:0/1387382424 wait complete. 2024-03-22T10:39:45.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.050+0000 7f9ba3800700 1 Processor -- start 2024-03-22T10:39:45.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.050+0000 7f9ba3800700 1 -- start start 2024-03-22T10:39:45.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.051+0000 7f9ba3800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c100150 0x7f9b9c195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.051+0000 7f9ba3800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c101350 0x7f9b9c195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.051+0000 7f9ba3800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c196580 con 0x7f9b9c101350 2024-03-22T10:39:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.051+0000 7f9ba3800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1966c0 con 0x7f9b9c100150 2024-03-22T10:39:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.051+0000 7f9b9be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c101350 0x7f9b9c195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.051+0000 7f9b9be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c101350 0x7f9b9c195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44808/0 (socket says 172.21.15.115:44808) 2024-03-22T10:39:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.051+0000 7f9b9be00700 1 -- 172.21.15.115:0/1739327536 learned_addr learned my addr 172.21.15.115:0/1739327536 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.052+0000 7f9ba0e00700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c100150 0x7f9b9c195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.052+0000 7f9b9be00700 1 -- 172.21.15.115:0/1739327536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c100150 msgr2=0x7f9b9c195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.052+0000 7f9b9be00700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c100150 0x7f9b9c195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.052+0000 7f9b9be00700 1 -- 172.21.15.115:0/1739327536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b900095f0 con 0x7f9b9c101350 2024-03-22T10:39:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.052+0000 7f9b9be00700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c101350 0x7f9b9c195f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800b4b0 tx=0x7f9b8800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:45.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.053+0000 7f9b99600700 1 -- 172.21.15.115:0/1739327536 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b88011800 con 0x7f9b9c101350 2024-03-22T10:39:45.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.053+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c19b170 con 0x7f9b9c101350 2024-03-22T10:39:45.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.053+0000 7f9b99600700 1 -- 172.21.15.115:0/1739327536 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9b88011e10 con 0x7f9b9c101350 2024-03-22T10:39:45.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.053+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c19b740 con 0x7f9b9c101350 2024-03-22T10:39:45.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.053+0000 7f9b99600700 1 -- 172.21.15.115:0/1739327536 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b8800f2e0 con 0x7f9b9c101350 2024-03-22T10:39:45.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.055+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c108d20 con 0x7f9b9c101350 2024-03-22T10:39:45.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.056+0000 7f9b99600700 1 -- 172.21.15.115:0/1739327536 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f9b88010380 con 0x7f9b9c101350 2024-03-22T10:39:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.057+0000 7f9b99600700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b8406c680 0x7f9b8406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:45.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.057+0000 7f9b99600700 1 -- 172.21.15.115:0/1739327536 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f9b8805d810 con 0x7f9b9c101350 2024-03-22T10:39:45.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.057+0000 7f9ba0e00700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b8406c680 0x7f9b8406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:45.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.058+0000 7f9ba0e00700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b8406c680 0x7f9b8406eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b90006010 tx=0x7f9b90005280 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:45.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.065+0000 7f9b99600700 1 -- 172.21.15.115:0/1739327536 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9b880562b0 con 0x7f9b9c101350 2024-03-22T10:39:45.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:45 smithi115 ceph-mon[29111]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.322+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b9c105ca0 con 0x7f9b8406c680 2024-03-22T10:39:45.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.342+0000 7f9b99600700 1 -- 172.21.15.115:0/1739327536 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f9b9c105ca0 con 0x7f9b8406c680 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (6m) 41s ago 8m 13.4M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (8m) 41s ago 8m 7486k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (7m) 46s ago 7m 7398k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (8m) 41s ago 8m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (6m) 46s ago 6m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (6m) 41s ago 7m 73.5M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (72s) 41s ago 71s 14.7M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (63s) 41s ago 62s 11.7M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (59s) 46s ago 59s 11.5M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:39:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (68s) 46s ago 67s 14.2M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:9283,8765,8443 running (9m) 41s ago 9m 488M - 18.2.0 dc2bc1663786 932346df0e0f 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (6m) 46s ago 6m 437M - 18.2.0 dc2bc1663786 3884fe56fbc9 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (10m) 41s ago 10m 44.2M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (6m) 46s ago 6m 33.9M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (8m) 41s ago 8m 7746k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (6m) 46s ago 6m 7591k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (5m) 41s ago 5m 45.7M 3947M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (5m) 41s ago 5m 47.1M 3947M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (4m) 41s ago 4m 46.0M 3947M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (4m) 46s ago 4m 45.6M 4971M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (3m) 46s ago 3m 43.1M 4971M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (2m) 46s ago 2m 42.8M 4971M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:39:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (5m) 41s ago 7m 57.0M - 2.43.0 a07b618ecd1d 9fcacfb2c3cb 2024-03-22T10:39:45.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b8406c680 msgr2=0x7f9b8406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:45.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b8406c680 0x7f9b8406eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b90006010 tx=0x7f9b90005280 comp rx=0 tx=0).stop 2024-03-22T10:39:45.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c101350 msgr2=0x7f9b9c195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:45.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c101350 0x7f9b9c195f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800b4b0 tx=0x7f9b8800b7c0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 shutdown_connections 2024-03-22T10:39:45.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f9b8406c680 0x7f9b8406eb30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c100150 0x7f9b9c195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 --2- 172.21.15.115:0/1739327536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b9c101350 0x7f9b9c195f60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 >> 172.21.15.115:0/1739327536 conn(0x7f9b9c0fb6e0 msgr2=0x7f9b9c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:45.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.346+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 shutdown_connections 2024-03-22T10:39:45.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.347+0000 7f9ba3800700 1 -- 172.21.15.115:0/1739327536 wait complete. 2024-03-22T10:39:45.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:45 smithi150 ceph-mon[34518]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:45.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.573+0000 7f22e7e00700 1 -- 172.21.15.115:0/1426619408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80fffa0 msgr2=0x7f22e81003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:45.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.573+0000 7f22e7e00700 1 --2- 172.21.15.115:0/1426619408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80fffa0 0x7f22e81003b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009910 tx=0x7f22d8009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:45.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.573+0000 7f22e7e00700 1 -- 172.21.15.115:0/1426619408 shutdown_connections 2024-03-22T10:39:45.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.573+0000 7f22e7e00700 1 --2- 172.21.15.115:0/1426619408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e81011a0 0x7f22e81015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.573+0000 7f22e7e00700 1 --2- 172.21.15.115:0/1426619408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80fffa0 0x7f22e81003b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.573+0000 7f22e7e00700 1 -- 172.21.15.115:0/1426619408 >> 172.21.15.115:0/1426619408 conn(0x7f22e80fb510 msgr2=0x7f22e80fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:45.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.573+0000 7f22e7e00700 1 -- 172.21.15.115:0/1426619408 shutdown_connections 2024-03-22T10:39:45.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.573+0000 7f22e7e00700 1 -- 172.21.15.115:0/1426619408 wait complete. 2024-03-22T10:39:45.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.574+0000 7f22e7e00700 1 Processor -- start 2024-03-22T10:39:45.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.574+0000 7f22e7e00700 1 -- start start 2024-03-22T10:39:45.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.575+0000 7f22e7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e80fffa0 0x7f22e8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:45.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.575+0000 7f22e7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e81011a0 0x7f22e8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:45.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.575+0000 7f22e7e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e81985b0 con 0x7f22e81011a0 2024-03-22T10:39:45.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.575+0000 7f22e7e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e81986f0 con 0x7f22e80fffa0 2024-03-22T10:39:45.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.575+0000 7f22e6000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e81011a0 0x7f22e8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:45.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.575+0000 7f22e6000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e81011a0 0x7f22e8197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44822/0 (socket says 172.21.15.115:44822) 2024-03-22T10:39:45.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.575+0000 7f22e6000700 1 -- 172.21.15.115:0/2404900416 learned_addr learned my addr 172.21.15.115:0/2404900416 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:45.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.576+0000 7f22e6a00700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e80fffa0 0x7f22e8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.576+0000 7f22e6000700 1 -- 172.21.15.115:0/2404900416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e80fffa0 msgr2=0x7f22e8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.576+0000 7f22e6000700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e80fffa0 0x7f22e8197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.576+0000 7f22e6000700 1 -- 172.21.15.115:0/2404900416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80095f0 con 0x7f22e81011a0 2024-03-22T10:39:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.576+0000 7f22e6a00700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e80fffa0 0x7f22e8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:39:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.576+0000 7f22e6000700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e81011a0 0x7f22e8197f90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f22d400b4b0 tx=0x7f22d400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.577+0000 7f22df400700 1 -- 172.21.15.115:0/2404900416 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22d4011800 con 0x7f22e81011a0 2024-03-22T10:39:45.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.577+0000 7f22df400700 1 -- 172.21.15.115:0/2404900416 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f22d4011e10 con 0x7f22e81011a0 2024-03-22T10:39:45.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.577+0000 7f22df400700 1 -- 172.21.15.115:0/2404900416 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22d400f2e0 con 0x7f22e81011a0 2024-03-22T10:39:45.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.577+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e819d1a0 con 0x7f22e81011a0 2024-03-22T10:39:45.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.577+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e819d770 con 0x7f22e81011a0 2024-03-22T10:39:45.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.579+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e8105280 con 0x7f22e81011a0 2024-03-22T10:39:45.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.581+0000 7f22df400700 1 -- 172.21.15.115:0/2404900416 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f22d4010380 con 0x7f22e81011a0 2024-03-22T10:39:45.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.582+0000 7f22df400700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f22cc06c680 0x7f22cc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:45.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.582+0000 7f22df400700 1 -- 172.21.15.115:0/2404900416 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f22d408b960 con 0x7f22e81011a0 2024-03-22T10:39:45.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.582+0000 7f22e6a00700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f22cc06c680 0x7f22cc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:45.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.583+0000 7f22e6a00700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f22cc06c680 0x7f22cc06eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22d8000c00 tx=0x7f22d800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:45.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:45.586+0000 7f22df400700 1 -- 172.21.15.115:0/2404900416 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f22d4053720 con 0x7f22e81011a0 2024-03-22T10:39:45.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.004+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f22e802cc30 con 0x7f22e81011a0 2024-03-22T10:39:45.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.005+0000 7f22df400700 1 -- 172.21.15.115:0/2404900416 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f22d4059bf0 con 0x7f22e81011a0 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:39:45.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:39:45.998 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:39:45.998 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-03-22T10:39:45.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:39:45.998 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f22cc06c680 msgr2=0x7f22cc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f22cc06c680 0x7f22cc06eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22d8000c00 tx=0x7f22d800b540 comp rx=0 tx=0).stop 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e81011a0 msgr2=0x7f22e8197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e81011a0 0x7f22e8197f90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f22d400b4b0 tx=0x7f22d400b870 comp rx=0 tx=0).stop 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 shutdown_connections 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f22cc06c680 0x7f22cc06eb30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e80fffa0 0x7f22e8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 --2- 172.21.15.115:0/2404900416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e81011a0 0x7f22e8197f90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 >> 172.21.15.115:0/2404900416 conn(0x7f22e80fb510 msgr2=0x7f22e81023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:46.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 shutdown_connections 2024-03-22T10:39:46.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.008+0000 7f22e7e00700 1 -- 172.21.15.115:0/2404900416 wait complete. 2024-03-22T10:39:46.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.223+0000 7fbf0ee00700 1 -- 172.21.15.115:0/868343668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08101350 msgr2=0x7fbf081017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:46.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.223+0000 7fbf0ee00700 1 --2- 172.21.15.115:0/868343668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08101350 0x7fbf081017a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009910 tx=0x7fbef4009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:46.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.223+0000 7fbf0ee00700 1 -- 172.21.15.115:0/868343668 shutdown_connections 2024-03-22T10:39:46.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.223+0000 7fbf0ee00700 1 --2- 172.21.15.115:0/868343668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08101350 0x7fbf081017a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.223+0000 7fbf0ee00700 1 --2- 172.21.15.115:0/868343668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08100150 0x7fbf08100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.223+0000 7fbf0ee00700 1 -- 172.21.15.115:0/868343668 >> 172.21.15.115:0/868343668 conn(0x7fbf080fb6e0 msgr2=0x7fbf080fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:46.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.223+0000 7fbf0ee00700 1 -- 172.21.15.115:0/868343668 shutdown_connections 2024-03-22T10:39:46.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.223+0000 7fbf0ee00700 1 -- 172.21.15.115:0/868343668 wait complete. 2024-03-22T10:39:46.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.225+0000 7fbf0ee00700 1 Processor -- start 2024-03-22T10:39:46.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.225+0000 7fbf0ee00700 1 -- start start 2024-03-22T10:39:46.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.225+0000 7fbf0ee00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08100150 0x7fbf08195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:46.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.225+0000 7fbf0ee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08101350 0x7fbf08195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:46.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.225+0000 7fbf0ee00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf081964c0 con 0x7fbf08100150 2024-03-22T10:39:46.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.225+0000 7fbf0ee00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf08196600 con 0x7fbf08101350 2024-03-22T10:39:46.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.226+0000 7fbf07e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08100150 0x7fbf08195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:46.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.226+0000 7fbf07e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08100150 0x7fbf08195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44836/0 (socket says 172.21.15.115:44836) 2024-03-22T10:39:46.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.226+0000 7fbf07e00700 1 -- 172.21.15.115:0/3653035332 learned_addr learned my addr 172.21.15.115:0/3653035332 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:46.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.226+0000 7fbf07400700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08101350 0x7fbf08195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:46.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.226+0000 7fbf07e00700 1 -- 172.21.15.115:0/3653035332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08101350 msgr2=0x7fbf08195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:46.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.226+0000 7fbf07e00700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08101350 0x7fbf08195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.226+0000 7fbf07e00700 1 -- 172.21.15.115:0/3653035332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40095f0 con 0x7fbf08100150 2024-03-22T10:39:46.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.227+0000 7fbf07e00700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08100150 0x7fbf08195960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbef800d900 tx=0x7fbef800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:46.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.227+0000 7fbf04c00700 1 -- 172.21.15.115:0/3653035332 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbef80041f0 con 0x7fbf08100150 2024-03-22T10:39:46.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.227+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf080730a0 con 0x7fbf08100150 2024-03-22T10:39:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.227+0000 7fbf04c00700 1 -- 172.21.15.115:0/3653035332 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbef8004350 con 0x7fbf08100150 2024-03-22T10:39:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.227+0000 7fbf04c00700 1 -- 172.21.15.115:0/3653035332 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbef8003da0 con 0x7fbf08100150 2024-03-22T10:39:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.227+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf080735f0 con 0x7fbf08100150 2024-03-22T10:39:46.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.229+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf08108ca0 con 0x7fbf08100150 2024-03-22T10:39:46.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.230+0000 7fbf04c00700 1 -- 172.21.15.115:0/3653035332 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fbef8010460 con 0x7fbf08100150 2024-03-22T10:39:46.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.231+0000 7fbf04c00700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbeec06c680 0x7fbeec06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:46.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.231+0000 7fbf04c00700 1 -- 172.21.15.115:0/3653035332 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbef808b750 con 0x7fbf08100150 2024-03-22T10:39:46.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.231+0000 7fbf07400700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbeec06c680 0x7fbeec06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:46.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.232+0000 7fbf07400700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbeec06c680 0x7fbeec06eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbef4006010 tx=0x7fbef400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:46.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.239+0000 7fbf04c00700 1 -- 172.21.15.115:0/3653035332 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbef8059570 con 0x7fbf08100150 2024-03-22T10:39:46.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:46 smithi115 ceph-mon[29111]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:46.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:46 smithi115 ceph-mon[29111]: pgmap v253: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:39:46.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:46 smithi115 ceph-mon[29111]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:46.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:46 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2404900416' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:39:46.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:46 smithi150 ceph-mon[34518]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:46.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:46 smithi150 ceph-mon[34518]: pgmap v253: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:39:46.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:46 smithi150 ceph-mon[34518]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:46.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:46 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2404900416' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:39:46.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.586+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf08073230 con 0x7fbf08100150 2024-03-22T10:39:46.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.587+0000 7fbf04c00700 1 -- 172.21.15.115:0/3653035332 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1839 (secure 0 0 0) 0x7fbf08073230 con 0x7fbf08100150 2024-03-22T10:39:46.579 INFO:teuthology.orchestra.run.smithi115.stdout:e12 2024-03-22T10:39:46.579 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:39:46.579 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 12 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:38:50.921733+0000 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:39:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:39:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:39:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:39:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:39:46.582 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:39:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:39:46.582 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:39:46.582 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:39:46.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.591+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbeec06c680 msgr2=0x7fbeec06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:46.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.591+0000 7fbf0ee00700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbeec06c680 0x7fbeec06eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbef4006010 tx=0x7fbef400b540 comp rx=0 tx=0).stop 2024-03-22T10:39:46.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.591+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08100150 msgr2=0x7fbf08195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:46.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.591+0000 7fbf0ee00700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08100150 0x7fbf08195960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbef800d900 tx=0x7fbef800dcc0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.591+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 shutdown_connections 2024-03-22T10:39:46.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.591+0000 7fbf0ee00700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fbeec06c680 0x7fbeec06eb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.591+0000 7fbf0ee00700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf08100150 0x7fbf08195960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.591+0000 7fbf0ee00700 1 --2- 172.21.15.115:0/3653035332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08101350 0x7fbf08195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.591+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 >> 172.21.15.115:0/3653035332 conn(0x7fbf080fb6e0 msgr2=0x7fbf08104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:46.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.592+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 shutdown_connections 2024-03-22T10:39:46.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.592+0000 7fbf0ee00700 1 -- 172.21.15.115:0/3653035332 wait complete. 2024-03-22T10:39:46.585 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 12 2024-03-22T10:39:46.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.788+0000 7f64a3000700 1 -- 172.21.15.115:0/2846885702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c101350 msgr2=0x7f649c1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:46.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.788+0000 7f64a3000700 1 --2- 172.21.15.115:0/2846885702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c101350 0x7f649c1017a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6488009910 tx=0x7f6488009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:46.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.788+0000 7f64a3000700 1 -- 172.21.15.115:0/2846885702 shutdown_connections 2024-03-22T10:39:46.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.788+0000 7f64a3000700 1 --2- 172.21.15.115:0/2846885702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c101350 0x7f649c1017a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.788+0000 7f64a3000700 1 --2- 172.21.15.115:0/2846885702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c100150 0x7f649c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.788+0000 7f64a3000700 1 -- 172.21.15.115:0/2846885702 >> 172.21.15.115:0/2846885702 conn(0x7f649c0fb6e0 msgr2=0x7f649c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:46.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.789+0000 7f64a3000700 1 -- 172.21.15.115:0/2846885702 shutdown_connections 2024-03-22T10:39:46.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.789+0000 7f64a3000700 1 -- 172.21.15.115:0/2846885702 wait complete. 2024-03-22T10:39:46.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.790+0000 7f64a3000700 1 Processor -- start 2024-03-22T10:39:46.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.790+0000 7f64a3000700 1 -- start start 2024-03-22T10:39:46.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.791+0000 7f64a3000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c100150 0x7f649c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:46.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.791+0000 7f64a3000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c101350 0x7f649c073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:46.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.791+0000 7f64a3000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c1995b0 con 0x7f649c100150 2024-03-22T10:39:46.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.791+0000 7f64a3000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c1996f0 con 0x7f649c101350 2024-03-22T10:39:46.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.791+0000 7f649be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c100150 0x7f649c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:46.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.791+0000 7f649be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c100150 0x7f649c074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44854/0 (socket says 172.21.15.115:44854) 2024-03-22T10:39:46.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.791+0000 7f649be00700 1 -- 172.21.15.115:0/1242343810 learned_addr learned my addr 172.21.15.115:0/1242343810 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:46.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.792+0000 7f649b400700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c101350 0x7f649c073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:46.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.792+0000 7f649be00700 1 -- 172.21.15.115:0/1242343810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c101350 msgr2=0x7f649c073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:46.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.792+0000 7f649be00700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c101350 0x7f649c073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:46.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.792+0000 7f649be00700 1 -- 172.21.15.115:0/1242343810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880095f0 con 0x7f649c100150 2024-03-22T10:39:46.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.792+0000 7f649be00700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c100150 0x7f649c074af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f648c00d900 tx=0x7f648c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:46.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.793+0000 7f6498c00700 1 -- 172.21.15.115:0/1242343810 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f648c0041f0 con 0x7f649c100150 2024-03-22T10:39:46.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.793+0000 7f6498c00700 1 -- 172.21.15.115:0/1242343810 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f648c004350 con 0x7f649c100150 2024-03-22T10:39:46.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.793+0000 7f6498c00700 1 -- 172.21.15.115:0/1242343810 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f648c003da0 con 0x7f649c100150 2024-03-22T10:39:46.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.793+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649c1999d0 con 0x7f649c100150 2024-03-22T10:39:46.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.793+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c199f20 con 0x7f649c100150 2024-03-22T10:39:46.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.794+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c108ca0 con 0x7f649c100150 2024-03-22T10:39:46.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.796+0000 7f6498c00700 1 -- 172.21.15.115:0/1242343810 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f648c009730 con 0x7f649c100150 2024-03-22T10:39:46.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.797+0000 7f6498c00700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f648006c680 0x7f648006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:46.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.797+0000 7f6498c00700 1 -- 172.21.15.115:0/1242343810 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f648c08ba00 con 0x7f649c100150 2024-03-22T10:39:46.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.797+0000 7f649b400700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f648006c680 0x7f648006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:46.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.799+0000 7f649b400700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f648006c680 0x7f648006eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6488006010 tx=0x7f648800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:46.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:46.805+0000 7f6498c00700 1 -- 172.21.15.115:0/1242343810 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f648c059b50 con 0x7f649c100150 2024-03-22T10:39:47.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.129+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f649c074030 con 0x7f648006c680 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.130+0000 7f6498c00700 1 -- 172.21.15.115:0/1242343810 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f648801e040 con 0x7f648006c680 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [], 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "", 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 image", 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:39:47.123 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:39:47.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.134+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f648006c680 msgr2=0x7f648006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:47.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.134+0000 7f64a3000700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f648006c680 0x7f648006eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6488006010 tx=0x7f648800b540 comp rx=0 tx=0).stop 2024-03-22T10:39:47.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.134+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c100150 msgr2=0x7f649c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:47.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.134+0000 7f64a3000700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c100150 0x7f649c074af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f648c00d900 tx=0x7f648c00dcc0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.134+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 shutdown_connections 2024-03-22T10:39:47.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.134+0000 7f64a3000700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f648006c680 0x7f648006eb30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.134+0000 7f64a3000700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f649c100150 0x7f649c074af0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.134+0000 7f64a3000700 1 --2- 172.21.15.115:0/1242343810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649c101350 0x7f649c073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.134+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 >> 172.21.15.115:0/1242343810 conn(0x7f649c0fb6e0 msgr2=0x7f649c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:47.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.138+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 shutdown_connections 2024-03-22T10:39:47.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.138+0000 7f64a3000700 1 -- 172.21.15.115:0/1242343810 wait complete. 2024-03-22T10:39:47.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.345+0000 7ff062c00700 1 -- 172.21.15.115:0/3245136410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c100be0 msgr2=0x7ff05c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:47.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.345+0000 7ff062c00700 1 --2- 172.21.15.115:0/3245136410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c100be0 0x7ff05c102fc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009910 tx=0x7ff04c009c20 comp rx=0 tx=0).stop 2024-03-22T10:39:47.338 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:47 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3653035332' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:39:47.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.345+0000 7ff062c00700 1 -- 172.21.15.115:0/3245136410 shutdown_connections 2024-03-22T10:39:47.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.345+0000 7ff062c00700 1 --2- 172.21.15.115:0/3245136410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff05c103500 0x7ff05c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.345+0000 7ff062c00700 1 --2- 172.21.15.115:0/3245136410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c100be0 0x7ff05c102fc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.345+0000 7ff062c00700 1 -- 172.21.15.115:0/3245136410 >> 172.21.15.115:0/3245136410 conn(0x7ff05c0fa5e0 msgr2=0x7ff05c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:47.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.345+0000 7ff062c00700 1 -- 172.21.15.115:0/3245136410 shutdown_connections 2024-03-22T10:39:47.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.345+0000 7ff062c00700 1 -- 172.21.15.115:0/3245136410 wait complete. 2024-03-22T10:39:47.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.346+0000 7ff062c00700 1 Processor -- start 2024-03-22T10:39:47.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.347+0000 7ff062c00700 1 -- start start 2024-03-22T10:39:47.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.347+0000 7ff062c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff05c100be0 0x7ff05c071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:47.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.347+0000 7ff062c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c103500 0x7ff05c072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:47.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.347+0000 7ff062c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c072840 con 0x7ff05c103500 2024-03-22T10:39:47.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.347+0000 7ff062c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c072980 con 0x7ff05c100be0 2024-03-22T10:39:47.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.347+0000 7ff05b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c103500 0x7ff05c072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:47.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.347+0000 7ff05b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c103500 0x7ff05c072220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44870/0 (socket says 172.21.15.115:44870) 2024-03-22T10:39:47.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.348+0000 7ff05b400700 1 -- 172.21.15.115:0/970606465 learned_addr learned my addr 172.21.15.115:0/970606465 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:39:47.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.348+0000 7ff05b400700 1 -- 172.21.15.115:0/970606465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff05c100be0 msgr2=0x7ff05c071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.348+0000 7ff05b400700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff05c100be0 0x7ff05c071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.348+0000 7ff05b400700 1 -- 172.21.15.115:0/970606465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0095f0 con 0x7ff05c103500 2024-03-22T10:39:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.348+0000 7ff05b400700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c103500 0x7ff05c072220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff04800c8a0 tx=0x7ff04800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.349+0000 7ff058c00700 1 -- 172.21.15.115:0/970606465 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff04800cea0 con 0x7ff05c103500 2024-03-22T10:39:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.349+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c1a3850 con 0x7ff05c103500 2024-03-22T10:39:47.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.349+0000 7ff058c00700 1 -- 172.21.15.115:0/970606465 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff048004830 con 0x7ff05c103500 2024-03-22T10:39:47.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.349+0000 7ff058c00700 1 -- 172.21.15.115:0/970606465 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0480055b0 con 0x7ff05c103500 2024-03-22T10:39:47.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.349+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c1a3da0 con 0x7ff05c103500 2024-03-22T10:39:47.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.351+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c06bf10 con 0x7ff05c103500 2024-03-22T10:39:47.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.352+0000 7ff058c00700 1 -- 172.21.15.115:0/970606465 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7ff04800f450 con 0x7ff05c103500 2024-03-22T10:39:47.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.353+0000 7ff058c00700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff04006c680 0x7ff04006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:39:47.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.354+0000 7ff058c00700 1 -- 172.21.15.115:0/970606465 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff04808b5e0 con 0x7ff05c103500 2024-03-22T10:39:47.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.355+0000 7ff05be00700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff04006c680 0x7ff04006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:39:47.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.356+0000 7ff05be00700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff04006c680 0x7ff04006eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff04c005090 tx=0x7ff04c005fb0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:39:47.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.359+0000 7ff058c00700 1 -- 172.21.15.115:0/970606465 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff048059730 con 0x7ff05c103500 2024-03-22T10:39:47.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:47 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3653035332' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:39:47.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.831+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff05c02cc30 con 0x7ff05c103500 2024-03-22T10:39:47.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.832+0000 7ff058c00700 1 -- 172.21.15.115:0/970606465 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff0480592c0 con 0x7ff05c103500 2024-03-22T10:39:47.824 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:39:47.824 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:39:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:39:47.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.836+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff04006c680 msgr2=0x7ff04006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:47.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.836+0000 7ff062c00700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff04006c680 0x7ff04006eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff04c005090 tx=0x7ff04c005fb0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.836+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c103500 msgr2=0x7ff05c072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:39:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.836+0000 7ff062c00700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c103500 0x7ff05c072220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff04800c8a0 tx=0x7ff04800cc60 comp rx=0 tx=0).stop 2024-03-22T10:39:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.836+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 shutdown_connections 2024-03-22T10:39:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.836+0000 7ff062c00700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ff04006c680 0x7ff04006eb30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.836+0000 7ff062c00700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff05c100be0 0x7ff05c071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:39:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.836+0000 7ff062c00700 1 --2- 172.21.15.115:0/970606465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05c103500 0x7ff05c072220 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff04800c8a0 tx=0x7ff04800cc60 comp rx=0 tx=0).stop 2024-03-22T10:39:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.836+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 >> 172.21.15.115:0/970606465 conn(0x7ff05c0fa5e0 msgr2=0x7ff05c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:39:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.837+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 shutdown_connections 2024-03-22T10:39:47.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:39:47.837+0000 7ff062c00700 1 -- 172.21.15.115:0/970606465 wait complete. 2024-03-22T10:39:48.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:48 smithi115 ceph-mon[29111]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:48.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:48 smithi115 ceph-mon[29111]: pgmap v254: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:39:48.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:48 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/970606465' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:39:48.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:48 smithi150 ceph-mon[34518]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:39:48.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:48 smithi150 ceph-mon[34518]: pgmap v254: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:39:48.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:48 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/970606465' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:39:50.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:50 smithi150 ceph-mon[34518]: pgmap v255: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:50.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:50 smithi115 ceph-mon[29111]: pgmap v255: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:52.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:52 smithi150 ceph-mon[34518]: pgmap v256: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:52.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:52 smithi115 ceph-mon[29111]: pgmap v256: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:54.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:54 smithi150 ceph-mon[34518]: pgmap v257: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:54.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:54 smithi115 ceph-mon[29111]: pgmap v257: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:55.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:39:55.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:39:56.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:56 smithi150 ceph-mon[34518]: pgmap v258: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:56.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:56 smithi115 ceph-mon[29111]: pgmap v258: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:58.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:39:58 smithi150 ceph-mon[34518]: pgmap v259: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:39:58.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:39:58 smithi115 ceph-mon[29111]: pgmap v259: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:00.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:00 smithi150 ceph-mon[34518]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:00.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:00 smithi150 ceph-mon[34518]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:40:00.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:00 smithi150 ceph-mon[34518]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:40:00.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:00 smithi150 ceph-mon[34518]: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:40:00.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:00 smithi115 ceph-mon[29111]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:00.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:00 smithi115 ceph-mon[29111]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:40:00.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:00 smithi115 ceph-mon[29111]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:40:00.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:00 smithi115 ceph-mon[29111]: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:40:02.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:02 smithi150 ceph-mon[34518]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:02.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:02 smithi115 ceph-mon[29111]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:04.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:04 smithi150 ceph-mon[34518]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:04.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:04 smithi115 ceph-mon[29111]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:06.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:06 smithi150 ceph-mon[34518]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:06.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:06 smithi115 ceph-mon[29111]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:08.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:08 smithi150 ceph-mon[34518]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:08.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:08 smithi115 ceph-mon[29111]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:10.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:10 smithi150 ceph-mon[34518]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:10.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:40:10.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:10 smithi115 ceph-mon[29111]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:10.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:40:12.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:12 smithi150 ceph-mon[34518]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:12.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:12 smithi115 ceph-mon[29111]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:14.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:14 smithi150 ceph-mon[34518]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:14.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:14 smithi115 ceph-mon[29111]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:16.630 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:16 smithi115 ceph-mon[29111]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:16.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:16 smithi150 ceph-mon[34518]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.056+0000 7fb43fe00700 1 -- 172.21.15.115:0/1090301773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb438103500 msgr2=0x7fb4381058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.056+0000 7fb43fe00700 1 --2- 172.21.15.115:0/1090301773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb438103500 0x7fb4381058e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb424009910 tx=0x7fb424009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.056+0000 7fb43fe00700 1 -- 172.21.15.115:0/1090301773 shutdown_connections 2024-03-22T10:40:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.056+0000 7fb43fe00700 1 --2- 172.21.15.115:0/1090301773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb438103500 0x7fb4381058e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.056+0000 7fb43fe00700 1 --2- 172.21.15.115:0/1090301773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438100be0 0x7fb438102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.056+0000 7fb43fe00700 1 -- 172.21.15.115:0/1090301773 >> 172.21.15.115:0/1090301773 conn(0x7fb4380fa5e0 msgr2=0x7fb4380fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.056+0000 7fb43fe00700 1 -- 172.21.15.115:0/1090301773 shutdown_connections 2024-03-22T10:40:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.056+0000 7fb43fe00700 1 -- 172.21.15.115:0/1090301773 wait complete. 2024-03-22T10:40:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.057+0000 7fb43fe00700 1 Processor -- start 2024-03-22T10:40:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.058+0000 7fb43fe00700 1 -- start start 2024-03-22T10:40:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.058+0000 7fb43fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438100be0 0x7fb438197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.058+0000 7fb43fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb438103500 0x7fb438198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.058+0000 7fb43fe00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4381986a0 con 0x7fb438103500 2024-03-22T10:40:18.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.058+0000 7fb43fe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4381987e0 con 0x7fb438100be0 2024-03-22T10:40:18.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.059+0000 7fb43d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438100be0 0x7fb438197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:18.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.059+0000 7fb43d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438100be0 0x7fb438197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:54000/0 (socket says 172.21.15.115:54000) 2024-03-22T10:40:18.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.059+0000 7fb43d400700 1 -- 172.21.15.115:0/1333213570 learned_addr learned my addr 172.21.15.115:0/1333213570 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:18.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.059+0000 7fb43ca00700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb438103500 0x7fb438198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:18.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.059+0000 7fb43ca00700 1 -- 172.21.15.115:0/1333213570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438100be0 msgr2=0x7fb438197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:18.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.059+0000 7fb43ca00700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438100be0 0x7fb438197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.059+0000 7fb43ca00700 1 -- 172.21.15.115:0/1333213570 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4240095f0 con 0x7fb438103500 2024-03-22T10:40:18.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.059+0000 7fb43ca00700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb438103500 0x7fb438198080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb424005b40 tx=0x7fb42400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:18.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.060+0000 7fb436000700 1 -- 172.21.15.115:0/1333213570 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb42401d070 con 0x7fb438103500 2024-03-22T10:40:18.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.060+0000 7fb436000700 1 -- 172.21.15.115:0/1333213570 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb424005dc0 con 0x7fb438103500 2024-03-22T10:40:18.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.060+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43819d230 con 0x7fb438103500 2024-03-22T10:40:18.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.060+0000 7fb436000700 1 -- 172.21.15.115:0/1333213570 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb424022470 con 0x7fb438103500 2024-03-22T10:40:18.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.060+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43819d720 con 0x7fb438103500 2024-03-22T10:40:18.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.062+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb418005320 con 0x7fb438103500 2024-03-22T10:40:18.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.063+0000 7fb436000700 1 -- 172.21.15.115:0/1333213570 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb4240050d0 con 0x7fb438103500 2024-03-22T10:40:18.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.064+0000 7fb436000700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb42006c6d0 0x7fb42006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:18.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.065+0000 7fb436000700 1 -- 172.21.15.115:0/1333213570 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb42408cb50 con 0x7fb438103500 2024-03-22T10:40:18.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.065+0000 7fb43d400700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb42006c6d0 0x7fb42006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:18.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.066+0000 7fb43d400700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb42006c6d0 0x7fb42006eb80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009d20 tx=0x7fb42c009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.073+0000 7fb436000700 1 -- 172.21.15.115:0/1333213570 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb42405ade0 con 0x7fb438103500 2024-03-22T10:40:18.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.301+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb418000bf0 con 0x7fb42006c6d0 2024-03-22T10:40:18.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.302+0000 7fb436000700 1 -- 172.21.15.115:0/1333213570 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fb418000bf0 con 0x7fb42006c6d0 2024-03-22T10:40:18.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.305+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb42006c6d0 msgr2=0x7fb42006eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:18.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.305+0000 7fb43fe00700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb42006c6d0 0x7fb42006eb80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009d20 tx=0x7fb42c009260 comp rx=0 tx=0).stop 2024-03-22T10:40:18.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.305+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb438103500 msgr2=0x7fb438198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:18.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.305+0000 7fb43fe00700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb438103500 0x7fb438198080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb424005b40 tx=0x7fb42400fa00 comp rx=0 tx=0).stop 2024-03-22T10:40:18.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.306+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 shutdown_connections 2024-03-22T10:40:18.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.306+0000 7fb43fe00700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb42006c6d0 0x7fb42006eb80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.306+0000 7fb43fe00700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438100be0 0x7fb438197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.306+0000 7fb43fe00700 1 --2- 172.21.15.115:0/1333213570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb438103500 0x7fb438198080 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.306+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 >> 172.21.15.115:0/1333213570 conn(0x7fb4380fa5e0 msgr2=0x7fb438104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:18.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.306+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 shutdown_connections 2024-03-22T10:40:18.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.306+0000 7fb43fe00700 1 -- 172.21.15.115:0/1333213570 wait complete. 2024-03-22T10:40:18.317 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:40:18.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.518+0000 7f15c6600700 1 -- 172.21.15.115:0/673545762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c0074d80 msgr2=0x7f15c00731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:18.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.518+0000 7f15c6600700 1 --2- 172.21.15.115:0/673545762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c0074d80 0x7f15c00731e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009940 tx=0x7f15b0009c50 comp rx=0 tx=0).stop 2024-03-22T10:40:18.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.518+0000 7f15c6600700 1 -- 172.21.15.115:0/673545762 shutdown_connections 2024-03-22T10:40:18.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.518+0000 7f15c6600700 1 --2- 172.21.15.115:0/673545762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c00737b0 0x7f15c0073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.518+0000 7f15c6600700 1 --2- 172.21.15.115:0/673545762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c0074d80 0x7f15c00731e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.518+0000 7f15c6600700 1 -- 172.21.15.115:0/673545762 >> 172.21.15.115:0/673545762 conn(0x7f15c00fb6e0 msgr2=0x7f15c00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:18.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.518+0000 7f15c6600700 1 -- 172.21.15.115:0/673545762 shutdown_connections 2024-03-22T10:40:18.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.518+0000 7f15c6600700 1 -- 172.21.15.115:0/673545762 wait complete. 2024-03-22T10:40:18.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.519+0000 7f15c6600700 1 Processor -- start 2024-03-22T10:40:18.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.519+0000 7f15c6600700 1 -- start start 2024-03-22T10:40:18.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15c6600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c00737b0 0x7f15c0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:18.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15c6600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c0074d80 0x7f15c0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:18.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15c6600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0198670 con 0x7f15c00737b0 2024-03-22T10:40:18.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15c6600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01987b0 con 0x7f15c0074d80 2024-03-22T10:40:18.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15bf400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c00737b0 0x7f15c0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:18.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15bf400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c00737b0 0x7f15c0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58150/0 (socket says 172.21.15.115:58150) 2024-03-22T10:40:18.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15bea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c0074d80 0x7f15c0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:18.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15bf400700 1 -- 172.21.15.115:0/2094821423 learned_addr learned my addr 172.21.15.115:0/2094821423 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:18.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15bf400700 1 -- 172.21.15.115:0/2094821423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c0074d80 msgr2=0x7f15c0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:18.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15bf400700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c0074d80 0x7f15c0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.520+0000 7f15bf400700 1 -- 172.21.15.115:0/2094821423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b00095f0 con 0x7f15c00737b0 2024-03-22T10:40:18.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.521+0000 7f15bf400700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c00737b0 0x7f15c0197b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f15b000be80 tx=0x7f15b000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:18.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.521+0000 7f15b7e00700 1 -- 172.21.15.115:0/2094821423 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15b001d070 con 0x7f15c00737b0 2024-03-22T10:40:18.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.521+0000 7f15b7e00700 1 -- 172.21.15.115:0/2094821423 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f15b000f460 con 0x7f15c00737b0 2024-03-22T10:40:18.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.521+0000 7f15b7e00700 1 -- 172.21.15.115:0/2094821423 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15b0021640 con 0x7f15c00737b0 2024-03-22T10:40:18.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.521+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c019d200 con 0x7f15c00737b0 2024-03-22T10:40:18.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.521+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c019d6f0 con 0x7f15c00737b0 2024-03-22T10:40:18.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.523+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c010ae90 con 0x7f15c00737b0 2024-03-22T10:40:18.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.525+0000 7f15b7e00700 1 -- 172.21.15.115:0/2094821423 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f15b000f5d0 con 0x7f15c00737b0 2024-03-22T10:40:18.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.526+0000 7f15b7e00700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f15a406c680 0x7f15a406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:18.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.526+0000 7f15b7e00700 1 -- 172.21.15.115:0/2094821423 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f15b008ce40 con 0x7f15c00737b0 2024-03-22T10:40:18.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.526+0000 7f15bea00700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f15a406c680 0x7f15a406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:18.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.527+0000 7f15bea00700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f15a406c680 0x7f15a406eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009c60 tx=0x7f15ac009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.530+0000 7f15b7e00700 1 -- 172.21.15.115:0/2094821423 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f15b005b0d0 con 0x7f15c00737b0 2024-03-22T10:40:18.565 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:18 smithi115 ceph-mon[29111]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:18.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:18 smithi150 ceph-mon[34518]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:18.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.876+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15c0107650 con 0x7f15a406c680 2024-03-22T10:40:18.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.878+0000 7f15b7e00700 1 -- 172.21.15.115:0/2094821423 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f15c0107650 con 0x7f15a406c680 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f15a406c680 msgr2=0x7f15a406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f15a406c680 0x7f15a406eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009c60 tx=0x7f15ac009260 comp rx=0 tx=0).stop 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c00737b0 msgr2=0x7f15c0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c00737b0 0x7f15c0197b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f15b000be80 tx=0x7f15b000beb0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 shutdown_connections 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f15a406c680 0x7f15a406eb30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c00737b0 0x7f15c0197b10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 --2- 172.21.15.115:0/2094821423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c0074d80 0x7f15c0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 >> 172.21.15.115:0/2094821423 conn(0x7f15c00fb6e0 msgr2=0x7f15c0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.880+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 shutdown_connections 2024-03-22T10:40:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:18.881+0000 7f15c6600700 1 -- 172.21.15.115:0/2094821423 wait complete. 2024-03-22T10:40:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.041+0000 7f752c800700 1 -- 172.21.15.115:0/2697015289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7524071910 msgr2=0x7f7524071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.041+0000 7f752c800700 1 --2- 172.21.15.115:0/2697015289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7524071910 0x7f7524071d20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7518009910 tx=0x7f7518009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.041+0000 7f752c800700 1 -- 172.21.15.115:0/2697015289 shutdown_connections 2024-03-22T10:40:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.041+0000 7f752c800700 1 --2- 172.21.15.115:0/2697015289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75240722f0 0x7f752410bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.041+0000 7f752c800700 1 --2- 172.21.15.115:0/2697015289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7524071910 0x7f7524071d20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.041+0000 7f752c800700 1 -- 172.21.15.115:0/2697015289 >> 172.21.15.115:0/2697015289 conn(0x7f752406d050 msgr2=0x7f752406f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.042+0000 7f752c800700 1 -- 172.21.15.115:0/2697015289 shutdown_connections 2024-03-22T10:40:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.042+0000 7f752c800700 1 -- 172.21.15.115:0/2697015289 wait complete. 2024-03-22T10:40:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.043+0000 7f752c800700 1 Processor -- start 2024-03-22T10:40:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.043+0000 7f752c800700 1 -- start start 2024-03-22T10:40:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.044+0000 7f752c800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7524071910 0x7f75241a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.044+0000 7f752c800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75240722f0 0x7f75241a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.045+0000 7f7529e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7524071910 0x7f75241a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.045+0000 7f7529400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75240722f0 0x7f75241a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.045+0000 7f752c800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75241a52d0 con 0x7f75240722f0 2024-03-22T10:40:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.045+0000 7f7529400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75240722f0 0x7f75241a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58178/0 (socket says 172.21.15.115:58178) 2024-03-22T10:40:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.045+0000 7f7529400700 1 -- 172.21.15.115:0/4077137317 learned_addr learned my addr 172.21.15.115:0/4077137317 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.045+0000 7f7529e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7524071910 0x7f75241a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:54036/0 (socket says 172.21.15.115:54036) 2024-03-22T10:40:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.045+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75241a5410 con 0x7f7524071910 2024-03-22T10:40:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.046+0000 7f7529400700 1 -- 172.21.15.115:0/4077137317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7524071910 msgr2=0x7f75241a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.046+0000 7f7529400700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7524071910 0x7f75241a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.046+0000 7f7529400700 1 -- 172.21.15.115:0/4077137317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7514009520 con 0x7f75240722f0 2024-03-22T10:40:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.046+0000 7f7529400700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75240722f0 0x7f75241a4cb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f751400eab0 tx=0x7f751400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.046+0000 7f751ea00700 1 -- 172.21.15.115:0/4077137317 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f751400cba0 con 0x7f75240722f0 2024-03-22T10:40:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.047+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75180095f0 con 0x7f75240722f0 2024-03-22T10:40:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.047+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75241aa280 con 0x7f75240722f0 2024-03-22T10:40:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.047+0000 7f751ea00700 1 -- 172.21.15.115:0/4077137317 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f75140044d0 con 0x7f75240722f0 2024-03-22T10:40:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.047+0000 7f751ea00700 1 -- 172.21.15.115:0/4077137317 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7514005020 con 0x7f75240722f0 2024-03-22T10:40:19.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.048+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7508005320 con 0x7f75240722f0 2024-03-22T10:40:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.051+0000 7f751ea00700 1 -- 172.21.15.115:0/4077137317 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f751400cd00 con 0x7f75240722f0 2024-03-22T10:40:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.051+0000 7f751ea00700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f750c06c680 0x7f750c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.052+0000 7f751ea00700 1 -- 172.21.15.115:0/4077137317 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7514014070 con 0x7f75240722f0 2024-03-22T10:40:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.052+0000 7f7529e00700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f750c06c680 0x7f750c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:19.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.053+0000 7f7529e00700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f750c06c680 0x7f750c06eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7518009fd0 tx=0x7f7518005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.060+0000 7f751ea00700 1 -- 172.21.15.115:0/4077137317 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7514059780 con 0x7f75240722f0 2024-03-22T10:40:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.442+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7508000bf0 con 0x7f750c06c680 2024-03-22T10:40:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.453+0000 7f751ea00700 1 -- 172.21.15.115:0/4077137317 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7f7508000bf0 con 0x7f750c06c680 2024-03-22T10:40:19.446 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (6m) 75s ago 8m 13.4M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (9m) 75s ago 9m 7486k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (7m) 80s ago 7m 7398k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (8m) 75s ago 8m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (7m) 80s ago 7m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (6m) 75s ago 8m 73.5M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (106s) 75s ago 105s 14.7M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (97s) 75s ago 97s 11.7M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (93s) 80s ago 93s 11.5M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (102s) 80s ago 101s 14.2M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:40:19.447 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:9283,8765,8443 running (10m) 75s ago 10m 488M - 18.2.0 dc2bc1663786 932346df0e0f 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (7m) 80s ago 7m 437M - 18.2.0 dc2bc1663786 3884fe56fbc9 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (10m) 75s ago 10m 44.2M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (7m) 80s ago 7m 33.9M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (8m) 75s ago 8m 7746k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (7m) 80s ago 7m 7591k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (6m) 75s ago 6m 45.7M 3947M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (5m) 75s ago 5m 47.1M 3947M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (5m) 75s ago 5m 46.0M 3947M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (4m) 80s ago 4m 45.6M 4971M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (4m) 80s ago 4m 43.1M 4971M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (3m) 80s ago 3m 42.8M 4971M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:40:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (6m) 75s ago 8m 57.0M - 2.43.0 a07b618ecd1d 9fcacfb2c3cb 2024-03-22T10:40:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.456+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f750c06c680 msgr2=0x7f750c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.456+0000 7f752c800700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f750c06c680 0x7f750c06eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7518009fd0 tx=0x7f7518005fd0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.456+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75240722f0 msgr2=0x7f75241a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.456+0000 7f752c800700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75240722f0 0x7f75241a4cb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f751400eab0 tx=0x7f751400edc0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.457+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 shutdown_connections 2024-03-22T10:40:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.457+0000 7f752c800700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f750c06c680 0x7f750c06eb30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.457+0000 7f752c800700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7524071910 0x7f75241a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.457+0000 7f752c800700 1 --2- 172.21.15.115:0/4077137317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75240722f0 0x7f75241a4cb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.457+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 >> 172.21.15.115:0/4077137317 conn(0x7f752406d050 msgr2=0x7f752410b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.457+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 shutdown_connections 2024-03-22T10:40:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.457+0000 7f752c800700 1 -- 172.21.15.115:0/4077137317 wait complete. 2024-03-22T10:40:19.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.647+0000 7fdf26800700 1 -- 172.21.15.115:0/3380276965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20100250 msgr2=0x7fdf201006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:19.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.647+0000 7fdf26800700 1 --2- 172.21.15.115:0/3380276965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20100250 0x7fdf201006a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009910 tx=0x7fdf0c009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:19.641 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:19 smithi115 ceph-mon[29111]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:19.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.648+0000 7fdf26800700 1 -- 172.21.15.115:0/3380276965 shutdown_connections 2024-03-22T10:40:19.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.648+0000 7fdf26800700 1 --2- 172.21.15.115:0/3380276965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20100250 0x7fdf201006a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.648+0000 7fdf26800700 1 --2- 172.21.15.115:0/3380276965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf200ff050 0x7fdf200ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.648+0000 7fdf26800700 1 -- 172.21.15.115:0/3380276965 >> 172.21.15.115:0/3380276965 conn(0x7fdf200fa5e0 msgr2=0x7fdf200fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:19.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.648+0000 7fdf26800700 1 -- 172.21.15.115:0/3380276965 shutdown_connections 2024-03-22T10:40:19.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.648+0000 7fdf26800700 1 -- 172.21.15.115:0/3380276965 wait complete. 2024-03-22T10:40:19.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.649+0000 7fdf26800700 1 Processor -- start 2024-03-22T10:40:19.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.650+0000 7fdf26800700 1 -- start start 2024-03-22T10:40:19.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.650+0000 7fdf26800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf200ff050 0x7fdf20102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:19.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.650+0000 7fdf26800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100250 0x7fdf20101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:19.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.650+0000 7fdf26800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf20101b70 con 0x7fdf200ff050 2024-03-22T10:40:19.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.650+0000 7fdf26800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf20101cb0 con 0x7fdf20100250 2024-03-22T10:40:19.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.650+0000 7fdf1f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf200ff050 0x7fdf20102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:19.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.650+0000 7fdf1f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf200ff050 0x7fdf20102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58192/0 (socket says 172.21.15.115:58192) 2024-03-22T10:40:19.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.650+0000 7fdf1f400700 1 -- 172.21.15.115:0/147970067 learned_addr learned my addr 172.21.15.115:0/147970067 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:19.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.650+0000 7fdf1ea00700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100250 0x7fdf20101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:19.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.651+0000 7fdf1f400700 1 -- 172.21.15.115:0/147970067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100250 msgr2=0x7fdf20101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:19.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.651+0000 7fdf1f400700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100250 0x7fdf20101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:19.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.651+0000 7fdf1f400700 1 -- 172.21.15.115:0/147970067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0095f0 con 0x7fdf200ff050 2024-03-22T10:40:19.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.651+0000 7fdf1f400700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf200ff050 0x7fdf20102fe0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e8f0 tx=0x7fdf1000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:19.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.652+0000 7fdf17e00700 1 -- 172.21.15.115:0/147970067 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf10009790 con 0x7fdf200ff050 2024-03-22T10:40:19.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.652+0000 7fdf17e00700 1 -- 172.21.15.115:0/147970067 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf1000c810 con 0x7fdf200ff050 2024-03-22T10:40:19.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.652+0000 7fdf17e00700 1 -- 172.21.15.115:0/147970067 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf10003d20 con 0x7fdf200ff050 2024-03-22T10:40:19.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.652+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf20101f90 con 0x7fdf200ff050 2024-03-22T10:40:19.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.652+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf20102480 con 0x7fdf200ff050 2024-03-22T10:40:19.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.654+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf20107ba0 con 0x7fdf200ff050 2024-03-22T10:40:19.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.654+0000 7fdf17e00700 1 -- 172.21.15.115:0/147970067 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fdf100041f0 con 0x7fdf200ff050 2024-03-22T10:40:19.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.655+0000 7fdf17e00700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdf0406c680 0x7fdf0406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:19.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:19 smithi150 ceph-mon[34518]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:19.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.656+0000 7fdf17e00700 1 -- 172.21.15.115:0/147970067 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fdf10014070 con 0x7fdf200ff050 2024-03-22T10:40:19.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.656+0000 7fdf1ea00700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdf0406c680 0x7fdf0406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:19.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.657+0000 7fdf1ea00700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdf0406c680 0x7fdf0406eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009fd0 tx=0x7fdf0c005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:19.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:19.663+0000 7fdf17e00700 1 -- 172.21.15.115:0/147970067 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdf10059d80 con 0x7fdf200ff050 2024-03-22T10:40:20.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.098+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf2019ded0 con 0x7fdf200ff050 2024-03-22T10:40:20.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.099+0000 7fdf17e00700 1 -- 172.21.15.115:0/147970067 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fdf10059910 con 0x7fdf200ff050 2024-03-22T10:40:20.091 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:40:20.091 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:40:20.092 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.101+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdf0406c680 msgr2=0x7fdf0406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.101+0000 7fdf26800700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdf0406c680 0x7fdf0406eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009fd0 tx=0x7fdf0c005fd0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.101+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf200ff050 msgr2=0x7fdf20102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.101+0000 7fdf26800700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf200ff050 0x7fdf20102fe0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e8f0 tx=0x7fdf1000ec00 comp rx=0 tx=0).stop 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.102+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 shutdown_connections 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.102+0000 7fdf26800700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fdf0406c680 0x7fdf0406eb30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.102+0000 7fdf26800700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf200ff050 0x7fdf20102fe0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.102+0000 7fdf26800700 1 --2- 172.21.15.115:0/147970067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100250 0x7fdf20101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.102+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 >> 172.21.15.115:0/147970067 conn(0x7fdf200fa5e0 msgr2=0x7fdf200683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.102+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 shutdown_connections 2024-03-22T10:40:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.102+0000 7fdf26800700 1 -- 172.21.15.115:0/147970067 wait complete. 2024-03-22T10:40:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.267+0000 7f2aeee00700 1 -- 172.21.15.115:0/2759150130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae80fffb0 msgr2=0x7f2ae81003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.267+0000 7f2aeee00700 1 --2- 172.21.15.115:0/2759150130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae80fffb0 0x7f2ae81003c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009910 tx=0x7f2adc009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.267+0000 7f2aeee00700 1 -- 172.21.15.115:0/2759150130 shutdown_connections 2024-03-22T10:40:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.267+0000 7f2aeee00700 1 --2- 172.21.15.115:0/2759150130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae81011b0 0x7f2ae8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.267+0000 7f2aeee00700 1 --2- 172.21.15.115:0/2759150130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae80fffb0 0x7f2ae81003c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.267+0000 7f2aeee00700 1 -- 172.21.15.115:0/2759150130 >> 172.21.15.115:0/2759150130 conn(0x7f2ae80fb560 msgr2=0x7f2ae80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.267+0000 7f2aeee00700 1 -- 172.21.15.115:0/2759150130 shutdown_connections 2024-03-22T10:40:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.267+0000 7f2aeee00700 1 -- 172.21.15.115:0/2759150130 wait complete. 2024-03-22T10:40:20.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.269+0000 7f2aeee00700 1 Processor -- start 2024-03-22T10:40:20.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.269+0000 7f2aeee00700 1 -- start start 2024-03-22T10:40:20.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.269+0000 7f2aeee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae80fffb0 0x7f2ae8195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:20.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.269+0000 7f2aeee00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae81011b0 0x7f2ae8195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:20.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.269+0000 7f2aeee00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8196390 con 0x7f2ae81011b0 2024-03-22T10:40:20.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.269+0000 7f2aeee00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae81964d0 con 0x7f2ae80fffb0 2024-03-22T10:40:20.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.270+0000 7f2aed000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae81011b0 0x7f2ae8195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:20.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.270+0000 7f2aeda00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae80fffb0 0x7f2ae8195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:20.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.270+0000 7f2aed000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae81011b0 0x7f2ae8195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58212/0 (socket says 172.21.15.115:58212) 2024-03-22T10:40:20.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.270+0000 7f2aed000700 1 -- 172.21.15.115:0/1350848780 learned_addr learned my addr 172.21.15.115:0/1350848780 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:20.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.270+0000 7f2aed000700 1 -- 172.21.15.115:0/1350848780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae80fffb0 msgr2=0x7f2ae8195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:20.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.270+0000 7f2aed000700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae80fffb0 0x7f2ae8195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.270+0000 7f2aed000700 1 -- 172.21.15.115:0/1350848780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0095f0 con 0x7f2ae81011b0 2024-03-22T10:40:20.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.271+0000 7f2aed000700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae81011b0 0x7f2ae8195d70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400b4b0 tx=0x7f2ad400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:20.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.271+0000 7f2ae6000700 1 -- 172.21.15.115:0/1350848780 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ad4011800 con 0x7f2ae81011b0 2024-03-22T10:40:20.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.271+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae819af80 con 0x7f2ae81011b0 2024-03-22T10:40:20.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.271+0000 7f2ae6000700 1 -- 172.21.15.115:0/1350848780 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2ad4011e10 con 0x7f2ae81011b0 2024-03-22T10:40:20.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.271+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae819b520 con 0x7f2ae81011b0 2024-03-22T10:40:20.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.272+0000 7f2ae6000700 1 -- 172.21.15.115:0/1350848780 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ad4011960 con 0x7f2ae81011b0 2024-03-22T10:40:20.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.273+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac8005320 con 0x7f2ae81011b0 2024-03-22T10:40:20.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.275+0000 7f2ae6000700 1 -- 172.21.15.115:0/1350848780 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f2ad4024440 con 0x7f2ae81011b0 2024-03-22T10:40:20.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.277+0000 7f2ae6000700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f2ad006c680 0x7f2ad006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:20.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.277+0000 7f2ae6000700 1 -- 172.21.15.115:0/1350848780 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2ad408b990 con 0x7f2ae81011b0 2024-03-22T10:40:20.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.277+0000 7f2aeda00700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f2ad006c680 0x7f2ad006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:20.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.278+0000 7f2aeda00700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f2ad006c680 0x7f2ad006eb30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc00b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:20.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.280+0000 7f2ae6000700 1 -- 172.21.15.115:0/1350848780 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2ad4053720 con 0x7f2ae81011b0 2024-03-22T10:40:20.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:20 smithi115 ceph-mon[29111]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:20.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:20 smithi115 ceph-mon[29111]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:20.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:20 smithi115 ceph-mon[29111]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:20.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:20 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:40:20.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:20 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:40:20.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:20 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:40:20.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:20 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/147970067' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:40:20.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.641+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2ac8005cc0 con 0x7f2ae81011b0 2024-03-22T10:40:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.642+0000 7f2ae6000700 1 -- 172.21.15.115:0/1350848780 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1839 (secure 0 0 0) 0x7f2ad4015740 con 0x7f2ae81011b0 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:e12 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 12 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:38:50.921733+0000 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:40:20.635 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:40:20.636 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:40:20.637 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:40:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f2ad006c680 msgr2=0x7f2ad006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f2ad006c680 0x7f2ad006eb30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc00b560 comp rx=0 tx=0).stop 2024-03-22T10:40:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae81011b0 msgr2=0x7f2ae8195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae81011b0 0x7f2ae8195d70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400b4b0 tx=0x7f2ad400b7c0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 shutdown_connections 2024-03-22T10:40:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f2ad006c680 0x7f2ad006eb30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae80fffb0 0x7f2ae8195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 --2- 172.21.15.115:0/1350848780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ae81011b0 0x7f2ae8195d70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 >> 172.21.15.115:0/1350848780 conn(0x7f2ae80fb560 msgr2=0x7f2ae81043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:20.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 shutdown_connections 2024-03-22T10:40:20.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.645+0000 7f2aeee00700 1 -- 172.21.15.115:0/1350848780 wait complete. 2024-03-22T10:40:20.639 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 12 2024-03-22T10:40:20.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:20 smithi150 ceph-mon[34518]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:20.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:20 smithi150 ceph-mon[34518]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:20.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:20 smithi150 ceph-mon[34518]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:20.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:20 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:40:20.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:20 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:40:20.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:20 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:40:20.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:20 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/147970067' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:40:20.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.845+0000 7f1a5e400700 1 -- 172.21.15.115:0/350550371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a58100be0 msgr2=0x7f1a58102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:20.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.845+0000 7f1a5e400700 1 --2- 172.21.15.115:0/350550371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a58100be0 0x7f1a58102fc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009910 tx=0x7f1a48009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:20.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.845+0000 7f1a5e400700 1 -- 172.21.15.115:0/350550371 shutdown_connections 2024-03-22T10:40:20.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.845+0000 7f1a5e400700 1 --2- 172.21.15.115:0/350550371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a58103500 0x7f1a581058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.845+0000 7f1a5e400700 1 --2- 172.21.15.115:0/350550371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a58100be0 0x7f1a58102fc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.845+0000 7f1a5e400700 1 -- 172.21.15.115:0/350550371 >> 172.21.15.115:0/350550371 conn(0x7f1a580fa5e0 msgr2=0x7f1a580fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:20.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.845+0000 7f1a5e400700 1 -- 172.21.15.115:0/350550371 shutdown_connections 2024-03-22T10:40:20.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.845+0000 7f1a5e400700 1 -- 172.21.15.115:0/350550371 wait complete. 2024-03-22T10:40:20.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.846+0000 7f1a5e400700 1 Processor -- start 2024-03-22T10:40:20.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.847+0000 7f1a5e400700 1 -- start start 2024-03-22T10:40:20.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.847+0000 7f1a5e400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a58100be0 0x7f1a58197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:20.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.847+0000 7f1a5e400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a58103500 0x7f1a58198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.847+0000 7f1a5e400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a58198660 con 0x7f1a58100be0 2024-03-22T10:40:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.847+0000 7f1a5e400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a581987a0 con 0x7f1a58103500 2024-03-22T10:40:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.848+0000 7f1a57400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a58100be0 0x7f1a58197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.848+0000 7f1a56a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a58103500 0x7f1a58198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.848+0000 7f1a56a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a58103500 0x7f1a58198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:54070/0 (socket says 172.21.15.115:54070) 2024-03-22T10:40:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.848+0000 7f1a56a00700 1 -- 172.21.15.115:0/1430109287 learned_addr learned my addr 172.21.15.115:0/1430109287 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.849+0000 7f1a56a00700 1 -- 172.21.15.115:0/1430109287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a58100be0 msgr2=0x7f1a58197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.849+0000 7f1a56a00700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a58100be0 0x7f1a58197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.849+0000 7f1a56a00700 1 -- 172.21.15.115:0/1430109287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a480095f0 con 0x7f1a58103500 2024-03-22T10:40:20.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.849+0000 7f1a56a00700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a58103500 0x7f1a58198040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400e920 tx=0x7f1a4400ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:20.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.850+0000 7f1a4fe00700 1 -- 172.21.15.115:0/1430109287 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a4400c9d0 con 0x7f1a58103500 2024-03-22T10:40:20.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.850+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5819d250 con 0x7f1a58103500 2024-03-22T10:40:20.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.850+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5819d7a0 con 0x7f1a58103500 2024-03-22T10:40:20.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.850+0000 7f1a4fe00700 1 -- 172.21.15.115:0/1430109287 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a44010430 con 0x7f1a58103500 2024-03-22T10:40:20.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.850+0000 7f1a4fe00700 1 -- 172.21.15.115:0/1430109287 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a44018650 con 0x7f1a58103500 2024-03-22T10:40:20.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.852+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a38005320 con 0x7f1a58103500 2024-03-22T10:40:20.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.853+0000 7f1a4fe00700 1 -- 172.21.15.115:0/1430109287 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f1a4400cb30 con 0x7f1a58103500 2024-03-22T10:40:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.854+0000 7f1a4fe00700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1a3c06c680 0x7f1a3c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.855+0000 7f1a4fe00700 1 -- 172.21.15.115:0/1430109287 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f1a44014070 con 0x7f1a58103500 2024-03-22T10:40:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.855+0000 7f1a57400700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1a3c06c680 0x7f1a3c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:20.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.856+0000 7f1a57400700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1a3c06c680 0x7f1a3c06eb30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009fd0 tx=0x7f1a48005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:20.863+0000 7f1a4fe00700 1 -- 172.21.15.115:0/1430109287 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1a44051720 con 0x7f1a58103500 2024-03-22T10:40:21.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.175+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a38000bf0 con 0x7f1a3c06c680 2024-03-22T10:40:21.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.177+0000 7f1a4fe00700 1 -- 172.21.15.115:0/1430109287 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f1a38000bf0 con 0x7f1a3c06c680 2024-03-22T10:40:21.170 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:40:21.170 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:40:21.170 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:40:21.170 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:40:21.170 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [], 2024-03-22T10:40:21.170 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "0/23 daemons upgraded", 2024-03-22T10:40:21.171 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc image on host smithi150", 2024-03-22T10:40:21.171 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:40:21.171 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:40:21.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.180+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1a3c06c680 msgr2=0x7f1a3c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:21.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.180+0000 7f1a5e400700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1a3c06c680 0x7f1a3c06eb30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009fd0 tx=0x7f1a48005fd0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.180+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a58103500 msgr2=0x7f1a58198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:21.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.180+0000 7f1a5e400700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a58103500 0x7f1a58198040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400e920 tx=0x7f1a4400ec30 comp rx=0 tx=0).stop 2024-03-22T10:40:21.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.180+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 shutdown_connections 2024-03-22T10:40:21.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.180+0000 7f1a5e400700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f1a3c06c680 0x7f1a3c06eb30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.180+0000 7f1a5e400700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a58100be0 0x7f1a58197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.180+0000 7f1a5e400700 1 --2- 172.21.15.115:0/1430109287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a58103500 0x7f1a58198040 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.180+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 >> 172.21.15.115:0/1430109287 conn(0x7f1a580fa5e0 msgr2=0x7f1a580ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:21.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.181+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 shutdown_connections 2024-03-22T10:40:21.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.181+0000 7f1a5e400700 1 -- 172.21.15.115:0/1430109287 wait complete. 2024-03-22T10:40:21.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.358+0000 7f0083e00700 1 -- 172.21.15.115:0/427500540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00840fff90 msgr2=0x7f00841003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.358+0000 7f0083e00700 1 --2- 172.21.15.115:0/427500540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00840fff90 0x7f00841003a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0074009910 tx=0x7f0074009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.358+0000 7f0083e00700 1 -- 172.21.15.115:0/427500540 shutdown_connections 2024-03-22T10:40:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.358+0000 7f0083e00700 1 --2- 172.21.15.115:0/427500540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0084101190 0x7f00841015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.358+0000 7f0083e00700 1 --2- 172.21.15.115:0/427500540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00840fff90 0x7f00841003a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.358+0000 7f0083e00700 1 -- 172.21.15.115:0/427500540 >> 172.21.15.115:0/427500540 conn(0x7f00840fb500 msgr2=0x7f00840fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:21.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.358+0000 7f0083e00700 1 -- 172.21.15.115:0/427500540 shutdown_connections 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.358+0000 7f0083e00700 1 -- 172.21.15.115:0/427500540 wait complete. 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.360+0000 7f0083e00700 1 Processor -- start 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.360+0000 7f0083e00700 1 -- start start 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.360+0000 7f0083e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00840fff90 0x7f008419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.360+0000 7f0083e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084101190 0x7f008419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.360+0000 7f0083e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008419c920 con 0x7f0084101190 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.360+0000 7f0083e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008419ca60 con 0x7f00840fff90 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.361+0000 7f0082000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084101190 0x7f008419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.361+0000 7f0082000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084101190 0x7f008419c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58260/0 (socket says 172.21.15.115:58260) 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.361+0000 7f0082000700 1 -- 172.21.15.115:0/4147157839 learned_addr learned my addr 172.21.15.115:0/4147157839 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.361+0000 7f0082a00700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00840fff90 0x7f008419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.361+0000 7f0082000700 1 -- 172.21.15.115:0/4147157839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00840fff90 msgr2=0x7f008419bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.361+0000 7f0082000700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00840fff90 0x7f008419bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.361+0000 7f0082000700 1 -- 172.21.15.115:0/4147157839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740095f0 con 0x7f0084101190 2024-03-22T10:40:21.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.362+0000 7f0082000700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084101190 0x7f008419c300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f007000d8d0 tx=0x7f007000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:21.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.362+0000 7f007b400700 1 -- 172.21.15.115:0/4147157839 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0070009940 con 0x7f0084101190 2024-03-22T10:40:21.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.362+0000 7f007b400700 1 -- 172.21.15.115:0/4147157839 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0070010460 con 0x7f0084101190 2024-03-22T10:40:21.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.362+0000 7f007b400700 1 -- 172.21.15.115:0/4147157839 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f007000f5f0 con 0x7f0084101190 2024-03-22T10:40:21.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.362+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00841a1510 con 0x7f0084101190 2024-03-22T10:40:21.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.362+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00841a1a60 con 0x7f0084101190 2024-03-22T10:40:21.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.364+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0084105270 con 0x7f0084101190 2024-03-22T10:40:21.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.365+0000 7f007b400700 1 -- 172.21.15.115:0/4147157839 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f00700105d0 con 0x7f0084101190 2024-03-22T10:40:21.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.367+0000 7f007b400700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f006806c6d0 0x7f006806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:21.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.367+0000 7f007b400700 1 -- 172.21.15.115:0/4147157839 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f007008b300 con 0x7f0084101190 2024-03-22T10:40:21.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.367+0000 7f0082a00700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f006806c6d0 0x7f006806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:21.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.368+0000 7f0082a00700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f006806c6d0 0x7f006806eb80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00740098e0 tx=0x7f0074005c00 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:21.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.374+0000 7f007b400700 1 -- 172.21.15.115:0/4147157839 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0070059450 con 0x7f0084101190 2024-03-22T10:40:21.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:21 smithi150 ceph-mon[34518]: Upgrade: Target is version 19.0.0-2145-g3be9a5c9 (unknown) 2024-03-22T10:40:21.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:21 smithi150 ceph-mon[34518]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc'] 2024-03-22T10:40:21.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:21 smithi150 ceph-mon[34518]: Upgrade: Need to upgrade myself (mgr.smithi115.iqzwsz) 2024-03-22T10:40:21.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:21 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1350848780' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:40:21.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:21 smithi150 ceph-mon[34518]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc on smithi150 2024-03-22T10:40:21.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:21 smithi115 ceph-mon[29111]: Upgrade: Target is version 19.0.0-2145-g3be9a5c9 (unknown) 2024-03-22T10:40:21.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:21 smithi115 ceph-mon[29111]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc'] 2024-03-22T10:40:21.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:21 smithi115 ceph-mon[29111]: Upgrade: Need to upgrade myself (mgr.smithi115.iqzwsz) 2024-03-22T10:40:21.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:21 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1350848780' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:40:21.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:21 smithi115 ceph-mon[29111]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc on smithi150 2024-03-22T10:40:21.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.778+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f008402cc30 con 0x7f0084101190 2024-03-22T10:40:21.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.779+0000 7f007b400700 1 -- 172.21.15.115:0/4147157839 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f00700200e0 con 0x7f0084101190 2024-03-22T10:40:21.771 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:40:21.772 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:40:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:40:21.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f006806c6d0 msgr2=0x7f006806eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:21.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f006806c6d0 0x7f006806eb80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00740098e0 tx=0x7f0074005c00 comp rx=0 tx=0).stop 2024-03-22T10:40:21.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084101190 msgr2=0x7f008419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:21.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084101190 0x7f008419c300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f007000d8d0 tx=0x7f007000dc90 comp rx=0 tx=0).stop 2024-03-22T10:40:21.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 shutdown_connections 2024-03-22T10:40:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f006806c6d0 0x7f006806eb80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00840fff90 0x7f008419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 --2- 172.21.15.115:0/4147157839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0084101190 0x7f008419c300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 >> 172.21.15.115:0/4147157839 conn(0x7f00840fb500 msgr2=0x7f00841023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.782+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 shutdown_connections 2024-03-22T10:40:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:21.783+0000 7f0083e00700 1 -- 172.21.15.115:0/4147157839 wait complete. 2024-03-22T10:40:22.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:22 smithi150 ceph-mon[34518]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:22.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:22 smithi150 ceph-mon[34518]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:22.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:22 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/4147157839' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:40:22.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:22 smithi115 ceph-mon[29111]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:22.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:22 smithi115 ceph-mon[29111]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:22.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:22 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/4147157839' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:40:24.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:24 smithi150 ceph-mon[34518]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:24.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:24 smithi115 ceph-mon[29111]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:25.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:40:25.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:40:26.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:26 smithi150 ceph-mon[34518]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:26.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:26 smithi115 ceph-mon[29111]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:28.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:28 smithi150 ceph-mon[34518]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:28.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:28 smithi115 ceph-mon[29111]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:30.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:30 smithi150 ceph-mon[34518]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:30.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:30 smithi115 ceph-mon[29111]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:32.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:32 smithi150 ceph-mon[34518]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:32.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:32 smithi115 ceph-mon[29111]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:34 smithi150 ceph-mon[34518]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:34.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:34 smithi115 ceph-mon[29111]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:36.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:36 smithi150 ceph-mon[34518]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:36.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:36 smithi115 ceph-mon[29111]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:38.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:38 smithi150 ceph-mon[34518]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:38.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:38 smithi115 ceph-mon[29111]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:40.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:40 smithi150 ceph-mon[34518]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:40.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:40 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:40:40.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:40 smithi115 ceph-mon[29111]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:40.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:40 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:40:42.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:42 smithi150 ceph-mon[34518]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:42.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:42 smithi115 ceph-mon[29111]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:44.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:44 smithi150 ceph-mon[34518]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:44.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:44 smithi115 ceph-mon[29111]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:46 smithi115 ceph-mon[29111]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:46.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:46 smithi150 ceph-mon[34518]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:48.299 INFO:tasks.workunit.client.0.smithi115.stderr:Updating files: 63% (7739/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-22T10:40:48.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:48 smithi150 ceph-mon[34518]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr:state without impacting any branches by switching back to a branch. 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr: git switch -c 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr:Or undo this operation with: 2024-03-22T10:40:48.753 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-03-22T10:40:48.754 INFO:tasks.workunit.client.0.smithi115.stderr: git switch - 2024-03-22T10:40:48.754 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-03-22T10:40:48.754 INFO:tasks.workunit.client.0.smithi115.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-22T10:40:48.754 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-03-22T10:40:48.754 INFO:tasks.workunit.client.0.smithi115.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-22T10:40:48.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:48 smithi115 ceph-mon[29111]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:48.764 DEBUG:teuthology.orchestra.run.smithi115:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-22T10:40:48.826 INFO:tasks.workunit.client.0.smithi115.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-22T10:40:48.833 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T10:40:48.833 INFO:tasks.workunit.client.0.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-22T10:40:49.803 INFO:tasks.workunit.client.0.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-22T10:40:49.952 INFO:tasks.workunit.client.0.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-22T10:40:50.028 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T10:40:50.033 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T10:40:50.033 INFO:tasks.workunit.client.0.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-22T10:40:50.117 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T10:40:50.127 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:40:50.127 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-22T10:40:50.176 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-22T10:40:50.178 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-22T10:40:50.178 DEBUG:teuthology.orchestra.run.smithi115:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-22T10:40:50.262 INFO:tasks.workunit.client.0.smithi115.stderr:+ mkdir -p fsstress 2024-03-22T10:40:50.268 INFO:tasks.workunit.client.0.smithi115.stderr:+ pushd fsstress 2024-03-22T10:40:50.271 INFO:tasks.workunit.client.0.smithi115.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T10:40:50.271 INFO:tasks.workunit.client.0.smithi115.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-22T10:40:50.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:50 smithi150 ceph-mon[34518]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:50.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:50 smithi115 ceph-mon[29111]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.969+0000 7fce51c00700 1 -- 172.21.15.115:0/522354148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 msgr2=0x7fce4c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.969+0000 7fce51c00700 1 --2- 172.21.15.115:0/522354148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 0x7fce4c100560 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009910 tx=0x7fce3c009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.969+0000 7fce51c00700 1 -- 172.21.15.115:0/522354148 shutdown_connections 2024-03-22T10:40:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.969+0000 7fce51c00700 1 --2- 172.21.15.115:0/522354148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce4c101350 0x7fce4c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.969+0000 7fce51c00700 1 --2- 172.21.15.115:0/522354148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 0x7fce4c100560 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.969+0000 7fce51c00700 1 -- 172.21.15.115:0/522354148 >> 172.21.15.115:0/522354148 conn(0x7fce4c0fb6e0 msgr2=0x7fce4c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.969+0000 7fce51c00700 1 -- 172.21.15.115:0/522354148 shutdown_connections 2024-03-22T10:40:51.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.969+0000 7fce51c00700 1 -- 172.21.15.115:0/522354148 wait complete. 2024-03-22T10:40:51.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.971+0000 7fce51c00700 1 Processor -- start 2024-03-22T10:40:51.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.971+0000 7fce51c00700 1 -- start start 2024-03-22T10:40:51.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.971+0000 7fce51c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 0x7fce4c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:51.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.971+0000 7fce51c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce4c101350 0x7fce4c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:51.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.971+0000 7fce51c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c198700 con 0x7fce4c100150 2024-03-22T10:40:51.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.971+0000 7fce51c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c198840 con 0x7fce4c101350 2024-03-22T10:40:51.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.972+0000 7fce4aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 0x7fce4c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:51.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.972+0000 7fce4aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 0x7fce4c197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51772/0 (socket says 172.21.15.115:51772) 2024-03-22T10:40:51.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.972+0000 7fce4aa00700 1 -- 172.21.15.115:0/3315385649 learned_addr learned my addr 172.21.15.115:0/3315385649 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:51.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.972+0000 7fce4a000700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce4c101350 0x7fce4c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:51.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.972+0000 7fce4aa00700 1 -- 172.21.15.115:0/3315385649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce4c101350 msgr2=0x7fce4c1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:51.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.972+0000 7fce4aa00700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce4c101350 0x7fce4c1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:51.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.972+0000 7fce4aa00700 1 -- 172.21.15.115:0/3315385649 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0095f0 con 0x7fce4c100150 2024-03-22T10:40:51.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.973+0000 7fce4aa00700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 0x7fce4c197ba0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00bd70 tx=0x7fce3c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:51.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.973+0000 7fce43400700 1 -- 172.21.15.115:0/3315385649 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3c01d070 con 0x7fce4c100150 2024-03-22T10:40:51.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.973+0000 7fce43400700 1 -- 172.21.15.115:0/3315385649 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fce3c00f460 con 0x7fce4c100150 2024-03-22T10:40:51.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.973+0000 7fce43400700 1 -- 172.21.15.115:0/3315385649 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3c021640 con 0x7fce4c100150 2024-03-22T10:40:51.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.973+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c19d290 con 0x7fce4c100150 2024-03-22T10:40:51.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.973+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c19d780 con 0x7fce4c100150 2024-03-22T10:40:51.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.975+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c005320 con 0x7fce4c100150 2024-03-22T10:40:51.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.976+0000 7fce43400700 1 -- 172.21.15.115:0/3315385649 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fce3c00fab0 con 0x7fce4c100150 2024-03-22T10:40:51.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.977+0000 7fce43400700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fce3006c680 0x7fce3006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:51.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.977+0000 7fce43400700 1 -- 172.21.15.115:0/3315385649 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fce3c08cca0 con 0x7fce4c100150 2024-03-22T10:40:51.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.977+0000 7fce4a000700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fce3006c680 0x7fce3006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:51.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.978+0000 7fce4a000700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fce3006c680 0x7fce3006eb30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fce38005fd0 tx=0x7fce3800a560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:51.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:51.983+0000 7fce43400700 1 -- 172.21.15.115:0/3315385649 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fce3c05adf0 con 0x7fce4c100150 2024-03-22T10:40:52.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.236+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce2c000bf0 con 0x7fce3006c680 2024-03-22T10:40:52.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.238+0000 7fce43400700 1 -- 172.21.15.115:0/3315385649 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fce2c000bf0 con 0x7fce3006c680 2024-03-22T10:40:52.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.241+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fce3006c680 msgr2=0x7fce3006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:52.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.241+0000 7fce51c00700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fce3006c680 0x7fce3006eb30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fce38005fd0 tx=0x7fce3800a560 comp rx=0 tx=0).stop 2024-03-22T10:40:52.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.241+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 msgr2=0x7fce4c197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:52.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.241+0000 7fce51c00700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 0x7fce4c197ba0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00bd70 tx=0x7fce3c00be50 comp rx=0 tx=0).stop 2024-03-22T10:40:52.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.242+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 shutdown_connections 2024-03-22T10:40:52.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.242+0000 7fce51c00700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fce3006c680 0x7fce3006eb30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.242+0000 7fce51c00700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce4c100150 0x7fce4c197ba0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.242+0000 7fce51c00700 1 --2- 172.21.15.115:0/3315385649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce4c101350 0x7fce4c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.242+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 >> 172.21.15.115:0/3315385649 conn(0x7fce4c0fb6e0 msgr2=0x7fce4c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:52.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.242+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 shutdown_connections 2024-03-22T10:40:52.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.242+0000 7fce51c00700 1 -- 172.21.15.115:0/3315385649 wait complete. 2024-03-22T10:40:52.250 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:40:52.359 INFO:tasks.workunit.client.1.smithi150.stderr:Updating files: 56% (6914/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 93% (11492/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-22T10:40:52.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.405+0000 7fc050600700 1 -- 172.21.15.115:0/1982894927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 msgr2=0x7fc048100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:52.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.405+0000 7fc050600700 1 --2- 172.21.15.115:0/1982894927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 0x7fc048100e10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009910 tx=0x7fc03c009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:52.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.406+0000 7fc050600700 1 -- 172.21.15.115:0/1982894927 shutdown_connections 2024-03-22T10:40:52.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.406+0000 7fc050600700 1 --2- 172.21.15.115:0/1982894927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc048101350 0x7fc0481037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.406+0000 7fc050600700 1 --2- 172.21.15.115:0/1982894927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 0x7fc048100e10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.406+0000 7fc050600700 1 -- 172.21.15.115:0/1982894927 >> 172.21.15.115:0/1982894927 conn(0x7fc0480fa600 msgr2=0x7fc0480fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:52.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.406+0000 7fc050600700 1 -- 172.21.15.115:0/1982894927 shutdown_connections 2024-03-22T10:40:52.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.406+0000 7fc050600700 1 -- 172.21.15.115:0/1982894927 wait complete. 2024-03-22T10:40:52.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.407+0000 7fc050600700 1 Processor -- start 2024-03-22T10:40:52.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.408+0000 7fc050600700 1 -- start start 2024-03-22T10:40:52.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.408+0000 7fc050600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 0x7fc04819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:52.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.408+0000 7fc050600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc048101350 0x7fc04819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:52.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.408+0000 7fc050600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04819cad0 con 0x7fc0480fe9f0 2024-03-22T10:40:52.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.408+0000 7fc050600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04819cc10 con 0x7fc048101350 2024-03-22T10:40:52.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.408+0000 7fc04dc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 0x7fc04819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.408+0000 7fc04dc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 0x7fc04819bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51796/0 (socket says 172.21.15.115:51796) 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.408+0000 7fc04dc00700 1 -- 172.21.15.115:0/1303574749 learned_addr learned my addr 172.21.15.115:0/1303574749 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.408+0000 7fc04d200700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc048101350 0x7fc04819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.409+0000 7fc04dc00700 1 -- 172.21.15.115:0/1303574749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc048101350 msgr2=0x7fc04819c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.409+0000 7fc04dc00700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc048101350 0x7fc04819c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.409+0000 7fc04dc00700 1 -- 172.21.15.115:0/1303574749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c0095f0 con 0x7fc0480fe9f0 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.409+0000 7fc04dc00700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 0x7fc04819bf70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0092e0 tx=0x7fc03c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.409+0000 7fc046000700 1 -- 172.21.15.115:0/1303574749 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03c01d070 con 0x7fc0480fe9f0 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.410+0000 7fc046000700 1 -- 172.21.15.115:0/1303574749 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc03c005dc0 con 0x7fc0480fe9f0 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.410+0000 7fc046000700 1 -- 172.21.15.115:0/1303574749 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03c017630 con 0x7fc0480fe9f0 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.410+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0481a1660 con 0x7fc0480fe9f0 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.410+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0481a1b50 con 0x7fc0480fe9f0 2024-03-22T10:40:52.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.411+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc048196190 con 0x7fc0480fe9f0 2024-03-22T10:40:52.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.412+0000 7fc046000700 1 -- 172.21.15.115:0/1303574749 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fc03c003680 con 0x7fc0480fe9f0 2024-03-22T10:40:52.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.413+0000 7fc046000700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc03006c680 0x7fc03006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:52.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.413+0000 7fc046000700 1 -- 172.21.15.115:0/1303574749 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc03c08cd00 con 0x7fc0480fe9f0 2024-03-22T10:40:52.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.413+0000 7fc04d200700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc03006c680 0x7fc03006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:52.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.417+0000 7fc04d200700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc03006c680 0x7fc03006eb30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc034009c60 tx=0x7fc034009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:52.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.421+0000 7fc046000700 1 -- 172.21.15.115:0/1303574749 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc03c05ae50 con 0x7fc0480fe9f0 2024-03-22T10:40:52.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:52 smithi115 ceph-mon[29111]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:52.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:52 smithi150 ceph-mon[34518]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:52.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.701+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc048061190 con 0x7fc03006c680 2024-03-22T10:40:52.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.703+0000 7fc046000700 1 -- 172.21.15.115:0/1303574749 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fc048061190 con 0x7fc03006c680 2024-03-22T10:40:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc03006c680 msgr2=0x7fc03006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc03006c680 0x7fc03006eb30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc034009c60 tx=0x7fc034009260 comp rx=0 tx=0).stop 2024-03-22T10:40:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 msgr2=0x7fc04819bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 0x7fc04819bf70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0092e0 tx=0x7fc03c00f690 comp rx=0 tx=0).stop 2024-03-22T10:40:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 shutdown_connections 2024-03-22T10:40:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fc03006c680 0x7fc03006eb30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0480fe9f0 0x7fc04819bf70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 --2- 172.21.15.115:0/1303574749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc048101350 0x7fc04819c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 >> 172.21.15.115:0/1303574749 conn(0x7fc0480fa600 msgr2=0x7fc0480fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 shutdown_connections 2024-03-22T10:40:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.706+0000 7fc050600700 1 -- 172.21.15.115:0/1303574749 wait complete. 2024-03-22T10:40:52.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.865+0000 7faccb400700 1 -- 172.21.15.115:0/2511826018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4100be0 msgr2=0x7facc4102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:52.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.865+0000 7faccb400700 1 --2- 172.21.15.115:0/2511826018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4100be0 0x7facc4102fc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7facb8009910 tx=0x7facb8009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:52.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.866+0000 7faccb400700 1 -- 172.21.15.115:0/2511826018 shutdown_connections 2024-03-22T10:40:52.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.866+0000 7faccb400700 1 --2- 172.21.15.115:0/2511826018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facc4103500 0x7facc41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.866+0000 7faccb400700 1 --2- 172.21.15.115:0/2511826018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4100be0 0x7facc4102fc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.866+0000 7faccb400700 1 -- 172.21.15.115:0/2511826018 >> 172.21.15.115:0/2511826018 conn(0x7facc40fa5e0 msgr2=0x7facc40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:52.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.866+0000 7faccb400700 1 -- 172.21.15.115:0/2511826018 shutdown_connections 2024-03-22T10:40:52.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.866+0000 7faccb400700 1 -- 172.21.15.115:0/2511826018 wait complete. 2024-03-22T10:40:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.867+0000 7faccb400700 1 Processor -- start 2024-03-22T10:40:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.867+0000 7faccb400700 1 -- start start 2024-03-22T10:40:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.867+0000 7faccb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facc4100be0 0x7facc4100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.867+0000 7faccb400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4103500 0x7facc40feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.867+0000 7faccb400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40ff0d0 con 0x7facc4103500 2024-03-22T10:40:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.867+0000 7faccb400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40ff210 con 0x7facc4100be0 2024-03-22T10:40:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.868+0000 7facc3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4103500 0x7facc40feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.868+0000 7facc3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4103500 0x7facc40feb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51804/0 (socket says 172.21.15.115:51804) 2024-03-22T10:40:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.868+0000 7facc3e00700 1 -- 172.21.15.115:0/729113820 learned_addr learned my addr 172.21.15.115:0/729113820 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.868+0000 7facc8a00700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facc4100be0 0x7facc4100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.868+0000 7facc3e00700 1 -- 172.21.15.115:0/729113820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facc4100be0 msgr2=0x7facc4100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.868+0000 7facc3e00700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facc4100be0 0x7facc4100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.868+0000 7facc3e00700 1 -- 172.21.15.115:0/729113820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb80095f0 con 0x7facc4103500 2024-03-22T10:40:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.868+0000 7facc3e00700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4103500 0x7facc40feb90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7facb000b4b0 tx=0x7facb000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.869+0000 7facc1600700 1 -- 172.21.15.115:0/729113820 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facb0011800 con 0x7facc4103500 2024-03-22T10:40:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.869+0000 7facc1600700 1 -- 172.21.15.115:0/729113820 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7facb0011e10 con 0x7facc4103500 2024-03-22T10:40:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.869+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc40ff4f0 con 0x7facc4103500 2024-03-22T10:40:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.869+0000 7facc1600700 1 -- 172.21.15.115:0/729113820 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facb000f2e0 con 0x7facc4103500 2024-03-22T10:40:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.869+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc419da00 con 0x7facc4103500 2024-03-22T10:40:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.870+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca4005320 con 0x7facc4103500 2024-03-22T10:40:52.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.871+0000 7facc1600700 1 -- 172.21.15.115:0/729113820 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7facb0011960 con 0x7facc4103500 2024-03-22T10:40:52.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.872+0000 7facc1600700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7facac06c680 0x7facac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:52.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.872+0000 7facc1600700 1 -- 172.21.15.115:0/729113820 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7facb008b450 con 0x7facc4103500 2024-03-22T10:40:52.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.872+0000 7facc8a00700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7facac06c680 0x7facac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:52.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.873+0000 7facc8a00700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7facac06c680 0x7facac06eb30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7facb8006010 tx=0x7facb800b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:52.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:52.877+0000 7facc1600700 1 -- 172.21.15.115:0/729113820 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7facb0059520 con 0x7facc4103500 2024-03-22T10:40:52.898 INFO:tasks.workunit.client.1.smithi150.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-22T10:40:52.898 INFO:tasks.workunit.client.1.smithi150.stderr: 2024-03-22T10:40:52.898 INFO:tasks.workunit.client.1.smithi150.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr:state without impacting any branches by switching back to a branch. 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr: 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr: 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr: git switch -c 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr: 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr:Or undo this operation with: 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr: 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr: git switch - 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr: 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr: 2024-03-22T10:40:52.899 INFO:tasks.workunit.client.1.smithi150.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-22T10:40:52.910 DEBUG:teuthology.orchestra.run.smithi150:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-22T10:40:53.034 INFO:tasks.workunit.client.1.smithi150.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-22T10:40:53.041 INFO:tasks.workunit.client.1.smithi150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-22T10:40:53.041 INFO:tasks.workunit.client.1.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-22T10:40:53.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.114+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faca4000bf0 con 0x7facac06c680 2024-03-22T10:40:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.144+0000 7facc1600700 1 -- 172.21.15.115:0/729113820 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7faca4000bf0 con 0x7facac06c680 2024-03-22T10:40:53.136 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:40:53.136 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (7m) 109s ago 9m 13.4M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:40:53.136 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (9m) 109s ago 9m 7486k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (8m) 114s ago 8m 7398k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (9m) 109s ago 9m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (8m) 114s ago 8m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (7m) 109s ago 8m 73.5M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (2m) 109s ago 2m 14.7M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (2m) 109s ago 2m 11.7M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (2m) 114s ago 2m 11.5M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (2m) 114s ago 2m 14.2M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:9283,8765,8443 running (10m) 109s ago 10m 488M - 18.2.0 dc2bc1663786 932346df0e0f 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (7m) 114s ago 7m 437M - 18.2.0 dc2bc1663786 3884fe56fbc9 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (11m) 109s ago 11m 44.2M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (7m) 114s ago 7m 33.9M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:40:53.137 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (9m) 109s ago 9m 7746k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:40:53.138 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (8m) 114s ago 8m 7591k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:40:53.138 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (6m) 109s ago 6m 45.7M 3947M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:40:53.138 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (6m) 109s ago 6m 47.1M 3947M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:40:53.138 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (5m) 109s ago 5m 46.0M 3947M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:40:53.138 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (5m) 114s ago 5m 45.6M 4971M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:40:53.138 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (4m) 114s ago 4m 43.1M 4971M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:40:53.138 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (4m) 114s ago 4m 42.8M 4971M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:40:53.138 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (7m) 109s ago 8m 57.0M - 2.43.0 a07b618ecd1d 9fcacfb2c3cb 2024-03-22T10:40:53.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7facac06c680 msgr2=0x7facac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7facac06c680 0x7facac06eb30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7facb8006010 tx=0x7facb800b560 comp rx=0 tx=0).stop 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4103500 msgr2=0x7facc40feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4103500 0x7facc40feb90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7facb000b4b0 tx=0x7facb000b7c0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 shutdown_connections 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7facac06c680 0x7facac06eb30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facc4100be0 0x7facc4100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 --2- 172.21.15.115:0/729113820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facc4103500 0x7facc40feb90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 >> 172.21.15.115:0/729113820 conn(0x7facc40fa5e0 msgr2=0x7facc40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 shutdown_connections 2024-03-22T10:40:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.148+0000 7faccb400700 1 -- 172.21.15.115:0/729113820 wait complete. 2024-03-22T10:40:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.300+0000 7fb383200700 1 -- 172.21.15.115:0/2292194666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb384101190 msgr2=0x7fb3841015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.300+0000 7fb383200700 1 --2- 172.21.15.115:0/2292194666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb384101190 0x7fb3841015e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009910 tx=0x7fb36c009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.300+0000 7fb383200700 1 -- 172.21.15.115:0/2292194666 shutdown_connections 2024-03-22T10:40:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.300+0000 7fb383200700 1 --2- 172.21.15.115:0/2292194666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb384101190 0x7fb3841015e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.300+0000 7fb383200700 1 --2- 172.21.15.115:0/2292194666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3840fff90 0x7fb3841003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.300+0000 7fb383200700 1 -- 172.21.15.115:0/2292194666 >> 172.21.15.115:0/2292194666 conn(0x7fb3840fb500 msgr2=0x7fb3840fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.300+0000 7fb383200700 1 -- 172.21.15.115:0/2292194666 shutdown_connections 2024-03-22T10:40:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.300+0000 7fb383200700 1 -- 172.21.15.115:0/2292194666 wait complete. 2024-03-22T10:40:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.301+0000 7fb383200700 1 Processor -- start 2024-03-22T10:40:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.301+0000 7fb383200700 1 -- start start 2024-03-22T10:40:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.301+0000 7fb383200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3840fff90 0x7fb3841957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.301+0000 7fb383200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb384101190 0x7fb384195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.302+0000 7fb383200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb384196330 con 0x7fb3840fff90 2024-03-22T10:40:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.302+0000 7fb383200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb384196470 con 0x7fb384101190 2024-03-22T10:40:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.302+0000 7fb381e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3840fff90 0x7fb3841957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.302+0000 7fb381e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3840fff90 0x7fb3841957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51814/0 (socket says 172.21.15.115:51814) 2024-03-22T10:40:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.302+0000 7fb381e00700 1 -- 172.21.15.115:0/3967324264 learned_addr learned my addr 172.21.15.115:0/3967324264 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.302+0000 7fb379400700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb384101190 0x7fb384195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.302+0000 7fb381e00700 1 -- 172.21.15.115:0/3967324264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb384101190 msgr2=0x7fb384195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.302+0000 7fb381e00700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb384101190 0x7fb384195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.302+0000 7fb381e00700 1 -- 172.21.15.115:0/3967324264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c0095f0 con 0x7fb3840fff90 2024-03-22T10:40:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.303+0000 7fb381e00700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3840fff90 0x7fb3841957d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb37400ada0 tx=0x7fb37400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.303+0000 7fb37b400700 1 -- 172.21.15.115:0/3967324264 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb37400f070 con 0x7fb3840fff90 2024-03-22T10:40:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.303+0000 7fb37b400700 1 -- 172.21.15.115:0/3967324264 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb37400ae90 con 0x7fb3840fff90 2024-03-22T10:40:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.303+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38419af20 con 0x7fb3840fff90 2024-03-22T10:40:53.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.303+0000 7fb37b400700 1 -- 172.21.15.115:0/3967324264 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3740083f0 con 0x7fb3840fff90 2024-03-22T10:40:53.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.304+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38419b3e0 con 0x7fb3840fff90 2024-03-22T10:40:53.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.305+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb384108ae0 con 0x7fb3840fff90 2024-03-22T10:40:53.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.305+0000 7fb37b400700 1 -- 172.21.15.115:0/3967324264 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb37400cad0 con 0x7fb3840fff90 2024-03-22T10:40:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.306+0000 7fb37b400700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb36806c680 0x7fb36806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.306+0000 7fb37b400700 1 -- 172.21.15.115:0/3967324264 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb37408b120 con 0x7fb3840fff90 2024-03-22T10:40:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.306+0000 7fb379400700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb36806c680 0x7fb36806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.307+0000 7fb379400700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb36806c680 0x7fb36806eb30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb36c006010 tx=0x7fb36c00b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.311+0000 7fb37b400700 1 -- 172.21.15.115:0/3967324264 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb374059310 con 0x7fb3840fff90 2024-03-22T10:40:53.390 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:53 smithi115 ceph-mon[29111]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:53.391 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:53 smithi115 ceph-mon[29111]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:53.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.652+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb38419bf30 con 0x7fb3840fff90 2024-03-22T10:40:53.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.653+0000 7fb37b400700 1 -- 172.21.15.115:0/3967324264 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fb374058ea0 con 0x7fb3840fff90 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:40:53.646 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:40:53.647 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:40:53.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:40:53.647 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:40:53.647 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-03-22T10:40:53.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:40:53.647 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:40:53.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:53 smithi150 ceph-mon[34518]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:53.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:53 smithi150 ceph-mon[34518]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:53.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.657+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb36806c680 msgr2=0x7fb36806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:53.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.657+0000 7fb383200700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb36806c680 0x7fb36806eb30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb36c006010 tx=0x7fb36c00b560 comp rx=0 tx=0).stop 2024-03-22T10:40:53.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.657+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3840fff90 msgr2=0x7fb3841957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:53.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.657+0000 7fb383200700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3840fff90 0x7fb3841957d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb37400ada0 tx=0x7fb37400c330 comp rx=0 tx=0).stop 2024-03-22T10:40:53.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.657+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 shutdown_connections 2024-03-22T10:40:53.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.658+0000 7fb383200700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb36806c680 0x7fb36806eb30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.658+0000 7fb383200700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3840fff90 0x7fb3841957d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.658+0000 7fb383200700 1 --2- 172.21.15.115:0/3967324264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb384101190 0x7fb384195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.658+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 >> 172.21.15.115:0/3967324264 conn(0x7fb3840fb500 msgr2=0x7fb3841043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:53.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.658+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 shutdown_connections 2024-03-22T10:40:53.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.658+0000 7fb383200700 1 -- 172.21.15.115:0/3967324264 wait complete. 2024-03-22T10:40:53.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.813+0000 7f69b5200700 1 -- 172.21.15.115:0/2675988636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0100c40 msgr2=0x7f69b0103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:53.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.813+0000 7f69b5200700 1 --2- 172.21.15.115:0/2675988636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0100c40 0x7f69b0103020 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009910 tx=0x7f69a0009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:53.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.813+0000 7f69b5200700 1 -- 172.21.15.115:0/2675988636 shutdown_connections 2024-03-22T10:40:53.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.813+0000 7f69b5200700 1 --2- 172.21.15.115:0/2675988636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69b0103560 0x7f69b0105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.813+0000 7f69b5200700 1 --2- 172.21.15.115:0/2675988636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0100c40 0x7f69b0103020 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.813+0000 7f69b5200700 1 -- 172.21.15.115:0/2675988636 >> 172.21.15.115:0/2675988636 conn(0x7f69b00fa5e0 msgr2=0x7f69b00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:53.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.813+0000 7f69b5200700 1 -- 172.21.15.115:0/2675988636 shutdown_connections 2024-03-22T10:40:53.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.813+0000 7f69b5200700 1 -- 172.21.15.115:0/2675988636 wait complete. 2024-03-22T10:40:53.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.814+0000 7f69b5200700 1 Processor -- start 2024-03-22T10:40:53.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.814+0000 7f69b5200700 1 -- start start 2024-03-22T10:40:53.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69b5200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69b0100c40 0x7f69b0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:53.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69b5200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0103560 0x7f69b0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:53.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69b5200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0198660 con 0x7f69b0103560 2024-03-22T10:40:53.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69b5200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b01987a0 con 0x7f69b0100c40 2024-03-22T10:40:53.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69ad400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0103560 0x7f69b0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:53.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69ad400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0103560 0x7f69b0198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51836/0 (socket says 172.21.15.115:51836) 2024-03-22T10:40:53.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69ad400700 1 -- 172.21.15.115:0/4115766196 learned_addr learned my addr 172.21.15.115:0/4115766196 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:53.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69ade00700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69b0100c40 0x7f69b0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:53.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69ad400700 1 -- 172.21.15.115:0/4115766196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69b0100c40 msgr2=0x7f69b0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:53.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69ad400700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69b0100c40 0x7f69b0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:53.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.815+0000 7f69ad400700 1 -- 172.21.15.115:0/4115766196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00095f0 con 0x7f69b0103560 2024-03-22T10:40:53.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.816+0000 7f69ad400700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0103560 0x7f69b0198040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f699c00e920 tx=0x7f699c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:53.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.816+0000 7f69a6a00700 1 -- 172.21.15.115:0/4115766196 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699c009790 con 0x7f69b0103560 2024-03-22T10:40:53.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.816+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b019d250 con 0x7f69b0103560 2024-03-22T10:40:53.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.816+0000 7f69a6a00700 1 -- 172.21.15.115:0/4115766196 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f699c00cba0 con 0x7f69b0103560 2024-03-22T10:40:53.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.816+0000 7f69a6a00700 1 -- 172.21.15.115:0/4115766196 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699c018630 con 0x7f69b0103560 2024-03-22T10:40:53.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.816+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b019d7a0 con 0x7f69b0103560 2024-03-22T10:40:53.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.818+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b00fc1c0 con 0x7f69b0103560 2024-03-22T10:40:53.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.818+0000 7f69a6a00700 1 -- 172.21.15.115:0/4115766196 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f699c00cd10 con 0x7f69b0103560 2024-03-22T10:40:53.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.819+0000 7f69a6a00700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f699406c680 0x7f699406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:53.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.819+0000 7f69a6a00700 1 -- 172.21.15.115:0/4115766196 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f699c014070 con 0x7f69b0103560 2024-03-22T10:40:53.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.820+0000 7f69ade00700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f699406c680 0x7f699406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:53.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.820+0000 7f69ade00700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f699406c680 0x7f699406eb30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f69a00059d0 tx=0x7f69a0005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:53.824+0000 7f69a6a00700 1 -- 172.21.15.115:0/4115766196 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f699c051720 con 0x7f69b0103560 2024-03-22T10:40:54.005 INFO:tasks.workunit.client.1.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-22T10:40:54.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.123+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f69b002cc30 con 0x7f69b0103560 2024-03-22T10:40:54.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.124+0000 7f69a6a00700 1 -- 172.21.15.115:0/4115766196 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1839 (secure 0 0 0) 0x7f699c059c40 con 0x7f69b0103560 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:e12 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 12 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:38:50.921733+0000 2024-03-22T10:40:54.117 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:40:54.118 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:40:54.119 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f699406c680 msgr2=0x7f699406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f699406c680 0x7f699406eb30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f69a00059d0 tx=0x7f69a0005940 comp rx=0 tx=0).stop 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0103560 msgr2=0x7f69b0198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0103560 0x7f69b0198040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f699c00e920 tx=0x7f699c00ece0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 shutdown_connections 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f699406c680 0x7f699406eb30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69b0100c40 0x7f69b0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 --2- 172.21.15.115:0/4115766196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69b0103560 0x7f69b0198040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 >> 172.21.15.115:0/4115766196 conn(0x7f69b00fa5e0 msgr2=0x7f69b00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 shutdown_connections 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.127+0000 7f69b5200700 1 -- 172.21.15.115:0/4115766196 wait complete. 2024-03-22T10:40:54.121 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 12 2024-03-22T10:40:54.129 INFO:tasks.workunit.client.1.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-22T10:40:54.215 INFO:tasks.workunit.client.1.smithi150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-22T10:40:54.221 INFO:tasks.workunit.client.1.smithi150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-22T10:40:54.221 INFO:tasks.workunit.client.1.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-22T10:40:54.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.290+0000 7f5f5e800700 1 -- 172.21.15.115:0/271843378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f58100a80 msgr2=0x7f5f58102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:54.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.290+0000 7f5f5e800700 1 --2- 172.21.15.115:0/271843378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f58100a80 0x7f5f58102e60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009910 tx=0x7f5f4c009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:54.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.290+0000 7f5f5e800700 1 -- 172.21.15.115:0/271843378 shutdown_connections 2024-03-22T10:40:54.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.290+0000 7f5f5e800700 1 --2- 172.21.15.115:0/271843378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f581033a0 0x7f5f58105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.290+0000 7f5f5e800700 1 --2- 172.21.15.115:0/271843378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f58100a80 0x7f5f58102e60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.290+0000 7f5f5e800700 1 -- 172.21.15.115:0/271843378 >> 172.21.15.115:0/271843378 conn(0x7f5f580fa440 msgr2=0x7f5f580fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:54.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.290+0000 7f5f5e800700 1 -- 172.21.15.115:0/271843378 shutdown_connections 2024-03-22T10:40:54.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.290+0000 7f5f5e800700 1 -- 172.21.15.115:0/271843378 wait complete. 2024-03-22T10:40:54.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.291+0000 7f5f5e800700 1 Processor -- start 2024-03-22T10:40:54.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.292+0000 7f5f5e800700 1 -- start start 2024-03-22T10:40:54.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.292+0000 7f5f5e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f58100a80 0x7f5f581957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:54.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.292+0000 7f5f5e800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f581033a0 0x7f5f58195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:54.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.292+0000 7f5f5e800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58196320 con 0x7f5f581033a0 2024-03-22T10:40:54.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.292+0000 7f5f5e800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58196460 con 0x7f5f58100a80 2024-03-22T10:40:54.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.292+0000 7f5f5ca00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f581033a0 0x7f5f58195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:54.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.292+0000 7f5f5ca00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f581033a0 0x7f5f58195d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51840/0 (socket says 172.21.15.115:51840) 2024-03-22T10:40:54.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.292+0000 7f5f5d400700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f58100a80 0x7f5f581957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:54.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.292+0000 7f5f5ca00700 1 -- 172.21.15.115:0/901574427 learned_addr learned my addr 172.21.15.115:0/901574427 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.293+0000 7f5f5ca00700 1 -- 172.21.15.115:0/901574427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f58100a80 msgr2=0x7f5f581957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.293+0000 7f5f5ca00700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f58100a80 0x7f5f581957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.293+0000 7f5f5ca00700 1 -- 172.21.15.115:0/901574427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0095f0 con 0x7f5f581033a0 2024-03-22T10:40:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.293+0000 7f5f5d400700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f58100a80 0x7f5f581957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T10:40:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.293+0000 7f5f5ca00700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f581033a0 0x7f5f58195d00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400d8d0 tx=0x7f5f4400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.293+0000 7f5f56000700 1 -- 172.21.15.115:0/901574427 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f440098a0 con 0x7f5f581033a0 2024-03-22T10:40:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.293+0000 7f5f56000700 1 -- 172.21.15.115:0/901574427 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5f44010460 con 0x7f5f581033a0 2024-03-22T10:40:54.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.294+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5819af10 con 0x7f5f581033a0 2024-03-22T10:40:54.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.294+0000 7f5f56000700 1 -- 172.21.15.115:0/901574427 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f4400f5f0 con 0x7f5f581033a0 2024-03-22T10:40:54.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.294+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5819b430 con 0x7f5f581033a0 2024-03-22T10:40:54.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.295+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5818f9b0 con 0x7f5f581033a0 2024-03-22T10:40:54.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.296+0000 7f5f56000700 1 -- 172.21.15.115:0/901574427 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f5f44009a00 con 0x7f5f581033a0 2024-03-22T10:40:54.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.297+0000 7f5f56000700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f5f4006c680 0x7f5f4006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:54.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.297+0000 7f5f56000700 1 -- 172.21.15.115:0/901574427 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5f4408b7d0 con 0x7f5f581033a0 2024-03-22T10:40:54.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.297+0000 7f5f5d400700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f5f4006c680 0x7f5f4006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:54.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.298+0000 7f5f5d400700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f5f4006c680 0x7f5f4006eb30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c006010 tx=0x7f5f4c00b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:54.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.301+0000 7f5f56000700 1 -- 172.21.15.115:0/901574427 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5f440598a0 con 0x7f5f581033a0 2024-03-22T10:40:54.313 INFO:tasks.workunit.client.1.smithi150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-22T10:40:54.321 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-22T10:40:54.321 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-22T10:40:54.369 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-22T10:40:54.370 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-22T10:40:54.370 DEBUG:teuthology.orchestra.run.smithi150:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-22T10:40:54.392 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:54 smithi115 ceph-mon[29111]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:54.393 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:54 smithi115 ceph-mon[29111]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:54.443 INFO:tasks.workunit.client.0.smithi115.stderr:+ tar xzf ltp-full.tgz 2024-03-22T10:40:54.494 INFO:tasks.workunit.client.1.smithi150.stderr:+ mkdir -p fsstress 2024-03-22T10:40:54.499 INFO:tasks.workunit.client.1.smithi150.stderr:+ pushd fsstress 2024-03-22T10:40:54.503 INFO:tasks.workunit.client.1.smithi150.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T10:40:54.503 INFO:tasks.workunit.client.1.smithi150.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-22T10:40:54.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.607+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f58061190 con 0x7f5f4006c680 2024-03-22T10:40:54.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.609+0000 7f5f56000700 1 -- 172.21.15.115:0/901574427 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f5f58061190 con 0x7f5f4006c680 2024-03-22T10:40:54.601 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:40:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:40:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:40:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:40:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [], 2024-03-22T10:40:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "0/23 daemons upgraded", 2024-03-22T10:40:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc image on host smithi150", 2024-03-22T10:40:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:40:54.602 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:40:54.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.613+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f5f4006c680 msgr2=0x7f5f4006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:54.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.613+0000 7f5f5e800700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f5f4006c680 0x7f5f4006eb30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c006010 tx=0x7f5f4c00b560 comp rx=0 tx=0).stop 2024-03-22T10:40:54.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.613+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f581033a0 msgr2=0x7f5f58195d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:54.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.613+0000 7f5f5e800700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f581033a0 0x7f5f58195d00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400d8d0 tx=0x7f5f4400dbe0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.613+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 shutdown_connections 2024-03-22T10:40:54.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.613+0000 7f5f5e800700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f5f4006c680 0x7f5f4006eb30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.613+0000 7f5f5e800700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f58100a80 0x7f5f581957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.613+0000 7f5f5e800700 1 --2- 172.21.15.115:0/901574427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f581033a0 0x7f5f58195d00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.613+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 >> 172.21.15.115:0/901574427 conn(0x7f5f580fa440 msgr2=0x7f5f580fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:54.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.614+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 shutdown_connections 2024-03-22T10:40:54.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.614+0000 7f5f5e800700 1 -- 172.21.15.115:0/901574427 wait complete. 2024-03-22T10:40:54.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:54 smithi150 ceph-mon[34518]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:54.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:54 smithi150 ceph-mon[34518]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:54.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:54 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3967324264' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:40:54.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:54 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/4115766196' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:40:54.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:54 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3967324264' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:40:54.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:54 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/4115766196' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:40:54.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.792+0000 7f4a15800700 1 -- 172.21.15.115:0/1056912029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 msgr2=0x7f4a101058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:54.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.792+0000 7f4a15800700 1 --2- 172.21.15.115:0/1056912029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 0x7f4a101058e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009910 tx=0x7f49fc009c20 comp rx=0 tx=0).stop 2024-03-22T10:40:54.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.792+0000 7f4a15800700 1 -- 172.21.15.115:0/1056912029 shutdown_connections 2024-03-22T10:40:54.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.792+0000 7f4a15800700 1 --2- 172.21.15.115:0/1056912029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 0x7f4a101058e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.792+0000 7f4a15800700 1 --2- 172.21.15.115:0/1056912029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a10100be0 0x7f4a10102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.792+0000 7f4a15800700 1 -- 172.21.15.115:0/1056912029 >> 172.21.15.115:0/1056912029 conn(0x7f4a100fa5e0 msgr2=0x7f4a100fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:54.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.793+0000 7f4a15800700 1 -- 172.21.15.115:0/1056912029 shutdown_connections 2024-03-22T10:40:54.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.793+0000 7f4a15800700 1 -- 172.21.15.115:0/1056912029 wait complete. 2024-03-22T10:40:54.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.794+0000 7f4a15800700 1 Processor -- start 2024-03-22T10:40:54.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.794+0000 7f4a15800700 1 -- start start 2024-03-22T10:40:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.794+0000 7f4a15800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a10100be0 0x7f4a10197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.794+0000 7f4a15800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 0x7f4a101980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.794+0000 7f4a15800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a101986c0 con 0x7f4a10103500 2024-03-22T10:40:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.794+0000 7f4a15800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10198800 con 0x7f4a10100be0 2024-03-22T10:40:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.795+0000 7f4a0e000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 0x7f4a101980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.795+0000 7f4a0e000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 0x7f4a101980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51846/0 (socket says 172.21.15.115:51846) 2024-03-22T10:40:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.795+0000 7f4a0e000700 1 -- 172.21.15.115:0/867375958 learned_addr learned my addr 172.21.15.115:0/867375958 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:40:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.795+0000 7f4a0ea00700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a10100be0 0x7f4a10197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.795+0000 7f4a0e000700 1 -- 172.21.15.115:0/867375958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a10100be0 msgr2=0x7f4a10197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.795+0000 7f4a0e000700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a10100be0 0x7f4a10197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.795+0000 7f4a0e000700 1 -- 172.21.15.115:0/867375958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0095f0 con 0x7f4a10103500 2024-03-22T10:40:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.796+0000 7f4a0e000700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 0x7f4a101980a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b5c0 tx=0x7f49fc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.796+0000 7f4a07400700 1 -- 172.21.15.115:0/867375958 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49fc01d070 con 0x7f4a10103500 2024-03-22T10:40:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.796+0000 7f4a07400700 1 -- 172.21.15.115:0/867375958 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f49fc00f460 con 0x7f4a10103500 2024-03-22T10:40:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.796+0000 7f4a07400700 1 -- 172.21.15.115:0/867375958 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49fc021640 con 0x7f4a10103500 2024-03-22T10:40:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.796+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1019d250 con 0x7f4a10103500 2024-03-22T10:40:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.796+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1019d740 con 0x7f4a10103500 2024-03-22T10:40:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.798+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a100fc1c0 con 0x7f4a10103500 2024-03-22T10:40:54.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.799+0000 7f4a07400700 1 -- 172.21.15.115:0/867375958 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f49fc00f5d0 con 0x7f4a10103500 2024-03-22T10:40:54.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.800+0000 7f4a07400700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f49f406c680 0x7f49f406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:40:54.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.800+0000 7f4a07400700 1 -- 172.21.15.115:0/867375958 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f49fc08cc60 con 0x7f4a10103500 2024-03-22T10:40:54.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.800+0000 7f4a0ea00700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f49f406c680 0x7f49f406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:40:54.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.801+0000 7f4a0ea00700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f49f406c680 0x7f49f406eb30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005950 tx=0x7f4a000058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:40:54.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:54.806+0000 7f4a07400700 1 -- 172.21.15.115:0/867375958 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f49fc05adb0 con 0x7f4a10103500 2024-03-22T10:40:55.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.156+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4a1002cc30 con 0x7f4a10103500 2024-03-22T10:40:55.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.157+0000 7f4a07400700 1 -- 172.21.15.115:0/867375958 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f49fc05a940 con 0x7f4a10103500 2024-03-22T10:40:55.149 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:40:55.149 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:40:55.149 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:40:55.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.159+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f49f406c680 msgr2=0x7f49f406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:55.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.159+0000 7f4a15800700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f49f406c680 0x7f49f406eb30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005950 tx=0x7f4a000058e0 comp rx=0 tx=0).stop 2024-03-22T10:40:55.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.159+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 msgr2=0x7f4a101980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:40:55.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.159+0000 7f4a15800700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 0x7f4a101980a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b5c0 tx=0x7f49fc00be20 comp rx=0 tx=0).stop 2024-03-22T10:40:55.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.160+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 shutdown_connections 2024-03-22T10:40:55.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.160+0000 7f4a15800700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f49f406c680 0x7f49f406eb30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:55.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.160+0000 7f4a15800700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a10100be0 0x7f4a10197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:55.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.160+0000 7f4a15800700 1 --2- 172.21.15.115:0/867375958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a10103500 0x7f4a101980a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:40:55.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.160+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 >> 172.21.15.115:0/867375958 conn(0x7f4a100fa5e0 msgr2=0x7f4a100ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:40:55.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.160+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 shutdown_connections 2024-03-22T10:40:55.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:40:55.160+0000 7f4a15800700 1 -- 172.21.15.115:0/867375958 wait complete. 2024-03-22T10:40:55.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:55 smithi115 ceph-mon[29111]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:55.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:55 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/867375958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:40:55.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:55 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:40:55.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:55 smithi150 ceph-mon[34518]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:40:55.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:55 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/867375958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:40:55.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:55 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:40:56.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:56 smithi115 ceph-mon[29111]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:56.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:56 smithi150 ceph-mon[34518]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:40:58.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:40:58 smithi115 ceph-mon[29111]: pgmap v289: 65 pgs: 65 active+clean; 7.5 MiB data, 188 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 590 KiB/s wr, 7 op/s 2024-03-22T10:40:58.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:40:58 smithi150 ceph-mon[34518]: pgmap v289: 65 pgs: 65 active+clean; 7.5 MiB data, 188 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 590 KiB/s wr, 7 op/s 2024-03-22T10:41:00.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:00 smithi115 ceph-mon[29111]: pgmap v290: 65 pgs: 65 active+clean; 12 MiB data, 196 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 957 KiB/s wr, 10 op/s 2024-03-22T10:41:00.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:00 smithi150 ceph-mon[34518]: pgmap v290: 65 pgs: 65 active+clean; 12 MiB data, 196 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 957 KiB/s wr, 10 op/s 2024-03-22T10:41:01.119 INFO:tasks.workunit.client.1.smithi150.stderr:+ tar xzf ltp-full.tgz 2024-03-22T10:41:02.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:02 smithi115 ceph-mon[29111]: pgmap v291: 65 pgs: 65 active+clean; 14 MiB data, 200 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 16 op/s 2024-03-22T10:41:02.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:02 smithi150 ceph-mon[34518]: pgmap v291: 65 pgs: 65 active+clean; 14 MiB data, 200 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 16 op/s 2024-03-22T10:41:04.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:04 smithi150 ceph-mon[34518]: pgmap v292: 65 pgs: 65 active+clean; 20 MiB data, 220 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 25 op/s 2024-03-22T10:41:04.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:04 smithi115 ceph-mon[29111]: pgmap v292: 65 pgs: 65 active+clean; 20 MiB data, 220 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 25 op/s 2024-03-22T10:41:05.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:05 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:41:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:05 smithi150 ceph-mon[34518]: Upgrade: Updating mgr.smithi150.fmoumv 2024-03-22T10:41:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:41:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:41:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:05 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:41:05.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:05 smithi115 ceph-mon[29111]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:41:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:05 smithi115 ceph-mon[29111]: Upgrade: Updating mgr.smithi150.fmoumv 2024-03-22T10:41:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:41:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:41:05.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:05 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:41:06.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:06 smithi150 ceph-mon[34518]: Deploying daemon mgr.smithi150.fmoumv on smithi150 2024-03-22T10:41:06.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:06 smithi150 ceph-mon[34518]: pgmap v293: 65 pgs: 65 active+clean; 25 MiB data, 237 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 36 op/s 2024-03-22T10:41:06.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:06 smithi115 ceph-mon[29111]: Deploying daemon mgr.smithi150.fmoumv on smithi150 2024-03-22T10:41:06.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:06 smithi115 ceph-mon[29111]: pgmap v293: 65 pgs: 65 active+clean; 25 MiB data, 237 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 36 op/s 2024-03-22T10:41:08.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:08 smithi150 ceph-mon[34518]: pgmap v294: 65 pgs: 65 active+clean; 30 MiB data, 248 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 61 op/s 2024-03-22T10:41:08.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:08 smithi115 ceph-mon[29111]: pgmap v294: 65 pgs: 65 active+clean; 30 MiB data, 248 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 61 op/s 2024-03-22T10:41:10.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:10 smithi150 ceph-mon[34518]: pgmap v295: 65 pgs: 65 active+clean; 30 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 59 op/s 2024-03-22T10:41:10.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:10 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:10.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:10 smithi115 ceph-mon[29111]: pgmap v295: 65 pgs: 65 active+clean; 30 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 59 op/s 2024-03-22T10:41:10.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:10 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:12.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:12 smithi150 ceph-mon[34518]: pgmap v296: 65 pgs: 65 active+clean; 31 MiB data, 254 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 65 op/s 2024-03-22T10:41:12.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:12 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:12.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:12 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:12.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:12 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:41:12.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:12 smithi115 ceph-mon[29111]: pgmap v296: 65 pgs: 65 active+clean; 31 MiB data, 254 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 65 op/s 2024-03-22T10:41:12.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:12 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:12.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:12 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:12.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:12 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:41:13.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:13 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:13.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:13 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:14.125 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:13 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:14.125 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:13 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:14 smithi115 ceph-mon[29111]: pgmap v297: 65 pgs: 65 active+clean; 36 MiB data, 268 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 75 op/s 2024-03-22T10:41:15.049 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:14 smithi150 ceph-mon[34518]: pgmap v297: 65 pgs: 65 active+clean; 36 MiB data, 268 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 75 op/s 2024-03-22T10:41:16.978 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:16 smithi115 ceph-mon[29111]: pgmap v298: 65 pgs: 65 active+clean; 36 MiB data, 283 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 73 op/s 2024-03-22T10:41:16.978 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:16.978 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:16 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:17.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:16 smithi150 ceph-mon[34518]: pgmap v298: 65 pgs: 65 active+clean; 36 MiB data, 283 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 73 op/s 2024-03-22T10:41:17.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:17.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:16 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:18.866 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:18 smithi150 ceph-mon[34518]: pgmap v299: 65 pgs: 65 active+clean; 45 MiB data, 305 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 88 op/s 2024-03-22T10:41:18.866 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:18.866 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:18 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:19.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:18 smithi115 ceph-mon[29111]: pgmap v299: 65 pgs: 65 active+clean; 45 MiB data, 305 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 88 op/s 2024-03-22T10:41:19.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:19.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:18 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:21.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:20 smithi115 ceph-mon[29111]: pgmap v300: 65 pgs: 65 active+clean; 45 MiB data, 306 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 72 op/s 2024-03-22T10:41:21.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:20 smithi150 ceph-mon[34518]: pgmap v300: 65 pgs: 65 active+clean; 45 MiB data, 306 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 72 op/s 2024-03-22T10:41:22.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:22 smithi150 ceph-mon[34518]: pgmap v301: 65 pgs: 65 active+clean; 50 MiB data, 325 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 76 op/s 2024-03-22T10:41:22.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:22 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/crt"}]: dispatch 2024-03-22T10:41:22.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:22 smithi150 ceph-mon[34518]: Standby manager daemon smithi150.fmoumv restarted 2024-03-22T10:41:22.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:22 smithi150 ceph-mon[34518]: Standby manager daemon smithi150.fmoumv started 2024-03-22T10:41:22.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:22 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:41:22.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:22 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/key"}]: dispatch 2024-03-22T10:41:22.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:22 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:41:23.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:22 smithi115 ceph-mon[29111]: pgmap v301: 65 pgs: 65 active+clean; 50 MiB data, 325 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 76 op/s 2024-03-22T10:41:23.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:22 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/crt"}]: dispatch 2024-03-22T10:41:23.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:22 smithi115 ceph-mon[29111]: Standby manager daemon smithi150.fmoumv restarted 2024-03-22T10:41:23.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:22 smithi115 ceph-mon[29111]: Standby manager daemon smithi150.fmoumv started 2024-03-22T10:41:23.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:22 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:41:23.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:22 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/key"}]: dispatch 2024-03-22T10:41:23.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:22 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:41:24.045 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:23 smithi150 ceph-mon[34518]: mgrmap e20: smithi115.iqzwsz(active, since 8m), standbys: smithi150.fmoumv 2024-03-22T10:41:24.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:23 smithi115 ceph-mon[29111]: mgrmap e20: smithi115.iqzwsz(active, since 8m), standbys: smithi150.fmoumv 2024-03-22T10:41:25.097 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:24 smithi150 ceph-mon[34518]: pgmap v302: 65 pgs: 65 active+clean; 56 MiB data, 340 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 83 op/s 2024-03-22T10:41:25.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:24 smithi115 ceph-mon[29111]: pgmap v302: 65 pgs: 65 active+clean; 56 MiB data, 340 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 83 op/s 2024-03-22T10:41:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.346+0000 7f4cb8200700 1 -- 172.21.15.115:0/3678141735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0100190 msgr2=0x7f4cb01005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.346+0000 7f4cb8200700 1 --2- 172.21.15.115:0/3678141735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0100190 0x7f4cb01005a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009910 tx=0x7f4ca4009c20 comp rx=0 tx=0).stop 2024-03-22T10:41:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.346+0000 7f4cb8200700 1 -- 172.21.15.115:0/3678141735 shutdown_connections 2024-03-22T10:41:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.346+0000 7f4cb8200700 1 --2- 172.21.15.115:0/3678141735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cb0101390 0x7f4cb01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.346+0000 7f4cb8200700 1 --2- 172.21.15.115:0/3678141735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0100190 0x7f4cb01005a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.346+0000 7f4cb8200700 1 -- 172.21.15.115:0/3678141735 >> 172.21.15.115:0/3678141735 conn(0x7f4cb00fb740 msgr2=0x7f4cb00fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.347+0000 7f4cb8200700 1 -- 172.21.15.115:0/3678141735 shutdown_connections 2024-03-22T10:41:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.347+0000 7f4cb8200700 1 -- 172.21.15.115:0/3678141735 wait complete. 2024-03-22T10:41:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.348+0000 7f4cb8200700 1 Processor -- start 2024-03-22T10:41:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.348+0000 7f4cb8200700 1 -- start start 2024-03-22T10:41:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.349+0000 7f4cb8200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cb0100190 0x7f4cb0197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.349+0000 7f4cb8200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0101390 0x7f4cb01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.349+0000 7f4cb8200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0198710 con 0x7f4cb0101390 2024-03-22T10:41:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.349+0000 7f4cb8200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0198850 con 0x7f4cb0100190 2024-03-22T10:41:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.349+0000 7f4cb4e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0101390 0x7f4cb01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.349+0000 7f4cb4e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0101390 0x7f4cb01980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35150/0 (socket says 172.21.15.115:35150) 2024-03-22T10:41:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.349+0000 7f4cb4e00700 1 -- 172.21.15.115:0/3048554249 learned_addr learned my addr 172.21.15.115:0/3048554249 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:41:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.349+0000 7f4cb5800700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cb0100190 0x7f4cb0197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:25.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.350+0000 7f4cb4e00700 1 -- 172.21.15.115:0/3048554249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cb0100190 msgr2=0x7f4cb0197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:25.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.350+0000 7f4cb4e00700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cb0100190 0x7f4cb0197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.350+0000 7f4cb4e00700 1 -- 172.21.15.115:0/3048554249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca40095f0 con 0x7f4cb0101390 2024-03-22T10:41:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.350+0000 7f4cb4e00700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0101390 0x7f4cb01980f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00d8d0 tx=0x7f4c9c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.350+0000 7f4cae000700 1 -- 172.21.15.115:0/3048554249 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c0098a0 con 0x7f4cb0101390 2024-03-22T10:41:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.350+0000 7f4cae000700 1 -- 172.21.15.115:0/3048554249 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c9c010460 con 0x7f4cb0101390 2024-03-22T10:41:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.350+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb019d300 con 0x7f4cb0101390 2024-03-22T10:41:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.350+0000 7f4cae000700 1 -- 172.21.15.115:0/3048554249 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c00f5f0 con 0x7f4cb0101390 2024-03-22T10:41:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.351+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb019d850 con 0x7f4cb0101390 2024-03-22T10:41:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.352+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb0105470 con 0x7f4cb0101390 2024-03-22T10:41:25.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.353+0000 7f4cae000700 1 -- 172.21.15.115:0/3048554249 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 90806+0+0 (secure 0 0 0) 0x7f4c9c009a00 con 0x7f4cb0101390 2024-03-22T10:41:25.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.354+0000 7f4cae000700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4c9806d150 0x7f4c9806f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:25.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.354+0000 7f4cae000700 1 -- 172.21.15.115:0/3048554249 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4c9c08c590 con 0x7f4cb0101390 2024-03-22T10:41:25.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.355+0000 7f4cb5800700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4c9806d150 0x7f4c9806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:25.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.356+0000 7f4cb5800700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4c9806d150 0x7f4c9806f600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4006010 tx=0x7f4ca400b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.361+0000 7f4cae000700 1 -- 172.21.15.115:0/3048554249 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4c9c05a6e0 con 0x7f4cb0101390 2024-03-22T10:41:25.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.625+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cb0061190 con 0x7f4c9806d150 2024-03-22T10:41:25.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.626+0000 7f4cae000700 1 -- 172.21.15.115:0/3048554249 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f4cb0061190 con 0x7f4c9806d150 2024-03-22T10:41:25.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.629+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4c9806d150 msgr2=0x7f4c9806f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:25.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.629+0000 7f4cb8200700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4c9806d150 0x7f4c9806f600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4006010 tx=0x7f4ca400b560 comp rx=0 tx=0).stop 2024-03-22T10:41:25.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.629+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0101390 msgr2=0x7f4cb01980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:25.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.629+0000 7f4cb8200700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0101390 0x7f4cb01980f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00d8d0 tx=0x7f4c9c00dbe0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.630+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 shutdown_connections 2024-03-22T10:41:25.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.630+0000 7f4cb8200700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f4c9806d150 0x7f4c9806f600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.630+0000 7f4cb8200700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cb0100190 0x7f4cb0197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.630+0000 7f4cb8200700 1 --2- 172.21.15.115:0/3048554249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cb0101390 0x7f4cb01980f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.630+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 >> 172.21.15.115:0/3048554249 conn(0x7f4cb00fb740 msgr2=0x7f4cb01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:25.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.630+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 shutdown_connections 2024-03-22T10:41:25.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.630+0000 7f4cb8200700 1 -- 172.21.15.115:0/3048554249 wait complete. 2024-03-22T10:41:25.639 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:41:25.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.809+0000 7fb467c00700 1 -- 172.21.15.115:0/3490562977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 msgr2=0x7fb460073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.809+0000 7fb467c00700 1 --2- 172.21.15.115:0/3490562977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 0x7fb460073bc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009910 tx=0x7fb44c009c20 comp rx=0 tx=0).stop 2024-03-22T10:41:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.810+0000 7fb467c00700 1 -- 172.21.15.115:0/3490562977 shutdown_connections 2024-03-22T10:41:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.810+0000 7fb467c00700 1 --2- 172.21.15.115:0/3490562977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 0x7fb460073bc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.810+0000 7fb467c00700 1 --2- 172.21.15.115:0/3490562977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460074d20 0x7fb460073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.810+0000 7fb467c00700 1 -- 172.21.15.115:0/3490562977 >> 172.21.15.115:0/3490562977 conn(0x7fb4600fb6e0 msgr2=0x7fb4600fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.810+0000 7fb467c00700 1 -- 172.21.15.115:0/3490562977 shutdown_connections 2024-03-22T10:41:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.810+0000 7fb467c00700 1 -- 172.21.15.115:0/3490562977 wait complete. 2024-03-22T10:41:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.811+0000 7fb467c00700 1 Processor -- start 2024-03-22T10:41:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.811+0000 7fb467c00700 1 -- start start 2024-03-22T10:41:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.812+0000 7fb467c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 0x7fb460197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.812+0000 7fb467c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460074d20 0x7fb460198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.812+0000 7fb467c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb460198690 con 0x7fb460073750 2024-03-22T10:41:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.812+0000 7fb467c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4601987d0 con 0x7fb460074d20 2024-03-22T10:41:25.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.812+0000 7fb465200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 0x7fb460197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:25.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.812+0000 7fb465200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 0x7fb460197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35166/0 (socket says 172.21.15.115:35166) 2024-03-22T10:41:25.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.812+0000 7fb465200700 1 -- 172.21.15.115:0/3315842592 learned_addr learned my addr 172.21.15.115:0/3315842592 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:41:25.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.813+0000 7fb45fe00700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460074d20 0x7fb460198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:25.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.813+0000 7fb465200700 1 -- 172.21.15.115:0/3315842592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460074d20 msgr2=0x7fb460198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:25.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.813+0000 7fb465200700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460074d20 0x7fb460198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:25.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.813+0000 7fb465200700 1 -- 172.21.15.115:0/3315842592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c0095f0 con 0x7fb460073750 2024-03-22T10:41:25.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.813+0000 7fb465200700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 0x7fb460197b30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb45400d8d0 tx=0x7fb45400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:25.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.813+0000 7fb45d600700 1 -- 172.21.15.115:0/3315842592 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb454009940 con 0x7fb460073750 2024-03-22T10:41:25.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.813+0000 7fb45d600700 1 -- 172.21.15.115:0/3315842592 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb454010460 con 0x7fb460073750 2024-03-22T10:41:25.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.813+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46019d280 con 0x7fb460073750 2024-03-22T10:41:25.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.814+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46019d7d0 con 0x7fb460073750 2024-03-22T10:41:25.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.814+0000 7fb45d600700 1 -- 172.21.15.115:0/3315842592 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb45400f5f0 con 0x7fb460073750 2024-03-22T10:41:25.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.815+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46010aeb0 con 0x7fb460073750 2024-03-22T10:41:25.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.817+0000 7fb45d600700 1 -- 172.21.15.115:0/3315842592 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 90806+0+0 (secure 0 0 0) 0x7fb4540105d0 con 0x7fb460073750 2024-03-22T10:41:25.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.818+0000 7fb45d600700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb44806d150 0x7fb44806f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:25.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.818+0000 7fb45d600700 1 -- 172.21.15.115:0/3315842592 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb45408c760 con 0x7fb460073750 2024-03-22T10:41:25.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.819+0000 7fb45fe00700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb44806d150 0x7fb44806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:25.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.820+0000 7fb45fe00700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb44806d150 0x7fb44806f600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005b40 tx=0x7fb44c005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:25.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:25.826+0000 7fb45d600700 1 -- 172.21.15.115:0/3315842592 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb45405a8d0 con 0x7fb460073750 2024-03-22T10:41:26.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:25 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.098+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb46019df20 con 0x7fb44806d150 2024-03-22T10:41:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.100+0000 7fb45d600700 1 -- 172.21.15.115:0/3315842592 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fb46019df20 con 0x7fb44806d150 2024-03-22T10:41:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.103+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb44806d150 msgr2=0x7fb44806f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.103+0000 7fb467c00700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb44806d150 0x7fb44806f600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005b40 tx=0x7fb44c005a90 comp rx=0 tx=0).stop 2024-03-22T10:41:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.103+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 msgr2=0x7fb460197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.103+0000 7fb467c00700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 0x7fb460197b30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb45400d8d0 tx=0x7fb45400dc90 comp rx=0 tx=0).stop 2024-03-22T10:41:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.103+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 shutdown_connections 2024-03-22T10:41:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.103+0000 7fb467c00700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb44806d150 0x7fb44806f600 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.103+0000 7fb467c00700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb460073750 0x7fb460197b30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.103+0000 7fb467c00700 1 --2- 172.21.15.115:0/3315842592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460074d20 0x7fb460198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.103+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 >> 172.21.15.115:0/3315842592 conn(0x7fb4600fb6e0 msgr2=0x7fb460105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.104+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 shutdown_connections 2024-03-22T10:41:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.104+0000 7fb467c00700 1 -- 172.21.15.115:0/3315842592 wait complete. 2024-03-22T10:41:26.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:25 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:26.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.308+0000 7fb349200700 1 -- 172.21.15.115:0/1538315938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344100150 msgr2=0x7fb344100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:26.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.308+0000 7fb349200700 1 --2- 172.21.15.115:0/1538315938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344100150 0x7fb344100560 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb334009910 tx=0x7fb334009c20 comp rx=0 tx=0).stop 2024-03-22T10:41:26.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.309+0000 7fb349200700 1 -- 172.21.15.115:0/1538315938 shutdown_connections 2024-03-22T10:41:26.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.309+0000 7fb349200700 1 --2- 172.21.15.115:0/1538315938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb344101350 0x7fb3441017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.309+0000 7fb349200700 1 --2- 172.21.15.115:0/1538315938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344100150 0x7fb344100560 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.309+0000 7fb349200700 1 -- 172.21.15.115:0/1538315938 >> 172.21.15.115:0/1538315938 conn(0x7fb3440fb6e0 msgr2=0x7fb3440fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:26.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.310+0000 7fb349200700 1 -- 172.21.15.115:0/1538315938 shutdown_connections 2024-03-22T10:41:26.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.310+0000 7fb349200700 1 -- 172.21.15.115:0/1538315938 wait complete. 2024-03-22T10:41:26.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.311+0000 7fb349200700 1 Processor -- start 2024-03-22T10:41:26.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.311+0000 7fb349200700 1 -- start start 2024-03-22T10:41:26.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.311+0000 7fb349200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb344100150 0x7fb344193790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:26.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.311+0000 7fb349200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344101350 0x7fb344193cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:26.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.311+0000 7fb349200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3441942f0 con 0x7fb344101350 2024-03-22T10:41:26.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.311+0000 7fb349200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344194430 con 0x7fb344100150 2024-03-22T10:41:26.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.312+0000 7fb341400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344101350 0x7fb344193cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:26.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.312+0000 7fb341e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb344100150 0x7fb344193790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:26.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.312+0000 7fb341400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344101350 0x7fb344193cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35178/0 (socket says 172.21.15.115:35178) 2024-03-22T10:41:26.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.312+0000 7fb341400700 1 -- 172.21.15.115:0/3036953208 learned_addr learned my addr 172.21.15.115:0/3036953208 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:41:26.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.312+0000 7fb341e00700 1 -- 172.21.15.115:0/3036953208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344101350 msgr2=0x7fb344193cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:26.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.312+0000 7fb341e00700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344101350 0x7fb344193cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.313+0000 7fb341e00700 1 -- 172.21.15.115:0/3036953208 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340095f0 con 0x7fb344100150 2024-03-22T10:41:26.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.313+0000 7fb341e00700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb344100150 0x7fb344193790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb334009910 tx=0x7fb33400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:26.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.313+0000 7fb33aa00700 1 -- 172.21.15.115:0/3036953208 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33401d070 con 0x7fb344100150 2024-03-22T10:41:26.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.314+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb344198e80 con 0x7fb344100150 2024-03-22T10:41:26.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.314+0000 7fb33aa00700 1 -- 172.21.15.115:0/3036953208 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb334005dc0 con 0x7fb344100150 2024-03-22T10:41:26.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.314+0000 7fb33aa00700 1 -- 172.21.15.115:0/3036953208 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb334021410 con 0x7fb344100150 2024-03-22T10:41:26.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.314+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3441992f0 con 0x7fb344100150 2024-03-22T10:41:26.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.315+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb344106ad0 con 0x7fb344100150 2024-03-22T10:41:26.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.316+0000 7fb33aa00700 1 -- 172.21.15.115:0/3036953208 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 20) v1 ==== 90806+0+0 (secure 0 0 0) 0x7fb33400f460 con 0x7fb344100150 2024-03-22T10:41:26.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.318+0000 7fb33aa00700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb328071560 0x7fb328073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:26.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.318+0000 7fb33aa00700 1 -- 172.21.15.115:0/3036953208 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb33408d920 con 0x7fb344100150 2024-03-22T10:41:26.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.318+0000 7fb341400700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb328071560 0x7fb328073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:26.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.319+0000 7fb341400700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb328071560 0x7fb328073a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb330005950 tx=0x7fb33000b500 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.323+0000 7fb33aa00700 1 -- 172.21.15.115:0/3036953208 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb33405ba70 con 0x7fb344100150 2024-03-22T10:41:26.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.611+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 --> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb344105ca0 con 0x7fb328071560 2024-03-22T10:41:26.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.627+0000 7fb33aa00700 1 -- 172.21.15.115:0/3036953208 <== mgr.14213 v2:172.21.15.115:6800/1251137983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb344105ca0 con 0x7fb328071560 2024-03-22T10:41:26.620 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (7m) 2m ago 9m 13.4M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (10m) 2m ago 10m 7486k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (8m) 10s ago 8m 7578k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (10m) 2m ago 10m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (8m) 10s ago 8m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (7m) 2m ago 9m 73.5M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (2m) 2m ago 2m 14.7M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (2m) 2m ago 2m 11.7M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (2m) 10s ago 2m 12.7M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (2m) 10s ago 2m 33.1M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:9283,8765,8443 running (11m) 2m ago 11m 488M - 18.2.0 dc2bc1663786 932346df0e0f 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (15s) 10s ago 8m 53.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 02c6d1a3b66d 2024-03-22T10:41:26.621 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (11m) 2m ago 11m 44.2M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (8m) 10s ago 8m 35.1M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (9m) 2m ago 9m 7746k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (8m) 10s ago 8m 7884k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (7m) 2m ago 7m 45.7M 3947M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (6m) 2m ago 6m 47.1M 3947M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (6m) 2m ago 6m 46.0M 3947M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (5m) 10s ago 5m 54.4M 4971M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (5m) 10s ago 5m 53.6M 4971M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (4m) 10s ago 4m 53.4M 4971M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:41:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (7m) 2m ago 9m 57.0M - 2.43.0 a07b618ecd1d 9fcacfb2c3cb 2024-03-22T10:41:26.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.631+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb328071560 msgr2=0x7fb328073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:26.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.631+0000 7fb349200700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb328071560 0x7fb328073a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb330005950 tx=0x7fb33000b500 comp rx=0 tx=0).stop 2024-03-22T10:41:26.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.631+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb344100150 msgr2=0x7fb344193790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:26.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.631+0000 7fb349200700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb344100150 0x7fb344193790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb334009910 tx=0x7fb33400bd70 comp rx=0 tx=0).stop 2024-03-22T10:41:26.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.631+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 shutdown_connections 2024-03-22T10:41:26.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.631+0000 7fb349200700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7fb328071560 0x7fb328073a10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.631+0000 7fb349200700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb344100150 0x7fb344193790 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.631+0000 7fb349200700 1 --2- 172.21.15.115:0/3036953208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344101350 0x7fb344193cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.631+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 >> 172.21.15.115:0/3036953208 conn(0x7fb3440fb6e0 msgr2=0x7fb344104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:26.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.632+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 shutdown_connections 2024-03-22T10:41:26.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.632+0000 7fb349200700 1 -- 172.21.15.115:0/3036953208 wait complete. 2024-03-22T10:41:26.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.810+0000 7ffa47600700 1 -- 172.21.15.115:0/1714290482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa40100170 msgr2=0x7ffa40100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:26.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.810+0000 7ffa47600700 1 --2- 172.21.15.115:0/1714290482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa40100170 0x7ffa40100580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009870 tx=0x7ffa34009b80 comp rx=0 tx=0).stop 2024-03-22T10:41:26.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.810+0000 7ffa47600700 1 -- 172.21.15.115:0/1714290482 shutdown_connections 2024-03-22T10:41:26.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.810+0000 7ffa47600700 1 --2- 172.21.15.115:0/1714290482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa40101370 0x7ffa401017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.810+0000 7ffa47600700 1 --2- 172.21.15.115:0/1714290482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa40100170 0x7ffa40100580 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.810+0000 7ffa47600700 1 -- 172.21.15.115:0/1714290482 >> 172.21.15.115:0/1714290482 conn(0x7ffa400fb6e0 msgr2=0x7ffa400fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:26.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.810+0000 7ffa47600700 1 -- 172.21.15.115:0/1714290482 shutdown_connections 2024-03-22T10:41:26.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.810+0000 7ffa47600700 1 -- 172.21.15.115:0/1714290482 wait complete. 2024-03-22T10:41:26.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.811+0000 7ffa47600700 1 Processor -- start 2024-03-22T10:41:26.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.812+0000 7ffa47600700 1 -- start start 2024-03-22T10:41:26.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.812+0000 7ffa47600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa40100170 0x7ffa40197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:26.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.812+0000 7ffa47600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa40101370 0x7ffa401980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:26.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.812+0000 7ffa47600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401986e0 con 0x7ffa40101370 2024-03-22T10:41:26.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.812+0000 7ffa47600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40198820 con 0x7ffa40100170 2024-03-22T10:41:26.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.813+0000 7ffa3fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa40101370 0x7ffa401980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:26.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.813+0000 7ffa3fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa40101370 0x7ffa401980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35200/0 (socket says 172.21.15.115:35200) 2024-03-22T10:41:26.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.813+0000 7ffa3fe00700 1 -- 172.21.15.115:0/926500879 learned_addr learned my addr 172.21.15.115:0/926500879 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:41:26.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.813+0000 7ffa44c00700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa40100170 0x7ffa40197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:26.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.813+0000 7ffa3fe00700 1 -- 172.21.15.115:0/926500879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa40100170 msgr2=0x7ffa40197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:26.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.813+0000 7ffa3fe00700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa40100170 0x7ffa40197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:26.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.813+0000 7ffa3fe00700 1 -- 172.21.15.115:0/926500879 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa34009520 con 0x7ffa40101370 2024-03-22T10:41:26.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.814+0000 7ffa3fe00700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa40101370 0x7ffa401980c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000e940 tx=0x7ffa3000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:26.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.814+0000 7ffa3d600700 1 -- 172.21.15.115:0/926500879 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3000eee0 con 0x7ffa40101370 2024-03-22T10:41:26.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.814+0000 7ffa3d600700 1 -- 172.21.15.115:0/926500879 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffa30004ce0 con 0x7ffa40101370 2024-03-22T10:41:26.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.814+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4019d2d0 con 0x7ffa40101370 2024-03-22T10:41:26.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.814+0000 7ffa3d600700 1 -- 172.21.15.115:0/926500879 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa30005650 con 0x7ffa40101370 2024-03-22T10:41:26.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.814+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4019d8a0 con 0x7ffa40101370 2024-03-22T10:41:26.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.816+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa40105450 con 0x7ffa40101370 2024-03-22T10:41:26.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.817+0000 7ffa3d600700 1 -- 172.21.15.115:0/926500879 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 90806+0+0 (secure 0 0 0) 0x7ffa30004750 con 0x7ffa40101370 2024-03-22T10:41:26.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.817+0000 7ffa3d600700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ffa2806d150 0x7ffa2806f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:26.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.817+0000 7ffa3d600700 1 -- 172.21.15.115:0/926500879 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ffa30014070 con 0x7ffa40101370 2024-03-22T10:41:26.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.817+0000 7ffa44c00700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ffa2806d150 0x7ffa2806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:26.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.819+0000 7ffa44c00700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ffa2806d150 0x7ffa2806f600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400b6f0 tx=0x7ffa34019040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:26.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:26.823+0000 7ffa3d600700 1 -- 172.21.15.115:0/926500879 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffa3005ac00 con 0x7ffa40101370 2024-03-22T10:41:27.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:26 smithi115 ceph-mon[29111]: pgmap v303: 65 pgs: 65 active+clean; 57 MiB data, 358 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 71 op/s 2024-03-22T10:41:27.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:26 smithi115 ceph-mon[29111]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:41:27.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:26 smithi150 ceph-mon[34518]: pgmap v303: 65 pgs: 65 active+clean; 57 MiB data, 358 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 71 op/s 2024-03-22T10:41:27.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:26 smithi150 ceph-mon[34518]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:41:27.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.212+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffa4002cc30 con 0x7ffa40101370 2024-03-22T10:41:27.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.212+0000 7ffa3d600700 1 -- 172.21.15.115:0/926500879 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7ffa3005a790 con 0x7ffa40101370 2024-03-22T10:41:27.205 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:41:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:41:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:41:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:41:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-03-22T10:41:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-22T10:41:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:41:27.207 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.215+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ffa2806d150 msgr2=0x7ffa2806f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.215+0000 7ffa47600700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ffa2806d150 0x7ffa2806f600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400b6f0 tx=0x7ffa34019040 comp rx=0 tx=0).stop 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.215+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa40101370 msgr2=0x7ffa401980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.215+0000 7ffa47600700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa40101370 0x7ffa401980c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000e940 tx=0x7ffa3000ed00 comp rx=0 tx=0).stop 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.216+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 shutdown_connections 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.216+0000 7ffa47600700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7ffa2806d150 0x7ffa2806f600 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.216+0000 7ffa47600700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa40100170 0x7ffa40197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.216+0000 7ffa47600700 1 --2- 172.21.15.115:0/926500879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa40101370 0x7ffa401980c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.216+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 >> 172.21.15.115:0/926500879 conn(0x7ffa400fb6e0 msgr2=0x7ffa40102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.216+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 shutdown_connections 2024-03-22T10:41:27.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.216+0000 7ffa47600700 1 -- 172.21.15.115:0/926500879 wait complete. 2024-03-22T10:41:27.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.400+0000 7f5a1c600700 1 -- 172.21.15.115:0/3186563806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 msgr2=0x7f5a141058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:27.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.400+0000 7f5a1c600700 1 --2- 172.21.15.115:0/3186563806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 0x7f5a141058e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009910 tx=0x7f5a00009c20 comp rx=0 tx=0).stop 2024-03-22T10:41:27.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.400+0000 7f5a1c600700 1 -- 172.21.15.115:0/3186563806 shutdown_connections 2024-03-22T10:41:27.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.400+0000 7f5a1c600700 1 --2- 172.21.15.115:0/3186563806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 0x7f5a141058e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:27.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.400+0000 7f5a1c600700 1 --2- 172.21.15.115:0/3186563806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a14100be0 0x7f5a14102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:27.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.400+0000 7f5a1c600700 1 -- 172.21.15.115:0/3186563806 >> 172.21.15.115:0/3186563806 conn(0x7f5a140fa5e0 msgr2=0x7f5a140fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:27.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.400+0000 7f5a1c600700 1 -- 172.21.15.115:0/3186563806 shutdown_connections 2024-03-22T10:41:27.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.400+0000 7f5a1c600700 1 -- 172.21.15.115:0/3186563806 wait complete. 2024-03-22T10:41:27.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.402+0000 7f5a1c600700 1 Processor -- start 2024-03-22T10:41:27.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.403+0000 7f5a1c600700 1 -- start start 2024-03-22T10:41:27.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.403+0000 7f5a1c600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a14100be0 0x7f5a14197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.403+0000 7f5a1c600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 0x7f5a14198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.403+0000 7f5a1c600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14198620 con 0x7f5a14103500 2024-03-22T10:41:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.403+0000 7f5a1c600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14198760 con 0x7f5a14100be0 2024-03-22T10:41:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.404+0000 7f5a19200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 0x7f5a14198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.404+0000 7f5a19200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 0x7f5a14198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35220/0 (socket says 172.21.15.115:35220) 2024-03-22T10:41:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.404+0000 7f5a19200700 1 -- 172.21.15.115:0/655498059 learned_addr learned my addr 172.21.15.115:0/655498059 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:41:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.404+0000 7f5a19c00700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a14100be0 0x7f5a14197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.404+0000 7f5a19200700 1 -- 172.21.15.115:0/655498059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a14100be0 msgr2=0x7f5a14197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.404+0000 7f5a19200700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a14100be0 0x7f5a14197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.404+0000 7f5a19200700 1 -- 172.21.15.115:0/655498059 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a000095f0 con 0x7f5a14103500 2024-03-22T10:41:27.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.405+0000 7f5a19200700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 0x7f5a14198090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000b5c0 tx=0x7f5a0000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:27.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.406+0000 7f5a12000700 1 -- 172.21.15.115:0/655498059 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a0001d070 con 0x7f5a14103500 2024-03-22T10:41:27.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.406+0000 7f5a12000700 1 -- 172.21.15.115:0/655498059 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a00022470 con 0x7f5a14103500 2024-03-22T10:41:27.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.406+0000 7f5a1c600700 1 -- 172.21.15.115:0/655498059 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1419d1c0 con 0x7f5a14103500 2024-03-22T10:41:27.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.406+0000 7f5a12000700 1 -- 172.21.15.115:0/655498059 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a0000f690 con 0x7f5a14103500 2024-03-22T10:41:27.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.406+0000 7f5a1c600700 1 -- 172.21.15.115:0/655498059 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1419d680 con 0x7f5a14103500 2024-03-22T10:41:27.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.408+0000 7f5a1c600700 1 -- 172.21.15.115:0/655498059 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a14191d70 con 0x7f5a14103500 2024-03-22T10:41:27.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.409+0000 7f5a12000700 1 -- 172.21.15.115:0/655498059 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 90806+0+0 (secure 0 0 0) 0x7f5a000225e0 con 0x7f5a14103500 2024-03-22T10:41:27.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.410+0000 7f5a12000700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f59fc06d1a0 0x7f59fc06f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:27.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.410+0000 7f5a12000700 1 -- 172.21.15.115:0/655498059 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5a0008dd20 con 0x7f5a14103500 2024-03-22T10:41:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.411+0000 7f5a19c00700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f59fc06d1a0 0x7f59fc06f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:27.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.412+0000 7f5a19c00700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f59fc06d1a0 0x7f59fc06f650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800ac50 tx=0x7f5a0800a380 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:27.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.417+0000 7f5a12000700 1 -- 172.21.15.115:0/655498059 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5a0005bfb0 con 0x7f5a14103500 2024-03-22T10:41:27.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.681+0000 7f5a12000700 1 -- 172.21.15.115:0/655498059 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 21) v1 ==== 45768+0+0 (secure 0 0 0) 0x7f5a0006fad0 con 0x7f5a14103500 2024-03-22T10:41:27.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.681+0000 7f5a12000700 1 -- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f59fc06d1a0 msgr2=0x7f59fc06f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:27.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.681+0000 7f5a12000700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f59fc06d1a0 0x7f59fc06f650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800ac50 tx=0x7f5a0800a380 comp rx=0 tx=0).stop 2024-03-22T10:41:27.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.813+0000 7f5a1c600700 1 -- 172.21.15.115:0/655498059 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5a1402cc30 con 0x7f5a14103500 2024-03-22T10:41:27.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.814+0000 7f5a12000700 1 -- 172.21.15.115:0/655498059 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1856 (secure 0 0 0) 0x7f5a00027750 con 0x7f5a14103500 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:e13 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 13 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:41:27.806 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:41:04.070466+0000 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:41:27.807 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:41:27.808 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:41:27.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.817+0000 7f5a1c600700 1 -- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 msgr2=0x7f5a14198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:27.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.817+0000 7f5a1c600700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 0x7f5a14198090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000b5c0 tx=0x7f5a0000bd70 comp rx=0 tx=0).stop 2024-03-22T10:41:27.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.817+0000 7f5a1c600700 1 -- 172.21.15.115:0/655498059 shutdown_connections 2024-03-22T10:41:27.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.817+0000 7f5a1c600700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:6800/1251137983,v1:172.21.15.115:6801/1251137983] conn(0x7f59fc06d1a0 0x7f59fc06f650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:27.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.817+0000 7f5a1c600700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a14100be0 0x7f5a14197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.817+0000 7f5a1c600700 1 --2- 172.21.15.115:0/655498059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a14103500 0x7f5a14198090 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.817+0000 7f5a1c600700 1 -- 172.21.15.115:0/655498059 >> 172.21.15.115:0/655498059 conn(0x7f5a140fa5e0 msgr2=0x7f5a140fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.818+0000 7f5a1c600700 1 -- 172.21.15.115:0/655498059 shutdown_connections 2024-03-22T10:41:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:27.818+0000 7f5a1c600700 1 -- 172.21.15.115:0/655498059 wait complete. 2024-03-22T10:41:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 13 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/926500879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:41:28.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr fail", "who": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: osdmap e46: 6 total, 6 up, 6 in 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "mgr fail", "who": "smithi115.iqzwsz"}]': finished 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: mgrmap e21: smithi150.fmoumv(active, starting, since 0.011766s) 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr metadata", "who": "smithi150.fmoumv", "id": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:41:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:41:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:41:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:41:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:41:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:41:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: Manager daemon smithi150.fmoumv is now available 2024-03-22T10:41:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:41:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:27 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:41:28.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.026+0000 7f6679000700 1 -- 172.21.15.115:0/342961524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674100be0 msgr2=0x7f6674102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:28.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.026+0000 7f6679000700 1 --2- 172.21.15.115:0/342961524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674100be0 0x7f6674102fc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f665c009910 tx=0x7f665c009c20 comp rx=0 tx=0).stop 2024-03-22T10:41:28.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.026+0000 7f6679000700 1 -- 172.21.15.115:0/342961524 shutdown_connections 2024-03-22T10:41:28.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.026+0000 7f6679000700 1 --2- 172.21.15.115:0/342961524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6674103500 0x7f66741058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.026+0000 7f6679000700 1 --2- 172.21.15.115:0/342961524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674100be0 0x7f6674102fc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.026+0000 7f6679000700 1 -- 172.21.15.115:0/342961524 >> 172.21.15.115:0/342961524 conn(0x7f66740fa5e0 msgr2=0x7f66740fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:28.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.027+0000 7f6679000700 1 -- 172.21.15.115:0/342961524 shutdown_connections 2024-03-22T10:41:28.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.027+0000 7f6679000700 1 -- 172.21.15.115:0/342961524 wait complete. 2024-03-22T10:41:28.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.028+0000 7f6679000700 1 Processor -- start 2024-03-22T10:41:28.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.029+0000 7f6679000700 1 -- start start 2024-03-22T10:41:28.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.029+0000 7f6679000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6674100be0 0x7f667419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:28.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.029+0000 7f6679000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674103500 0x7f667419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:28.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.029+0000 7f6679000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667419cac0 con 0x7f6674103500 2024-03-22T10:41:28.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.029+0000 7f6679000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667419cc00 con 0x7f6674100be0 2024-03-22T10:41:28.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.030+0000 7f6671400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674103500 0x7f667419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:28.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.030+0000 7f6671400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674103500 0x7f667419c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35276/0 (socket says 172.21.15.115:35276) 2024-03-22T10:41:28.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.030+0000 7f6671400700 1 -- 172.21.15.115:0/138497686 learned_addr learned my addr 172.21.15.115:0/138497686 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:41:28.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.030+0000 7f6671e00700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6674100be0 0x7f667419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:28.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.031+0000 7f6671400700 1 -- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6674100be0 msgr2=0x7f667419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:28.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.031+0000 7f6671400700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6674100be0 0x7f667419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.031+0000 7f6671400700 1 -- 172.21.15.115:0/138497686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0095f0 con 0x7f6674103500 2024-03-22T10:41:28.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.031+0000 7f6671400700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674103500 0x7f667419c4a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f666400e920 tx=0x7f666400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:28.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.032+0000 7f666aa00700 1 -- 172.21.15.115:0/138497686 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666400cba0 con 0x7f6674103500 2024-03-22T10:41:28.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.032+0000 7f6679000700 1 -- 172.21.15.115:0/138497686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66741a16b0 con 0x7f6674103500 2024-03-22T10:41:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.032+0000 7f666aa00700 1 -- 172.21.15.115:0/138497686 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f666400cd00 con 0x7f6674103500 2024-03-22T10:41:28.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.032+0000 7f666aa00700 1 -- 172.21.15.115:0/138497686 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6664018770 con 0x7f6674103500 2024-03-22T10:41:28.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.032+0000 7f6679000700 1 -- 172.21.15.115:0/138497686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66741a1c00 con 0x7f6674103500 2024-03-22T10:41:28.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.035+0000 7f6679000700 1 -- 172.21.15.115:0/138497686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6674108cb0 con 0x7f6674103500 2024-03-22T10:41:28.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.035+0000 7f666aa00700 1 -- 172.21.15.115:0/138497686 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 21) v1 ==== 45768+0+0 (secure 0 0 0) 0x7f6664010430 con 0x7f6674103500 2024-03-22T10:41:28.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.036+0000 7f666aa00700 1 -- 172.21.15.115:0/138497686 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f6664014070 con 0x7f6674103500 2024-03-22T10:41:28.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.047+0000 7f666aa00700 1 -- 172.21.15.115:0/138497686 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6664051070 con 0x7f6674103500 2024-03-22T10:41:28.084 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:41:28.084 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:41:28.084 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/926500879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:41:28.084 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:28.084 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:28.084 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:41:28.084 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:41:28.084 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:41:28.084 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr fail", "who": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: osdmap e46: 6 total, 6 up, 6 in 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.14213 172.21.15.115:0/2099099465' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "mgr fail", "who": "smithi115.iqzwsz"}]': finished 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: mgrmap e21: smithi150.fmoumv(active, starting, since 0.011766s) 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr metadata", "who": "smithi150.fmoumv", "id": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:41:28.085 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:41:28.086 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:41:28.086 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:41:28.086 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:41:28.086 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:41:28.086 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:41:28.086 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:41:28.086 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: Manager daemon smithi150.fmoumv is now available 2024-03-22T10:41:28.086 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:41:28.086 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:27 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:41:28.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.684+0000 7f666aa00700 1 -- 172.21.15.115:0/138497686 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 22) v1 ==== 46191+0+0 (secure 0 0 0) 0x7f6664007d30 con 0x7f6674103500 2024-03-22T10:41:28.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.685+0000 7f666aa00700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f665803e080 0x7f6658040460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:28.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.685+0000 7f666aa00700 1 -- 172.21.15.115:0/138497686 --> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66580409a0 con 0x7f665803e080 2024-03-22T10:41:28.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.694+0000 7f6671e00700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f665803e080 0x7f6658040460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:28.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.697+0000 7f6671e00700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f665803e080 0x7f6658040460 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f665c006010 tx=0x7f665c009f70 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:28.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.702+0000 7f666aa00700 1 -- 172.21.15.115:0/138497686 <== mgr.24365 v2:172.21.15.150:6828/3274858902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f66580409a0 con 0x7f665803e080 2024-03-22T10:41:28.695 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:41:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:41:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:41:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:41:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [], 2024-03-22T10:41:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "1/23 daemons upgraded", 2024-03-22T10:41:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "", 2024-03-22T10:41:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:41:28.696 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:41:28.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.705+0000 7f6679000700 1 -- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f665803e080 msgr2=0x7f6658040460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:28.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.705+0000 7f6679000700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f665803e080 0x7f6658040460 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f665c006010 tx=0x7f665c009f70 comp rx=0 tx=0).stop 2024-03-22T10:41:28.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.705+0000 7f6679000700 1 -- 172.21.15.115:0/138497686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674103500 msgr2=0x7f667419c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:28.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.705+0000 7f6679000700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674103500 0x7f667419c4a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f666400e920 tx=0x7f666400ece0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.705+0000 7f6679000700 1 -- 172.21.15.115:0/138497686 shutdown_connections 2024-03-22T10:41:28.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.705+0000 7f6679000700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f665803e080 0x7f6658040460 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.705+0000 7f6679000700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6674100be0 0x7f667419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.705+0000 7f6679000700 1 --2- 172.21.15.115:0/138497686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6674103500 0x7f667419c4a0 secure :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f666400e920 tx=0x7f666400ece0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.705+0000 7f6679000700 1 -- 172.21.15.115:0/138497686 >> 172.21.15.115:0/138497686 conn(0x7f66740fa5e0 msgr2=0x7f66740fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:28.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.706+0000 7f6679000700 1 -- 172.21.15.115:0/138497686 shutdown_connections 2024-03-22T10:41:28.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.706+0000 7f6679000700 1 -- 172.21.15.115:0/138497686 wait complete. 2024-03-22T10:41:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.904+0000 7fbbb2000700 1 -- 172.21.15.115:0/356464092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 msgr2=0x7fbbac0ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.904+0000 7fbbb2000700 1 --2- 172.21.15.115:0/356464092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 0x7fbbac0ff460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009910 tx=0x7fbb9c009c20 comp rx=0 tx=0).stop 2024-03-22T10:41:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.904+0000 7fbbb2000700 1 -- 172.21.15.115:0/356464092 shutdown_connections 2024-03-22T10:41:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.904+0000 7fbbb2000700 1 --2- 172.21.15.115:0/356464092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbac100250 0x7fbbac1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.904+0000 7fbbb2000700 1 --2- 172.21.15.115:0/356464092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 0x7fbbac0ff460 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.904+0000 7fbbb2000700 1 -- 172.21.15.115:0/356464092 >> 172.21.15.115:0/356464092 conn(0x7fbbac0fa5e0 msgr2=0x7fbbac0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:28.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.905+0000 7fbbb2000700 1 -- 172.21.15.115:0/356464092 shutdown_connections 2024-03-22T10:41:28.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.905+0000 7fbbb2000700 1 -- 172.21.15.115:0/356464092 wait complete. 2024-03-22T10:41:28.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.907+0000 7fbbb2000700 1 Processor -- start 2024-03-22T10:41:28.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.907+0000 7fbbb2000700 1 -- start start 2024-03-22T10:41:28.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.907+0000 7fbbb2000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 0x7fbbac102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:28.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.907+0000 7fbbb2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbac100250 0x7fbbac101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:28.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.907+0000 7fbbb2000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac101b70 con 0x7fbbac0ff050 2024-03-22T10:41:28.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.907+0000 7fbbb2000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac101cb0 con 0x7fbbac100250 2024-03-22T10:41:28.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.908+0000 7fbbab400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 0x7fbbac102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:28.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.908+0000 7fbbab400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 0x7fbbac102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35282/0 (socket says 172.21.15.115:35282) 2024-03-22T10:41:28.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.908+0000 7fbbab400700 1 -- 172.21.15.115:0/3596086330 learned_addr learned my addr 172.21.15.115:0/3596086330 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:41:28.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.908+0000 7fbbaaa00700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbac100250 0x7fbbac101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:28.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.909+0000 7fbbab400700 1 -- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbac100250 msgr2=0x7fbbac101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:28.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.909+0000 7fbbab400700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbac100250 0x7fbbac101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:28.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.909+0000 7fbbab400700 1 -- 172.21.15.115:0/3596086330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c0095f0 con 0x7fbbac0ff050 2024-03-22T10:41:28.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.909+0000 7fbbab400700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 0x7fbbac102fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00f690 tx=0x7fbb9c00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.910+0000 7fbba3e00700 1 -- 172.21.15.115:0/3596086330 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb9c01d070 con 0x7fbbac0ff050 2024-03-22T10:41:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.910+0000 7fbba3e00700 1 -- 172.21.15.115:0/3596086330 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c005dc0 con 0x7fbbac0ff050 2024-03-22T10:41:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.910+0000 7fbba3e00700 1 -- 172.21.15.115:0/3596086330 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb9c017630 con 0x7fbbac0ff050 2024-03-22T10:41:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.911+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac101f30 con 0x7fbbac0ff050 2024-03-22T10:41:28.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.911+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac102420 con 0x7fbbac0ff050 2024-03-22T10:41:28.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.913+0000 7fbba3e00700 1 -- 172.21.15.115:0/3596086330 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 22) v1 ==== 46191+0+0 (secure 0 0 0) 0x7fbb9c003910 con 0x7fbbac0ff050 2024-03-22T10:41:28.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.913+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac107ba0 con 0x7fbbac0ff050 2024-03-22T10:41:28.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.914+0000 7fbba3e00700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fbb90039200 0x7fbb9003b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:28.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.914+0000 7fbba3e00700 1 -- 172.21.15.115:0/3596086330 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fbb9c04e6c0 con 0x7fbbac0ff050 2024-03-22T10:41:28.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.916+0000 7fbbaaa00700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fbb90039200 0x7fbb9003b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:28.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.918+0000 7fbbaaa00700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fbb90039200 0x7fbb9003b6b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009c60 tx=0x7fbb98009260 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:28.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:28.925+0000 7fbba3e00700 1 -- 172.21.15.115:0/3596086330 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fbb9c0163b0 con 0x7fbbac0ff050 2024-03-22T10:41:29.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:28 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/655498059' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:41:29.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:28 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:29.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:28 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi150.fmoumv/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:41:29.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:28 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi150.fmoumv/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:41:29.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:28 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi150.fmoumv/trash_purge_schedule"}]: dispatch 2024-03-22T10:41:29.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:28 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi150.fmoumv/trash_purge_schedule"}]: dispatch 2024-03-22T10:41:29.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:28 smithi115 ceph-mon[29111]: mgrmap e22: smithi150.fmoumv(active, since 1.01296s) 2024-03-22T10:41:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:28 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/655498059' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:41:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:28 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:28 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi150.fmoumv/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:41:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:28 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi150.fmoumv/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:41:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:28 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi150.fmoumv/trash_purge_schedule"}]: dispatch 2024-03-22T10:41:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:28 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi150.fmoumv/trash_purge_schedule"}]: dispatch 2024-03-22T10:41:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:28 smithi150 ceph-mon[34518]: mgrmap e22: smithi150.fmoumv(active, since 1.01296s) 2024-03-22T10:41:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.383+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbbac19de80 con 0x7fbbac0ff050 2024-03-22T10:41:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.384+0000 7fbba3e00700 1 -- 172.21.15.115:0/3596086330 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fbb9c021c50 con 0x7fbbac0ff050 2024-03-22T10:41:29.376 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:41:29.376 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:41:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:41:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fbb90039200 msgr2=0x7fbb9003b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fbb90039200 0x7fbb9003b6b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009c60 tx=0x7fbb98009260 comp rx=0 tx=0).stop 2024-03-22T10:41:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 msgr2=0x7fbbac102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 0x7fbbac102fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00f690 tx=0x7fbb9c00f6c0 comp rx=0 tx=0).stop 2024-03-22T10:41:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 shutdown_connections 2024-03-22T10:41:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fbb90039200 0x7fbb9003b6b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0ff050 0x7fbbac102fe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 --2- 172.21.15.115:0/3596086330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbac100250 0x7fbbac101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 >> 172.21.15.115:0/3596086330 conn(0x7fbbac0fa5e0 msgr2=0x7fbbac0683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 shutdown_connections 2024-03-22T10:41:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:29.388+0000 7fbbb2000700 1 -- 172.21.15.115:0/3596086330 wait complete. 2024-03-22T10:41:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:29 smithi150 ceph-mon[34518]: [22/Mar/2024:10:41:28] ENGINE Bus STARTING 2024-03-22T10:41:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:29 smithi150 ceph-mon[34518]: [22/Mar/2024:10:41:28] ENGINE Serving on https://172.21.15.150:7150 2024-03-22T10:41:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:29 smithi150 ceph-mon[34518]: [22/Mar/2024:10:41:28] ENGINE Serving on http://172.21.15.150:8765 2024-03-22T10:41:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:29 smithi150 ceph-mon[34518]: [22/Mar/2024:10:41:28] ENGINE Bus STARTED 2024-03-22T10:41:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:29 smithi150 ceph-mon[34518]: Deploying cephadm binary to smithi150 2024-03-22T10:41:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:29 smithi150 ceph-mon[34518]: Deploying cephadm binary to smithi115 2024-03-22T10:41:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:29 smithi150 ceph-mon[34518]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:41:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:29 smithi150 ceph-mon[34518]: pgmap v3: 65 pgs: 65 active+clean; 73 MiB data, 407 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:41:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:29 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3596086330' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:41:30.219 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:29 smithi115 ceph-mon[29111]: [22/Mar/2024:10:41:28] ENGINE Bus STARTING 2024-03-22T10:41:30.219 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:29 smithi115 ceph-mon[29111]: [22/Mar/2024:10:41:28] ENGINE Serving on https://172.21.15.150:7150 2024-03-22T10:41:30.219 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:29 smithi115 ceph-mon[29111]: [22/Mar/2024:10:41:28] ENGINE Serving on http://172.21.15.150:8765 2024-03-22T10:41:30.219 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:29 smithi115 ceph-mon[29111]: [22/Mar/2024:10:41:28] ENGINE Bus STARTED 2024-03-22T10:41:30.220 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:29 smithi115 ceph-mon[29111]: Deploying cephadm binary to smithi150 2024-03-22T10:41:30.220 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:29 smithi115 ceph-mon[29111]: Deploying cephadm binary to smithi115 2024-03-22T10:41:30.220 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:29 smithi115 ceph-mon[29111]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:41:30.220 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:29 smithi115 ceph-mon[29111]: pgmap v3: 65 pgs: 65 active+clean; 73 MiB data, 407 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:41:30.220 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:29 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3596086330' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:41:31.068 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:30 smithi115 ceph-mon[29111]: pgmap v4: 65 pgs: 65 active+clean; 73 MiB data, 407 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:41:31.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:30 smithi150 ceph-mon[34518]: pgmap v4: 65 pgs: 65 active+clean; 73 MiB data, 407 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:41:32.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:31 smithi150 ceph-mon[34518]: mgrmap e23: smithi150.fmoumv(active, since 3s) 2024-03-22T10:41:32.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:31 smithi115 ceph-mon[29111]: mgrmap e23: smithi150.fmoumv(active, since 3s) 2024-03-22T10:41:33.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:32 smithi150 ceph-mon[34518]: pgmap v5: 65 pgs: 65 active+clean; 73 MiB data, 407 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:41:33.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:32 smithi115 ceph-mon[29111]: pgmap v5: 65 pgs: 65 active+clean; 73 MiB data, 407 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:41:34.254 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:34 smithi115 ceph-mon[29111]: pgmap v6: 65 pgs: 65 active+clean; 80 MiB data, 441 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 95 op/s 2024-03-22T10:41:34.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:34 smithi150 ceph-mon[34518]: pgmap v6: 65 pgs: 65 active+clean; 80 MiB data, 441 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 95 op/s 2024-03-22T10:41:36.755 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:36 smithi115 ceph-mon[29111]: pgmap v7: 65 pgs: 65 active+clean; 89 MiB data, 460 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.3 MiB/s wr, 122 op/s 2024-03-22T10:41:37.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:36 smithi150 ceph-mon[34518]: pgmap v7: 65 pgs: 65 active+clean; 89 MiB data, 460 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.3 MiB/s wr, 122 op/s 2024-03-22T10:41:38.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:38 smithi115 ceph-mon[29111]: pgmap v8: 65 pgs: 65 active+clean; 89 MiB data, 460 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-03-22T10:41:38.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:38 smithi115 ceph-mon[29111]: Standby manager daemon smithi115.iqzwsz started 2024-03-22T10:41:38.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:38 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.115:0/1090942604' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.iqzwsz/crt"}]: dispatch 2024-03-22T10:41:38.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:38 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.115:0/1090942604' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:41:38.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:38 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.115:0/1090942604' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.iqzwsz/key"}]: dispatch 2024-03-22T10:41:38.869 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:38 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.115:0/1090942604' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:41:39.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:38 smithi150 ceph-mon[34518]: pgmap v8: 65 pgs: 65 active+clean; 89 MiB data, 460 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-03-22T10:41:39.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:38 smithi150 ceph-mon[34518]: Standby manager daemon smithi115.iqzwsz started 2024-03-22T10:41:39.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:38 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.115:0/1090942604' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.iqzwsz/crt"}]: dispatch 2024-03-22T10:41:39.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:38 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.115:0/1090942604' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:41:39.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:38 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.115:0/1090942604' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.iqzwsz/key"}]: dispatch 2024-03-22T10:41:39.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:38 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.115:0/1090942604' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:41:40.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:40 smithi150 ceph-mon[34518]: mgrmap e24: smithi150.fmoumv(active, since 11s), standbys: smithi115.iqzwsz 2024-03-22T10:41:40.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:40 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:41:40.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:40 smithi150 ceph-mon[34518]: pgmap v9: 65 pgs: 65 active+clean; 104 MiB data, 520 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 2.9 MiB/s wr, 120 op/s 2024-03-22T10:41:40.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:40 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:40.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:40 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:40.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:40 smithi115 ceph-mon[29111]: mgrmap e24: smithi150.fmoumv(active, since 11s), standbys: smithi115.iqzwsz 2024-03-22T10:41:40.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:40 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:41:40.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:40 smithi115 ceph-mon[29111]: pgmap v9: 65 pgs: 65 active+clean; 104 MiB data, 520 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 2.9 MiB/s wr, 120 op/s 2024-03-22T10:41:40.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:40 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:40.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:40 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:43.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:42 smithi115 ceph-mon[29111]: pgmap v10: 65 pgs: 65 active+clean; 104 MiB data, 520 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 110 op/s 2024-03-22T10:41:43.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:42 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:43.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:42 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:43.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:42 smithi150 ceph-mon[34518]: pgmap v10: 65 pgs: 65 active+clean; 104 MiB data, 520 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 110 op/s 2024-03-22T10:41:43.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:42 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:43.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:42 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:44.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:43 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:44.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:43 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:44.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:43 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:44.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:43 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:44.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:43 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:44.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:43 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:45.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:44 smithi115 ceph-mon[29111]: pgmap v11: 65 pgs: 65 active+clean; 113 MiB data, 552 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 145 op/s 2024-03-22T10:41:45.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:44 smithi150 ceph-mon[34518]: pgmap v11: 65 pgs: 65 active+clean; 113 MiB data, 552 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 145 op/s 2024-03-22T10:41:46.622 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:46.622 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:46.622 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.622 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.623 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.623 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.623 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.623 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.623 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: Adjusting osd_memory_target on smithi115 to 1217M 2024-03-22T10:41:46.623 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:46.623 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:46 smithi150 ceph-mon[34518]: pgmap v12: 65 pgs: 65 active+clean; 113 MiB data, 555 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.8 MiB/s wr, 112 op/s 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: Adjusting osd_memory_target on smithi115 to 1217M 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:46.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:46 smithi115 ceph-mon[29111]: pgmap v12: 65 pgs: 65 active+clean; 113 MiB data, 555 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.8 MiB/s wr, 112 op/s 2024-03-22T10:41:49.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:48 smithi115 ceph-mon[29111]: pgmap v13: 65 pgs: 65 active+clean; 113 MiB data, 555 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 80 op/s 2024-03-22T10:41:49.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:48 smithi150 ceph-mon[34518]: pgmap v13: 65 pgs: 65 active+clean; 113 MiB data, 555 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 80 op/s 2024-03-22T10:41:50.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:41:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:49 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:50.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:41:50.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:49 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:41:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:50 smithi150 ceph-mon[34518]: Adjusting osd_memory_target on smithi150 to 2241M 2024-03-22T10:41:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:50 smithi150 ceph-mon[34518]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-22T10:41:51.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:50 smithi150 ceph-mon[34518]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-22T10:41:51.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:50 smithi150 ceph-mon[34518]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:41:51.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:50 smithi150 ceph-mon[34518]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:41:51.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:50 smithi150 ceph-mon[34518]: pgmap v14: 65 pgs: 65 active+clean; 121 MiB data, 586 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.7 MiB/s wr, 113 op/s 2024-03-22T10:41:51.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:50 smithi115 ceph-mon[29111]: Adjusting osd_memory_target on smithi150 to 2241M 2024-03-22T10:41:51.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:50 smithi115 ceph-mon[29111]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-22T10:41:51.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:50 smithi115 ceph-mon[29111]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-22T10:41:51.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:50 smithi115 ceph-mon[29111]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:41:51.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:50 smithi115 ceph-mon[29111]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:41:51.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:50 smithi115 ceph-mon[29111]: pgmap v14: 65 pgs: 65 active+clean; 121 MiB data, 586 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.7 MiB/s wr, 113 op/s 2024-03-22T10:41:52.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:51 smithi150 ceph-mon[34518]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:41:52.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:51 smithi150 ceph-mon[34518]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:41:52.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:51 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:51 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:51 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:51 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:51 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:51 smithi115 ceph-mon[29111]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:41:52.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:51 smithi115 ceph-mon[29111]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:41:52.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:51 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:51 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:51 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:51 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:51 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.877 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:52 smithi150 ceph-mon[34518]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:41:52.877 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:52 smithi150 ceph-mon[34518]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:41:52.877 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:52 smithi150 ceph-mon[34518]: pgmap v15: 65 pgs: 65 active+clean; 121 MiB data, 586 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 74 op/s 2024-03-22T10:41:52.877 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:52 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.877 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:52 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.877 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:52 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:52.877 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:52 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:52 smithi115 ceph-mon[29111]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:41:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:52 smithi115 ceph-mon[29111]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:41:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:52 smithi115 ceph-mon[29111]: pgmap v15: 65 pgs: 65 active+clean; 121 MiB data, 586 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 74 op/s 2024-03-22T10:41:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:52 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:52 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:52 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:52 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:41:54.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:53 smithi150 ceph-mon[34518]: Reconfiguring prometheus.smithi115 (dependencies changed)... 2024-03-22T10:41:54.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:53 smithi150 ceph-mon[34518]: Reconfiguring daemon prometheus.smithi115 on smithi115 2024-03-22T10:41:54.170 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:53 smithi115 ceph-mon[29111]: Reconfiguring prometheus.smithi115 (dependencies changed)... 2024-03-22T10:41:54.170 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:53 smithi115 ceph-mon[29111]: Reconfiguring daemon prometheus.smithi115 on smithi115 2024-03-22T10:41:55.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:54 smithi115 ceph-mon[29111]: pgmap v16: 65 pgs: 65 active+clean; 131 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.3 MiB/s wr, 122 op/s 2024-03-22T10:41:55.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:54 smithi150 ceph-mon[34518]: pgmap v16: 65 pgs: 65 active+clean; 131 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.3 MiB/s wr, 122 op/s 2024-03-22T10:41:57.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:56 smithi150 ceph-mon[34518]: pgmap v17: 65 pgs: 65 active+clean; 131 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 87 op/s 2024-03-22T10:41:57.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:56 smithi115 ceph-mon[29111]: pgmap v17: 65 pgs: 65 active+clean; 131 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 87 op/s 2024-03-22T10:41:58.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:57 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:58.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:57 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:41:59.167 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:58 smithi115 ceph-mon[29111]: pgmap v18: 65 pgs: 65 active+clean; 131 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 80 op/s 2024-03-22T10:41:59.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:58 smithi150 ceph-mon[34518]: pgmap v18: 65 pgs: 65 active+clean; 131 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 80 op/s 2024-03-22T10:41:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.575+0000 7f7782400700 1 -- 172.21.15.115:0/2492897471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 msgr2=0x7f777c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.575+0000 7f7782400700 1 --2- 172.21.15.115:0/2492897471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 0x7f777c102fe0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f776c009910 tx=0x7f776c009c20 comp rx=0 tx=0).stop 2024-03-22T10:41:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.575+0000 7f7782400700 1 -- 172.21.15.115:0/2492897471 shutdown_connections 2024-03-22T10:41:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.575+0000 7f7782400700 1 --2- 172.21.15.115:0/2492897471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f777c103520 0x7f777c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:59.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.575+0000 7f7782400700 1 --2- 172.21.15.115:0/2492897471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 0x7f777c102fe0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:59.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.575+0000 7f7782400700 1 -- 172.21.15.115:0/2492897471 >> 172.21.15.115:0/2492897471 conn(0x7f777c0fa600 msgr2=0x7f777c0fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:59.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.575+0000 7f7782400700 1 -- 172.21.15.115:0/2492897471 shutdown_connections 2024-03-22T10:41:59.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.575+0000 7f7782400700 1 -- 172.21.15.115:0/2492897471 wait complete. 2024-03-22T10:41:59.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.577+0000 7f7782400700 1 Processor -- start 2024-03-22T10:41:59.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.577+0000 7f7782400700 1 -- start start 2024-03-22T10:41:59.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.577+0000 7f7782400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 0x7f777c1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:59.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.577+0000 7f7782400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f777c103520 0x7f777c195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:59.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.577+0000 7f7782400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c196500 con 0x7f777c100c00 2024-03-22T10:41:59.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.577+0000 7f7782400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c196640 con 0x7f777c103520 2024-03-22T10:41:59.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.578+0000 7f777b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 0x7f777c1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.578+0000 7f777b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 0x7f777c1959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35392/0 (socket says 172.21.15.115:35392) 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.578+0000 7f777b400700 1 -- 172.21.15.115:0/420185221 learned_addr learned my addr 172.21.15.115:0/420185221 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.578+0000 7f777aa00700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f777c103520 0x7f777c195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.578+0000 7f777b400700 1 -- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f777c103520 msgr2=0x7f777c195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.578+0000 7f777b400700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f777c103520 0x7f777c195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.578+0000 7f777b400700 1 -- 172.21.15.115:0/420185221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c0095f0 con 0x7f777c100c00 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.579+0000 7f777b400700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 0x7f777c1959a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f776c005b40 tx=0x7f776c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.579+0000 7f7773e00700 1 -- 172.21.15.115:0/420185221 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f776c01d070 con 0x7f777c100c00 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.579+0000 7f7773e00700 1 -- 172.21.15.115:0/420185221 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f776c022470 con 0x7f777c100c00 2024-03-22T10:41:59.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.579+0000 7f7773e00700 1 -- 172.21.15.115:0/420185221 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f776c00f690 con 0x7f777c100c00 2024-03-22T10:41:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.579+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777c19b040 con 0x7f777c100c00 2024-03-22T10:41:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.579+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c0febc0 con 0x7f777c100c00 2024-03-22T10:41:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.581+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777c18fb90 con 0x7f777c100c00 2024-03-22T10:41:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.581+0000 7f7773e00700 1 -- 172.21.15.115:0/420185221 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 90908+0+0 (secure 0 0 0) 0x7f776c022a10 con 0x7f777c100c00 2024-03-22T10:41:59.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.582+0000 7f7773e00700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f77600716d0 0x7f7760073b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:41:59.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.583+0000 7f7773e00700 1 -- 172.21.15.115:0/420185221 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f776c08df10 con 0x7f777c100c00 2024-03-22T10:41:59.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.583+0000 7f777aa00700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f77600716d0 0x7f7760073b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:41:59.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.585+0000 7f777aa00700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f77600716d0 0x7f7760073b80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7768005950 tx=0x7f776800a300 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:41:59.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.589+0000 7f7773e00700 1 -- 172.21.15.115:0/420185221 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f776c05b4f0 con 0x7f777c100c00 2024-03-22T10:41:59.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.923+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 --> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f777c061190 con 0x7f77600716d0 2024-03-22T10:41:59.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.927+0000 7f7773e00700 1 -- 172.21.15.115:0/420185221 <== mgr.24365 v2:172.21.15.150:6828/3274858902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f777c061190 con 0x7f77600716d0 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f77600716d0 msgr2=0x7f7760073b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f77600716d0 0x7f7760073b80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7768005950 tx=0x7f776800a300 comp rx=0 tx=0).stop 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 msgr2=0x7f777c1959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 0x7f777c1959a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f776c005b40 tx=0x7f776c00bd70 comp rx=0 tx=0).stop 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 shutdown_connections 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f77600716d0 0x7f7760073b80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f777c100c00 0x7f777c1959a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 --2- 172.21.15.115:0/420185221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f777c103520 0x7f777c195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 >> 172.21.15.115:0/420185221 conn(0x7f777c0fa600 msgr2=0x7f777c0fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.930+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 shutdown_connections 2024-03-22T10:41:59.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:41:59.931+0000 7f7782400700 1 -- 172.21.15.115:0/420185221 wait complete. 2024-03-22T10:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:42:00.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:41:59 smithi115 ceph-mon[29111]: pgmap v19: 65 pgs: 65 active+clean; 142 MiB data, 659 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.5 MiB/s wr, 126 op/s 2024-03-22T10:42:00.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.128+0000 7f6625c00700 1 -- 172.21.15.115:0/2375645379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6620101350 msgr2=0x7f66201017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:00.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.128+0000 7f6625c00700 1 --2- 172.21.15.115:0/2375645379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6620101350 0x7f66201017a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f660c009910 tx=0x7f660c009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:00.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.128+0000 7f6625c00700 1 -- 172.21.15.115:0/2375645379 shutdown_connections 2024-03-22T10:42:00.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.128+0000 7f6625c00700 1 --2- 172.21.15.115:0/2375645379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6620101350 0x7f66201017a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.128+0000 7f6625c00700 1 --2- 172.21.15.115:0/2375645379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6620100150 0x7f6620100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.128+0000 7f6625c00700 1 -- 172.21.15.115:0/2375645379 >> 172.21.15.115:0/2375645379 conn(0x7f66200fb6e0 msgr2=0x7f66200fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:00.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.128+0000 7f6625c00700 1 -- 172.21.15.115:0/2375645379 shutdown_connections 2024-03-22T10:42:00.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.129+0000 7f6625c00700 1 -- 172.21.15.115:0/2375645379 wait complete. 2024-03-22T10:42:00.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.130+0000 7f6625c00700 1 Processor -- start 2024-03-22T10:42:00.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.130+0000 7f6625c00700 1 -- start start 2024-03-22T10:42:00.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.130+0000 7f6625c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6620100150 0x7f6620071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:00.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.130+0000 7f6625c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6620101350 0x7f6620072240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:00.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.131+0000 7f6625c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6620072860 con 0x7f6620101350 2024-03-22T10:42:00.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.131+0000 7f6625c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66201a14e0 con 0x7f6620100150 2024-03-22T10:42:00.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.131+0000 7f661ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6620100150 0x7f6620071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:00.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.131+0000 7f661ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6620100150 0x7f6620071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:40936/0 (socket says 172.21.15.115:40936) 2024-03-22T10:42:00.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.131+0000 7f661ea00700 1 -- 172.21.15.115:0/1570239124 learned_addr learned my addr 172.21.15.115:0/1570239124 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:00.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.131+0000 7f661e000700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6620101350 0x7f6620072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:00.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.132+0000 7f661ea00700 1 -- 172.21.15.115:0/1570239124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6620101350 msgr2=0x7f6620072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:00.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.132+0000 7f661ea00700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6620101350 0x7f6620072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.132+0000 7f661ea00700 1 -- 172.21.15.115:0/1570239124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661000f790 con 0x7f6620100150 2024-03-22T10:42:00.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.132+0000 7f661ea00700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6620100150 0x7f6620071d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6610009d30 tx=0x7f66100074a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:00.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.133+0000 7f6617400700 1 -- 172.21.15.115:0/1570239124 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6610007db0 con 0x7f6620100150 2024-03-22T10:42:00.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.133+0000 7f6617400700 1 -- 172.21.15.115:0/1570239124 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6610015040 con 0x7f6620100150 2024-03-22T10:42:00.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.133+0000 7f6617400700 1 -- 172.21.15.115:0/1570239124 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6610018610 con 0x7f6620100150 2024-03-22T10:42:00.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.134+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c0095f0 con 0x7f6620100150 2024-03-22T10:42:00.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.134+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66201a1a20 con 0x7f6620100150 2024-03-22T10:42:00.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.135+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662004ea50 con 0x7f6620100150 2024-03-22T10:42:00.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.138+0000 7f6617400700 1 -- 172.21.15.115:0/1570239124 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 24) v1 ==== 90908+0+0 (secure 0 0 0) 0x7f6610019440 con 0x7f6620100150 2024-03-22T10:42:00.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.138+0000 7f6617400700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f660406d280 0x7f660406f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:00.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.139+0000 7f6617400700 1 -- 172.21.15.115:0/1570239124 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f6610090490 con 0x7f6620100150 2024-03-22T10:42:00.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.139+0000 7f661e000700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f660406d280 0x7f660406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:00.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.140+0000 7f661e000700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f660406d280 0x7f660406f730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f660c009fd0 tx=0x7f660c005fd0 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:00.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.147+0000 7f6617400700 1 -- 172.21.15.115:0/1570239124 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f6610059dd0 con 0x7f6620100150 2024-03-22T10:42:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:41:59 smithi150 ceph-mon[34518]: pgmap v19: 65 pgs: 65 active+clean; 142 MiB data, 659 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.5 MiB/s wr, 126 op/s 2024-03-22T10:42:00.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.455+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 --> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6620105ca0 con 0x7f660406d280 2024-03-22T10:42:00.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.458+0000 7f6617400700 1 -- 172.21.15.115:0/1570239124 <== mgr.24365 v2:172.21.15.150:6828/3274858902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f6620105ca0 con 0x7f660406d280 2024-03-22T10:42:00.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.461+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f660406d280 msgr2=0x7f660406f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:00.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.461+0000 7f6625c00700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f660406d280 0x7f660406f730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f660c009fd0 tx=0x7f660c005fd0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.461+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6620100150 msgr2=0x7f6620071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:00.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.461+0000 7f6625c00700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6620100150 0x7f6620071d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6610009d30 tx=0x7f66100074a0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.461+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 shutdown_connections 2024-03-22T10:42:00.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.461+0000 7f6625c00700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f660406d280 0x7f660406f730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.461+0000 7f6625c00700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6620100150 0x7f6620071d00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.461+0000 7f6625c00700 1 --2- 172.21.15.115:0/1570239124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6620101350 0x7f6620072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.461+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 >> 172.21.15.115:0/1570239124 conn(0x7f66200fb6e0 msgr2=0x7f6620104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:00.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.462+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 shutdown_connections 2024-03-22T10:42:00.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.462+0000 7f6625c00700 1 -- 172.21.15.115:0/1570239124 wait complete. 2024-03-22T10:42:00.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.634+0000 7f362c400700 1 -- 172.21.15.115:0/3904523164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624100150 msgr2=0x7f3624100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:00.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.634+0000 7f362c400700 1 --2- 172.21.15.115:0/3904523164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624100150 0x7f3624100560 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3618009910 tx=0x7f3618009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:00.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.634+0000 7f362c400700 1 -- 172.21.15.115:0/3904523164 shutdown_connections 2024-03-22T10:42:00.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.634+0000 7f362c400700 1 --2- 172.21.15.115:0/3904523164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3624101350 0x7f36241017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.634+0000 7f362c400700 1 --2- 172.21.15.115:0/3904523164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624100150 0x7f3624100560 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.634+0000 7f362c400700 1 -- 172.21.15.115:0/3904523164 >> 172.21.15.115:0/3904523164 conn(0x7f36240fb6e0 msgr2=0x7f36240fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:00.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.635+0000 7f362c400700 1 -- 172.21.15.115:0/3904523164 shutdown_connections 2024-03-22T10:42:00.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.635+0000 7f362c400700 1 -- 172.21.15.115:0/3904523164 wait complete. 2024-03-22T10:42:00.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.636+0000 7f362c400700 1 Processor -- start 2024-03-22T10:42:00.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.636+0000 7f362c400700 1 -- start start 2024-03-22T10:42:00.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.636+0000 7f362c400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3624100150 0x7f3624074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:00.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.636+0000 7f362c400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624101350 0x7f3624073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:00.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.636+0000 7f362c400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3624073680 con 0x7f3624101350 2024-03-22T10:42:00.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.636+0000 7f362c400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36240737c0 con 0x7f3624100150 2024-03-22T10:42:00.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.637+0000 7f3629000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624101350 0x7f3624073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:00.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.637+0000 7f3629000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624101350 0x7f3624073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35434/0 (socket says 172.21.15.115:35434) 2024-03-22T10:42:00.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.637+0000 7f3629000700 1 -- 172.21.15.115:0/611875419 learned_addr learned my addr 172.21.15.115:0/611875419 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:00.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.637+0000 7f3629a00700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3624100150 0x7f3624074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:00.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.638+0000 7f3629000700 1 -- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3624100150 msgr2=0x7f3624074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:00.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.638+0000 7f3629000700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3624100150 0x7f3624074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:00.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.638+0000 7f3629000700 1 -- 172.21.15.115:0/611875419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180095f0 con 0x7f3624101350 2024-03-22T10:42:00.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.638+0000 7f3629000700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624101350 0x7f3624073140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f361000d8d0 tx=0x7f361000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:00.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.639+0000 7f3622000700 1 -- 172.21.15.115:0/611875419 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36100098a0 con 0x7f3624101350 2024-03-22T10:42:00.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.639+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3624073aa0 con 0x7f3624101350 2024-03-22T10:42:00.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.639+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362419d9f0 con 0x7f3624101350 2024-03-22T10:42:00.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.639+0000 7f3622000700 1 -- 172.21.15.115:0/611875419 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3610010460 con 0x7f3624101350 2024-03-22T10:42:00.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.639+0000 7f3622000700 1 -- 172.21.15.115:0/611875419 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361000f5f0 con 0x7f3624101350 2024-03-22T10:42:00.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.640+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3604005320 con 0x7f3624101350 2024-03-22T10:42:00.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.641+0000 7f3622000700 1 -- 172.21.15.115:0/611875419 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 90908+0+0 (secure 0 0 0) 0x7f3610009a00 con 0x7f3624101350 2024-03-22T10:42:00.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.642+0000 7f3622000700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f360c06d2d0 0x7f360c06f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:00.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.643+0000 7f3622000700 1 -- 172.21.15.115:0/611875419 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f361008c890 con 0x7f3624101350 2024-03-22T10:42:00.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.644+0000 7f3629a00700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f360c06d2d0 0x7f360c06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:00.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.645+0000 7f3629a00700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f360c06d2d0 0x7f360c06f780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3618009fd0 tx=0x7f3618005fb0 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:00.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.649+0000 7f3622000700 1 -- 172.21.15.115:0/611875419 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f3610059c80 con 0x7f3624101350 2024-03-22T10:42:00.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:00.966+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 --> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3604000bf0 con 0x7f360c06d2d0 2024-03-22T10:42:00.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.004+0000 7f3622000700 1 -- 172.21.15.115:0/611875419 <== mgr.24365 v2:172.21.15.150:6828/3274858902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3604000bf0 con 0x7f360c06d2d0 2024-03-22T10:42:00.997 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:42:00.997 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (8m) 17s ago 10m 14.1M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:42:00.997 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (10m) 17s ago 10m 7746k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:42:00.997 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (9m) 21s ago 9m 7608k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:42:00.997 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (10m) 17s ago 10m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:42:00.997 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (9m) 21s ago 9m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (8m) 17s ago 10m 73.6M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (3m) 17s ago 3m 79.8M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (3m) 17s ago 3m 13.3M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (3m) 21s ago 3m 13.3M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (3m) 21s ago 3m 73.8M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:9283,8765,8443 running (12m) 17s ago 12m 335M - 18.2.0 dc2bc1663786 932346df0e0f 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (49s) 21s ago 9m 486M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 02c6d1a3b66d 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (12m) 17s ago 12m 49.2M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (8m) 21s ago 8m 38.4M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (10m) 17s ago 10m 7801k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (9m) 21s ago 9m 8174k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:42:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (8m) 17s ago 8m 72.5M 1217M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:42:00.999 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (7m) 17s ago 7m 78.7M 1217M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:42:00.999 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (6m) 17s ago 6m 63.4M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:42:00.999 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (6m) 21s ago 6m 77.2M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:42:00.999 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (5m) 21s ago 5m 66.9M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:42:00.999 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (5m) 21s ago 5m 66.9M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:42:00.999 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 starting - - - - 2024-03-22T10:42:00.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f360c06d2d0 msgr2=0x7f360c06f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f360c06d2d0 0x7f360c06f780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3618009fd0 tx=0x7f3618005fb0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624101350 msgr2=0x7f3624073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624101350 0x7f3624073140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f361000d8d0 tx=0x7f361000dbe0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 shutdown_connections 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f360c06d2d0 0x7f360c06f780 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3624100150 0x7f3624074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 --2- 172.21.15.115:0/611875419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3624101350 0x7f3624073140 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 >> 172.21.15.115:0/611875419 conn(0x7f36240fb6e0 msgr2=0x7f3624104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 shutdown_connections 2024-03-22T10:42:01.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.007+0000 7f362c400700 1 -- 172.21.15.115:0/611875419 wait complete. 2024-03-22T10:42:01.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.179+0000 7f2d11200700 1 -- 172.21.15.115:0/829540843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 msgr2=0x7f2d0c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:01.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.179+0000 7f2d11200700 1 --2- 172.21.15.115:0/829540843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 0x7f2d0c100560 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009910 tx=0x7f2cfc009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:01.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.179+0000 7f2d11200700 1 -- 172.21.15.115:0/829540843 shutdown_connections 2024-03-22T10:42:01.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.179+0000 7f2d11200700 1 --2- 172.21.15.115:0/829540843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d0c101350 0x7f2d0c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.179+0000 7f2d11200700 1 --2- 172.21.15.115:0/829540843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 0x7f2d0c100560 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.179+0000 7f2d11200700 1 -- 172.21.15.115:0/829540843 >> 172.21.15.115:0/829540843 conn(0x7f2d0c0fb6e0 msgr2=0x7f2d0c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:01.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.181+0000 7f2d11200700 1 -- 172.21.15.115:0/829540843 shutdown_connections 2024-03-22T10:42:01.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.181+0000 7f2d11200700 1 -- 172.21.15.115:0/829540843 wait complete. 2024-03-22T10:42:01.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.182+0000 7f2d11200700 1 Processor -- start 2024-03-22T10:42:01.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.182+0000 7f2d11200700 1 -- start start 2024-03-22T10:42:01.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.182+0000 7f2d11200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 0x7f2d0c1a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:01.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.183+0000 7f2d11200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d0c101350 0x7f2d0c1a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:01.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.183+0000 7f2d11200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1a5830 con 0x7f2d0c100150 2024-03-22T10:42:01.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.183+0000 7f2d11200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1a9da0 con 0x7f2d0c101350 2024-03-22T10:42:01.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.183+0000 7f2d09e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 0x7f2d0c1a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:01.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.183+0000 7f2d09e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 0x7f2d0c1a4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35452/0 (socket says 172.21.15.115:35452) 2024-03-22T10:42:01.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.183+0000 7f2d09e00700 1 -- 172.21.15.115:0/795599673 learned_addr learned my addr 172.21.15.115:0/795599673 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:01.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.183+0000 7f2d09400700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d0c101350 0x7f2d0c1a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:01.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.184+0000 7f2d09e00700 1 -- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d0c101350 msgr2=0x7f2d0c1a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:01.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.184+0000 7f2d09e00700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d0c101350 0x7f2d0c1a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.184+0000 7f2d09e00700 1 -- 172.21.15.115:0/795599673 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0095f0 con 0x7f2d0c100150 2024-03-22T10:42:01.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.184+0000 7f2d09e00700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 0x7f2d0c1a4cd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00bda0 tx=0x7f2cfc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:01.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.185+0000 7f2d02a00700 1 -- 172.21.15.115:0/795599673 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc01d070 con 0x7f2d0c100150 2024-03-22T10:42:01.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.185+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c1a9f40 con 0x7f2d0c100150 2024-03-22T10:42:01.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.185+0000 7f2d02a00700 1 -- 172.21.15.115:0/795599673 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc004c50 con 0x7f2d0c100150 2024-03-22T10:42:01.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.185+0000 7f2d02a00700 1 -- 172.21.15.115:0/795599673 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc00f690 con 0x7f2d0c100150 2024-03-22T10:42:01.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.186+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c1aa3d0 con 0x7f2d0c100150 2024-03-22T10:42:01.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.187+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c106b80 con 0x7f2d0c100150 2024-03-22T10:42:01.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.188+0000 7f2d02a00700 1 -- 172.21.15.115:0/795599673 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 90908+0+0 (secure 0 0 0) 0x7f2cfc00f7f0 con 0x7f2d0c100150 2024-03-22T10:42:01.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.189+0000 7f2d02a00700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f2cf006d290 0x7f2cf006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:01.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.189+0000 7f2d02a00700 1 -- 172.21.15.115:0/795599673 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f2cfc08e1c0 con 0x7f2d0c100150 2024-03-22T10:42:01.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.189+0000 7f2d09400700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f2cf006d290 0x7f2cf006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:01.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.191+0000 7f2d09400700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f2cf006d290 0x7f2cf006f740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8006fd0 tx=0x7f2cf8008040 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:01.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.195+0000 7f2d02a00700 1 -- 172.21.15.115:0/795599673 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f2cfc057b20 con 0x7f2d0c100150 2024-03-22T10:42:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:01 smithi115 ceph-mon[29111]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:01 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:01 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:01 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:42:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:01 smithi115 ceph-mon[29111]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:42:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:01 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:01 smithi115 ceph-mon[29111]: Upgrade: Need to upgrade myself (mgr.smithi150.fmoumv) 2024-03-22T10:42:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:01 smithi115 ceph-mon[29111]: Upgrade: Need to upgrade myself (mgr.smithi150.fmoumv) 2024-03-22T10:42:01.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:01 smithi115 ceph-mon[29111]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:01.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:01 smithi150 ceph-mon[34518]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:01.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:01 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:01.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:01 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:01.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:01 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:42:01.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:01 smithi150 ceph-mon[34518]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:42:01.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:01 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:01.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:01 smithi150 ceph-mon[34518]: Upgrade: Need to upgrade myself (mgr.smithi150.fmoumv) 2024-03-22T10:42:01.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:01 smithi150 ceph-mon[34518]: Upgrade: Need to upgrade myself (mgr.smithi150.fmoumv) 2024-03-22T10:42:01.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:01 smithi150 ceph-mon[34518]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:01.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.581+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2d0c1aaa60 con 0x7f2d0c100150 2024-03-22T10:42:01.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.582+0000 7f2d02a00700 1 -- 172.21.15.115:0/795599673 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f2cfc05b1e0 con 0x7f2d0c100150 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:42:01.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:42:01.577 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:42:01.577 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-03-22T10:42:01.577 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-22T10:42:01.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:42:01.577 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:42:01.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.586+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f2cf006d290 msgr2=0x7f2cf006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:01.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.586+0000 7f2d11200700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f2cf006d290 0x7f2cf006f740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8006fd0 tx=0x7f2cf8008040 comp rx=0 tx=0).stop 2024-03-22T10:42:01.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.586+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 msgr2=0x7f2d0c1a4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:01.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.586+0000 7f2d11200700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 0x7f2d0c1a4cd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00bda0 tx=0x7f2cfc00be80 comp rx=0 tx=0).stop 2024-03-22T10:42:01.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.587+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 shutdown_connections 2024-03-22T10:42:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.587+0000 7f2d11200700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f2cf006d290 0x7f2cf006f740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.587+0000 7f2d11200700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d0c100150 0x7f2d0c1a4cd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.587+0000 7f2d11200700 1 --2- 172.21.15.115:0/795599673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d0c101350 0x7f2d0c1a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.587+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 >> 172.21.15.115:0/795599673 conn(0x7f2d0c0fb6e0 msgr2=0x7f2d0c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.587+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 shutdown_connections 2024-03-22T10:42:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.587+0000 7f2d11200700 1 -- 172.21.15.115:0/795599673 wait complete. 2024-03-22T10:42:01.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.781+0000 7fa762a00700 1 -- 172.21.15.115:0/1147516981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0ff380 msgr2=0x7fa75c0ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:01.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.781+0000 7fa762a00700 1 --2- 172.21.15.115:0/1147516981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0ff380 0x7fa75c0ff7f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa748009910 tx=0x7fa748009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:01.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.781+0000 7fa762a00700 1 -- 172.21.15.115:0/1147516981 shutdown_connections 2024-03-22T10:42:01.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.781+0000 7fa762a00700 1 --2- 172.21.15.115:0/1147516981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0ff380 0x7fa75c0ff7f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.781+0000 7fa762a00700 1 --2- 172.21.15.115:0/1147516981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa75c0fea30 0x7fa75c0fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.781+0000 7fa762a00700 1 -- 172.21.15.115:0/1147516981 >> 172.21.15.115:0/1147516981 conn(0x7fa75c0fa5e0 msgr2=0x7fa75c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:01.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.781+0000 7fa762a00700 1 -- 172.21.15.115:0/1147516981 shutdown_connections 2024-03-22T10:42:01.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.782+0000 7fa762a00700 1 -- 172.21.15.115:0/1147516981 wait complete. 2024-03-22T10:42:01.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.783+0000 7fa762a00700 1 Processor -- start 2024-03-22T10:42:01.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.783+0000 7fa762a00700 1 -- start start 2024-03-22T10:42:01.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.783+0000 7fa762a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0fea30 0x7fa75c071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:01.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.783+0000 7fa762a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa75c0ff380 0x7fa75c072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:01.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.783+0000 7fa762a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c072850 con 0x7fa75c0fea30 2024-03-22T10:42:01.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.783+0000 7fa762a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c072990 con 0x7fa75c0ff380 2024-03-22T10:42:01.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.784+0000 7fa75be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0fea30 0x7fa75c071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:01.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.784+0000 7fa75be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0fea30 0x7fa75c071cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35474/0 (socket says 172.21.15.115:35474) 2024-03-22T10:42:01.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.784+0000 7fa75be00700 1 -- 172.21.15.115:0/1758138409 learned_addr learned my addr 172.21.15.115:0/1758138409 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:01.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.784+0000 7fa75b400700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa75c0ff380 0x7fa75c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:01.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.785+0000 7fa75be00700 1 -- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa75c0ff380 msgr2=0x7fa75c072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:01.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.785+0000 7fa75be00700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa75c0ff380 0x7fa75c072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:01.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.785+0000 7fa75be00700 1 -- 172.21.15.115:0/1758138409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7480095f0 con 0x7fa75c0fea30 2024-03-22T10:42:01.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.785+0000 7fa75be00700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0fea30 0x7fa75c071cf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00b4b0 tx=0x7fa74c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:01.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.786+0000 7fa758c00700 1 -- 172.21.15.115:0/1758138409 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa74c011800 con 0x7fa75c0fea30 2024-03-22T10:42:01.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.786+0000 7fa758c00700 1 -- 172.21.15.115:0/1758138409 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa74c011e10 con 0x7fa75c0fea30 2024-03-22T10:42:01.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.786+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c1a5b10 con 0x7fa75c0fea30 2024-03-22T10:42:01.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.786+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c1a6060 con 0x7fa75c0fea30 2024-03-22T10:42:01.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.786+0000 7fa758c00700 1 -- 172.21.15.115:0/1758138409 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa74c00f2e0 con 0x7fa75c0fea30 2024-03-22T10:42:01.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.788+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c005320 con 0x7fa75c0fea30 2024-03-22T10:42:01.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.789+0000 7fa758c00700 1 -- 172.21.15.115:0/1758138409 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 90908+0+0 (secure 0 0 0) 0x7fa74c011960 con 0x7fa75c0fea30 2024-03-22T10:42:01.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.790+0000 7fa758c00700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fa74006d280 0x7fa74006f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:01.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.790+0000 7fa758c00700 1 -- 172.21.15.115:0/1758138409 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fa74c08c410 con 0x7fa75c0fea30 2024-03-22T10:42:01.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.792+0000 7fa75b400700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fa74006d280 0x7fa74006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:01.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.793+0000 7fa75b400700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fa74006d280 0x7fa74006f730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa748006010 tx=0x7fa74800b560 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:01.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:01.795+0000 7fa758c00700 1 -- 172.21.15.115:0/1758138409 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fa74c059910 con 0x7fa75c0fea30 2024-03-22T10:42:02.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.154+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa73c005cc0 con 0x7fa75c0fea30 2024-03-22T10:42:02.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.155+0000 7fa758c00700 1 -- 172.21.15.115:0/1758138409 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1856 (secure 0 0 0) 0x7fa74c015750 con 0x7fa75c0fea30 2024-03-22T10:42:02.148 INFO:teuthology.orchestra.run.smithi115.stdout:e13 2024-03-22T10:42:02.148 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:42:02.148 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:42:02.148 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:42:02.148 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:42:02.148 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:42:02.148 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:42:02.148 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 13 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:41:04.070466+0000 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:42:02.149 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:42:02.150 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:42:02.150 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:42:02.150 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:42:02.150 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:42:02.150 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:42:02.150 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:42:02.150 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:42:02.150 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:42:02.150 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:42:02.151 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:42:02.151 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:42:02.151 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:42:02.151 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:42:02.151 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:42:02.151 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:42:02.151 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:42:02.151 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:42:02.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.158+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fa74006d280 msgr2=0x7fa74006f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:02.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.158+0000 7fa762a00700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fa74006d280 0x7fa74006f730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa748006010 tx=0x7fa74800b560 comp rx=0 tx=0).stop 2024-03-22T10:42:02.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.158+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0fea30 msgr2=0x7fa75c071cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:02.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.158+0000 7fa762a00700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0fea30 0x7fa75c071cf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00b4b0 tx=0x7fa74c00b7c0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.159+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 shutdown_connections 2024-03-22T10:42:02.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.159+0000 7fa762a00700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7fa74006d280 0x7fa74006f730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.159+0000 7fa762a00700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c0fea30 0x7fa75c071cf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.159+0000 7fa762a00700 1 --2- 172.21.15.115:0/1758138409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa75c0ff380 0x7fa75c072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.159+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 >> 172.21.15.115:0/1758138409 conn(0x7fa75c0fa5e0 msgr2=0x7fa75c107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:02.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.159+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 shutdown_connections 2024-03-22T10:42:02.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.159+0000 7fa762a00700 1 -- 172.21.15.115:0/1758138409 wait complete. 2024-03-22T10:42:02.153 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 13 2024-03-22T10:42:02.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.351+0000 7f71dfe00700 1 -- 172.21.15.115:0/2320642181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81035b0 msgr2=0x7f71d8103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:02.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.351+0000 7f71dfe00700 1 --2- 172.21.15.115:0/2320642181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81035b0 0x7f71d8103a00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009910 tx=0x7f71c4009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:02.344 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:02 smithi115 ceph-mon[29111]: from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:02.344 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:02 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/795599673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:42:02.344 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:02 smithi115 ceph-mon[29111]: pgmap v20: 65 pgs: 65 active+clean; 142 MiB data, 659 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 93 op/s 2024-03-22T10:42:02.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.351+0000 7f71dfe00700 1 -- 172.21.15.115:0/2320642181 shutdown_connections 2024-03-22T10:42:02.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.351+0000 7f71dfe00700 1 --2- 172.21.15.115:0/2320642181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81035b0 0x7f71d8103a00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.351+0000 7f71dfe00700 1 --2- 172.21.15.115:0/2320642181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d81023b0 0x7f71d81027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.351+0000 7f71dfe00700 1 -- 172.21.15.115:0/2320642181 >> 172.21.15.115:0/2320642181 conn(0x7f71d80fd960 msgr2=0x7f71d80ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:02.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.351+0000 7f71dfe00700 1 -- 172.21.15.115:0/2320642181 shutdown_connections 2024-03-22T10:42:02.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.351+0000 7f71dfe00700 1 -- 172.21.15.115:0/2320642181 wait complete. 2024-03-22T10:42:02.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.352+0000 7f71dfe00700 1 Processor -- start 2024-03-22T10:42:02.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.353+0000 7f71dfe00700 1 -- start start 2024-03-22T10:42:02.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.353+0000 7f71dfe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81023b0 0x7f71d8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.353+0000 7f71dfe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d81035b0 0x7f71d81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.353+0000 7f71dfe00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d81987d0 con 0x7f71d81023b0 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.353+0000 7f71dfe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d819d1e0 con 0x7f71d81035b0 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.353+0000 7f71dd400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81023b0 0x7f71d8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.354+0000 7f71dd400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81023b0 0x7f71d8197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48032/0 (socket says 172.21.15.115:48032) 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.354+0000 7f71dd400700 1 -- 172.21.15.115:0/2878181468 learned_addr learned my addr 172.21.15.115:0/2878181468 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.354+0000 7f71dca00700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d81035b0 0x7f71d81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.354+0000 7f71dd400700 1 -- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d81035b0 msgr2=0x7f71d81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.354+0000 7f71dd400700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d81035b0 0x7f71d81981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.354+0000 7f71dd400700 1 -- 172.21.15.115:0/2878181468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40095f0 con 0x7f71d81023b0 2024-03-22T10:42:02.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.354+0000 7f71dd400700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81023b0 0x7f71d8197c70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00b480 tx=0x7f71cc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:02.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.355+0000 7f71d6000700 1 -- 172.21.15.115:0/2878181468 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71cc0118c0 con 0x7f71d81023b0 2024-03-22T10:42:02.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.355+0000 7f71d6000700 1 -- 172.21.15.115:0/2878181468 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f71cc010380 con 0x7f71d81023b0 2024-03-22T10:42:02.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.355+0000 7f71d6000700 1 -- 172.21.15.115:0/2878181468 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71cc00f4f0 con 0x7f71d81023b0 2024-03-22T10:42:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.356+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d819d3e0 con 0x7f71d81023b0 2024-03-22T10:42:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.356+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d819d980 con 0x7f71d81023b0 2024-03-22T10:42:02.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.357+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d810afa0 con 0x7f71d81023b0 2024-03-22T10:42:02.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.358+0000 7f71d6000700 1 -- 172.21.15.115:0/2878181468 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 90908+0+0 (secure 0 0 0) 0x7f71cc0104f0 con 0x7f71d81023b0 2024-03-22T10:42:02.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.360+0000 7f71d6000700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f71c006d280 0x7f71c006f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:02.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.360+0000 7f71d6000700 1 -- 172.21.15.115:0/2878181468 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f71cc08bfc0 con 0x7f71d81023b0 2024-03-22T10:42:02.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.361+0000 7f71dca00700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f71c006d280 0x7f71c006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:02.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.362+0000 7f71dca00700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f71c006d280 0x7f71c006f730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f71c4000c00 tx=0x7f71c4005fb0 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:02.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.365+0000 7f71d6000700 1 -- 172.21.15.115:0/2878181468 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f71cc055990 con 0x7f71d81023b0 2024-03-22T10:42:02.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:02 smithi150 ceph-mon[34518]: from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:02.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:02 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/795599673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:42:02.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:02 smithi150 ceph-mon[34518]: pgmap v20: 65 pgs: 65 active+clean; 142 MiB data, 659 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 93 op/s 2024-03-22T10:42:02.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.679+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 --> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71d8107f00 con 0x7f71c006d280 2024-03-22T10:42:02.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.682+0000 7f71d6000700 1 -- 172.21.15.115:0/2878181468 <== mgr.24365 v2:172.21.15.150:6828/3274858902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f71d8107f00 con 0x7f71c006d280 2024-03-22T10:42:02.674 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:42:02.674 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:42:02.674 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:42:02.674 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:42:02.675 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [], 2024-03-22T10:42:02.675 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "1/23 daemons upgraded", 2024-03-22T10:42:02.675 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading mgr daemons", 2024-03-22T10:42:02.675 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:42:02.675 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:42:02.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.685+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f71c006d280 msgr2=0x7f71c006f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:02.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.685+0000 7f71dfe00700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f71c006d280 0x7f71c006f730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f71c4000c00 tx=0x7f71c4005fb0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.685+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81023b0 msgr2=0x7f71d8197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:02.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.685+0000 7f71dfe00700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81023b0 0x7f71d8197c70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00b480 tx=0x7f71cc00b840 comp rx=0 tx=0).stop 2024-03-22T10:42:02.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.685+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 shutdown_connections 2024-03-22T10:42:02.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.685+0000 7f71dfe00700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f71c006d280 0x7f71c006f730 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f71c4000c00 tx=0x7f71c4005fb0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.685+0000 7f71dfe00700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71d81023b0 0x7f71d8197c70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.685+0000 7f71dfe00700 1 --2- 172.21.15.115:0/2878181468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d81035b0 0x7f71d81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.685+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 >> 172.21.15.115:0/2878181468 conn(0x7f71d80fd960 msgr2=0x7f71d81067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:02.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.686+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 shutdown_connections 2024-03-22T10:42:02.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.686+0000 7f71dfe00700 1 -- 172.21.15.115:0/2878181468 wait complete. 2024-03-22T10:42:02.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.863+0000 7f19ef000700 1 -- 172.21.15.115:0/2158162674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 msgr2=0x7f19e80fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:02.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.863+0000 7f19ef000700 1 --2- 172.21.15.115:0/2158162674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 0x7f19e80fee60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009910 tx=0x7f19d8009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:02.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.863+0000 7f19ef000700 1 -- 172.21.15.115:0/2158162674 shutdown_connections 2024-03-22T10:42:02.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.863+0000 7f19ef000700 1 --2- 172.21.15.115:0/2158162674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e80ff3a0 0x7f19e80ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.863+0000 7f19ef000700 1 --2- 172.21.15.115:0/2158162674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 0x7f19e80fee60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.863+0000 7f19ef000700 1 -- 172.21.15.115:0/2158162674 >> 172.21.15.115:0/2158162674 conn(0x7f19e80fa5e0 msgr2=0x7f19e80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:02.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.863+0000 7f19ef000700 1 -- 172.21.15.115:0/2158162674 shutdown_connections 2024-03-22T10:42:02.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.863+0000 7f19ef000700 1 -- 172.21.15.115:0/2158162674 wait complete. 2024-03-22T10:42:02.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.865+0000 7f19ef000700 1 Processor -- start 2024-03-22T10:42:02.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.865+0000 7f19ef000700 1 -- start start 2024-03-22T10:42:02.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.865+0000 7f19ef000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 0x7f19e8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:02.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.865+0000 7f19ef000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e80ff3a0 0x7f19e8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:02.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.865+0000 7f19ef000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81986a0 con 0x7f19e80fea50 2024-03-22T10:42:02.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.865+0000 7f19ef000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81987e0 con 0x7f19e80ff3a0 2024-03-22T10:42:02.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.866+0000 7f19e7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 0x7f19e8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:02.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.866+0000 7f19e7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 0x7f19e8197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48044/0 (socket says 172.21.15.115:48044) 2024-03-22T10:42:02.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.866+0000 7f19e7e00700 1 -- 172.21.15.115:0/3256522681 learned_addr learned my addr 172.21.15.115:0/3256522681 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:02.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.866+0000 7f19e7400700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e80ff3a0 0x7f19e8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:02.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.866+0000 7f19e7e00700 1 -- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e80ff3a0 msgr2=0x7f19e8198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:02.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.866+0000 7f19e7e00700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e80ff3a0 0x7f19e8198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:02.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.866+0000 7f19e7e00700 1 -- 172.21.15.115:0/3256522681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80095f0 con 0x7f19e80fea50 2024-03-22T10:42:02.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.867+0000 7f19e7e00700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 0x7f19e8197b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f19d80092e0 tx=0x7f19d800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:02.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.867+0000 7f19e4c00700 1 -- 172.21.15.115:0/3256522681 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d801d070 con 0x7f19e80fea50 2024-03-22T10:42:02.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.867+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e819d230 con 0x7f19e80fea50 2024-03-22T10:42:02.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.867+0000 7f19e4c00700 1 -- 172.21.15.115:0/3256522681 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f19d8005dc0 con 0x7f19e80fea50 2024-03-22T10:42:02.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.867+0000 7f19e4c00700 1 -- 172.21.15.115:0/3256522681 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d8017630 con 0x7f19e80fea50 2024-03-22T10:42:02.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.867+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e819d720 con 0x7f19e80fea50 2024-03-22T10:42:02.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.869+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e810ae80 con 0x7f19e80fea50 2024-03-22T10:42:02.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.871+0000 7f19e4c00700 1 -- 172.21.15.115:0/3256522681 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 90908+0+0 (secure 0 0 0) 0x7f19d8003680 con 0x7f19e80fea50 2024-03-22T10:42:02.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.872+0000 7f19e4c00700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f19cc06d280 0x7f19cc06f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:02.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.872+0000 7f19e4c00700 1 -- 172.21.15.115:0/3256522681 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f19d808dee0 con 0x7f19e80fea50 2024-03-22T10:42:02.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.875+0000 7f19e7400700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f19cc06d280 0x7f19cc06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:02.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.876+0000 7f19e7400700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f19cc06d280 0x7f19cc06f730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009c60 tx=0x7f19d4009260 comp rx=0 tx=0).ready entity=mgr.24365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:02.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:02.877+0000 7f19e4c00700 1 -- 172.21.15.115:0/3256522681 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f19d805b2d0 con 0x7f19e80fea50 2024-03-22T10:42:03.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.314+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f19e810b070 con 0x7f19e80fea50 2024-03-22T10:42:03.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.315+0000 7f19e4c00700 1 -- 172.21.15.115:0/3256522681 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f19d805ae60 con 0x7f19e80fea50 2024-03-22T10:42:03.308 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:42:03.308 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:42:03.308 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:42:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.318+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f19cc06d280 msgr2=0x7f19cc06f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.318+0000 7f19ef000700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f19cc06d280 0x7f19cc06f730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009c60 tx=0x7f19d4009260 comp rx=0 tx=0).stop 2024-03-22T10:42:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.318+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 msgr2=0x7f19e8197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.318+0000 7f19ef000700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 0x7f19e8197b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f19d80092e0 tx=0x7f19d800f690 comp rx=0 tx=0).stop 2024-03-22T10:42:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.318+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 shutdown_connections 2024-03-22T10:42:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.318+0000 7f19ef000700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:6828/3274858902,v1:172.21.15.150:6829/3274858902] conn(0x7f19cc06d280 0x7f19cc06f730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.318+0000 7f19ef000700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19e80fea50 0x7f19e8197b40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.318+0000 7f19ef000700 1 --2- 172.21.15.115:0/3256522681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e80ff3a0 0x7f19e8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.318+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 >> 172.21.15.115:0/3256522681 conn(0x7f19e80fa5e0 msgr2=0x7f19e8107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:03.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.319+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 shutdown_connections 2024-03-22T10:42:03.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:03.319+0000 7f19ef000700 1 -- 172.21.15.115:0/3256522681 wait complete. 2024-03-22T10:42:03.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:03 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1758138409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:42:03.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:03 smithi150 ceph-mon[34518]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:03.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:03 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:03.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:03 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.iqzwsz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:42:03.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:03 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.iqzwsz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:42:03.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:03 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:42:03.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:03 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:42:03.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:03 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1758138409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:42:03.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:03 smithi115 ceph-mon[29111]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:03.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:03 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:03.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:03 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.iqzwsz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:42:03.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:03 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.iqzwsz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:42:03.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:03 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:42:03.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:03 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:42:04.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:04 smithi150 ceph-mon[34518]: Upgrade: Updating mgr.smithi115.iqzwsz 2024-03-22T10:42:04.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:04 smithi150 ceph-mon[34518]: Deploying daemon mgr.smithi115.iqzwsz on smithi115 2024-03-22T10:42:04.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:04 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3256522681' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:42:04.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:04 smithi150 ceph-mon[34518]: pgmap v21: 65 pgs: 65 active+clean; 154 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 176 op/s 2024-03-22T10:42:04.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:04 smithi115 ceph-mon[29111]: Upgrade: Updating mgr.smithi115.iqzwsz 2024-03-22T10:42:04.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:04 smithi115 ceph-mon[29111]: Deploying daemon mgr.smithi115.iqzwsz on smithi115 2024-03-22T10:42:04.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:04 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/3256522681' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:42:04.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:04 smithi115 ceph-mon[29111]: pgmap v21: 65 pgs: 65 active+clean; 154 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 176 op/s 2024-03-22T10:42:07.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:06 smithi150 ceph-mon[34518]: pgmap v22: 65 pgs: 65 active+clean; 154 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 128 op/s 2024-03-22T10:42:07.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:06 smithi115 ceph-mon[29111]: pgmap v22: 65 pgs: 65 active+clean; 154 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 128 op/s 2024-03-22T10:42:09.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:08 smithi115 ceph-mon[29111]: pgmap v23: 65 pgs: 65 active+clean; 154 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 128 op/s 2024-03-22T10:42:09.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:08 smithi150 ceph-mon[34518]: pgmap v23: 65 pgs: 65 active+clean; 154 MiB data, 706 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 128 op/s 2024-03-22T10:42:11.140 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:10 smithi115 ceph-mon[29111]: pgmap v24: 65 pgs: 65 active+clean; 166 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 189 op/s 2024-03-22T10:42:11.140 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:10 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:11.141 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:10 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:11.141 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:10 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:11.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:10 smithi150 ceph-mon[34518]: pgmap v24: 65 pgs: 65 active+clean; 166 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 189 op/s 2024-03-22T10:42:11.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:10 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:11.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:10 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:11.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:10 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:12.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:12 smithi115 ceph-mon[29111]: pgmap v25: 65 pgs: 65 active+clean; 166 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 143 op/s 2024-03-22T10:42:13.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:12 smithi150 ceph-mon[34518]: pgmap v25: 65 pgs: 65 active+clean; 166 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 143 op/s 2024-03-22T10:42:13.867 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:13 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:13.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:13 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:14 smithi115 ceph-mon[29111]: pgmap v26: 65 pgs: 65 active+clean; 175 MiB data, 761 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 245 op/s 2024-03-22T10:42:15.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:14 smithi150 ceph-mon[34518]: pgmap v26: 65 pgs: 65 active+clean; 175 MiB data, 761 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 245 op/s 2024-03-22T10:42:17.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:16 smithi115 ceph-mon[29111]: pgmap v27: 65 pgs: 65 active+clean; 175 MiB data, 761 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 162 op/s 2024-03-22T10:42:17.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:16 smithi150 ceph-mon[34518]: pgmap v27: 65 pgs: 65 active+clean; 175 MiB data, 761 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 162 op/s 2024-03-22T10:42:19.103 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:18 smithi115 ceph-mon[29111]: pgmap v28: 65 pgs: 65 active+clean; 175 MiB data, 761 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 162 op/s 2024-03-22T10:42:19.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:18 smithi150 ceph-mon[34518]: pgmap v28: 65 pgs: 65 active+clean; 175 MiB data, 761 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 162 op/s 2024-03-22T10:42:21.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:20 smithi150 ceph-mon[34518]: pgmap v29: 65 pgs: 65 active+clean; 186 MiB data, 784 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.7 MiB/s wr, 255 op/s 2024-03-22T10:42:21.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:20 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:21.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:20 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:21.150 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:20 smithi115 ceph-mon[29111]: pgmap v29: 65 pgs: 65 active+clean; 186 MiB data, 784 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.7 MiB/s wr, 255 op/s 2024-03-22T10:42:21.150 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:20 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:21.150 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:20 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:22.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:21 smithi150 ceph-mon[34518]: Standby manager daemon smithi115.iqzwsz restarted 2024-03-22T10:42:22.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:21 smithi150 ceph-mon[34518]: Standby manager daemon smithi115.iqzwsz started 2024-03-22T10:42:22.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:21 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.iqzwsz/crt"}]: dispatch 2024-03-22T10:42:22.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:21 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:42:22.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:21 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.iqzwsz/key"}]: dispatch 2024-03-22T10:42:22.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:21 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:42:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:21 smithi115 ceph-mon[29111]: Standby manager daemon smithi115.iqzwsz restarted 2024-03-22T10:42:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:21 smithi115 ceph-mon[29111]: Standby manager daemon smithi115.iqzwsz started 2024-03-22T10:42:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:21 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.iqzwsz/crt"}]: dispatch 2024-03-22T10:42:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:21 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:42:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:21 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.iqzwsz/key"}]: dispatch 2024-03-22T10:42:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:21 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:42:23.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:22 smithi150 ceph-mon[34518]: pgmap v30: 65 pgs: 65 active+clean; 186 MiB data, 784 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-22T10:42:23.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:22 smithi150 ceph-mon[34518]: mgrmap e25: smithi150.fmoumv(active, since 54s), standbys: smithi115.iqzwsz 2024-03-22T10:42:23.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:22 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:23.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:22 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:23.189 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:22 smithi115 ceph-mon[29111]: pgmap v30: 65 pgs: 65 active+clean; 186 MiB data, 784 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-22T10:42:23.190 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:22 smithi115 ceph-mon[29111]: mgrmap e25: smithi150.fmoumv(active, since 54s), standbys: smithi115.iqzwsz 2024-03-22T10:42:23.190 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:22 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:23.190 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:22 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:24.292 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:24 smithi115 ceph-mon[29111]: pgmap v31: 65 pgs: 65 active+clean; 199 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 269 op/s 2024-03-22T10:42:24.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:24 smithi150 ceph-mon[34518]: pgmap v31: 65 pgs: 65 active+clean; 199 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 269 op/s 2024-03-22T10:42:27.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:26 smithi150 ceph-mon[34518]: pgmap v32: 65 pgs: 65 active+clean; 199 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 167 op/s 2024-03-22T10:42:27.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:26 smithi115 ceph-mon[29111]: pgmap v32: 65 pgs: 65 active+clean; 199 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 167 op/s 2024-03-22T10:42:27.355 INFO:tasks.workunit.client.0.smithi115.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-22T10:42:27.379 INFO:tasks.workunit.client.0.smithi115.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T10:42:27.379 INFO:tasks.workunit.client.0.smithi115.stderr:+ make 2024-03-22T10:42:27.709 INFO:tasks.workunit.client.0.smithi115.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-22T10:42:29.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: pgmap v33: 65 pgs: 65 active+clean; 199 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 167 op/s 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr fail", "who": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr fail", "who": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd='[{"prefix": "mgr fail", "who": "smithi150.fmoumv"}]': finished 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: mgrmap e26: smithi115.iqzwsz(active, starting, since 0.0112649s) 2024-03-22T10:42:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: Manager daemon smithi115.iqzwsz is now available 2024-03-22T10:42:29.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:29.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:29.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:29.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:42:29.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:28 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/trash_purge_schedule"}]: dispatch 2024-03-22T10:42:29.014 INFO:tasks.workunit.client.0.smithi115.stderr:++ readlink -f fsstress 2024-03-22T10:42:29.018 INFO:tasks.workunit.client.0.smithi115.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-22T10:42:29.018 INFO:tasks.workunit.client.0.smithi115.stderr:+ popd 2024-03-22T10:42:29.020 INFO:tasks.workunit.client.0.smithi115.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T10:42:29.020 INFO:tasks.workunit.client.0.smithi115.stderr:+ popd 2024-03-22T10:42:29.022 INFO:tasks.workunit.client.0.smithi115.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-22T10:42:29.023 INFO:tasks.workunit.client.0.smithi115.stderr:++ mktemp -d -p . 2024-03-22T10:42:29.027 INFO:tasks.workunit.client.0.smithi115.stderr:+ T=./tmp.JwwKOLKHxY 2024-03-22T10:42:29.027 INFO:tasks.workunit.client.0.smithi115.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.JwwKOLKHxY -l 1 -n 1000 -p 10 -v 2024-03-22T10:42:29.036 INFO:tasks.workunit.client.0.smithi115.stdout:seed = 1711133824 2024-03-22T10:42:29.047 INFO:tasks.workunit.client.0.smithi115.stdout:0/0: creat f0 x:0 0 0 2024-03-22T10:42:29.047 INFO:tasks.workunit.client.0.smithi115.stdout:2/0: write - no filename 2024-03-22T10:42:29.049 INFO:tasks.workunit.client.0.smithi115.stdout:1/0: write - no filename 2024-03-22T10:42:29.052 INFO:tasks.workunit.client.0.smithi115.stdout:0/1: rename f0 to f1 0 2024-03-22T10:42:29.052 INFO:tasks.workunit.client.0.smithi115.stdout:3/0: link - no file 2024-03-22T10:42:29.052 INFO:tasks.workunit.client.0.smithi115.stdout:3/1: dwrite - no filename 2024-03-22T10:42:29.053 INFO:tasks.workunit.client.0.smithi115.stdout:3/2: chown . 5929 1 2024-03-22T10:42:29.053 INFO:tasks.workunit.client.0.smithi115.stdout:3/3: dwrite - no filename 2024-03-22T10:42:29.054 INFO:tasks.workunit.client.0.smithi115.stdout:2/1: creat f0 x:0 0 0 2024-03-22T10:42:29.056 INFO:tasks.workunit.client.0.smithi115.stdout:0/2: mknod c2 0 2024-03-22T10:42:29.056 INFO:tasks.workunit.client.0.smithi115.stdout:4/0: truncate - no filename 2024-03-22T10:42:29.056 INFO:tasks.workunit.client.0.smithi115.stdout:4/1: truncate - no filename 2024-03-22T10:42:29.056 INFO:tasks.workunit.client.0.smithi115.stdout:4/2: write - no filename 2024-03-22T10:42:29.056 INFO:tasks.workunit.client.0.smithi115.stdout:4/3: read - no filename 2024-03-22T10:42:29.057 INFO:tasks.workunit.client.0.smithi115.stdout:0/3: read - f1 zero size 2024-03-22T10:42:29.060 INFO:tasks.workunit.client.0.smithi115.stdout:1/1: mkdir d0 0 2024-03-22T10:42:29.060 INFO:tasks.workunit.client.0.smithi115.stdout:1/2: unlink - no file 2024-03-22T10:42:29.060 INFO:tasks.workunit.client.0.smithi115.stdout:2/2: mknod c1 0 2024-03-22T10:42:29.063 INFO:tasks.workunit.client.0.smithi115.stdout:2/3: write f0 [546187,30028] 0 2024-03-22T10:42:29.064 INFO:tasks.workunit.client.0.smithi115.stdout:5/0: truncate - no filename 2024-03-22T10:42:29.065 INFO:tasks.workunit.client.0.smithi115.stdout:3/4: mkdir d0 0 2024-03-22T10:42:29.065 INFO:tasks.workunit.client.0.smithi115.stdout:3/5: truncate - no filename 2024-03-22T10:42:29.068 INFO:tasks.workunit.client.0.smithi115.stdout:0/4: write f1 [857691,125699] 0 2024-03-22T10:42:29.070 INFO:tasks.workunit.client.0.smithi115.stdout:0/5: chown f1 2122 1 2024-03-22T10:42:29.071 INFO:tasks.workunit.client.0.smithi115.stdout:2/4: dread f0 [0,4194304] 0 2024-03-22T10:42:29.074 INFO:tasks.workunit.client.0.smithi115.stdout:2/5: write f0 [293956,49248] 0 2024-03-22T10:42:29.074 INFO:tasks.workunit.client.0.smithi115.stdout:4/4: symlink l0 0 2024-03-22T10:42:29.075 INFO:tasks.workunit.client.0.smithi115.stdout:6/0: chown . 21 1 2024-03-22T10:42:29.077 INFO:tasks.workunit.client.0.smithi115.stdout:1/3: getdents d0 0 2024-03-22T10:42:29.077 INFO:tasks.workunit.client.0.smithi115.stdout:1/4: dread - no filename 2024-03-22T10:42:29.077 INFO:tasks.workunit.client.0.smithi115.stdout:1/5: truncate - no filename 2024-03-22T10:42:29.077 INFO:tasks.workunit.client.0.smithi115.stdout:1/6: dread - no filename 2024-03-22T10:42:29.077 INFO:tasks.workunit.client.0.smithi115.stdout:1/7: write - no filename 2024-03-22T10:42:29.077 INFO:tasks.workunit.client.0.smithi115.stdout:1/8: unlink - no file 2024-03-22T10:42:29.077 INFO:tasks.workunit.client.0.smithi115.stdout:1/9: write - no filename 2024-03-22T10:42:29.078 INFO:tasks.workunit.client.0.smithi115.stdout:0/6: mkdir d3 0 2024-03-22T10:42:29.080 INFO:tasks.workunit.client.0.smithi115.stdout:0/7: chown c2 254633308 1 2024-03-22T10:42:29.092 INFO:tasks.workunit.client.0.smithi115.stdout:5/1: symlink l0 0 2024-03-22T10:42:29.092 INFO:tasks.workunit.client.0.smithi115.stdout:4/5: mkdir d1 0 2024-03-22T10:42:29.092 INFO:tasks.workunit.client.0.smithi115.stdout:4/6: dwrite - no filename 2024-03-22T10:42:29.093 INFO:tasks.workunit.client.0.smithi115.stdout:7/0: readlink - no filename 2024-03-22T10:42:29.094 INFO:tasks.workunit.client.0.smithi115.stdout:2/6: creat f2 x:0 0 0 2024-03-22T10:42:29.095 INFO:tasks.workunit.client.0.smithi115.stdout:3/6: rmdir d0 0 2024-03-22T10:42:29.097 INFO:tasks.workunit.client.0.smithi115.stdout:5/2: creat f1 x:0 0 0 2024-03-22T10:42:29.112 INFO:tasks.workunit.client.0.smithi115.stdout:0/8: mkdir d3/d4 0 2024-03-22T10:42:29.112 INFO:tasks.workunit.client.0.smithi115.stdout:0/9: readlink - no filename 2024-03-22T10:42:29.113 INFO:tasks.workunit.client.0.smithi115.stdout:1/10: creat d0/f1 x:0 0 0 2024-03-22T10:42:29.117 INFO:tasks.workunit.client.0.smithi115.stdout:2/7: unlink f2 0 2024-03-22T10:42:29.119 INFO:tasks.workunit.client.0.smithi115.stdout:3/7: creat f1 x:0 0 0 2024-03-22T10:42:29.120 INFO:tasks.workunit.client.0.smithi115.stdout:3/8: stat f1 0 2024-03-22T10:42:29.121 INFO:tasks.workunit.client.0.smithi115.stdout:3/9: rmdir - no directory 2024-03-22T10:42:29.141 INFO:tasks.workunit.client.0.smithi115.stdout:0/10: dwrite f1 [0,4194304] 0 2024-03-22T10:42:29.142 INFO:tasks.workunit.client.0.smithi115.stdout:0/11: write f1 [2143187,50252] 0 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: pgmap v33: 65 pgs: 65 active+clean; 199 MiB data, 810 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 167 op/s 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 172.21.15.150:0/307909864' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr fail", "who": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "mgr fail", "who": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T10:42:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.24365 ' entity='mgr.smithi150.fmoumv' cmd='[{"prefix": "mgr fail", "who": "smithi150.fmoumv"}]': finished 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: mgrmap e26: smithi115.iqzwsz(active, starting, since 0.0112649s) 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:42:29.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: Manager daemon smithi115.iqzwsz is now available 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:42:29.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:28 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/trash_purge_schedule"}]: dispatch 2024-03-22T10:42:29.163 INFO:tasks.workunit.client.0.smithi115.stdout:0/12: dwrite f1 [0,4194304] 0 2024-03-22T10:42:29.167 INFO:tasks.workunit.client.0.smithi115.stdout:0/13: read f1 [3434141,4243] 0 2024-03-22T10:42:29.198 INFO:tasks.workunit.client.0.smithi115.stdout:0/14: fdatasync f1 0 2024-03-22T10:42:29.207 INFO:tasks.workunit.client.0.smithi115.stdout:4/7: creat d1/f2 x:0 0 0 2024-03-22T10:42:29.212 INFO:tasks.workunit.client.0.smithi115.stdout:7/1: creat f0 x:0 0 0 2024-03-22T10:42:29.214 INFO:tasks.workunit.client.0.smithi115.stdout:2/8: fdatasync f0 0 2024-03-22T10:42:29.214 INFO:tasks.workunit.client.0.smithi115.stdout:9/0: dwrite - no filename 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/1: read - no filename 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/2: dread - no filename 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/3: dread - no filename 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/4: truncate - no filename 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/5: dread - no filename 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/6: fdatasync - no filename 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/7: fdatasync - no filename 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:3/10: creat f2 x:0 0 0 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/8: chown . 1293 1 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/9: dwrite - no filename 2024-03-22T10:42:29.215 INFO:tasks.workunit.client.0.smithi115.stdout:3/11: rmdir - no directory 2024-03-22T10:42:29.217 INFO:tasks.workunit.client.0.smithi115.stdout:0/15: mkdir d3/d5 0 2024-03-22T10:42:29.219 INFO:tasks.workunit.client.0.smithi115.stdout:8/0: creat f0 x:0 0 0 2024-03-22T10:42:29.219 INFO:tasks.workunit.client.0.smithi115.stdout:8/1: rmdir - no directory 2024-03-22T10:42:29.220 INFO:tasks.workunit.client.0.smithi115.stdout:2/9: write f0 [1621758,59307] 0 2024-03-22T10:42:29.223 INFO:tasks.workunit.client.0.smithi115.stdout:2/10: truncate f0 2627152 0 2024-03-22T10:42:29.226 INFO:tasks.workunit.client.0.smithi115.stdout:7/2: rename f0 to f1 0 2024-03-22T10:42:29.229 INFO:tasks.workunit.client.0.smithi115.stdout:3/12: mkdir d3 0 2024-03-22T10:42:29.233 INFO:tasks.workunit.client.0.smithi115.stdout:4/8: dwrite d1/f2 [0,4194304] 0 2024-03-22T10:42:29.233 INFO:tasks.workunit.client.0.smithi115.stdout:3/13: write f2 [1017239,1864] 0 2024-03-22T10:42:29.236 INFO:tasks.workunit.client.0.smithi115.stdout:3/14: rename d3 to d3/d4 22 2024-03-22T10:42:29.238 INFO:tasks.workunit.client.0.smithi115.stdout:0/16: write f1 [747296,15464] 0 2024-03-22T10:42:29.239 INFO:tasks.workunit.client.0.smithi115.stdout:3/15: stat f2 0 2024-03-22T10:42:29.249 INFO:tasks.workunit.client.0.smithi115.stdout:9/10: creat f0 x:0 0 0 2024-03-22T10:42:29.263 INFO:tasks.workunit.client.0.smithi115.stdout:0/17: rename c2 to d3/d5/c6 0 2024-03-22T10:42:29.264 INFO:tasks.workunit.client.0.smithi115.stdout:3/16: symlink d3/l5 0 2024-03-22T10:42:29.268 INFO:tasks.workunit.client.0.smithi115.stdout:3/17: write f1 [228723,1722] 0 2024-03-22T10:42:29.276 INFO:tasks.workunit.client.0.smithi115.stdout:3/18: mknod d3/c6 0 2024-03-22T10:42:29.276 INFO:tasks.workunit.client.0.smithi115.stdout:7/3: dwrite f1 [0,4194304] 0 2024-03-22T10:42:29.281 INFO:tasks.workunit.client.0.smithi115.stdout:5/3: fsync f1 0 2024-03-22T10:42:29.282 INFO:tasks.workunit.client.0.smithi115.stdout:1/11: getdents d0 0 2024-03-22T10:42:29.282 INFO:tasks.workunit.client.0.smithi115.stdout:7/4: chown f1 770 1 2024-03-22T10:42:29.286 INFO:tasks.workunit.client.0.smithi115.stdout:3/19: rename f2 to d3/f7 0 2024-03-22T10:42:29.288 INFO:tasks.workunit.client.0.smithi115.stdout:5/4: mkdir d2 0 2024-03-22T10:42:29.290 INFO:tasks.workunit.client.0.smithi115.stdout:7/5: mkdir d2 0 2024-03-22T10:42:29.294 INFO:tasks.workunit.client.0.smithi115.stdout:4/9: fdatasync d1/f2 0 2024-03-22T10:42:29.294 INFO:tasks.workunit.client.0.smithi115.stdout:3/20: creat d3/f8 x:0 0 0 2024-03-22T10:42:29.294 INFO:tasks.workunit.client.0.smithi115.stdout:5/5: write f1 [610020,74607] 0 2024-03-22T10:42:29.305 INFO:tasks.workunit.client.0.smithi115.stdout:7/6: creat d2/f3 x:0 0 0 2024-03-22T10:42:29.306 INFO:tasks.workunit.client.0.smithi115.stdout:7/7: readlink - no filename 2024-03-22T10:42:29.311 INFO:tasks.workunit.client.0.smithi115.stdout:5/6: dread f1 [0,4194304] 0 2024-03-22T10:42:29.312 INFO:tasks.workunit.client.0.smithi115.stdout:0/18: dwrite f1 [0,4194304] 0 2024-03-22T10:42:29.321 INFO:tasks.workunit.client.0.smithi115.stdout:5/7: readlink l0 0 2024-03-22T10:42:29.322 INFO:tasks.workunit.client.0.smithi115.stdout:0/19: rmdir d3/d5 39 2024-03-22T10:42:29.346 INFO:tasks.workunit.client.0.smithi115.stdout:1/12: dwrite d0/f1 [0,4194304] 0 2024-03-22T10:42:29.358 INFO:tasks.workunit.client.0.smithi115.stdout:1/13: rename d0 to d0/d2 22 2024-03-22T10:42:29.361 INFO:tasks.workunit.client.0.smithi115.stdout:3/21: dwrite f1 [0,4194304] 0 2024-03-22T10:42:29.372 INFO:tasks.workunit.client.0.smithi115.stdout:4/10: dwrite d1/f2 [0,4194304] 0 2024-03-22T10:42:29.374 INFO:tasks.workunit.client.0.smithi115.stdout:1/14: write d0/f1 [2173068,29319] 0 2024-03-22T10:42:29.374 INFO:tasks.workunit.client.0.smithi115.stdout:7/8: dwrite d2/f3 [0,4194304] 0 2024-03-22T10:42:29.377 INFO:tasks.workunit.client.0.smithi115.stdout:7/9: chown f1 0 1 2024-03-22T10:42:29.381 INFO:tasks.workunit.client.0.smithi115.stdout:0/20: dwrite f1 [0,4194304] 0 2024-03-22T10:42:29.407 INFO:tasks.workunit.client.0.smithi115.stdout:3/22: dwrite d3/f8 [0,4194304] 0 2024-03-22T10:42:29.421 INFO:tasks.workunit.client.0.smithi115.stdout:0/21: dwrite f1 [0,4194304] 0 2024-03-22T10:42:29.427 INFO:tasks.workunit.client.0.smithi115.stdout:4/11: rename l0 to d1/l3 0 2024-03-22T10:42:29.431 INFO:tasks.workunit.client.0.smithi115.stdout:1/15: fsync d0/f1 0 2024-03-22T10:42:29.436 INFO:tasks.workunit.client.0.smithi115.stdout:5/8: link l0 d2/l3 0 2024-03-22T10:42:29.436 INFO:tasks.workunit.client.0.smithi115.stdout:1/16: write d0/f1 [3091657,48138] 0 2024-03-22T10:42:29.445 INFO:tasks.workunit.client.0.smithi115.stdout:4/12: unlink d1/f2 0 2024-03-22T10:42:29.445 INFO:tasks.workunit.client.0.smithi115.stdout:4/13: write - no filename 2024-03-22T10:42:29.449 INFO:tasks.workunit.client.0.smithi115.stdout:1/17: creat d0/f3 x:0 0 0 2024-03-22T10:42:29.453 INFO:tasks.workunit.client.0.smithi115.stdout:4/14: symlink d1/l4 0 2024-03-22T10:42:29.454 INFO:tasks.workunit.client.0.smithi115.stdout:5/9: creat d2/f4 x:0 0 0 2024-03-22T10:42:29.458 INFO:tasks.workunit.client.0.smithi115.stdout:4/15: mknod d1/c5 0 2024-03-22T10:42:29.463 INFO:tasks.workunit.client.0.smithi115.stdout:5/10: creat d2/f5 x:0 0 0 2024-03-22T10:42:29.466 INFO:tasks.workunit.client.0.smithi115.stdout:4/16: creat d1/f6 x:0 0 0 2024-03-22T10:42:29.468 INFO:tasks.workunit.client.0.smithi115.stdout:4/17: truncate d1/f6 173068 0 2024-03-22T10:42:29.475 INFO:tasks.workunit.client.0.smithi115.stdout:4/18: dread d1/f6 [0,4194304] 0 2024-03-22T10:42:29.476 INFO:tasks.workunit.client.0.smithi115.stdout:8/2: getdents . 0 2024-03-22T10:42:29.479 INFO:tasks.workunit.client.0.smithi115.stdout:8/3: dread - f0 zero size 2024-03-22T10:42:29.484 INFO:tasks.workunit.client.0.smithi115.stdout:1/18: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:42:29.484 INFO:tasks.workunit.client.0.smithi115.stdout:8/4: write f0 [106812,122341] 0 2024-03-22T10:42:29.485 INFO:tasks.workunit.client.0.smithi115.stdout:9/11: fsync f0 0 2024-03-22T10:42:29.485 INFO:tasks.workunit.client.0.smithi115.stdout:5/11: link d2/l3 d2/l6 0 2024-03-22T10:42:29.486 INFO:tasks.workunit.client.0.smithi115.stdout:9/12: dread - f0 zero size 2024-03-22T10:42:29.488 INFO:tasks.workunit.client.0.smithi115.stdout:5/12: write d2/f5 [115490,59577] 0 2024-03-22T10:42:29.490 INFO:tasks.workunit.client.0.smithi115.stdout:5/13: dread - d2/f4 zero size 2024-03-22T10:42:29.491 INFO:tasks.workunit.client.0.smithi115.stdout:5/14: chown f1 3226 1 2024-03-22T10:42:29.494 INFO:tasks.workunit.client.0.smithi115.stdout:0/22: rename d3/d5 to d3/d7 0 2024-03-22T10:42:29.501 INFO:tasks.workunit.client.0.smithi115.stdout:1/19: creat d0/f4 x:0 0 0 2024-03-22T10:42:29.512 INFO:tasks.workunit.client.0.smithi115.stdout:0/23: unlink f1 0 2024-03-22T10:42:29.519 INFO:tasks.workunit.client.0.smithi115.stdout:8/5: dwrite f0 [0,4194304] 0 2024-03-22T10:42:29.520 INFO:tasks.workunit.client.0.smithi115.stdout:8/6: readlink - no filename 2024-03-22T10:42:29.522 INFO:tasks.workunit.client.0.smithi115.stdout:8/7: write f0 [3127400,60803] 0 2024-03-22T10:42:29.530 INFO:tasks.workunit.client.0.smithi115.stdout:5/15: dwrite d2/f5 [0,4194304] 0 2024-03-22T10:42:29.533 INFO:tasks.workunit.client.0.smithi115.stdout:9/13: getdents . 0 2024-03-22T10:42:29.536 INFO:tasks.workunit.client.0.smithi115.stdout:9/14: truncate f0 514162 0 2024-03-22T10:42:29.541 INFO:tasks.workunit.client.0.smithi115.stdout:5/16: fdatasync f1 0 2024-03-22T10:42:29.543 INFO:tasks.workunit.client.0.smithi115.stdout:4/19: readlink d1/l3 0 2024-03-22T10:42:29.543 INFO:tasks.workunit.client.0.smithi115.stdout:5/17: fsync f1 0 2024-03-22T10:42:29.543 INFO:tasks.workunit.client.0.smithi115.stdout:7/10: truncate f1 1550171 0 2024-03-22T10:42:29.546 INFO:tasks.workunit.client.0.smithi115.stdout:0/24: getdents d3/d7 0 2024-03-22T10:42:29.546 INFO:tasks.workunit.client.0.smithi115.stdout:0/25: fsync - no filename 2024-03-22T10:42:29.548 INFO:tasks.workunit.client.0.smithi115.stdout:8/8: dread f0 [0,4194304] 0 2024-03-22T10:42:29.549 INFO:tasks.workunit.client.0.smithi115.stdout:7/11: symlink d2/l4 0 2024-03-22T10:42:29.550 INFO:tasks.workunit.client.0.smithi115.stdout:0/26: chown d3/d7/c6 2816 1 2024-03-22T10:42:29.550 INFO:tasks.workunit.client.0.smithi115.stdout:0/27: dwrite - no filename 2024-03-22T10:42:29.550 INFO:tasks.workunit.client.0.smithi115.stdout:0/28: read - no filename 2024-03-22T10:42:29.557 INFO:tasks.workunit.client.0.smithi115.stdout:8/9: write f0 [2693773,77112] 0 2024-03-22T10:42:29.558 INFO:tasks.workunit.client.0.smithi115.stdout:0/29: creat d3/d7/f8 x:0 0 0 2024-03-22T10:42:29.563 INFO:tasks.workunit.client.0.smithi115.stdout:0/30: symlink d3/l9 0 2024-03-22T10:42:29.569 INFO:tasks.workunit.client.0.smithi115.stdout:1/20: rmdir d0 39 2024-03-22T10:42:29.570 INFO:tasks.workunit.client.0.smithi115.stdout:8/10: dread f0 [0,4194304] 0 2024-03-22T10:42:29.573 INFO:tasks.workunit.client.0.smithi115.stdout:8/11: chown f0 1 1 2024-03-22T10:42:29.574 INFO:tasks.workunit.client.0.smithi115.stdout:0/31: mkdir d3/da 0 2024-03-22T10:42:29.575 INFO:tasks.workunit.client.0.smithi115.stdout:1/21: write d0/f1 [1859259,98957] 0 2024-03-22T10:42:29.576 INFO:tasks.workunit.client.0.smithi115.stdout:0/32: chown d3/d7 5418 1 2024-03-22T10:42:29.582 INFO:tasks.workunit.client.0.smithi115.stdout:0/33: write d3/d7/f8 [594198,102768] 0 2024-03-22T10:42:29.587 INFO:tasks.workunit.client.0.smithi115.stdout:4/20: dwrite d1/f6 [0,4194304] 0 2024-03-22T10:42:29.594 INFO:tasks.workunit.client.0.smithi115.stdout:1/22: dread d0/f3 [0,4194304] 0 2024-03-22T10:42:29.597 INFO:tasks.workunit.client.0.smithi115.stdout:8/12: dread f0 [0,4194304] 0 2024-03-22T10:42:29.601 INFO:tasks.workunit.client.0.smithi115.stdout:1/23: symlink d0/l5 0 2024-03-22T10:42:29.603 INFO:tasks.workunit.client.0.smithi115.stdout:0/34: getdents d3/d7 0 2024-03-22T10:42:29.605 INFO:tasks.workunit.client.0.smithi115.stdout:0/35: stat d3/d7/c6 0 2024-03-22T10:42:29.606 INFO:tasks.workunit.client.0.smithi115.stdout:0/36: chown d3/d7 44 1 2024-03-22T10:42:29.608 INFO:tasks.workunit.client.0.smithi115.stdout:1/24: write d0/f3 [4433054,52473] 0 2024-03-22T10:42:29.608 INFO:tasks.workunit.client.0.smithi115.stdout:0/37: chown d3/d4 1654 1 2024-03-22T10:42:29.614 INFO:tasks.workunit.client.0.smithi115.stdout:0/38: mknod d3/d7/cb 0 2024-03-22T10:42:29.624 INFO:tasks.workunit.client.0.smithi115.stdout:0/39: getdents d3 0 2024-03-22T10:42:29.628 INFO:tasks.workunit.client.0.smithi115.stdout:0/40: creat d3/fc x:0 0 0 2024-03-22T10:42:29.629 INFO:tasks.workunit.client.0.smithi115.stdout:8/13: dwrite f0 [0,4194304] 0 2024-03-22T10:42:29.633 INFO:tasks.workunit.client.0.smithi115.stdout:1/25: dwrite d0/f1 [0,4194304] 0 2024-03-22T10:42:29.633 INFO:tasks.workunit.client.0.smithi115.stdout:8/14: mknod c1 0 2024-03-22T10:42:29.635 INFO:tasks.workunit.client.0.smithi115.stdout:1/26: dread - d0/f4 zero size 2024-03-22T10:42:29.637 INFO:tasks.workunit.client.0.smithi115.stdout:8/15: creat f2 x:0 0 0 2024-03-22T10:42:29.648 INFO:tasks.workunit.client.0.smithi115.stdout:8/16: symlink l3 0 2024-03-22T10:42:29.649 INFO:tasks.workunit.client.0.smithi115.stdout:8/17: stat f2 0 2024-03-22T10:42:29.652 INFO:tasks.workunit.client.0.smithi115.stdout:8/18: dread - f2 zero size 2024-03-22T10:42:29.654 INFO:tasks.workunit.client.0.smithi115.stdout:1/27: dread d0/f3 [0,4194304] 0 2024-03-22T10:42:29.656 INFO:tasks.workunit.client.0.smithi115.stdout:1/28: write d0/f1 [4325179,24102] 0 2024-03-22T10:42:29.661 INFO:tasks.workunit.client.0.smithi115.stdout:1/29: mknod d0/c6 0 2024-03-22T10:42:29.664 INFO:tasks.workunit.client.0.smithi115.stdout:0/41: write d3/d7/f8 [1337611,38749] 0 2024-03-22T10:42:29.666 INFO:tasks.workunit.client.0.smithi115.stdout:0/42: fdatasync d3/fc 0 2024-03-22T10:42:29.667 INFO:tasks.workunit.client.0.smithi115.stdout:1/30: creat d0/f7 x:0 0 0 2024-03-22T10:42:29.669 INFO:tasks.workunit.client.0.smithi115.stdout:1/31: write d0/f7 [254767,17840] 0 2024-03-22T10:42:29.679 INFO:tasks.workunit.client.0.smithi115.stdout:8/19: truncate f0 3994835 0 2024-03-22T10:42:29.684 INFO:tasks.workunit.client.0.smithi115.stdout:0/43: dread d3/d7/f8 [0,4194304] 0 2024-03-22T10:42:29.687 INFO:tasks.workunit.client.0.smithi115.stdout:1/32: dread d0/f3 [0,4194304] 0 2024-03-22T10:42:29.688 INFO:tasks.workunit.client.0.smithi115.stdout:8/20: fsync f0 0 2024-03-22T10:42:29.690 INFO:tasks.workunit.client.0.smithi115.stdout:1/33: truncate d0/f7 882732 0 2024-03-22T10:42:29.693 INFO:tasks.workunit.client.0.smithi115.stdout:0/44: dread d3/d7/f8 [0,4194304] 0 2024-03-22T10:42:29.697 INFO:tasks.workunit.client.0.smithi115.stdout:8/21: link f0 f4 0 2024-03-22T10:42:29.698 INFO:tasks.workunit.client.0.smithi115.stdout:0/45: readlink d3/l9 0 2024-03-22T10:42:29.702 INFO:tasks.workunit.client.0.smithi115.stdout:8/22: write f2 [574557,57128] 0 2024-03-22T10:42:29.702 INFO:tasks.workunit.client.0.smithi115.stdout:8/23: creat f5 x:0 0 0 2024-03-22T10:42:29.705 INFO:tasks.workunit.client.0.smithi115.stdout:0/46: dread d3/d7/f8 [0,4194304] 0 2024-03-22T10:42:29.706 INFO:tasks.workunit.client.0.smithi115.stdout:1/34: truncate d0/f3 34653 0 2024-03-22T10:42:29.707 INFO:tasks.workunit.client.0.smithi115.stdout:0/47: readlink d3/l9 0 2024-03-22T10:42:29.711 INFO:tasks.workunit.client.0.smithi115.stdout:1/35: mkdir d0/d8 0 2024-03-22T10:42:29.713 INFO:tasks.workunit.client.0.smithi115.stdout:1/36: read - d0/f4 zero size 2024-03-22T10:42:29.713 INFO:tasks.workunit.client.0.smithi115.stdout:0/48: write d3/d7/f8 [1678079,90159] 0 2024-03-22T10:42:29.714 INFO:tasks.workunit.client.0.smithi115.stdout:0/49: read - d3/fc zero size 2024-03-22T10:42:29.725 INFO:tasks.workunit.client.0.smithi115.stdout:8/24: dwrite f5 [0,4194304] 0 2024-03-22T10:42:29.727 INFO:tasks.workunit.client.0.smithi115.stdout:0/50: dread d3/d7/f8 [0,4194304] 0 2024-03-22T10:42:29.735 INFO:tasks.workunit.client.0.smithi115.stdout:8/25: getdents . 0 2024-03-22T10:42:29.735 INFO:tasks.workunit.client.0.smithi115.stdout:8/26: rmdir - no directory 2024-03-22T10:42:29.748 INFO:tasks.workunit.client.0.smithi115.stdout:1/37: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:42:29.755 INFO:tasks.workunit.client.0.smithi115.stdout:1/38: unlink d0/f4 0 2024-03-22T10:42:29.760 INFO:tasks.workunit.client.0.smithi115.stdout:0/51: dwrite d3/d7/f8 [0,4194304] 0 2024-03-22T10:42:29.766 INFO:tasks.workunit.client.0.smithi115.stdout:1/39: getdents d0/d8 0 2024-03-22T10:42:29.769 INFO:tasks.workunit.client.0.smithi115.stdout:0/52: rmdir d3/d7 39 2024-03-22T10:42:29.769 INFO:tasks.workunit.client.0.smithi115.stdout:8/27: dwrite f2 [0,4194304] 0 2024-03-22T10:42:29.770 INFO:tasks.workunit.client.0.smithi115.stdout:1/40: read d0/f1 [3856886,35523] 0 2024-03-22T10:42:29.774 INFO:tasks.workunit.client.0.smithi115.stdout:1/41: write d0/f1 [2172713,81684] 0 2024-03-22T10:42:29.781 INFO:tasks.workunit.client.0.smithi115.stdout:8/28: mkdir d6 0 2024-03-22T10:42:29.789 INFO:tasks.workunit.client.0.smithi115.stdout:1/42: getdents d0/d8 0 2024-03-22T10:42:29.790 INFO:tasks.workunit.client.0.smithi115.stdout:8/29: creat d6/f7 x:0 0 0 2024-03-22T10:42:29.799 INFO:tasks.workunit.client.0.smithi115.stdout:8/30: mknod d6/c8 0 2024-03-22T10:42:29.809 INFO:tasks.workunit.client.0.smithi115.stdout:0/53: dwrite d3/fc [0,4194304] 0 2024-03-22T10:42:29.817 INFO:tasks.workunit.client.0.smithi115.stdout:0/54: creat d3/d7/fd x:0 0 0 2024-03-22T10:42:29.821 INFO:tasks.workunit.client.0.smithi115.stdout:0/55: chown d3/da 2 1 2024-03-22T10:42:29.822 INFO:tasks.workunit.client.0.smithi115.stdout:1/43: dwrite d0/f1 [0,4194304] 0 2024-03-22T10:42:29.824 INFO:tasks.workunit.client.0.smithi115.stdout:0/56: write d3/fc [2680555,98720] 0 2024-03-22T10:42:29.826 INFO:tasks.workunit.client.0.smithi115.stdout:1/44: unlink d0/l5 0 2024-03-22T10:42:29.826 INFO:tasks.workunit.client.0.smithi115.stdout:0/57: dread - d3/d7/fd zero size 2024-03-22T10:42:29.828 INFO:tasks.workunit.client.0.smithi115.stdout:0/58: read - d3/d7/fd zero size 2024-03-22T10:42:29.829 INFO:tasks.workunit.client.0.smithi115.stdout:8/31: dwrite f2 [0,4194304] 0 2024-03-22T10:42:29.829 INFO:tasks.workunit.client.0.smithi115.stdout:1/45: chown d0/c6 120602 1 2024-03-22T10:42:29.831 INFO:tasks.workunit.client.0.smithi115.stdout:0/59: write d3/d7/fd [258777,102458] 0 2024-03-22T10:42:29.836 INFO:tasks.workunit.client.0.smithi115.stdout:8/32: symlink d6/l9 0 2024-03-22T10:42:29.839 INFO:tasks.workunit.client.0.smithi115.stdout:1/46: link d0/f1 d0/d8/f9 0 2024-03-22T10:42:29.839 INFO:tasks.workunit.client.0.smithi115.stdout:8/33: dread - d6/f7 zero size 2024-03-22T10:42:29.848 INFO:tasks.workunit.client.0.smithi115.stdout:1/47: symlink d0/d8/la 0 2024-03-22T10:42:29.858 INFO:tasks.workunit.client.0.smithi115.stdout:1/48: dread d0/f7 [0,4194304] 0 2024-03-22T10:42:29.862 INFO:tasks.workunit.client.0.smithi115.stdout:1/49: mkdir d0/d8/db 0 2024-03-22T10:42:29.865 INFO:tasks.workunit.client.0.smithi115.stdout:1/50: creat d0/fc x:0 0 0 2024-03-22T10:42:29.865 INFO:tasks.workunit.client.0.smithi115.stdout:8/34: dwrite f4 [0,4194304] 0 2024-03-22T10:42:29.868 INFO:tasks.workunit.client.0.smithi115.stdout:1/51: symlink d0/d8/ld 0 2024-03-22T10:42:29.869 INFO:tasks.workunit.client.0.smithi115.stdout:8/35: rename d6/f7 to d6/fa 0 2024-03-22T10:42:29.886 INFO:tasks.workunit.client.0.smithi115.stdout:1/52: dread d0/d8/f9 [0,4194304] 0 2024-03-22T10:42:29.889 INFO:tasks.workunit.client.0.smithi115.stdout:8/36: dwrite f5 [0,4194304] 0 2024-03-22T10:42:29.895 INFO:tasks.workunit.client.0.smithi115.stdout:8/37: getdents d6 0 2024-03-22T10:42:29.897 INFO:tasks.workunit.client.0.smithi115.stdout:8/38: fsync d6/fa 0 2024-03-22T10:42:29.900 INFO:tasks.workunit.client.0.smithi115.stdout:8/39: write f0 [2265357,79075] 0 2024-03-22T10:42:29.902 INFO:tasks.workunit.client.0.smithi115.stdout:8/40: chown d6 134843 1 2024-03-22T10:42:29.902 INFO:tasks.workunit.client.0.smithi115.stdout:1/53: dread d0/f1 [0,4194304] 0 2024-03-22T10:42:29.906 INFO:tasks.workunit.client.0.smithi115.stdout:1/54: mkdir d0/d8/de 0 2024-03-22T10:42:29.909 INFO:tasks.workunit.client.0.smithi115.stdout:1/55: fsync d0/f1 0 2024-03-22T10:42:29.917 INFO:tasks.workunit.client.0.smithi115.stdout:8/41: dread f5 [0,4194304] 0 2024-03-22T10:42:29.917 INFO:tasks.workunit.client.0.smithi115.stdout:8/42: chown l3 1960 1 2024-03-22T10:42:29.921 INFO:tasks.workunit.client.0.smithi115.stdout:8/43: creat d6/fb x:0 0 0 2024-03-22T10:42:29.922 INFO:tasks.workunit.client.0.smithi115.stdout:1/56: dread d0/f1 [0,4194304] 0 2024-03-22T10:42:29.924 INFO:tasks.workunit.client.0.smithi115.stdout:8/44: creat d6/fc x:0 0 0 2024-03-22T10:42:29.927 INFO:tasks.workunit.client.0.smithi115.stdout:1/57: rename d0/d8/la to d0/d8/lf 0 2024-03-22T10:42:29.931 INFO:tasks.workunit.client.0.smithi115.stdout:1/58: getdents d0 0 2024-03-22T10:42:29.936 INFO:tasks.workunit.client.0.smithi115.stdout:1/59: mknod d0/d8/db/c10 0 2024-03-22T10:42:29.936 INFO:tasks.workunit.client.0.smithi115.stdout:8/45: dread f5 [0,4194304] 0 2024-03-22T10:42:29.941 INFO:tasks.workunit.client.0.smithi115.stdout:1/60: symlink d0/d8/l11 0 2024-03-22T10:42:29.949 INFO:tasks.workunit.client.0.smithi115.stdout:1/61: write d0/f7 [609367,29745] 0 2024-03-22T10:42:29.950 INFO:tasks.workunit.client.0.smithi115.stdout:8/46: dread f5 [0,4194304] 0 2024-03-22T10:42:29.951 INFO:tasks.workunit.client.0.smithi115.stdout:1/62: fsync d0/f3 0 2024-03-22T10:42:29.951 INFO:tasks.workunit.client.0.smithi115.stdout:8/47: chown l3 63852226 1 2024-03-22T10:42:29.954 INFO:tasks.workunit.client.0.smithi115.stdout:8/48: truncate d6/fb 412970 0 2024-03-22T10:42:29.956 INFO:tasks.workunit.client.0.smithi115.stdout:1/63: mkdir d0/d12 0 2024-03-22T10:42:29.957 INFO:tasks.workunit.client.0.smithi115.stdout:1/64: fsync d0/f1 0 2024-03-22T10:42:29.958 INFO:tasks.workunit.client.0.smithi115.stdout:8/49: mkdir d6/dd 0 2024-03-22T10:42:29.960 INFO:tasks.workunit.client.0.smithi115.stdout:8/50: write d6/fa [567977,74350] 0 2024-03-22T10:42:29.961 INFO:tasks.workunit.client.0.smithi115.stdout:1/65: symlink d0/l13 0 2024-03-22T10:42:29.965 INFO:tasks.workunit.client.0.smithi115.stdout:8/51: mknod d6/ce 0 2024-03-22T10:42:29.966 INFO:tasks.workunit.client.0.smithi115.stdout:1/66: creat d0/f14 x:0 0 0 2024-03-22T10:42:29.971 INFO:tasks.workunit.client.0.smithi115.stdout:8/52: link f4 d6/dd/ff 0 2024-03-22T10:42:29.973 INFO:tasks.workunit.client.0.smithi115.stdout:8/53: creat d6/f10 x:0 0 0 2024-03-22T10:42:29.974 INFO:tasks.workunit.client.0.smithi115.stdout:8/54: dread - d6/f10 zero size 2024-03-22T10:42:29.976 INFO:tasks.workunit.client.0.smithi115.stdout:8/55: chown d6/ce 119 1 2024-03-22T10:42:29.984 INFO:tasks.workunit.client.0.smithi115.stdout:1/67: dwrite d0/f14 [0,4194304] 0 2024-03-22T10:42:29.990 INFO:tasks.workunit.client.0.smithi115.stdout:1/68: symlink d0/d8/db/l15 0 2024-03-22T10:42:29.990 INFO:tasks.workunit.client.0.smithi115.stdout:8/56: truncate f5 4035992 0 2024-03-22T10:42:29.991 INFO:tasks.workunit.client.0.smithi115.stdout:1/69: dread - d0/fc zero size 2024-03-22T10:42:29.992 INFO:tasks.workunit.client.0.smithi115.stdout:1/70: stat d0/d8 0 2024-03-22T10:42:29.995 INFO:tasks.workunit.client.0.smithi115.stdout:1/71: creat d0/d8/db/f16 x:0 0 0 2024-03-22T10:42:30.007 INFO:tasks.workunit.client.0.smithi115.stdout:8/57: write f0 [261072,74928] 0 2024-03-22T10:42:30.010 INFO:tasks.workunit.client.0.smithi115.stdout:1/72: dread d0/f1 [0,4194304] 0 2024-03-22T10:42:30.013 INFO:tasks.workunit.client.0.smithi115.stdout:8/58: creat d6/dd/f11 x:0 0 0 2024-03-22T10:42:30.014 INFO:tasks.workunit.client.0.smithi115.stdout:1/73: mkdir d0/d12/d17 0 2024-03-22T10:42:30.015 INFO:tasks.workunit.client.0.smithi115.stdout:8/59: mkdir d6/dd/d12 0 2024-03-22T10:42:30.016 INFO:tasks.workunit.client.0.smithi115.stdout:1/74: rmdir d0/d8/db 39 2024-03-22T10:42:30.020 INFO:tasks.workunit.client.0.smithi115.stdout:8/60: rmdir d6/dd 39 2024-03-22T10:42:30.021 INFO:tasks.workunit.client.0.smithi115.stdout:8/61: chown l3 17 1 2024-03-22T10:42:30.024 INFO:tasks.workunit.client.0.smithi115.stdout:8/62: symlink d6/l13 0 2024-03-22T10:42:30.027 INFO:tasks.workunit.client.0.smithi115.stdout:8/63: mknod d6/dd/d12/c14 0 2024-03-22T10:42:30.028 INFO:tasks.workunit.client.0.smithi115.stdout:8/64: truncate d6/fb 489577 0 2024-03-22T10:42:30.045 INFO:tasks.workunit.client.0.smithi115.stdout:8/65: dwrite d6/fc [0,4194304] 0 2024-03-22T10:42:30.063 INFO:tasks.workunit.client.0.smithi115.stdout:8/66: dwrite d6/dd/f11 [0,4194304] 0 2024-03-22T10:42:30.065 INFO:tasks.workunit.client.0.smithi115.stdout:8/67: creat d6/dd/d12/f15 x:0 0 0 2024-03-22T10:42:30.068 INFO:tasks.workunit.client.0.smithi115.stdout:8/68: creat d6/f16 x:0 0 0 2024-03-22T10:42:30.074 INFO:tasks.workunit.client.0.smithi115.stdout:8/69: creat d6/f17 x:0 0 0 2024-03-22T10:42:30.076 INFO:tasks.workunit.client.0.smithi115.stdout:8/70: mknod d6/c18 0 2024-03-22T10:42:30.088 INFO:tasks.workunit.client.0.smithi115.stdout:8/71: dread f4 [0,4194304] 0 2024-03-22T10:42:30.092 INFO:tasks.workunit.client.0.smithi115.stdout:8/72: rename f0 to d6/dd/d12/f19 0 2024-03-22T10:42:30.094 INFO:tasks.workunit.client.0.smithi115.stdout:8/73: dread - d6/dd/d12/f15 zero size 2024-03-22T10:42:30.095 INFO:tasks.workunit.client.0.smithi115.stdout:8/74: write d6/f16 [136868,58584] 0 2024-03-22T10:42:30.109 INFO:tasks.workunit.client.0.smithi115.stdout:8/75: dwrite f4 [0,4194304] 0 2024-03-22T10:42:30.118 INFO:tasks.workunit.client.0.smithi115.stdout:8/76: write f2 [2056960,119119] 0 2024-03-22T10:42:30.172 INFO:tasks.workunit.client.0.smithi115.stdout:8/77: creat d6/dd/d12/f1a x:0 0 0 2024-03-22T10:42:30.175 INFO:tasks.workunit.client.0.smithi115.stdout:8/78: truncate d6/dd/d12/f15 696398 0 2024-03-22T10:42:30.185 INFO:tasks.workunit.client.0.smithi115.stdout:8/79: dread d6/dd/f11 [0,4194304] 0 2024-03-22T10:42:30.188 INFO:tasks.workunit.client.0.smithi115.stdout:8/80: symlink d6/dd/l1b 0 2024-03-22T10:42:30.195 INFO:tasks.workunit.client.0.smithi115.stdout:8/81: rename d6/dd/l1b to d6/dd/d12/l1c 0 2024-03-22T10:42:30.199 INFO:tasks.workunit.client.0.smithi115.stdout:8/82: chown d6/dd 6 1 2024-03-22T10:42:30.200 INFO:tasks.workunit.client.0.smithi115.stdout:8/83: chown c1 7582 1 2024-03-22T10:42:30.217 INFO:tasks.workunit.client.0.smithi115.stdout:8/84: dwrite d6/dd/d12/f1a [0,4194304] 0 2024-03-22T10:42:30.221 INFO:tasks.workunit.client.0.smithi115.stdout:8/85: creat d6/dd/d12/f1d x:0 0 0 2024-03-22T10:42:30.224 INFO:tasks.workunit.client.0.smithi115.stdout:8/86: unlink d6/dd/d12/f1a 0 2024-03-22T10:42:30.244 INFO:tasks.workunit.client.0.smithi115.stdout:8/87: dwrite f4 [4194304,4194304] 0 2024-03-22T10:42:30.244 INFO:tasks.workunit.client.0.smithi115.stdout:8/88: readlink l3 0 2024-03-22T10:42:30.248 INFO:tasks.workunit.client.0.smithi115.stdout:8/89: truncate f5 2160524 0 2024-03-22T10:42:30.252 INFO:tasks.workunit.client.0.smithi115.stdout:8/90: symlink d6/dd/d12/l1e 0 2024-03-22T10:42:30.257 INFO:tasks.workunit.client.0.smithi115.stdout:8/91: link d6/ce d6/dd/c1f 0 2024-03-22T10:42:30.262 INFO:tasks.workunit.client.0.smithi115.stdout:8/92: creat d6/f20 x:0 0 0 2024-03-22T10:42:30.265 INFO:tasks.workunit.client.0.smithi115.stdout:8/93: symlink d6/dd/l21 0 2024-03-22T10:42:30.278 INFO:tasks.workunit.client.0.smithi115.stdout:8/94: truncate d6/fa 108598 0 2024-03-22T10:42:30.287 INFO:tasks.workunit.client.0.smithi115.stdout:8/95: dread d6/fa [0,4194304] 0 2024-03-22T10:42:30.299 INFO:tasks.workunit.client.0.smithi115.stdout:8/96: dread d6/dd/ff [0,4194304] 0 2024-03-22T10:42:30.301 INFO:tasks.workunit.client.0.smithi115.stdout:8/97: creat d6/dd/d12/f22 x:0 0 0 2024-03-22T10:42:30.305 INFO:tasks.workunit.client.0.smithi115.stdout:8/98: rename d6/f10 to d6/f23 0 2024-03-22T10:42:30.307 INFO:tasks.workunit.client.0.smithi115.stdout:8/99: mkdir d6/dd/d24 0 2024-03-22T10:42:30.312 INFO:tasks.workunit.client.0.smithi115.stdout:8/100: link d6/dd/d12/c14 d6/dd/d12/c25 0 2024-03-22T10:42:30.314 INFO:tasks.workunit.client.0.smithi115.stdout:8/101: rmdir d6/dd 39 2024-03-22T10:42:30.320 INFO:tasks.workunit.client.0.smithi115.stdout:8/102: dread f5 [0,4194304] 0 2024-03-22T10:42:30.322 INFO:tasks.workunit.client.0.smithi115.stdout:8/103: fdatasync d6/dd/d12/f22 0 2024-03-22T10:42:30.325 INFO:tasks.workunit.client.0.smithi115.stdout:8/104: getdents d6/dd/d24 0 2024-03-22T10:42:30.328 INFO:tasks.workunit.client.0.smithi115.stdout:8/105: creat d6/dd/f26 x:0 0 0 2024-03-22T10:42:30.336 INFO:tasks.workunit.client.0.smithi115.stdout:8/106: getdents d6/dd/d12 0 2024-03-22T10:42:30.340 INFO:tasks.workunit.client.0.smithi115.stdout:8/107: rename d6/dd/ff to d6/dd/d12/f27 0 2024-03-22T10:42:30.343 INFO:tasks.workunit.client.0.smithi115.stdout:8/108: creat d6/f28 x:0 0 0 2024-03-22T10:42:30.344 INFO:tasks.workunit.client.0.smithi115.stdout:8/109: truncate d6/f17 857279 0 2024-03-22T10:42:30.347 INFO:tasks.workunit.client.0.smithi115.stdout:8/110: creat d6/dd/f29 x:0 0 0 2024-03-22T10:42:30.347 INFO:tasks.workunit.client.0.smithi115.stdout:8/111: stat d6/l9 0 2024-03-22T10:42:30.350 INFO:tasks.workunit.client.0.smithi115.stdout:8/112: dread d6/f17 [0,4194304] 0 2024-03-22T10:42:30.358 INFO:tasks.workunit.client.0.smithi115.stdout:8/113: truncate f4 834031 0 2024-03-22T10:42:30.371 INFO:tasks.workunit.client.0.smithi115.stdout:8/114: dwrite d6/f28 [0,4194304] 0 2024-03-22T10:42:30.378 INFO:tasks.workunit.client.0.smithi115.stdout:8/115: mknod d6/dd/d12/c2a 0 2024-03-22T10:42:30.385 INFO:tasks.workunit.client.0.smithi115.stdout:8/116: rename d6/dd/l21 to d6/dd/d12/l2b 0 2024-03-22T10:42:30.398 INFO:tasks.workunit.client.0.smithi115.stdout:8/117: dwrite d6/f20 [0,4194304] 0 2024-03-22T10:42:30.406 INFO:tasks.workunit.client.0.smithi115.stdout:8/118: write d6/f17 [353420,97606] 0 2024-03-22T10:42:30.409 INFO:tasks.workunit.client.0.smithi115.stdout:8/119: dread d6/f16 [0,4194304] 0 2024-03-22T10:42:30.411 INFO:tasks.workunit.client.0.smithi115.stdout:8/120: dread d6/f16 [0,4194304] 0 2024-03-22T10:42:30.414 INFO:tasks.workunit.client.0.smithi115.stdout:8/121: truncate d6/dd/f26 812746 0 2024-03-22T10:42:30.417 INFO:tasks.workunit.client.0.smithi115.stdout:8/122: mknod d6/dd/d12/c2c 0 2024-03-22T10:42:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:30 smithi115 ceph-mon[29111]: mgrmap e27: smithi115.iqzwsz(active, since 1.0304s) 2024-03-22T10:42:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:30 smithi115 ceph-mon[29111]: [22/Mar/2024:10:42:29] ENGINE Bus STARTING 2024-03-22T10:42:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:30 smithi115 ceph-mon[29111]: [22/Mar/2024:10:42:29] ENGINE Serving on http://172.21.15.115:8765 2024-03-22T10:42:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:30 smithi115 ceph-mon[29111]: [22/Mar/2024:10:42:29] ENGINE Serving on https://172.21.15.115:7150 2024-03-22T10:42:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:30 smithi115 ceph-mon[29111]: [22/Mar/2024:10:42:29] ENGINE Bus STARTED 2024-03-22T10:42:30.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:30 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:30.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:30 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:30.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:30 smithi150 ceph-mon[34518]: mgrmap e27: smithi115.iqzwsz(active, since 1.0304s) 2024-03-22T10:42:30.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:30 smithi150 ceph-mon[34518]: [22/Mar/2024:10:42:29] ENGINE Bus STARTING 2024-03-22T10:42:30.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:30 smithi150 ceph-mon[34518]: [22/Mar/2024:10:42:29] ENGINE Serving on http://172.21.15.115:8765 2024-03-22T10:42:30.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:30 smithi150 ceph-mon[34518]: [22/Mar/2024:10:42:29] ENGINE Serving on https://172.21.15.115:7150 2024-03-22T10:42:30.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:30 smithi150 ceph-mon[34518]: [22/Mar/2024:10:42:29] ENGINE Bus STARTED 2024-03-22T10:42:30.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:30 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:30.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:30 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:31.024 INFO:tasks.workunit.client.0.smithi115.stdout:6/1: sync 2024-03-22T10:42:31.029 INFO:tasks.workunit.client.0.smithi115.stdout:6/2: creat f0 x:0 0 0 2024-03-22T10:42:31.030 INFO:tasks.workunit.client.0.smithi115.stdout:6/3: fdatasync f0 0 2024-03-22T10:42:31.035 INFO:tasks.workunit.client.0.smithi115.stdout:6/4: symlink l1 0 2024-03-22T10:42:31.039 INFO:tasks.workunit.client.0.smithi115.stdout:6/5: creat f2 x:0 0 0 2024-03-22T10:42:31.054 INFO:tasks.workunit.client.0.smithi115.stdout:6/6: getdents . 0 2024-03-22T10:42:31.061 INFO:tasks.workunit.client.0.smithi115.stdout:6/7: rename f0 to f3 0 2024-03-22T10:42:31.065 INFO:tasks.workunit.client.0.smithi115.stdout:6/8: mkdir d4 0 2024-03-22T10:42:31.066 INFO:tasks.workunit.client.0.smithi115.stdout:6/9: dread - f2 zero size 2024-03-22T10:42:31.073 INFO:tasks.workunit.client.0.smithi115.stdout:6/10: link f3 d4/f5 0 2024-03-22T10:42:31.074 INFO:tasks.workunit.client.0.smithi115.stdout:6/11: readlink l1 0 2024-03-22T10:42:31.076 INFO:tasks.workunit.client.0.smithi115.stdout:6/12: chown f2 15 1 2024-03-22T10:42:31.117 INFO:tasks.workunit.client.0.smithi115.stdout:6/13: dwrite f2 [0,4194304] 0 2024-03-22T10:42:31.149 INFO:tasks.workunit.client.0.smithi115.stdout:3/23: sync 2024-03-22T10:42:31.151 INFO:tasks.workunit.client.0.smithi115.stdout:2/11: sync 2024-03-22T10:42:31.154 INFO:tasks.workunit.client.0.smithi115.stdout:3/24: readlink d3/l5 0 2024-03-22T10:42:31.170 INFO:tasks.workunit.client.0.smithi115.stdout:3/25: dwrite f1 [0,4194304] 0 2024-03-22T10:42:31.213 INFO:tasks.workunit.client.0.smithi115.stdout:9/15: sync 2024-03-22T10:42:31.213 INFO:tasks.workunit.client.0.smithi115.stdout:5/18: sync 2024-03-22T10:42:31.216 INFO:tasks.workunit.client.0.smithi115.stdout:5/19: write d2/f4 [553105,29861] 0 2024-03-22T10:42:31.228 INFO:tasks.workunit.client.0.smithi115.stdout:2/12: mknod c3 0 2024-03-22T10:42:31.228 INFO:tasks.workunit.client.0.smithi115.stdout:2/13: fdatasync f0 0 2024-03-22T10:42:31.229 INFO:tasks.workunit.client.0.smithi115.stdout:3/26: creat d3/f9 x:0 0 0 2024-03-22T10:42:31.230 INFO:tasks.workunit.client.0.smithi115.stdout:9/16: creat f1 x:0 0 0 2024-03-22T10:42:31.231 INFO:tasks.workunit.client.0.smithi115.stdout:3/27: dread - d3/f9 zero size 2024-03-22T10:42:31.232 INFO:tasks.workunit.client.0.smithi115.stdout:3/28: rename d3 to d3/da 22 2024-03-22T10:42:31.233 INFO:tasks.workunit.client.0.smithi115.stdout:3/29: chown d3 2155103 1 2024-03-22T10:42:31.234 INFO:tasks.workunit.client.0.smithi115.stdout:2/14: unlink f0 0 2024-03-22T10:42:31.236 INFO:tasks.workunit.client.0.smithi115.stdout:2/15: chown c1 41252598 1 2024-03-22T10:42:31.236 INFO:tasks.workunit.client.0.smithi115.stdout:2/16: rmdir - no directory 2024-03-22T10:42:31.238 INFO:tasks.workunit.client.0.smithi115.stdout:5/20: unlink d2/l6 0 2024-03-22T10:42:31.247 INFO:tasks.workunit.client.0.smithi115.stdout:5/21: unlink d2/l3 0 2024-03-22T10:42:31.254 INFO:tasks.workunit.client.0.smithi115.stdout:7/12: sync 2024-03-22T10:42:31.254 INFO:tasks.workunit.client.0.smithi115.stdout:1/75: sync 2024-03-22T10:42:31.255 INFO:tasks.workunit.client.0.smithi115.stdout:4/21: sync 2024-03-22T10:42:31.255 INFO:tasks.workunit.client.0.smithi115.stdout:8/123: sync 2024-03-22T10:42:31.255 INFO:tasks.workunit.client.0.smithi115.stdout:0/60: sync 2024-03-22T10:42:31.256 INFO:tasks.workunit.client.0.smithi115.stdout:8/124: stat d6/dd/d12/c2c 0 2024-03-22T10:42:31.261 INFO:tasks.workunit.client.0.smithi115.stdout:5/22: dread d2/f5 [0,4194304] 0 2024-03-22T10:42:31.263 INFO:tasks.workunit.client.0.smithi115.stdout:4/22: link d1/f6 d1/f7 0 2024-03-22T10:42:31.264 INFO:tasks.workunit.client.0.smithi115.stdout:5/23: read d2/f5 [3910200,119844] 0 2024-03-22T10:42:31.265 INFO:tasks.workunit.client.0.smithi115.stdout:0/61: creat d3/da/fe x:0 0 0 2024-03-22T10:42:31.269 INFO:tasks.workunit.client.0.smithi115.stdout:5/24: dread f1 [0,4194304] 0 2024-03-22T10:42:31.270 INFO:tasks.workunit.client.0.smithi115.stdout:4/23: write d1/f7 [211075,36587] 0 2024-03-22T10:42:31.274 INFO:tasks.workunit.client.0.smithi115.stdout:7/13: dread d2/f3 [0,4194304] 0 2024-03-22T10:42:31.275 INFO:tasks.workunit.client.0.smithi115.stdout:1/76: link d0/d8/db/f16 d0/d8/db/f18 0 2024-03-22T10:42:31.280 INFO:tasks.workunit.client.0.smithi115.stdout:1/77: write d0/f14 [339549,77312] 0 2024-03-22T10:42:31.282 INFO:tasks.workunit.client.0.smithi115.stdout:8/125: dwrite d6/dd/d12/f15 [0,4194304] 0 2024-03-22T10:42:31.289 INFO:tasks.workunit.client.0.smithi115.stdout:1/78: symlink d0/d8/l19 0 2024-03-22T10:42:31.292 INFO:tasks.workunit.client.0.smithi115.stdout:8/126: dread d6/f17 [0,4194304] 0 2024-03-22T10:42:31.293 INFO:tasks.workunit.client.0.smithi115.stdout:7/14: dread d2/f3 [0,4194304] 0 2024-03-22T10:42:31.295 INFO:tasks.workunit.client.0.smithi115.stdout:8/127: write d6/dd/f29 [862822,78348] 0 2024-03-22T10:42:31.295 INFO:tasks.workunit.client.0.smithi115.stdout:1/79: mkdir d0/d8/de/d1a 0 2024-03-22T10:42:31.296 INFO:tasks.workunit.client.0.smithi115.stdout:4/24: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:31.297 INFO:tasks.workunit.client.0.smithi115.stdout:8/128: chown d6/fc 0 1 2024-03-22T10:42:31.301 INFO:tasks.workunit.client.0.smithi115.stdout:4/25: unlink d1/f6 0 2024-03-22T10:42:31.304 INFO:tasks.workunit.client.0.smithi115.stdout:4/26: mknod d1/c8 0 2024-03-22T10:42:31.307 INFO:tasks.workunit.client.0.smithi115.stdout:4/27: creat d1/f9 x:0 0 0 2024-03-22T10:42:31.309 INFO:tasks.workunit.client.0.smithi115.stdout:1/80: dread d0/f14 [0,4194304] 0 2024-03-22T10:42:31.311 INFO:tasks.workunit.client.0.smithi115.stdout:4/28: creat d1/fa x:0 0 0 2024-03-22T10:42:31.314 INFO:tasks.workunit.client.0.smithi115.stdout:1/81: mkdir d0/d12/d1b 0 2024-03-22T10:42:31.315 INFO:tasks.workunit.client.0.smithi115.stdout:4/29: write d1/f9 [790387,119349] 0 2024-03-22T10:42:31.321 INFO:tasks.workunit.client.0.smithi115.stdout:1/82: creat d0/d12/f1c x:0 0 0 2024-03-22T10:42:31.323 INFO:tasks.workunit.client.0.smithi115.stdout:4/30: truncate d1/fa 219734 0 2024-03-22T10:42:31.325 INFO:tasks.workunit.client.0.smithi115.stdout:8/129: dwrite d6/fa [0,4194304] 0 2024-03-22T10:42:31.325 INFO:tasks.workunit.client.0.smithi115.stdout:4/31: write d1/fa [1121750,84264] 0 2024-03-22T10:42:31.327 INFO:tasks.workunit.client.0.smithi115.stdout:8/130: fsync d6/dd/f26 0 2024-03-22T10:42:31.338 INFO:tasks.workunit.client.0.smithi115.stdout:4/32: creat d1/fb x:0 0 0 2024-03-22T10:42:31.341 INFO:tasks.workunit.client.0.smithi115.stdout:4/33: mkdir d1/dc 0 2024-03-22T10:42:31.342 INFO:tasks.workunit.client.0.smithi115.stdout:4/34: dread - d1/fb zero size 2024-03-22T10:42:31.348 INFO:tasks.workunit.client.0.smithi115.stdout:2/17: getdents . 0 2024-03-22T10:42:31.348 INFO:tasks.workunit.client.0.smithi115.stdout:9/17: truncate f0 94146 0 2024-03-22T10:42:31.349 INFO:tasks.workunit.client.0.smithi115.stdout:4/35: write d1/fb [766883,17691] 0 2024-03-22T10:42:31.349 INFO:tasks.workunit.client.0.smithi115.stdout:3/30: truncate d3/f7 36489 0 2024-03-22T10:42:31.352 INFO:tasks.workunit.client.0.smithi115.stdout:9/18: chown f1 19916 1 2024-03-22T10:42:31.356 INFO:tasks.workunit.client.0.smithi115.stdout:2/18: creat f4 x:0 0 0 2024-03-22T10:42:31.356 INFO:tasks.workunit.client.0.smithi115.stdout:4/36: chown d1/c5 0 1 2024-03-22T10:42:31.356 INFO:tasks.workunit.client.0.smithi115.stdout:9/19: dread - f1 zero size 2024-03-22T10:42:31.358 INFO:tasks.workunit.client.0.smithi115.stdout:3/31: rename f1 to d3/fb 0 2024-03-22T10:42:31.359 INFO:tasks.workunit.client.0.smithi115.stdout:4/37: fdatasync d1/f9 0 2024-03-22T10:42:31.359 INFO:tasks.workunit.client.0.smithi115.stdout:0/62: truncate d3/fc 1344415 0 2024-03-22T10:42:31.361 INFO:tasks.workunit.client.0.smithi115.stdout:2/19: symlink l5 0 2024-03-22T10:42:31.363 INFO:tasks.workunit.client.0.smithi115.stdout:2/20: chown c3 0 1 2024-03-22T10:42:31.364 INFO:tasks.workunit.client.0.smithi115.stdout:2/21: dread - f4 zero size 2024-03-22T10:42:31.364 INFO:tasks.workunit.client.0.smithi115.stdout:6/14: dwrite f3 [0,4194304] 0 2024-03-22T10:42:31.365 INFO:tasks.workunit.client.0.smithi115.stdout:9/20: mknod c2 0 2024-03-22T10:42:31.369 INFO:tasks.workunit.client.0.smithi115.stdout:7/15: truncate d2/f3 1005360 0 2024-03-22T10:42:31.373 INFO:tasks.workunit.client.0.smithi115.stdout:3/32: write d3/fb [3555202,112473] 0 2024-03-22T10:42:31.373 INFO:tasks.workunit.client.0.smithi115.stdout:9/21: creat f3 x:0 0 0 2024-03-22T10:42:31.379 INFO:tasks.workunit.client.0.smithi115.stdout:3/33: read d3/fb [3097801,85800] 0 2024-03-22T10:42:31.382 INFO:tasks.workunit.client.0.smithi115.stdout:7/16: creat d2/f5 x:0 0 0 2024-03-22T10:42:31.386 INFO:tasks.workunit.client.0.smithi115.stdout:4/38: dwrite d1/fb [0,4194304] 0 2024-03-22T10:42:31.393 INFO:tasks.workunit.client.0.smithi115.stdout:6/15: dread f2 [0,4194304] 0 2024-03-22T10:42:31.393 INFO:tasks.workunit.client.0.smithi115.stdout:8/131: truncate d6/dd/f11 3995640 0 2024-03-22T10:42:31.395 INFO:tasks.workunit.client.0.smithi115.stdout:0/63: mknod d3/d4/cf 0 2024-03-22T10:42:31.404 INFO:tasks.workunit.client.0.smithi115.stdout:2/22: fsync f4 0 2024-03-22T10:42:31.404 INFO:tasks.workunit.client.0.smithi115.stdout:2/23: rmdir - no directory 2024-03-22T10:42:31.407 INFO:tasks.workunit.client.0.smithi115.stdout:2/24: chown l5 639832705 1 2024-03-22T10:42:31.412 INFO:tasks.workunit.client.0.smithi115.stdout:7/17: chown d2/f3 446505 1 2024-03-22T10:42:31.413 INFO:tasks.workunit.client.0.smithi115.stdout:3/34: dwrite d3/f9 [0,4194304] 0 2024-03-22T10:42:31.415 INFO:tasks.workunit.client.0.smithi115.stdout:7/18: chown d2/l4 2394151 1 2024-03-22T10:42:31.422 INFO:tasks.workunit.client.0.smithi115.stdout:9/22: dread f0 [0,4194304] 0 2024-03-22T10:42:31.426 INFO:tasks.workunit.client.0.smithi115.stdout:2/25: symlink l6 0 2024-03-22T10:42:31.426 INFO:tasks.workunit.client.0.smithi115.stdout:2/26: rmdir - no directory 2024-03-22T10:42:31.427 INFO:tasks.workunit.client.0.smithi115.stdout:0/64: fdatasync d3/fc 0 2024-03-22T10:42:31.429 INFO:tasks.workunit.client.0.smithi115.stdout:8/132: symlink d6/dd/d24/l2d 0 2024-03-22T10:42:31.434 INFO:tasks.workunit.client.0.smithi115.stdout:9/23: mknod c4 0 2024-03-22T10:42:31.435 INFO:tasks.workunit.client.0.smithi115.stdout:2/27: rename c3 to c7 0 2024-03-22T10:42:31.439 INFO:tasks.workunit.client.0.smithi115.stdout:8/133: chown d6/dd/d12/f19 29668 1 2024-03-22T10:42:31.445 INFO:tasks.workunit.client.0.smithi115.stdout:6/16: truncate f3 2763843 0 2024-03-22T10:42:31.449 INFO:tasks.workunit.client.0.smithi115.stdout:6/17: chown d4/f5 0 1 2024-03-22T10:42:31.459 INFO:tasks.workunit.client.0.smithi115.stdout:9/24: link f3 f5 0 2024-03-22T10:42:31.460 INFO:tasks.workunit.client.0.smithi115.stdout:0/65: link d3/da/fe d3/f10 0 2024-03-22T10:42:31.463 INFO:tasks.workunit.client.0.smithi115.stdout:4/39: creat d1/fd x:0 0 0 2024-03-22T10:42:31.467 INFO:tasks.workunit.client.0.smithi115.stdout:2/28: dwrite f4 [0,4194304] 0 2024-03-22T10:42:31.469 INFO:tasks.workunit.client.0.smithi115.stdout:9/25: write f0 [790267,51083] 0 2024-03-22T10:42:31.469 INFO:tasks.workunit.client.0.smithi115.stdout:9/26: readlink - no filename 2024-03-22T10:42:31.470 INFO:tasks.workunit.client.0.smithi115.stdout:7/19: chown d2/f3 852930 1 2024-03-22T10:42:31.471 INFO:tasks.workunit.client.0.smithi115.stdout:0/66: rename d3/f10 to d3/d4/f11 0 2024-03-22T10:42:31.473 INFO:tasks.workunit.client.0.smithi115.stdout:8/134: dwrite d6/fb [0,4194304] 0 2024-03-22T10:42:31.477 INFO:tasks.workunit.client.0.smithi115.stdout:2/29: creat f8 x:0 0 0 2024-03-22T10:42:31.478 INFO:tasks.workunit.client.0.smithi115.stdout:2/30: write f4 [1888138,64863] 0 2024-03-22T10:42:31.485 INFO:tasks.workunit.client.0.smithi115.stdout:8/135: rmdir d6/dd 39 2024-03-22T10:42:31.488 INFO:tasks.workunit.client.0.smithi115.stdout:2/31: creat f9 x:0 0 0 2024-03-22T10:42:31.495 INFO:tasks.workunit.client.0.smithi115.stdout:2/32: write f4 [180191,48995] 0 2024-03-22T10:42:31.495 INFO:tasks.workunit.client.0.smithi115.stdout:4/40: mknod d1/dc/ce 0 2024-03-22T10:42:31.502 INFO:tasks.workunit.client.0.smithi115.stdout:9/27: dwrite f0 [0,4194304] 0 2024-03-22T10:42:31.507 INFO:tasks.workunit.client.0.smithi115.stdout:4/41: write d1/fa [389436,37374] 0 2024-03-22T10:42:31.507 INFO:tasks.workunit.client.0.smithi115.stdout:2/33: getdents . 0 2024-03-22T10:42:31.509 INFO:tasks.workunit.client.0.smithi115.stdout:9/28: mkdir d6 0 2024-03-22T10:42:31.511 INFO:tasks.workunit.client.0.smithi115.stdout:2/34: dread - f8 zero size 2024-03-22T10:42:31.512 INFO:tasks.workunit.client.0.smithi115.stdout:2/35: rmdir - no directory 2024-03-22T10:42:31.514 INFO:tasks.workunit.client.0.smithi115.stdout:9/29: mkdir d6/d7 0 2024-03-22T10:42:31.515 INFO:tasks.workunit.client.0.smithi115.stdout:2/36: chown l5 89856 1 2024-03-22T10:42:31.516 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:31 smithi150 ceph-mon[34518]: mgrmap e28: smithi115.iqzwsz(active, since 2s) 2024-03-22T10:42:31.516 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:31 smithi150 ceph-mon[34518]: pgmap v4: 65 pgs: 65 active+clean; 234 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:42:31.522 INFO:tasks.workunit.client.0.smithi115.stdout:8/136: dwrite d6/f20 [0,4194304] 0 2024-03-22T10:42:31.523 INFO:tasks.workunit.client.0.smithi115.stdout:4/42: getdents d1/dc 0 2024-03-22T10:42:31.523 INFO:tasks.workunit.client.0.smithi115.stdout:7/20: dwrite f1 [0,4194304] 0 2024-03-22T10:42:31.523 INFO:tasks.workunit.client.0.smithi115.stdout:8/137: chown f2 349 1 2024-03-22T10:42:31.523 INFO:tasks.workunit.client.0.smithi115.stdout:8/138: write f2 [1020488,33426] 0 2024-03-22T10:42:31.526 INFO:tasks.workunit.client.0.smithi115.stdout:7/21: creat d2/f6 x:0 0 0 2024-03-22T10:42:31.527 INFO:tasks.workunit.client.0.smithi115.stdout:8/139: dread d6/f16 [0,4194304] 0 2024-03-22T10:42:31.529 INFO:tasks.workunit.client.0.smithi115.stdout:7/22: chown d2/f3 3 1 2024-03-22T10:42:31.530 INFO:tasks.workunit.client.0.smithi115.stdout:7/23: stat d2/f3 0 2024-03-22T10:42:31.532 INFO:tasks.workunit.client.0.smithi115.stdout:8/140: chown d6/c8 3542241 1 2024-03-22T10:42:31.535 INFO:tasks.workunit.client.0.smithi115.stdout:7/24: mkdir d2/d7 0 2024-03-22T10:42:31.543 INFO:tasks.workunit.client.0.smithi115.stdout:4/43: dread d1/fa [0,4194304] 0 2024-03-22T10:42:31.552 INFO:tasks.workunit.client.0.smithi115.stdout:9/30: dwrite f3 [0,4194304] 0 2024-03-22T10:42:31.555 INFO:tasks.workunit.client.0.smithi115.stdout:9/31: write f3 [2822720,81136] 0 2024-03-22T10:42:31.557 INFO:tasks.workunit.client.0.smithi115.stdout:7/25: write d2/f3 [1338429,98452] 0 2024-03-22T10:42:31.558 INFO:tasks.workunit.client.0.smithi115.stdout:8/141: link d6/dd/d12/f19 d6/dd/f2e 0 2024-03-22T10:42:31.559 INFO:tasks.workunit.client.0.smithi115.stdout:4/44: creat d1/ff x:0 0 0 2024-03-22T10:42:31.560 INFO:tasks.workunit.client.0.smithi115.stdout:9/32: mkdir d6/d8 0 2024-03-22T10:42:31.560 INFO:tasks.workunit.client.0.smithi115.stdout:7/26: truncate d2/f5 1036446 0 2024-03-22T10:42:31.562 INFO:tasks.workunit.client.0.smithi115.stdout:9/33: write f1 [503704,50410] 0 2024-03-22T10:42:31.569 INFO:tasks.workunit.client.0.smithi115.stdout:4/45: creat d1/f10 x:0 0 0 2024-03-22T10:42:31.569 INFO:tasks.workunit.client.0.smithi115.stdout:8/142: truncate d6/dd/f2e 1182965 0 2024-03-22T10:42:31.571 INFO:tasks.workunit.client.0.smithi115.stdout:7/27: getdents d2 0 2024-03-22T10:42:31.577 INFO:tasks.workunit.client.0.smithi115.stdout:8/143: dread d6/f17 [0,4194304] 0 2024-03-22T10:42:31.578 INFO:tasks.workunit.client.0.smithi115.stdout:4/46: write d1/fa [723351,108322] 0 2024-03-22T10:42:31.581 INFO:tasks.workunit.client.0.smithi115.stdout:8/144: stat d6/dd/d12/c2c 0 2024-03-22T10:42:31.582 INFO:tasks.workunit.client.0.smithi115.stdout:7/28: getdents d2 0 2024-03-22T10:42:31.582 INFO:tasks.workunit.client.0.smithi115.stdout:4/47: read - d1/f10 zero size 2024-03-22T10:42:31.584 INFO:tasks.workunit.client.0.smithi115.stdout:4/48: chown d1/ff 482828661 1 2024-03-22T10:42:31.585 INFO:tasks.workunit.client.0.smithi115.stdout:8/145: unlink f2 0 2024-03-22T10:42:31.585 INFO:tasks.workunit.client.0.smithi115.stdout:7/29: mknod d2/c8 0 2024-03-22T10:42:31.588 INFO:tasks.workunit.client.0.smithi115.stdout:8/146: mknod d6/dd/c2f 0 2024-03-22T10:42:31.589 INFO:tasks.workunit.client.0.smithi115.stdout:9/34: dwrite f3 [0,4194304] 0 2024-03-22T10:42:31.589 INFO:tasks.workunit.client.0.smithi115.stdout:7/30: mkdir d2/d7/d9 0 2024-03-22T10:42:31.590 INFO:tasks.workunit.client.0.smithi115.stdout:7/31: fsync d2/f3 0 2024-03-22T10:42:31.591 INFO:tasks.workunit.client.0.smithi115.stdout:4/49: symlink d1/dc/l11 0 2024-03-22T10:42:31.595 INFO:tasks.workunit.client.0.smithi115.stdout:8/147: dread d6/f17 [0,4194304] 0 2024-03-22T10:42:31.595 INFO:tasks.workunit.client.0.smithi115.stdout:7/32: truncate d2/f6 448178 0 2024-03-22T10:42:31.595 INFO:tasks.workunit.client.0.smithi115.stdout:7/33: stat d2/f3 0 2024-03-22T10:42:31.597 INFO:tasks.workunit.client.0.smithi115.stdout:9/35: getdents d6/d7 0 2024-03-22T10:42:31.600 INFO:tasks.workunit.client.0.smithi115.stdout:7/34: unlink d2/f3 0 2024-03-22T10:42:31.602 INFO:tasks.workunit.client.0.smithi115.stdout:7/35: truncate f1 4720504 0 2024-03-22T10:42:31.605 INFO:tasks.workunit.client.0.smithi115.stdout:9/36: rename c4 to d6/d7/c9 0 2024-03-22T10:42:31.605 INFO:tasks.workunit.client.0.smithi115.stdout:9/37: write f1 [1213907,31361] 0 2024-03-22T10:42:31.605 INFO:tasks.workunit.client.0.smithi115.stdout:4/50: dread d1/f9 [0,4194304] 0 2024-03-22T10:42:31.607 INFO:tasks.workunit.client.0.smithi115.stdout:7/36: mkdir d2/d7/da 0 2024-03-22T10:42:31.608 INFO:tasks.workunit.client.0.smithi115.stdout:7/37: chown d2 36 1 2024-03-22T10:42:31.610 INFO:tasks.workunit.client.0.smithi115.stdout:4/51: mknod d1/dc/c12 0 2024-03-22T10:42:31.613 INFO:tasks.workunit.client.0.smithi115.stdout:4/52: readlink d1/l4 0 2024-03-22T10:42:31.618 INFO:tasks.workunit.client.0.smithi115.stdout:7/38: rmdir d2/d7/da 0 2024-03-22T10:42:31.620 INFO:tasks.workunit.client.0.smithi115.stdout:9/38: dread f3 [0,4194304] 0 2024-03-22T10:42:31.621 INFO:tasks.workunit.client.0.smithi115.stdout:9/39: chown d6/d8 5 1 2024-03-22T10:42:31.623 INFO:tasks.workunit.client.0.smithi115.stdout:4/53: link d1/c8 d1/dc/c13 0 2024-03-22T10:42:31.627 INFO:tasks.workunit.client.0.smithi115.stdout:9/40: creat d6/fa x:0 0 0 2024-03-22T10:42:31.628 INFO:tasks.workunit.client.0.smithi115.stdout:8/148: dwrite f4 [0,4194304] 0 2024-03-22T10:42:31.630 INFO:tasks.workunit.client.0.smithi115.stdout:8/149: fsync d6/dd/f26 0 2024-03-22T10:42:31.632 INFO:tasks.workunit.client.0.smithi115.stdout:9/41: symlink d6/d7/lb 0 2024-03-22T10:42:31.635 INFO:tasks.workunit.client.0.smithi115.stdout:7/39: dread f1 [0,4194304] 0 2024-03-22T10:42:31.635 INFO:tasks.workunit.client.0.smithi115.stdout:5/25: sync 2024-03-22T10:42:31.636 INFO:tasks.workunit.client.0.smithi115.stdout:9/42: creat d6/fc x:0 0 0 2024-03-22T10:42:31.641 INFO:tasks.workunit.client.0.smithi115.stdout:5/26: fsync d2/f5 0 2024-03-22T10:42:31.641 INFO:tasks.workunit.client.0.smithi115.stdout:8/150: link d6/dd/d12/c25 d6/c30 0 2024-03-22T10:42:31.641 INFO:tasks.workunit.client.0.smithi115.stdout:9/43: creat d6/d8/fd x:0 0 0 2024-03-22T10:42:31.642 INFO:tasks.workunit.client.0.smithi115.stdout:9/44: chown d6/d7 16 1 2024-03-22T10:42:31.645 INFO:tasks.workunit.client.0.smithi115.stdout:9/45: write f1 [1164981,71026] 0 2024-03-22T10:42:31.647 INFO:tasks.workunit.client.0.smithi115.stdout:8/151: unlink d6/dd/d12/f22 0 2024-03-22T10:42:31.649 INFO:tasks.workunit.client.0.smithi115.stdout:9/46: unlink c2 0 2024-03-22T10:42:31.649 INFO:tasks.workunit.client.0.smithi115.stdout:8/152: stat d6/dd/d24 0 2024-03-22T10:42:31.654 INFO:tasks.workunit.client.0.smithi115.stdout:8/153: chown d6/dd/d24/l2d 339783 1 2024-03-22T10:42:31.661 INFO:tasks.workunit.client.0.smithi115.stdout:7/40: dwrite f1 [0,4194304] 0 2024-03-22T10:42:31.667 INFO:tasks.workunit.client.0.smithi115.stdout:7/41: creat d2/d7/fb x:0 0 0 2024-03-22T10:42:31.668 INFO:tasks.workunit.client.0.smithi115.stdout:5/27: dwrite f1 [0,4194304] 0 2024-03-22T10:42:31.670 INFO:tasks.workunit.client.0.smithi115.stdout:7/42: dread - d2/d7/fb zero size 2024-03-22T10:42:31.671 INFO:tasks.workunit.client.0.smithi115.stdout:5/28: truncate f1 4306471 0 2024-03-22T10:42:31.674 INFO:tasks.workunit.client.0.smithi115.stdout:5/29: rename d2 to d2/d7 22 2024-03-22T10:42:31.676 INFO:tasks.workunit.client.0.smithi115.stdout:7/43: read - d2/d7/fb zero size 2024-03-22T10:42:31.678 INFO:tasks.workunit.client.0.smithi115.stdout:5/30: read f1 [3654466,91277] 0 2024-03-22T10:42:31.681 INFO:tasks.workunit.client.0.smithi115.stdout:8/154: dread d6/dd/d12/f19 [0,4194304] 0 2024-03-22T10:42:31.681 INFO:tasks.workunit.client.0.smithi115.stdout:9/47: dwrite f3 [4194304,4194304] 0 2024-03-22T10:42:31.688 INFO:tasks.workunit.client.0.smithi115.stdout:9/48: mknod d6/d8/ce 0 2024-03-22T10:42:31.688 INFO:tasks.workunit.client.0.smithi115.stdout:8/155: chown d6/dd/d12/c25 963360227 1 2024-03-22T10:42:31.690 INFO:tasks.workunit.client.0.smithi115.stdout:9/49: chown d6/fa 445300 1 2024-03-22T10:42:31.690 INFO:tasks.workunit.client.0.smithi115.stdout:8/156: write d6/f28 [117909,42438] 0 2024-03-22T10:42:31.705 INFO:tasks.workunit.client.0.smithi115.stdout:8/157: read d6/f20 [2102053,47929] 0 2024-03-22T10:42:31.712 INFO:tasks.workunit.client.0.smithi115.stdout:8/158: write d6/dd/f26 [1819213,49603] 0 2024-03-22T10:42:31.718 INFO:tasks.workunit.client.0.smithi115.stdout:7/44: dwrite d2/f6 [0,4194304] 0 2024-03-22T10:42:31.723 INFO:tasks.workunit.client.0.smithi115.stdout:8/159: dread - d6/dd/d12/f1d zero size 2024-03-22T10:42:31.728 INFO:tasks.workunit.client.0.smithi115.stdout:5/31: dwrite d2/f4 [0,4194304] 0 2024-03-22T10:42:31.729 INFO:tasks.workunit.client.0.smithi115.stdout:8/160: chown d6/dd/f29 98820197 1 2024-03-22T10:42:31.732 INFO:tasks.workunit.client.0.smithi115.stdout:8/161: stat d6/f20 0 2024-03-22T10:42:31.732 INFO:tasks.workunit.client.0.smithi115.stdout:5/32: chown d2/f4 598013 1 2024-03-22T10:42:31.732 INFO:tasks.workunit.client.0.smithi115.stdout:9/50: dwrite d6/fc [0,4194304] 0 2024-03-22T10:42:31.737 INFO:tasks.workunit.client.0.smithi115.stdout:5/33: mknod d2/c8 0 2024-03-22T10:42:31.737 INFO:tasks.workunit.client.0.smithi115.stdout:8/162: mkdir d6/dd/d12/d31 0 2024-03-22T10:42:31.739 INFO:tasks.workunit.client.0.smithi115.stdout:5/34: write d2/f5 [477699,81823] 0 2024-03-22T10:42:31.741 INFO:tasks.workunit.client.0.smithi115.stdout:9/51: link f3 d6/d7/ff 0 2024-03-22T10:42:31.742 INFO:tasks.workunit.client.0.smithi115.stdout:5/35: chown d2/c8 2549742 1 2024-03-22T10:42:31.746 INFO:tasks.workunit.client.0.smithi115.stdout:8/163: mknod d6/dd/d12/d31/c32 0 2024-03-22T10:42:31.746 INFO:tasks.workunit.client.0.smithi115.stdout:9/52: symlink d6/l10 0 2024-03-22T10:42:31.746 INFO:tasks.workunit.client.0.smithi115.stdout:9/53: fsync f0 0 2024-03-22T10:42:31.749 INFO:tasks.workunit.client.0.smithi115.stdout:9/54: creat d6/d7/f11 x:0 0 0 2024-03-22T10:42:31.752 INFO:tasks.workunit.client.0.smithi115.stdout:8/164: rename d6/dd/d12/f19 to d6/dd/f33 0 2024-03-22T10:42:31.754 INFO:tasks.workunit.client.0.smithi115.stdout:7/45: fsync d2/f6 0 2024-03-22T10:42:31.757 INFO:tasks.workunit.client.0.smithi115.stdout:5/36: rmdir d2 39 2024-03-22T10:42:31.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:31 smithi115 ceph-mon[29111]: mgrmap e28: smithi115.iqzwsz(active, since 2s) 2024-03-22T10:42:31.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:31 smithi115 ceph-mon[29111]: pgmap v4: 65 pgs: 65 active+clean; 234 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:42:31.764 INFO:tasks.workunit.client.0.smithi115.stdout:5/37: symlink d2/l9 0 2024-03-22T10:42:31.766 INFO:tasks.workunit.client.0.smithi115.stdout:9/55: truncate f0 654815 0 2024-03-22T10:42:31.768 INFO:tasks.workunit.client.0.smithi115.stdout:5/38: creat d2/fa x:0 0 0 2024-03-22T10:42:31.768 INFO:tasks.workunit.client.0.smithi115.stdout:8/165: fsync d6/dd/f2e 0 2024-03-22T10:42:31.769 INFO:tasks.workunit.client.0.smithi115.stdout:9/56: chown d6/d7/ff 323443724 1 2024-03-22T10:42:31.773 INFO:tasks.workunit.client.0.smithi115.stdout:5/39: write d2/f4 [3718898,125311] 0 2024-03-22T10:42:31.774 INFO:tasks.workunit.client.0.smithi115.stdout:8/166: creat d6/dd/f34 x:0 0 0 2024-03-22T10:42:31.776 INFO:tasks.workunit.client.0.smithi115.stdout:9/57: creat d6/f12 x:0 0 0 2024-03-22T10:42:31.778 INFO:tasks.workunit.client.0.smithi115.stdout:7/46: dwrite d2/d7/fb [0,4194304] 0 2024-03-22T10:42:31.780 INFO:tasks.workunit.client.0.smithi115.stdout:9/58: write d6/d8/fd [223906,51943] 0 2024-03-22T10:42:31.781 INFO:tasks.workunit.client.0.smithi115.stdout:8/167: mkdir d6/dd/d35 0 2024-03-22T10:42:31.786 INFO:tasks.workunit.client.0.smithi115.stdout:9/59: chown d6/d7/c9 225548 1 2024-03-22T10:42:31.786 INFO:tasks.workunit.client.0.smithi115.stdout:8/168: dread d6/f16 [0,4194304] 0 2024-03-22T10:42:31.790 INFO:tasks.workunit.client.0.smithi115.stdout:9/60: write f1 [1050865,42540] 0 2024-03-22T10:42:31.795 INFO:tasks.workunit.client.0.smithi115.stdout:8/169: getdents d6/dd/d24 0 2024-03-22T10:42:31.795 INFO:tasks.workunit.client.0.smithi115.stdout:9/61: chown d6/f12 1 1 2024-03-22T10:42:31.798 INFO:tasks.workunit.client.0.smithi115.stdout:9/62: fdatasync f1 0 2024-03-22T10:42:31.798 INFO:tasks.workunit.client.0.smithi115.stdout:8/170: fdatasync d6/fc 0 2024-03-22T10:42:31.801 INFO:tasks.workunit.client.0.smithi115.stdout:9/63: read f3 [5793206,86379] 0 2024-03-22T10:42:31.804 INFO:tasks.workunit.client.0.smithi115.stdout:8/171: mknod d6/dd/c36 0 2024-03-22T10:42:31.806 INFO:tasks.workunit.client.0.smithi115.stdout:7/47: dwrite f1 [4194304,4194304] 0 2024-03-22T10:42:31.807 INFO:tasks.workunit.client.0.smithi115.stdout:9/64: read f3 [449259,103765] 0 2024-03-22T10:42:31.807 INFO:tasks.workunit.client.0.smithi115.stdout:5/40: dwrite f1 [0,4194304] 0 2024-03-22T10:42:31.812 INFO:tasks.workunit.client.0.smithi115.stdout:7/48: write d2/f5 [1731734,3174] 0 2024-03-22T10:42:31.814 INFO:tasks.workunit.client.0.smithi115.stdout:5/41: mknod d2/cb 0 2024-03-22T10:42:31.816 INFO:tasks.workunit.client.0.smithi115.stdout:7/49: symlink d2/d7/d9/lc 0 2024-03-22T10:42:31.820 INFO:tasks.workunit.client.0.smithi115.stdout:5/42: creat d2/fc x:0 0 0 2024-03-22T10:42:31.822 INFO:tasks.workunit.client.0.smithi115.stdout:7/50: unlink d2/c8 0 2024-03-22T10:42:31.827 INFO:tasks.workunit.client.0.smithi115.stdout:5/43: rename d2/cb to d2/cd 0 2024-03-22T10:42:31.833 INFO:tasks.workunit.client.0.smithi115.stdout:7/51: rename d2/d7/d9/lc to d2/d7/ld 0 2024-03-22T10:42:31.833 INFO:tasks.workunit.client.0.smithi115.stdout:1/83: sync 2024-03-22T10:42:31.834 INFO:tasks.workunit.client.0.smithi115.stdout:9/65: dwrite d6/f12 [0,4194304] 0 2024-03-22T10:42:31.835 INFO:tasks.workunit.client.0.smithi115.stdout:5/44: rename d2/fa to d2/fe 0 2024-03-22T10:42:31.836 INFO:tasks.workunit.client.0.smithi115.stdout:8/172: dwrite d6/dd/f2e [0,4194304] 0 2024-03-22T10:42:31.837 INFO:tasks.workunit.client.0.smithi115.stdout:5/45: stat d2/l9 0 2024-03-22T10:42:31.838 INFO:tasks.workunit.client.0.smithi115.stdout:8/173: chown d6/fc 0 1 2024-03-22T10:42:31.839 INFO:tasks.workunit.client.0.smithi115.stdout:1/84: truncate d0/fc 551442 0 2024-03-22T10:42:31.840 INFO:tasks.workunit.client.0.smithi115.stdout:1/85: chown d0/d8 1783084187 1 2024-03-22T10:42:31.851 INFO:tasks.workunit.client.0.smithi115.stdout:8/174: getdents d6/dd/d35 0 2024-03-22T10:42:31.855 INFO:tasks.workunit.client.0.smithi115.stdout:8/175: chown f4 498635621 1 2024-03-22T10:42:31.865 INFO:tasks.workunit.client.0.smithi115.stdout:9/66: dread d6/fc [0,4194304] 0 2024-03-22T10:42:31.866 INFO:tasks.workunit.client.0.smithi115.stdout:7/52: dwrite f1 [0,4194304] 0 2024-03-22T10:42:31.869 INFO:tasks.workunit.client.0.smithi115.stdout:8/176: link d6/dd/f34 d6/dd/d12/f37 0 2024-03-22T10:42:31.870 INFO:tasks.workunit.client.0.smithi115.stdout:9/67: read f5 [2124707,75242] 0 2024-03-22T10:42:31.875 INFO:tasks.workunit.client.0.smithi115.stdout:8/177: rmdir d6/dd/d24 39 2024-03-22T10:42:31.877 INFO:tasks.workunit.client.0.smithi115.stdout:5/46: dwrite d2/fc [0,4194304] 0 2024-03-22T10:42:31.880 INFO:tasks.workunit.client.0.smithi115.stdout:7/53: rename d2/d7/ld to d2/le 0 2024-03-22T10:42:31.880 INFO:tasks.workunit.client.0.smithi115.stdout:1/86: dwrite d0/d12/f1c [0,4194304] 0 2024-03-22T10:42:31.883 INFO:tasks.workunit.client.0.smithi115.stdout:5/47: mkdir d2/df 0 2024-03-22T10:42:31.885 INFO:tasks.workunit.client.0.smithi115.stdout:8/178: mknod d6/dd/d12/c38 0 2024-03-22T10:42:31.887 INFO:tasks.workunit.client.0.smithi115.stdout:5/48: readlink l0 0 2024-03-22T10:42:31.887 INFO:tasks.workunit.client.0.smithi115.stdout:9/68: getdents d6 0 2024-03-22T10:42:31.890 INFO:tasks.workunit.client.0.smithi115.stdout:7/54: symlink d2/d7/d9/lf 0 2024-03-22T10:42:31.891 INFO:tasks.workunit.client.0.smithi115.stdout:1/87: creat d0/f1d x:0 0 0 2024-03-22T10:42:31.893 INFO:tasks.workunit.client.0.smithi115.stdout:5/49: rename f1 to d2/df/f10 0 2024-03-22T10:42:31.893 INFO:tasks.workunit.client.0.smithi115.stdout:9/69: write d6/fc [1297587,13044] 0 2024-03-22T10:42:31.893 INFO:tasks.workunit.client.0.smithi115.stdout:8/179: mknod d6/dd/d35/c39 0 2024-03-22T10:42:31.898 INFO:tasks.workunit.client.0.smithi115.stdout:9/70: symlink d6/d8/l13 0 2024-03-22T10:42:31.901 INFO:tasks.workunit.client.0.smithi115.stdout:5/50: rename d2/f5 to d2/f11 0 2024-03-22T10:42:31.907 INFO:tasks.workunit.client.0.smithi115.stdout:5/51: link d2/df/f10 d2/f12 0 2024-03-22T10:42:31.907 INFO:tasks.workunit.client.0.smithi115.stdout:8/180: dread d6/dd/f2e [0,4194304] 0 2024-03-22T10:42:31.909 INFO:tasks.workunit.client.0.smithi115.stdout:7/55: getdents d2/d7/d9 0 2024-03-22T10:42:31.911 INFO:tasks.workunit.client.0.smithi115.stdout:7/56: write d2/d7/fb [4763724,12305] 0 2024-03-22T10:42:31.913 INFO:tasks.workunit.client.0.smithi115.stdout:8/181: mkdir d6/dd/d24/d3a 0 2024-03-22T10:42:31.914 INFO:tasks.workunit.client.0.smithi115.stdout:1/88: write d0/d8/db/f18 [290921,16877] 0 2024-03-22T10:42:31.915 INFO:tasks.workunit.client.0.smithi115.stdout:7/57: write d2/d7/fb [4843877,45373] 0 2024-03-22T10:42:31.916 INFO:tasks.workunit.client.0.smithi115.stdout:9/71: getdents d6/d8 0 2024-03-22T10:42:31.918 INFO:tasks.workunit.client.0.smithi115.stdout:1/89: chown d0/d8/l11 1 1 2024-03-22T10:42:31.918 INFO:tasks.workunit.client.0.smithi115.stdout:9/72: truncate f1 1810230 0 2024-03-22T10:42:31.920 INFO:tasks.workunit.client.0.smithi115.stdout:9/73: fsync d6/f12 0 2024-03-22T10:42:31.920 INFO:tasks.workunit.client.0.smithi115.stdout:7/58: mkdir d2/d7/d9/d10 0 2024-03-22T10:42:31.920 INFO:tasks.workunit.client.0.smithi115.stdout:8/182: write d6/dd/f33 [2029698,43897] 0 2024-03-22T10:42:31.928 INFO:tasks.workunit.client.0.smithi115.stdout:9/74: truncate d6/d8/fd 853118 0 2024-03-22T10:42:31.928 INFO:tasks.workunit.client.0.smithi115.stdout:1/90: creat d0/d8/de/d1a/f1e x:0 0 0 2024-03-22T10:42:31.929 INFO:tasks.workunit.client.0.smithi115.stdout:8/183: creat d6/dd/d12/f3b x:0 0 0 2024-03-22T10:42:31.935 INFO:tasks.workunit.client.0.smithi115.stdout:1/91: write d0/fc [67330,6246] 0 2024-03-22T10:42:31.935 INFO:tasks.workunit.client.0.smithi115.stdout:9/75: chown d6/d7/f11 11048 1 2024-03-22T10:42:31.937 INFO:tasks.workunit.client.0.smithi115.stdout:8/184: creat d6/dd/d24/d3a/f3c x:0 0 0 2024-03-22T10:42:31.939 INFO:tasks.workunit.client.0.smithi115.stdout:5/52: dwrite d2/fe [0,4194304] 0 2024-03-22T10:42:31.942 INFO:tasks.workunit.client.0.smithi115.stdout:8/185: symlink d6/dd/d12/l3d 0 2024-03-22T10:42:31.943 INFO:tasks.workunit.client.0.smithi115.stdout:5/53: mkdir d2/df/d13 0 2024-03-22T10:42:31.943 INFO:tasks.workunit.client.0.smithi115.stdout:8/186: read d6/f17 [721027,23962] 0 2024-03-22T10:42:31.944 INFO:tasks.workunit.client.0.smithi115.stdout:1/92: symlink d0/d8/l1f 0 2024-03-22T10:42:31.945 INFO:tasks.workunit.client.0.smithi115.stdout:1/93: chown d0/d8 65278253 1 2024-03-22T10:42:31.945 INFO:tasks.workunit.client.0.smithi115.stdout:5/54: rmdir d2/df 39 2024-03-22T10:42:31.950 INFO:tasks.workunit.client.0.smithi115.stdout:7/59: dread d2/f6 [0,4194304] 0 2024-03-22T10:42:31.950 INFO:tasks.workunit.client.0.smithi115.stdout:1/94: fsync d0/f3 0 2024-03-22T10:42:31.950 INFO:tasks.workunit.client.0.smithi115.stdout:2/37: sync 2024-03-22T10:42:31.951 INFO:tasks.workunit.client.0.smithi115.stdout:3/35: sync 2024-03-22T10:42:31.951 INFO:tasks.workunit.client.0.smithi115.stdout:6/18: sync 2024-03-22T10:42:31.951 INFO:tasks.workunit.client.0.smithi115.stdout:8/187: creat d6/dd/f3e x:0 0 0 2024-03-22T10:42:31.952 INFO:tasks.workunit.client.0.smithi115.stdout:5/55: mknod d2/df/c14 0 2024-03-22T10:42:31.955 INFO:tasks.workunit.client.0.smithi115.stdout:5/56: readlink l0 0 2024-03-22T10:42:31.955 INFO:tasks.workunit.client.0.smithi115.stdout:2/38: mkdir da 0 2024-03-22T10:42:31.955 INFO:tasks.workunit.client.0.smithi115.stdout:7/60: mkdir d2/d7/d11 0 2024-03-22T10:42:31.955 INFO:tasks.workunit.client.0.smithi115.stdout:4/54: sync 2024-03-22T10:42:31.955 INFO:tasks.workunit.client.0.smithi115.stdout:0/67: sync 2024-03-22T10:42:31.956 INFO:tasks.workunit.client.0.smithi115.stdout:1/95: write d0/d8/db/f18 [1143026,79688] 0 2024-03-22T10:42:31.960 INFO:tasks.workunit.client.0.smithi115.stdout:7/61: write d2/d7/fb [3497350,116478] 0 2024-03-22T10:42:31.963 INFO:tasks.workunit.client.0.smithi115.stdout:5/57: fdatasync d2/df/f10 0 2024-03-22T10:42:31.965 INFO:tasks.workunit.client.0.smithi115.stdout:9/76: dwrite d6/f12 [0,4194304] 0 2024-03-22T10:42:31.967 INFO:tasks.workunit.client.0.smithi115.stdout:7/62: write f1 [4009326,106624] 0 2024-03-22T10:42:31.970 INFO:tasks.workunit.client.0.smithi115.stdout:4/55: mkdir d1/dc/d14 0 2024-03-22T10:42:31.972 INFO:tasks.workunit.client.0.smithi115.stdout:3/36: truncate d3/fb 3352316 0 2024-03-22T10:42:31.972 INFO:tasks.workunit.client.0.smithi115.stdout:8/188: write d6/dd/f34 [583211,49207] 0 2024-03-22T10:42:31.972 INFO:tasks.workunit.client.0.smithi115.stdout:5/58: rmdir d2/df 39 2024-03-22T10:42:31.974 INFO:tasks.workunit.client.0.smithi115.stdout:4/56: fsync d1/ff 0 2024-03-22T10:42:31.981 INFO:tasks.workunit.client.0.smithi115.stdout:7/63: rename d2/d7/d9 to d2/d7/d12 0 2024-03-22T10:42:31.984 INFO:tasks.workunit.client.0.smithi115.stdout:0/68: write d3/d4/f11 [387640,85324] 0 2024-03-22T10:42:31.985 INFO:tasks.workunit.client.0.smithi115.stdout:3/37: creat d3/fc x:0 0 0 2024-03-22T10:42:31.988 INFO:tasks.workunit.client.0.smithi115.stdout:8/189: creat d6/dd/d24/f3f x:0 0 0 2024-03-22T10:42:31.988 INFO:tasks.workunit.client.0.smithi115.stdout:1/96: truncate d0/f1 2009807 0 2024-03-22T10:42:31.989 INFO:tasks.workunit.client.0.smithi115.stdout:5/59: mkdir d2/d15 0 2024-03-22T10:42:31.989 INFO:tasks.workunit.client.0.smithi115.stdout:2/39: dwrite f8 [0,4194304] 0 2024-03-22T10:42:31.992 INFO:tasks.workunit.client.0.smithi115.stdout:1/97: read d0/f14 [457838,83302] 0 2024-03-22T10:42:31.999 INFO:tasks.workunit.client.0.smithi115.stdout:8/190: write f4 [229033,92370] 0 2024-03-22T10:42:32.001 INFO:tasks.workunit.client.0.smithi115.stdout:5/60: unlink d2/f12 0 2024-03-22T10:42:32.002 INFO:tasks.workunit.client.0.smithi115.stdout:2/40: unlink f4 0 2024-03-22T10:42:32.009 INFO:tasks.workunit.client.0.smithi115.stdout:0/69: symlink d3/l12 0 2024-03-22T10:42:32.010 INFO:tasks.workunit.client.0.smithi115.stdout:3/38: stat d3/fb 0 2024-03-22T10:42:32.012 INFO:tasks.workunit.client.0.smithi115.stdout:2/41: unlink f9 0 2024-03-22T10:42:32.016 INFO:tasks.workunit.client.0.smithi115.stdout:4/57: dwrite d1/fa [0,4194304] 0 2024-03-22T10:42:32.019 INFO:tasks.workunit.client.0.smithi115.stdout:8/191: mknod d6/c40 0 2024-03-22T10:42:32.020 INFO:tasks.workunit.client.0.smithi115.stdout:3/39: mkdir d3/dd 0 2024-03-22T10:42:32.021 INFO:tasks.workunit.client.0.smithi115.stdout:9/77: dwrite d6/d7/ff [0,4194304] 0 2024-03-22T10:42:32.024 INFO:tasks.workunit.client.0.smithi115.stdout:3/40: chown d3/fc 163082 1 2024-03-22T10:42:32.027 INFO:tasks.workunit.client.0.smithi115.stdout:7/64: dread d2/d7/fb [0,4194304] 0 2024-03-22T10:42:32.029 INFO:tasks.workunit.client.0.smithi115.stdout:4/58: write d1/fb [436650,13365] 0 2024-03-22T10:42:32.029 INFO:tasks.workunit.client.0.smithi115.stdout:9/78: mknod d6/d7/c14 0 2024-03-22T10:42:32.030 INFO:tasks.workunit.client.0.smithi115.stdout:0/70: dread d3/da/fe [0,4194304] 0 2024-03-22T10:42:32.031 INFO:tasks.workunit.client.0.smithi115.stdout:8/192: mkdir d6/dd/d35/d41 0 2024-03-22T10:42:32.031 INFO:tasks.workunit.client.0.smithi115.stdout:7/65: write d2/f5 [2031840,37017] 0 2024-03-22T10:42:32.035 INFO:tasks.workunit.client.0.smithi115.stdout:9/79: truncate d6/fa 555474 0 2024-03-22T10:42:32.036 INFO:tasks.workunit.client.0.smithi115.stdout:7/66: readlink d2/d7/d12/lf 0 2024-03-22T10:42:32.036 INFO:tasks.workunit.client.0.smithi115.stdout:9/80: chown d6 455 1 2024-03-22T10:42:32.037 INFO:tasks.workunit.client.0.smithi115.stdout:4/59: mknod d1/c15 0 2024-03-22T10:42:32.038 INFO:tasks.workunit.client.0.smithi115.stdout:2/42: dread f8 [0,4194304] 0 2024-03-22T10:42:32.040 INFO:tasks.workunit.client.0.smithi115.stdout:8/193: dread - d6/dd/d24/d3a/f3c zero size 2024-03-22T10:42:32.040 INFO:tasks.workunit.client.0.smithi115.stdout:2/43: chown l6 45790273 1 2024-03-22T10:42:32.044 INFO:tasks.workunit.client.0.smithi115.stdout:2/44: fsync f8 0 2024-03-22T10:42:32.045 INFO:tasks.workunit.client.0.smithi115.stdout:2/45: chown da 199338647 1 2024-03-22T10:42:32.049 INFO:tasks.workunit.client.0.smithi115.stdout:2/46: write f8 [3153731,6689] 0 2024-03-22T10:42:32.051 INFO:tasks.workunit.client.0.smithi115.stdout:1/98: truncate d0/d8/db/f16 220831 0 2024-03-22T10:42:32.051 INFO:tasks.workunit.client.0.smithi115.stdout:7/67: rename d2/f6 to d2/d7/f13 0 2024-03-22T10:42:32.054 INFO:tasks.workunit.client.0.smithi115.stdout:2/47: getdents da 0 2024-03-22T10:42:32.057 INFO:tasks.workunit.client.0.smithi115.stdout:9/81: fdatasync f5 0 2024-03-22T10:42:32.057 INFO:tasks.workunit.client.0.smithi115.stdout:5/61: truncate d2/f11 3176306 0 2024-03-22T10:42:32.061 INFO:tasks.workunit.client.0.smithi115.stdout:2/48: rename f8 to da/fb 0 2024-03-22T10:42:32.061 INFO:tasks.workunit.client.0.smithi115.stdout:4/60: dread d1/fb [0,4194304] 0 2024-03-22T10:42:32.064 INFO:tasks.workunit.client.0.smithi115.stdout:7/68: mknod d2/d7/d12/d10/c14 0 2024-03-22T10:42:32.064 INFO:tasks.workunit.client.0.smithi115.stdout:4/61: fdatasync d1/ff 0 2024-03-22T10:42:32.065 INFO:tasks.workunit.client.0.smithi115.stdout:2/49: chown c1 5 1 2024-03-22T10:42:32.066 INFO:tasks.workunit.client.0.smithi115.stdout:3/41: truncate d3/f9 3820907 0 2024-03-22T10:42:32.068 INFO:tasks.workunit.client.0.smithi115.stdout:5/62: write d2/f4 [4948511,111424] 0 2024-03-22T10:42:32.069 INFO:tasks.workunit.client.0.smithi115.stdout:1/99: write d0/d8/db/f18 [838623,26613] 0 2024-03-22T10:42:32.070 INFO:tasks.workunit.client.0.smithi115.stdout:8/194: dwrite f4 [0,4194304] 0 2024-03-22T10:42:32.070 INFO:tasks.workunit.client.0.smithi115.stdout:9/82: truncate d6/d7/f11 784624 0 2024-03-22T10:42:32.073 INFO:tasks.workunit.client.0.smithi115.stdout:7/69: fsync d2/f5 0 2024-03-22T10:42:32.074 INFO:tasks.workunit.client.0.smithi115.stdout:9/83: stat d6/d8 0 2024-03-22T10:42:32.076 INFO:tasks.workunit.client.0.smithi115.stdout:2/50: write da/fb [2682530,23289] 0 2024-03-22T10:42:32.076 INFO:tasks.workunit.client.0.smithi115.stdout:8/195: fdatasync d6/fa 0 2024-03-22T10:42:32.077 INFO:tasks.workunit.client.0.smithi115.stdout:7/70: chown d2/d7/fb 3076 1 2024-03-22T10:42:32.080 INFO:tasks.workunit.client.0.smithi115.stdout:0/71: dwrite d3/da/fe [0,4194304] 0 2024-03-22T10:42:32.080 INFO:tasks.workunit.client.0.smithi115.stdout:4/62: mkdir d1/d16 0 2024-03-22T10:42:32.083 INFO:tasks.workunit.client.0.smithi115.stdout:8/196: write d6/dd/f33 [4041591,92200] 0 2024-03-22T10:42:32.084 INFO:tasks.workunit.client.0.smithi115.stdout:3/42: creat d3/fe x:0 0 0 2024-03-22T10:42:32.086 INFO:tasks.workunit.client.0.smithi115.stdout:9/84: symlink d6/l15 0 2024-03-22T10:42:32.086 INFO:tasks.workunit.client.0.smithi115.stdout:4/63: chown d1/fa 2 1 2024-03-22T10:42:32.088 INFO:tasks.workunit.client.0.smithi115.stdout:0/72: symlink d3/da/l13 0 2024-03-22T10:42:32.092 INFO:tasks.workunit.client.0.smithi115.stdout:0/73: write d3/fc [2305669,91666] 0 2024-03-22T10:42:32.095 INFO:tasks.workunit.client.0.smithi115.stdout:1/100: symlink d0/d12/d1b/l20 0 2024-03-22T10:42:32.096 INFO:tasks.workunit.client.0.smithi115.stdout:1/101: chown d0/d8/de 1809 1 2024-03-22T10:42:32.099 INFO:tasks.workunit.client.0.smithi115.stdout:1/102: read d0/f7 [282766,72845] 0 2024-03-22T10:42:32.103 INFO:tasks.workunit.client.0.smithi115.stdout:4/64: mkdir d1/dc/d17 0 2024-03-22T10:42:32.103 INFO:tasks.workunit.client.0.smithi115.stdout:1/103: truncate d0/f1d 873786 0 2024-03-22T10:42:32.104 INFO:tasks.workunit.client.0.smithi115.stdout:4/65: dread - d1/ff zero size 2024-03-22T10:42:32.106 INFO:tasks.workunit.client.0.smithi115.stdout:9/85: creat d6/d7/f16 x:0 0 0 2024-03-22T10:42:32.107 INFO:tasks.workunit.client.0.smithi115.stdout:5/63: dread d2/df/f10 [0,4194304] 0 2024-03-22T10:42:32.107 INFO:tasks.workunit.client.0.smithi115.stdout:4/66: chown d1/dc/l11 83585 1 2024-03-22T10:42:32.109 INFO:tasks.workunit.client.0.smithi115.stdout:2/51: truncate da/fb 1786742 0 2024-03-22T10:42:32.109 INFO:tasks.workunit.client.0.smithi115.stdout:3/43: mkdir d3/dd/df 0 2024-03-22T10:42:32.109 INFO:tasks.workunit.client.0.smithi115.stdout:2/52: readlink l5 0 2024-03-22T10:42:32.113 INFO:tasks.workunit.client.0.smithi115.stdout:8/197: rename d6/dd/d12/f15 to d6/dd/d35/d41/f42 0 2024-03-22T10:42:32.114 INFO:tasks.workunit.client.0.smithi115.stdout:3/44: write d3/fc [967624,130120] 0 2024-03-22T10:42:32.117 INFO:tasks.workunit.client.0.smithi115.stdout:3/45: readlink d3/l5 0 2024-03-22T10:42:32.117 INFO:tasks.workunit.client.0.smithi115.stdout:9/86: creat d6/d7/f17 x:0 0 0 2024-03-22T10:42:32.118 INFO:tasks.workunit.client.0.smithi115.stdout:4/67: unlink d1/fb 0 2024-03-22T10:42:32.128 INFO:tasks.workunit.client.0.smithi115.stdout:1/104: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:42:32.128 INFO:tasks.workunit.client.0.smithi115.stdout:7/71: truncate d2/d7/fb 2532774 0 2024-03-22T10:42:32.131 INFO:tasks.workunit.client.0.smithi115.stdout:8/198: mkdir d6/dd/d24/d3a/d43 0 2024-03-22T10:42:32.132 INFO:tasks.workunit.client.0.smithi115.stdout:7/72: chown f1 16 1 2024-03-22T10:42:32.133 INFO:tasks.workunit.client.0.smithi115.stdout:7/73: chown d2/d7/d11 375220 1 2024-03-22T10:42:32.134 INFO:tasks.workunit.client.0.smithi115.stdout:5/64: creat d2/df/d13/f16 x:0 0 0 2024-03-22T10:42:32.136 INFO:tasks.workunit.client.0.smithi115.stdout:5/65: chown d2/f4 20104425 1 2024-03-22T10:42:32.136 INFO:tasks.workunit.client.0.smithi115.stdout:9/87: symlink d6/d8/l18 0 2024-03-22T10:42:32.136 INFO:tasks.workunit.client.0.smithi115.stdout:8/199: chown d6/dd/d12/c2c 47250 1 2024-03-22T10:42:32.142 INFO:tasks.workunit.client.0.smithi115.stdout:6/19: sync 2024-03-22T10:42:32.156 INFO:tasks.workunit.client.0.smithi115.stdout:1/105: write d0/f14 [3868664,26664] 0 2024-03-22T10:42:32.156 INFO:tasks.workunit.client.0.smithi115.stdout:5/66: rmdir d2/df 39 2024-03-22T10:42:32.156 INFO:tasks.workunit.client.0.smithi115.stdout:9/88: mkdir d6/d7/d19 0 2024-03-22T10:42:32.158 INFO:tasks.workunit.client.0.smithi115.stdout:0/74: truncate d3/da/fe 1371307 0 2024-03-22T10:42:32.161 INFO:tasks.workunit.client.0.smithi115.stdout:3/46: dwrite d3/f8 [4194304,4194304] 0 2024-03-22T10:42:32.170 INFO:tasks.workunit.client.0.smithi115.stdout:6/20: dread f2 [0,4194304] 0 2024-03-22T10:42:32.171 INFO:tasks.workunit.client.0.smithi115.stdout:6/21: readlink l1 0 2024-03-22T10:42:32.180 INFO:tasks.workunit.client.0.smithi115.stdout:0/75: dwrite d3/d7/f8 [0,4194304] 0 2024-03-22T10:42:32.187 INFO:tasks.workunit.client.0.smithi115.stdout:6/22: dread d4/f5 [0,4194304] 0 2024-03-22T10:42:32.188 INFO:tasks.workunit.client.0.smithi115.stdout:6/23: readlink l1 0 2024-03-22T10:42:32.192 INFO:tasks.workunit.client.0.smithi115.stdout:3/47: dwrite d3/f8 [8388608,4194304] 0 2024-03-22T10:42:32.216 INFO:tasks.workunit.client.0.smithi115.stdout:9/89: creat d6/d8/f1a x:0 0 0 2024-03-22T10:42:32.217 INFO:tasks.workunit.client.0.smithi115.stdout:9/90: stat d6/d8/ce 0 2024-03-22T10:42:32.219 INFO:tasks.workunit.client.0.smithi115.stdout:2/53: dread da/fb [0,4194304] 0 2024-03-22T10:42:32.219 INFO:tasks.workunit.client.0.smithi115.stdout:9/91: write d6/f12 [2277102,8749] 0 2024-03-22T10:42:32.220 INFO:tasks.workunit.client.0.smithi115.stdout:8/200: creat d6/f44 x:0 0 0 2024-03-22T10:42:32.220 INFO:tasks.workunit.client.0.smithi115.stdout:5/67: mknod d2/df/d13/c17 0 2024-03-22T10:42:32.221 INFO:tasks.workunit.client.0.smithi115.stdout:1/106: truncate d0/f1 1222722 0 2024-03-22T10:42:32.223 INFO:tasks.workunit.client.0.smithi115.stdout:1/107: chown d0/d8 52 1 2024-03-22T10:42:32.224 INFO:tasks.workunit.client.0.smithi115.stdout:8/201: write d6/dd/f2e [1743008,107322] 0 2024-03-22T10:42:32.224 INFO:tasks.workunit.client.0.smithi115.stdout:9/92: truncate d6/d8/f1a 397463 0 2024-03-22T10:42:32.225 INFO:tasks.workunit.client.0.smithi115.stdout:6/24: symlink d4/l6 0 2024-03-22T10:42:32.227 INFO:tasks.workunit.client.0.smithi115.stdout:8/202: truncate d6/dd/d24/d3a/f3c 319472 0 2024-03-22T10:42:32.233 INFO:tasks.workunit.client.0.smithi115.stdout:5/68: creat d2/df/d13/f18 x:0 0 0 2024-03-22T10:42:32.236 INFO:tasks.workunit.client.0.smithi115.stdout:9/93: mknod d6/c1b 0 2024-03-22T10:42:32.239 INFO:tasks.workunit.client.0.smithi115.stdout:3/48: rmdir d3/dd/df 0 2024-03-22T10:42:32.239 INFO:tasks.workunit.client.0.smithi115.stdout:6/25: creat d4/f7 x:0 0 0 2024-03-22T10:42:32.241 INFO:tasks.workunit.client.0.smithi115.stdout:8/203: unlink d6/dd/f3e 0 2024-03-22T10:42:32.243 INFO:tasks.workunit.client.0.smithi115.stdout:0/76: getdents d3 0 2024-03-22T10:42:32.249 INFO:tasks.workunit.client.0.smithi115.stdout:9/94: rename d6/fc to d6/d8/f1c 0 2024-03-22T10:42:32.255 INFO:tasks.workunit.client.0.smithi115.stdout:8/204: rmdir d6/dd/d12 39 2024-03-22T10:42:32.257 INFO:tasks.workunit.client.0.smithi115.stdout:0/77: mkdir d3/d4/d14 0 2024-03-22T10:42:32.258 INFO:tasks.workunit.client.0.smithi115.stdout:3/49: dwrite d3/fc [0,4194304] 0 2024-03-22T10:42:32.261 INFO:tasks.workunit.client.0.smithi115.stdout:3/50: chown d3/fc 52906 1 2024-03-22T10:42:32.261 INFO:tasks.workunit.client.0.smithi115.stdout:8/205: fdatasync d6/f20 0 2024-03-22T10:42:32.266 INFO:tasks.workunit.client.0.smithi115.stdout:8/206: write d6/dd/d24/f3f [640932,42178] 0 2024-03-22T10:42:32.268 INFO:tasks.workunit.client.0.smithi115.stdout:3/51: truncate d3/fb 2772329 0 2024-03-22T10:42:32.271 INFO:tasks.workunit.client.0.smithi115.stdout:8/207: creat d6/dd/d24/d3a/f45 x:0 0 0 2024-03-22T10:42:32.277 INFO:tasks.workunit.client.0.smithi115.stdout:4/68: sync 2024-03-22T10:42:32.285 INFO:tasks.workunit.client.0.smithi115.stdout:9/95: dwrite d6/d8/f1c [0,4194304] 0 2024-03-22T10:42:32.287 INFO:tasks.workunit.client.0.smithi115.stdout:0/78: dwrite d3/d7/fd [0,4194304] 0 2024-03-22T10:42:32.291 INFO:tasks.workunit.client.0.smithi115.stdout:7/74: truncate f1 6650877 0 2024-03-22T10:42:32.300 INFO:tasks.workunit.client.0.smithi115.stdout:7/75: chown d2/l4 76946358 1 2024-03-22T10:42:32.303 INFO:tasks.workunit.client.0.smithi115.stdout:5/69: rmdir d2/df/d13 39 2024-03-22T10:42:32.304 INFO:tasks.workunit.client.0.smithi115.stdout:6/26: rmdir d4 39 2024-03-22T10:42:32.308 INFO:tasks.workunit.client.0.smithi115.stdout:8/208: dwrite d6/f23 [0,4194304] 0 2024-03-22T10:42:32.315 INFO:tasks.workunit.client.0.smithi115.stdout:5/70: write d2/f4 [678905,123357] 0 2024-03-22T10:42:32.318 INFO:tasks.workunit.client.0.smithi115.stdout:7/76: mknod d2/c15 0 2024-03-22T10:42:32.320 INFO:tasks.workunit.client.0.smithi115.stdout:4/69: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:32.326 INFO:tasks.workunit.client.0.smithi115.stdout:0/79: dread d3/d4/f11 [0,4194304] 0 2024-03-22T10:42:32.328 INFO:tasks.workunit.client.0.smithi115.stdout:9/96: dwrite d6/d8/f1c [0,4194304] 0 2024-03-22T10:42:32.328 INFO:tasks.workunit.client.0.smithi115.stdout:3/52: unlink d3/fb 0 2024-03-22T10:42:32.333 INFO:tasks.workunit.client.0.smithi115.stdout:6/27: rename l1 to d4/l8 0 2024-03-22T10:42:32.335 INFO:tasks.workunit.client.0.smithi115.stdout:2/54: dwrite da/fb [0,4194304] 0 2024-03-22T10:42:32.337 INFO:tasks.workunit.client.0.smithi115.stdout:7/77: creat d2/d7/d12/d10/f16 x:0 0 0 2024-03-22T10:42:32.337 INFO:tasks.workunit.client.0.smithi115.stdout:4/70: readlink d1/l3 0 2024-03-22T10:42:32.338 INFO:tasks.workunit.client.0.smithi115.stdout:1/108: write d0/f1 [1581698,125149] 0 2024-03-22T10:42:32.343 INFO:tasks.workunit.client.0.smithi115.stdout:8/209: truncate d6/dd/d12/f3b 753725 0 2024-03-22T10:42:32.345 INFO:tasks.workunit.client.0.smithi115.stdout:1/109: write d0/d8/db/f18 [1176487,38072] 0 2024-03-22T10:42:32.345 INFO:tasks.workunit.client.0.smithi115.stdout:2/55: write da/fb [2951795,13094] 0 2024-03-22T10:42:32.346 INFO:tasks.workunit.client.0.smithi115.stdout:3/53: unlink d3/l5 0 2024-03-22T10:42:32.348 INFO:tasks.workunit.client.0.smithi115.stdout:6/28: unlink f2 0 2024-03-22T10:42:32.358 INFO:tasks.workunit.client.0.smithi115.stdout:9/97: unlink d6/d7/c9 0 2024-03-22T10:42:32.359 INFO:tasks.workunit.client.0.smithi115.stdout:4/71: rename d1/fa to d1/d16/f18 0 2024-03-22T10:42:32.366 INFO:tasks.workunit.client.0.smithi115.stdout:7/78: creat d2/d7/d12/d10/f17 x:0 0 0 2024-03-22T10:42:32.369 INFO:tasks.workunit.client.0.smithi115.stdout:1/110: rmdir d0/d8/de/d1a 39 2024-03-22T10:42:32.372 INFO:tasks.workunit.client.0.smithi115.stdout:2/56: creat da/fc x:0 0 0 2024-03-22T10:42:32.373 INFO:tasks.workunit.client.0.smithi115.stdout:6/29: fsync d4/f5 0 2024-03-22T10:42:32.376 INFO:tasks.workunit.client.0.smithi115.stdout:9/98: mknod d6/d7/c1d 0 2024-03-22T10:42:32.383 INFO:tasks.workunit.client.0.smithi115.stdout:6/30: mknod d4/c9 0 2024-03-22T10:42:32.385 INFO:tasks.workunit.client.0.smithi115.stdout:0/80: dwrite d3/da/fe [0,4194304] 0 2024-03-22T10:42:32.387 INFO:tasks.workunit.client.0.smithi115.stdout:9/99: creat d6/d7/f1e x:0 0 0 2024-03-22T10:42:32.391 INFO:tasks.workunit.client.0.smithi115.stdout:8/210: getdents d6/dd/d24 0 2024-03-22T10:42:32.395 INFO:tasks.workunit.client.0.smithi115.stdout:3/54: getdents d3/dd 0 2024-03-22T10:42:32.402 INFO:tasks.workunit.client.0.smithi115.stdout:4/72: getdents d1 0 2024-03-22T10:42:32.406 INFO:tasks.workunit.client.0.smithi115.stdout:9/100: symlink d6/d7/d19/l1f 0 2024-03-22T10:42:32.410 INFO:tasks.workunit.client.0.smithi115.stdout:4/73: unlink d1/d16/f18 0 2024-03-22T10:42:32.414 INFO:tasks.workunit.client.0.smithi115.stdout:8/211: mkdir d6/dd/d46 0 2024-03-22T10:42:32.426 INFO:tasks.workunit.client.0.smithi115.stdout:4/74: getdents d1/dc 0 2024-03-22T10:42:32.430 INFO:tasks.workunit.client.0.smithi115.stdout:8/212: rmdir d6 39 2024-03-22T10:42:32.435 INFO:tasks.workunit.client.0.smithi115.stdout:7/79: dwrite d2/d7/f13 [4194304,4194304] 0 2024-03-22T10:42:32.444 INFO:tasks.workunit.client.0.smithi115.stdout:0/81: dwrite d3/da/fe [0,4194304] 0 2024-03-22T10:42:32.447 INFO:tasks.workunit.client.0.smithi115.stdout:6/31: dwrite d4/f7 [0,4194304] 0 2024-03-22T10:42:32.450 INFO:tasks.workunit.client.0.smithi115.stdout:8/213: write d6/fc [2720417,37375] 0 2024-03-22T10:42:32.451 INFO:tasks.workunit.client.0.smithi115.stdout:4/75: fdatasync d1/f7 0 2024-03-22T10:42:32.453 INFO:tasks.workunit.client.0.smithi115.stdout:6/32: readlink d4/l6 0 2024-03-22T10:42:32.454 INFO:tasks.workunit.client.0.smithi115.stdout:9/101: dread f3 [0,4194304] 0 2024-03-22T10:42:32.457 INFO:tasks.workunit.client.0.smithi115.stdout:6/33: rmdir d4 39 2024-03-22T10:42:32.458 INFO:tasks.workunit.client.0.smithi115.stdout:4/76: read - d1/fd zero size 2024-03-22T10:42:32.502 INFO:tasks.workunit.client.0.smithi115.stdout:9/102: write d6/d8/f1c [153655,6922] 0 2024-03-22T10:42:32.503 INFO:tasks.workunit.client.0.smithi115.stdout:0/82: rename d3/d7/f8 to d3/d4/d14/f15 0 2024-03-22T10:42:32.505 INFO:tasks.workunit.client.0.smithi115.stdout:3/55: dwrite d3/fe [0,4194304] 0 2024-03-22T10:42:32.508 INFO:tasks.workunit.client.0.smithi115.stdout:0/83: chown d3/d4/cf 4167262 1 2024-03-22T10:42:32.511 INFO:tasks.workunit.client.0.smithi115.stdout:6/34: link d4/l8 d4/la 0 2024-03-22T10:42:32.516 INFO:tasks.workunit.client.0.smithi115.stdout:6/35: chown d4 126791 1 2024-03-22T10:42:32.517 INFO:tasks.workunit.client.0.smithi115.stdout:9/103: dread d6/d8/f1a [0,4194304] 0 2024-03-22T10:42:32.524 INFO:tasks.workunit.client.0.smithi115.stdout:6/36: fsync d4/f5 0 2024-03-22T10:42:32.525 INFO:tasks.workunit.client.0.smithi115.stdout:3/56: write d3/f7 [538326,117607] 0 2024-03-22T10:42:32.525 INFO:tasks.workunit.client.0.smithi115.stdout:7/80: dwrite d2/d7/d12/d10/f17 [0,4194304] 0 2024-03-22T10:42:32.525 INFO:tasks.workunit.client.0.smithi115.stdout:9/104: mkdir d6/d7/d20 0 2024-03-22T10:42:32.530 INFO:tasks.workunit.client.0.smithi115.stdout:6/37: write f3 [1851179,37709] 0 2024-03-22T10:42:32.531 INFO:tasks.workunit.client.0.smithi115.stdout:7/81: readlink d2/l4 0 2024-03-22T10:42:32.533 INFO:tasks.workunit.client.0.smithi115.stdout:9/105: chown d6/fa 62059949 1 2024-03-22T10:42:32.536 INFO:tasks.workunit.client.0.smithi115.stdout:8/214: dwrite d6/dd/d12/f3b [0,4194304] 0 2024-03-22T10:42:32.537 INFO:tasks.workunit.client.0.smithi115.stdout:6/38: read d4/f5 [2642175,106696] 0 2024-03-22T10:42:32.537 INFO:tasks.workunit.client.0.smithi115.stdout:9/106: chown d6/d7/d19 11834 1 2024-03-22T10:42:32.539 INFO:tasks.workunit.client.0.smithi115.stdout:9/107: read f5 [6018553,95685] 0 2024-03-22T10:42:32.540 INFO:tasks.workunit.client.0.smithi115.stdout:3/57: dread d3/f7 [0,4194304] 0 2024-03-22T10:42:32.544 INFO:tasks.workunit.client.0.smithi115.stdout:9/108: truncate d6/fa 1014610 0 2024-03-22T10:42:32.544 INFO:tasks.workunit.client.0.smithi115.stdout:0/84: dwrite d3/fc [0,4194304] 0 2024-03-22T10:42:32.618 INFO:tasks.workunit.client.0.smithi115.stdout:6/39: mknod d4/cb 0 2024-03-22T10:42:32.620 INFO:tasks.workunit.client.0.smithi115.stdout:8/215: rmdir d6/dd/d35 39 2024-03-22T10:42:32.621 INFO:tasks.workunit.client.0.smithi115.stdout:9/109: creat d6/d7/d19/f21 x:0 0 0 2024-03-22T10:42:32.623 INFO:tasks.workunit.client.0.smithi115.stdout:9/110: chown d6/l15 7 1 2024-03-22T10:42:32.625 INFO:tasks.workunit.client.0.smithi115.stdout:9/111: readlink d6/d8/l13 0 2024-03-22T10:42:32.629 INFO:tasks.workunit.client.0.smithi115.stdout:8/216: symlink d6/dd/d24/l47 0 2024-03-22T10:42:32.631 INFO:tasks.workunit.client.0.smithi115.stdout:9/112: mkdir d6/d7/d20/d22 0 2024-03-22T10:42:32.633 INFO:tasks.workunit.client.0.smithi115.stdout:7/82: dread d2/d7/fb [0,4194304] 0 2024-03-22T10:42:32.634 INFO:tasks.workunit.client.0.smithi115.stdout:9/113: write d6/f12 [224082,4956] 0 2024-03-22T10:42:32.636 INFO:tasks.workunit.client.0.smithi115.stdout:8/217: rmdir d6/dd/d24/d3a/d43 0 2024-03-22T10:42:32.638 INFO:tasks.workunit.client.0.smithi115.stdout:7/83: symlink d2/d7/l18 0 2024-03-22T10:42:32.640 INFO:tasks.workunit.client.0.smithi115.stdout:9/114: write f1 [954489,14203] 0 2024-03-22T10:42:32.640 INFO:tasks.workunit.client.0.smithi115.stdout:0/85: truncate d3/d4/d14/f15 2236896 0 2024-03-22T10:42:32.644 INFO:tasks.workunit.client.0.smithi115.stdout:0/86: fdatasync d3/d4/f11 0 2024-03-22T10:42:32.646 INFO:tasks.workunit.client.0.smithi115.stdout:6/40: dwrite d4/f7 [0,4194304] 0 2024-03-22T10:42:32.648 INFO:tasks.workunit.client.0.smithi115.stdout:0/87: write d3/d4/f11 [2121513,68046] 0 2024-03-22T10:42:32.649 INFO:tasks.workunit.client.0.smithi115.stdout:8/218: getdents d6/dd/d12/d31 0 2024-03-22T10:42:32.652 INFO:tasks.workunit.client.0.smithi115.stdout:0/88: mknod d3/d4/c16 0 2024-03-22T10:42:32.653 INFO:tasks.workunit.client.0.smithi115.stdout:3/58: dwrite d3/f9 [0,4194304] 0 2024-03-22T10:42:32.655 INFO:tasks.workunit.client.0.smithi115.stdout:8/219: symlink d6/dd/d24/d3a/l48 0 2024-03-22T10:42:32.658 INFO:tasks.workunit.client.0.smithi115.stdout:3/59: rmdir d3 39 2024-03-22T10:42:32.659 INFO:tasks.workunit.client.0.smithi115.stdout:8/220: dread - d6/f44 zero size 2024-03-22T10:42:32.664 INFO:tasks.workunit.client.0.smithi115.stdout:0/89: rename d3/da/fe to d3/f17 0 2024-03-22T10:42:32.680 INFO:tasks.workunit.client.0.smithi115.stdout:7/84: dwrite d2/d7/d12/d10/f16 [0,4194304] 0 2024-03-22T10:42:32.683 INFO:tasks.workunit.client.0.smithi115.stdout:9/115: dwrite d6/f12 [0,4194304] 0 2024-03-22T10:42:32.683 INFO:tasks.workunit.client.0.smithi115.stdout:0/90: read d3/fc [2113600,2079] 0 2024-03-22T10:42:32.689 INFO:tasks.workunit.client.0.smithi115.stdout:9/116: read f5 [1014972,11821] 0 2024-03-22T10:42:32.690 INFO:tasks.workunit.client.0.smithi115.stdout:7/85: mkdir d2/d7/d12/d10/d19 0 2024-03-22T10:42:32.692 INFO:tasks.workunit.client.0.smithi115.stdout:6/41: dwrite f3 [0,4194304] 0 2024-03-22T10:42:32.698 INFO:tasks.workunit.client.0.smithi115.stdout:6/42: write d4/f5 [4160275,76955] 0 2024-03-22T10:42:32.698 INFO:tasks.workunit.client.0.smithi115.stdout:9/117: mkdir d6/d7/d20/d22/d23 0 2024-03-22T10:42:32.699 INFO:tasks.workunit.client.0.smithi115.stdout:8/221: dread d6/f20 [0,4194304] 0 2024-03-22T10:42:32.700 INFO:tasks.workunit.client.0.smithi115.stdout:3/60: dwrite d3/fc [0,4194304] 0 2024-03-22T10:42:32.702 INFO:tasks.workunit.client.0.smithi115.stdout:0/91: truncate d3/f17 2897733 0 2024-03-22T10:42:32.702 INFO:tasks.workunit.client.0.smithi115.stdout:7/86: creat d2/d7/d12/d10/d19/f1a x:0 0 0 2024-03-22T10:42:32.703 INFO:tasks.workunit.client.0.smithi115.stdout:8/222: read - d6/f44 zero size 2024-03-22T10:42:32.708 INFO:tasks.workunit.client.0.smithi115.stdout:6/43: mknod d4/cc 0 2024-03-22T10:42:32.714 INFO:tasks.workunit.client.0.smithi115.stdout:8/223: mkdir d6/d49 0 2024-03-22T10:42:32.718 INFO:tasks.workunit.client.0.smithi115.stdout:8/224: getdents d6/d49 0 2024-03-22T10:42:32.728 INFO:tasks.workunit.client.0.smithi115.stdout:5/71: sync 2024-03-22T10:42:32.729 INFO:tasks.workunit.client.0.smithi115.stdout:0/92: dread d3/d7/fd [0,4194304] 0 2024-03-22T10:42:32.732 INFO:tasks.workunit.client.0.smithi115.stdout:9/118: write f0 [1003663,53015] 0 2024-03-22T10:42:32.739 INFO:tasks.workunit.client.0.smithi115.stdout:7/87: dwrite d2/d7/d12/d10/f16 [0,4194304] 0 2024-03-22T10:42:32.741 INFO:tasks.workunit.client.0.smithi115.stdout:6/44: dwrite f3 [0,4194304] 0 2024-03-22T10:42:32.741 INFO:tasks.workunit.client.0.smithi115.stdout:9/119: creat d6/d8/f24 x:0 0 0 2024-03-22T10:42:32.751 INFO:tasks.workunit.client.0.smithi115.stdout:8/225: dread d6/dd/d12/f3b [0,4194304] 0 2024-03-22T10:42:32.753 INFO:tasks.workunit.client.0.smithi115.stdout:8/226: stat d6/fc 0 2024-03-22T10:42:32.754 INFO:tasks.workunit.client.0.smithi115.stdout:9/120: creat d6/d7/d19/f25 x:0 0 0 2024-03-22T10:42:32.755 INFO:tasks.workunit.client.0.smithi115.stdout:5/72: getdents d2/df/d13 0 2024-03-22T10:42:32.755 INFO:tasks.workunit.client.0.smithi115.stdout:8/227: fsync d6/f44 0 2024-03-22T10:42:32.756 INFO:tasks.workunit.client.0.smithi115.stdout:0/93: truncate d3/d4/d14/f15 2619851 0 2024-03-22T10:42:32.756 INFO:tasks.workunit.client.0.smithi115.stdout:9/121: write f3 [3391009,23577] 0 2024-03-22T10:42:32.757 INFO:tasks.workunit.client.0.smithi115.stdout:5/73: stat d2/c8 0 2024-03-22T10:42:32.757 INFO:tasks.workunit.client.0.smithi115.stdout:3/61: fsync d3/fc 0 2024-03-22T10:42:32.761 INFO:tasks.workunit.client.0.smithi115.stdout:3/62: stat d3/f8 0 2024-03-22T10:42:32.762 INFO:tasks.workunit.client.0.smithi115.stdout:8/228: chown d6/dd/d12/c2a 1319274 1 2024-03-22T10:42:32.765 INFO:tasks.workunit.client.0.smithi115.stdout:6/45: truncate d4/f5 445973 0 2024-03-22T10:42:32.768 INFO:tasks.workunit.client.0.smithi115.stdout:5/74: mknod d2/d15/c19 0 2024-03-22T10:42:32.769 INFO:tasks.workunit.client.0.smithi115.stdout:9/122: symlink d6/d7/d20/l26 0 2024-03-22T10:42:32.770 INFO:tasks.workunit.client.0.smithi115.stdout:6/46: symlink d4/ld 0 2024-03-22T10:42:32.773 INFO:tasks.workunit.client.0.smithi115.stdout:5/75: rmdir d2/d15 39 2024-03-22T10:42:32.821 INFO:tasks.workunit.client.0.smithi115.stdout:8/229: rename d6/dd/d35 to d6/dd/d24/d4a 0 2024-03-22T10:42:32.822 INFO:tasks.workunit.client.0.smithi115.stdout:6/47: chown d4/l8 0 1 2024-03-22T10:42:32.822 INFO:tasks.workunit.client.0.smithi115.stdout:7/88: dwrite d2/d7/d12/d10/f17 [4194304,4194304] 0 2024-03-22T10:42:32.822 INFO:tasks.workunit.client.0.smithi115.stdout:0/94: truncate d3/fc 3182718 0 2024-03-22T10:42:32.824 INFO:tasks.workunit.client.0.smithi115.stdout:6/48: chown d4/f5 1 1 2024-03-22T10:42:32.826 INFO:tasks.workunit.client.0.smithi115.stdout:5/76: truncate d2/df/f10 5250008 0 2024-03-22T10:42:32.826 INFO:tasks.workunit.client.0.smithi115.stdout:9/123: getdents d6/d7/d20 0 2024-03-22T10:42:32.827 INFO:tasks.workunit.client.0.smithi115.stdout:0/95: read d3/d7/fd [1881137,55383] 0 2024-03-22T10:42:32.827 INFO:tasks.workunit.client.0.smithi115.stdout:6/49: chown d4/cb 221853 1 2024-03-22T10:42:32.829 INFO:tasks.workunit.client.0.smithi115.stdout:9/124: write f3 [8447354,16019] 0 2024-03-22T10:42:32.833 INFO:tasks.workunit.client.0.smithi115.stdout:8/230: truncate d6/dd/f11 2127601 0 2024-03-22T10:42:32.835 INFO:tasks.workunit.client.0.smithi115.stdout:6/50: symlink d4/le 0 2024-03-22T10:42:32.837 INFO:tasks.workunit.client.0.smithi115.stdout:9/125: creat d6/d7/d19/f27 x:0 0 0 2024-03-22T10:42:32.837 INFO:tasks.workunit.client.0.smithi115.stdout:8/231: dread d6/dd/d24/d3a/f3c [0,4194304] 0 2024-03-22T10:42:32.838 INFO:tasks.workunit.client.0.smithi115.stdout:6/51: fsync d4/f5 0 2024-03-22T10:42:32.840 INFO:tasks.workunit.client.0.smithi115.stdout:0/96: rename d3/d7/fd to d3/f18 0 2024-03-22T10:42:32.841 INFO:tasks.workunit.client.0.smithi115.stdout:5/77: rmdir d2 39 2024-03-22T10:42:32.842 INFO:tasks.workunit.client.0.smithi115.stdout:7/89: dread d2/d7/d12/d10/f16 [0,4194304] 0 2024-03-22T10:42:32.844 INFO:tasks.workunit.client.0.smithi115.stdout:6/52: mknod d4/cf 0 2024-03-22T10:42:32.847 INFO:tasks.workunit.client.0.smithi115.stdout:8/232: creat d6/dd/d24/d4a/d41/f4b x:0 0 0 2024-03-22T10:42:32.850 INFO:tasks.workunit.client.0.smithi115.stdout:5/78: write d2/df/d13/f18 [855470,88521] 0 2024-03-22T10:42:32.852 INFO:tasks.workunit.client.0.smithi115.stdout:8/233: write d6/dd/d12/f1d [222540,88549] 0 2024-03-22T10:42:32.863 INFO:tasks.workunit.client.0.smithi115.stdout:0/97: write d3/f17 [2600616,75469] 0 2024-03-22T10:42:32.866 INFO:tasks.workunit.client.0.smithi115.stdout:9/126: dwrite d6/d7/f17 [0,4194304] 0 2024-03-22T10:42:32.872 INFO:tasks.workunit.client.0.smithi115.stdout:9/127: chown d6/d8/fd 187183747 1 2024-03-22T10:42:32.872 INFO:tasks.workunit.client.0.smithi115.stdout:8/234: truncate d6/dd/d12/f3b 2995007 0 2024-03-22T10:42:32.876 INFO:tasks.workunit.client.0.smithi115.stdout:7/90: dwrite d2/f5 [0,4194304] 0 2024-03-22T10:42:32.878 INFO:tasks.workunit.client.0.smithi115.stdout:6/53: dwrite f3 [0,4194304] 0 2024-03-22T10:42:32.884 INFO:tasks.workunit.client.0.smithi115.stdout:5/79: dread d2/fc [0,4194304] 0 2024-03-22T10:42:32.884 INFO:tasks.workunit.client.0.smithi115.stdout:0/98: write d3/d4/d14/f15 [2239909,8700] 0 2024-03-22T10:42:32.884 INFO:tasks.workunit.client.0.smithi115.stdout:8/235: rename d6/dd/d24 to d6/dd/d24/d4a/d41/d4c 22 2024-03-22T10:42:32.884 INFO:tasks.workunit.client.0.smithi115.stdout:7/91: dread - d2/d7/d12/d10/d19/f1a zero size 2024-03-22T10:42:32.884 INFO:tasks.workunit.client.0.smithi115.stdout:6/54: truncate d4/f5 4840570 0 2024-03-22T10:42:32.886 INFO:tasks.workunit.client.0.smithi115.stdout:0/99: write d3/d4/f11 [1834966,23221] 0 2024-03-22T10:42:32.887 INFO:tasks.workunit.client.0.smithi115.stdout:5/80: write d2/fc [2219963,70611] 0 2024-03-22T10:42:32.893 INFO:tasks.workunit.client.0.smithi115.stdout:7/92: creat d2/d7/d12/f1b x:0 0 0 2024-03-22T10:42:32.893 INFO:tasks.workunit.client.0.smithi115.stdout:8/236: creat d6/dd/d24/d4a/f4d x:0 0 0 2024-03-22T10:42:32.894 INFO:tasks.workunit.client.0.smithi115.stdout:9/128: link d6/d8/l13 d6/d7/d20/d22/d23/l28 0 2024-03-22T10:42:32.894 INFO:tasks.workunit.client.0.smithi115.stdout:0/100: creat d3/d4/d14/f19 x:0 0 0 2024-03-22T10:42:32.897 INFO:tasks.workunit.client.0.smithi115.stdout:8/237: dread d6/f16 [0,4194304] 0 2024-03-22T10:42:32.903 INFO:tasks.workunit.client.0.smithi115.stdout:5/81: mknod d2/d15/c1a 0 2024-03-22T10:42:32.905 INFO:tasks.workunit.client.0.smithi115.stdout:8/238: creat d6/dd/d24/d4a/f4e x:0 0 0 2024-03-22T10:42:32.909 INFO:tasks.workunit.client.0.smithi115.stdout:0/101: creat d3/f1a x:0 0 0 2024-03-22T10:42:32.910 INFO:tasks.workunit.client.0.smithi115.stdout:8/239: write d6/fc [503297,45958] 0 2024-03-22T10:42:32.913 INFO:tasks.workunit.client.0.smithi115.stdout:5/82: write d2/df/d13/f16 [472170,82102] 0 2024-03-22T10:42:32.917 INFO:tasks.workunit.client.0.smithi115.stdout:8/240: chown d6/f17 0 1 2024-03-22T10:42:32.920 INFO:tasks.workunit.client.0.smithi115.stdout:0/102: fdatasync d3/d4/f11 0 2024-03-22T10:42:32.922 INFO:tasks.workunit.client.0.smithi115.stdout:6/55: dwrite f3 [0,4194304] 0 2024-03-22T10:42:32.923 INFO:tasks.workunit.client.0.smithi115.stdout:9/129: dwrite f0 [0,4194304] 0 2024-03-22T10:42:32.924 INFO:tasks.workunit.client.0.smithi115.stdout:1/111: sync 2024-03-22T10:42:32.924 INFO:tasks.workunit.client.0.smithi115.stdout:4/77: sync 2024-03-22T10:42:32.924 INFO:tasks.workunit.client.0.smithi115.stdout:2/57: sync 2024-03-22T10:42:32.924 INFO:tasks.workunit.client.0.smithi115.stdout:8/241: truncate d6/f16 918327 0 2024-03-22T10:42:32.926 INFO:tasks.workunit.client.0.smithi115.stdout:6/56: creat d4/f10 x:0 0 0 2024-03-22T10:42:32.927 INFO:tasks.workunit.client.0.smithi115.stdout:4/78: stat d1/dc/d17 0 2024-03-22T10:42:32.929 INFO:tasks.workunit.client.0.smithi115.stdout:2/58: dread - da/fc zero size 2024-03-22T10:42:32.931 INFO:tasks.workunit.client.0.smithi115.stdout:7/93: dwrite d2/d7/f13 [4194304,4194304] 0 2024-03-22T10:42:32.931 INFO:tasks.workunit.client.0.smithi115.stdout:1/112: symlink d0/d8/db/l21 0 2024-03-22T10:42:32.939 INFO:tasks.workunit.client.0.smithi115.stdout:7/94: stat d2/d7/d12/f1b 0 2024-03-22T10:42:32.941 INFO:tasks.workunit.client.0.smithi115.stdout:6/57: mkdir d4/d11 0 2024-03-22T10:42:32.942 INFO:tasks.workunit.client.0.smithi115.stdout:9/130: rmdir d6/d7/d20/d22 39 2024-03-22T10:42:32.943 INFO:tasks.workunit.client.0.smithi115.stdout:2/59: rename c7 to da/cd 0 2024-03-22T10:42:32.946 INFO:tasks.workunit.client.0.smithi115.stdout:1/113: fsync d0/f7 0 2024-03-22T10:42:32.952 INFO:tasks.workunit.client.0.smithi115.stdout:6/58: write f3 [1801035,111443] 0 2024-03-22T10:42:32.955 INFO:tasks.workunit.client.0.smithi115.stdout:2/60: creat da/fe x:0 0 0 2024-03-22T10:42:32.961 INFO:tasks.workunit.client.0.smithi115.stdout:2/61: dread - da/fc zero size 2024-03-22T10:42:32.963 INFO:tasks.workunit.client.0.smithi115.stdout:9/131: write d6/d7/d19/f21 [968383,38678] 0 2024-03-22T10:42:32.964 INFO:tasks.workunit.client.0.smithi115.stdout:0/103: dwrite d3/d4/d14/f19 [0,4194304] 0 2024-03-22T10:42:32.969 INFO:tasks.workunit.client.0.smithi115.stdout:6/59: mkdir d4/d11/d12 0 2024-03-22T10:42:32.970 INFO:tasks.workunit.client.0.smithi115.stdout:8/242: dwrite d6/f20 [0,4194304] 0 2024-03-22T10:42:32.971 INFO:tasks.workunit.client.0.smithi115.stdout:1/114: rmdir d0 39 2024-03-22T10:42:32.973 INFO:tasks.workunit.client.0.smithi115.stdout:0/104: rmdir d3 39 2024-03-22T10:42:32.976 INFO:tasks.workunit.client.0.smithi115.stdout:6/60: chown d4/le 30041577 1 2024-03-22T10:42:32.982 INFO:tasks.workunit.client.0.smithi115.stdout:6/61: mkdir d4/d13 0 2024-03-22T10:42:32.984 INFO:tasks.workunit.client.0.smithi115.stdout:6/62: fdatasync d4/f5 0 2024-03-22T10:42:32.985 INFO:tasks.workunit.client.0.smithi115.stdout:4/79: dwrite d1/f10 [0,4194304] 0 2024-03-22T10:42:32.986 INFO:tasks.workunit.client.0.smithi115.stdout:7/95: dread d2/d7/f13 [0,4194304] 0 2024-03-22T10:42:32.987 INFO:tasks.workunit.client.0.smithi115.stdout:0/105: readlink d3/l9 0 2024-03-22T10:42:32.990 INFO:tasks.workunit.client.0.smithi115.stdout:6/63: write d4/f5 [4021036,15466] 0 2024-03-22T10:42:32.990 INFO:tasks.workunit.client.0.smithi115.stdout:7/96: read d2/d7/fb [314085,75274] 0 2024-03-22T10:42:32.991 INFO:tasks.workunit.client.0.smithi115.stdout:4/80: write d1/f7 [4853433,76188] 0 2024-03-22T10:42:32.994 INFO:tasks.workunit.client.0.smithi115.stdout:6/64: write d4/f5 [337085,90182] 0 2024-03-22T10:42:32.996 INFO:tasks.workunit.client.0.smithi115.stdout:1/115: rename d0/d12/d1b to d0/d22 0 2024-03-22T10:42:32.997 INFO:tasks.workunit.client.0.smithi115.stdout:9/132: getdents d6/d8 0 2024-03-22T10:42:32.998 INFO:tasks.workunit.client.0.smithi115.stdout:4/81: symlink d1/dc/l19 0 2024-03-22T10:42:32.999 INFO:tasks.workunit.client.0.smithi115.stdout:1/116: readlink d0/d8/db/l21 0 2024-03-22T10:42:33.000 INFO:tasks.workunit.client.0.smithi115.stdout:0/106: mkdir d3/d7/d1b 0 2024-03-22T10:42:33.001 INFO:tasks.workunit.client.0.smithi115.stdout:2/62: dread da/fb [0,4194304] 0 2024-03-22T10:42:33.002 INFO:tasks.workunit.client.0.smithi115.stdout:8/243: dread d6/dd/d12/f3b [0,4194304] 0 2024-03-22T10:42:33.004 INFO:tasks.workunit.client.0.smithi115.stdout:9/133: write d6/d7/f1e [1004830,28762] 0 2024-03-22T10:42:33.004 INFO:tasks.workunit.client.0.smithi115.stdout:6/65: symlink d4/d13/l14 0 2024-03-22T10:42:33.009 INFO:tasks.workunit.client.0.smithi115.stdout:1/117: creat d0/d12/d17/f23 x:0 0 0 2024-03-22T10:42:33.012 INFO:tasks.workunit.client.0.smithi115.stdout:8/244: mknod d6/dd/d12/d31/c4f 0 2024-03-22T10:42:33.013 INFO:tasks.workunit.client.0.smithi115.stdout:1/118: write d0/f1d [1670279,78818] 0 2024-03-22T10:42:33.015 INFO:tasks.workunit.client.0.smithi115.stdout:4/82: dread d1/f7 [4194304,4194304] 0 2024-03-22T10:42:33.020 INFO:tasks.workunit.client.0.smithi115.stdout:2/63: getdents da 0 2024-03-22T10:42:33.022 INFO:tasks.workunit.client.0.smithi115.stdout:9/134: link d6/l10 d6/d7/d19/l29 0 2024-03-22T10:42:33.022 INFO:tasks.workunit.client.0.smithi115.stdout:3/63: sync 2024-03-22T10:42:33.022 INFO:tasks.workunit.client.0.smithi115.stdout:7/97: dwrite d2/d7/f13 [0,4194304] 0 2024-03-22T10:42:33.026 INFO:tasks.workunit.client.0.smithi115.stdout:4/83: truncate d1/f10 4195493 0 2024-03-22T10:42:33.028 INFO:tasks.workunit.client.0.smithi115.stdout:2/64: fdatasync da/fe 0 2024-03-22T10:42:33.028 INFO:tasks.workunit.client.0.smithi115.stdout:6/66: getdents d4/d13 0 2024-03-22T10:42:33.028 INFO:tasks.workunit.client.0.smithi115.stdout:1/119: rename d0/fc to d0/d8/de/d1a/f24 0 2024-03-22T10:42:33.029 INFO:tasks.workunit.client.0.smithi115.stdout:9/135: chown d6/d8/l18 403616 1 2024-03-22T10:42:33.031 INFO:tasks.workunit.client.0.smithi115.stdout:4/84: chown d1/dc/ce 4002 1 2024-03-22T10:42:33.036 INFO:tasks.workunit.client.0.smithi115.stdout:0/107: dwrite d3/d4/d14/f19 [0,4194304] 0 2024-03-22T10:42:33.036 INFO:tasks.workunit.client.0.smithi115.stdout:8/245: link l3 d6/dd/d24/d4a/l50 0 2024-03-22T10:42:33.037 INFO:tasks.workunit.client.0.smithi115.stdout:6/67: mknod d4/d11/c15 0 2024-03-22T10:42:33.039 INFO:tasks.workunit.client.0.smithi115.stdout:6/68: fdatasync f3 0 2024-03-22T10:42:33.040 INFO:tasks.workunit.client.0.smithi115.stdout:4/85: write d1/f9 [324535,116058] 0 2024-03-22T10:42:33.041 INFO:tasks.workunit.client.0.smithi115.stdout:2/65: creat da/ff x:0 0 0 2024-03-22T10:42:33.045 INFO:tasks.workunit.client.0.smithi115.stdout:0/108: write d3/d4/f11 [342988,114152] 0 2024-03-22T10:42:33.048 INFO:tasks.workunit.client.0.smithi115.stdout:8/246: rmdir d6/dd/d46 0 2024-03-22T10:42:33.050 INFO:tasks.workunit.client.0.smithi115.stdout:6/69: mknod d4/d13/c16 0 2024-03-22T10:42:33.053 INFO:tasks.workunit.client.0.smithi115.stdout:4/86: write d1/f7 [3188638,95412] 0 2024-03-22T10:42:33.054 INFO:tasks.workunit.client.0.smithi115.stdout:8/247: chown d6/c8 26638 1 2024-03-22T10:42:33.057 INFO:tasks.workunit.client.0.smithi115.stdout:6/70: symlink d4/l17 0 2024-03-22T10:42:33.059 INFO:tasks.workunit.client.0.smithi115.stdout:0/109: mkdir d3/d7/d1c 0 2024-03-22T10:42:33.062 INFO:tasks.workunit.client.0.smithi115.stdout:8/248: fsync f5 0 2024-03-22T10:42:33.065 INFO:tasks.workunit.client.0.smithi115.stdout:6/71: write f3 [1257381,14212] 0 2024-03-22T10:42:33.065 INFO:tasks.workunit.client.0.smithi115.stdout:2/66: rmdir da 39 2024-03-22T10:42:33.067 INFO:tasks.workunit.client.0.smithi115.stdout:7/98: dwrite d2/d7/d12/d10/d19/f1a [0,4194304] 0 2024-03-22T10:42:33.072 INFO:tasks.workunit.client.0.smithi115.stdout:1/120: dwrite d0/d8/f9 [0,4194304] 0 2024-03-22T10:42:33.073 INFO:tasks.workunit.client.0.smithi115.stdout:0/110: rename d3/f18 to d3/d7/f1d 0 2024-03-22T10:42:33.073 INFO:tasks.workunit.client.0.smithi115.stdout:6/72: write f3 [1964927,129456] 0 2024-03-22T10:42:33.074 INFO:tasks.workunit.client.0.smithi115.stdout:9/136: dwrite d6/d7/d19/f21 [0,4194304] 0 2024-03-22T10:42:33.074 INFO:tasks.workunit.client.0.smithi115.stdout:7/99: chown d2/d7/fb 336641 1 2024-03-22T10:42:33.079 INFO:tasks.workunit.client.0.smithi115.stdout:9/137: chown d6/d8/ce 240735 1 2024-03-22T10:42:33.079 INFO:tasks.workunit.client.0.smithi115.stdout:2/67: write da/fc [959504,67067] 0 2024-03-22T10:42:33.084 INFO:tasks.workunit.client.0.smithi115.stdout:2/68: truncate da/ff 221367 0 2024-03-22T10:42:33.084 INFO:tasks.workunit.client.0.smithi115.stdout:7/100: symlink d2/d7/d12/l1c 0 2024-03-22T10:42:33.085 INFO:tasks.workunit.client.0.smithi115.stdout:6/73: symlink d4/l18 0 2024-03-22T10:42:33.090 INFO:tasks.workunit.client.0.smithi115.stdout:2/69: write da/fe [182043,74823] 0 2024-03-22T10:42:33.091 INFO:tasks.workunit.client.0.smithi115.stdout:1/121: mkdir d0/d8/db/d25 0 2024-03-22T10:42:33.095 INFO:tasks.workunit.client.0.smithi115.stdout:8/249: write d6/dd/d12/f3b [1011863,110841] 0 2024-03-22T10:42:33.097 INFO:tasks.workunit.client.0.smithi115.stdout:2/70: unlink da/fc 0 2024-03-22T10:42:33.097 INFO:tasks.workunit.client.0.smithi115.stdout:0/111: dread d3/fc [0,4194304] 0 2024-03-22T10:42:33.104 INFO:tasks.workunit.client.0.smithi115.stdout:2/71: mknod da/c10 0 2024-03-22T10:42:33.109 INFO:tasks.workunit.client.0.smithi115.stdout:7/101: dwrite d2/d7/d12/d10/d19/f1a [0,4194304] 0 2024-03-22T10:42:33.109 INFO:tasks.workunit.client.0.smithi115.stdout:1/122: creat d0/d8/de/f26 x:0 0 0 2024-03-22T10:42:33.113 INFO:tasks.workunit.client.0.smithi115.stdout:6/74: dwrite d4/f7 [0,4194304] 0 2024-03-22T10:42:33.113 INFO:tasks.workunit.client.0.smithi115.stdout:6/75: chown d4/d13/c16 1584530 1 2024-03-22T10:42:33.114 INFO:tasks.workunit.client.0.smithi115.stdout:2/72: write da/fb [1770203,100656] 0 2024-03-22T10:42:33.118 INFO:tasks.workunit.client.0.smithi115.stdout:1/123: creat d0/d8/db/f27 x:0 0 0 2024-03-22T10:42:33.119 INFO:tasks.workunit.client.0.smithi115.stdout:0/112: write d3/fc [824794,54268] 0 2024-03-22T10:42:33.122 INFO:tasks.workunit.client.0.smithi115.stdout:6/76: mknod d4/d13/c19 0 2024-03-22T10:42:33.122 INFO:tasks.workunit.client.0.smithi115.stdout:2/73: write da/fe [88337,44094] 0 2024-03-22T10:42:33.125 INFO:tasks.workunit.client.0.smithi115.stdout:6/77: write f3 [2506052,50289] 0 2024-03-22T10:42:33.126 INFO:tasks.workunit.client.0.smithi115.stdout:2/74: unlink da/cd 0 2024-03-22T10:42:33.133 INFO:tasks.workunit.client.0.smithi115.stdout:8/250: dwrite f5 [0,4194304] 0 2024-03-22T10:42:33.139 INFO:tasks.workunit.client.0.smithi115.stdout:7/102: dread d2/d7/d12/d10/f17 [4194304,4194304] 0 2024-03-22T10:42:33.140 INFO:tasks.workunit.client.0.smithi115.stdout:8/251: creat d6/dd/d24/d4a/f51 x:0 0 0 2024-03-22T10:42:33.145 INFO:tasks.workunit.client.0.smithi115.stdout:2/75: rmdir da 39 2024-03-22T10:42:33.149 INFO:tasks.workunit.client.0.smithi115.stdout:0/113: dwrite d3/fc [0,4194304] 0 2024-03-22T10:42:33.150 INFO:tasks.workunit.client.0.smithi115.stdout:1/124: dwrite d0/d12/d17/f23 [0,4194304] 0 2024-03-22T10:42:33.150 INFO:tasks.workunit.client.0.smithi115.stdout:6/78: dread d4/f7 [0,4194304] 0 2024-03-22T10:42:33.150 INFO:tasks.workunit.client.0.smithi115.stdout:0/114: fsync d3/f17 0 2024-03-22T10:42:33.161 INFO:tasks.workunit.client.0.smithi115.stdout:7/103: creat d2/d7/d12/d10/f1d x:0 0 0 2024-03-22T10:42:33.163 INFO:tasks.workunit.client.0.smithi115.stdout:0/115: rename d3/d4/d14/f19 to d3/d7/d1c/f1e 0 2024-03-22T10:42:33.167 INFO:tasks.workunit.client.0.smithi115.stdout:0/116: write d3/d4/d14/f15 [2283329,113841] 0 2024-03-22T10:42:33.170 INFO:tasks.workunit.client.0.smithi115.stdout:8/252: dwrite d6/dd/d24/d4a/f51 [0,4194304] 0 2024-03-22T10:42:33.175 INFO:tasks.workunit.client.0.smithi115.stdout:5/83: sync 2024-03-22T10:42:33.175 INFO:tasks.workunit.client.0.smithi115.stdout:2/76: dwrite da/fe [0,4194304] 0 2024-03-22T10:42:33.175 INFO:tasks.workunit.client.0.smithi115.stdout:1/125: unlink d0/d8/l11 0 2024-03-22T10:42:33.176 INFO:tasks.workunit.client.0.smithi115.stdout:2/77: read da/ff [216326,58486] 0 2024-03-22T10:42:33.177 INFO:tasks.workunit.client.0.smithi115.stdout:6/79: getdents d4/d11/d12 0 2024-03-22T10:42:33.178 INFO:tasks.workunit.client.0.smithi115.stdout:7/104: creat d2/d7/d12/d10/f1e x:0 0 0 2024-03-22T10:42:33.178 INFO:tasks.workunit.client.0.smithi115.stdout:0/117: unlink d3/f17 0 2024-03-22T10:42:33.179 INFO:tasks.workunit.client.0.smithi115.stdout:1/126: fdatasync d0/d8/de/d1a/f1e 0 2024-03-22T10:42:33.180 INFO:tasks.workunit.client.0.smithi115.stdout:8/253: mknod d6/dd/d12/d31/c52 0 2024-03-22T10:42:33.182 INFO:tasks.workunit.client.0.smithi115.stdout:6/80: write d4/f10 [58642,35147] 0 2024-03-22T10:42:33.182 INFO:tasks.workunit.client.0.smithi115.stdout:5/84: mkdir d2/d15/d1b 0 2024-03-22T10:42:33.185 INFO:tasks.workunit.client.0.smithi115.stdout:8/254: stat d6/fa 0 2024-03-22T10:42:33.187 INFO:tasks.workunit.client.0.smithi115.stdout:7/105: symlink d2/d7/l1f 0 2024-03-22T10:42:33.194 INFO:tasks.workunit.client.0.smithi115.stdout:1/127: rmdir d0 39 2024-03-22T10:42:33.197 INFO:tasks.workunit.client.0.smithi115.stdout:7/106: creat d2/d7/f20 x:0 0 0 2024-03-22T10:42:33.200 INFO:tasks.workunit.client.0.smithi115.stdout:8/255: dread d6/dd/d12/f37 [0,4194304] 0 2024-03-22T10:42:33.200 INFO:tasks.workunit.client.0.smithi115.stdout:5/85: dread d2/df/d13/f16 [0,4194304] 0 2024-03-22T10:42:33.201 INFO:tasks.workunit.client.0.smithi115.stdout:6/81: mknod d4/d11/d12/c1a 0 2024-03-22T10:42:33.202 INFO:tasks.workunit.client.0.smithi115.stdout:1/128: read - d0/d8/de/f26 zero size 2024-03-22T10:42:33.205 INFO:tasks.workunit.client.0.smithi115.stdout:7/107: creat d2/d7/d11/f21 x:0 0 0 2024-03-22T10:42:33.206 INFO:tasks.workunit.client.0.smithi115.stdout:8/256: symlink d6/dd/d24/l53 0 2024-03-22T10:42:33.207 INFO:tasks.workunit.client.0.smithi115.stdout:6/82: write d4/f5 [4915222,48789] 0 2024-03-22T10:42:33.207 INFO:tasks.workunit.client.0.smithi115.stdout:1/129: fdatasync d0/d8/de/f26 0 2024-03-22T10:42:33.209 INFO:tasks.workunit.client.0.smithi115.stdout:5/86: creat d2/d15/f1c x:0 0 0 2024-03-22T10:42:33.209 INFO:tasks.workunit.client.0.smithi115.stdout:0/118: dwrite d3/fc [4194304,4194304] 0 2024-03-22T10:42:33.210 INFO:tasks.workunit.client.0.smithi115.stdout:7/108: dread - d2/d7/d12/f1b zero size 2024-03-22T10:42:33.212 INFO:tasks.workunit.client.0.smithi115.stdout:1/130: chown d0/d8/db/l21 6 1 2024-03-22T10:42:33.213 INFO:tasks.workunit.client.0.smithi115.stdout:8/257: creat d6/dd/d24/d3a/f54 x:0 0 0 2024-03-22T10:42:33.214 INFO:tasks.workunit.client.0.smithi115.stdout:1/131: write d0/d8/db/f27 [822530,19981] 0 2024-03-22T10:42:33.214 INFO:tasks.workunit.client.0.smithi115.stdout:6/83: symlink d4/d11/d12/l1b 0 2024-03-22T10:42:33.216 INFO:tasks.workunit.client.0.smithi115.stdout:1/132: write d0/d8/db/f18 [993500,65778] 0 2024-03-22T10:42:33.218 INFO:tasks.workunit.client.0.smithi115.stdout:1/133: chown d0/f3 3 1 2024-03-22T10:42:33.222 INFO:tasks.workunit.client.0.smithi115.stdout:2/78: write da/fe [4608281,71036] 0 2024-03-22T10:42:33.229 INFO:tasks.workunit.client.0.smithi115.stdout:1/134: mknod d0/d8/db/c28 0 2024-03-22T10:42:33.230 INFO:tasks.workunit.client.0.smithi115.stdout:8/258: symlink d6/l55 0 2024-03-22T10:42:33.234 INFO:tasks.workunit.client.0.smithi115.stdout:0/119: dwrite d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:33.236 INFO:tasks.workunit.client.0.smithi115.stdout:8/259: write d6/dd/d24/d4a/f4d [756346,64105] 0 2024-03-22T10:42:33.237 INFO:tasks.workunit.client.0.smithi115.stdout:5/87: getdents d2 0 2024-03-22T10:42:33.239 INFO:tasks.workunit.client.0.smithi115.stdout:5/88: fsync d2/fc 0 2024-03-22T10:42:33.239 INFO:tasks.workunit.client.0.smithi115.stdout:7/109: getdents d2/d7/d12 0 2024-03-22T10:42:33.240 INFO:tasks.workunit.client.0.smithi115.stdout:7/110: chown d2 21365189 1 2024-03-22T10:42:33.242 INFO:tasks.workunit.client.0.smithi115.stdout:0/120: rmdir d3/d7 39 2024-03-22T10:42:33.244 INFO:tasks.workunit.client.0.smithi115.stdout:2/79: dread da/fb [0,4194304] 0 2024-03-22T10:42:33.245 INFO:tasks.workunit.client.0.smithi115.stdout:5/89: unlink d2/df/c14 0 2024-03-22T10:42:33.247 INFO:tasks.workunit.client.0.smithi115.stdout:5/90: fdatasync d2/f4 0 2024-03-22T10:42:33.248 INFO:tasks.workunit.client.0.smithi115.stdout:5/91: chown d2/df/d13 3 1 2024-03-22T10:42:33.256 INFO:tasks.workunit.client.0.smithi115.stdout:7/111: dread d2/d7/d12/d10/f16 [0,4194304] 0 2024-03-22T10:42:33.258 INFO:tasks.workunit.client.0.smithi115.stdout:2/80: dread da/fe [0,4194304] 0 2024-03-22T10:42:33.258 INFO:tasks.workunit.client.0.smithi115.stdout:6/84: truncate d4/f5 3382523 0 2024-03-22T10:42:33.260 INFO:tasks.workunit.client.0.smithi115.stdout:0/121: stat d3/d4/cf 0 2024-03-22T10:42:33.261 INFO:tasks.workunit.client.0.smithi115.stdout:7/112: mknod d2/d7/d11/c22 0 2024-03-22T10:42:33.262 INFO:tasks.workunit.client.0.smithi115.stdout:1/135: truncate d0/d8/db/f16 155075 0 2024-03-22T10:42:33.266 INFO:tasks.workunit.client.0.smithi115.stdout:2/81: write da/fe [4835147,59843] 0 2024-03-22T10:42:33.267 INFO:tasks.workunit.client.0.smithi115.stdout:8/260: truncate d6/dd/f33 1361218 0 2024-03-22T10:42:33.270 INFO:tasks.workunit.client.0.smithi115.stdout:7/113: mkdir d2/d7/d12/d10/d23 0 2024-03-22T10:42:33.271 INFO:tasks.workunit.client.0.smithi115.stdout:1/136: rename d0/d8/db/l15 to d0/d12/l29 0 2024-03-22T10:42:33.272 INFO:tasks.workunit.client.0.smithi115.stdout:6/85: write d4/f7 [1476398,112058] 0 2024-03-22T10:42:33.274 INFO:tasks.workunit.client.0.smithi115.stdout:8/261: mknod d6/dd/d12/d31/c56 0 2024-03-22T10:42:33.276 INFO:tasks.workunit.client.0.smithi115.stdout:7/114: chown d2/l4 392 1 2024-03-22T10:42:33.278 INFO:tasks.workunit.client.0.smithi115.stdout:5/92: truncate d2/df/d13/f16 250747 0 2024-03-22T10:42:33.279 INFO:tasks.workunit.client.0.smithi115.stdout:6/86: rmdir d4/d13 39 2024-03-22T10:42:33.283 INFO:tasks.workunit.client.0.smithi115.stdout:2/82: write da/ff [1179137,4948] 0 2024-03-22T10:42:33.283 INFO:tasks.workunit.client.0.smithi115.stdout:6/87: write d4/f7 [3999878,91507] 0 2024-03-22T10:42:33.284 INFO:tasks.workunit.client.0.smithi115.stdout:8/262: rmdir d6/dd/d12/d31 39 2024-03-22T10:42:33.285 INFO:tasks.workunit.client.0.smithi115.stdout:5/93: mknod d2/d15/c1d 0 2024-03-22T10:42:33.287 INFO:tasks.workunit.client.0.smithi115.stdout:1/137: dread d0/d8/de/d1a/f24 [0,4194304] 0 2024-03-22T10:42:33.287 INFO:tasks.workunit.client.0.smithi115.stdout:0/122: dwrite d3/f1a [0,4194304] 0 2024-03-22T10:42:33.343 INFO:tasks.workunit.client.0.smithi115.stdout:1/138: unlink d0/f1d 0 2024-03-22T10:42:33.347 INFO:tasks.workunit.client.0.smithi115.stdout:5/94: fsync d2/df/d13/f16 0 2024-03-22T10:42:33.347 INFO:tasks.workunit.client.0.smithi115.stdout:0/123: readlink d3/l12 0 2024-03-22T10:42:33.350 INFO:tasks.workunit.client.0.smithi115.stdout:1/139: symlink d0/d8/de/l2a 0 2024-03-22T10:42:33.358 INFO:tasks.workunit.client.0.smithi115.stdout:8/263: dwrite d6/f44 [0,4194304] 0 2024-03-22T10:42:33.360 INFO:tasks.workunit.client.0.smithi115.stdout:0/124: mkdir d3/d7/d1b/d1f 0 2024-03-22T10:42:33.374 INFO:tasks.workunit.client.0.smithi115.stdout:6/88: dwrite f3 [0,4194304] 0 2024-03-22T10:42:33.384 INFO:tasks.workunit.client.0.smithi115.stdout:6/89: read f3 [4137091,109949] 0 2024-03-22T10:42:33.396 INFO:tasks.workunit.client.0.smithi115.stdout:0/125: dread d3/d7/d1c/f1e [0,4194304] 0 2024-03-22T10:42:33.396 INFO:tasks.workunit.client.0.smithi115.stdout:5/95: dwrite d2/d15/f1c [0,4194304] 0 2024-03-22T10:42:33.399 INFO:tasks.workunit.client.0.smithi115.stdout:8/264: dwrite d6/dd/f26 [0,4194304] 0 2024-03-22T10:42:33.400 INFO:tasks.workunit.client.0.smithi115.stdout:1/140: dwrite d0/f7 [0,4194304] 0 2024-03-22T10:42:33.403 INFO:tasks.workunit.client.0.smithi115.stdout:5/96: chown d2/df/d13 4 1 2024-03-22T10:42:33.404 INFO:tasks.workunit.client.0.smithi115.stdout:8/265: readlink d6/dd/d24/l47 0 2024-03-22T10:42:33.404 INFO:tasks.workunit.client.0.smithi115.stdout:5/97: chown d2/d15 14 1 2024-03-22T10:42:33.409 INFO:tasks.workunit.client.0.smithi115.stdout:0/126: mknod d3/da/c20 0 2024-03-22T10:42:33.411 INFO:tasks.workunit.client.0.smithi115.stdout:5/98: mkdir d2/df/d1e 0 2024-03-22T10:42:33.413 INFO:tasks.workunit.client.0.smithi115.stdout:8/266: symlink d6/dd/l57 0 2024-03-22T10:42:33.415 INFO:tasks.workunit.client.0.smithi115.stdout:5/99: write d2/f4 [1621837,128203] 0 2024-03-22T10:42:33.416 INFO:tasks.workunit.client.0.smithi115.stdout:8/267: dread - d6/dd/d24/d3a/f54 zero size 2024-03-22T10:42:33.418 INFO:tasks.workunit.client.0.smithi115.stdout:5/100: chown d2/df 530701 1 2024-03-22T10:42:33.420 INFO:tasks.workunit.client.0.smithi115.stdout:8/268: stat f4 0 2024-03-22T10:42:33.425 INFO:tasks.workunit.client.0.smithi115.stdout:8/269: read d6/f16 [462990,87885] 0 2024-03-22T10:42:33.434 INFO:tasks.workunit.client.0.smithi115.stdout:1/141: dwrite d0/f14 [0,4194304] 0 2024-03-22T10:42:33.439 INFO:tasks.workunit.client.0.smithi115.stdout:0/127: dwrite d3/fc [4194304,4194304] 0 2024-03-22T10:42:33.442 INFO:tasks.workunit.client.0.smithi115.stdout:1/142: mkdir d0/d12/d2b 0 2024-03-22T10:42:33.445 INFO:tasks.workunit.client.0.smithi115.stdout:1/143: chown d0/d8/de 12 1 2024-03-22T10:42:33.458 INFO:tasks.workunit.client.0.smithi115.stdout:1/144: creat d0/d8/db/f2c x:0 0 0 2024-03-22T10:42:33.468 INFO:tasks.workunit.client.0.smithi115.stdout:1/145: write d0/f1 [4613763,117221] 0 2024-03-22T10:42:33.469 INFO:tasks.workunit.client.0.smithi115.stdout:5/101: dwrite d2/f11 [0,4194304] 0 2024-03-22T10:42:33.469 INFO:tasks.workunit.client.0.smithi115.stdout:8/270: dwrite d6/fc [0,4194304] 0 2024-03-22T10:42:33.480 INFO:tasks.workunit.client.0.smithi115.stdout:0/128: dwrite d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:33.482 INFO:tasks.workunit.client.0.smithi115.stdout:0/129: fdatasync d3/fc 0 2024-03-22T10:42:33.490 INFO:tasks.workunit.client.0.smithi115.stdout:4/87: sync 2024-03-22T10:42:33.493 INFO:tasks.workunit.client.0.smithi115.stdout:9/138: sync 2024-03-22T10:42:33.493 INFO:tasks.workunit.client.0.smithi115.stdout:3/64: sync 2024-03-22T10:42:33.500 INFO:tasks.workunit.client.0.smithi115.stdout:4/88: write d1/fd [514921,32221] 0 2024-03-22T10:42:33.501 INFO:tasks.workunit.client.0.smithi115.stdout:9/139: fsync d6/d7/d19/f27 0 2024-03-22T10:42:33.506 INFO:tasks.workunit.client.0.smithi115.stdout:0/130: symlink d3/l21 0 2024-03-22T10:42:33.507 INFO:tasks.workunit.client.0.smithi115.stdout:4/89: creat d1/dc/f1a x:0 0 0 2024-03-22T10:42:33.509 INFO:tasks.workunit.client.0.smithi115.stdout:8/271: dwrite d6/dd/d24/d3a/f45 [0,4194304] 0 2024-03-22T10:42:33.517 INFO:tasks.workunit.client.0.smithi115.stdout:4/90: write d1/fd [997205,39483] 0 2024-03-22T10:42:33.517 INFO:tasks.workunit.client.0.smithi115.stdout:5/102: dread d2/f11 [0,4194304] 0 2024-03-22T10:42:33.518 INFO:tasks.workunit.client.0.smithi115.stdout:9/140: rename f1 to d6/d7/f2a 0 2024-03-22T10:42:33.523 INFO:tasks.workunit.client.0.smithi115.stdout:0/131: creat d3/d7/d1c/f22 x:0 0 0 2024-03-22T10:42:33.525 INFO:tasks.workunit.client.0.smithi115.stdout:8/272: symlink d6/dd/l58 0 2024-03-22T10:42:33.527 INFO:tasks.workunit.client.0.smithi115.stdout:5/103: write d2/fc [185272,117880] 0 2024-03-22T10:42:33.529 INFO:tasks.workunit.client.0.smithi115.stdout:1/146: dwrite d0/d8/db/f16 [0,4194304] 0 2024-03-22T10:42:33.538 INFO:tasks.workunit.client.0.smithi115.stdout:5/104: stat d2/df/d13 0 2024-03-22T10:42:33.538 INFO:tasks.workunit.client.0.smithi115.stdout:9/141: creat d6/d7/d20/d22/f2b x:0 0 0 2024-03-22T10:42:33.538 INFO:tasks.workunit.client.0.smithi115.stdout:1/147: creat d0/d8/db/f2d x:0 0 0 2024-03-22T10:42:33.539 INFO:tasks.workunit.client.0.smithi115.stdout:9/142: write d6/fa [390488,20747] 0 2024-03-22T10:42:33.539 INFO:tasks.workunit.client.0.smithi115.stdout:3/65: dwrite d3/fe [0,4194304] 0 2024-03-22T10:42:33.543 INFO:tasks.workunit.client.0.smithi115.stdout:5/105: creat d2/f1f x:0 0 0 2024-03-22T10:42:33.546 INFO:tasks.workunit.client.0.smithi115.stdout:4/91: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:33.553 INFO:tasks.workunit.client.0.smithi115.stdout:8/273: rename d6/dd/d24/d4a/f4e to d6/f59 0 2024-03-22T10:42:33.555 INFO:tasks.workunit.client.0.smithi115.stdout:9/143: creat d6/d7/d20/d22/f2c x:0 0 0 2024-03-22T10:42:33.556 INFO:tasks.workunit.client.0.smithi115.stdout:0/132: link d3/f1a d3/d4/d14/f23 0 2024-03-22T10:42:33.557 INFO:tasks.workunit.client.0.smithi115.stdout:9/144: fsync d6/d7/f17 0 2024-03-22T10:42:33.560 INFO:tasks.workunit.client.0.smithi115.stdout:1/148: symlink d0/d8/db/d25/l2e 0 2024-03-22T10:42:33.565 INFO:tasks.workunit.client.0.smithi115.stdout:3/66: rmdir d3/dd 0 2024-03-22T10:42:33.565 INFO:tasks.workunit.client.0.smithi115.stdout:1/149: write d0/f3 [2771703,42037] 0 2024-03-22T10:42:33.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.576+0000 7fdec0000700 1 -- 172.21.15.115:0/1915460954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeb8074d80 msgr2=0x7fdeb80731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:33.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.576+0000 7fdec0000700 1 --2- 172.21.15.115:0/1915460954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeb8074d80 0x7fdeb80731e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009910 tx=0x7fdea4009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:33.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.577+0000 7fdec0000700 1 -- 172.21.15.115:0/1915460954 shutdown_connections 2024-03-22T10:42:33.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.577+0000 7fdec0000700 1 --2- 172.21.15.115:0/1915460954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb80737b0 0x7fdeb8073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:33.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.577+0000 7fdec0000700 1 --2- 172.21.15.115:0/1915460954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeb8074d80 0x7fdeb80731e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:33.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.577+0000 7fdec0000700 1 -- 172.21.15.115:0/1915460954 >> 172.21.15.115:0/1915460954 conn(0x7fdeb80fb6e0 msgr2=0x7fdeb80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:33.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.577+0000 7fdec0000700 1 -- 172.21.15.115:0/1915460954 shutdown_connections 2024-03-22T10:42:33.569 INFO:tasks.workunit.client.0.smithi115.stdout:0/133: creat d3/d7/d1b/f24 x:0 0 0 2024-03-22T10:42:33.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.577+0000 7fdec0000700 1 -- 172.21.15.115:0/1915460954 wait complete. 2024-03-22T10:42:33.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.580+0000 7fdec0000700 1 Processor -- start 2024-03-22T10:42:33.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.582+0000 7fdec0000700 1 -- start start 2024-03-22T10:42:33.574 INFO:tasks.workunit.client.0.smithi115.stdout:4/92: dwrite d1/ff [0,4194304] 0 2024-03-22T10:42:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.582+0000 7fdec0000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeb80737b0 0x7fdeb819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.583+0000 7fdec0000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb8074d80 0x7fdeb819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.583+0000 7fdec0000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb819cae0 con 0x7fdeb80737b0 2024-03-22T10:42:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.583+0000 7fdec0000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb819cc20 con 0x7fdeb8074d80 2024-03-22T10:42:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.583+0000 7fdebcc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb8074d80 0x7fdeb819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.583+0000 7fdebd600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeb80737b0 0x7fdeb819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.583+0000 7fdebcc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb8074d80 0x7fdeb819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:54556/0 (socket says 172.21.15.115:54556) 2024-03-22T10:42:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.583+0000 7fdebcc00700 1 -- 172.21.15.115:0/4096290832 learned_addr learned my addr 172.21.15.115:0/4096290832 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:33.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.584+0000 7fdebcc00700 1 -- 172.21.15.115:0/4096290832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeb80737b0 msgr2=0x7fdeb819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:33.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.584+0000 7fdebcc00700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeb80737b0 0x7fdeb819bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:33.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.584+0000 7fdebcc00700 1 -- 172.21.15.115:0/4096290832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea40095f0 con 0x7fdeb8074d80 2024-03-22T10:42:33.577 INFO:tasks.workunit.client.0.smithi115.stdout:8/274: rename d6/dd/d12/d31/c4f to d6/dd/d24/c5a 0 2024-03-22T10:42:33.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.586+0000 7fdebcc00700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb8074d80 0x7fdeb819c4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b4b0 tx=0x7fdeac00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:33.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.587+0000 7fdeb6000700 1 -- 172.21.15.115:0/4096290832 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeac011800 con 0x7fdeb8074d80 2024-03-22T10:42:33.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.587+0000 7fdeb6000700 1 -- 172.21.15.115:0/4096290832 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdeac011e10 con 0x7fdeb8074d80 2024-03-22T10:42:33.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.587+0000 7fdeb6000700 1 -- 172.21.15.115:0/4096290832 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeac00f2e0 con 0x7fdeb8074d80 2024-03-22T10:42:33.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.587+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb81a16d0 con 0x7fdeb8074d80 2024-03-22T10:42:33.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.587+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb81a1ca0 con 0x7fdeb8074d80 2024-03-22T10:42:33.579 INFO:tasks.workunit.client.0.smithi115.stdout:7/115: sync 2024-03-22T10:42:33.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.590+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb810f300 con 0x7fdeb8074d80 2024-03-22T10:42:33.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.591+0000 7fdeb6000700 1 -- 172.21.15.115:0/4096290832 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 29) v1 ==== 46247+0+0 (secure 0 0 0) 0x7fdeac00f440 con 0x7fdeb8074d80 2024-03-22T10:42:33.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.592+0000 7fdeb6000700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdea0039230 0x7fdea003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:33.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.592+0000 7fdeb6000700 1 -- 172.21.15.115:0/4096290832 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fdeac04cfe0 con 0x7fdeb8074d80 2024-03-22T10:42:33.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.592+0000 7fdebd600700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdea0039230 0x7fdea003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:33.584 INFO:tasks.workunit.client.0.smithi115.stdout:1/150: creat d0/d8/de/d1a/f2f x:0 0 0 2024-03-22T10:42:33.584 INFO:tasks.workunit.client.0.smithi115.stdout:7/116: dread - d2/d7/f20 zero size 2024-03-22T10:42:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.593+0000 7fdebd600700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdea0039230 0x7fdea003b6e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdea4000c00 tx=0x7fdea4005fb0 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:33.587 INFO:tasks.workunit.client.0.smithi115.stdout:4/93: fdatasync d1/f7 0 2024-03-22T10:42:33.589 INFO:tasks.workunit.client.0.smithi115.stdout:9/145: dread d6/d7/f17 [0,4194304] 0 2024-03-22T10:42:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:33.601+0000 7fdeb6000700 1 -- 172.21.15.115:0/4096290832 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fdeac017890 con 0x7fdeb8074d80 2024-03-22T10:42:33.593 INFO:tasks.workunit.client.0.smithi115.stdout:4/94: read d1/fd [150672,76858] 0 2024-03-22T10:42:33.593 INFO:tasks.workunit.client.0.smithi115.stdout:8/275: creat d6/dd/d12/d31/f5b x:0 0 0 2024-03-22T10:42:33.595 INFO:tasks.workunit.client.0.smithi115.stdout:3/67: dwrite d3/f9 [0,4194304] 0 2024-03-22T10:42:33.597 INFO:tasks.workunit.client.0.smithi115.stdout:0/134: dread d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:33.598 INFO:tasks.workunit.client.0.smithi115.stdout:1/151: creat d0/d12/f30 x:0 0 0 2024-03-22T10:42:33.600 INFO:tasks.workunit.client.0.smithi115.stdout:5/106: readlink d2/l9 0 2024-03-22T10:42:33.600 INFO:tasks.workunit.client.0.smithi115.stdout:3/68: chown d3/f9 1 1 2024-03-22T10:42:33.602 INFO:tasks.workunit.client.0.smithi115.stdout:9/146: readlink d6/d8/l13 0 2024-03-22T10:42:33.608 INFO:tasks.workunit.client.0.smithi115.stdout:0/135: fdatasync d3/f1a 0 2024-03-22T10:42:33.612 INFO:tasks.workunit.client.0.smithi115.stdout:1/152: creat d0/d12/d17/f31 x:0 0 0 2024-03-22T10:42:33.614 INFO:tasks.workunit.client.0.smithi115.stdout:5/107: write d2/df/d13/f16 [363712,55303] 0 2024-03-22T10:42:33.616 INFO:tasks.workunit.client.0.smithi115.stdout:4/95: creat d1/dc/d14/f1b x:0 0 0 2024-03-22T10:42:33.616 INFO:tasks.workunit.client.0.smithi115.stdout:5/108: chown d2 10207 1 2024-03-22T10:42:33.617 INFO:tasks.workunit.client.0.smithi115.stdout:5/109: chown d2 1 1 2024-03-22T10:42:33.620 INFO:tasks.workunit.client.0.smithi115.stdout:7/117: link d2/d7/d11/c22 d2/d7/d12/d10/d19/c24 0 2024-03-22T10:42:33.622 INFO:tasks.workunit.client.0.smithi115.stdout:4/96: stat d1/c8 0 2024-03-22T10:42:33.625 INFO:tasks.workunit.client.0.smithi115.stdout:3/69: dwrite d3/fe [0,4194304] 0 2024-03-22T10:42:33.628 INFO:tasks.workunit.client.0.smithi115.stdout:0/136: mknod d3/d4/c25 0 2024-03-22T10:42:33.630 INFO:tasks.workunit.client.0.smithi115.stdout:7/118: unlink d2/d7/d12/lf 0 2024-03-22T10:42:33.634 INFO:tasks.workunit.client.0.smithi115.stdout:3/70: symlink d3/l10 0 2024-03-22T10:42:33.641 INFO:tasks.workunit.client.0.smithi115.stdout:7/119: creat d2/d7/d12/d10/f25 x:0 0 0 2024-03-22T10:42:33.643 INFO:tasks.workunit.client.0.smithi115.stdout:4/97: truncate d1/f7 2564996 0 2024-03-22T10:42:33.644 INFO:tasks.workunit.client.0.smithi115.stdout:0/137: rename d3/d4/f11 to d3/da/f26 0 2024-03-22T10:42:33.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:33 smithi150 ceph-mon[34518]: pgmap v5: 65 pgs: 65 active+clean; 234 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:42:33.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:33 smithi150 ceph-mon[34518]: mgrmap e29: smithi115.iqzwsz(active, since 4s) 2024-03-22T10:42:33.651 INFO:tasks.workunit.client.0.smithi115.stdout:0/138: mknod d3/da/c27 0 2024-03-22T10:42:33.654 INFO:tasks.workunit.client.0.smithi115.stdout:1/153: dwrite d0/f7 [4194304,4194304] 0 2024-03-22T10:42:33.654 INFO:tasks.workunit.client.0.smithi115.stdout:4/98: truncate d1/dc/d14/f1b 41811 0 2024-03-22T10:42:33.658 INFO:tasks.workunit.client.0.smithi115.stdout:3/71: getdents d3 0 2024-03-22T10:42:33.659 INFO:tasks.workunit.client.0.smithi115.stdout:7/120: getdents d2 0 2024-03-22T10:42:33.661 INFO:tasks.workunit.client.0.smithi115.stdout:5/110: dwrite d2/fe [0,4194304] 0 2024-03-22T10:42:33.661 INFO:tasks.workunit.client.0.smithi115.stdout:1/154: dread - d0/d8/de/d1a/f1e zero size 2024-03-22T10:42:33.663 INFO:tasks.workunit.client.0.smithi115.stdout:9/147: dwrite d6/d7/f2a [0,4194304] 0 2024-03-22T10:42:33.664 INFO:tasks.workunit.client.0.smithi115.stdout:7/121: chown d2/d7/d12/d10/d19 1 1 2024-03-22T10:42:33.665 INFO:tasks.workunit.client.0.smithi115.stdout:4/99: rmdir d1/dc 39 2024-03-22T10:42:33.668 INFO:tasks.workunit.client.0.smithi115.stdout:1/155: creat d0/d8/de/d1a/f32 x:0 0 0 2024-03-22T10:42:33.669 INFO:tasks.workunit.client.0.smithi115.stdout:4/100: stat d1/dc/c12 0 2024-03-22T10:42:33.670 INFO:tasks.workunit.client.0.smithi115.stdout:4/101: stat d1/dc/l11 0 2024-03-22T10:42:33.674 INFO:tasks.workunit.client.0.smithi115.stdout:3/72: rename d3/fe to d3/f11 0 2024-03-22T10:42:33.678 INFO:tasks.workunit.client.0.smithi115.stdout:9/148: creat d6/d7/f2d x:0 0 0 2024-03-22T10:42:33.684 INFO:tasks.workunit.client.0.smithi115.stdout:4/102: rename d1/dc/f1a to d1/d16/f1c 0 2024-03-22T10:42:33.685 INFO:tasks.workunit.client.0.smithi115.stdout:4/103: fdatasync d1/f9 0 2024-03-22T10:42:33.686 INFO:tasks.workunit.client.0.smithi115.stdout:1/156: creat d0/d12/d17/f33 x:0 0 0 2024-03-22T10:42:33.688 INFO:tasks.workunit.client.0.smithi115.stdout:3/73: creat d3/f12 x:0 0 0 2024-03-22T10:42:33.689 INFO:tasks.workunit.client.0.smithi115.stdout:5/111: symlink d2/d15/d1b/l20 0 2024-03-22T10:42:33.692 INFO:tasks.workunit.client.0.smithi115.stdout:3/74: mknod d3/c13 0 2024-03-22T10:42:33.694 INFO:tasks.workunit.client.0.smithi115.stdout:7/122: getdents d2/d7/d12/d10 0 2024-03-22T10:42:33.696 INFO:tasks.workunit.client.0.smithi115.stdout:4/104: link d1/f9 d1/dc/d14/f1d 0 2024-03-22T10:42:33.699 INFO:tasks.workunit.client.0.smithi115.stdout:4/105: chown d1/dc/l19 1798 1 2024-03-22T10:42:33.702 INFO:tasks.workunit.client.0.smithi115.stdout:7/123: dread d2/d7/fb [0,4194304] 0 2024-03-22T10:42:33.709 INFO:tasks.workunit.client.0.smithi115.stdout:3/75: rmdir d3 39 2024-03-22T10:42:33.712 INFO:tasks.workunit.client.0.smithi115.stdout:4/106: dread d1/f9 [0,4194304] 0 2024-03-22T10:42:33.712 INFO:tasks.workunit.client.0.smithi115.stdout:1/157: truncate d0/d8/db/f16 3110176 0 2024-03-22T10:42:33.713 INFO:tasks.workunit.client.0.smithi115.stdout:3/76: mknod d3/c14 0 2024-03-22T10:42:33.713 INFO:tasks.workunit.client.0.smithi115.stdout:6/90: sync 2024-03-22T10:42:33.714 INFO:tasks.workunit.client.0.smithi115.stdout:2/83: sync 2024-03-22T10:42:33.715 INFO:tasks.workunit.client.0.smithi115.stdout:4/107: write d1/dc/d14/f1b [252663,38576] 0 2024-03-22T10:42:33.715 INFO:tasks.workunit.client.0.smithi115.stdout:7/124: write d2/d7/fb [570818,55800] 0 2024-03-22T10:42:33.716 INFO:tasks.workunit.client.0.smithi115.stdout:4/108: chown d1/d16 14850 1 2024-03-22T10:42:33.719 INFO:tasks.workunit.client.0.smithi115.stdout:7/125: truncate d2/f5 4814821 0 2024-03-22T10:42:33.722 INFO:tasks.workunit.client.0.smithi115.stdout:6/91: mkdir d4/d11/d12/d1c 0 2024-03-22T10:42:33.724 INFO:tasks.workunit.client.0.smithi115.stdout:7/126: write d2/d7/f13 [8419246,33511] 0 2024-03-22T10:42:33.724 INFO:tasks.workunit.client.0.smithi115.stdout:6/92: fsync f3 0 2024-03-22T10:42:33.724 INFO:tasks.workunit.client.0.smithi115.stdout:4/109: dread d1/dc/d14/f1b [0,4194304] 0 2024-03-22T10:42:33.729 INFO:tasks.workunit.client.0.smithi115.stdout:4/110: mknod d1/dc/d17/c1e 0 2024-03-22T10:42:33.729 INFO:tasks.workunit.client.0.smithi115.stdout:7/127: creat d2/d7/d12/d10/f26 x:0 0 0 2024-03-22T10:42:33.731 INFO:tasks.workunit.client.0.smithi115.stdout:2/84: dread da/fe [0,4194304] 0 2024-03-22T10:42:33.731 INFO:tasks.workunit.client.0.smithi115.stdout:7/128: chown d2/d7/d12/d10/f17 795237 1 2024-03-22T10:42:33.732 INFO:tasks.workunit.client.0.smithi115.stdout:1/158: dwrite d0/f1 [0,4194304] 0 2024-03-22T10:42:33.736 INFO:tasks.workunit.client.0.smithi115.stdout:7/129: fdatasync d2/d7/d12/d10/f16 0 2024-03-22T10:42:33.736 INFO:tasks.workunit.client.0.smithi115.stdout:4/111: rename d1/f10 to d1/f1f 0 2024-03-22T10:42:33.740 INFO:tasks.workunit.client.0.smithi115.stdout:7/130: chown d2/d7/d12/d10/f26 12261 1 2024-03-22T10:42:33.743 INFO:tasks.workunit.client.0.smithi115.stdout:7/131: chown d2/d7/d12/d10/f1d 77 1 2024-03-22T10:42:33.744 INFO:tasks.workunit.client.0.smithi115.stdout:7/132: readlink d2/d7/l18 0 2024-03-22T10:42:33.744 INFO:tasks.workunit.client.0.smithi115.stdout:4/112: creat d1/dc/f20 x:0 0 0 2024-03-22T10:42:33.746 INFO:tasks.workunit.client.0.smithi115.stdout:1/159: getdents d0/d22 0 2024-03-22T10:42:33.746 INFO:tasks.workunit.client.0.smithi115.stdout:6/93: dread d4/f5 [0,4194304] 0 2024-03-22T10:42:33.750 INFO:tasks.workunit.client.0.smithi115.stdout:7/133: write d2/d7/d12/d10/f1e [226926,64421] 0 2024-03-22T10:42:33.750 INFO:tasks.workunit.client.0.smithi115.stdout:4/113: write d1/f1f [1504242,123208] 0 2024-03-22T10:42:33.754 INFO:tasks.workunit.client.0.smithi115.stdout:7/134: fdatasync d2/d7/d12/d10/f25 0 2024-03-22T10:42:33.756 INFO:tasks.workunit.client.0.smithi115.stdout:7/135: chown d2/le 901 1 2024-03-22T10:42:33.756 INFO:tasks.workunit.client.0.smithi115.stdout:1/160: symlink d0/d8/de/l34 0 2024-03-22T10:42:33.760 INFO:tasks.workunit.client.0.smithi115.stdout:2/85: dwrite da/fe [4194304,4194304] 0 2024-03-22T10:42:33.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:33 smithi115 ceph-mon[29111]: pgmap v5: 65 pgs: 65 active+clean; 234 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:42:33.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:33 smithi115 ceph-mon[29111]: mgrmap e29: smithi115.iqzwsz(active, since 4s) 2024-03-22T10:42:33.766 INFO:tasks.workunit.client.0.smithi115.stdout:1/161: truncate d0/d12/d17/f33 477480 0 2024-03-22T10:42:33.768 INFO:tasks.workunit.client.0.smithi115.stdout:7/136: rename d2/d7/d12/d10/f1d to d2/d7/d12/d10/f27 0 2024-03-22T10:42:33.779 INFO:tasks.workunit.client.0.smithi115.stdout:7/137: creat d2/d7/d12/d10/f28 x:0 0 0 2024-03-22T10:42:33.781 INFO:tasks.workunit.client.0.smithi115.stdout:4/114: dwrite d1/ff [0,4194304] 0 2024-03-22T10:42:33.781 INFO:tasks.workunit.client.0.smithi115.stdout:1/162: creat d0/d12/d2b/f35 x:0 0 0 2024-03-22T10:42:33.785 INFO:tasks.workunit.client.0.smithi115.stdout:2/86: truncate da/ff 1019416 0 2024-03-22T10:42:33.786 INFO:tasks.workunit.client.0.smithi115.stdout:4/115: dread - d1/dc/f20 zero size 2024-03-22T10:42:33.786 INFO:tasks.workunit.client.0.smithi115.stdout:1/163: dread - d0/d12/f30 zero size 2024-03-22T10:42:33.788 INFO:tasks.workunit.client.0.smithi115.stdout:6/94: dwrite f3 [4194304,4194304] 0 2024-03-22T10:42:33.793 INFO:tasks.workunit.client.0.smithi115.stdout:7/138: rename d2/le to d2/d7/l29 0 2024-03-22T10:42:33.798 INFO:tasks.workunit.client.0.smithi115.stdout:2/87: rename l5 to da/l11 0 2024-03-22T10:42:33.799 INFO:tasks.workunit.client.0.smithi115.stdout:7/139: creat d2/d7/d11/f2a x:0 0 0 2024-03-22T10:42:33.804 INFO:tasks.workunit.client.0.smithi115.stdout:1/164: dread d0/f1 [0,4194304] 0 2024-03-22T10:42:33.805 INFO:tasks.workunit.client.0.smithi115.stdout:4/116: rename d1/f1f to d1/dc/d17/f21 0 2024-03-22T10:42:33.808 INFO:tasks.workunit.client.0.smithi115.stdout:7/140: truncate d2/d7/d12/d10/f1e 910971 0 2024-03-22T10:42:33.808 INFO:tasks.workunit.client.0.smithi115.stdout:6/95: dread d4/f7 [0,4194304] 0 2024-03-22T10:42:33.810 INFO:tasks.workunit.client.0.smithi115.stdout:2/88: creat da/f12 x:0 0 0 2024-03-22T10:42:33.812 INFO:tasks.workunit.client.0.smithi115.stdout:4/117: rmdir d1/dc/d14 39 2024-03-22T10:42:33.816 INFO:tasks.workunit.client.0.smithi115.stdout:7/141: mknod d2/d7/d12/c2b 0 2024-03-22T10:42:33.816 INFO:tasks.workunit.client.0.smithi115.stdout:6/96: rename d4/d13 to d4/d13/d1d 22 2024-03-22T10:42:33.821 INFO:tasks.workunit.client.0.smithi115.stdout:1/165: creat d0/d8/f36 x:0 0 0 2024-03-22T10:42:33.822 INFO:tasks.workunit.client.0.smithi115.stdout:7/142: write d2/d7/d12/d10/f17 [6709922,83037] 0 2024-03-22T10:42:33.822 INFO:tasks.workunit.client.0.smithi115.stdout:6/97: symlink d4/d13/l1e 0 2024-03-22T10:42:33.829 INFO:tasks.workunit.client.0.smithi115.stdout:7/143: write d2/d7/d12/d10/f26 [730391,111719] 0 2024-03-22T10:42:33.832 INFO:tasks.workunit.client.0.smithi115.stdout:7/144: chown d2/f5 39780724 1 2024-03-22T10:42:33.837 INFO:tasks.workunit.client.0.smithi115.stdout:7/145: mknod d2/d7/d12/c2c 0 2024-03-22T10:42:33.839 INFO:tasks.workunit.client.0.smithi115.stdout:4/118: dread d1/fd [0,4194304] 0 2024-03-22T10:42:33.843 INFO:tasks.workunit.client.0.smithi115.stdout:7/146: mknod d2/d7/d11/c2d 0 2024-03-22T10:42:33.845 INFO:tasks.workunit.client.0.smithi115.stdout:6/98: dread d4/f7 [0,4194304] 0 2024-03-22T10:42:33.846 INFO:tasks.workunit.client.0.smithi115.stdout:7/147: stat d2/d7/d12/d10/f28 0 2024-03-22T10:42:33.847 INFO:tasks.workunit.client.0.smithi115.stdout:7/148: chown d2/d7 465752 1 2024-03-22T10:42:33.848 INFO:tasks.workunit.client.0.smithi115.stdout:4/119: rmdir d1/dc 39 2024-03-22T10:42:33.849 INFO:tasks.workunit.client.0.smithi115.stdout:2/89: fsync da/f12 0 2024-03-22T10:42:33.850 INFO:tasks.workunit.client.0.smithi115.stdout:1/166: dwrite d0/f7 [8388608,4194304] 0 2024-03-22T10:42:33.857 INFO:tasks.workunit.client.0.smithi115.stdout:7/149: rename d2/d7/d12/d10/f1e to d2/d7/d12/d10/d23/f2e 0 2024-03-22T10:42:33.866 INFO:tasks.workunit.client.0.smithi115.stdout:7/150: mknod d2/d7/d12/d10/d19/c2f 0 2024-03-22T10:42:33.873 INFO:tasks.workunit.client.0.smithi115.stdout:6/99: dwrite f3 [4194304,4194304] 0 2024-03-22T10:42:33.875 INFO:tasks.workunit.client.0.smithi115.stdout:7/151: write d2/d7/d12/f1b [273306,28076] 0 2024-03-22T10:42:33.875 INFO:tasks.workunit.client.0.smithi115.stdout:1/167: dread d0/f7 [8388608,4194304] 0 2024-03-22T10:42:33.879 INFO:tasks.workunit.client.0.smithi115.stdout:6/100: creat d4/d11/f1f x:0 0 0 2024-03-22T10:42:33.880 INFO:tasks.workunit.client.0.smithi115.stdout:1/168: mkdir d0/d12/d17/d37 0 2024-03-22T10:42:33.882 INFO:tasks.workunit.client.0.smithi115.stdout:7/152: write d2/d7/d12/d10/d19/f1a [1731768,10945] 0 2024-03-22T10:42:33.884 INFO:tasks.workunit.client.0.smithi115.stdout:4/120: dwrite d1/dc/d14/f1d [0,4194304] 0 2024-03-22T10:42:33.887 INFO:tasks.workunit.client.0.smithi115.stdout:4/121: chown d1/dc/d14/f1d 3 1 2024-03-22T10:42:33.889 INFO:tasks.workunit.client.0.smithi115.stdout:7/153: mknod d2/d7/d12/d10/c30 0 2024-03-22T10:42:33.891 INFO:tasks.workunit.client.0.smithi115.stdout:1/169: mknod d0/d8/de/c38 0 2024-03-22T10:42:33.897 INFO:tasks.workunit.client.0.smithi115.stdout:4/122: rename d1/l3 to d1/dc/d17/l22 0 2024-03-22T10:42:33.901 INFO:tasks.workunit.client.0.smithi115.stdout:6/101: rename d4/l6 to d4/l20 0 2024-03-22T10:42:33.902 INFO:tasks.workunit.client.0.smithi115.stdout:7/154: dread d2/d7/d12/f1b [0,4194304] 0 2024-03-22T10:42:33.905 INFO:tasks.workunit.client.0.smithi115.stdout:2/90: dwrite da/ff [0,4194304] 0 2024-03-22T10:42:33.907 INFO:tasks.workunit.client.0.smithi115.stdout:6/102: write d4/f5 [4698288,63121] 0 2024-03-22T10:42:33.910 INFO:tasks.workunit.client.0.smithi115.stdout:4/123: rmdir d1/dc 39 2024-03-22T10:42:33.910 INFO:tasks.workunit.client.0.smithi115.stdout:7/155: mkdir d2/d7/d31 0 2024-03-22T10:42:33.913 INFO:tasks.workunit.client.0.smithi115.stdout:7/156: dread - d2/d7/d12/d10/f28 zero size 2024-03-22T10:42:33.913 INFO:tasks.workunit.client.0.smithi115.stdout:2/91: symlink da/l13 0 2024-03-22T10:42:33.915 INFO:tasks.workunit.client.0.smithi115.stdout:6/103: rename d4/f7 to d4/d11/d12/f21 0 2024-03-22T10:42:33.918 INFO:tasks.workunit.client.0.smithi115.stdout:6/104: readlink d4/l17 0 2024-03-22T10:42:33.919 INFO:tasks.workunit.client.0.smithi115.stdout:7/157: creat d2/f32 x:0 0 0 2024-03-22T10:42:33.924 INFO:tasks.workunit.client.0.smithi115.stdout:6/105: write d4/d11/f1f [908821,6073] 0 2024-03-22T10:42:33.930 INFO:tasks.workunit.client.0.smithi115.stdout:1/170: dwrite d0/d8/de/d1a/f32 [0,4194304] 0 2024-03-22T10:42:33.933 INFO:tasks.workunit.client.0.smithi115.stdout:2/92: dread da/ff [0,4194304] 0 2024-03-22T10:42:33.935 INFO:tasks.workunit.client.0.smithi115.stdout:4/124: creat d1/dc/f23 x:0 0 0 2024-03-22T10:42:33.939 INFO:tasks.workunit.client.0.smithi115.stdout:1/171: rmdir d0/d22 39 2024-03-22T10:42:33.940 INFO:tasks.workunit.client.0.smithi115.stdout:2/93: creat da/f14 x:0 0 0 2024-03-22T10:42:33.943 INFO:tasks.workunit.client.0.smithi115.stdout:7/158: rmdir d2 39 2024-03-22T10:42:33.945 INFO:tasks.workunit.client.0.smithi115.stdout:1/172: creat d0/d12/d17/f39 x:0 0 0 2024-03-22T10:42:33.948 INFO:tasks.workunit.client.0.smithi115.stdout:6/106: dwrite d4/f5 [0,4194304] 0 2024-03-22T10:42:33.951 INFO:tasks.workunit.client.0.smithi115.stdout:1/173: mknod d0/d12/d2b/c3a 0 2024-03-22T10:42:33.953 INFO:tasks.workunit.client.0.smithi115.stdout:1/174: read - d0/d12/f30 zero size 2024-03-22T10:42:33.957 INFO:tasks.workunit.client.0.smithi115.stdout:1/175: write d0/d12/d17/f33 [330255,131025] 0 2024-03-22T10:42:33.957 INFO:tasks.workunit.client.0.smithi115.stdout:4/125: truncate d1/ff 3177108 0 2024-03-22T10:42:33.959 INFO:tasks.workunit.client.0.smithi115.stdout:6/107: link d4/c9 d4/d11/c22 0 2024-03-22T10:42:33.963 INFO:tasks.workunit.client.0.smithi115.stdout:1/176: read - d0/d12/d17/f39 zero size 2024-03-22T10:42:33.971 INFO:tasks.workunit.client.0.smithi115.stdout:1/177: readlink d0/d8/lf 0 2024-03-22T10:42:33.973 INFO:tasks.workunit.client.0.smithi115.stdout:2/94: dwrite da/fb [0,4194304] 0 2024-03-22T10:42:33.976 INFO:tasks.workunit.client.0.smithi115.stdout:1/178: mknod d0/d12/d17/d37/c3b 0 2024-03-22T10:42:33.977 INFO:tasks.workunit.client.0.smithi115.stdout:7/159: dwrite d2/d7/f13 [4194304,4194304] 0 2024-03-22T10:42:33.980 INFO:tasks.workunit.client.0.smithi115.stdout:2/95: rename da/ff to da/f15 0 2024-03-22T10:42:33.981 INFO:tasks.workunit.client.0.smithi115.stdout:4/126: write d1/dc/d14/f1b [1335416,36916] 0 2024-03-22T10:42:33.986 INFO:tasks.workunit.client.0.smithi115.stdout:1/179: getdents d0 0 2024-03-22T10:42:33.986 INFO:tasks.workunit.client.0.smithi115.stdout:4/127: chown d1/d16/f1c 103726648 1 2024-03-22T10:42:33.993 INFO:tasks.workunit.client.0.smithi115.stdout:1/180: creat d0/d12/d17/d37/f3c x:0 0 0 2024-03-22T10:42:33.995 INFO:tasks.workunit.client.0.smithi115.stdout:6/108: dwrite d4/d11/f1f [0,4194304] 0 2024-03-22T10:42:33.998 INFO:tasks.workunit.client.0.smithi115.stdout:6/109: write d4/d11/f1f [2753116,38672] 0 2024-03-22T10:42:34.006 INFO:tasks.workunit.client.0.smithi115.stdout:1/181: creat d0/d22/f3d x:0 0 0 2024-03-22T10:42:34.010 INFO:tasks.workunit.client.0.smithi115.stdout:1/182: write d0/d8/db/f27 [1370918,82190] 0 2024-03-22T10:42:34.013 INFO:tasks.workunit.client.0.smithi115.stdout:2/96: dwrite da/fe [0,4194304] 0 2024-03-22T10:42:34.020 INFO:tasks.workunit.client.0.smithi115.stdout:2/97: dread - da/f12 zero size 2024-03-22T10:42:34.021 INFO:tasks.workunit.client.0.smithi115.stdout:7/160: dwrite d2/d7/fb [0,4194304] 0 2024-03-22T10:42:34.023 INFO:tasks.workunit.client.0.smithi115.stdout:1/183: creat d0/d12/d17/f3e x:0 0 0 2024-03-22T10:42:34.026 INFO:tasks.workunit.client.0.smithi115.stdout:8/276: sync 2024-03-22T10:42:34.029 INFO:tasks.workunit.client.0.smithi115.stdout:6/110: dread d4/d11/f1f [0,4194304] 0 2024-03-22T10:42:34.031 INFO:tasks.workunit.client.0.smithi115.stdout:4/128: dwrite d1/f7 [0,4194304] 0 2024-03-22T10:42:34.031 INFO:tasks.workunit.client.0.smithi115.stdout:8/277: truncate d6/f59 1003727 0 2024-03-22T10:42:34.034 INFO:tasks.workunit.client.0.smithi115.stdout:7/161: rename d2/d7/d31 to d2/d7/d12/d10/d23/d33 0 2024-03-22T10:42:34.047 INFO:tasks.workunit.client.0.smithi115.stdout:1/184: dread d0/f1 [0,4194304] 0 2024-03-22T10:42:34.048 INFO:tasks.workunit.client.0.smithi115.stdout:2/98: truncate da/fe 1362429 0 2024-03-22T10:42:34.048 INFO:tasks.workunit.client.0.smithi115.stdout:4/129: symlink d1/dc/l24 0 2024-03-22T10:42:34.051 INFO:tasks.workunit.client.0.smithi115.stdout:8/278: link d6/dd/d12/f3b d6/dd/d12/d31/f5c 0 2024-03-22T10:42:34.052 INFO:tasks.workunit.client.0.smithi115.stdout:4/130: chown d1/dc/c13 13649225 1 2024-03-22T10:42:34.052 INFO:tasks.workunit.client.0.smithi115.stdout:2/99: creat da/f16 x:0 0 0 2024-03-22T10:42:34.054 INFO:tasks.workunit.client.0.smithi115.stdout:4/131: readlink d1/dc/l11 0 2024-03-22T10:42:34.055 INFO:tasks.workunit.client.0.smithi115.stdout:1/185: symlink d0/d8/db/l3f 0 2024-03-22T10:42:34.055 INFO:tasks.workunit.client.0.smithi115.stdout:7/162: dread d2/f5 [0,4194304] 0 2024-03-22T10:42:34.058 INFO:tasks.workunit.client.0.smithi115.stdout:6/111: dread d4/f5 [0,4194304] 0 2024-03-22T10:42:34.059 INFO:tasks.workunit.client.0.smithi115.stdout:1/186: mknod d0/d12/c40 0 2024-03-22T10:42:34.064 INFO:tasks.workunit.client.0.smithi115.stdout:4/132: creat d1/dc/f25 x:0 0 0 2024-03-22T10:42:34.064 INFO:tasks.workunit.client.0.smithi115.stdout:7/163: symlink d2/d7/d12/d10/d23/l34 0 2024-03-22T10:42:34.067 INFO:tasks.workunit.client.0.smithi115.stdout:6/112: write d4/d11/d12/f21 [3714835,85076] 0 2024-03-22T10:42:34.071 INFO:tasks.workunit.client.0.smithi115.stdout:4/133: creat d1/d16/f26 x:0 0 0 2024-03-22T10:42:34.073 INFO:tasks.workunit.client.0.smithi115.stdout:4/134: dread - d1/dc/f25 zero size 2024-03-22T10:42:34.078 INFO:tasks.workunit.client.0.smithi115.stdout:4/135: symlink d1/d16/l27 0 2024-03-22T10:42:34.079 INFO:tasks.workunit.client.0.smithi115.stdout:2/100: getdents da 0 2024-03-22T10:42:34.083 INFO:tasks.workunit.client.0.smithi115.stdout:4/136: mkdir d1/d16/d28 0 2024-03-22T10:42:34.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.090+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdeb8103200 con 0x7fdea0039230 2024-03-22T10:42:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.093+0000 7fdeb6000700 1 -- 172.21.15.115:0/4096290832 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fdeb8103200 con 0x7fdea0039230 2024-03-22T10:42:34.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.096+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdea0039230 msgr2=0x7fdea003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:34.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.096+0000 7fdec0000700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdea0039230 0x7fdea003b6e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdea4000c00 tx=0x7fdea4005fb0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.096+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb8074d80 msgr2=0x7fdeb819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:34.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.096+0000 7fdec0000700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb8074d80 0x7fdeb819c4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b4b0 tx=0x7fdeac00b870 comp rx=0 tx=0).stop 2024-03-22T10:42:34.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.097+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 shutdown_connections 2024-03-22T10:42:34.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.097+0000 7fdec0000700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdea0039230 0x7fdea003b6e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.097+0000 7fdec0000700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeb80737b0 0x7fdeb819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.097+0000 7fdec0000700 1 --2- 172.21.15.115:0/4096290832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb8074d80 0x7fdeb819c4c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.097+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 >> 172.21.15.115:0/4096290832 conn(0x7fdeb80fb6e0 msgr2=0x7fdeb8101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:34.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.099+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 shutdown_connections 2024-03-22T10:42:34.091 INFO:tasks.workunit.client.0.smithi115.stdout:6/113: dread d4/f5 [0,4194304] 0 2024-03-22T10:42:34.092 INFO:tasks.workunit.client.0.smithi115.stdout:1/187: getdents d0/d8/de/d1a 0 2024-03-22T10:42:34.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.099+0000 7fdec0000700 1 -- 172.21.15.115:0/4096290832 wait complete. 2024-03-22T10:42:34.097 INFO:tasks.workunit.client.0.smithi115.stdout:7/164: dwrite d2/d7/d12/d10/f27 [0,4194304] 0 2024-03-22T10:42:34.099 INFO:tasks.workunit.client.0.smithi115.stdout:2/101: dread da/f15 [0,4194304] 0 2024-03-22T10:42:34.103 INFO:tasks.workunit.client.0.smithi115.stdout:7/165: dread - d2/d7/d12/d10/f28 zero size 2024-03-22T10:42:34.105 INFO:tasks.workunit.client.0.smithi115.stdout:8/279: dread d6/f23 [0,4194304] 0 2024-03-22T10:42:34.106 INFO:tasks.workunit.client.0.smithi115.stdout:2/102: unlink da/fe 0 2024-03-22T10:42:34.111 INFO:tasks.workunit.client.0.smithi115.stdout:4/137: dwrite d1/dc/d14/f1b [0,4194304] 0 2024-03-22T10:42:34.116 INFO:tasks.workunit.client.0.smithi115.stdout:1/188: dread d0/d8/f9 [0,4194304] 0 2024-03-22T10:42:34.119 INFO:tasks.workunit.client.0.smithi115.stdout:1/189: write d0/f3 [1311627,68228] 0 2024-03-22T10:42:34.120 INFO:tasks.workunit.client.0.smithi115.stdout:4/138: symlink d1/dc/d17/l29 0 2024-03-22T10:42:34.120 INFO:tasks.workunit.client.0.smithi115.stdout:6/114: dread d4/d11/f1f [0,4194304] 0 2024-03-22T10:42:34.121 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:42:34.122 INFO:tasks.workunit.client.0.smithi115.stdout:1/190: fsync d0/f1 0 2024-03-22T10:42:34.127 INFO:tasks.workunit.client.0.smithi115.stdout:4/139: mknod d1/d16/d28/c2a 0 2024-03-22T10:42:34.134 INFO:tasks.workunit.client.0.smithi115.stdout:4/140: mkdir d1/d2b 0 2024-03-22T10:42:34.138 INFO:tasks.workunit.client.0.smithi115.stdout:7/166: dwrite d2/d7/d12/d10/f16 [0,4194304] 0 2024-03-22T10:42:34.144 INFO:tasks.workunit.client.0.smithi115.stdout:8/280: dread d6/dd/f33 [0,4194304] 0 2024-03-22T10:42:34.145 INFO:tasks.workunit.client.0.smithi115.stdout:2/103: dwrite da/f16 [0,4194304] 0 2024-03-22T10:42:34.147 INFO:tasks.workunit.client.0.smithi115.stdout:8/281: readlink d6/dd/d12/l1c 0 2024-03-22T10:42:34.147 INFO:tasks.workunit.client.0.smithi115.stdout:4/141: rename d1/dc/l19 to d1/dc/d14/l2c 0 2024-03-22T10:42:34.149 INFO:tasks.workunit.client.0.smithi115.stdout:1/191: dread d0/d8/db/f27 [0,4194304] 0 2024-03-22T10:42:34.153 INFO:tasks.workunit.client.0.smithi115.stdout:2/104: symlink da/l17 0 2024-03-22T10:42:34.160 INFO:tasks.workunit.client.0.smithi115.stdout:4/142: mknod d1/c2d 0 2024-03-22T10:42:34.164 INFO:tasks.workunit.client.0.smithi115.stdout:2/105: rename da/l13 to da/l18 0 2024-03-22T10:42:34.170 INFO:tasks.workunit.client.0.smithi115.stdout:4/143: symlink d1/d16/l2e 0 2024-03-22T10:42:34.172 INFO:tasks.workunit.client.0.smithi115.stdout:6/115: dwrite d4/d11/d12/f21 [0,4194304] 0 2024-03-22T10:42:34.174 INFO:tasks.workunit.client.0.smithi115.stdout:2/106: write da/f12 [649913,122570] 0 2024-03-22T10:42:34.175 INFO:tasks.workunit.client.0.smithi115.stdout:1/192: write d0/d8/db/f27 [478867,10799] 0 2024-03-22T10:42:34.183 INFO:tasks.workunit.client.0.smithi115.stdout:2/107: mknod da/c19 0 2024-03-22T10:42:34.183 INFO:tasks.workunit.client.0.smithi115.stdout:1/193: fsync d0/d8/f9 0 2024-03-22T10:42:34.183 INFO:tasks.workunit.client.0.smithi115.stdout:6/116: creat d4/d11/f23 x:0 0 0 2024-03-22T10:42:34.184 INFO:tasks.workunit.client.0.smithi115.stdout:7/167: dwrite d2/d7/d12/d10/f25 [0,4194304] 0 2024-03-22T10:42:34.189 INFO:tasks.workunit.client.0.smithi115.stdout:8/282: dwrite d6/fa [0,4194304] 0 2024-03-22T10:42:34.194 INFO:tasks.workunit.client.0.smithi115.stdout:8/283: chown d6/l13 3441 1 2024-03-22T10:42:34.202 INFO:tasks.workunit.client.0.smithi115.stdout:1/194: creat d0/d22/f41 x:0 0 0 2024-03-22T10:42:34.208 INFO:tasks.workunit.client.0.smithi115.stdout:8/284: symlink d6/dd/l5d 0 2024-03-22T10:42:34.213 INFO:tasks.workunit.client.0.smithi115.stdout:2/108: dwrite da/f16 [0,4194304] 0 2024-03-22T10:42:34.213 INFO:tasks.workunit.client.0.smithi115.stdout:1/195: creat d0/d12/d17/f42 x:0 0 0 2024-03-22T10:42:34.216 INFO:tasks.workunit.client.0.smithi115.stdout:3/77: sync 2024-03-22T10:42:34.216 INFO:tasks.workunit.client.0.smithi115.stdout:5/112: sync 2024-03-22T10:42:34.216 INFO:tasks.workunit.client.0.smithi115.stdout:9/149: sync 2024-03-22T10:42:34.217 INFO:tasks.workunit.client.0.smithi115.stdout:0/139: sync 2024-03-22T10:42:34.218 INFO:tasks.workunit.client.0.smithi115.stdout:7/168: getdents d2/d7/d12/d10 0 2024-03-22T10:42:34.221 INFO:tasks.workunit.client.0.smithi115.stdout:3/78: chown d3/c14 18155017 1 2024-03-22T10:42:34.225 INFO:tasks.workunit.client.0.smithi115.stdout:4/144: truncate d1/dc/d14/f1d 1594933 0 2024-03-22T10:42:34.226 INFO:tasks.workunit.client.0.smithi115.stdout:9/150: write d6/d7/f16 [483436,89763] 0 2024-03-22T10:42:34.227 INFO:tasks.workunit.client.0.smithi115.stdout:2/109: mknod da/c1a 0 2024-03-22T10:42:34.236 INFO:tasks.workunit.client.0.smithi115.stdout:3/79: rename d3/fc to d3/f15 0 2024-03-22T10:42:34.238 INFO:tasks.workunit.client.0.smithi115.stdout:6/117: dwrite d4/d11/f1f [0,4194304] 0 2024-03-22T10:42:34.245 INFO:tasks.workunit.client.0.smithi115.stdout:9/151: creat d6/f2e x:0 0 0 2024-03-22T10:42:34.245 INFO:tasks.workunit.client.0.smithi115.stdout:2/110: creat da/f1b x:0 0 0 2024-03-22T10:42:34.251 INFO:tasks.workunit.client.0.smithi115.stdout:5/113: rename l0 to d2/df/d1e/l21 0 2024-03-22T10:42:34.254 INFO:tasks.workunit.client.0.smithi115.stdout:3/80: mkdir d3/d16 0 2024-03-22T10:42:34.256 INFO:tasks.workunit.client.0.smithi115.stdout:8/285: dwrite d6/dd/d12/f1d [0,4194304] 0 2024-03-22T10:42:34.257 INFO:tasks.workunit.client.0.smithi115.stdout:7/169: write d2/f5 [3414572,130982] 0 2024-03-22T10:42:34.265 INFO:tasks.workunit.client.0.smithi115.stdout:1/196: dwrite d0/d22/f41 [0,4194304] 0 2024-03-22T10:42:34.270 INFO:tasks.workunit.client.0.smithi115.stdout:3/81: symlink d3/l17 0 2024-03-22T10:42:34.278 INFO:tasks.workunit.client.0.smithi115.stdout:4/145: link d1/dc/d14/l2c d1/l2f 0 2024-03-22T10:42:34.282 INFO:tasks.workunit.client.0.smithi115.stdout:7/170: symlink d2/d7/d12/d10/d23/l35 0 2024-03-22T10:42:34.283 INFO:tasks.workunit.client.0.smithi115.stdout:1/197: read d0/d8/de/d1a/f24 [295400,2564] 0 2024-03-22T10:42:34.286 INFO:tasks.workunit.client.0.smithi115.stdout:0/140: dwrite d3/f1a [4194304,4194304] 0 2024-03-22T10:42:34.286 INFO:tasks.workunit.client.0.smithi115.stdout:9/152: link d6/d7/f17 d6/d7/d20/f2f 0 2024-03-22T10:42:34.286 INFO:tasks.workunit.client.0.smithi115.stdout:3/82: creat d3/d16/f18 x:0 0 0 2024-03-22T10:42:34.287 INFO:tasks.workunit.client.0.smithi115.stdout:6/118: dread f3 [4194304,4194304] 0 2024-03-22T10:42:34.291 INFO:tasks.workunit.client.0.smithi115.stdout:8/286: link d6/dd/d24/l2d d6/dd/d12/l5e 0 2024-03-22T10:42:34.292 INFO:tasks.workunit.client.0.smithi115.stdout:5/114: dread d2/fe [0,4194304] 0 2024-03-22T10:42:34.295 INFO:tasks.workunit.client.0.smithi115.stdout:9/153: rmdir d6/d7/d20/d22 39 2024-03-22T10:42:34.301 INFO:tasks.workunit.client.0.smithi115.stdout:7/171: rename d2/d7/d12/d10/f27 to d2/d7/d12/d10/f36 0 2024-03-22T10:42:34.307 INFO:tasks.workunit.client.0.smithi115.stdout:8/287: rmdir d6/dd/d12/d31 39 2024-03-22T10:42:34.310 INFO:tasks.workunit.client.0.smithi115.stdout:9/154: mkdir d6/d7/d30 0 2024-03-22T10:42:34.313 INFO:tasks.workunit.client.0.smithi115.stdout:1/198: dwrite d0/d12/d17/f3e [0,4194304] 0 2024-03-22T10:42:34.313 INFO:tasks.workunit.client.0.smithi115.stdout:6/119: creat d4/f24 x:0 0 0 2024-03-22T10:42:34.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.326+0000 7f2b0be00700 1 -- 172.21.15.115:0/3774080892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074d80 msgr2=0x7f2b040731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:34.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.326+0000 7f2b0be00700 1 --2- 172.21.15.115:0/3774080892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074d80 0x7f2b040731e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009910 tx=0x7f2af8009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:34.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.326+0000 7f2b0be00700 1 -- 172.21.15.115:0/3774080892 shutdown_connections 2024-03-22T10:42:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.326+0000 7f2b0be00700 1 --2- 172.21.15.115:0/3774080892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b040737b0 0x7f2b04073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.326+0000 7f2b0be00700 1 --2- 172.21.15.115:0/3774080892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074d80 0x7f2b040731e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.326+0000 7f2b0be00700 1 -- 172.21.15.115:0/3774080892 >> 172.21.15.115:0/3774080892 conn(0x7f2b040fb6e0 msgr2=0x7f2b040fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.327+0000 7f2b0be00700 1 -- 172.21.15.115:0/3774080892 shutdown_connections 2024-03-22T10:42:34.320 INFO:tasks.workunit.client.0.smithi115.stdout:6/120: truncate d4/d11/f1f 4678601 0 2024-03-22T10:42:34.321 INFO:tasks.workunit.client.0.smithi115.stdout:1/199: symlink d0/d12/l43 0 2024-03-22T10:42:34.322 INFO:tasks.workunit.client.0.smithi115.stdout:6/121: chown d4/d13 3795733 1 2024-03-22T10:42:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.328+0000 7f2b0be00700 1 -- 172.21.15.115:0/3774080892 wait complete. 2024-03-22T10:42:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.329+0000 7f2b0be00700 1 Processor -- start 2024-03-22T10:42:34.323 INFO:tasks.workunit.client.0.smithi115.stdout:4/146: truncate d1/f7 1443140 0 2024-03-22T10:42:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.332+0000 7f2b0be00700 1 -- start start 2024-03-22T10:42:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.333+0000 7f2b0be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b040737b0 0x7f2b04071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.334+0000 7f2b0be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074d80 0x7f2b04072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.334+0000 7f2b0be00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b04072620 con 0x7f2b04074d80 2024-03-22T10:42:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.334+0000 7f2b0be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b04072760 con 0x7f2b040737b0 2024-03-22T10:42:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.335+0000 7f2b09400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b040737b0 0x7f2b04071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.335+0000 7f2b09400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b040737b0 0x7f2b04071ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:54570/0 (socket says 172.21.15.115:54570) 2024-03-22T10:42:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.336+0000 7f2b09400700 1 -- 172.21.15.115:0/1008944598 learned_addr learned my addr 172.21.15.115:0/1008944598 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.336+0000 7f2b08a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074d80 0x7f2b04072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.337+0000 7f2b08a00700 1 -- 172.21.15.115:0/1008944598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b040737b0 msgr2=0x7f2b04071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.337+0000 7f2b08a00700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b040737b0 0x7f2b04071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.337+0000 7f2b08a00700 1 -- 172.21.15.115:0/1008944598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80095f0 con 0x7f2b04074d80 2024-03-22T10:42:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.339+0000 7f2b08a00700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074d80 0x7f2b04072000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2af000d8d0 tx=0x7f2af000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:34.331 INFO:tasks.workunit.client.0.smithi115.stdout:3/83: dwrite d3/f7 [0,4194304] 0 2024-03-22T10:42:34.331 INFO:tasks.workunit.client.0.smithi115.stdout:6/122: chown d4/ld 1304387 1 2024-03-22T10:42:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.341+0000 7f2b02000700 1 -- 172.21.15.115:0/1008944598 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af000f860 con 0x7f2b04074d80 2024-03-22T10:42:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.341+0000 7f2b02000700 1 -- 172.21.15.115:0/1008944598 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2af000fe70 con 0x7f2b04074d80 2024-03-22T10:42:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.341+0000 7f2b0be00700 1 -- 172.21.15.115:0/1008944598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b041a9d10 con 0x7f2b04074d80 2024-03-22T10:42:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.341+0000 7f2b0be00700 1 -- 172.21.15.115:0/1008944598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b041aa260 con 0x7f2b04074d80 2024-03-22T10:42:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.342+0000 7f2b02000700 1 -- 172.21.15.115:0/1008944598 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af000e5e0 con 0x7f2b04074d80 2024-03-22T10:42:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.343+0000 7f2af7400700 1 -- 172.21.15.115:0/1008944598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0410f280 con 0x7f2b04074d80 2024-03-22T10:42:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.345+0000 7f2b02000700 1 -- 172.21.15.115:0/1008944598 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 29) v1 ==== 46247+0+0 (secure 0 0 0) 0x7f2af000f9c0 con 0x7f2b04074d80 2024-03-22T10:42:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.346+0000 7f2b02000700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2aec039280 0x7f2aec03b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.346+0000 7f2b02000700 1 -- 172.21.15.115:0/1008944598 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f2af004d3c0 con 0x7f2b04074d80 2024-03-22T10:42:34.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.347+0000 7f2b09400700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2aec039280 0x7f2aec03b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:34.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.349+0000 7f2b09400700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2aec039280 0x7f2aec03b730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2af8005090 tx=0x7f2af801a040 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:34.344 INFO:tasks.workunit.client.0.smithi115.stdout:9/155: dwrite f3 [4194304,4194304] 0 2024-03-22T10:42:34.344 INFO:tasks.workunit.client.0.smithi115.stdout:1/200: creat d0/d8/db/d25/f44 x:0 0 0 2024-03-22T10:42:34.348 INFO:tasks.workunit.client.0.smithi115.stdout:7/172: link f1 d2/d7/d12/d10/d19/f37 0 2024-03-22T10:42:34.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.356+0000 7f2b02000700 1 -- 172.21.15.115:0/1008944598 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f2af0024ba0 con 0x7f2b04074d80 2024-03-22T10:42:34.348 INFO:tasks.workunit.client.0.smithi115.stdout:9/156: write f5 [2039280,97573] 0 2024-03-22T10:42:34.349 INFO:tasks.workunit.client.0.smithi115.stdout:5/115: dwrite d2/fe [0,4194304] 0 2024-03-22T10:42:34.352 INFO:tasks.workunit.client.0.smithi115.stdout:9/157: write d6/d8/fd [628348,36815] 0 2024-03-22T10:42:34.354 INFO:tasks.workunit.client.0.smithi115.stdout:6/123: creat d4/d13/f25 x:0 0 0 2024-03-22T10:42:34.356 INFO:tasks.workunit.client.0.smithi115.stdout:4/147: dread d1/f9 [0,4194304] 0 2024-03-22T10:42:34.356 INFO:tasks.workunit.client.0.smithi115.stdout:3/84: mkdir d3/d19 0 2024-03-22T10:42:34.360 INFO:tasks.workunit.client.0.smithi115.stdout:8/288: dread d6/dd/d12/d31/f5c [0,4194304] 0 2024-03-22T10:42:34.365 INFO:tasks.workunit.client.0.smithi115.stdout:1/201: rmdir d0 39 2024-03-22T10:42:34.366 INFO:tasks.workunit.client.0.smithi115.stdout:5/116: symlink d2/d15/l22 0 2024-03-22T10:42:34.368 INFO:tasks.workunit.client.0.smithi115.stdout:5/117: readlink d2/d15/d1b/l20 0 2024-03-22T10:42:34.376 INFO:tasks.workunit.client.0.smithi115.stdout:9/158: creat d6/d7/d19/f31 x:0 0 0 2024-03-22T10:42:34.385 INFO:tasks.workunit.client.0.smithi115.stdout:1/202: chown d0/d12 411390 1 2024-03-22T10:42:34.387 INFO:tasks.workunit.client.0.smithi115.stdout:8/289: dread d6/dd/d12/d31/f5c [0,4194304] 0 2024-03-22T10:42:34.388 INFO:tasks.workunit.client.0.smithi115.stdout:3/85: dwrite d3/f8 [12582912,4194304] 0 2024-03-22T10:42:34.392 INFO:tasks.workunit.client.0.smithi115.stdout:6/124: mknod d4/c26 0 2024-03-22T10:42:34.402 INFO:tasks.workunit.client.0.smithi115.stdout:5/118: dread d2/d15/f1c [0,4194304] 0 2024-03-22T10:42:34.408 INFO:tasks.workunit.client.0.smithi115.stdout:9/159: mknod d6/d7/d30/c32 0 2024-03-22T10:42:34.409 INFO:tasks.workunit.client.0.smithi115.stdout:8/290: creat d6/dd/d24/d4a/f5f x:0 0 0 2024-03-22T10:42:34.409 INFO:tasks.workunit.client.0.smithi115.stdout:1/203: symlink d0/d8/de/l45 0 2024-03-22T10:42:34.410 INFO:tasks.workunit.client.0.smithi115.stdout:3/86: creat d3/d19/f1a x:0 0 0 2024-03-22T10:42:34.411 INFO:tasks.workunit.client.0.smithi115.stdout:1/204: readlink d0/d8/ld 0 2024-03-22T10:42:34.411 INFO:tasks.workunit.client.0.smithi115.stdout:6/125: dread d4/d11/d12/f21 [0,4194304] 0 2024-03-22T10:42:34.413 INFO:tasks.workunit.client.0.smithi115.stdout:5/119: creat d2/df/d13/f23 x:0 0 0 2024-03-22T10:42:34.414 INFO:tasks.workunit.client.0.smithi115.stdout:6/126: chown d4/f5 41 1 2024-03-22T10:42:34.416 INFO:tasks.workunit.client.0.smithi115.stdout:1/205: creat d0/d22/f46 x:0 0 0 2024-03-22T10:42:34.416 INFO:tasks.workunit.client.0.smithi115.stdout:5/120: chown d2/d15/c19 276650 1 2024-03-22T10:42:34.416 INFO:tasks.workunit.client.0.smithi115.stdout:3/87: mkdir d3/d19/d1b 0 2024-03-22T10:42:34.420 INFO:tasks.workunit.client.0.smithi115.stdout:8/291: link d6/dd/d12/c2a d6/d49/c60 0 2024-03-22T10:42:34.422 INFO:tasks.workunit.client.0.smithi115.stdout:5/121: fsync d2/df/d13/f16 0 2024-03-22T10:42:34.422 INFO:tasks.workunit.client.0.smithi115.stdout:1/206: truncate d0/d12/d17/f42 263939 0 2024-03-22T10:42:34.424 INFO:tasks.workunit.client.0.smithi115.stdout:8/292: read f5 [414983,1217] 0 2024-03-22T10:42:34.429 INFO:tasks.workunit.client.0.smithi115.stdout:4/148: dwrite d1/f9 [0,4194304] 0 2024-03-22T10:42:34.430 INFO:tasks.workunit.client.0.smithi115.stdout:1/207: symlink d0/d12/d2b/l47 0 2024-03-22T10:42:34.433 INFO:tasks.workunit.client.0.smithi115.stdout:1/208: readlink d0/d8/db/l21 0 2024-03-22T10:42:34.435 INFO:tasks.workunit.client.0.smithi115.stdout:8/293: readlink l3 0 2024-03-22T10:42:34.441 INFO:tasks.workunit.client.0.smithi115.stdout:1/209: write d0/d12/d17/d37/f3c [287728,108140] 0 2024-03-22T10:42:34.441 INFO:tasks.workunit.client.0.smithi115.stdout:8/294: chown d6/dd/d12/d31 0 1 2024-03-22T10:42:34.444 INFO:tasks.workunit.client.0.smithi115.stdout:6/127: dwrite d4/d11/f23 [0,4194304] 0 2024-03-22T10:42:34.448 INFO:tasks.workunit.client.0.smithi115.stdout:5/122: dread d2/df/d13/f18 [0,4194304] 0 2024-03-22T10:42:34.448 INFO:tasks.workunit.client.0.smithi115.stdout:6/128: fsync d4/f24 0 2024-03-22T10:42:34.450 INFO:tasks.workunit.client.0.smithi115.stdout:4/149: mkdir d1/d30 0 2024-03-22T10:42:34.451 INFO:tasks.workunit.client.0.smithi115.stdout:5/123: stat d2/df/d13 0 2024-03-22T10:42:34.455 INFO:tasks.workunit.client.0.smithi115.stdout:8/295: creat d6/dd/d24/d4a/f61 x:0 0 0 2024-03-22T10:42:34.456 INFO:tasks.workunit.client.0.smithi115.stdout:5/124: rename d2/d15/d1b to d2/d15/d1b/d24 22 2024-03-22T10:42:34.456 INFO:tasks.workunit.client.0.smithi115.stdout:6/129: fdatasync d4/d11/d12/f21 0 2024-03-22T10:42:34.459 INFO:tasks.workunit.client.0.smithi115.stdout:3/88: dwrite d3/f11 [0,4194304] 0 2024-03-22T10:42:34.463 INFO:tasks.workunit.client.0.smithi115.stdout:8/296: write d6/dd/d24/f3f [1144604,23176] 0 2024-03-22T10:42:34.465 INFO:tasks.workunit.client.0.smithi115.stdout:3/89: write d3/f9 [4994904,107878] 0 2024-03-22T10:42:34.465 INFO:tasks.workunit.client.0.smithi115.stdout:8/297: fdatasync d6/dd/f26 0 2024-03-22T10:42:34.467 INFO:tasks.workunit.client.0.smithi115.stdout:3/90: write d3/f8 [7490026,12008] 0 2024-03-22T10:42:34.467 INFO:tasks.workunit.client.0.smithi115.stdout:1/210: dread d0/d8/de/d1a/f32 [0,4194304] 0 2024-03-22T10:42:34.468 INFO:tasks.workunit.client.0.smithi115.stdout:6/130: unlink d4/cc 0 2024-03-22T10:42:34.470 INFO:tasks.workunit.client.0.smithi115.stdout:6/131: fdatasync d4/d11/d12/f21 0 2024-03-22T10:42:34.475 INFO:tasks.workunit.client.0.smithi115.stdout:4/150: rmdir d1/d2b 0 2024-03-22T10:42:34.478 INFO:tasks.workunit.client.0.smithi115.stdout:5/125: fdatasync d2/df/d13/f18 0 2024-03-22T10:42:34.482 INFO:tasks.workunit.client.0.smithi115.stdout:3/91: rename d3/f15 to d3/d16/f1c 0 2024-03-22T10:42:34.483 INFO:tasks.workunit.client.0.smithi115.stdout:5/126: write d2/df/d13/f18 [1880341,1650] 0 2024-03-22T10:42:34.493 INFO:tasks.workunit.client.0.smithi115.stdout:4/151: dwrite d1/f9 [0,4194304] 0 2024-03-22T10:42:34.509 INFO:tasks.workunit.client.0.smithi115.stdout:8/298: creat d6/dd/d24/d4a/d41/f62 x:0 0 0 2024-03-22T10:42:34.512 INFO:tasks.workunit.client.0.smithi115.stdout:8/299: read d6/fc [732423,24419] 0 2024-03-22T10:42:34.516 INFO:tasks.workunit.client.0.smithi115.stdout:4/152: dwrite d1/dc/f25 [0,4194304] 0 2024-03-22T10:42:34.517 INFO:tasks.workunit.client.0.smithi115.stdout:3/92: creat d3/d19/f1d x:0 0 0 2024-03-22T10:42:34.525 INFO:tasks.workunit.client.0.smithi115.stdout:1/211: link d0/d8/f9 d0/d12/f48 0 2024-03-22T10:42:34.528 INFO:tasks.workunit.client.0.smithi115.stdout:1/212: dread - d0/d8/de/f26 zero size 2024-03-22T10:42:34.529 INFO:tasks.workunit.client.0.smithi115.stdout:1/213: dread - d0/d12/d17/f31 zero size 2024-03-22T10:42:34.535 INFO:tasks.workunit.client.0.smithi115.stdout:4/153: unlink d1/d16/l27 0 2024-03-22T10:42:34.537 INFO:tasks.workunit.client.0.smithi115.stdout:3/93: rename d3/l17 to d3/d16/l1e 0 2024-03-22T10:42:34.539 INFO:tasks.workunit.client.0.smithi115.stdout:5/127: dwrite d2/fc [0,4194304] 0 2024-03-22T10:42:34.541 INFO:tasks.workunit.client.0.smithi115.stdout:5/128: fsync d2/df/d13/f16 0 2024-03-22T10:42:34.546 INFO:tasks.workunit.client.0.smithi115.stdout:1/214: creat d0/d12/d17/d37/f49 x:0 0 0 2024-03-22T10:42:34.549 INFO:tasks.workunit.client.0.smithi115.stdout:6/132: getdents d4/d13 0 2024-03-22T10:42:34.551 INFO:tasks.workunit.client.0.smithi115.stdout:5/129: symlink d2/df/d13/l25 0 2024-03-22T10:42:34.553 INFO:tasks.workunit.client.0.smithi115.stdout:1/215: creat d0/d8/de/d1a/f4a x:0 0 0 2024-03-22T10:42:34.556 INFO:tasks.workunit.client.0.smithi115.stdout:1/216: symlink d0/d12/l4b 0 2024-03-22T10:42:34.559 INFO:tasks.workunit.client.0.smithi115.stdout:5/130: symlink d2/df/l26 0 2024-03-22T10:42:34.560 INFO:tasks.workunit.client.0.smithi115.stdout:6/133: rename d4/l8 to d4/d11/d12/l27 0 2024-03-22T10:42:34.562 INFO:tasks.workunit.client.0.smithi115.stdout:6/134: stat d4/f5 0 2024-03-22T10:42:34.565 INFO:tasks.workunit.client.0.smithi115.stdout:3/94: dwrite d3/f11 [4194304,4194304] 0 2024-03-22T10:42:34.565 INFO:tasks.workunit.client.0.smithi115.stdout:5/131: chown d2/d15/c1d 296930905 1 2024-03-22T10:42:34.569 INFO:tasks.workunit.client.0.smithi115.stdout:4/154: dread d1/ff [0,4194304] 0 2024-03-22T10:42:34.572 INFO:tasks.workunit.client.0.smithi115.stdout:8/300: dwrite d6/dd/f2e [0,4194304] 0 2024-03-22T10:42:34.573 INFO:tasks.workunit.client.0.smithi115.stdout:3/95: mknod d3/c1f 0 2024-03-22T10:42:34.580 INFO:tasks.workunit.client.0.smithi115.stdout:3/96: symlink d3/l20 0 2024-03-22T10:42:34.581 INFO:tasks.workunit.client.0.smithi115.stdout:8/301: dread - d6/dd/d24/d4a/d41/f62 zero size 2024-03-22T10:42:34.584 INFO:tasks.workunit.client.0.smithi115.stdout:6/135: rename d4/d11/d12/c1a to d4/c28 0 2024-03-22T10:42:34.586 INFO:tasks.workunit.client.0.smithi115.stdout:3/97: write d3/d16/f18 [431427,34193] 0 2024-03-22T10:42:34.587 INFO:tasks.workunit.client.0.smithi115.stdout:1/217: dread d0/d8/f9 [0,4194304] 0 2024-03-22T10:42:34.587 INFO:tasks.workunit.client.0.smithi115.stdout:6/136: fdatasync d4/f5 0 2024-03-22T10:42:34.593 INFO:tasks.workunit.client.0.smithi115.stdout:5/132: dwrite d2/fe [0,4194304] 0 2024-03-22T10:42:34.593 INFO:tasks.workunit.client.0.smithi115.stdout:3/98: unlink d3/l20 0 2024-03-22T10:42:34.596 INFO:tasks.workunit.client.0.smithi115.stdout:1/218: chown d0/d12/d17/d37/f3c 8511 1 2024-03-22T10:42:34.597 INFO:tasks.workunit.client.0.smithi115.stdout:3/99: fsync d3/f9 0 2024-03-22T10:42:34.598 INFO:tasks.workunit.client.0.smithi115.stdout:6/137: creat d4/d11/f29 x:0 0 0 2024-03-22T10:42:34.608 INFO:tasks.workunit.client.0.smithi115.stdout:4/155: dwrite d1/dc/d14/f1b [4194304,4194304] 0 2024-03-22T10:42:34.609 INFO:tasks.workunit.client.0.smithi115.stdout:6/138: creat d4/d11/f2a x:0 0 0 2024-03-22T10:42:34.625 INFO:tasks.workunit.client.0.smithi115.stdout:1/219: dread d0/d8/f9 [0,4194304] 0 2024-03-22T10:42:34.628 INFO:tasks.workunit.client.0.smithi115.stdout:5/133: dwrite d2/d15/f1c [0,4194304] 0 2024-03-22T10:42:34.632 INFO:tasks.workunit.client.0.smithi115.stdout:8/302: dwrite d6/dd/d24/d3a/f3c [0,4194304] 0 2024-03-22T10:42:34.632 INFO:tasks.workunit.client.0.smithi115.stdout:6/139: dread d4/d11/f23 [0,4194304] 0 2024-03-22T10:42:34.633 INFO:tasks.workunit.client.0.smithi115.stdout:1/220: write d0/d12/d17/f31 [967745,27415] 0 2024-03-22T10:42:34.634 INFO:tasks.workunit.client.0.smithi115.stdout:5/134: mkdir d2/df/d27 0 2024-03-22T10:42:34.638 INFO:tasks.workunit.client.0.smithi115.stdout:4/156: dread d1/dc/d14/f1b [4194304,4194304] 0 2024-03-22T10:42:34.640 INFO:tasks.workunit.client.0.smithi115.stdout:1/221: write d0/d8/db/d25/f44 [607020,18827] 0 2024-03-22T10:42:34.641 INFO:tasks.workunit.client.0.smithi115.stdout:8/303: write d6/f16 [1663270,70266] 0 2024-03-22T10:42:34.642 INFO:tasks.workunit.client.0.smithi115.stdout:1/222: readlink d0/d12/d2b/l47 0 2024-03-22T10:42:34.644 INFO:tasks.workunit.client.0.smithi115.stdout:8/304: chown d6/dd/l58 509231 1 2024-03-22T10:42:34.646 INFO:tasks.workunit.client.0.smithi115.stdout:1/223: chown d0/d8/db/d25/f44 13 1 2024-03-22T10:42:34.647 INFO:tasks.workunit.client.0.smithi115.stdout:3/100: truncate d3/f7 3378404 0 2024-03-22T10:42:34.647 INFO:tasks.workunit.client.0.smithi115.stdout:4/157: creat d1/dc/d17/f31 x:0 0 0 2024-03-22T10:42:34.649 INFO:tasks.workunit.client.0.smithi115.stdout:6/140: mknod d4/c2b 0 2024-03-22T10:42:34.650 INFO:tasks.workunit.client.0.smithi115.stdout:8/305: chown d6/dd/d24/f3f 22 1 2024-03-22T10:42:34.650 INFO:tasks.workunit.client.0.smithi115.stdout:3/101: fdatasync d3/d19/f1a 0 2024-03-22T10:42:34.652 INFO:tasks.workunit.client.0.smithi115.stdout:6/141: readlink d4/ld 0 2024-03-22T10:42:34.652 INFO:tasks.workunit.client.0.smithi115.stdout:4/158: write d1/dc/d17/f31 [384863,71605] 0 2024-03-22T10:42:34.658 INFO:tasks.workunit.client.0.smithi115.stdout:5/135: dread d2/d15/f1c [0,4194304] 0 2024-03-22T10:42:34.659 INFO:tasks.workunit.client.0.smithi115.stdout:3/102: rename d3/f11 to d3/d16/f21 0 2024-03-22T10:42:34.661 INFO:tasks.workunit.client.0.smithi115.stdout:6/142: write d4/d11/d12/f21 [125627,21112] 0 2024-03-22T10:42:34.663 INFO:tasks.workunit.client.0.smithi115.stdout:3/103: chown d3/d16/f18 8410 1 2024-03-22T10:42:34.663 INFO:tasks.workunit.client.0.smithi115.stdout:4/159: mkdir d1/dc/d17/d32 0 2024-03-22T10:42:34.663 INFO:tasks.workunit.client.0.smithi115.stdout:6/143: fsync d4/d11/f1f 0 2024-03-22T10:42:34.666 INFO:tasks.workunit.client.0.smithi115.stdout:5/136: mknod d2/d15/d1b/c28 0 2024-03-22T10:42:34.669 INFO:tasks.workunit.client.0.smithi115.stdout:5/137: fsync d2/fc 0 2024-03-22T10:42:34.670 INFO:tasks.workunit.client.0.smithi115.stdout:3/104: write d3/d16/f18 [1218166,27960] 0 2024-03-22T10:42:34.673 INFO:tasks.workunit.client.0.smithi115.stdout:4/160: truncate d1/dc/f20 413547 0 2024-03-22T10:42:34.678 INFO:tasks.workunit.client.0.smithi115.stdout:1/224: dread d0/d12/d17/f3e [0,4194304] 0 2024-03-22T10:42:34.681 INFO:tasks.workunit.client.0.smithi115.stdout:1/225: readlink d0/d8/ld 0 2024-03-22T10:42:34.681 INFO:tasks.workunit.client.0.smithi115.stdout:5/138: dread d2/df/d13/f16 [0,4194304] 0 2024-03-22T10:42:34.684 INFO:tasks.workunit.client.0.smithi115.stdout:4/161: dread d1/ff [0,4194304] 0 2024-03-22T10:42:34.684 INFO:tasks.workunit.client.0.smithi115.stdout:5/139: chown d2/d15/l22 22877543 1 2024-03-22T10:42:34.687 INFO:tasks.workunit.client.0.smithi115.stdout:1/226: write d0/d8/f9 [3260506,81129] 0 2024-03-22T10:42:34.688 INFO:tasks.workunit.client.0.smithi115.stdout:8/306: dread d6/dd/f33 [0,4194304] 0 2024-03-22T10:42:34.689 INFO:tasks.workunit.client.0.smithi115.stdout:5/140: mknod d2/df/d13/c29 0 2024-03-22T10:42:34.691 INFO:tasks.workunit.client.0.smithi115.stdout:1/227: write d0/f3 [3585263,43848] 0 2024-03-22T10:42:34.694 INFO:tasks.workunit.client.0.smithi115.stdout:1/228: fdatasync d0/d12/d17/f31 0 2024-03-22T10:42:34.695 INFO:tasks.workunit.client.0.smithi115.stdout:8/307: fsync d6/dd/d24/d4a/d41/f4b 0 2024-03-22T10:42:34.698 INFO:tasks.workunit.client.0.smithi115.stdout:8/308: fsync d6/dd/f29 0 2024-03-22T10:42:34.698 INFO:tasks.workunit.client.0.smithi115.stdout:6/144: dwrite d4/d11/d12/f21 [0,4194304] 0 2024-03-22T10:42:34.698 INFO:tasks.workunit.client.0.smithi115.stdout:5/141: write d2/f4 [5255304,8429] 0 2024-03-22T10:42:34.699 INFO:tasks.workunit.client.0.smithi115.stdout:8/309: fdatasync d6/f44 0 2024-03-22T10:42:34.701 INFO:tasks.workunit.client.0.smithi115.stdout:5/142: write d2/f1f [854557,90471] 0 2024-03-22T10:42:34.705 INFO:tasks.workunit.client.0.smithi115.stdout:6/145: mknod d4/d11/d12/c2c 0 2024-03-22T10:42:34.705 INFO:tasks.workunit.client.0.smithi115.stdout:3/105: dread d3/f7 [0,4194304] 0 2024-03-22T10:42:34.706 INFO:tasks.workunit.client.0.smithi115.stdout:8/310: mknod d6/dd/d24/d4a/d41/c63 0 2024-03-22T10:42:34.710 INFO:tasks.workunit.client.0.smithi115.stdout:5/143: creat d2/df/d1e/f2a x:0 0 0 2024-03-22T10:42:34.712 INFO:tasks.workunit.client.0.smithi115.stdout:4/162: dwrite d1/dc/f25 [4194304,4194304] 0 2024-03-22T10:42:34.714 INFO:tasks.workunit.client.0.smithi115.stdout:1/229: creat d0/d8/de/f4c x:0 0 0 2024-03-22T10:42:34.714 INFO:tasks.workunit.client.0.smithi115.stdout:8/311: symlink d6/dd/d12/d31/l64 0 2024-03-22T10:42:34.715 INFO:tasks.workunit.client.0.smithi115.stdout:1/230: stat d0/d8/de/l45 0 2024-03-22T10:42:34.718 INFO:tasks.workunit.client.0.smithi115.stdout:2/111: sync 2024-03-22T10:42:34.721 INFO:tasks.workunit.client.0.smithi115.stdout:2/112: chown c1 861869 1 2024-03-22T10:42:34.721 INFO:tasks.workunit.client.0.smithi115.stdout:6/146: creat d4/d11/d12/d1c/f2d x:0 0 0 2024-03-22T10:42:34.722 INFO:tasks.workunit.client.0.smithi115.stdout:2/113: truncate da/f16 4291321 0 2024-03-22T10:42:34.724 INFO:tasks.workunit.client.0.smithi115.stdout:6/147: dread - d4/d13/f25 zero size 2024-03-22T10:42:34.731 INFO:tasks.workunit.client.0.smithi115.stdout:2/114: write da/f12 [49554,71508] 0 2024-03-22T10:42:34.733 INFO:tasks.workunit.client.0.smithi115.stdout:3/106: creat d3/d19/d1b/f22 x:0 0 0 2024-03-22T10:42:34.736 INFO:tasks.workunit.client.0.smithi115.stdout:2/115: readlink da/l11 0 2024-03-22T10:42:34.736 INFO:tasks.workunit.client.0.smithi115.stdout:1/231: mkdir d0/d8/d4d 0 2024-03-22T10:42:34.739 INFO:tasks.workunit.client.0.smithi115.stdout:6/148: creat d4/d13/f2e x:0 0 0 2024-03-22T10:42:34.741 INFO:tasks.workunit.client.0.smithi115.stdout:8/312: dread d6/dd/f33 [0,4194304] 0 2024-03-22T10:42:34.744 INFO:tasks.workunit.client.0.smithi115.stdout:2/116: mkdir da/d1c 0 2024-03-22T10:42:34.745 INFO:tasks.workunit.client.0.smithi115.stdout:2/117: stat da/c10 0 2024-03-22T10:42:34.746 INFO:tasks.workunit.client.0.smithi115.stdout:1/232: write d0/d12/d17/d37/f3c [1216220,17286] 0 2024-03-22T10:42:34.751 INFO:tasks.workunit.client.0.smithi115.stdout:6/149: chown d4/d13/f2e 58 1 2024-03-22T10:42:34.754 INFO:tasks.workunit.client.0.smithi115.stdout:1/233: chown d0/d12/d17/f23 583651 1 2024-03-22T10:42:34.756 INFO:tasks.workunit.client.0.smithi115.stdout:4/163: dwrite d1/dc/d17/f31 [0,4194304] 0 2024-03-22T10:42:34.756 INFO:tasks.workunit.client.0.smithi115.stdout:1/234: chown d0/d12/d17/d37 973 1 2024-03-22T10:42:34.756 INFO:tasks.workunit.client.0.smithi115.stdout:6/150: mknod d4/d11/d12/c2f 0 2024-03-22T10:42:34.758 INFO:tasks.workunit.client.0.smithi115.stdout:5/144: dread d2/f11 [0,4194304] 0 2024-03-22T10:42:34.761 INFO:tasks.workunit.client.0.smithi115.stdout:1/235: write d0/f7 [5395315,18543] 0 2024-03-22T10:42:34.766 INFO:tasks.workunit.client.0.smithi115.stdout:4/164: mkdir d1/dc/d14/d33 0 2024-03-22T10:42:34.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.776+0000 7f2af7400700 1 -- 172.21.15.115:0/1008944598 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b04061d80 con 0x7f2aec039280 2024-03-22T10:42:34.770 INFO:tasks.workunit.client.0.smithi115.stdout:5/145: mkdir d2/df/d1e/d2b 0 2024-03-22T10:42:34.772 INFO:tasks.workunit.client.0.smithi115.stdout:4/165: write d1/dc/d17/f31 [2573062,14022] 0 2024-03-22T10:42:34.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.779+0000 7f2b02000700 1 -- 172.21.15.115:0/1008944598 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f2b04061d80 con 0x7f2aec039280 2024-03-22T10:42:34.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.782+0000 7f2af7400700 1 -- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2aec039280 msgr2=0x7f2aec03b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:34.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.782+0000 7f2af7400700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2aec039280 0x7f2aec03b730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2af8005090 tx=0x7f2af801a040 comp rx=0 tx=0).stop 2024-03-22T10:42:34.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.782+0000 7f2af7400700 1 -- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074d80 msgr2=0x7f2b04072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:34.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.782+0000 7f2af7400700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074d80 0x7f2b04072000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2af000d8d0 tx=0x7f2af000dbe0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.782+0000 7f2af7400700 1 -- 172.21.15.115:0/1008944598 shutdown_connections 2024-03-22T10:42:34.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.782+0000 7f2af7400700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2aec039280 0x7f2aec03b730 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.782+0000 7f2af7400700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b040737b0 0x7f2b04071ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.782+0000 7f2af7400700 1 --2- 172.21.15.115:0/1008944598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b04074d80 0x7f2b04072000 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:34.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.782+0000 7f2af7400700 1 -- 172.21.15.115:0/1008944598 >> 172.21.15.115:0/1008944598 conn(0x7f2b040fb6e0 msgr2=0x7f2b040fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:34.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.784+0000 7f2af7400700 1 -- 172.21.15.115:0/1008944598 shutdown_connections 2024-03-22T10:42:34.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:34.784+0000 7f2af7400700 1 -- 172.21.15.115:0/1008944598 wait complete. 2024-03-22T10:42:34.778 INFO:tasks.workunit.client.0.smithi115.stdout:6/151: creat d4/f30 x:0 0 0 2024-03-22T10:42:34.778 INFO:tasks.workunit.client.0.smithi115.stdout:5/146: write d2/fc [3391110,101545] 0 2024-03-22T10:42:34.783 INFO:tasks.workunit.client.0.smithi115.stdout:3/107: dread d3/f8 [0,4194304] 0 2024-03-22T10:42:34.786 INFO:tasks.workunit.client.0.smithi115.stdout:8/313: dwrite d6/dd/d24/d4a/f51 [0,4194304] 0 2024-03-22T10:42:34.786 INFO:tasks.workunit.client.0.smithi115.stdout:3/108: read d3/f7 [2742928,70701] 0 2024-03-22T10:42:34.792 INFO:tasks.workunit.client.0.smithi115.stdout:3/109: chown d3/c14 0 1 2024-03-22T10:42:34.794 INFO:tasks.workunit.client.0.smithi115.stdout:1/236: dwrite d0/d22/f46 [0,4194304] 0 2024-03-22T10:42:34.797 INFO:tasks.workunit.client.0.smithi115.stdout:1/237: write d0/f14 [4605183,17726] 0 2024-03-22T10:42:34.801 INFO:tasks.workunit.client.0.smithi115.stdout:0/141: sync 2024-03-22T10:42:34.801 INFO:tasks.workunit.client.0.smithi115.stdout:9/160: sync 2024-03-22T10:42:34.802 INFO:tasks.workunit.client.0.smithi115.stdout:7/173: sync 2024-03-22T10:42:34.802 INFO:tasks.workunit.client.0.smithi115.stdout:9/161: chown d6/d7/d19/f21 0 1 2024-03-22T10:42:34.802 INFO:tasks.workunit.client.0.smithi115.stdout:8/314: symlink d6/dd/d24/d4a/l65 0 2024-03-22T10:42:34.803 INFO:tasks.workunit.client.0.smithi115.stdout:0/142: write d3/d4/d14/f23 [2264665,77618] 0 2024-03-22T10:42:34.805 INFO:tasks.workunit.client.0.smithi115.stdout:3/110: creat d3/f23 x:0 0 0 2024-03-22T10:42:34.807 INFO:tasks.workunit.client.0.smithi115.stdout:7/174: write d2/f5 [4506197,99775] 0 2024-03-22T10:42:34.807 INFO:tasks.workunit.client.0.smithi115.stdout:8/315: stat d6/dd/d24/d3a 0 2024-03-22T10:42:34.808 INFO:tasks.workunit.client.0.smithi115.stdout:4/166: getdents d1/d16 0 2024-03-22T10:42:34.828 INFO:tasks.workunit.client.0.smithi115.stdout:8/316: chown d6/f17 1 1 2024-03-22T10:42:34.829 INFO:tasks.workunit.client.0.smithi115.stdout:1/238: rename d0/d8/db/c10 to d0/d8/de/d1a/c4e 0 2024-03-22T10:42:34.830 INFO:tasks.workunit.client.0.smithi115.stdout:3/111: truncate d3/d19/d1b/f22 840004 0 2024-03-22T10:42:34.833 INFO:tasks.workunit.client.0.smithi115.stdout:9/162: mknod d6/d7/d19/c33 0 2024-03-22T10:42:34.833 INFO:tasks.workunit.client.0.smithi115.stdout:8/317: write d6/dd/d24/d3a/f54 [729784,81663] 0 2024-03-22T10:42:34.834 INFO:tasks.workunit.client.0.smithi115.stdout:3/112: write d3/f9 [538760,126286] 0 2024-03-22T10:42:34.834 INFO:tasks.workunit.client.0.smithi115.stdout:5/147: link d2/l9 d2/d15/d1b/l2c 0 2024-03-22T10:42:34.836 INFO:tasks.workunit.client.0.smithi115.stdout:8/318: chown d6/dd/d24/l47 52494 1 2024-03-22T10:42:34.836 INFO:tasks.workunit.client.0.smithi115.stdout:3/113: dread - d3/d19/f1a zero size 2024-03-22T10:42:34.836 INFO:tasks.workunit.client.0.smithi115.stdout:0/143: symlink d3/d4/d14/l28 0 2024-03-22T10:42:34.836 INFO:tasks.workunit.client.0.smithi115.stdout:4/167: creat d1/dc/d17/f34 x:0 0 0 2024-03-22T10:42:34.837 INFO:tasks.workunit.client.0.smithi115.stdout:7/175: creat d2/d7/d12/d10/d23/f38 x:0 0 0 2024-03-22T10:42:34.839 INFO:tasks.workunit.client.0.smithi115.stdout:3/114: chown d3/d19/f1d 726454786 1 2024-03-22T10:42:34.840 INFO:tasks.workunit.client.0.smithi115.stdout:6/152: getdents d4/d13 0 2024-03-22T10:42:34.844 INFO:tasks.workunit.client.0.smithi115.stdout:4/168: mkdir d1/d16/d35 0 2024-03-22T10:42:34.847 INFO:tasks.workunit.client.0.smithi115.stdout:0/144: symlink d3/d7/d1b/l29 0 2024-03-22T10:42:34.849 INFO:tasks.workunit.client.0.smithi115.stdout:7/176: mkdir d2/d7/d12/d10/d23/d39 0 2024-03-22T10:42:34.854 INFO:tasks.workunit.client.0.smithi115.stdout:8/319: dwrite d6/dd/d12/f1d [0,4194304] 0 2024-03-22T10:42:34.855 INFO:tasks.workunit.client.0.smithi115.stdout:5/148: unlink d2/c8 0 2024-03-22T10:42:34.856 INFO:tasks.workunit.client.0.smithi115.stdout:4/169: mkdir d1/d16/d36 0 2024-03-22T10:42:34.859 INFO:tasks.workunit.client.0.smithi115.stdout:0/145: unlink d3/fc 0 2024-03-22T10:42:34.860 INFO:tasks.workunit.client.0.smithi115.stdout:7/177: mkdir d2/d3a 0 2024-03-22T10:42:34.860 INFO:tasks.workunit.client.0.smithi115.stdout:0/146: chown d3/l12 74 1 2024-03-22T10:42:34.864 INFO:tasks.workunit.client.0.smithi115.stdout:7/178: read d2/d7/d12/d10/f17 [4667687,40480] 0 2024-03-22T10:42:34.864 INFO:tasks.workunit.client.0.smithi115.stdout:9/163: write d6/d7/f17 [3993809,49056] 0 2024-03-22T10:42:34.866 INFO:tasks.workunit.client.0.smithi115.stdout:9/164: chown d6/d8/l18 9167 1 2024-03-22T10:42:34.869 INFO:tasks.workunit.client.0.smithi115.stdout:9/165: write d6/d7/d19/f21 [1351200,60683] 0 2024-03-22T10:42:34.876 INFO:tasks.workunit.client.0.smithi115.stdout:7/179: creat d2/d7/d12/d10/d23/d33/f3b x:0 0 0 2024-03-22T10:42:34.880 INFO:tasks.workunit.client.0.smithi115.stdout:4/170: rename d1/dc/d17/l22 to d1/dc/d14/d33/l37 0 2024-03-22T10:42:34.881 INFO:tasks.workunit.client.0.smithi115.stdout:6/153: getdents d4/d11/d12/d1c 0 2024-03-22T10:42:34.884 INFO:tasks.workunit.client.0.smithi115.stdout:3/115: getdents d3/d19/d1b 0 2024-03-22T10:42:34.887 INFO:tasks.workunit.client.0.smithi115.stdout:7/180: symlink d2/l3c 0 2024-03-22T10:42:34.887 INFO:tasks.workunit.client.0.smithi115.stdout:3/116: write d3/d19/f1a [999979,99263] 0 2024-03-22T10:42:34.888 INFO:tasks.workunit.client.0.smithi115.stdout:1/239: dwrite d0/d8/db/f16 [0,4194304] 0 2024-03-22T10:42:34.891 INFO:tasks.workunit.client.0.smithi115.stdout:9/166: rename d6/d7/c14 to d6/d7/d19/c34 0 2024-03-22T10:42:34.895 INFO:tasks.workunit.client.0.smithi115.stdout:4/171: mkdir d1/d30/d38 0 2024-03-22T10:42:34.896 INFO:tasks.workunit.client.0.smithi115.stdout:1/240: symlink d0/d12/l4f 0 2024-03-22T10:42:34.897 INFO:tasks.workunit.client.0.smithi115.stdout:8/320: write d6/dd/f11 [2789517,126746] 0 2024-03-22T10:42:34.899 INFO:tasks.workunit.client.0.smithi115.stdout:9/167: dread - d6/d7/d20/d22/f2b zero size 2024-03-22T10:42:34.899 INFO:tasks.workunit.client.0.smithi115.stdout:4/172: stat d1/dc/d17/f31 0 2024-03-22T10:42:34.902 INFO:tasks.workunit.client.0.smithi115.stdout:7/181: rename d2/d7/d11 to d2/d7/d12/d10/d23/d3d 0 2024-03-22T10:42:34.904 INFO:tasks.workunit.client.0.smithi115.stdout:5/149: truncate d2/fc 1968211 0 2024-03-22T10:42:34.906 INFO:tasks.workunit.client.0.smithi115.stdout:1/241: symlink d0/d8/de/d1a/l50 0 2024-03-22T10:42:34.907 INFO:tasks.workunit.client.0.smithi115.stdout:8/321: read d6/dd/d24/d3a/f54 [655242,73527] 0 2024-03-22T10:42:34.911 INFO:tasks.workunit.client.0.smithi115.stdout:9/168: symlink d6/d7/d20/l35 0 2024-03-22T10:42:34.913 INFO:tasks.workunit.client.0.smithi115.stdout:8/322: dread d6/dd/d12/f37 [0,4194304] 0 2024-03-22T10:42:34.916 INFO:tasks.workunit.client.0.smithi115.stdout:8/323: fsync d6/f28 0 2024-03-22T10:42:34.917 INFO:tasks.workunit.client.0.smithi115.stdout:1/242: rename d0/d8/f36 to d0/d12/f51 0 2024-03-22T10:42:34.921 INFO:tasks.workunit.client.0.smithi115.stdout:9/169: symlink d6/d7/d19/l36 0 2024-03-22T10:42:34.923 INFO:tasks.workunit.client.0.smithi115.stdout:1/243: write d0/d12/d17/f33 [730333,14924] 0 2024-03-22T10:42:34.928 INFO:tasks.workunit.client.0.smithi115.stdout:8/324: chown d6/dd/d24/l2d 20304 1 2024-03-22T10:42:34.928 INFO:tasks.workunit.client.0.smithi115.stdout:9/170: mkdir d6/d7/d30/d37 0 2024-03-22T10:42:34.933 INFO:tasks.workunit.client.0.smithi115.stdout:1/244: write d0/d22/f46 [4290122,1357] 0 2024-03-22T10:42:34.936 INFO:tasks.workunit.client.0.smithi115.stdout:8/325: readlink d6/dd/d24/l2d 0 2024-03-22T10:42:34.944 INFO:tasks.workunit.client.0.smithi115.stdout:7/182: dwrite d2/d7/d12/d10/f25 [4194304,4194304] 0 2024-03-22T10:42:34.944 INFO:tasks.workunit.client.0.smithi115.stdout:4/173: dwrite d1/dc/f20 [0,4194304] 0 2024-03-22T10:42:34.947 INFO:tasks.workunit.client.0.smithi115.stdout:1/245: truncate d0/d12/f51 535514 0 2024-03-22T10:42:34.950 INFO:tasks.workunit.client.0.smithi115.stdout:8/326: write d6/dd/d24/d4a/f61 [805838,116231] 0 2024-03-22T10:42:34.952 INFO:tasks.workunit.client.0.smithi115.stdout:5/150: dwrite d2/df/d13/f18 [0,4194304] 0 2024-03-22T10:42:34.954 INFO:tasks.workunit.client.0.smithi115.stdout:1/246: write d0/f7 [12181303,107109] 0 2024-03-22T10:42:34.957 INFO:tasks.workunit.client.0.smithi115.stdout:8/327: write d6/dd/f26 [4426580,45124] 0 2024-03-22T10:42:34.958 INFO:tasks.workunit.client.0.smithi115.stdout:7/183: creat d2/d7/d12/d10/d23/d39/f3e x:0 0 0 2024-03-22T10:42:34.959 INFO:tasks.workunit.client.0.smithi115.stdout:1/247: write d0/d12/d17/f39 [148979,119101] 0 2024-03-22T10:42:34.962 INFO:tasks.workunit.client.0.smithi115.stdout:5/151: mknod d2/df/d13/c2d 0 2024-03-22T10:42:34.967 INFO:tasks.workunit.client.0.smithi115.stdout:9/171: truncate f5 1809783 0 2024-03-22T10:42:34.971 INFO:tasks.workunit.client.0.smithi115.stdout:7/184: chown f1 1045 1 2024-03-22T10:42:34.971 INFO:tasks.workunit.client.0.smithi115.stdout:4/174: write d1/ff [2269613,50382] 0 2024-03-22T10:42:34.972 INFO:tasks.workunit.client.0.smithi115.stdout:9/172: read d6/d8/fd [722993,44406] 0 2024-03-22T10:42:34.976 INFO:tasks.workunit.client.0.smithi115.stdout:8/328: write d6/dd/d24/d4a/d41/f42 [736510,84968] 0 2024-03-22T10:42:34.979 INFO:tasks.workunit.client.0.smithi115.stdout:7/185: read - d2/d7/d12/d10/d23/d39/f3e zero size 2024-03-22T10:42:34.982 INFO:tasks.workunit.client.0.smithi115.stdout:9/173: chown d6/d7/ff 6803780 1 2024-03-22T10:42:34.984 INFO:tasks.workunit.client.0.smithi115.stdout:9/174: chown d6/d7/f2a 8 1 2024-03-22T10:42:34.984 INFO:tasks.workunit.client.0.smithi115.stdout:5/152: link d2/fe d2/f2e 0 2024-03-22T10:42:34.985 INFO:tasks.workunit.client.0.smithi115.stdout:7/186: fsync f1 0 2024-03-22T10:42:34.987 INFO:tasks.workunit.client.0.smithi115.stdout:8/329: creat d6/dd/d24/d4a/f66 x:0 0 0 2024-03-22T10:42:34.989 INFO:tasks.workunit.client.0.smithi115.stdout:7/187: creat d2/f3f x:0 0 0 2024-03-22T10:42:34.990 INFO:tasks.workunit.client.0.smithi115.stdout:1/248: dwrite d0/d12/d17/d37/f3c [0,4194304] 0 2024-03-22T10:42:34.993 INFO:tasks.workunit.client.0.smithi115.stdout:9/175: mkdir d6/d7/d20/d38 0 2024-03-22T10:42:34.994 INFO:tasks.workunit.client.0.smithi115.stdout:4/175: dread d1/dc/d14/f1b [0,4194304] 0 2024-03-22T10:42:35.000 INFO:tasks.workunit.client.0.smithi115.stdout:5/153: rename d2/df/d13/c2d to d2/d15/c2f 0 2024-03-22T10:42:35.004 INFO:tasks.workunit.client.0.smithi115.stdout:8/330: symlink d6/d49/l67 0 2024-03-22T10:42:35.006 INFO:tasks.workunit.client.0.smithi115.stdout:9/176: unlink f5 0 2024-03-22T10:42:35.010 INFO:tasks.workunit.client.0.smithi115.stdout:1/249: rename d0/d8/db/f27 to d0/f52 0 2024-03-22T10:42:35.011 INFO:tasks.workunit.client.0.smithi115.stdout:8/331: unlink d6/dd/d24/d4a/d41/f42 0 2024-03-22T10:42:35.012 INFO:tasks.workunit.client.0.smithi115.stdout:5/154: mknod d2/df/d27/c30 0 2024-03-22T10:42:35.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.021+0000 7fd795400700 1 -- 172.21.15.115:0/2482840126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7901067a0 msgr2=0x7fd790106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:35.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.021+0000 7fd795400700 1 --2- 172.21.15.115:0/2482840126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7901067a0 0x7fd790106bb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd78000b0a0 tx=0x7fd78000b3b0 comp rx=0 tx=0).stop 2024-03-22T10:42:35.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.022+0000 7fd795400700 1 -- 172.21.15.115:0/2482840126 shutdown_connections 2024-03-22T10:42:35.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.022+0000 7fd795400700 1 --2- 172.21.15.115:0/2482840126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7901079a0 0x7fd790107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:35.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.022+0000 7fd795400700 1 --2- 172.21.15.115:0/2482840126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7901067a0 0x7fd790106bb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:35.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.022+0000 7fd795400700 1 -- 172.21.15.115:0/2482840126 >> 172.21.15.115:0/2482840126 conn(0x7fd790069ae0 msgr2=0x7fd790069f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:35.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.022+0000 7fd795400700 1 -- 172.21.15.115:0/2482840126 shutdown_connections 2024-03-22T10:42:35.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.023+0000 7fd795400700 1 -- 172.21.15.115:0/2482840126 wait complete. 2024-03-22T10:42:35.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.024+0000 7fd795400700 1 Processor -- start 2024-03-22T10:42:35.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.024+0000 7fd795400700 1 -- start start 2024-03-22T10:42:35.016 INFO:tasks.workunit.client.0.smithi115.stdout:7/188: rename d2/f32 to d2/d7/d12/d10/f40 0 2024-03-22T10:42:35.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.025+0000 7fd795400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7901067a0 0x7fd79019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:35.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.025+0000 7fd795400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7901079a0 0x7fd79019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:35.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.025+0000 7fd795400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79019cc80 con 0x7fd7901067a0 2024-03-22T10:42:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.025+0000 7fd795400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79019cdc0 con 0x7fd7901079a0 2024-03-22T10:42:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.025+0000 7fd785e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7901079a0 0x7fd79019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.025+0000 7fd785e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7901079a0 0x7fd79019c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:54584/0 (socket says 172.21.15.115:54584) 2024-03-22T10:42:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.025+0000 7fd785e00700 1 -- 172.21.15.115:0/210820463 learned_addr learned my addr 172.21.15.115:0/210820463 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:35.017 INFO:tasks.workunit.client.0.smithi115.stdout:8/332: mkdir d6/d49/d68 0 2024-03-22T10:42:35.018 INFO:tasks.workunit.client.0.smithi115.stdout:4/176: dwrite d1/dc/d14/f1d [4194304,4194304] 0 2024-03-22T10:42:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.026+0000 7fd78e800700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7901067a0 0x7fd79019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.026+0000 7fd785e00700 1 -- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7901067a0 msgr2=0x7fd79019c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.026+0000 7fd785e00700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7901067a0 0x7fd79019c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.026+0000 7fd785e00700 1 -- 172.21.15.115:0/210820463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780009d00 con 0x7fd7901079a0 2024-03-22T10:42:35.020 INFO:tasks.workunit.client.0.smithi115.stdout:5/155: symlink d2/df/d1e/l31 0 2024-03-22T10:42:35.020 INFO:tasks.workunit.client.0.smithi115.stdout:4/177: chown d1/dc/d14 15689 1 2024-03-22T10:42:35.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.027+0000 7fd785e00700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7901079a0 0x7fd79019c660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd77800b4b0 tx=0x7fd77800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:35.021 INFO:tasks.workunit.client.0.smithi115.stdout:8/333: rmdir d6/d49 39 2024-03-22T10:42:35.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.029+0000 7fd787e00700 1 -- 172.21.15.115:0/210820463 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd778011800 con 0x7fd7901079a0 2024-03-22T10:42:35.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.029+0000 7fd787e00700 1 -- 172.21.15.115:0/210820463 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd778011e10 con 0x7fd7901079a0 2024-03-22T10:42:35.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.029+0000 7fd787e00700 1 -- 172.21.15.115:0/210820463 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd77800f2e0 con 0x7fd7901079a0 2024-03-22T10:42:35.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.031+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7901a1870 con 0x7fd7901079a0 2024-03-22T10:42:35.023 INFO:tasks.workunit.client.0.smithi115.stdout:7/189: write d2/d7/d12/d10/d23/f38 [756481,35730] 0 2024-03-22T10:42:35.023 INFO:tasks.workunit.client.0.smithi115.stdout:4/178: dread - d1/d16/f26 zero size 2024-03-22T10:42:35.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.031+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7901a1e10 con 0x7fd7901079a0 2024-03-22T10:42:35.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.032+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79010f370 con 0x7fd7901079a0 2024-03-22T10:42:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.036+0000 7fd787e00700 1 -- 172.21.15.115:0/210820463 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 29) v1 ==== 46247+0+0 (secure 0 0 0) 0x7fd778010380 con 0x7fd7901079a0 2024-03-22T10:42:35.028 INFO:tasks.workunit.client.0.smithi115.stdout:4/179: rename d1 to d1/dc/d14/d33/d39 22 2024-03-22T10:42:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.036+0000 7fd787e00700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fd774039230 0x7fd77403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.036+0000 7fd787e00700 1 -- 172.21.15.115:0/210820463 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fd77804d4b0 con 0x7fd7901079a0 2024-03-22T10:42:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.037+0000 7fd78e800700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fd774039230 0x7fd77403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:35.029 INFO:tasks.workunit.client.0.smithi115.stdout:8/334: truncate d6/dd/d24/d4a/f4d 1159010 0 2024-03-22T10:42:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.038+0000 7fd78e800700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fd774039230 0x7fd77403b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd780009fd0 tx=0x7fd780007a60 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.046+0000 7fd787e00700 1 -- 172.21.15.115:0/210820463 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fd778019590 con 0x7fd7901079a0 2024-03-22T10:42:35.038 INFO:tasks.workunit.client.0.smithi115.stdout:4/180: fdatasync d1/dc/d14/f1b 0 2024-03-22T10:42:35.044 INFO:tasks.workunit.client.0.smithi115.stdout:9/177: truncate d6/d7/ff 1985753 0 2024-03-22T10:42:35.048 INFO:tasks.workunit.client.0.smithi115.stdout:5/156: dread d2/fe [0,4194304] 0 2024-03-22T10:42:35.051 INFO:tasks.workunit.client.0.smithi115.stdout:4/181: rename d1/dc/l24 to d1/d16/d36/l3a 0 2024-03-22T10:42:35.052 INFO:tasks.workunit.client.0.smithi115.stdout:8/335: dwrite d6/dd/d24/d3a/f45 [0,4194304] 0 2024-03-22T10:42:35.054 INFO:tasks.workunit.client.0.smithi115.stdout:4/182: truncate d1/d16/f1c 355690 0 2024-03-22T10:42:35.055 INFO:tasks.workunit.client.0.smithi115.stdout:7/190: dwrite d2/d7/d12/d10/f40 [0,4194304] 0 2024-03-22T10:42:35.057 INFO:tasks.workunit.client.0.smithi115.stdout:5/157: mkdir d2/d15/d32 0 2024-03-22T10:42:35.059 INFO:tasks.workunit.client.0.smithi115.stdout:7/191: fsync d2/d7/d12/d10/d23/d33/f3b 0 2024-03-22T10:42:35.060 INFO:tasks.workunit.client.0.smithi115.stdout:7/192: dread - d2/f3f zero size 2024-03-22T10:42:35.061 INFO:tasks.workunit.client.0.smithi115.stdout:8/336: mknod d6/dd/d24/c69 0 2024-03-22T10:42:35.062 INFO:tasks.workunit.client.0.smithi115.stdout:5/158: read d2/df/d13/f18 [1601656,86435] 0 2024-03-22T10:42:35.062 INFO:tasks.workunit.client.0.smithi115.stdout:4/183: mkdir d1/d16/d3b 0 2024-03-22T10:42:35.069 INFO:tasks.workunit.client.0.smithi115.stdout:1/250: creat d0/d8/f53 x:0 0 0 2024-03-22T10:42:35.069 INFO:tasks.workunit.client.0.smithi115.stdout:8/337: chown d6/dd/d24/d4a/f61 6 1 2024-03-22T10:42:35.072 INFO:tasks.workunit.client.0.smithi115.stdout:9/178: dwrite d6/d7/d19/f27 [0,4194304] 0 2024-03-22T10:42:35.076 INFO:tasks.workunit.client.0.smithi115.stdout:5/159: unlink d2/d15/c19 0 2024-03-22T10:42:35.078 INFO:tasks.workunit.client.0.smithi115.stdout:7/193: stat d2/d7/d12/d10/d23/d3d/c22 0 2024-03-22T10:42:35.081 INFO:tasks.workunit.client.0.smithi115.stdout:4/184: rename d1/dc/l11 to d1/dc/d17/d32/l3c 0 2024-03-22T10:42:35.085 INFO:tasks.workunit.client.0.smithi115.stdout:7/194: truncate d2/d7/d12/d10/d23/d3d/f2a 581341 0 2024-03-22T10:42:35.086 INFO:tasks.workunit.client.0.smithi115.stdout:5/160: mkdir d2/d15/d1b/d33 0 2024-03-22T10:42:35.095 INFO:tasks.workunit.client.0.smithi115.stdout:1/251: rename d0/d8/de/d1a/f1e to d0/d8/db/d25/f54 0 2024-03-22T10:42:35.097 INFO:tasks.workunit.client.0.smithi115.stdout:7/195: rmdir d2/d7/d12/d10/d23/d39 39 2024-03-22T10:42:35.102 INFO:tasks.workunit.client.0.smithi115.stdout:9/179: rename d6/d7/c1d to d6/d7/d30/d37/c39 0 2024-03-22T10:42:35.103 INFO:tasks.workunit.client.0.smithi115.stdout:8/338: creat d6/dd/f6a x:0 0 0 2024-03-22T10:42:35.109 INFO:tasks.workunit.client.0.smithi115.stdout:1/252: fsync d0/f1 0 2024-03-22T10:42:35.112 INFO:tasks.workunit.client.0.smithi115.stdout:4/185: dwrite d1/d16/f26 [0,4194304] 0 2024-03-22T10:42:35.117 INFO:tasks.workunit.client.0.smithi115.stdout:5/161: dwrite d2/df/d1e/f2a [0,4194304] 0 2024-03-22T10:42:35.117 INFO:tasks.workunit.client.0.smithi115.stdout:4/186: stat d1/c8 0 2024-03-22T10:42:35.118 INFO:tasks.workunit.client.0.smithi115.stdout:5/162: fdatasync d2/f1f 0 2024-03-22T10:42:35.126 INFO:tasks.workunit.client.0.smithi115.stdout:8/339: dread d6/dd/f33 [0,4194304] 0 2024-03-22T10:42:35.133 INFO:tasks.workunit.client.0.smithi115.stdout:2/118: sync 2024-03-22T10:42:35.145 INFO:tasks.workunit.client.0.smithi115.stdout:2/119: dwrite da/f14 [0,4194304] 0 2024-03-22T10:42:35.181 INFO:tasks.workunit.client.0.smithi115.stdout:4/187: creat d1/dc/d17/f3d x:0 0 0 2024-03-22T10:42:35.184 INFO:tasks.workunit.client.0.smithi115.stdout:2/120: creat da/f1d x:0 0 0 2024-03-22T10:42:35.188 INFO:tasks.workunit.client.0.smithi115.stdout:7/196: dwrite d2/d7/d12/d10/d19/f37 [0,4194304] 0 2024-03-22T10:42:35.193 INFO:tasks.workunit.client.0.smithi115.stdout:1/253: rename d0/d12/f51 to d0/f55 0 2024-03-22T10:42:35.195 INFO:tasks.workunit.client.0.smithi115.stdout:8/340: link d6/dd/d24/d4a/l65 d6/dd/d24/d3a/l6b 0 2024-03-22T10:42:35.195 INFO:tasks.workunit.client.0.smithi115.stdout:4/188: rmdir d1/dc/d17 39 2024-03-22T10:42:35.197 INFO:tasks.workunit.client.0.smithi115.stdout:2/121: write da/f14 [5227120,101122] 0 2024-03-22T10:42:35.197 INFO:tasks.workunit.client.0.smithi115.stdout:7/197: mknod d2/d7/d12/d10/d23/d33/c41 0 2024-03-22T10:42:35.203 INFO:tasks.workunit.client.0.smithi115.stdout:1/254: write d0/d22/f3d [778803,72254] 0 2024-03-22T10:42:35.205 INFO:tasks.workunit.client.0.smithi115.stdout:5/163: dwrite d2/fe [0,4194304] 0 2024-03-22T10:42:35.208 INFO:tasks.workunit.client.0.smithi115.stdout:1/255: dread - d0/d12/f30 zero size 2024-03-22T10:42:35.213 INFO:tasks.workunit.client.0.smithi115.stdout:1/256: mkdir d0/d8/d4d/d56 0 2024-03-22T10:42:35.216 INFO:tasks.workunit.client.0.smithi115.stdout:7/198: rename d2/d7/d12/d10/d23/d39 to d2/d7/d12/d10/d19/d42 0 2024-03-22T10:42:35.219 INFO:tasks.workunit.client.0.smithi115.stdout:2/122: truncate da/f16 2328563 0 2024-03-22T10:42:35.225 INFO:tasks.workunit.client.0.smithi115.stdout:5/164: rename d2/df/d13/c29 to d2/d15/d1b/c34 0 2024-03-22T10:42:35.225 INFO:tasks.workunit.client.0.smithi115.stdout:4/189: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:35.227 INFO:tasks.workunit.client.0.smithi115.stdout:7/199: symlink d2/d7/d12/d10/d23/d33/l43 0 2024-03-22T10:42:35.228 INFO:tasks.workunit.client.0.smithi115.stdout:8/341: dwrite d6/dd/d12/d31/f5b [0,4194304] 0 2024-03-22T10:42:35.229 INFO:tasks.workunit.client.0.smithi115.stdout:2/123: creat da/d1c/f1e x:0 0 0 2024-03-22T10:42:35.231 INFO:tasks.workunit.client.0.smithi115.stdout:7/200: readlink d2/d7/l18 0 2024-03-22T10:42:35.234 INFO:tasks.workunit.client.0.smithi115.stdout:1/257: rename d0/d8/db/f16 to d0/d8/d4d/d56/f57 0 2024-03-22T10:42:35.248 INFO:tasks.workunit.client.0.smithi115.stdout:7/201: mknod d2/d7/d12/d10/d23/d3d/c44 0 2024-03-22T10:42:35.248 INFO:tasks.workunit.client.0.smithi115.stdout:4/190: rename d1/ff to d1/dc/d14/f3e 0 2024-03-22T10:42:35.249 INFO:tasks.workunit.client.0.smithi115.stdout:1/258: mkdir d0/d58 0 2024-03-22T10:42:35.253 INFO:tasks.workunit.client.0.smithi115.stdout:2/124: write da/f16 [2461059,116559] 0 2024-03-22T10:42:35.253 INFO:tasks.workunit.client.0.smithi115.stdout:1/259: creat d0/d12/d2b/f59 x:0 0 0 2024-03-22T10:42:35.254 INFO:tasks.workunit.client.0.smithi115.stdout:7/202: mkdir d2/d3a/d45 0 2024-03-22T10:42:35.256 INFO:tasks.workunit.client.0.smithi115.stdout:5/165: write d2/df/f10 [3042462,48914] 0 2024-03-22T10:42:35.262 INFO:tasks.workunit.client.0.smithi115.stdout:2/125: creat da/d1c/f1f x:0 0 0 2024-03-22T10:42:35.263 INFO:tasks.workunit.client.0.smithi115.stdout:1/260: creat d0/d8/d4d/f5a x:0 0 0 2024-03-22T10:42:35.274 INFO:tasks.workunit.client.0.smithi115.stdout:8/342: dwrite d6/dd/d24/d3a/f54 [0,4194304] 0 2024-03-22T10:42:35.281 INFO:tasks.workunit.client.0.smithi115.stdout:1/261: fdatasync d0/f52 0 2024-03-22T10:42:35.283 INFO:tasks.workunit.client.0.smithi115.stdout:4/191: dwrite d1/dc/d14/f1b [0,4194304] 0 2024-03-22T10:42:35.285 INFO:tasks.workunit.client.0.smithi115.stdout:8/343: write d6/dd/f2e [4079532,7239] 0 2024-03-22T10:42:35.291 INFO:tasks.workunit.client.0.smithi115.stdout:1/262: rmdir d0/d8/d4d 39 2024-03-22T10:42:35.296 INFO:tasks.workunit.client.0.smithi115.stdout:4/192: symlink d1/d16/d35/l3f 0 2024-03-22T10:42:35.296 INFO:tasks.workunit.client.0.smithi115.stdout:7/203: dwrite d2/d7/fb [4194304,4194304] 0 2024-03-22T10:42:35.301 INFO:tasks.workunit.client.0.smithi115.stdout:6/154: sync 2024-03-22T10:42:35.301 INFO:tasks.workunit.client.0.smithi115.stdout:0/147: sync 2024-03-22T10:42:35.302 INFO:tasks.workunit.client.0.smithi115.stdout:1/263: symlink d0/d12/d17/d37/l5b 0 2024-03-22T10:42:35.303 INFO:tasks.workunit.client.0.smithi115.stdout:2/126: dwrite da/d1c/f1e [0,4194304] 0 2024-03-22T10:42:35.314 INFO:tasks.workunit.client.0.smithi115.stdout:6/155: creat d4/d11/d12/f31 x:0 0 0 2024-03-22T10:42:35.314 INFO:tasks.workunit.client.0.smithi115.stdout:5/166: dwrite d2/f4 [4194304,4194304] 0 2024-03-22T10:42:35.315 INFO:tasks.workunit.client.0.smithi115.stdout:6/156: chown d4 1240028 1 2024-03-22T10:42:35.319 INFO:tasks.workunit.client.0.smithi115.stdout:4/193: mkdir d1/d16/d35/d40 0 2024-03-22T10:42:35.320 INFO:tasks.workunit.client.0.smithi115.stdout:2/127: fdatasync da/f15 0 2024-03-22T10:42:35.324 INFO:tasks.workunit.client.0.smithi115.stdout:5/167: symlink d2/d15/l35 0 2024-03-22T10:42:35.324 INFO:tasks.workunit.client.0.smithi115.stdout:5/168: chown d2 126825 1 2024-03-22T10:42:35.326 INFO:tasks.workunit.client.0.smithi115.stdout:8/344: dread d6/dd/f2e [0,4194304] 0 2024-03-22T10:42:35.327 INFO:tasks.workunit.client.0.smithi115.stdout:6/157: symlink d4/d11/d12/d1c/l32 0 2024-03-22T10:42:35.331 INFO:tasks.workunit.client.0.smithi115.stdout:7/204: dread d2/f5 [4194304,4194304] 0 2024-03-22T10:42:35.340 INFO:tasks.workunit.client.0.smithi115.stdout:8/345: symlink d6/d49/d68/l6c 0 2024-03-22T10:42:35.342 INFO:tasks.workunit.client.0.smithi115.stdout:7/205: chown d2/d7/d12/d10/d19/d42/f3e 55 1 2024-03-22T10:42:35.348 INFO:tasks.workunit.client.0.smithi115.stdout:1/264: dread d0/d8/f9 [0,4194304] 0 2024-03-22T10:42:35.350 INFO:tasks.workunit.client.0.smithi115.stdout:8/346: write d6/dd/d12/f1d [2813705,58978] 0 2024-03-22T10:42:35.350 INFO:tasks.workunit.client.0.smithi115.stdout:7/206: readlink d2/d7/d12/l1c 0 2024-03-22T10:42:35.350 INFO:tasks.workunit.client.0.smithi115.stdout:2/128: truncate da/f14 418098 0 2024-03-22T10:42:35.355 INFO:tasks.workunit.client.0.smithi115.stdout:4/194: dwrite d1/f9 [4194304,4194304] 0 2024-03-22T10:42:35.358 INFO:tasks.workunit.client.0.smithi115.stdout:8/347: chown d6/dd/d24/d4a/d41/f4b 2181 1 2024-03-22T10:42:35.358 INFO:tasks.workunit.client.0.smithi115.stdout:1/265: mkdir d0/d12/d17/d37/d5c 0 2024-03-22T10:42:35.360 INFO:tasks.workunit.client.0.smithi115.stdout:3/117: sync 2024-03-22T10:42:35.360 INFO:tasks.workunit.client.0.smithi115.stdout:9/180: sync 2024-03-22T10:42:35.360 INFO:tasks.workunit.client.0.smithi115.stdout:6/158: dwrite f3 [8388608,4194304] 0 2024-03-22T10:42:35.362 INFO:tasks.workunit.client.0.smithi115.stdout:7/207: mkdir d2/d7/d12/d10/d19/d42/d46 0 2024-03-22T10:42:35.362 INFO:tasks.workunit.client.0.smithi115.stdout:5/169: dread d2/fe [0,4194304] 0 2024-03-22T10:42:35.363 INFO:tasks.workunit.client.0.smithi115.stdout:4/195: creat d1/d30/f41 x:0 0 0 2024-03-22T10:42:35.364 INFO:tasks.workunit.client.0.smithi115.stdout:8/348: unlink d6/dd/f34 0 2024-03-22T10:42:35.374 INFO:tasks.workunit.client.0.smithi115.stdout:3/118: write d3/d16/f18 [329478,69239] 0 2024-03-22T10:42:35.374 INFO:tasks.workunit.client.0.smithi115.stdout:1/266: fsync d0/d12/d17/d37/f3c 0 2024-03-22T10:42:35.375 INFO:tasks.workunit.client.0.smithi115.stdout:0/148: dread d3/f1a [0,4194304] 0 2024-03-22T10:42:35.375 INFO:tasks.workunit.client.0.smithi115.stdout:3/119: chown d3/d19/d1b/f22 23 1 2024-03-22T10:42:35.378 INFO:tasks.workunit.client.0.smithi115.stdout:9/181: creat d6/d7/d19/f3a x:0 0 0 2024-03-22T10:42:35.381 INFO:tasks.workunit.client.0.smithi115.stdout:1/267: creat d0/d12/f5d x:0 0 0 2024-03-22T10:42:35.389 INFO:tasks.workunit.client.0.smithi115.stdout:9/182: mkdir d6/d7/d30/d3b 0 2024-03-22T10:42:35.392 INFO:tasks.workunit.client.0.smithi115.stdout:7/208: getdents d2/d3a 0 2024-03-22T10:42:35.395 INFO:tasks.workunit.client.0.smithi115.stdout:6/159: dwrite d4/f30 [0,4194304] 0 2024-03-22T10:42:35.396 INFO:tasks.workunit.client.0.smithi115.stdout:1/268: rmdir d0/d12/d2b 39 2024-03-22T10:42:35.398 INFO:tasks.workunit.client.0.smithi115.stdout:8/349: dread d6/dd/d12/f1d [0,4194304] 0 2024-03-22T10:42:35.404 INFO:tasks.workunit.client.0.smithi115.stdout:2/129: truncate da/fb 1363945 0 2024-03-22T10:42:35.405 INFO:tasks.workunit.client.0.smithi115.stdout:7/209: dread - d2/d7/d12/d10/d19/d42/f3e zero size 2024-03-22T10:42:35.407 INFO:tasks.workunit.client.0.smithi115.stdout:0/149: write d3/d7/d1c/f1e [1055324,78984] 0 2024-03-22T10:42:35.407 INFO:tasks.workunit.client.0.smithi115.stdout:2/130: readlink da/l11 0 2024-03-22T10:42:35.408 INFO:tasks.workunit.client.0.smithi115.stdout:5/170: dwrite d2/fe [0,4194304] 0 2024-03-22T10:42:35.415 INFO:tasks.workunit.client.0.smithi115.stdout:3/120: dwrite d3/d16/f1c [0,4194304] 0 2024-03-22T10:42:35.419 INFO:tasks.workunit.client.0.smithi115.stdout:3/121: chown d3/c1f 118 1 2024-03-22T10:42:35.419 INFO:tasks.workunit.client.0.smithi115.stdout:4/196: dwrite d1/dc/d17/f31 [0,4194304] 0 2024-03-22T10:42:35.421 INFO:tasks.workunit.client.0.smithi115.stdout:2/131: mknod da/c20 0 2024-03-22T10:42:35.422 INFO:tasks.workunit.client.0.smithi115.stdout:7/210: rmdir d2/d7/d12 39 2024-03-22T10:42:35.426 INFO:tasks.workunit.client.0.smithi115.stdout:0/150: creat d3/d7/d1b/f2a x:0 0 0 2024-03-22T10:42:35.435 INFO:tasks.workunit.client.0.smithi115.stdout:9/183: truncate d6/d7/d19/f21 246891 0 2024-03-22T10:42:35.440 INFO:tasks.workunit.client.0.smithi115.stdout:2/132: symlink da/d1c/l21 0 2024-03-22T10:42:35.441 INFO:tasks.workunit.client.0.smithi115.stdout:1/269: dwrite d0/d8/f53 [0,4194304] 0 2024-03-22T10:42:35.445 INFO:tasks.workunit.client.0.smithi115.stdout:2/133: write da/f15 [4353451,36840] 0 2024-03-22T10:42:35.449 INFO:tasks.workunit.client.0.smithi115.stdout:6/160: dwrite d4/d11/f2a [0,4194304] 0 2024-03-22T10:42:35.449 INFO:tasks.workunit.client.0.smithi115.stdout:9/184: truncate d6/d7/d20/d22/f2c 880814 0 2024-03-22T10:42:35.450 INFO:tasks.workunit.client.0.smithi115.stdout:9/185: stat d6/d7/f17 0 2024-03-22T10:42:35.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.461+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd79010c2f0 con 0x7fd774039230 2024-03-22T10:42:35.458 INFO:tasks.workunit.client.0.smithi115.stdout:0/151: mknod d3/d4/c2b 0 2024-03-22T10:42:35.467 INFO:tasks.workunit.client.0.smithi115.stdout:2/134: mkdir da/d1c/d22 0 2024-03-22T10:42:35.468 INFO:tasks.workunit.client.0.smithi115.stdout:9/186: mkdir d6/d8/d3c 0 2024-03-22T10:42:35.470 INFO:tasks.workunit.client.0.smithi115.stdout:6/161: mkdir d4/d13/d33 0 2024-03-22T10:42:35.477 INFO:tasks.workunit.client.0.smithi115.stdout:4/197: getdents d1/dc/d17/d32 0 2024-03-22T10:42:35.478 INFO:tasks.workunit.client.0.smithi115.stdout:2/135: symlink da/l23 0 2024-03-22T10:42:35.481 INFO:tasks.workunit.client.0.smithi115.stdout:9/187: rename d6/d7/d20/d22/f2c to d6/d7/d30/d3b/f3d 0 2024-03-22T10:42:35.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.491+0000 7fd787e00700 1 -- 172.21.15.115:0/210820463 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd79010c2f0 con 0x7fd774039230 2024-03-22T10:42:35.483 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:42:35.483 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (9m) 15s ago 10m 14.1M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:42:35.483 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (11m) 15s ago 11m 7792k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:42:35.483 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (9m) 0s ago 9m 7684k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (11m) 15s ago 11m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (9m) 0s ago 9m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (8m) 15s ago 10m 73.8M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (4m) 15s ago 4m 104M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (3m) 15s ago 3m 13.7M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (3m) 0s ago 3m 13.9M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (3m) 0s ago 3m 125M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (26s) 15s ago 12m 65.0M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (84s) 0s ago 9m 101M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 02c6d1a3b66d 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (12m) 15s ago 12m 50.2M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (9m) 0s ago 9m 42.3M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (11m) 15s ago 11m 7792k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (9m) 0s ago 9m 8048k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (8m) 15s ago 8m 79.3M 1217M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:42:35.484 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (8m) 15s ago 8m 87.1M 1217M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:42:35.485 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (7m) 15s ago 7m 73.7M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:42:35.485 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (6m) 0s ago 6m 108M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:42:35.485 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (6m) 0s ago 6m 104M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:42:35.485 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (5m) 0s ago 5m 90.0M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:42:35.485 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (35s) 15s ago 10m 41.6M - 2.43.0 a07b618ecd1d 43302dacd900 2024-03-22T10:42:35.485 INFO:tasks.workunit.client.0.smithi115.stdout:7/211: link d2/d7/l18 d2/d7/d12/d10/d19/d42/d46/l47 0 2024-03-22T10:42:35.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.496+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fd774039230 msgr2=0x7fd77403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.496+0000 7fd795400700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fd774039230 0x7fd77403b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd780009fd0 tx=0x7fd780007a60 comp rx=0 tx=0).stop 2024-03-22T10:42:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.496+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7901079a0 msgr2=0x7fd79019c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.496+0000 7fd795400700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7901079a0 0x7fd79019c660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd77800b4b0 tx=0x7fd77800b870 comp rx=0 tx=0).stop 2024-03-22T10:42:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.496+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 shutdown_connections 2024-03-22T10:42:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.496+0000 7fd795400700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fd774039230 0x7fd77403b6e0 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd780009fd0 tx=0x7fd780007a60 comp rx=0 tx=0).stop 2024-03-22T10:42:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.496+0000 7fd795400700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7901067a0 0x7fd79019c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.496+0000 7fd795400700 1 --2- 172.21.15.115:0/210820463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7901079a0 0x7fd79019c660 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.496+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 >> 172.21.15.115:0/210820463 conn(0x7fd790069ae0 msgr2=0x7fd79010abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:35.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.497+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 shutdown_connections 2024-03-22T10:42:35.490 INFO:tasks.workunit.client.0.smithi115.stdout:7/212: stat d2/d7/d12/d10/d23 0 2024-03-22T10:42:35.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.499+0000 7fd795400700 1 -- 172.21.15.115:0/210820463 wait complete. 2024-03-22T10:42:35.491 INFO:tasks.workunit.client.0.smithi115.stdout:0/152: dwrite d3/d7/d1c/f22 [0,4194304] 0 2024-03-22T10:42:35.495 INFO:tasks.workunit.client.0.smithi115.stdout:0/153: write d3/d7/d1b/f2a [410954,85227] 0 2024-03-22T10:42:35.497 INFO:tasks.workunit.client.0.smithi115.stdout:7/213: write d2/d7/d12/d10/d23/d3d/f21 [776618,21143] 0 2024-03-22T10:42:35.500 INFO:tasks.workunit.client.0.smithi115.stdout:6/162: rename d4/d11/f1f to d4/d13/d33/f34 0 2024-03-22T10:42:35.501 INFO:tasks.workunit.client.0.smithi115.stdout:1/270: getdents d0/d22 0 2024-03-22T10:42:35.506 INFO:tasks.workunit.client.0.smithi115.stdout:9/188: mknod d6/d7/d30/d3b/c3e 0 2024-03-22T10:42:35.507 INFO:tasks.workunit.client.0.smithi115.stdout:6/163: dread - d4/d11/d12/d1c/f2d zero size 2024-03-22T10:42:35.510 INFO:tasks.workunit.client.0.smithi115.stdout:9/189: write d6/d7/f2a [2342428,79321] 0 2024-03-22T10:42:35.511 INFO:tasks.workunit.client.0.smithi115.stdout:2/136: mknod da/d1c/d22/c24 0 2024-03-22T10:42:35.513 INFO:tasks.workunit.client.0.smithi115.stdout:9/190: dread - d6/f2e zero size 2024-03-22T10:42:35.513 INFO:tasks.workunit.client.0.smithi115.stdout:2/137: chown da/c19 0 1 2024-03-22T10:42:35.516 INFO:tasks.workunit.client.0.smithi115.stdout:4/198: rename d1/dc/d17 to d1/d30/d42 0 2024-03-22T10:42:35.518 INFO:tasks.workunit.client.0.smithi115.stdout:2/138: chown da/d1c/f1e 8080236 1 2024-03-22T10:42:35.518 INFO:tasks.workunit.client.0.smithi115.stdout:4/199: stat d1/d30/d42/f31 0 2024-03-22T10:42:35.520 INFO:tasks.workunit.client.0.smithi115.stdout:1/271: fdatasync d0/d8/de/d1a/f24 0 2024-03-22T10:42:35.523 INFO:tasks.workunit.client.0.smithi115.stdout:0/154: mknod d3/d4/c2c 0 2024-03-22T10:42:35.525 INFO:tasks.workunit.client.0.smithi115.stdout:0/155: chown d3/d7/d1b 81 1 2024-03-22T10:42:35.525 INFO:tasks.workunit.client.0.smithi115.stdout:7/214: dwrite d2/d7/f13 [4194304,4194304] 0 2024-03-22T10:42:35.530 INFO:tasks.workunit.client.0.smithi115.stdout:6/164: rename d4/c2b to d4/d13/c35 0 2024-03-22T10:42:35.530 INFO:tasks.workunit.client.0.smithi115.stdout:4/200: fsync d1/fd 0 2024-03-22T10:42:35.535 INFO:tasks.workunit.client.0.smithi115.stdout:2/139: mkdir da/d25 0 2024-03-22T10:42:35.538 INFO:tasks.workunit.client.0.smithi115.stdout:1/272: write d0/d8/db/f18 [2267408,87060] 0 2024-03-22T10:42:35.546 INFO:tasks.workunit.client.0.smithi115.stdout:6/165: mkdir d4/d36 0 2024-03-22T10:42:35.549 INFO:tasks.workunit.client.0.smithi115.stdout:4/201: link d1/d16/d28/c2a d1/d30/d38/c43 0 2024-03-22T10:42:35.551 INFO:tasks.workunit.client.0.smithi115.stdout:1/273: link d0/d22/f41 d0/d22/f5e 0 2024-03-22T10:42:35.554 INFO:tasks.workunit.client.0.smithi115.stdout:2/140: dwrite da/f16 [0,4194304] 0 2024-03-22T10:42:35.556 INFO:tasks.workunit.client.0.smithi115.stdout:0/156: getdents d3/d4/d14 0 2024-03-22T10:42:35.557 INFO:tasks.workunit.client.0.smithi115.stdout:6/166: rename d4/d11/d12/c2c to d4/d13/c37 0 2024-03-22T10:42:35.559 INFO:tasks.workunit.client.0.smithi115.stdout:6/167: write d4/f5 [11670127,11948] 0 2024-03-22T10:42:35.561 INFO:tasks.workunit.client.0.smithi115.stdout:2/141: rmdir da 39 2024-03-22T10:42:35.567 INFO:tasks.workunit.client.0.smithi115.stdout:0/157: rename d3/da/c27 to d3/d4/d14/c2d 0 2024-03-22T10:42:35.570 INFO:tasks.workunit.client.0.smithi115.stdout:1/274: link d0/d8/de/d1a/f24 d0/d8/de/f5f 0 2024-03-22T10:42:35.578 INFO:tasks.workunit.client.0.smithi115.stdout:2/142: dread da/f12 [0,4194304] 0 2024-03-22T10:42:35.581 INFO:tasks.workunit.client.0.smithi115.stdout:9/191: dwrite f3 [0,4194304] 0 2024-03-22T10:42:35.591 INFO:tasks.workunit.client.0.smithi115.stdout:2/143: mknod da/d1c/d22/c26 0 2024-03-22T10:42:35.598 INFO:tasks.workunit.client.0.smithi115.stdout:4/202: dwrite d1/dc/d14/f3e [0,4194304] 0 2024-03-22T10:42:35.602 INFO:tasks.workunit.client.0.smithi115.stdout:0/158: dread d3/d4/d14/f23 [4194304,4194304] 0 2024-03-22T10:42:35.605 INFO:tasks.workunit.client.0.smithi115.stdout:6/168: getdents d4/d11/d12 0 2024-03-22T10:42:35.610 INFO:tasks.workunit.client.0.smithi115.stdout:2/144: mkdir da/d1c/d22/d27 0 2024-03-22T10:42:35.611 INFO:tasks.workunit.client.0.smithi115.stdout:1/275: dwrite d0/d22/f3d [0,4194304] 0 2024-03-22T10:42:35.614 INFO:tasks.workunit.client.0.smithi115.stdout:4/203: write d1/d30/d42/f21 [2195865,82052] 0 2024-03-22T10:42:35.625 INFO:tasks.workunit.client.0.smithi115.stdout:0/159: link d3/d7/cb d3/d7/c2e 0 2024-03-22T10:42:35.626 INFO:tasks.workunit.client.0.smithi115.stdout:9/192: dwrite d6/d8/f1c [4194304,4194304] 0 2024-03-22T10:42:35.626 INFO:tasks.workunit.client.0.smithi115.stdout:1/276: chown d0/d8/d4d/d56/f57 3676 1 2024-03-22T10:42:35.631 INFO:tasks.workunit.client.0.smithi115.stdout:4/204: link d1/f9 d1/d30/d42/d32/f44 0 2024-03-22T10:42:35.633 INFO:tasks.workunit.client.0.smithi115.stdout:1/277: chown d0/d8/l19 0 1 2024-03-22T10:42:35.634 INFO:tasks.workunit.client.0.smithi115.stdout:4/205: write d1/d30/d42/f34 [460052,105571] 0 2024-03-22T10:42:35.635 INFO:tasks.workunit.client.0.smithi115.stdout:9/193: write d6/d7/d19/f3a [742978,72986] 0 2024-03-22T10:42:35.638 INFO:tasks.workunit.client.0.smithi115.stdout:2/145: dread da/f16 [0,4194304] 0 2024-03-22T10:42:35.642 INFO:tasks.workunit.client.0.smithi115.stdout:4/206: rmdir d1 39 2024-03-22T10:42:35.643 INFO:tasks.workunit.client.0.smithi115.stdout:6/169: dread d4/d11/f2a [0,4194304] 0 2024-03-22T10:42:35.648 INFO:tasks.workunit.client.0.smithi115.stdout:0/160: chown d3/d7/cb 521311 1 2024-03-22T10:42:35.654 INFO:tasks.workunit.client.0.smithi115.stdout:6/170: write d4/f24 [790939,10865] 0 2024-03-22T10:42:35.655 INFO:tasks.workunit.client.0.smithi115.stdout:2/146: creat da/d1c/d22/d27/f28 x:0 0 0 2024-03-22T10:42:35.661 INFO:tasks.workunit.client.0.smithi115.stdout:4/207: chown d1/d30/d42/l29 4034 1 2024-03-22T10:42:35.663 INFO:tasks.workunit.client.0.smithi115.stdout:6/171: dread - d4/d13/f2e zero size 2024-03-22T10:42:35.669 INFO:tasks.workunit.client.0.smithi115.stdout:1/278: dwrite d0/d12/f5d [0,4194304] 0 2024-03-22T10:42:35.669 INFO:tasks.workunit.client.0.smithi115.stdout:4/208: creat d1/d30/d42/f45 x:0 0 0 2024-03-22T10:42:35.676 INFO:tasks.workunit.client.0.smithi115.stdout:6/172: rmdir d4/d11/d12 39 2024-03-22T10:42:35.678 INFO:tasks.workunit.client.0.smithi115.stdout:6/173: fsync d4/f10 0 2024-03-22T10:42:35.680 INFO:tasks.workunit.client.0.smithi115.stdout:2/147: getdents da/d1c/d22 0 2024-03-22T10:42:35.680 INFO:tasks.workunit.client.0.smithi115.stdout:9/194: dwrite d6/f2e [0,4194304] 0 2024-03-22T10:42:35.683 INFO:tasks.workunit.client.0.smithi115.stdout:0/161: dwrite d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:35.684 INFO:tasks.workunit.client.0.smithi115.stdout:4/209: mknod d1/dc/d14/c46 0 2024-03-22T10:42:35.685 INFO:tasks.workunit.client.0.smithi115.stdout:1/279: write d0/d12/d2b/f59 [717030,54285] 0 2024-03-22T10:42:35.704 INFO:tasks.workunit.client.0.smithi115.stdout:1/280: rename d0/d12/l4f to d0/d8/de/d1a/l60 0 2024-03-22T10:42:35.708 INFO:tasks.workunit.client.0.smithi115.stdout:9/195: dwrite d6/f2e [0,4194304] 0 2024-03-22T10:42:35.714 INFO:tasks.workunit.client.0.smithi115.stdout:1/281: creat d0/d8/de/d1a/f61 x:0 0 0 2024-03-22T10:42:35.717 INFO:tasks.workunit.client.0.smithi115.stdout:0/162: dread d3/d7/d1c/f22 [0,4194304] 0 2024-03-22T10:42:35.720 INFO:tasks.workunit.client.0.smithi115.stdout:9/196: mkdir d6/d7/d20/d22/d3f 0 2024-03-22T10:42:35.720 INFO:tasks.workunit.client.0.smithi115.stdout:1/282: mknod d0/d8/d4d/c62 0 2024-03-22T10:42:35.720 INFO:tasks.workunit.client.0.smithi115.stdout:4/210: dwrite d1/dc/d14/f1d [4194304,4194304] 0 2024-03-22T10:42:35.729 INFO:tasks.workunit.client.0.smithi115.stdout:9/197: creat d6/d7/d20/d38/f40 x:0 0 0 2024-03-22T10:42:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.738+0000 7f848c400700 1 -- 172.21.15.115:0/2895326600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840ff380 msgr2=0x7f84840ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.738+0000 7f848c400700 1 --2- 172.21.15.115:0/2895326600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840ff380 0x7f84840ff7f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8470009870 tx=0x7f8470009b80 comp rx=0 tx=0).stop 2024-03-22T10:42:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.739+0000 7f848c400700 1 -- 172.21.15.115:0/2895326600 shutdown_connections 2024-03-22T10:42:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.739+0000 7f848c400700 1 --2- 172.21.15.115:0/2895326600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840ff380 0x7f84840ff7f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.739+0000 7f848c400700 1 --2- 172.21.15.115:0/2895326600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84840fea30 0x7f84840fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.739+0000 7f848c400700 1 -- 172.21.15.115:0/2895326600 >> 172.21.15.115:0/2895326600 conn(0x7f84840fa5e0 msgr2=0x7f84840fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.740+0000 7f848c400700 1 -- 172.21.15.115:0/2895326600 shutdown_connections 2024-03-22T10:42:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.741+0000 7f848c400700 1 -- 172.21.15.115:0/2895326600 wait complete. 2024-03-22T10:42:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.742+0000 7f848c400700 1 Processor -- start 2024-03-22T10:42:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.742+0000 7f848c400700 1 -- start start 2024-03-22T10:42:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.743+0000 7f848c400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840fea30 0x7f84841045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.743+0000 7f848c400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84840ff380 0x7f8484104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.743+0000 7f848c400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84841010a0 con 0x7f84840ff380 2024-03-22T10:42:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.743+0000 7f848c400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8484101210 con 0x7f84840fea30 2024-03-22T10:42:35.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.743+0000 7f8489a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840fea30 0x7f84841045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:35.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.743+0000 7f8489a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840fea30 0x7f84841045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:54598/0 (socket says 172.21.15.115:54598) 2024-03-22T10:42:35.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.743+0000 7f8489a00700 1 -- 172.21.15.115:0/351355899 learned_addr learned my addr 172.21.15.115:0/351355899 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:35.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.744+0000 7f8489000700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84840ff380 0x7f8484104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:35.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.745+0000 7f8489a00700 1 -- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84840ff380 msgr2=0x7f8484104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:35.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.745+0000 7f8489a00700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84840ff380 0x7f8484104af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:35.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.745+0000 7f8489a00700 1 -- 172.21.15.115:0/351355899 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470009520 con 0x7f84840fea30 2024-03-22T10:42:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.746+0000 7f8489a00700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840fea30 0x7f84841045b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f847800e940 tx=0x7f847800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.746+0000 7f8482000700 1 -- 172.21.15.115:0/351355899 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f847800eee0 con 0x7f84840fea30 2024-03-22T10:42:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.746+0000 7f8482000700 1 -- 172.21.15.115:0/351355899 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8478004ce0 con 0x7f84840fea30 2024-03-22T10:42:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.746+0000 7f8482000700 1 -- 172.21.15.115:0/351355899 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8478005650 con 0x7f84840fea30 2024-03-22T10:42:35.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.747+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84841014f0 con 0x7f84840fea30 2024-03-22T10:42:35.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.747+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8484101a70 con 0x7f84840fea30 2024-03-22T10:42:35.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.750+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848410ae80 con 0x7f84840fea30 2024-03-22T10:42:35.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.751+0000 7f8482000700 1 -- 172.21.15.115:0/351355899 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 29) v1 ==== 46247+0+0 (secure 0 0 0) 0x7f8478021440 con 0x7f84840fea30 2024-03-22T10:42:35.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.751+0000 7f8482000700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f846c039280 0x7f846c03b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:35.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.751+0000 7f8482000700 1 -- 172.21.15.115:0/351355899 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f8478014070 con 0x7f84840fea30 2024-03-22T10:42:35.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.752+0000 7f8489000700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f846c039280 0x7f846c03b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:35.744 INFO:tasks.workunit.client.0.smithi115.stdout:9/198: link d6/d8/ce d6/d7/d30/d37/c41 0 2024-03-22T10:42:35.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.754+0000 7f8489000700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f846c039280 0x7f846c03b730 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8470000c00 tx=0x7f8470019040 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:35.748 INFO:tasks.workunit.client.0.smithi115.stdout:4/211: dread d1/fd [0,4194304] 0 2024-03-22T10:42:35.748 INFO:tasks.workunit.client.0.smithi115.stdout:1/283: dwrite d0/f1 [0,4194304] 0 2024-03-22T10:42:35.750 INFO:tasks.workunit.client.0.smithi115.stdout:4/212: dread - d1/d30/f41 zero size 2024-03-22T10:42:35.751 INFO:tasks.workunit.client.0.smithi115.stdout:1/284: dread - d0/d8/de/d1a/f4a zero size 2024-03-22T10:42:35.755 INFO:tasks.workunit.client.0.smithi115.stdout:1/285: dread - d0/d8/de/d1a/f2f zero size 2024-03-22T10:42:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:35.763+0000 7f8482000700 1 -- 172.21.15.115:0/351355899 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f8478016330 con 0x7f84840fea30 2024-03-22T10:42:35.760 INFO:tasks.workunit.client.0.smithi115.stdout:6/174: dread f3 [4194304,4194304] 0 2024-03-22T10:42:35.774 INFO:tasks.workunit.client.0.smithi115.stdout:1/286: dread d0/d8/db/f18 [0,4194304] 0 2024-03-22T10:42:35.776 INFO:tasks.workunit.client.0.smithi115.stdout:1/287: read - d0/d8/d4d/f5a zero size 2024-03-22T10:42:35.782 INFO:tasks.workunit.client.0.smithi115.stdout:6/175: dread d4/d11/d12/f21 [0,4194304] 0 2024-03-22T10:42:35.783 INFO:tasks.workunit.client.0.smithi115.stdout:9/199: dread d6/f12 [0,4194304] 0 2024-03-22T10:42:35.783 INFO:tasks.workunit.client.0.smithi115.stdout:1/288: truncate d0/f52 1241232 0 2024-03-22T10:42:35.786 INFO:tasks.workunit.client.0.smithi115.stdout:9/200: write d6/d7/f1e [1968728,11322] 0 2024-03-22T10:42:35.787 INFO:tasks.workunit.client.0.smithi115.stdout:6/176: dread - d4/d11/d12/d1c/f2d zero size 2024-03-22T10:42:35.789 INFO:tasks.workunit.client.0.smithi115.stdout:9/201: stat d6/d7/f17 0 2024-03-22T10:42:35.790 INFO:tasks.workunit.client.0.smithi115.stdout:1/289: write d0/d8/de/d1a/f32 [2816850,80686] 0 2024-03-22T10:42:35.790 INFO:tasks.workunit.client.0.smithi115.stdout:6/177: rename d4/d13 to d4/d13/d33/d38 22 2024-03-22T10:42:35.796 INFO:tasks.workunit.client.0.smithi115.stdout:1/290: chown d0/d8/ld 378 1 2024-03-22T10:42:35.799 INFO:tasks.workunit.client.0.smithi115.stdout:8/350: sync 2024-03-22T10:42:35.799 INFO:tasks.workunit.client.0.smithi115.stdout:5/171: sync 2024-03-22T10:42:35.799 INFO:tasks.workunit.client.0.smithi115.stdout:3/122: sync 2024-03-22T10:42:35.799 INFO:tasks.workunit.client.0.smithi115.stdout:7/215: sync 2024-03-22T10:42:35.799 INFO:tasks.workunit.client.0.smithi115.stdout:6/178: write d4/f30 [1895757,101492] 0 2024-03-22T10:42:35.806 INFO:tasks.workunit.client.0.smithi115.stdout:1/291: mknod d0/d8/c63 0 2024-03-22T10:42:35.810 INFO:tasks.workunit.client.0.smithi115.stdout:6/179: stat d4/l18 0 2024-03-22T10:42:35.812 INFO:tasks.workunit.client.0.smithi115.stdout:5/172: unlink d2/d15/f1c 0 2024-03-22T10:42:35.814 INFO:tasks.workunit.client.0.smithi115.stdout:3/123: mknod d3/c24 0 2024-03-22T10:42:35.827 INFO:tasks.workunit.client.0.smithi115.stdout:3/124: write d3/d16/f21 [3796760,25031] 0 2024-03-22T10:42:35.827 INFO:tasks.workunit.client.0.smithi115.stdout:9/202: dwrite d6/d7/d19/f25 [0,4194304] 0 2024-03-22T10:42:35.832 INFO:tasks.workunit.client.0.smithi115.stdout:6/180: mknod d4/c39 0 2024-03-22T10:42:35.834 INFO:tasks.workunit.client.0.smithi115.stdout:4/213: truncate d1/dc/d14/f3e 1855258 0 2024-03-22T10:42:35.843 INFO:tasks.workunit.client.0.smithi115.stdout:9/203: mknod d6/d7/d30/d3b/c42 0 2024-03-22T10:42:35.849 INFO:tasks.workunit.client.0.smithi115.stdout:8/351: dwrite d6/fb [0,4194304] 0 2024-03-22T10:42:35.857 INFO:tasks.workunit.client.0.smithi115.stdout:8/352: unlink d6/dd/d24/d3a/f45 0 2024-03-22T10:42:35.862 INFO:tasks.workunit.client.0.smithi115.stdout:3/125: dread d3/f8 [4194304,4194304] 0 2024-03-22T10:42:35.863 INFO:tasks.workunit.client.0.smithi115.stdout:3/126: fsync d3/d19/f1d 0 2024-03-22T10:42:35.864 INFO:tasks.workunit.client.0.smithi115.stdout:9/204: rename d6/d7/lb to d6/d7/d20/d22/d23/l43 0 2024-03-22T10:42:35.867 INFO:tasks.workunit.client.0.smithi115.stdout:8/353: write d6/fb [1002934,30290] 0 2024-03-22T10:42:35.873 INFO:tasks.workunit.client.0.smithi115.stdout:8/354: fsync d6/dd/f11 0 2024-03-22T10:42:35.873 INFO:tasks.workunit.client.0.smithi115.stdout:1/292: dread d0/d12/f1c [0,4194304] 0 2024-03-22T10:42:35.874 INFO:tasks.workunit.client.0.smithi115.stdout:3/127: creat d3/f25 x:0 0 0 2024-03-22T10:42:35.879 INFO:tasks.workunit.client.0.smithi115.stdout:6/181: dread d4/f30 [0,4194304] 0 2024-03-22T10:42:35.881 INFO:tasks.workunit.client.0.smithi115.stdout:9/205: stat d6/d7/d30/d37/c41 0 2024-03-22T10:42:35.883 INFO:tasks.workunit.client.0.smithi115.stdout:4/214: dwrite d1/d30/f41 [0,4194304] 0 2024-03-22T10:42:35.883 INFO:tasks.workunit.client.0.smithi115.stdout:3/128: symlink d3/d19/d1b/l26 0 2024-03-22T10:42:35.884 INFO:tasks.workunit.client.0.smithi115.stdout:6/182: write d4/f10 [1080210,30871] 0 2024-03-22T10:42:35.890 INFO:tasks.workunit.client.0.smithi115.stdout:4/215: write d1/d30/d42/f31 [3428432,68771] 0 2024-03-22T10:42:35.891 INFO:tasks.workunit.client.0.smithi115.stdout:8/355: symlink d6/dd/d24/d4a/d41/l6d 0 2024-03-22T10:42:35.892 INFO:tasks.workunit.client.0.smithi115.stdout:1/293: mknod d0/d8/d4d/c64 0 2024-03-22T10:42:35.903 INFO:tasks.workunit.client.0.smithi115.stdout:3/129: fsync d3/f25 0 2024-03-22T10:42:35.912 INFO:tasks.workunit.client.0.smithi115.stdout:8/356: dread d6/dd/f2e [0,4194304] 0 2024-03-22T10:42:35.913 INFO:tasks.workunit.client.0.smithi115.stdout:6/183: dread d4/d11/f2a [0,4194304] 0 2024-03-22T10:42:35.917 INFO:tasks.workunit.client.0.smithi115.stdout:5/173: dread d2/f11 [0,4194304] 0 2024-03-22T10:42:35.919 INFO:tasks.workunit.client.0.smithi115.stdout:1/294: dread d0/f52 [0,4194304] 0 2024-03-22T10:42:35.922 INFO:tasks.workunit.client.0.smithi115.stdout:6/184: rename d4 to d4/d3a 22 2024-03-22T10:42:35.929 INFO:tasks.workunit.client.0.smithi115.stdout:8/357: mknod d6/dd/d12/d31/c6e 0 2024-03-22T10:42:35.929 INFO:tasks.workunit.client.0.smithi115.stdout:5/174: creat d2/df/d1e/f36 x:0 0 0 2024-03-22T10:42:35.930 INFO:tasks.workunit.client.0.smithi115.stdout:9/206: dwrite d6/d7/d19/f25 [0,4194304] 0 2024-03-22T10:42:35.932 INFO:tasks.workunit.client.0.smithi115.stdout:1/295: truncate d0/d12/d17/f33 145811 0 2024-03-22T10:42:35.937 INFO:tasks.workunit.client.0.smithi115.stdout:6/185: rename d4/d13/c37 to d4/d13/c3b 0 2024-03-22T10:42:35.939 INFO:tasks.workunit.client.0.smithi115.stdout:4/216: dwrite d1/dc/d14/f3e [0,4194304] 0 2024-03-22T10:42:35.945 INFO:tasks.workunit.client.0.smithi115.stdout:6/186: truncate d4/d11/d12/d1c/f2d 119145 0 2024-03-22T10:42:35.945 INFO:tasks.workunit.client.0.smithi115.stdout:5/175: write d2/df/d13/f23 [45842,31989] 0 2024-03-22T10:42:35.946 INFO:tasks.workunit.client.0.smithi115.stdout:8/358: fdatasync d6/dd/d12/f3b 0 2024-03-22T10:42:35.947 INFO:tasks.workunit.client.0.smithi115.stdout:5/176: chown d2/df/d27 1771103 1 2024-03-22T10:42:35.948 INFO:tasks.workunit.client.0.smithi115.stdout:9/207: dread d6/d7/f1e [0,4194304] 0 2024-03-22T10:42:35.953 INFO:tasks.workunit.client.0.smithi115.stdout:6/187: mknod d4/c3c 0 2024-03-22T10:42:35.956 INFO:tasks.workunit.client.0.smithi115.stdout:4/217: rename d1/dc/d14 to d1/dc/d14/d33/d47 22 2024-03-22T10:42:35.958 INFO:tasks.workunit.client.0.smithi115.stdout:9/208: write d6/d7/d19/f27 [3576562,81868] 0 2024-03-22T10:42:35.959 INFO:tasks.workunit.client.0.smithi115.stdout:8/359: truncate d6/dd/d24/d4a/f66 349405 0 2024-03-22T10:42:35.963 INFO:tasks.workunit.client.0.smithi115.stdout:6/188: fdatasync d4/d11/d12/f21 0 2024-03-22T10:42:35.974 INFO:tasks.workunit.client.0.smithi115.stdout:3/130: dread d3/d16/f21 [4194304,4194304] 0 2024-03-22T10:42:35.976 INFO:tasks.workunit.client.0.smithi115.stdout:8/360: dread d6/dd/f29 [0,4194304] 0 2024-03-22T10:42:35.977 INFO:tasks.workunit.client.0.smithi115.stdout:5/177: dread d2/f2e [0,4194304] 0 2024-03-22T10:42:35.985 INFO:tasks.workunit.client.0.smithi115.stdout:3/131: creat d3/d19/d1b/f27 x:0 0 0 2024-03-22T10:42:35.987 INFO:tasks.workunit.client.0.smithi115.stdout:0/163: sync 2024-03-22T10:42:35.987 INFO:tasks.workunit.client.0.smithi115.stdout:2/148: sync 2024-03-22T10:42:35.987 INFO:tasks.workunit.client.0.smithi115.stdout:7/216: sync 2024-03-22T10:42:35.989 INFO:tasks.workunit.client.0.smithi115.stdout:9/209: truncate d6/d7/f11 488894 0 2024-03-22T10:42:35.992 INFO:tasks.workunit.client.0.smithi115.stdout:2/149: chown da/c1a 49899693 1 2024-03-22T10:42:35.992 INFO:tasks.workunit.client.0.smithi115.stdout:0/164: readlink d3/da/l13 0 2024-03-22T10:42:35.993 INFO:tasks.workunit.client.0.smithi115.stdout:1/296: dwrite d0/d12/d17/f33 [0,4194304] 0 2024-03-22T10:42:35.993 INFO:tasks.workunit.client.0.smithi115.stdout:3/132: mknod d3/c28 0 2024-03-22T10:42:35.995 INFO:tasks.workunit.client.0.smithi115.stdout:7/217: dread - d2/d7/d12/d10/f28 zero size 2024-03-22T10:42:35.995 INFO:tasks.workunit.client.0.smithi115.stdout:4/218: dwrite d1/dc/d14/f1d [4194304,4194304] 0 2024-03-22T10:42:36.005 INFO:tasks.workunit.client.0.smithi115.stdout:9/210: mknod d6/d7/d30/c44 0 2024-03-22T10:42:36.005 INFO:tasks.workunit.client.0.smithi115.stdout:0/165: read d3/f1a [357985,120988] 0 2024-03-22T10:42:36.005 INFO:tasks.workunit.client.0.smithi115.stdout:9/211: chown d6/d7 98263 1 2024-03-22T10:42:36.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:35 smithi115 ceph-mon[29111]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:36.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:35 smithi115 ceph-mon[29111]: pgmap v6: 65 pgs: 65 active+clean; 234 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:42:36.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:35 smithi115 ceph-mon[29111]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:36.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:35 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:36.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:35 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:36.011 INFO:tasks.workunit.client.0.smithi115.stdout:1/297: creat d0/d8/d4d/f65 x:0 0 0 2024-03-22T10:42:36.013 INFO:tasks.workunit.client.0.smithi115.stdout:2/150: dread da/f12 [0,4194304] 0 2024-03-22T10:42:36.018 INFO:tasks.workunit.client.0.smithi115.stdout:9/212: symlink d6/d7/d20/d22/d3f/l45 0 2024-03-22T10:42:36.023 INFO:tasks.workunit.client.0.smithi115.stdout:0/166: write d3/d4/d14/f23 [7492263,31456] 0 2024-03-22T10:42:36.023 INFO:tasks.workunit.client.0.smithi115.stdout:5/178: dwrite d2/f4 [4194304,4194304] 0 2024-03-22T10:42:36.025 INFO:tasks.workunit.client.0.smithi115.stdout:3/133: dread d3/d16/f18 [0,4194304] 0 2024-03-22T10:42:36.026 INFO:tasks.workunit.client.0.smithi115.stdout:9/213: read - d6/d7/d20/d22/f2b zero size 2024-03-22T10:42:36.030 INFO:tasks.workunit.client.0.smithi115.stdout:7/218: dread f1 [0,4194304] 0 2024-03-22T10:42:36.031 INFO:tasks.workunit.client.0.smithi115.stdout:9/214: dread - d6/d7/f2d zero size 2024-03-22T10:42:36.033 INFO:tasks.workunit.client.0.smithi115.stdout:6/189: dread d4/d11/f23 [0,4194304] 0 2024-03-22T10:42:36.034 INFO:tasks.workunit.client.0.smithi115.stdout:8/361: dread d6/f44 [0,4194304] 0 2024-03-22T10:42:36.037 INFO:tasks.workunit.client.0.smithi115.stdout:4/219: fdatasync d1/f9 0 2024-03-22T10:42:36.039 INFO:tasks.workunit.client.0.smithi115.stdout:6/190: chown d4/d13/d33 0 1 2024-03-22T10:42:36.062 INFO:tasks.workunit.client.0.smithi115.stdout:7/219: chown d2/d7/d12/d10/d23/d3d/f2a 2148159 1 2024-03-22T10:42:36.063 INFO:tasks.workunit.client.0.smithi115.stdout:3/134: dread d3/d19/d1b/f22 [0,4194304] 0 2024-03-22T10:42:36.064 INFO:tasks.workunit.client.0.smithi115.stdout:6/191: unlink f3 0 2024-03-22T10:42:36.069 INFO:tasks.workunit.client.0.smithi115.stdout:3/135: readlink d3/d16/l1e 0 2024-03-22T10:42:36.073 INFO:tasks.workunit.client.0.smithi115.stdout:2/151: dwrite da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:36.073 INFO:tasks.workunit.client.0.smithi115.stdout:1/298: dwrite d0/d12/d17/f23 [0,4194304] 0 2024-03-22T10:42:36.075 INFO:tasks.workunit.client.0.smithi115.stdout:7/220: creat d2/d7/d12/d10/d19/d42/d46/f48 x:0 0 0 2024-03-22T10:42:36.075 INFO:tasks.workunit.client.0.smithi115.stdout:2/152: dread - da/f1d zero size 2024-03-22T10:42:36.081 INFO:tasks.workunit.client.0.smithi115.stdout:8/362: truncate d6/f20 3076018 0 2024-03-22T10:42:36.082 INFO:tasks.workunit.client.0.smithi115.stdout:9/215: dwrite d6/d7/f1e [0,4194304] 0 2024-03-22T10:42:36.093 INFO:tasks.workunit.client.0.smithi115.stdout:9/216: creat d6/d7/d30/d3b/f46 x:0 0 0 2024-03-22T10:42:36.106 INFO:tasks.workunit.client.0.smithi115.stdout:1/299: creat d0/d12/f66 x:0 0 0 2024-03-22T10:42:36.108 INFO:tasks.workunit.client.0.smithi115.stdout:3/136: dwrite d3/f12 [0,4194304] 0 2024-03-22T10:42:36.114 INFO:tasks.workunit.client.0.smithi115.stdout:7/221: getdents d2/d7/d12 0 2024-03-22T10:42:36.129 INFO:tasks.workunit.client.0.smithi115.stdout:7/222: write d2/d7/d12/d10/d19/f37 [5639407,16598] 0 2024-03-22T10:42:36.137 INFO:tasks.workunit.client.0.smithi115.stdout:2/153: dwrite da/f12 [0,4194304] 0 2024-03-22T10:42:36.139 INFO:tasks.workunit.client.0.smithi115.stdout:2/154: chown c1 138686448 1 2024-03-22T10:42:36.139 INFO:tasks.workunit.client.0.smithi115.stdout:4/220: dread d1/dc/f25 [4194304,4194304] 0 2024-03-22T10:42:36.142 INFO:tasks.workunit.client.0.smithi115.stdout:1/300: write d0/f52 [50500,28273] 0 2024-03-22T10:42:36.144 INFO:tasks.workunit.client.0.smithi115.stdout:8/363: dwrite f5 [0,4194304] 0 2024-03-22T10:42:36.144 INFO:tasks.workunit.client.0.smithi115.stdout:9/217: dwrite d6/d8/f24 [0,4194304] 0 2024-03-22T10:42:36.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:35 smithi150 ceph-mon[34518]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:36.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:35 smithi150 ceph-mon[34518]: pgmap v6: 65 pgs: 65 active+clean; 234 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:42:36.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:35 smithi150 ceph-mon[34518]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:36.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:35 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:36.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:35 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:36.157 INFO:tasks.workunit.client.0.smithi115.stdout:2/155: symlink da/d1c/d22/l29 0 2024-03-22T10:42:36.158 INFO:tasks.workunit.client.0.smithi115.stdout:7/223: rename d2/d7/l18 to d2/d7/d12/d10/d23/l49 0 2024-03-22T10:42:36.162 INFO:tasks.workunit.client.0.smithi115.stdout:2/156: fdatasync da/d1c/d22/d27/f28 0 2024-03-22T10:42:36.164 INFO:tasks.workunit.client.0.smithi115.stdout:4/221: creat d1/d16/d35/d40/f48 x:0 0 0 2024-03-22T10:42:36.171 INFO:tasks.workunit.client.0.smithi115.stdout:4/222: write d1/d30/d42/f45 [789503,26215] 0 2024-03-22T10:42:36.173 INFO:tasks.workunit.client.0.smithi115.stdout:9/218: getdents d6/d7/d30/d3b 0 2024-03-22T10:42:36.179 INFO:tasks.workunit.client.0.smithi115.stdout:9/219: read d6/d7/f1e [2143313,74704] 0 2024-03-22T10:42:36.186 INFO:tasks.workunit.client.0.smithi115.stdout:1/301: dwrite d0/d8/de/d1a/f2f [0,4194304] 0 2024-03-22T10:42:36.186 INFO:tasks.workunit.client.0.smithi115.stdout:8/364: dwrite d6/f17 [0,4194304] 0 2024-03-22T10:42:36.187 INFO:tasks.workunit.client.0.smithi115.stdout:4/223: rename d1/d30/d38 to d1/dc/d14/d33/d49 0 2024-03-22T10:42:36.192 INFO:tasks.workunit.client.0.smithi115.stdout:7/224: truncate d2/d7/d12/d10/d19/f37 1369536 0 2024-03-22T10:42:36.197 INFO:tasks.workunit.client.0.smithi115.stdout:9/220: rename d6/d7/f2a to d6/d8/f47 0 2024-03-22T10:42:36.198 INFO:tasks.workunit.client.0.smithi115.stdout:4/224: creat d1/d30/f4a x:0 0 0 2024-03-22T10:42:36.200 INFO:tasks.workunit.client.0.smithi115.stdout:9/221: write d6/d7/d20/d22/f2b [402874,97322] 0 2024-03-22T10:42:36.201 INFO:tasks.workunit.client.0.smithi115.stdout:1/302: mkdir d0/d12/d17/d67 0 2024-03-22T10:42:36.206 INFO:tasks.workunit.client.0.smithi115.stdout:8/365: rmdir d6/dd/d12/d31 39 2024-03-22T10:42:36.206 INFO:tasks.workunit.client.0.smithi115.stdout:7/225: mkdir d2/d7/d12/d10/d23/d3d/d4a 0 2024-03-22T10:42:36.208 INFO:tasks.workunit.client.0.smithi115.stdout:4/225: symlink d1/d30/l4b 0 2024-03-22T10:42:36.209 INFO:tasks.workunit.client.0.smithi115.stdout:9/222: mkdir d6/d7/d48 0 2024-03-22T10:42:36.212 INFO:tasks.workunit.client.0.smithi115.stdout:1/303: mkdir d0/d68 0 2024-03-22T10:42:36.212 INFO:tasks.workunit.client.0.smithi115.stdout:7/226: creat d2/d7/d12/f4b x:0 0 0 2024-03-22T10:42:36.213 INFO:tasks.workunit.client.0.smithi115.stdout:9/223: write d6/d7/f2d [997166,104322] 0 2024-03-22T10:42:36.215 INFO:tasks.workunit.client.0.smithi115.stdout:8/366: rename d6/dd/f11 to d6/d49/d68/f6f 0 2024-03-22T10:42:36.216 INFO:tasks.workunit.client.0.smithi115.stdout:9/224: write f3 [2137340,30336] 0 2024-03-22T10:42:36.220 INFO:tasks.workunit.client.0.smithi115.stdout:4/226: mkdir d1/d30/d4c 0 2024-03-22T10:42:36.221 INFO:tasks.workunit.client.0.smithi115.stdout:7/227: mkdir d2/d4c 0 2024-03-22T10:42:36.222 INFO:tasks.workunit.client.0.smithi115.stdout:9/225: symlink d6/l49 0 2024-03-22T10:42:36.226 INFO:tasks.workunit.client.0.smithi115.stdout:4/227: symlink d1/dc/d14/d33/l4d 0 2024-03-22T10:42:36.226 INFO:tasks.workunit.client.0.smithi115.stdout:8/367: creat d6/dd/f70 x:0 0 0 2024-03-22T10:42:36.226 INFO:tasks.workunit.client.0.smithi115.stdout:9/226: symlink d6/d7/d20/d22/d23/l4a 0 2024-03-22T10:42:36.228 INFO:tasks.workunit.client.0.smithi115.stdout:7/228: rename d2/d7/d12/d10/d23/d33/c41 to d2/d7/d12/d10/d19/d42/c4d 0 2024-03-22T10:42:36.230 INFO:tasks.workunit.client.0.smithi115.stdout:9/227: creat d6/d7/d19/f4b x:0 0 0 2024-03-22T10:42:36.232 INFO:tasks.workunit.client.0.smithi115.stdout:8/368: symlink d6/d49/d68/l71 0 2024-03-22T10:42:36.241 INFO:tasks.workunit.client.0.smithi115.stdout:1/304: dwrite d0/d22/f46 [0,4194304] 0 2024-03-22T10:42:36.241 INFO:tasks.workunit.client.0.smithi115.stdout:1/305: fdatasync d0/d8/d4d/f65 0 2024-03-22T10:42:36.245 INFO:tasks.workunit.client.0.smithi115.stdout:9/228: fdatasync d6/d7/d19/f21 0 2024-03-22T10:42:36.253 INFO:tasks.workunit.client.0.smithi115.stdout:7/229: link d2/d7/d12/d10/f25 d2/d7/d12/d10/d23/f4e 0 2024-03-22T10:42:36.253 INFO:tasks.workunit.client.0.smithi115.stdout:8/369: dwrite d6/dd/d12/f27 [0,4194304] 0 2024-03-22T10:42:36.259 INFO:tasks.workunit.client.0.smithi115.stdout:9/229: unlink d6/d7/d19/f4b 0 2024-03-22T10:42:36.266 INFO:tasks.workunit.client.0.smithi115.stdout:1/306: truncate d0/d22/f5e 2070857 0 2024-03-22T10:42:36.272 INFO:tasks.workunit.client.0.smithi115.stdout:7/230: link d2/d7/d12/d10/f26 d2/d3a/d45/f4f 0 2024-03-22T10:42:36.275 INFO:tasks.workunit.client.0.smithi115.stdout:1/307: write d0/d8/db/d25/f44 [1638405,128870] 0 2024-03-22T10:42:36.278 INFO:tasks.workunit.client.0.smithi115.stdout:4/228: dread d1/dc/d14/f1b [0,4194304] 0 2024-03-22T10:42:36.282 INFO:tasks.workunit.client.0.smithi115.stdout:9/230: dwrite f0 [0,4194304] 0 2024-03-22T10:42:36.283 INFO:tasks.workunit.client.0.smithi115.stdout:8/370: link d6/dd/d12/d31/f5c d6/dd/d24/f72 0 2024-03-22T10:42:36.306 INFO:tasks.workunit.client.0.smithi115.stdout:8/371: link d6/dd/d24/l53 d6/d49/l73 0 2024-03-22T10:42:36.313 INFO:tasks.workunit.client.0.smithi115.stdout:9/231: dwrite d6/f2e [0,4194304] 0 2024-03-22T10:42:36.315 INFO:tasks.workunit.client.0.smithi115.stdout:4/229: dwrite d1/dc/d14/f3e [0,4194304] 0 2024-03-22T10:42:36.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.325+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f84841a23a0 con 0x7f84840fea30 2024-03-22T10:42:36.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.326+0000 7f8482000700 1 -- 172.21.15.115:0/351355899 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f847801fbd0 con 0x7f84840fea30 2024-03-22T10:42:36.319 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:42:36.319 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:42:36.319 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:42:36.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:42:36.320 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:42:36.321 INFO:tasks.workunit.client.0.smithi115.stdout:7/231: dwrite d2/f3f [0,4194304] 0 2024-03-22T10:42:36.323 INFO:tasks.workunit.client.0.smithi115.stdout:9/232: write d6/d7/f2d [177201,66209] 0 2024-03-22T10:42:36.323 INFO:tasks.workunit.client.0.smithi115.stdout:7/232: fdatasync d2/d7/d12/f4b 0 2024-03-22T10:42:36.324 INFO:tasks.workunit.client.0.smithi115.stdout:1/308: dwrite d0/d8/d4d/d56/f57 [0,4194304] 0 2024-03-22T10:42:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.334+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f846c039280 msgr2=0x7f846c03b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.334+0000 7f848c400700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f846c039280 0x7f846c03b730 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8470000c00 tx=0x7f8470019040 comp rx=0 tx=0).stop 2024-03-22T10:42:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.334+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840fea30 msgr2=0x7f84841045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.334+0000 7f848c400700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840fea30 0x7f84841045b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f847800e940 tx=0x7f847800ed00 comp rx=0 tx=0).stop 2024-03-22T10:42:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.335+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 shutdown_connections 2024-03-22T10:42:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.335+0000 7f848c400700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f846c039280 0x7f846c03b730 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.335+0000 7f848c400700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84840fea30 0x7f84841045b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.335+0000 7f848c400700 1 --2- 172.21.15.115:0/351355899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84840ff380 0x7f8484104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.335+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 >> 172.21.15.115:0/351355899 conn(0x7f84840fa5e0 msgr2=0x7f8484107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.335+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 shutdown_connections 2024-03-22T10:42:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.335+0000 7f848c400700 1 -- 172.21.15.115:0/351355899 wait complete. 2024-03-22T10:42:36.330 INFO:tasks.workunit.client.0.smithi115.stdout:7/233: unlink f1 0 2024-03-22T10:42:36.334 INFO:tasks.workunit.client.0.smithi115.stdout:1/309: symlink d0/d8/l69 0 2024-03-22T10:42:36.334 INFO:tasks.workunit.client.0.smithi115.stdout:9/233: creat d6/d7/d20/f4c x:0 0 0 2024-03-22T10:42:36.337 INFO:tasks.workunit.client.0.smithi115.stdout:9/234: stat d6/d8/fd 0 2024-03-22T10:42:36.338 INFO:tasks.workunit.client.0.smithi115.stdout:7/234: write d2/d7/d12/d10/f36 [861271,81987] 0 2024-03-22T10:42:36.341 INFO:tasks.workunit.client.0.smithi115.stdout:9/235: dread - d6/d7/d20/d38/f40 zero size 2024-03-22T10:42:36.343 INFO:tasks.workunit.client.0.smithi115.stdout:7/235: dread - d2/d7/d12/d10/d19/d42/d46/f48 zero size 2024-03-22T10:42:36.348 INFO:tasks.workunit.client.0.smithi115.stdout:7/236: rmdir d2/d7/d12/d10/d23/d33 39 2024-03-22T10:42:36.353 INFO:tasks.workunit.client.0.smithi115.stdout:7/237: mknod d2/d7/d12/d10/d19/c50 0 2024-03-22T10:42:36.362 INFO:tasks.workunit.client.0.smithi115.stdout:1/310: dwrite d0/d8/d4d/d56/f57 [0,4194304] 0 2024-03-22T10:42:36.368 INFO:tasks.workunit.client.0.smithi115.stdout:7/238: rename d2/d7/d12/d10/d23/d3d/c44 to d2/d7/d12/d10/d23/d33/c51 0 2024-03-22T10:42:36.374 INFO:tasks.workunit.client.0.smithi115.stdout:1/311: getdents d0/d12/d2b 0 2024-03-22T10:42:36.377 INFO:tasks.workunit.client.0.smithi115.stdout:1/312: dread - d0/d8/d4d/f65 zero size 2024-03-22T10:42:36.380 INFO:tasks.workunit.client.0.smithi115.stdout:1/313: symlink d0/d58/l6a 0 2024-03-22T10:42:36.383 INFO:tasks.workunit.client.0.smithi115.stdout:1/314: creat d0/d8/d4d/d56/f6b x:0 0 0 2024-03-22T10:42:36.386 INFO:tasks.workunit.client.0.smithi115.stdout:1/315: read d0/d12/f1c [746047,85683] 0 2024-03-22T10:42:36.399 INFO:tasks.workunit.client.0.smithi115.stdout:9/236: dread d6/d7/d19/f27 [0,4194304] 0 2024-03-22T10:42:36.400 INFO:tasks.workunit.client.0.smithi115.stdout:9/237: stat d6/d7/d20/d22/d3f 0 2024-03-22T10:42:36.404 INFO:tasks.workunit.client.0.smithi115.stdout:9/238: rename d6/d7/d30/d3b/f3d to d6/d7/d30/d37/f4d 0 2024-03-22T10:42:36.405 INFO:tasks.workunit.client.0.smithi115.stdout:9/239: write d6/d7/f17 [3039801,13592] 0 2024-03-22T10:42:36.408 INFO:tasks.workunit.client.0.smithi115.stdout:9/240: mkdir d6/d7/d19/d4e 0 2024-03-22T10:42:36.410 INFO:tasks.workunit.client.0.smithi115.stdout:9/241: write d6/d7/d30/d3b/f46 [606217,67197] 0 2024-03-22T10:42:36.423 INFO:tasks.workunit.client.0.smithi115.stdout:9/242: dwrite d6/d7/d19/f3a [0,4194304] 0 2024-03-22T10:42:36.429 INFO:tasks.workunit.client.0.smithi115.stdout:9/243: link d6/d8/l18 d6/d7/d30/d37/l4f 0 2024-03-22T10:42:36.431 INFO:tasks.workunit.client.0.smithi115.stdout:9/244: stat d6/l49 0 2024-03-22T10:42:36.436 INFO:tasks.workunit.client.0.smithi115.stdout:9/245: dread d6/d7/f11 [0,4194304] 0 2024-03-22T10:42:36.439 INFO:tasks.workunit.client.0.smithi115.stdout:9/246: creat d6/d7/d48/f50 x:0 0 0 2024-03-22T10:42:36.454 INFO:tasks.workunit.client.0.smithi115.stdout:9/247: dwrite d6/d7/d20/d22/f2b [0,4194304] 0 2024-03-22T10:42:36.481 INFO:tasks.workunit.client.0.smithi115.stdout:0/167: sync 2024-03-22T10:42:36.485 INFO:tasks.workunit.client.0.smithi115.stdout:0/168: mknod d3/c2f 0 2024-03-22T10:42:36.487 INFO:tasks.workunit.client.0.smithi115.stdout:9/248: dread d6/d7/f17 [0,4194304] 0 2024-03-22T10:42:36.487 INFO:tasks.workunit.client.0.smithi115.stdout:0/169: write d3/d7/d1c/f22 [1858595,65194] 0 2024-03-22T10:42:36.489 INFO:tasks.workunit.client.0.smithi115.stdout:9/249: stat d6/d7/d19/c33 0 2024-03-22T10:42:36.497 INFO:tasks.workunit.client.0.smithi115.stdout:9/250: link d6/d8/fd d6/d7/d20/d22/d23/f51 0 2024-03-22T10:42:36.511 INFO:tasks.workunit.client.0.smithi115.stdout:0/170: dwrite d3/da/f26 [0,4194304] 0 2024-03-22T10:42:36.512 INFO:tasks.workunit.client.0.smithi115.stdout:0/171: write d3/d4/d14/f15 [1387449,117341] 0 2024-03-22T10:42:36.513 INFO:tasks.workunit.client.0.smithi115.stdout:0/172: readlink d3/d4/d14/l28 0 2024-03-22T10:42:36.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.540+0000 7f5074e00700 1 -- 172.21.15.115:0/393963739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50700fffd0 msgr2=0x7f50701003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:36.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.540+0000 7f5074e00700 1 --2- 172.21.15.115:0/393963739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50700fffd0 0x7f50701003e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5060009910 tx=0x7f5060009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:36.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.540+0000 7f5074e00700 1 -- 172.21.15.115:0/393963739 shutdown_connections 2024-03-22T10:42:36.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.540+0000 7f5074e00700 1 --2- 172.21.15.115:0/393963739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50701011d0 0x7f5070101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:36.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.540+0000 7f5074e00700 1 --2- 172.21.15.115:0/393963739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50700fffd0 0x7f50701003e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:36.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.540+0000 7f5074e00700 1 -- 172.21.15.115:0/393963739 >> 172.21.15.115:0/393963739 conn(0x7f50700fb5a0 msgr2=0x7f50700fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:36.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.540+0000 7f5074e00700 1 -- 172.21.15.115:0/393963739 shutdown_connections 2024-03-22T10:42:36.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.540+0000 7f5074e00700 1 -- 172.21.15.115:0/393963739 wait complete. 2024-03-22T10:42:36.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.541+0000 7f5074e00700 1 Processor -- start 2024-03-22T10:42:36.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.541+0000 7f5074e00700 1 -- start start 2024-03-22T10:42:36.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.541+0000 7f5074e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50700fffd0 0x7f5070193660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:36.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.541+0000 7f5074e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50701011d0 0x7f5070193ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:36.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.541+0000 7f5074e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50701941c0 con 0x7f50700fffd0 2024-03-22T10:42:36.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.541+0000 7f5074e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5070194300 con 0x7f50701011d0 2024-03-22T10:42:36.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.542+0000 7f506f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50700fffd0 0x7f5070193660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:36.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.542+0000 7f506f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50700fffd0 0x7f5070193660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39202/0 (socket says 172.21.15.115:39202) 2024-03-22T10:42:36.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.542+0000 7f506f400700 1 -- 172.21.15.115:0/1159483323 learned_addr learned my addr 172.21.15.115:0/1159483323 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:36.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.542+0000 7f506ea00700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50701011d0 0x7f5070193ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:36.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.542+0000 7f506f400700 1 -- 172.21.15.115:0/1159483323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50701011d0 msgr2=0x7f5070193ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:36.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.542+0000 7f506f400700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50701011d0 0x7f5070193ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:36.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.542+0000 7f506f400700 1 -- 172.21.15.115:0/1159483323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600095f0 con 0x7f50700fffd0 2024-03-22T10:42:36.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.543+0000 7f506f400700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50700fffd0 0x7f5070193660 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f506000bde0 tx=0x7f506000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:36.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.543+0000 7f5067e00700 1 -- 172.21.15.115:0/1159483323 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5060012570 con 0x7f50700fffd0 2024-03-22T10:42:36.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.543+0000 7f5067e00700 1 -- 172.21.15.115:0/1159483323 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5060012cb0 con 0x7f50700fffd0 2024-03-22T10:42:36.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.543+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5070198d50 con 0x7f50700fffd0 2024-03-22T10:42:36.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.543+0000 7f5067e00700 1 -- 172.21.15.115:0/1159483323 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50600038b0 con 0x7f50700fffd0 2024-03-22T10:42:36.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.544+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5070199260 con 0x7f50700fffd0 2024-03-22T10:42:36.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.545+0000 7f5067e00700 1 -- 172.21.15.115:0/1159483323 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 29) v1 ==== 46247+0+0 (secure 0 0 0) 0x7f50600106e0 con 0x7f50700fffd0 2024-03-22T10:42:36.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.545+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5070106a00 con 0x7f50700fffd0 2024-03-22T10:42:36.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.545+0000 7f5067e00700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5054039230 0x7f505403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:36.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.546+0000 7f5067e00700 1 -- 172.21.15.115:0/1159483323 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f5060052820 con 0x7f50700fffd0 2024-03-22T10:42:36.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.546+0000 7f506ea00700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5054039230 0x7f505403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:36.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.547+0000 7f506ea00700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5054039230 0x7f505403b6e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f505c009a10 tx=0x7f505c009190 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.551+0000 7f5067e00700 1 -- 172.21.15.115:0/1159483323 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f5060012800 con 0x7f50700fffd0 2024-03-22T10:42:36.548 INFO:tasks.workunit.client.0.smithi115.stdout:9/251: write d6/d8/f1a [1091498,92757] 0 2024-03-22T10:42:36.551 INFO:tasks.workunit.client.0.smithi115.stdout:9/252: chown d6/d7/d19/c34 188335 1 2024-03-22T10:42:36.551 INFO:tasks.workunit.client.0.smithi115.stdout:0/173: dread d3/d4/d14/f23 [4194304,4194304] 0 2024-03-22T10:42:36.552 INFO:tasks.workunit.client.0.smithi115.stdout:0/174: readlink d3/l9 0 2024-03-22T10:42:36.553 INFO:tasks.workunit.client.0.smithi115.stdout:0/175: readlink d3/da/l13 0 2024-03-22T10:42:36.555 INFO:tasks.workunit.client.0.smithi115.stdout:0/176: chown d3/d7/d1b/l29 6451 1 2024-03-22T10:42:36.556 INFO:tasks.workunit.client.0.smithi115.stdout:9/253: link d6/l49 d6/d7/d20/d22/d23/l52 0 2024-03-22T10:42:36.558 INFO:tasks.workunit.client.0.smithi115.stdout:9/254: mkdir d6/d7/d48/d53 0 2024-03-22T10:42:36.560 INFO:tasks.workunit.client.0.smithi115.stdout:0/177: rename d3/d4/d14/l28 to d3/d7/d1c/l30 0 2024-03-22T10:42:36.562 INFO:tasks.workunit.client.0.smithi115.stdout:0/178: write d3/d7/d1c/f1e [1933949,117000] 0 2024-03-22T10:42:36.563 INFO:tasks.workunit.client.0.smithi115.stdout:9/255: rename d6/d8/f24 to d6/d8/d3c/f54 0 2024-03-22T10:42:36.566 INFO:tasks.workunit.client.0.smithi115.stdout:0/179: dread - d3/d7/d1b/f24 zero size 2024-03-22T10:42:36.571 INFO:tasks.workunit.client.0.smithi115.stdout:9/256: dread d6/d7/d30/d3b/f46 [0,4194304] 0 2024-03-22T10:42:36.572 INFO:tasks.workunit.client.0.smithi115.stdout:9/257: fdatasync d6/d7/f16 0 2024-03-22T10:42:36.575 INFO:tasks.workunit.client.0.smithi115.stdout:9/258: write d6/d7/f17 [5026069,23085] 0 2024-03-22T10:42:36.580 INFO:tasks.workunit.client.0.smithi115.stdout:0/180: dread d3/d4/d14/f23 [0,4194304] 0 2024-03-22T10:42:36.583 INFO:tasks.workunit.client.0.smithi115.stdout:0/181: symlink d3/da/l31 0 2024-03-22T10:42:36.584 INFO:tasks.workunit.client.0.smithi115.stdout:9/259: truncate d6/d7/f1e 503749 0 2024-03-22T10:42:36.585 INFO:tasks.workunit.client.0.smithi115.stdout:0/182: write d3/d7/d1b/f24 [292224,47093] 0 2024-03-22T10:42:36.588 INFO:tasks.workunit.client.0.smithi115.stdout:0/183: mknod d3/d7/d1b/c32 0 2024-03-22T10:42:36.588 INFO:tasks.workunit.client.0.smithi115.stdout:9/260: creat d6/d7/d20/f55 x:0 0 0 2024-03-22T10:42:36.591 INFO:tasks.workunit.client.0.smithi115.stdout:0/184: creat d3/f33 x:0 0 0 2024-03-22T10:42:36.591 INFO:tasks.workunit.client.0.smithi115.stdout:9/261: stat d6/d7/d30/d37/c39 0 2024-03-22T10:42:36.593 INFO:tasks.workunit.client.0.smithi115.stdout:0/185: creat d3/d4/d14/f34 x:0 0 0 2024-03-22T10:42:36.595 INFO:tasks.workunit.client.0.smithi115.stdout:0/186: mkdir d3/d7/d1b/d35 0 2024-03-22T10:42:36.599 INFO:tasks.workunit.client.0.smithi115.stdout:0/187: rename d3/d4/c16 to d3/da/c36 0 2024-03-22T10:42:36.601 INFO:tasks.workunit.client.0.smithi115.stdout:6/192: sync 2024-03-22T10:42:36.601 INFO:tasks.workunit.client.0.smithi115.stdout:5/179: sync 2024-03-22T10:42:36.601 INFO:tasks.workunit.client.0.smithi115.stdout:9/262: dread d6/d7/d19/f25 [0,4194304] 0 2024-03-22T10:42:36.603 INFO:tasks.workunit.client.0.smithi115.stdout:5/180: fsync d2/df/d13/f23 0 2024-03-22T10:42:36.605 INFO:tasks.workunit.client.0.smithi115.stdout:6/193: dread - d4/d13/f2e zero size 2024-03-22T10:42:36.607 INFO:tasks.workunit.client.0.smithi115.stdout:6/194: creat d4/d13/f3d x:0 0 0 2024-03-22T10:42:36.608 INFO:tasks.workunit.client.0.smithi115.stdout:5/181: mknod d2/d15/d1b/d33/c37 0 2024-03-22T10:42:36.610 INFO:tasks.workunit.client.0.smithi115.stdout:5/182: mkdir d2/d15/d1b/d33/d38 0 2024-03-22T10:42:36.612 INFO:tasks.workunit.client.0.smithi115.stdout:6/195: chown d4/d11/c22 12322 1 2024-03-22T10:42:36.614 INFO:tasks.workunit.client.0.smithi115.stdout:5/183: creat d2/d15/f39 x:0 0 0 2024-03-22T10:42:36.614 INFO:tasks.workunit.client.0.smithi115.stdout:6/196: readlink d4/la 0 2024-03-22T10:42:36.617 INFO:tasks.workunit.client.0.smithi115.stdout:0/188: dwrite d3/d7/d1b/f24 [0,4194304] 0 2024-03-22T10:42:36.621 INFO:tasks.workunit.client.0.smithi115.stdout:6/197: mknod d4/d13/c3e 0 2024-03-22T10:42:36.623 INFO:tasks.workunit.client.0.smithi115.stdout:9/263: dwrite d6/d7/d30/d3b/f46 [0,4194304] 0 2024-03-22T10:42:36.626 INFO:tasks.workunit.client.0.smithi115.stdout:9/264: chown d6/d7/ff 3192467 1 2024-03-22T10:42:36.627 INFO:tasks.workunit.client.0.smithi115.stdout:5/184: creat d2/d15/d1b/d33/d38/f3a x:0 0 0 2024-03-22T10:42:36.629 INFO:tasks.workunit.client.0.smithi115.stdout:6/198: chown d4/c28 61868400 1 2024-03-22T10:42:36.631 INFO:tasks.workunit.client.0.smithi115.stdout:0/189: rename d3/d4/d14/c2d to d3/d4/c37 0 2024-03-22T10:42:36.631 INFO:tasks.workunit.client.0.smithi115.stdout:6/199: read - d4/d11/f29 zero size 2024-03-22T10:42:36.634 INFO:tasks.workunit.client.0.smithi115.stdout:6/200: readlink d4/ld 0 2024-03-22T10:42:36.637 INFO:tasks.workunit.client.0.smithi115.stdout:9/265: rename d6/d8/fd to d6/d7/d20/d38/f56 0 2024-03-22T10:42:36.638 INFO:tasks.workunit.client.0.smithi115.stdout:0/190: mknod d3/d7/d1b/d1f/c38 0 2024-03-22T10:42:36.638 INFO:tasks.workunit.client.0.smithi115.stdout:2/157: sync 2024-03-22T10:42:36.638 INFO:tasks.workunit.client.0.smithi115.stdout:3/137: sync 2024-03-22T10:42:36.641 INFO:tasks.workunit.client.0.smithi115.stdout:6/201: mknod d4/d11/c3f 0 2024-03-22T10:42:36.643 INFO:tasks.workunit.client.0.smithi115.stdout:3/138: stat d3/d16/f18 0 2024-03-22T10:42:36.646 INFO:tasks.workunit.client.0.smithi115.stdout:6/202: write d4/d11/d12/f31 [1027348,63563] 0 2024-03-22T10:42:36.650 INFO:tasks.workunit.client.0.smithi115.stdout:3/139: write d3/d19/d1b/f27 [266439,56679] 0 2024-03-22T10:42:36.650 INFO:tasks.workunit.client.0.smithi115.stdout:2/158: mkdir da/d25/d2a 0 2024-03-22T10:42:36.651 INFO:tasks.workunit.client.0.smithi115.stdout:0/191: truncate d3/d7/d1b/f2a 1076069 0 2024-03-22T10:42:36.654 INFO:tasks.workunit.client.0.smithi115.stdout:5/185: dread d2/fe [0,4194304] 0 2024-03-22T10:42:36.655 INFO:tasks.workunit.client.0.smithi115.stdout:6/203: chown d4/l20 2081 1 2024-03-22T10:42:36.657 INFO:tasks.workunit.client.0.smithi115.stdout:5/186: chown d2/df/f10 206 1 2024-03-22T10:42:36.658 INFO:tasks.workunit.client.0.smithi115.stdout:9/266: creat d6/d7/d20/f57 x:0 0 0 2024-03-22T10:42:36.660 INFO:tasks.workunit.client.0.smithi115.stdout:3/140: unlink d3/d16/f18 0 2024-03-22T10:42:36.662 INFO:tasks.workunit.client.0.smithi115.stdout:0/192: mknod d3/da/c39 0 2024-03-22T10:42:36.662 INFO:tasks.workunit.client.0.smithi115.stdout:3/141: write d3/f25 [343817,122791] 0 2024-03-22T10:42:36.662 INFO:tasks.workunit.client.0.smithi115.stdout:9/267: read d6/d7/d19/f21 [17378,41056] 0 2024-03-22T10:42:36.664 INFO:tasks.workunit.client.0.smithi115.stdout:0/193: readlink d3/l12 0 2024-03-22T10:42:36.665 INFO:tasks.workunit.client.0.smithi115.stdout:6/204: chown d4/f24 193805767 1 2024-03-22T10:42:36.666 INFO:tasks.workunit.client.0.smithi115.stdout:3/142: write d3/d16/f1c [5171684,103550] 0 2024-03-22T10:42:36.671 INFO:tasks.workunit.client.0.smithi115.stdout:9/268: dread d6/d7/d30/d37/f4d [0,4194304] 0 2024-03-22T10:42:36.671 INFO:tasks.workunit.client.0.smithi115.stdout:0/194: truncate d3/d4/d14/f34 729724 0 2024-03-22T10:42:36.678 INFO:tasks.workunit.client.0.smithi115.stdout:5/187: rmdir d2/df/d1e/d2b 0 2024-03-22T10:42:36.681 INFO:tasks.workunit.client.0.smithi115.stdout:6/205: rename d4/d13/l14 to d4/d11/d12/d1c/l40 0 2024-03-22T10:42:36.690 INFO:tasks.workunit.client.0.smithi115.stdout:3/143: getdents d3 0 2024-03-22T10:42:36.693 INFO:tasks.workunit.client.0.smithi115.stdout:5/188: rename d2/df/d1e to d2/df/d3b 0 2024-03-22T10:42:36.695 INFO:tasks.workunit.client.0.smithi115.stdout:0/195: dwrite d3/d7/d1c/f22 [0,4194304] 0 2024-03-22T10:42:36.699 INFO:tasks.workunit.client.0.smithi115.stdout:2/159: dwrite da/f14 [0,4194304] 0 2024-03-22T10:42:36.700 INFO:tasks.workunit.client.0.smithi115.stdout:5/189: chown d2/df/d13/c17 50609 1 2024-03-22T10:42:36.701 INFO:tasks.workunit.client.0.smithi115.stdout:5/190: write d2/d15/f39 [13824,31549] 0 2024-03-22T10:42:36.702 INFO:tasks.workunit.client.0.smithi115.stdout:2/160: chown da/d1c/d22/c26 37133 1 2024-03-22T10:42:36.702 INFO:tasks.workunit.client.0.smithi115.stdout:9/269: truncate d6/d7/f17 3884434 0 2024-03-22T10:42:36.706 INFO:tasks.workunit.client.0.smithi115.stdout:2/161: chown da/d1c/d22/d27 2922 1 2024-03-22T10:42:36.711 INFO:tasks.workunit.client.0.smithi115.stdout:5/191: creat d2/df/f3c x:0 0 0 2024-03-22T10:42:36.711 INFO:tasks.workunit.client.0.smithi115.stdout:2/162: write da/f1b [405991,25205] 0 2024-03-22T10:42:36.713 INFO:tasks.workunit.client.0.smithi115.stdout:9/270: creat d6/d7/d30/d3b/f58 x:0 0 0 2024-03-22T10:42:36.718 INFO:tasks.workunit.client.0.smithi115.stdout:2/163: creat da/d1c/f2b x:0 0 0 2024-03-22T10:42:36.718 INFO:tasks.workunit.client.0.smithi115.stdout:6/206: dwrite d4/f5 [4194304,4194304] 0 2024-03-22T10:42:36.720 INFO:tasks.workunit.client.0.smithi115.stdout:5/192: creat d2/d15/d1b/d33/d38/f3d x:0 0 0 2024-03-22T10:42:36.721 INFO:tasks.workunit.client.0.smithi115.stdout:2/164: stat da/d1c/f2b 0 2024-03-22T10:42:36.722 INFO:tasks.workunit.client.0.smithi115.stdout:9/271: mkdir d6/d8/d3c/d59 0 2024-03-22T10:42:36.726 INFO:tasks.workunit.client.0.smithi115.stdout:2/165: readlink da/d1c/d22/l29 0 2024-03-22T10:42:36.729 INFO:tasks.workunit.client.0.smithi115.stdout:3/144: dwrite d3/f9 [0,4194304] 0 2024-03-22T10:42:36.731 INFO:tasks.workunit.client.0.smithi115.stdout:2/166: creat da/d1c/d22/d27/f2c x:0 0 0 2024-03-22T10:42:36.735 INFO:tasks.workunit.client.0.smithi115.stdout:3/145: mkdir d3/d16/d29 0 2024-03-22T10:42:36.741 INFO:tasks.workunit.client.0.smithi115.stdout:2/167: mknod da/d25/c2d 0 2024-03-22T10:42:36.750 INFO:tasks.workunit.client.0.smithi115.stdout:2/168: creat da/d1c/d22/f2e x:0 0 0 2024-03-22T10:42:36.751 INFO:tasks.workunit.client.0.smithi115.stdout:6/207: dwrite d4/d11/d12/d1c/f2d [0,4194304] 0 2024-03-22T10:42:36.751 INFO:tasks.workunit.client.0.smithi115.stdout:9/272: dwrite d6/d8/f1c [0,4194304] 0 2024-03-22T10:42:36.755 INFO:tasks.workunit.client.0.smithi115.stdout:2/169: symlink da/d1c/l2f 0 2024-03-22T10:42:36.765 INFO:tasks.workunit.client.0.smithi115.stdout:6/208: mknod d4/d11/d12/d1c/c41 0 2024-03-22T10:42:36.774 INFO:tasks.workunit.client.0.smithi115.stdout:6/209: creat d4/d13/d33/f42 x:0 0 0 2024-03-22T10:42:36.775 INFO:tasks.workunit.client.0.smithi115.stdout:5/193: dread d2/f1f [0,4194304] 0 2024-03-22T10:42:36.778 INFO:tasks.workunit.client.0.smithi115.stdout:5/194: write d2/df/f3c [226132,66450] 0 2024-03-22T10:42:36.780 INFO:tasks.workunit.client.0.smithi115.stdout:1/316: sync 2024-03-22T10:42:36.780 INFO:tasks.workunit.client.0.smithi115.stdout:7/239: sync 2024-03-22T10:42:36.780 INFO:tasks.workunit.client.0.smithi115.stdout:8/372: sync 2024-03-22T10:42:36.780 INFO:tasks.workunit.client.0.smithi115.stdout:4/230: sync 2024-03-22T10:42:36.783 INFO:tasks.workunit.client.0.smithi115.stdout:6/210: creat d4/d11/d12/d1c/f43 x:0 0 0 2024-03-22T10:42:36.785 INFO:tasks.workunit.client.0.smithi115.stdout:7/240: chown d2/d7/d12/c2b 901 1 2024-03-22T10:42:36.790 INFO:tasks.workunit.client.0.smithi115.stdout:4/231: write d1/d16/d35/d40/f48 [264228,130174] 0 2024-03-22T10:42:36.791 INFO:tasks.workunit.client.0.smithi115.stdout:5/195: mknod d2/d15/d1b/c3e 0 2024-03-22T10:42:36.791 INFO:tasks.workunit.client.0.smithi115.stdout:3/146: dwrite d3/f7 [0,4194304] 0 2024-03-22T10:42:36.795 INFO:tasks.workunit.client.0.smithi115.stdout:8/373: symlink d6/l74 0 2024-03-22T10:42:36.795 INFO:tasks.workunit.client.0.smithi115.stdout:5/196: write d2/d15/d1b/d33/d38/f3a [1032005,53853] 0 2024-03-22T10:42:36.797 INFO:tasks.workunit.client.0.smithi115.stdout:2/170: rename da/d1c/d22 to da/d25/d30 0 2024-03-22T10:42:36.800 INFO:tasks.workunit.client.0.smithi115.stdout:9/273: write d6/d7/f17 [2881148,4331] 0 2024-03-22T10:42:36.807 INFO:tasks.workunit.client.0.smithi115.stdout:5/197: mknod d2/df/d13/c3f 0 2024-03-22T10:42:36.807 INFO:tasks.workunit.client.0.smithi115.stdout:6/211: creat d4/d11/f44 x:0 0 0 2024-03-22T10:42:36.810 INFO:tasks.workunit.client.0.smithi115.stdout:9/274: truncate d6/d7/d48/f50 372194 0 2024-03-22T10:42:36.812 INFO:tasks.workunit.client.0.smithi115.stdout:8/374: symlink d6/dd/d12/l75 0 2024-03-22T10:42:36.815 INFO:tasks.workunit.client.0.smithi115.stdout:9/275: mknod d6/c5a 0 2024-03-22T10:42:36.822 INFO:tasks.workunit.client.0.smithi115.stdout:1/317: dwrite d0/d8/de/d1a/f4a [0,4194304] 0 2024-03-22T10:42:36.825 INFO:tasks.workunit.client.0.smithi115.stdout:6/212: truncate d4/d11/d12/d1c/f2d 4867333 0 2024-03-22T10:42:36.832 INFO:tasks.workunit.client.0.smithi115.stdout:7/241: dwrite d2/d7/f13 [4194304,4194304] 0 2024-03-22T10:42:36.834 INFO:tasks.workunit.client.0.smithi115.stdout:8/375: write d6/dd/d12/f27 [4320352,5352] 0 2024-03-22T10:42:36.834 INFO:tasks.workunit.client.0.smithi115.stdout:1/318: mknod d0/d8/d4d/d56/c6c 0 2024-03-22T10:42:36.836 INFO:tasks.workunit.client.0.smithi115.stdout:3/147: dwrite d3/f8 [8388608,4194304] 0 2024-03-22T10:42:36.840 INFO:tasks.workunit.client.0.smithi115.stdout:6/213: write d4/d11/f2a [4331997,55553] 0 2024-03-22T10:42:36.840 INFO:tasks.workunit.client.0.smithi115.stdout:1/319: readlink d0/d8/ld 0 2024-03-22T10:42:36.841 INFO:tasks.workunit.client.0.smithi115.stdout:7/242: chown d2/d7/f13 637 1 2024-03-22T10:42:36.845 INFO:tasks.workunit.client.0.smithi115.stdout:5/198: getdents d2/d15/d1b 0 2024-03-22T10:42:36.845 INFO:tasks.workunit.client.0.smithi115.stdout:2/171: rmdir da/d1c 39 2024-03-22T10:42:36.854 INFO:tasks.workunit.client.0.smithi115.stdout:1/320: readlink d0/d12/l29 0 2024-03-22T10:42:36.857 INFO:tasks.workunit.client.0.smithi115.stdout:5/199: write d2/df/d3b/f36 [894459,117820] 0 2024-03-22T10:42:36.861 INFO:tasks.workunit.client.0.smithi115.stdout:9/276: dread d6/d7/d30/d3b/f46 [0,4194304] 0 2024-03-22T10:42:36.867 INFO:tasks.workunit.client.0.smithi115.stdout:8/376: creat d6/dd/f76 x:0 0 0 2024-03-22T10:42:36.870 INFO:tasks.workunit.client.0.smithi115.stdout:5/200: fsync d2/fc 0 2024-03-22T10:42:36.872 INFO:tasks.workunit.client.0.smithi115.stdout:8/377: write f5 [3445464,108861] 0 2024-03-22T10:42:36.877 INFO:tasks.workunit.client.0.smithi115.stdout:3/148: write d3/d19/d1b/f22 [1598253,53781] 0 2024-03-22T10:42:36.881 INFO:tasks.workunit.client.0.smithi115.stdout:9/277: truncate d6/d7/d20/d38/f56 1005515 0 2024-03-22T10:42:36.884 INFO:tasks.workunit.client.0.smithi115.stdout:5/201: fsync d2/f11 0 2024-03-22T10:42:36.885 INFO:tasks.workunit.client.0.smithi115.stdout:2/172: dwrite da/f1b [0,4194304] 0 2024-03-22T10:42:36.885 INFO:tasks.workunit.client.0.smithi115.stdout:7/243: creat d2/d7/d12/d10/d19/d42/d46/f52 x:0 0 0 2024-03-22T10:42:36.888 INFO:tasks.workunit.client.0.smithi115.stdout:3/149: write d3/f25 [874450,53627] 0 2024-03-22T10:42:36.889 INFO:tasks.workunit.client.0.smithi115.stdout:8/378: chown d6/dd/d12/c14 57476330 1 2024-03-22T10:42:36.890 INFO:tasks.workunit.client.0.smithi115.stdout:6/214: link d4/d11/d12/d1c/l40 d4/l45 0 2024-03-22T10:42:36.893 INFO:tasks.workunit.client.0.smithi115.stdout:9/278: read d6/d7/d30/d37/f4d [36190,94167] 0 2024-03-22T10:42:36.893 INFO:tasks.workunit.client.0.smithi115.stdout:4/232: dread d1/dc/f25 [0,4194304] 0 2024-03-22T10:42:36.900 INFO:tasks.workunit.client.0.smithi115.stdout:6/215: read d4/d11/d12/f21 [686221,93875] 0 2024-03-22T10:42:36.904 INFO:tasks.workunit.client.0.smithi115.stdout:1/321: dwrite d0/d12/d17/f42 [0,4194304] 0 2024-03-22T10:42:36.904 INFO:tasks.workunit.client.0.smithi115.stdout:4/233: symlink d1/dc/d14/l4e 0 2024-03-22T10:42:36.912 INFO:tasks.workunit.client.0.smithi115.stdout:3/150: dread d3/f25 [0,4194304] 0 2024-03-22T10:42:36.913 INFO:tasks.workunit.client.0.smithi115.stdout:5/202: dread d2/df/d13/f16 [0,4194304] 0 2024-03-22T10:42:36.913 INFO:tasks.workunit.client.0.smithi115.stdout:4/234: fsync d1/dc/d14/f1b 0 2024-03-22T10:42:36.915 INFO:tasks.workunit.client.0.smithi115.stdout:6/216: creat d4/d11/d12/d1c/f46 x:0 0 0 2024-03-22T10:42:36.918 INFO:tasks.workunit.client.0.smithi115.stdout:4/235: write d1/d30/d42/f45 [1595902,78631] 0 2024-03-22T10:42:36.918 INFO:tasks.workunit.client.0.smithi115.stdout:3/151: symlink d3/d16/l2a 0 2024-03-22T10:42:36.920 INFO:tasks.workunit.client.0.smithi115.stdout:3/152: stat d3/f7 0 2024-03-22T10:42:36.921 INFO:tasks.workunit.client.0.smithi115.stdout:9/279: rename d6/d7/d20/d22/d23/l52 to d6/d7/d30/l5b 0 2024-03-22T10:42:36.925 INFO:tasks.workunit.client.0.smithi115.stdout:9/280: fsync d6/fa 0 2024-03-22T10:42:36.929 INFO:tasks.workunit.client.0.smithi115.stdout:2/173: dread da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:36.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.938+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5070199870 con 0x7f50700fffd0 2024-03-22T10:42:36.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.939+0000 7f5067e00700 1 -- 172.21.15.115:0/1159483323 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1856 (secure 0 0 0) 0x7f5060026070 con 0x7f50700fffd0 2024-03-22T10:42:36.930 INFO:teuthology.orchestra.run.smithi115.stdout:e13 2024-03-22T10:42:36.930 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:42:36.930 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:42:36.930 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:42:36.930 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:42:36.930 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:42:36.930 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 13 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:41:04.070466+0000 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:42:36.931 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:42:36.932 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:42:36.933 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:42:36.933 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:42:36.933 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:42:36.933 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:42:36.933 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:42:36.933 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:42:36.933 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:42:36.933 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:42:36.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.942+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5054039230 msgr2=0x7f505403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:36.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.942+0000 7f5074e00700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5054039230 0x7f505403b6e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f505c009a10 tx=0x7f505c009190 comp rx=0 tx=0).stop 2024-03-22T10:42:36.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.942+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50700fffd0 msgr2=0x7f5070193660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:36.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.942+0000 7f5074e00700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50700fffd0 0x7f5070193660 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f506000bde0 tx=0x7f506000be10 comp rx=0 tx=0).stop 2024-03-22T10:42:36.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.943+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 shutdown_connections 2024-03-22T10:42:36.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.943+0000 7f5074e00700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5054039230 0x7f505403b6e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:36.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.943+0000 7f5074e00700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50700fffd0 0x7f5070193660 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:36.935 INFO:tasks.workunit.client.0.smithi115.stdout:3/153: mkdir d3/d19/d2b 0 2024-03-22T10:42:36.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.943+0000 7f5074e00700 1 --2- 172.21.15.115:0/1159483323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50701011d0 0x7f5070193ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:36.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.943+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 >> 172.21.15.115:0/1159483323 conn(0x7f50700fb5a0 msgr2=0x7f5070104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:36.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.943+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 shutdown_connections 2024-03-22T10:42:36.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:36.943+0000 7f5074e00700 1 -- 172.21.15.115:0/1159483323 wait complete. 2024-03-22T10:42:36.936 INFO:tasks.workunit.client.0.smithi115.stdout:6/217: mknod d4/d11/c47 0 2024-03-22T10:42:36.937 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 13 2024-03-22T10:42:36.941 INFO:tasks.workunit.client.0.smithi115.stdout:5/203: getdents d2/d15 0 2024-03-22T10:42:36.945 INFO:tasks.workunit.client.0.smithi115.stdout:6/218: chown d4/d13/c3e 111 1 2024-03-22T10:42:36.945 INFO:tasks.workunit.client.0.smithi115.stdout:7/244: dwrite d2/d7/d12/f4b [0,4194304] 0 2024-03-22T10:42:36.945 INFO:tasks.workunit.client.0.smithi115.stdout:8/379: dwrite d6/dd/d12/f37 [0,4194304] 0 2024-03-22T10:42:36.950 INFO:tasks.workunit.client.0.smithi115.stdout:6/219: read d4/d11/d12/f21 [1303959,10863] 0 2024-03-22T10:42:36.951 INFO:tasks.workunit.client.0.smithi115.stdout:2/174: creat da/d25/d30/f31 x:0 0 0 2024-03-22T10:42:36.956 INFO:tasks.workunit.client.0.smithi115.stdout:6/220: chown d4/d11/d12/f21 61 1 2024-03-22T10:42:36.959 INFO:tasks.workunit.client.0.smithi115.stdout:7/245: rmdir d2/d7/d12/d10/d19/d42 39 2024-03-22T10:42:36.963 INFO:tasks.workunit.client.0.smithi115.stdout:8/380: rename d6/l55 to d6/d49/l77 0 2024-03-22T10:42:36.973 INFO:tasks.workunit.client.0.smithi115.stdout:9/281: dwrite d6/f12 [0,4194304] 0 2024-03-22T10:42:36.974 INFO:tasks.workunit.client.0.smithi115.stdout:2/175: rename da/d1c/l2f to da/d25/d2a/l32 0 2024-03-22T10:42:36.974 INFO:tasks.workunit.client.0.smithi115.stdout:3/154: dwrite d3/f25 [0,4194304] 0 2024-03-22T10:42:36.976 INFO:tasks.workunit.client.0.smithi115.stdout:6/221: mkdir d4/d36/d48 0 2024-03-22T10:42:36.978 INFO:tasks.workunit.client.0.smithi115.stdout:2/176: write da/d1c/f2b [118784,4141] 0 2024-03-22T10:42:36.981 INFO:tasks.workunit.client.0.smithi115.stdout:5/204: dwrite d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:36.986 INFO:tasks.workunit.client.0.smithi115.stdout:9/282: mknod d6/d7/d20/d38/c5c 0 2024-03-22T10:42:36.986 INFO:tasks.workunit.client.0.smithi115.stdout:3/155: write d3/d16/f21 [2079477,102643] 0 2024-03-22T10:42:36.987 INFO:tasks.workunit.client.0.smithi115.stdout:6/222: truncate d4/d11/d12/d1c/f43 375713 0 2024-03-22T10:42:36.987 INFO:tasks.workunit.client.0.smithi115.stdout:2/177: mknod da/d25/d30/d27/c33 0 2024-03-22T10:42:36.990 INFO:tasks.workunit.client.0.smithi115.stdout:9/283: creat d6/d7/f5d x:0 0 0 2024-03-22T10:42:36.992 INFO:tasks.workunit.client.0.smithi115.stdout:3/156: truncate d3/f23 879551 0 2024-03-22T10:42:36.995 INFO:tasks.workunit.client.0.smithi115.stdout:6/223: creat d4/d36/f49 x:0 0 0 2024-03-22T10:42:36.999 INFO:tasks.workunit.client.0.smithi115.stdout:9/284: mkdir d6/d5e 0 2024-03-22T10:42:37.003 INFO:tasks.workunit.client.0.smithi115.stdout:6/224: unlink d4/d13/f2e 0 2024-03-22T10:42:37.003 INFO:tasks.workunit.client.0.smithi115.stdout:2/178: link da/f14 da/d25/f34 0 2024-03-22T10:42:37.005 INFO:tasks.workunit.client.0.smithi115.stdout:5/205: dwrite d2/df/d13/f16 [0,4194304] 0 2024-03-22T10:42:37.008 INFO:tasks.workunit.client.0.smithi115.stdout:2/179: rename da/d25/d30 to da/d25/d30/d35 22 2024-03-22T10:42:37.008 INFO:tasks.workunit.client.0.smithi115.stdout:9/285: creat d6/d5e/f5f x:0 0 0 2024-03-22T10:42:37.012 INFO:tasks.workunit.client.0.smithi115.stdout:6/225: chown d4/c3c 0 1 2024-03-22T10:42:37.012 INFO:tasks.workunit.client.0.smithi115.stdout:2/180: write da/d1c/f1e [4151759,120091] 0 2024-03-22T10:42:37.014 INFO:tasks.workunit.client.0.smithi115.stdout:2/181: chown l6 111 1 2024-03-22T10:42:37.015 INFO:tasks.workunit.client.0.smithi115.stdout:6/226: readlink d4/d11/d12/d1c/l32 0 2024-03-22T10:42:37.016 INFO:tasks.workunit.client.0.smithi115.stdout:9/286: creat d6/d5e/f60 x:0 0 0 2024-03-22T10:42:37.018 INFO:tasks.workunit.client.0.smithi115.stdout:8/381: dread d6/dd/d12/d31/f5b [0,4194304] 0 2024-03-22T10:42:37.021 INFO:tasks.workunit.client.0.smithi115.stdout:9/287: mknod d6/d8/c61 0 2024-03-22T10:42:37.022 INFO:tasks.workunit.client.0.smithi115.stdout:6/227: symlink d4/d36/l4a 0 2024-03-22T10:42:37.022 INFO:tasks.workunit.client.0.smithi115.stdout:8/382: rmdir d6/dd/d12 39 2024-03-22T10:42:37.025 INFO:tasks.workunit.client.0.smithi115.stdout:7/246: dread d2/d7/d12/d10/f17 [4194304,4194304] 0 2024-03-22T10:42:37.026 INFO:tasks.workunit.client.0.smithi115.stdout:9/288: creat d6/f62 x:0 0 0 2024-03-22T10:42:37.028 INFO:tasks.workunit.client.0.smithi115.stdout:6/228: dread - d4/d13/d33/f42 zero size 2024-03-22T10:42:37.029 INFO:tasks.workunit.client.0.smithi115.stdout:8/383: symlink d6/dd/d12/l78 0 2024-03-22T10:42:37.032 INFO:tasks.workunit.client.0.smithi115.stdout:9/289: symlink d6/d7/d48/l63 0 2024-03-22T10:42:37.032 INFO:tasks.workunit.client.0.smithi115.stdout:7/247: mknod d2/d7/d12/d10/c53 0 2024-03-22T10:42:37.037 INFO:tasks.workunit.client.0.smithi115.stdout:5/206: dwrite d2/f4 [0,4194304] 0 2024-03-22T10:42:37.038 INFO:tasks.workunit.client.0.smithi115.stdout:7/248: read d2/f3f [2195396,81771] 0 2024-03-22T10:42:37.040 INFO:tasks.workunit.client.0.smithi115.stdout:8/384: symlink d6/dd/d24/d4a/d41/l79 0 2024-03-22T10:42:37.044 INFO:tasks.workunit.client.0.smithi115.stdout:5/207: rmdir d2/df/d27 39 2024-03-22T10:42:37.049 INFO:tasks.workunit.client.0.smithi115.stdout:7/249: dread - d2/d7/d12/d10/d19/d42/d46/f52 zero size 2024-03-22T10:42:37.050 INFO:tasks.workunit.client.0.smithi115.stdout:9/290: truncate d6/f2e 3867054 0 2024-03-22T10:42:37.058 INFO:tasks.workunit.client.0.smithi115.stdout:9/291: read d6/d7/d30/d3b/f46 [1475800,43287] 0 2024-03-22T10:42:37.058 INFO:tasks.workunit.client.0.smithi115.stdout:6/229: dwrite d4/f30 [0,4194304] 0 2024-03-22T10:42:37.058 INFO:tasks.workunit.client.0.smithi115.stdout:8/385: rename d6/dd/d12/l1c to d6/dd/l7a 0 2024-03-22T10:42:37.074 INFO:tasks.workunit.client.0.smithi115.stdout:2/182: fdatasync da/d1c/f1e 0 2024-03-22T10:42:37.080 INFO:tasks.workunit.client.0.smithi115.stdout:5/208: dwrite d2/f11 [0,4194304] 0 2024-03-22T10:42:37.086 INFO:tasks.workunit.client.0.smithi115.stdout:2/183: write da/d25/d30/f31 [880231,74198] 0 2024-03-22T10:42:37.086 INFO:tasks.workunit.client.0.smithi115.stdout:5/209: mknod d2/df/c40 0 2024-03-22T10:42:37.091 INFO:tasks.workunit.client.0.smithi115.stdout:5/210: readlink d2/df/d3b/l31 0 2024-03-22T10:42:37.093 INFO:tasks.workunit.client.0.smithi115.stdout:2/184: dread - da/d25/d30/d27/f2c zero size 2024-03-22T10:42:37.094 INFO:tasks.workunit.client.0.smithi115.stdout:9/292: dwrite f3 [0,4194304] 0 2024-03-22T10:42:37.097 INFO:tasks.workunit.client.0.smithi115.stdout:8/386: dwrite d6/dd/d12/d31/f5c [0,4194304] 0 2024-03-22T10:42:37.100 INFO:tasks.workunit.client.0.smithi115.stdout:2/185: write da/d25/d30/d27/f2c [23668,117710] 0 2024-03-22T10:42:37.100 INFO:tasks.workunit.client.0.smithi115.stdout:6/230: dwrite d4/d11/d12/d1c/f46 [0,4194304] 0 2024-03-22T10:42:37.107 INFO:tasks.workunit.client.0.smithi115.stdout:6/231: chown d4/d13/d33/f34 8080545 1 2024-03-22T10:42:37.107 INFO:tasks.workunit.client.0.smithi115.stdout:9/293: unlink d6/d7/d19/f3a 0 2024-03-22T10:42:37.109 INFO:tasks.workunit.client.0.smithi115.stdout:2/186: symlink da/d25/d30/d27/l36 0 2024-03-22T10:42:37.114 INFO:tasks.workunit.client.0.smithi115.stdout:6/232: truncate d4/d13/d33/f42 195778 0 2024-03-22T10:42:37.114 INFO:tasks.workunit.client.0.smithi115.stdout:9/294: mknod d6/d8/d3c/d59/c64 0 2024-03-22T10:42:37.115 INFO:tasks.workunit.client.0.smithi115.stdout:5/211: dread d2/f11 [0,4194304] 0 2024-03-22T10:42:37.115 INFO:tasks.workunit.client.0.smithi115.stdout:9/295: dread - d6/f62 zero size 2024-03-22T10:42:37.117 INFO:tasks.workunit.client.0.smithi115.stdout:6/233: creat d4/d13/f4b x:0 0 0 2024-03-22T10:42:37.118 INFO:tasks.workunit.client.0.smithi115.stdout:0/196: sync 2024-03-22T10:42:37.120 INFO:tasks.workunit.client.0.smithi115.stdout:5/212: dread d2/d15/f39 [0,4194304] 0 2024-03-22T10:42:37.124 INFO:tasks.workunit.client.0.smithi115.stdout:6/234: mknod d4/d11/c4c 0 2024-03-22T10:42:37.126 INFO:tasks.workunit.client.0.smithi115.stdout:9/296: symlink d6/d7/d48/d53/l65 0 2024-03-22T10:42:37.131 INFO:tasks.workunit.client.0.smithi115.stdout:5/213: symlink d2/d15/d32/l41 0 2024-03-22T10:42:37.132 INFO:tasks.workunit.client.0.smithi115.stdout:9/297: chown d6/d7/f16 879 1 2024-03-22T10:42:37.137 INFO:tasks.workunit.client.0.smithi115.stdout:9/298: creat d6/d7/f66 x:0 0 0 2024-03-22T10:42:37.139 INFO:tasks.workunit.client.0.smithi115.stdout:8/387: dwrite d6/dd/d12/d31/f5b [0,4194304] 0 2024-03-22T10:42:37.139 INFO:tasks.workunit.client.0.smithi115.stdout:5/214: truncate d2/d15/d1b/d33/d38/f3a 1218482 0 2024-03-22T10:42:37.143 INFO:tasks.workunit.client.0.smithi115.stdout:6/235: truncate d4/d11/f2a 324455 0 2024-03-22T10:42:37.143 INFO:tasks.workunit.client.0.smithi115.stdout:9/299: creat d6/d7/d20/d38/f67 x:0 0 0 2024-03-22T10:42:37.143 INFO:tasks.workunit.client.0.smithi115.stdout:5/215: chown d2/d15/d1b/c3e 22978 1 2024-03-22T10:42:37.145 INFO:tasks.workunit.client.0.smithi115.stdout:8/388: write d6/dd/d24/d4a/f4d [2083012,72478] 0 2024-03-22T10:42:37.146 INFO:tasks.workunit.client.0.smithi115.stdout:5/216: fsync d2/df/f10 0 2024-03-22T10:42:37.149 INFO:tasks.workunit.client.0.smithi115.stdout:8/389: mkdir d6/dd/d24/d4a/d7b 0 2024-03-22T10:42:37.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:36 smithi150 ceph-mon[34518]: from='client.24467 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:37.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:36 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/351355899' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:42:37.149 INFO:tasks.workunit.client.0.smithi115.stdout:0/197: dwrite d3/da/f26 [0,4194304] 0 2024-03-22T10:42:37.153 INFO:tasks.workunit.client.0.smithi115.stdout:5/217: rmdir d2/d15 39 2024-03-22T10:42:37.160 INFO:tasks.workunit.client.0.smithi115.stdout:5/218: symlink d2/df/d27/l42 0 2024-03-22T10:42:37.167 INFO:tasks.workunit.client.0.smithi115.stdout:6/236: truncate d4/d11/d12/d1c/f46 3251535 0 2024-03-22T10:42:37.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.183+0000 7f3d00400700 1 -- 172.21.15.115:0/1638320270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf8103570 msgr2=0x7f3cf8105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:37.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.183+0000 7f3d00400700 1 --2- 172.21.15.115:0/1638320270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf8103570 0x7f3cf8105950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009910 tx=0x7f3ce4009c20 comp rx=0 tx=0).stop 2024-03-22T10:42:37.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.183+0000 7f3d00400700 1 -- 172.21.15.115:0/1638320270 shutdown_connections 2024-03-22T10:42:37.175 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:36 smithi115 ceph-mon[29111]: from='client.24467 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:37.175 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:36 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/351355899' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:42:37.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.183+0000 7f3d00400700 1 --2- 172.21.15.115:0/1638320270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf8103570 0x7f3cf8105950 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.183+0000 7f3d00400700 1 --2- 172.21.15.115:0/1638320270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf8100c50 0x7f3cf8103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.183+0000 7f3d00400700 1 -- 172.21.15.115:0/1638320270 >> 172.21.15.115:0/1638320270 conn(0x7f3cf80fa5f0 msgr2=0x7f3cf80fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:37.176 INFO:tasks.workunit.client.0.smithi115.stdout:5/219: mkdir d2/d15/d1b/d33/d38/d43 0 2024-03-22T10:42:37.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.185+0000 7f3d00400700 1 -- 172.21.15.115:0/1638320270 shutdown_connections 2024-03-22T10:42:37.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.185+0000 7f3d00400700 1 -- 172.21.15.115:0/1638320270 wait complete. 2024-03-22T10:42:37.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.186+0000 7f3d00400700 1 Processor -- start 2024-03-22T10:42:37.178 INFO:tasks.workunit.client.0.smithi115.stdout:6/237: getdents d4/d36/d48 0 2024-03-22T10:42:37.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.186+0000 7f3d00400700 1 -- start start 2024-03-22T10:42:37.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.187+0000 7f3d00400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf8100c50 0x7f3cf810eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:37.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.187+0000 7f3d00400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf8103570 0x7f3cf810f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:37.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.187+0000 7f3d00400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf810f6c0 con 0x7f3cf8103570 2024-03-22T10:42:37.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.187+0000 7f3d00400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf810f800 con 0x7f3cf8100c50 2024-03-22T10:42:37.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.187+0000 7f3cfda00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf8100c50 0x7f3cf810eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:37.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.187+0000 7f3cfda00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf8100c50 0x7f3cf810eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:54626/0 (socket says 172.21.15.115:54626) 2024-03-22T10:42:37.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.187+0000 7f3cfda00700 1 -- 172.21.15.115:0/4066323250 learned_addr learned my addr 172.21.15.115:0/4066323250 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:37.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.187+0000 7f3cfda00700 1 -- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf8103570 msgr2=0x7f3cf810f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:37.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.188+0000 7f3cfd000700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf8103570 0x7f3cf810f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:37.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.188+0000 7f3cfda00700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf8103570 0x7f3cf810f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.188+0000 7f3cfda00700 1 -- 172.21.15.115:0/4066323250 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce40095f0 con 0x7f3cf8100c50 2024-03-22T10:42:37.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.188+0000 7f3cfd000700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf8103570 0x7f3cf810f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:42:37.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.188+0000 7f3cfda00700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf8100c50 0x7f3cf810eb60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00e920 tx=0x7f3cec00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:37.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.189+0000 7f3cf6000700 1 -- 172.21.15.115:0/4066323250 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cec00cba0 con 0x7f3cf8100c50 2024-03-22T10:42:37.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.189+0000 7f3cf6000700 1 -- 172.21.15.115:0/4066323250 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3cec00cd00 con 0x7f3cf8100c50 2024-03-22T10:42:37.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.189+0000 7f3cf6000700 1 -- 172.21.15.115:0/4066323250 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cec018770 con 0x7f3cf8100c50 2024-03-22T10:42:37.181 INFO:tasks.workunit.client.0.smithi115.stdout:5/220: symlink d2/df/d27/l44 0 2024-03-22T10:42:37.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.190+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf81122a0 con 0x7f3cf8100c50 2024-03-22T10:42:37.182 INFO:tasks.workunit.client.0.smithi115.stdout:8/390: dread d6/dd/d24/d4a/f4d [0,4194304] 0 2024-03-22T10:42:37.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.190+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf8112700 con 0x7f3cf8100c50 2024-03-22T10:42:37.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.191+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf8113af0 con 0x7f3cf8100c50 2024-03-22T10:42:37.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.191+0000 7f3cf6000700 1 -- 172.21.15.115:0/4066323250 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 29) v1 ==== 46247+0+0 (secure 0 0 0) 0x7f3cec010430 con 0x7f3cf8100c50 2024-03-22T10:42:37.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.192+0000 7f3cf6000700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f3ce0039230 0x7f3ce003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:37.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.192+0000 7f3cf6000700 1 -- 172.21.15.115:0/4066323250 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f3cec014070 con 0x7f3cf8100c50 2024-03-22T10:42:37.184 INFO:tasks.workunit.client.0.smithi115.stdout:0/198: dwrite d3/d4/d14/f23 [0,4194304] 0 2024-03-22T10:42:37.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.193+0000 7f3cfd000700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f3ce0039230 0x7f3ce003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:37.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.194+0000 7f3cfd000700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f3ce0039230 0x7f3ce003b6e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4000c00 tx=0x7f3ce4009f70 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:37.186 INFO:tasks.workunit.client.0.smithi115.stdout:0/199: stat d3 0 2024-03-22T10:42:37.187 INFO:tasks.workunit.client.0.smithi115.stdout:5/221: mknod d2/d15/d1b/d33/c45 0 2024-03-22T10:42:37.190 INFO:tasks.workunit.client.0.smithi115.stdout:5/222: readlink d2/d15/l35 0 2024-03-22T10:42:37.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.201+0000 7f3cf6000700 1 -- 172.21.15.115:0/4066323250 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f3cec01f900 con 0x7f3cf8100c50 2024-03-22T10:42:37.196 INFO:tasks.workunit.client.0.smithi115.stdout:0/200: symlink d3/l3a 0 2024-03-22T10:42:37.196 INFO:tasks.workunit.client.0.smithi115.stdout:8/391: getdents d6/dd/d24/d3a 0 2024-03-22T10:42:37.198 INFO:tasks.workunit.client.0.smithi115.stdout:6/238: dwrite d4/d13/f25 [0,4194304] 0 2024-03-22T10:42:37.204 INFO:tasks.workunit.client.0.smithi115.stdout:6/239: symlink d4/d13/d33/l4d 0 2024-03-22T10:42:37.210 INFO:tasks.workunit.client.0.smithi115.stdout:6/240: truncate d4/d11/f44 572377 0 2024-03-22T10:42:37.212 INFO:tasks.workunit.client.0.smithi115.stdout:0/201: truncate d3/d7/d1b/f2a 108623 0 2024-03-22T10:42:37.216 INFO:tasks.workunit.client.0.smithi115.stdout:8/392: dwrite f5 [4194304,4194304] 0 2024-03-22T10:42:37.218 INFO:tasks.workunit.client.0.smithi115.stdout:6/241: dread d4/d13/d33/f42 [0,4194304] 0 2024-03-22T10:42:37.220 INFO:tasks.workunit.client.0.smithi115.stdout:0/202: creat d3/d7/f3b x:0 0 0 2024-03-22T10:42:37.224 INFO:tasks.workunit.client.0.smithi115.stdout:0/203: write d3/d7/d1c/f22 [2313741,117988] 0 2024-03-22T10:42:37.227 INFO:tasks.workunit.client.0.smithi115.stdout:0/204: fsync d3/f33 0 2024-03-22T10:42:37.231 INFO:tasks.workunit.client.0.smithi115.stdout:0/205: symlink d3/da/l3c 0 2024-03-22T10:42:37.235 INFO:tasks.workunit.client.0.smithi115.stdout:0/206: chown d3/d7/d1b/f24 0 1 2024-03-22T10:42:37.237 INFO:tasks.workunit.client.0.smithi115.stdout:8/393: dwrite d6/dd/d24/d4a/f66 [0,4194304] 0 2024-03-22T10:42:37.251 INFO:tasks.workunit.client.0.smithi115.stdout:0/207: link d3/f33 d3/d7/f3d 0 2024-03-22T10:42:37.252 INFO:tasks.workunit.client.0.smithi115.stdout:8/394: rename d6/dd/d24/c5a to d6/dd/d24/d4a/d41/c7c 0 2024-03-22T10:42:37.253 INFO:tasks.workunit.client.0.smithi115.stdout:0/208: readlink d3/d7/d1b/l29 0 2024-03-22T10:42:37.258 INFO:tasks.workunit.client.0.smithi115.stdout:8/395: unlink d6/dd/d12/c2c 0 2024-03-22T10:42:37.263 INFO:tasks.workunit.client.0.smithi115.stdout:0/209: rename d3/d4/c37 to d3/d7/d1b/c3e 0 2024-03-22T10:42:37.265 INFO:tasks.workunit.client.0.smithi115.stdout:0/210: creat d3/d7/d1c/f3f x:0 0 0 2024-03-22T10:42:37.334 INFO:tasks.workunit.client.0.smithi115.stdout:8/396: rename d6/dd/d12/l5e to d6/dd/d12/d31/l7d 0 2024-03-22T10:42:37.337 INFO:tasks.workunit.client.0.smithi115.stdout:8/397: chown d6/dd/d12/d31/l64 299 1 2024-03-22T10:42:37.339 INFO:tasks.workunit.client.0.smithi115.stdout:0/211: mknod d3/d4/c40 0 2024-03-22T10:42:37.340 INFO:tasks.workunit.client.0.smithi115.stdout:8/398: readlink d6/dd/d12/l2b 0 2024-03-22T10:42:37.341 INFO:tasks.workunit.client.0.smithi115.stdout:8/399: rmdir d6/dd/d24/d3a 39 2024-03-22T10:42:37.344 INFO:tasks.workunit.client.0.smithi115.stdout:0/212: link d3/d4/d14/f23 d3/d7/d1b/d1f/f41 0 2024-03-22T10:42:37.349 INFO:tasks.workunit.client.0.smithi115.stdout:0/213: rename d3/d7/f3d to d3/d7/d1b/d35/f42 0 2024-03-22T10:42:37.360 INFO:tasks.workunit.client.0.smithi115.stdout:8/400: dwrite d6/dd/d24/d4a/f5f [0,4194304] 0 2024-03-22T10:42:37.362 INFO:tasks.workunit.client.0.smithi115.stdout:8/401: fsync d6/dd/f70 0 2024-03-22T10:42:37.365 INFO:tasks.workunit.client.0.smithi115.stdout:8/402: write d6/dd/d24/d4a/f66 [1786382,13103] 0 2024-03-22T10:42:37.374 INFO:tasks.workunit.client.0.smithi115.stdout:8/403: mknod d6/dd/d24/d4a/d7b/c7e 0 2024-03-22T10:42:37.379 INFO:tasks.workunit.client.0.smithi115.stdout:0/214: dread d3/d7/d1c/f1e [0,4194304] 0 2024-03-22T10:42:37.380 INFO:tasks.workunit.client.0.smithi115.stdout:8/404: rmdir d6/dd 39 2024-03-22T10:42:37.382 INFO:tasks.workunit.client.0.smithi115.stdout:0/215: chown d3/d7/d1b/l29 25 1 2024-03-22T10:42:37.383 INFO:tasks.workunit.client.0.smithi115.stdout:8/405: mknod d6/d49/d68/c7f 0 2024-03-22T10:42:37.384 INFO:tasks.workunit.client.0.smithi115.stdout:0/216: fsync d3/d7/d1c/f22 0 2024-03-22T10:42:37.391 INFO:tasks.workunit.client.0.smithi115.stdout:8/406: creat d6/dd/d24/d4a/f80 x:0 0 0 2024-03-22T10:42:37.398 INFO:tasks.workunit.client.0.smithi115.stdout:0/217: dread d3/d7/d1c/f22 [0,4194304] 0 2024-03-22T10:42:37.401 INFO:tasks.workunit.client.0.smithi115.stdout:0/218: symlink d3/d7/d1b/d1f/l43 0 2024-03-22T10:42:37.403 INFO:tasks.workunit.client.0.smithi115.stdout:0/219: mknod d3/da/c44 0 2024-03-22T10:42:37.410 INFO:tasks.workunit.client.0.smithi115.stdout:8/407: dwrite d6/dd/d24/d4a/f5f [4194304,4194304] 0 2024-03-22T10:42:37.415 INFO:tasks.workunit.client.0.smithi115.stdout:8/408: creat d6/dd/d24/d4a/f81 x:0 0 0 2024-03-22T10:42:37.419 INFO:tasks.workunit.client.0.smithi115.stdout:0/220: dread d3/d7/d1b/f24 [0,4194304] 0 2024-03-22T10:42:37.421 INFO:tasks.workunit.client.0.smithi115.stdout:0/221: read d3/f1a [966251,29701] 0 2024-03-22T10:42:37.425 INFO:tasks.workunit.client.0.smithi115.stdout:0/222: creat d3/d7/f45 x:0 0 0 2024-03-22T10:42:37.428 INFO:tasks.workunit.client.0.smithi115.stdout:0/223: creat d3/f46 x:0 0 0 2024-03-22T10:42:37.431 INFO:tasks.workunit.client.0.smithi115.stdout:0/224: rmdir d3/d7/d1c 39 2024-03-22T10:42:37.432 INFO:tasks.workunit.client.0.smithi115.stdout:0/225: stat d3/d4/d14 0 2024-03-22T10:42:37.434 INFO:tasks.workunit.client.0.smithi115.stdout:8/409: dwrite d6/dd/d24/d3a/f54 [0,4194304] 0 2024-03-22T10:42:37.467 INFO:tasks.workunit.client.0.smithi115.stdout:1/322: sync 2024-03-22T10:42:37.467 INFO:tasks.workunit.client.0.smithi115.stdout:4/236: sync 2024-03-22T10:42:37.468 INFO:tasks.workunit.client.0.smithi115.stdout:0/226: creat d3/d7/d1b/f47 x:0 0 0 2024-03-22T10:42:37.469 INFO:tasks.workunit.client.0.smithi115.stdout:8/410: read d6/dd/d12/d31/f5c [2241859,43163] 0 2024-03-22T10:42:37.471 INFO:tasks.workunit.client.0.smithi115.stdout:4/237: mkdir d1/d30/d42/d4f 0 2024-03-22T10:42:37.473 INFO:tasks.workunit.client.0.smithi115.stdout:1/323: creat d0/d8/d4d/d56/f6d x:0 0 0 2024-03-22T10:42:37.474 INFO:tasks.workunit.client.0.smithi115.stdout:4/238: truncate d1/d30/d42/f45 1983305 0 2024-03-22T10:42:37.475 INFO:tasks.workunit.client.0.smithi115.stdout:0/227: creat d3/d7/d1c/f48 x:0 0 0 2024-03-22T10:42:37.475 INFO:tasks.workunit.client.0.smithi115.stdout:8/411: symlink d6/dd/l82 0 2024-03-22T10:42:37.476 INFO:tasks.workunit.client.0.smithi115.stdout:8/412: stat d6/dd 0 2024-03-22T10:42:37.480 INFO:tasks.workunit.client.0.smithi115.stdout:0/228: fsync d3/d7/d1b/f2a 0 2024-03-22T10:42:37.480 INFO:tasks.workunit.client.0.smithi115.stdout:1/324: getdents d0/d12/d17/d37/d5c 0 2024-03-22T10:42:37.482 INFO:tasks.workunit.client.0.smithi115.stdout:1/325: fdatasync d0/d8/db/f18 0 2024-03-22T10:42:37.485 INFO:tasks.workunit.client.0.smithi115.stdout:1/326: write d0/d22/f3d [5242552,75089] 0 2024-03-22T10:42:37.489 INFO:tasks.workunit.client.0.smithi115.stdout:1/327: stat d0/d8/d4d/d56/f57 0 2024-03-22T10:42:37.493 INFO:tasks.workunit.client.0.smithi115.stdout:1/328: write d0/d8/de/d1a/f61 [349121,32604] 0 2024-03-22T10:42:37.497 INFO:tasks.workunit.client.0.smithi115.stdout:1/329: creat d0/d8/d4d/f6e x:0 0 0 2024-03-22T10:42:37.497 INFO:tasks.workunit.client.0.smithi115.stdout:8/413: dwrite d6/dd/d24/f3f [0,4194304] 0 2024-03-22T10:42:37.501 INFO:tasks.workunit.client.0.smithi115.stdout:1/330: readlink d0/d8/de/d1a/l50 0 2024-03-22T10:42:37.504 INFO:tasks.workunit.client.0.smithi115.stdout:0/229: dwrite d3/d4/d14/f15 [0,4194304] 0 2024-03-22T10:42:37.505 INFO:tasks.workunit.client.0.smithi115.stdout:1/331: creat d0/d12/d17/d37/f6f x:0 0 0 2024-03-22T10:42:37.509 INFO:tasks.workunit.client.0.smithi115.stdout:1/332: dread - d0/d8/d4d/d56/f6d zero size 2024-03-22T10:42:37.510 INFO:tasks.workunit.client.0.smithi115.stdout:1/333: write d0/d12/d2b/f59 [119143,128706] 0 2024-03-22T10:42:37.519 INFO:tasks.workunit.client.0.smithi115.stdout:8/414: dwrite d6/dd/d12/f27 [0,4194304] 0 2024-03-22T10:42:37.522 INFO:tasks.workunit.client.0.smithi115.stdout:8/415: readlink d6/dd/d24/d4a/d41/l79 0 2024-03-22T10:42:37.526 INFO:tasks.workunit.client.0.smithi115.stdout:1/334: rename d0/d12/d17/d37/f3c to d0/f70 0 2024-03-22T10:42:37.529 INFO:tasks.workunit.client.0.smithi115.stdout:0/230: dwrite d3/d4/d14/f15 [0,4194304] 0 2024-03-22T10:42:37.529 INFO:tasks.workunit.client.0.smithi115.stdout:1/335: write d0/d22/f3d [6310813,63708] 0 2024-03-22T10:42:37.532 INFO:tasks.workunit.client.0.smithi115.stdout:0/231: dread - d3/d7/d1b/f47 zero size 2024-03-22T10:42:37.542 INFO:tasks.workunit.client.0.smithi115.stdout:0/232: rmdir d3/d7/d1b/d35 39 2024-03-22T10:42:37.542 INFO:tasks.workunit.client.0.smithi115.stdout:8/416: write d6/f20 [382689,845] 0 2024-03-22T10:42:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.554+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cf8061190 con 0x7f3ce0039230 2024-03-22T10:42:37.549 INFO:tasks.workunit.client.0.smithi115.stdout:0/233: readlink d3/d7/d1c/l30 0 2024-03-22T10:42:37.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.561+0000 7f3cf6000700 1 -- 172.21.15.115:0/4066323250 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f3cf8061190 con 0x7f3ce0039230 2024-03-22T10:42:37.552 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr" 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "2/23 daemons upgraded", 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "", 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:42:37.553 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:42:37.555 INFO:tasks.workunit.client.0.smithi115.stdout:0/234: symlink d3/da/l49 0 2024-03-22T10:42:37.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.565+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f3ce0039230 msgr2=0x7f3ce003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:37.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.565+0000 7f3d00400700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f3ce0039230 0x7f3ce003b6e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4000c00 tx=0x7f3ce4009f70 comp rx=0 tx=0).stop 2024-03-22T10:42:37.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.565+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf8100c50 msgr2=0x7f3cf810eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:37.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.565+0000 7f3d00400700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf8100c50 0x7f3cf810eb60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00e920 tx=0x7f3cec00ece0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.566+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 shutdown_connections 2024-03-22T10:42:37.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.566+0000 7f3d00400700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f3ce0039230 0x7f3ce003b6e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.566+0000 7f3d00400700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf8100c50 0x7f3cf810eb60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.566+0000 7f3d00400700 1 --2- 172.21.15.115:0/4066323250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf8103570 0x7f3cf810f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.566+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 >> 172.21.15.115:0/4066323250 conn(0x7f3cf80fa5f0 msgr2=0x7f3cf80fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:37.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.567+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 shutdown_connections 2024-03-22T10:42:37.559 INFO:tasks.workunit.client.0.smithi115.stdout:1/336: dwrite d0/f14 [4194304,4194304] 0 2024-03-22T10:42:37.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.567+0000 7f3d00400700 1 -- 172.21.15.115:0/4066323250 wait complete. 2024-03-22T10:42:37.567 INFO:tasks.workunit.client.0.smithi115.stdout:0/235: unlink d3/d7/d1b/c3e 0 2024-03-22T10:42:37.572 INFO:tasks.workunit.client.0.smithi115.stdout:1/337: fsync d0/f55 0 2024-03-22T10:42:37.578 INFO:tasks.workunit.client.0.smithi115.stdout:3/157: sync 2024-03-22T10:42:37.581 INFO:tasks.workunit.client.0.smithi115.stdout:3/158: unlink d3/d16/l2a 0 2024-03-22T10:42:37.583 INFO:tasks.workunit.client.0.smithi115.stdout:3/159: fdatasync d3/f25 0 2024-03-22T10:42:37.587 INFO:tasks.workunit.client.0.smithi115.stdout:1/338: dread d0/d12/d2b/f59 [0,4194304] 0 2024-03-22T10:42:37.587 INFO:tasks.workunit.client.0.smithi115.stdout:3/160: creat d3/d19/d1b/f2c x:0 0 0 2024-03-22T10:42:37.588 INFO:tasks.workunit.client.0.smithi115.stdout:0/236: dwrite d3/d7/d1b/d1f/f41 [4194304,4194304] 0 2024-03-22T10:42:37.591 INFO:tasks.workunit.client.0.smithi115.stdout:3/161: unlink d3/c28 0 2024-03-22T10:42:37.596 INFO:tasks.workunit.client.0.smithi115.stdout:0/237: unlink d3/d7/d1c/f3f 0 2024-03-22T10:42:37.596 INFO:tasks.workunit.client.0.smithi115.stdout:8/417: dread d6/dd/d12/f1d [0,4194304] 0 2024-03-22T10:42:37.596 INFO:tasks.workunit.client.0.smithi115.stdout:3/162: mknod d3/c2d 0 2024-03-22T10:42:37.596 INFO:tasks.workunit.client.0.smithi115.stdout:8/418: readlink d6/l9 0 2024-03-22T10:42:37.598 INFO:tasks.workunit.client.0.smithi115.stdout:0/238: write d3/d7/d1b/f24 [4502494,86493] 0 2024-03-22T10:42:37.602 INFO:tasks.workunit.client.0.smithi115.stdout:8/419: creat d6/d49/f83 x:0 0 0 2024-03-22T10:42:37.602 INFO:tasks.workunit.client.0.smithi115.stdout:3/163: symlink d3/d19/l2e 0 2024-03-22T10:42:37.607 INFO:tasks.workunit.client.0.smithi115.stdout:8/420: creat d6/d49/f84 x:0 0 0 2024-03-22T10:42:37.620 INFO:tasks.workunit.client.0.smithi115.stdout:8/421: getdents d6/dd/d24/d4a/d41 0 2024-03-22T10:42:37.624 INFO:tasks.workunit.client.0.smithi115.stdout:3/164: dread d3/f7 [0,4194304] 0 2024-03-22T10:42:37.625 INFO:tasks.workunit.client.0.smithi115.stdout:0/239: dwrite d3/d7/d1c/f48 [0,4194304] 0 2024-03-22T10:42:37.633 INFO:tasks.workunit.client.0.smithi115.stdout:3/165: mkdir d3/d16/d2f 0 2024-03-22T10:42:37.636 INFO:tasks.workunit.client.0.smithi115.stdout:3/166: creat d3/f30 x:0 0 0 2024-03-22T10:42:37.638 INFO:tasks.workunit.client.0.smithi115.stdout:8/422: dread d6/dd/d12/f1d [0,4194304] 0 2024-03-22T10:42:37.639 INFO:tasks.workunit.client.0.smithi115.stdout:3/167: getdents d3/d19/d2b 0 2024-03-22T10:42:37.642 INFO:tasks.workunit.client.0.smithi115.stdout:0/240: dread d3/f1a [4194304,4194304] 0 2024-03-22T10:42:37.642 INFO:tasks.workunit.client.0.smithi115.stdout:8/423: symlink d6/dd/d12/d31/l85 0 2024-03-22T10:42:37.644 INFO:tasks.workunit.client.0.smithi115.stdout:3/168: write d3/f12 [2009923,128370] 0 2024-03-22T10:42:37.645 INFO:tasks.workunit.client.0.smithi115.stdout:0/241: stat d3/d7 0 2024-03-22T10:42:37.649 INFO:tasks.workunit.client.0.smithi115.stdout:8/424: symlink d6/dd/d24/d4a/l86 0 2024-03-22T10:42:37.652 INFO:tasks.workunit.client.0.smithi115.stdout:3/169: fsync d3/f7 0 2024-03-22T10:42:37.653 INFO:tasks.workunit.client.0.smithi115.stdout:0/242: write d3/f1a [4043493,48215] 0 2024-03-22T10:42:37.656 INFO:tasks.workunit.client.0.smithi115.stdout:0/243: fsync d3/d7/f45 0 2024-03-22T10:42:37.656 INFO:tasks.workunit.client.0.smithi115.stdout:3/170: creat d3/d19/f31 x:0 0 0 2024-03-22T10:42:37.663 INFO:tasks.workunit.client.0.smithi115.stdout:0/244: rmdir d3 39 2024-03-22T10:42:37.669 INFO:tasks.workunit.client.0.smithi115.stdout:0/245: stat d3/d7/d1b/d1f 0 2024-03-22T10:42:37.670 INFO:tasks.workunit.client.0.smithi115.stdout:0/246: chown d3/d4/cf 5035 1 2024-03-22T10:42:37.673 INFO:tasks.workunit.client.0.smithi115.stdout:0/247: readlink d3/l3a 0 2024-03-22T10:42:37.676 INFO:tasks.workunit.client.0.smithi115.stdout:8/425: dwrite d6/dd/d24/d4a/f4d [0,4194304] 0 2024-03-22T10:42:37.677 INFO:tasks.workunit.client.0.smithi115.stdout:3/171: dwrite d3/f9 [0,4194304] 0 2024-03-22T10:42:37.684 INFO:tasks.workunit.client.0.smithi115.stdout:3/172: creat d3/d16/d29/f32 x:0 0 0 2024-03-22T10:42:37.695 INFO:tasks.workunit.client.0.smithi115.stdout:0/248: dread d3/d7/d1b/d1f/f41 [4194304,4194304] 0 2024-03-22T10:42:37.702 INFO:tasks.workunit.client.0.smithi115.stdout:3/173: dwrite d3/d19/f1a [0,4194304] 0 2024-03-22T10:42:37.704 INFO:tasks.workunit.client.0.smithi115.stdout:0/249: link d3/d4/c2c d3/d4/d14/c4a 0 2024-03-22T10:42:37.704 INFO:tasks.workunit.client.0.smithi115.stdout:3/174: fdatasync d3/f23 0 2024-03-22T10:42:37.706 INFO:tasks.workunit.client.0.smithi115.stdout:0/250: chown d3/d7/d1b/f2a 866 1 2024-03-22T10:42:37.712 INFO:tasks.workunit.client.0.smithi115.stdout:0/251: mknod d3/d7/d1b/d1f/c4b 0 2024-03-22T10:42:37.716 INFO:tasks.workunit.client.0.smithi115.stdout:0/252: write d3/d4/d14/f15 [5039755,29424] 0 2024-03-22T10:42:37.719 INFO:tasks.workunit.client.0.smithi115.stdout:3/175: truncate d3/d16/f21 2562980 0 2024-03-22T10:42:37.721 INFO:tasks.workunit.client.0.smithi115.stdout:3/176: fsync d3/d19/d1b/f27 0 2024-03-22T10:42:37.722 INFO:tasks.workunit.client.0.smithi115.stdout:3/177: stat d3/l10 0 2024-03-22T10:42:37.725 INFO:tasks.workunit.client.0.smithi115.stdout:3/178: mkdir d3/d16/d2f/d33 0 2024-03-22T10:42:37.726 INFO:tasks.workunit.client.0.smithi115.stdout:3/179: stat d3/d19/d1b 0 2024-03-22T10:42:37.734 INFO:tasks.workunit.client.0.smithi115.stdout:7/250: sync 2024-03-22T10:42:37.734 INFO:tasks.workunit.client.0.smithi115.stdout:9/300: sync 2024-03-22T10:42:37.734 INFO:tasks.workunit.client.0.smithi115.stdout:2/187: sync 2024-03-22T10:42:37.740 INFO:tasks.workunit.client.0.smithi115.stdout:9/301: mkdir d6/d8/d3c/d59/d68 0 2024-03-22T10:42:37.746 INFO:tasks.workunit.client.0.smithi115.stdout:9/302: rmdir d6/d8/d3c/d59 39 2024-03-22T10:42:37.747 INFO:tasks.workunit.client.0.smithi115.stdout:3/180: dwrite d3/f30 [0,4194304] 0 2024-03-22T10:42:37.750 INFO:tasks.workunit.client.0.smithi115.stdout:9/303: chown d6/d5e/f60 1559344432 1 2024-03-22T10:42:37.761 INFO:tasks.workunit.client.0.smithi115.stdout:9/304: symlink d6/d7/d19/d4e/l69 0 2024-03-22T10:42:37.765 INFO:tasks.workunit.client.0.smithi115.stdout:3/181: dread d3/f8 [8388608,4194304] 0 2024-03-22T10:42:37.767 INFO:tasks.workunit.client.0.smithi115.stdout:9/305: mkdir d6/d7/d6a 0 2024-03-22T10:42:37.770 INFO:tasks.workunit.client.0.smithi115.stdout:3/182: dread - d3/d19/f31 zero size 2024-03-22T10:42:37.778 INFO:tasks.workunit.client.0.smithi115.stdout:3/183: symlink d3/d19/d1b/l34 0 2024-03-22T10:42:37.778 INFO:tasks.workunit.client.0.smithi115.stdout:7/251: dwrite d2/d3a/d45/f4f [0,4194304] 0 2024-03-22T10:42:37.778 INFO:tasks.workunit.client.0.smithi115.stdout:2/188: dread da/f14 [0,4194304] 0 2024-03-22T10:42:37.781 INFO:tasks.workunit.client.0.smithi115.stdout:9/306: link d6/d7/d48/l63 d6/d8/l6b 0 2024-03-22T10:42:37.782 INFO:tasks.workunit.client.0.smithi115.stdout:3/184: mkdir d3/d16/d29/d35 0 2024-03-22T10:42:37.784 INFO:tasks.workunit.client.0.smithi115.stdout:2/189: dread da/d25/d30/d27/f2c [0,4194304] 0 2024-03-22T10:42:37.785 INFO:tasks.workunit.client.0.smithi115.stdout:7/252: write d2/d7/d12/d10/d23/f2e [1705475,54731] 0 2024-03-22T10:42:37.785 INFO:tasks.workunit.client.0.smithi115.stdout:9/307: fsync d6/d7/f17 0 2024-03-22T10:42:37.788 INFO:tasks.workunit.client.0.smithi115.stdout:3/185: mkdir d3/d36 0 2024-03-22T10:42:37.791 INFO:tasks.workunit.client.0.smithi115.stdout:2/190: dread - da/d25/d30/d27/f28 zero size 2024-03-22T10:42:37.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.804+0000 7f584f800700 1 -- 172.21.15.115:0/2474840968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58480fe9d0 msgr2=0x7f5848100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:37.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.804+0000 7f584f800700 1 --2- 172.21.15.115:0/2474840968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58480fe9d0 0x7f5848100df0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5834009870 tx=0x7f5834009b80 comp rx=0 tx=0).stop 2024-03-22T10:42:37.796 INFO:tasks.workunit.client.0.smithi115.stdout:9/308: creat d6/d7/d19/d4e/f6c x:0 0 0 2024-03-22T10:42:37.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.804+0000 7f584f800700 1 -- 172.21.15.115:0/2474840968 shutdown_connections 2024-03-22T10:42:37.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.804+0000 7f584f800700 1 --2- 172.21.15.115:0/2474840968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5848101330 0x7f58481037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.804+0000 7f584f800700 1 --2- 172.21.15.115:0/2474840968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58480fe9d0 0x7f5848100df0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.804+0000 7f584f800700 1 -- 172.21.15.115:0/2474840968 >> 172.21.15.115:0/2474840968 conn(0x7f58480fa5e0 msgr2=0x7f58480fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:37.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.805+0000 7f584f800700 1 -- 172.21.15.115:0/2474840968 shutdown_connections 2024-03-22T10:42:37.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.805+0000 7f584f800700 1 -- 172.21.15.115:0/2474840968 wait complete. 2024-03-22T10:42:37.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.807+0000 7f584f800700 1 Processor -- start 2024-03-22T10:42:37.800 INFO:tasks.workunit.client.0.smithi115.stdout:9/309: truncate d6/d5e/f5f 586529 0 2024-03-22T10:42:37.801 INFO:tasks.workunit.client.0.smithi115.stdout:7/253: unlink d2/d7/d12/d10/c14 0 2024-03-22T10:42:37.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.809+0000 7f584f800700 1 -- start start 2024-03-22T10:42:37.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.810+0000 7f584f800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58480fe9d0 0x7f584819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:37.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.810+0000 7f584f800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5848101330 0x7f584819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:37.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.810+0000 7f584f800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584819cab0 con 0x7f58480fe9d0 2024-03-22T10:42:37.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.810+0000 7f584f800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584819cbf0 con 0x7f5848101330 2024-03-22T10:42:37.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.811+0000 7f584ce00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58480fe9d0 0x7f584819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:37.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.811+0000 7f5847e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5848101330 0x7f584819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:37.803 INFO:tasks.workunit.client.0.smithi115.stdout:9/310: chown d6/d7/d20/f2f 15 1 2024-03-22T10:42:37.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.811+0000 7f5847e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5848101330 0x7f584819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:54642/0 (socket says 172.21.15.115:54642) 2024-03-22T10:42:37.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.811+0000 7f5847e00700 1 -- 172.21.15.115:0/1774783633 learned_addr learned my addr 172.21.15.115:0/1774783633 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:42:37.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.812+0000 7f584ce00700 1 -- 172.21.15.115:0/1774783633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5848101330 msgr2=0x7f584819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:37.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.812+0000 7f584ce00700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5848101330 0x7f584819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:37.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.812+0000 7f584ce00700 1 -- 172.21.15.115:0/1774783633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5834009520 con 0x7f58480fe9d0 2024-03-22T10:42:37.804 INFO:tasks.workunit.client.0.smithi115.stdout:3/186: mknod d3/d16/d2f/c37 0 2024-03-22T10:42:37.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.814+0000 7f584ce00700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58480fe9d0 0x7f584819bf50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5834009230 tx=0x7f583400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:37.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.815+0000 7f5845600700 1 -- 172.21.15.115:0/1774783633 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f583401c070 con 0x7f58480fe9d0 2024-03-22T10:42:37.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.815+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58481a1640 con 0x7f58480fe9d0 2024-03-22T10:42:37.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.815+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58481a1bb0 con 0x7f58480fe9d0 2024-03-22T10:42:37.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.817+0000 7f5845600700 1 -- 172.21.15.115:0/1774783633 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f583400b4e0 con 0x7f58480fe9d0 2024-03-22T10:42:37.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.817+0000 7f5845600700 1 -- 172.21.15.115:0/1774783633 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5834017c60 con 0x7f58480fe9d0 2024-03-22T10:42:37.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.818+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5848196170 con 0x7f58480fe9d0 2024-03-22T10:42:37.810 INFO:tasks.workunit.client.0.smithi115.stdout:2/191: creat da/d25/f37 x:0 0 0 2024-03-22T10:42:37.811 INFO:tasks.workunit.client.0.smithi115.stdout:3/187: mknod d3/d16/d2f/c38 0 2024-03-22T10:42:37.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.820+0000 7f5845600700 1 -- 172.21.15.115:0/1774783633 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 29) v1 ==== 46247+0+0 (secure 0 0 0) 0x7f5834020070 con 0x7f58480fe9d0 2024-03-22T10:42:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.821+0000 7f5845600700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5830039230 0x7f583003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:42:37.812 INFO:tasks.workunit.client.0.smithi115.stdout:9/311: dread d6/d7/d30/d37/f4d [0,4194304] 0 2024-03-22T10:42:37.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.821+0000 7f5847e00700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5830039230 0x7f583003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:42:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.822+0000 7f5847e00700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5830039230 0x7f583003b6e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f583c005950 tx=0x7f583c009310 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:42:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.823+0000 7f5845600700 1 -- 172.21.15.115:0/1774783633 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f583404ee40 con 0x7f58480fe9d0 2024-03-22T10:42:37.814 INFO:tasks.workunit.client.0.smithi115.stdout:7/254: truncate d2/d7/d12/d10/d19/d42/d46/f48 161442 0 2024-03-22T10:42:37.814 INFO:tasks.workunit.client.0.smithi115.stdout:3/188: write d3/f23 [240187,88205] 0 2024-03-22T10:42:37.816 INFO:tasks.workunit.client.0.smithi115.stdout:3/189: chown d3/f23 7361 1 2024-03-22T10:42:37.816 INFO:tasks.workunit.client.0.smithi115.stdout:7/255: stat d2/d7/fb 0 2024-03-22T10:42:37.819 INFO:tasks.workunit.client.0.smithi115.stdout:9/312: write d6/d7/d19/f27 [1866306,107146] 0 2024-03-22T10:42:37.820 INFO:tasks.workunit.client.0.smithi115.stdout:7/256: fdatasync d2/d7/d12/d10/d23/d3d/f2a 0 2024-03-22T10:42:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:37.831+0000 7f5845600700 1 -- 172.21.15.115:0/1774783633 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f5834017480 con 0x7f58480fe9d0 2024-03-22T10:42:37.823 INFO:tasks.workunit.client.0.smithi115.stdout:3/190: creat d3/d16/d29/f39 x:0 0 0 2024-03-22T10:42:37.830 INFO:tasks.workunit.client.0.smithi115.stdout:9/313: readlink d6/l10 0 2024-03-22T10:42:37.832 INFO:tasks.workunit.client.0.smithi115.stdout:3/191: mkdir d3/d19/d1b/d3a 0 2024-03-22T10:42:37.834 INFO:tasks.workunit.client.0.smithi115.stdout:3/192: write d3/d19/d1b/f27 [1063262,67539] 0 2024-03-22T10:42:37.837 INFO:tasks.workunit.client.0.smithi115.stdout:9/314: fsync d6/d7/d19/f21 0 2024-03-22T10:42:37.841 INFO:tasks.workunit.client.0.smithi115.stdout:3/193: unlink d3/c13 0 2024-03-22T10:42:37.845 INFO:tasks.workunit.client.0.smithi115.stdout:9/315: rename d6/d7/d19/f21 to d6/d7/d19/d4e/f6d 0 2024-03-22T10:42:37.845 INFO:tasks.workunit.client.0.smithi115.stdout:3/194: write d3/d19/d1b/f22 [1506980,76080] 0 2024-03-22T10:42:37.849 INFO:tasks.workunit.client.0.smithi115.stdout:7/257: dwrite d2/d7/f13 [0,4194304] 0 2024-03-22T10:42:37.850 INFO:tasks.workunit.client.0.smithi115.stdout:3/195: write d3/f25 [311269,61962] 0 2024-03-22T10:42:37.853 INFO:tasks.workunit.client.0.smithi115.stdout:6/242: sync 2024-03-22T10:42:37.853 INFO:tasks.workunit.client.0.smithi115.stdout:5/223: sync 2024-03-22T10:42:37.854 INFO:tasks.workunit.client.0.smithi115.stdout:1/339: sync 2024-03-22T10:42:37.854 INFO:tasks.workunit.client.0.smithi115.stdout:4/239: sync 2024-03-22T10:42:37.854 INFO:tasks.workunit.client.0.smithi115.stdout:9/316: symlink d6/d5e/l6e 0 2024-03-22T10:42:37.857 INFO:tasks.workunit.client.0.smithi115.stdout:2/192: truncate da/f16 870442 0 2024-03-22T10:42:37.860 INFO:tasks.workunit.client.0.smithi115.stdout:4/240: write d1/d30/d42/f3d [880080,46903] 0 2024-03-22T10:42:37.867 INFO:tasks.workunit.client.0.smithi115.stdout:2/193: write da/d1c/f1f [2059004,114362] 0 2024-03-22T10:42:37.869 INFO:tasks.workunit.client.0.smithi115.stdout:1/340: link d0/f14 d0/d8/d4d/f71 0 2024-03-22T10:42:37.874 INFO:tasks.workunit.client.0.smithi115.stdout:9/317: mkdir d6/d7/d6a/d6f 0 2024-03-22T10:42:37.876 INFO:tasks.workunit.client.0.smithi115.stdout:6/243: dread d4/d11/f2a [0,4194304] 0 2024-03-22T10:42:37.879 INFO:tasks.workunit.client.0.smithi115.stdout:1/341: write d0/d8/db/f2c [339992,88123] 0 2024-03-22T10:42:37.880 INFO:tasks.workunit.client.0.smithi115.stdout:2/194: mknod da/d25/d2a/c38 0 2024-03-22T10:42:37.882 INFO:tasks.workunit.client.0.smithi115.stdout:2/195: rename da/d1c to da/d1c/d39 22 2024-03-22T10:42:37.883 INFO:tasks.workunit.client.0.smithi115.stdout:3/196: dwrite d3/f9 [0,4194304] 0 2024-03-22T10:42:37.888 INFO:tasks.workunit.client.0.smithi115.stdout:3/197: write d3/d19/f31 [108273,29515] 0 2024-03-22T10:42:37.893 INFO:tasks.workunit.client.0.smithi115.stdout:4/241: getdents d1/d16/d28 0 2024-03-22T10:42:37.894 INFO:tasks.workunit.client.0.smithi115.stdout:7/258: write d2/d7/d12/d10/f25 [966166,78962] 0 2024-03-22T10:42:37.896 INFO:tasks.workunit.client.0.smithi115.stdout:6/244: dread d4/d11/d12/f31 [0,4194304] 0 2024-03-22T10:42:37.897 INFO:tasks.workunit.client.0.smithi115.stdout:2/196: dread da/d1c/f2b [0,4194304] 0 2024-03-22T10:42:37.898 INFO:tasks.workunit.client.0.smithi115.stdout:1/342: symlink d0/d12/l72 0 2024-03-22T10:42:37.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:37 smithi150 ceph-mon[34518]: pgmap v7: 65 pgs: 65 active+clean; 558 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 99 MiB/s wr, 219 op/s 2024-03-22T10:42:37.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:37 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1159483323' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:42:37.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:37 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:37.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:37 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:37.903 INFO:tasks.workunit.client.0.smithi115.stdout:2/197: fdatasync da/f1d 0 2024-03-22T10:42:37.910 INFO:tasks.workunit.client.0.smithi115.stdout:4/242: dwrite d1/d30/d42/f45 [0,4194304] 0 2024-03-22T10:42:37.910 INFO:tasks.workunit.client.0.smithi115.stdout:3/198: mknod d3/d19/d1b/d3a/c3b 0 2024-03-22T10:42:37.914 INFO:tasks.workunit.client.0.smithi115.stdout:2/198: rename da/d25/c2d to da/d25/d2a/c3a 0 2024-03-22T10:42:37.918 INFO:tasks.workunit.client.0.smithi115.stdout:6/245: getdents d4/d36/d48 0 2024-03-22T10:42:37.925 INFO:tasks.workunit.client.0.smithi115.stdout:5/224: dread d2/f4 [4194304,4194304] 0 2024-03-22T10:42:37.928 INFO:tasks.workunit.client.0.smithi115.stdout:6/246: readlink d4/l18 0 2024-03-22T10:42:37.928 INFO:tasks.workunit.client.0.smithi115.stdout:3/199: unlink d3/d16/f21 0 2024-03-22T10:42:37.931 INFO:tasks.workunit.client.0.smithi115.stdout:2/199: truncate da/fb 875602 0 2024-03-22T10:42:37.931 INFO:tasks.workunit.client.0.smithi115.stdout:7/259: dwrite d2/d7/d12/d10/d23/f4e [0,4194304] 0 2024-03-22T10:42:37.935 INFO:tasks.workunit.client.0.smithi115.stdout:5/225: creat d2/d15/d32/f46 x:0 0 0 2024-03-22T10:42:37.937 INFO:tasks.workunit.client.0.smithi115.stdout:6/247: creat d4/d11/d12/f4e x:0 0 0 2024-03-22T10:42:37.947 INFO:tasks.workunit.client.0.smithi115.stdout:5/226: symlink d2/df/l47 0 2024-03-22T10:42:37.951 INFO:tasks.workunit.client.0.smithi115.stdout:7/260: link d2/d7/f13 d2/d7/d12/d10/d23/d3d/d4a/f54 0 2024-03-22T10:42:37.954 INFO:tasks.workunit.client.0.smithi115.stdout:4/243: dwrite d1/d16/f26 [0,4194304] 0 2024-03-22T10:42:37.956 INFO:tasks.workunit.client.0.smithi115.stdout:2/200: dwrite da/d25/d30/f31 [0,4194304] 0 2024-03-22T10:42:37.957 INFO:tasks.workunit.client.0.smithi115.stdout:3/200: getdents d3/d19/d1b 0 2024-03-22T10:42:37.959 INFO:tasks.workunit.client.0.smithi115.stdout:2/201: chown l6 0 1 2024-03-22T10:42:37.963 INFO:tasks.workunit.client.0.smithi115.stdout:6/248: symlink d4/d11/l4f 0 2024-03-22T10:42:37.963 INFO:tasks.workunit.client.0.smithi115.stdout:3/201: write d3/f9 [6078139,60330] 0 2024-03-22T10:42:37.967 INFO:tasks.workunit.client.0.smithi115.stdout:1/343: dread d0/f3 [0,4194304] 0 2024-03-22T10:42:37.968 INFO:tasks.workunit.client.0.smithi115.stdout:7/261: mknod d2/d7/d12/d10/d19/d42/c55 0 2024-03-22T10:42:37.972 INFO:tasks.workunit.client.0.smithi115.stdout:3/202: creat d3/d16/d29/f3c x:0 0 0 2024-03-22T10:42:37.973 INFO:tasks.workunit.client.0.smithi115.stdout:4/244: rename d1/d30/d42/f31 to d1/d16/d36/f50 0 2024-03-22T10:42:37.977 INFO:tasks.workunit.client.0.smithi115.stdout:1/344: unlink d0/d12/d17/f42 0 2024-03-22T10:42:37.977 INFO:tasks.workunit.client.0.smithi115.stdout:7/262: fsync d2/d7/d12/f1b 0 2024-03-22T10:42:37.979 INFO:tasks.workunit.client.0.smithi115.stdout:6/249: write d4/d13/d33/f34 [3116471,129091] 0 2024-03-22T10:42:37.983 INFO:tasks.workunit.client.0.smithi115.stdout:7/263: chown d2/d7/l1f 4 1 2024-03-22T10:42:37.983 INFO:tasks.workunit.client.0.smithi115.stdout:4/245: mknod d1/d30/d42/d32/c51 0 2024-03-22T10:42:37.987 INFO:tasks.workunit.client.0.smithi115.stdout:1/345: creat d0/d8/db/d25/f73 x:0 0 0 2024-03-22T10:42:37.991 INFO:tasks.workunit.client.0.smithi115.stdout:4/246: creat d1/d16/d36/f52 x:0 0 0 2024-03-22T10:42:37.991 INFO:tasks.workunit.client.0.smithi115.stdout:3/203: dread d3/f7 [0,4194304] 0 2024-03-22T10:42:37.992 INFO:tasks.workunit.client.0.smithi115.stdout:2/202: getdents da/d25/d2a 0 2024-03-22T10:42:37.993 INFO:tasks.workunit.client.0.smithi115.stdout:1/346: fdatasync d0/d12/d17/f3e 0 2024-03-22T10:42:37.994 INFO:tasks.workunit.client.0.smithi115.stdout:6/250: rename d4/cb to d4/d13/d33/c50 0 2024-03-22T10:42:37.996 INFO:tasks.workunit.client.0.smithi115.stdout:2/203: chown da/f15 0 1 2024-03-22T10:42:37.997 INFO:tasks.workunit.client.0.smithi115.stdout:6/251: fdatasync d4/d11/f29 0 2024-03-22T10:42:37.999 INFO:tasks.workunit.client.0.smithi115.stdout:5/227: dwrite d2/fc [0,4194304] 0 2024-03-22T10:42:38.000 INFO:tasks.workunit.client.0.smithi115.stdout:3/204: write d3/f8 [17804788,108765] 0 2024-03-22T10:42:38.001 INFO:tasks.workunit.client.0.smithi115.stdout:4/247: creat d1/d30/d42/d32/f53 x:0 0 0 2024-03-22T10:42:38.006 INFO:tasks.workunit.client.0.smithi115.stdout:3/205: write d3/f23 [1846220,31311] 0 2024-03-22T10:42:38.006 INFO:tasks.workunit.client.0.smithi115.stdout:2/204: symlink da/d25/l3b 0 2024-03-22T10:42:38.007 INFO:tasks.workunit.client.0.smithi115.stdout:7/264: dread d2/d7/d12/f4b [0,4194304] 0 2024-03-22T10:42:38.010 INFO:tasks.workunit.client.0.smithi115.stdout:5/228: symlink d2/d15/d1b/d33/l48 0 2024-03-22T10:42:38.013 INFO:tasks.workunit.client.0.smithi115.stdout:7/265: read d2/d7/d12/d10/d23/d3d/f21 [559324,86821] 0 2024-03-22T10:42:38.013 INFO:tasks.workunit.client.0.smithi115.stdout:4/248: readlink d1/dc/d14/d33/l37 0 2024-03-22T10:42:38.016 INFO:tasks.workunit.client.0.smithi115.stdout:1/347: symlink d0/d12/d17/d37/d5c/l74 0 2024-03-22T10:42:38.017 INFO:tasks.workunit.client.0.smithi115.stdout:4/249: write d1/dc/f20 [4562245,10602] 0 2024-03-22T10:42:38.017 INFO:tasks.workunit.client.0.smithi115.stdout:1/348: dread - d0/d8/de/f4c zero size 2024-03-22T10:42:38.018 INFO:tasks.workunit.client.0.smithi115.stdout:1/349: stat d0/d8/de 0 2024-03-22T10:42:38.020 INFO:tasks.workunit.client.0.smithi115.stdout:4/250: write d1/d16/d35/d40/f48 [1383867,67932] 0 2024-03-22T10:42:38.021 INFO:tasks.workunit.client.0.smithi115.stdout:2/205: read da/fb [6580,3666] 0 2024-03-22T10:42:38.022 INFO:tasks.workunit.client.0.smithi115.stdout:3/206: mknod d3/d16/d29/c3d 0 2024-03-22T10:42:38.023 INFO:tasks.workunit.client.0.smithi115.stdout:4/251: read d1/d16/f1c [211007,28598] 0 2024-03-22T10:42:38.027 INFO:tasks.workunit.client.0.smithi115.stdout:6/252: getdents d4 0 2024-03-22T10:42:38.028 INFO:tasks.workunit.client.0.smithi115.stdout:6/253: chown d4/d13 6048476 1 2024-03-22T10:42:38.028 INFO:tasks.workunit.client.0.smithi115.stdout:7/266: mknod d2/d7/d12/d10/d19/d42/c56 0 2024-03-22T10:42:38.029 INFO:tasks.workunit.client.0.smithi115.stdout:6/254: dread - d4/d11/d12/f4e zero size 2024-03-22T10:42:38.030 INFO:tasks.workunit.client.0.smithi115.stdout:1/350: mkdir d0/d22/d75 0 2024-03-22T10:42:38.030 INFO:tasks.workunit.client.0.smithi115.stdout:1/351: stat d0/f14 0 2024-03-22T10:42:38.032 INFO:tasks.workunit.client.0.smithi115.stdout:2/206: creat da/d25/d30/d27/f3c x:0 0 0 2024-03-22T10:42:38.033 INFO:tasks.workunit.client.0.smithi115.stdout:1/352: chown d0/d8/de/d1a/f2f 40 1 2024-03-22T10:42:38.033 INFO:tasks.workunit.client.0.smithi115.stdout:5/229: mkdir d2/d15/d1b/d33/d38/d43/d49 0 2024-03-22T10:42:38.034 INFO:tasks.workunit.client.0.smithi115.stdout:4/252: mkdir d1/d16/d35/d40/d54 0 2024-03-22T10:42:38.036 INFO:tasks.workunit.client.0.smithi115.stdout:4/253: stat d1/d30/d42/c1e 0 2024-03-22T10:42:38.036 INFO:tasks.workunit.client.0.smithi115.stdout:6/255: mknod d4/c51 0 2024-03-22T10:42:38.038 INFO:tasks.workunit.client.0.smithi115.stdout:4/254: readlink d1/d16/l2e 0 2024-03-22T10:42:38.038 INFO:tasks.workunit.client.0.smithi115.stdout:3/207: creat d3/d36/f3e x:0 0 0 2024-03-22T10:42:38.039 INFO:tasks.workunit.client.0.smithi115.stdout:5/230: mkdir d2/d15/d1b/d33/d38/d4a 0 2024-03-22T10:42:38.041 INFO:tasks.workunit.client.0.smithi115.stdout:7/267: truncate d2/d7/d12/d10/d19/f37 2056800 0 2024-03-22T10:42:38.041 INFO:tasks.workunit.client.0.smithi115.stdout:3/208: dread - d3/d36/f3e zero size 2024-03-22T10:42:38.043 INFO:tasks.workunit.client.0.smithi115.stdout:4/255: fsync d1/d30/d42/f21 0 2024-03-22T10:42:38.043 INFO:tasks.workunit.client.0.smithi115.stdout:5/231: dread - d2/d15/d1b/d33/d38/f3d zero size 2024-03-22T10:42:38.045 INFO:tasks.workunit.client.0.smithi115.stdout:5/232: readlink d2/df/d3b/l31 0 2024-03-22T10:42:38.047 INFO:tasks.workunit.client.0.smithi115.stdout:3/209: creat d3/d16/d2f/f3f x:0 0 0 2024-03-22T10:42:38.048 INFO:tasks.workunit.client.0.smithi115.stdout:7/268: write d2/d7/d12/d10/d19/d42/d46/f48 [150057,80271] 0 2024-03-22T10:42:38.048 INFO:tasks.workunit.client.0.smithi115.stdout:0/253: sync 2024-03-22T10:42:38.048 INFO:tasks.workunit.client.0.smithi115.stdout:8/426: sync 2024-03-22T10:42:38.049 INFO:tasks.workunit.client.0.smithi115.stdout:7/269: readlink d2/d7/d12/l1c 0 2024-03-22T10:42:38.052 INFO:tasks.workunit.client.0.smithi115.stdout:3/210: write d3/d16/d2f/f3f [874148,52502] 0 2024-03-22T10:42:38.052 INFO:tasks.workunit.client.0.smithi115.stdout:5/233: unlink d2/d15/d32/f46 0 2024-03-22T10:42:38.058 INFO:tasks.workunit.client.0.smithi115.stdout:1/353: truncate d0/d8/f9 1714104 0 2024-03-22T10:42:38.063 INFO:tasks.workunit.client.0.smithi115.stdout:0/254: mkdir d3/da/d4c 0 2024-03-22T10:42:38.063 INFO:tasks.workunit.client.0.smithi115.stdout:6/256: readlink d4/d36/l4a 0 2024-03-22T10:42:38.063 INFO:tasks.workunit.client.0.smithi115.stdout:7/270: symlink d2/d7/d12/d10/d23/d33/l57 0 2024-03-22T10:42:38.064 INFO:tasks.workunit.client.0.smithi115.stdout:4/256: rename d1/d16/d36 to d1/dc/d55 0 2024-03-22T10:42:38.065 INFO:tasks.workunit.client.0.smithi115.stdout:2/207: dread da/d1c/f1e [0,4194304] 0 2024-03-22T10:42:38.067 INFO:tasks.workunit.client.0.smithi115.stdout:4/257: fsync d1/d16/f26 0 2024-03-22T10:42:38.069 INFO:tasks.workunit.client.0.smithi115.stdout:2/208: readlink da/d25/d30/d27/l36 0 2024-03-22T10:42:38.070 INFO:tasks.workunit.client.0.smithi115.stdout:6/257: write d4/f5 [7166771,73223] 0 2024-03-22T10:42:38.078 INFO:tasks.workunit.client.0.smithi115.stdout:5/234: dread d2/f4 [0,4194304] 0 2024-03-22T10:42:38.082 INFO:tasks.workunit.client.0.smithi115.stdout:5/235: fsync d2/d15/d1b/d33/d38/f3d 0 2024-03-22T10:42:38.083 INFO:tasks.workunit.client.0.smithi115.stdout:3/211: truncate d3/f7 2628024 0 2024-03-22T10:42:38.086 INFO:tasks.workunit.client.0.smithi115.stdout:2/209: rename da/d25/d30/d27 to da/d25/d3d 0 2024-03-22T10:42:38.087 INFO:tasks.workunit.client.0.smithi115.stdout:6/258: chown d4/d11/d12/d1c/c41 461171511 1 2024-03-22T10:42:38.088 INFO:tasks.workunit.client.0.smithi115.stdout:8/427: dwrite d6/fc [0,4194304] 0 2024-03-22T10:42:38.088 INFO:tasks.workunit.client.0.smithi115.stdout:5/236: read - d2/d15/d1b/d33/d38/f3d zero size 2024-03-22T10:42:38.091 INFO:tasks.workunit.client.0.smithi115.stdout:4/258: dwrite d1/d30/d42/f34 [0,4194304] 0 2024-03-22T10:42:38.091 INFO:tasks.workunit.client.0.smithi115.stdout:7/271: unlink d2/d7/d12/d10/d23/d33/c51 0 2024-03-22T10:42:38.093 INFO:tasks.workunit.client.0.smithi115.stdout:8/428: chown d6/dd/d24/d4a/d41/f4b 18378 1 2024-03-22T10:42:38.095 INFO:tasks.workunit.client.0.smithi115.stdout:2/210: rename l6 to da/d1c/l3e 0 2024-03-22T10:42:38.095 INFO:tasks.workunit.client.0.smithi115.stdout:1/354: getdents d0/d12 0 2024-03-22T10:42:38.104 INFO:tasks.workunit.client.0.smithi115.stdout:5/237: write d2/f2e [527040,78089] 0 2024-03-22T10:42:38.105 INFO:tasks.workunit.client.0.smithi115.stdout:8/429: creat d6/dd/d24/d4a/f87 x:0 0 0 2024-03-22T10:42:38.106 INFO:tasks.workunit.client.0.smithi115.stdout:7/272: read d2/f3f [3740241,87142] 0 2024-03-22T10:42:38.108 INFO:tasks.workunit.client.0.smithi115.stdout:5/238: chown d2/d15/d1b/d33/c37 121594 1 2024-03-22T10:42:38.112 INFO:tasks.workunit.client.0.smithi115.stdout:0/255: link d3/d4/c2c d3/d4/c4d 0 2024-03-22T10:42:38.113 INFO:tasks.workunit.client.0.smithi115.stdout:6/259: rename d4/d11/f29 to d4/d11/f52 0 2024-03-22T10:42:38.116 INFO:tasks.workunit.client.0.smithi115.stdout:8/430: mkdir d6/d49/d88 0 2024-03-22T10:42:38.117 INFO:tasks.workunit.client.0.smithi115.stdout:7/273: creat d2/d7/d12/d10/d23/d33/f58 x:0 0 0 2024-03-22T10:42:38.119 INFO:tasks.workunit.client.0.smithi115.stdout:1/355: dread d0/d8/de/f5f [0,4194304] 0 2024-03-22T10:42:38.120 INFO:tasks.workunit.client.0.smithi115.stdout:9/318: sync 2024-03-22T10:42:38.123 INFO:tasks.workunit.client.0.smithi115.stdout:4/259: truncate d1/dc/d14/f1d 1075283 0 2024-03-22T10:42:38.124 INFO:tasks.workunit.client.0.smithi115.stdout:0/256: symlink d3/l4e 0 2024-03-22T10:42:38.127 INFO:tasks.workunit.client.0.smithi115.stdout:7/274: fsync d2/d7/d12/d10/f17 0 2024-03-22T10:42:38.131 INFO:tasks.workunit.client.0.smithi115.stdout:8/431: symlink d6/dd/d12/l89 0 2024-03-22T10:42:38.133 INFO:tasks.workunit.client.0.smithi115.stdout:2/211: dwrite da/d25/f37 [0,4194304] 0 2024-03-22T10:42:38.139 INFO:tasks.workunit.client.0.smithi115.stdout:2/212: chown da/d25/f37 0 1 2024-03-22T10:42:38.139 INFO:tasks.workunit.client.0.smithi115.stdout:5/239: dwrite d2/df/f3c [0,4194304] 0 2024-03-22T10:42:38.139 INFO:tasks.workunit.client.0.smithi115.stdout:8/432: stat d6/dd/d24/d3a 0 2024-03-22T10:42:38.141 INFO:tasks.workunit.client.0.smithi115.stdout:0/257: creat d3/d4/d14/f4f x:0 0 0 2024-03-22T10:42:38.142 INFO:tasks.workunit.client.0.smithi115.stdout:3/212: truncate d3/d19/f1a 1680152 0 2024-03-22T10:42:38.152 INFO:tasks.workunit.client.0.smithi115.stdout:8/433: stat d6/dd/d24/l53 0 2024-03-22T10:42:38.160 INFO:tasks.workunit.client.0.smithi115.stdout:7/275: creat d2/d7/d12/d10/d23/d3d/d4a/f59 x:0 0 0 2024-03-22T10:42:38.160 INFO:tasks.workunit.client.0.smithi115.stdout:1/356: write d0/f1 [1218675,2043] 0 2024-03-22T10:42:38.164 INFO:tasks.workunit.client.0.smithi115.stdout:4/260: rename d1/d30/d42/l29 to d1/dc/l56 0 2024-03-22T10:42:38.166 INFO:tasks.workunit.client.0.smithi115.stdout:5/240: symlink d2/d15/d1b/d33/d38/d43/d49/l4b 0 2024-03-22T10:42:38.167 INFO:tasks.workunit.client.0.smithi115.stdout:6/260: dread d4/f5 [0,4194304] 0 2024-03-22T10:42:38.173 INFO:tasks.workunit.client.0.smithi115.stdout:2/213: getdents da/d25 0 2024-03-22T10:42:38.173 INFO:tasks.workunit.client.0.smithi115.stdout:3/213: dread d3/f9 [4194304,4194304] 0 2024-03-22T10:42:38.183 INFO:tasks.workunit.client.0.smithi115.stdout:7/276: creat d2/d7/d12/d10/d23/d3d/f5a x:0 0 0 2024-03-22T10:42:38.184 INFO:tasks.workunit.client.0.smithi115.stdout:2/214: chown da/d1c/f2b 182216 1 2024-03-22T10:42:38.186 INFO:tasks.workunit.client.0.smithi115.stdout:9/319: dread d6/d8/f1c [4194304,4194304] 0 2024-03-22T10:42:38.189 INFO:tasks.workunit.client.0.smithi115.stdout:1/357: write d0/d8/de/f5f [1237003,36557] 0 2024-03-22T10:42:38.194 INFO:tasks.workunit.client.0.smithi115.stdout:0/258: rename d3/da/c36 to d3/d7/c50 0 2024-03-22T10:42:38.194 INFO:tasks.workunit.client.0.smithi115.stdout:1/358: write d0/d22/f46 [1236064,112445] 0 2024-03-22T10:42:38.195 INFO:tasks.workunit.client.0.smithi115.stdout:2/215: symlink da/d25/d3d/l3f 0 2024-03-22T10:42:38.199 INFO:tasks.workunit.client.0.smithi115.stdout:3/214: creat d3/d19/d2b/f40 x:0 0 0 2024-03-22T10:42:38.201 INFO:tasks.workunit.client.0.smithi115.stdout:4/261: link d1/f9 d1/f57 0 2024-03-22T10:42:38.203 INFO:tasks.workunit.client.0.smithi115.stdout:6/261: dwrite d4/d36/f49 [0,4194304] 0 2024-03-22T10:42:38.203 INFO:tasks.workunit.client.0.smithi115.stdout:8/434: dwrite d6/fc [0,4194304] 0 2024-03-22T10:42:38.204 INFO:tasks.workunit.client.0.smithi115.stdout:7/277: rename d2/d7/d12/f1b to d2/d7/d12/d10/f5b 0 2024-03-22T10:42:38.206 INFO:tasks.workunit.client.0.smithi115.stdout:5/241: getdents d2/df/d3b 0 2024-03-22T10:42:38.208 INFO:tasks.workunit.client.0.smithi115.stdout:1/359: mknod d0/d8/d4d/d56/c76 0 2024-03-22T10:42:38.208 INFO:tasks.workunit.client.0.smithi115.stdout:4/262: readlink d1/d16/d35/l3f 0 2024-03-22T10:42:38.210 INFO:tasks.workunit.client.0.smithi115.stdout:0/259: mknod d3/c51 0 2024-03-22T10:42:38.211 INFO:tasks.workunit.client.0.smithi115.stdout:5/242: write d2/f1f [516799,12074] 0 2024-03-22T10:42:38.213 INFO:tasks.workunit.client.0.smithi115.stdout:2/216: rename da/f12 to da/d25/f40 0 2024-03-22T10:42:38.213 INFO:tasks.workunit.client.0.smithi115.stdout:4/263: truncate d1/dc/f23 123983 0 2024-03-22T10:42:38.214 INFO:tasks.workunit.client.0.smithi115.stdout:7/278: write d2/d7/d12/d10/d23/d33/f3b [487767,115695] 0 2024-03-22T10:42:38.217 INFO:tasks.workunit.client.0.smithi115.stdout:6/262: chown d4/d13/f4b 955 1 2024-03-22T10:42:38.229 INFO:tasks.workunit.client.0.smithi115.stdout:4/264: write d1/d30/d42/f3d [627245,94120] 0 2024-03-22T10:42:38.230 INFO:tasks.workunit.client.0.smithi115.stdout:1/360: mkdir d0/d12/d2b/d77 0 2024-03-22T10:42:38.231 INFO:tasks.workunit.client.0.smithi115.stdout:3/215: rename d3/d16/d2f to d3/d36/d41 0 2024-03-22T10:42:38.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:37 smithi115 ceph-mon[29111]: pgmap v7: 65 pgs: 65 active+clean; 558 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 99 MiB/s wr, 219 op/s 2024-03-22T10:42:38.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:37 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1159483323' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:42:38.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:37 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:38.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:37 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:38.236 INFO:tasks.workunit.client.0.smithi115.stdout:6/263: creat d4/d36/f53 x:0 0 0 2024-03-22T10:42:38.237 INFO:tasks.workunit.client.0.smithi115.stdout:7/279: creat d2/d7/d12/d10/d23/d3d/d4a/f5c x:0 0 0 2024-03-22T10:42:38.239 INFO:tasks.workunit.client.0.smithi115.stdout:3/216: dread - d3/d16/d29/f39 zero size 2024-03-22T10:42:38.244 INFO:tasks.workunit.client.0.smithi115.stdout:9/320: dwrite d6/d7/d20/d22/f2b [4194304,4194304] 0 2024-03-22T10:42:38.247 INFO:tasks.workunit.client.0.smithi115.stdout:1/361: write d0/d8/de/d1a/f2f [2384375,57544] 0 2024-03-22T10:42:38.251 INFO:tasks.workunit.client.0.smithi115.stdout:0/260: write d3/f33 [1010079,109106] 0 2024-03-22T10:42:38.257 INFO:tasks.workunit.client.0.smithi115.stdout:0/261: chown d3/f33 1 1 2024-03-22T10:42:38.257 INFO:tasks.workunit.client.0.smithi115.stdout:9/321: readlink d6/d7/d30/l5b 0 2024-03-22T10:42:38.260 INFO:tasks.workunit.client.0.smithi115.stdout:8/435: dwrite d6/dd/d12/f3b [0,4194304] 0 2024-03-22T10:42:38.261 INFO:tasks.workunit.client.0.smithi115.stdout:3/217: creat d3/d19/d2b/f42 x:0 0 0 2024-03-22T10:42:38.269 INFO:tasks.workunit.client.0.smithi115.stdout:5/243: dwrite d2/df/f3c [0,4194304] 0 2024-03-22T10:42:38.276 INFO:tasks.workunit.client.0.smithi115.stdout:5/244: fdatasync d2/df/f10 0 2024-03-22T10:42:38.276 INFO:tasks.workunit.client.0.smithi115.stdout:7/280: rename d2/d7/d12/d10/d19/d42/c56 to d2/d7/d12/d10/d19/d42/d46/c5d 0 2024-03-22T10:42:38.279 INFO:tasks.workunit.client.0.smithi115.stdout:1/362: unlink d0/l13 0 2024-03-22T10:42:38.281 INFO:tasks.workunit.client.0.smithi115.stdout:6/264: dwrite d4/d11/d12/f4e [0,4194304] 0 2024-03-22T10:42:38.282 INFO:tasks.workunit.client.0.smithi115.stdout:4/265: dwrite d1/dc/f23 [0,4194304] 0 2024-03-22T10:42:38.285 INFO:tasks.workunit.client.0.smithi115.stdout:0/262: rmdir d3/d4/d14 39 2024-03-22T10:42:38.289 INFO:tasks.workunit.client.0.smithi115.stdout:3/218: read d3/d36/d41/f3f [694931,40808] 0 2024-03-22T10:42:38.291 INFO:tasks.workunit.client.0.smithi115.stdout:1/363: rmdir d0/d12/d17 39 2024-03-22T10:42:38.291 INFO:tasks.workunit.client.0.smithi115.stdout:5/245: creat d2/df/d13/f4c x:0 0 0 2024-03-22T10:42:38.299 INFO:tasks.workunit.client.0.smithi115.stdout:2/217: dread da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:38.300 INFO:tasks.workunit.client.0.smithi115.stdout:7/281: creat d2/d7/d12/d10/d19/d42/d46/f5e x:0 0 0 2024-03-22T10:42:38.305 INFO:tasks.workunit.client.0.smithi115.stdout:4/266: rename d1/dc/f20 to d1/dc/d14/f58 0 2024-03-22T10:42:38.306 INFO:tasks.workunit.client.0.smithi115.stdout:5/246: mknod d2/d15/d1b/d33/d38/d43/d49/c4d 0 2024-03-22T10:42:38.307 INFO:tasks.workunit.client.0.smithi115.stdout:8/436: dwrite d6/f23 [0,4194304] 0 2024-03-22T10:42:38.310 INFO:tasks.workunit.client.0.smithi115.stdout:5/247: chown d2/d15/d1b/c34 46 1 2024-03-22T10:42:38.312 INFO:tasks.workunit.client.0.smithi115.stdout:5/248: readlink d2/d15/d32/l41 0 2024-03-22T10:42:38.316 INFO:tasks.workunit.client.0.smithi115.stdout:2/218: rmdir da/d25 39 2024-03-22T10:42:38.317 INFO:tasks.workunit.client.0.smithi115.stdout:0/263: mkdir d3/d52 0 2024-03-22T10:42:38.319 INFO:tasks.workunit.client.0.smithi115.stdout:1/364: dread - d0/d12/f66 zero size 2024-03-22T10:42:38.319 INFO:tasks.workunit.client.0.smithi115.stdout:4/267: rmdir d1/d16/d35 39 2024-03-22T10:42:38.320 INFO:tasks.workunit.client.0.smithi115.stdout:7/282: creat d2/d3a/f5f x:0 0 0 2024-03-22T10:42:38.320 INFO:tasks.workunit.client.0.smithi115.stdout:0/264: chown d3/d7/d1c/l30 50856 1 2024-03-22T10:42:38.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.331+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f584802cc30 con 0x7f58480fe9d0 2024-03-22T10:42:38.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.331+0000 7f5845600700 1 -- 172.21.15.115:0/1774783633 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5834027070 con 0x7f58480fe9d0 2024-03-22T10:42:38.325 INFO:tasks.workunit.client.0.smithi115.stdout:7/283: fdatasync d2/d7/d12/d10/f25 0 2024-03-22T10:42:38.326 INFO:tasks.workunit.client.0.smithi115.stdout:0/265: stat d3/d7/c6 0 2024-03-22T10:42:38.327 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:42:38.327 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:42:38.327 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:42:38.327 INFO:tasks.workunit.client.0.smithi115.stdout:7/284: readlink d2/d7/l1f 0 2024-03-22T10:42:38.328 INFO:tasks.workunit.client.0.smithi115.stdout:6/265: getdents d4/d11/d12/d1c 0 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.335+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5830039230 msgr2=0x7f583003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.335+0000 7f584f800700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5830039230 0x7f583003b6e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f583c005950 tx=0x7f583c009310 comp rx=0 tx=0).stop 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.335+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58480fe9d0 msgr2=0x7f584819bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.335+0000 7f584f800700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58480fe9d0 0x7f584819bf50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5834009230 tx=0x7f583400f690 comp rx=0 tx=0).stop 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.335+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 shutdown_connections 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.335+0000 7f584f800700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f5830039230 0x7f583003b6e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.335+0000 7f584f800700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58480fe9d0 0x7f584819bf50 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5834009230 tx=0x7f583400f690 comp rx=0 tx=0).stop 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.335+0000 7f584f800700 1 --2- 172.21.15.115:0/1774783633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5848101330 0x7f584819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.335+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 >> 172.21.15.115:0/1774783633 conn(0x7f58480fa5e0 msgr2=0x7f58480fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.336+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 shutdown_connections 2024-03-22T10:42:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:42:38.337+0000 7f584f800700 1 -- 172.21.15.115:0/1774783633 wait complete. 2024-03-22T10:42:38.333 INFO:tasks.workunit.client.0.smithi115.stdout:8/437: creat d6/d49/d88/f8a x:0 0 0 2024-03-22T10:42:38.336 INFO:tasks.workunit.client.0.smithi115.stdout:1/365: rename d0/d8/l19 to d0/d58/l78 0 2024-03-22T10:42:38.337 INFO:tasks.workunit.client.0.smithi115.stdout:2/219: mknod da/c41 0 2024-03-22T10:42:38.340 INFO:tasks.workunit.client.0.smithi115.stdout:6/266: chown d4/d13/c35 90 1 2024-03-22T10:42:38.341 INFO:tasks.workunit.client.0.smithi115.stdout:8/438: symlink d6/d49/d88/l8b 0 2024-03-22T10:42:38.343 INFO:tasks.workunit.client.0.smithi115.stdout:6/267: dread - d4/d13/f3d zero size 2024-03-22T10:42:38.345 INFO:tasks.workunit.client.0.smithi115.stdout:6/268: chown d4/d36 2204 1 2024-03-22T10:42:38.348 INFO:tasks.workunit.client.0.smithi115.stdout:8/439: write d6/dd/d24/f72 [3140606,75045] 0 2024-03-22T10:42:38.350 INFO:tasks.workunit.client.0.smithi115.stdout:6/269: chown d4/l17 252428 1 2024-03-22T10:42:38.352 INFO:tasks.workunit.client.0.smithi115.stdout:2/220: write da/d25/d3d/f3c [453333,32250] 0 2024-03-22T10:42:38.356 INFO:tasks.workunit.client.0.smithi115.stdout:0/266: dwrite d3/d7/f45 [0,4194304] 0 2024-03-22T10:42:38.359 INFO:tasks.workunit.client.0.smithi115.stdout:1/366: truncate d0/d8/db/d25/f44 2715481 0 2024-03-22T10:42:38.359 INFO:tasks.workunit.client.0.smithi115.stdout:8/440: chown d6/dd/d12/d31/c52 629624595 1 2024-03-22T10:42:38.361 INFO:tasks.workunit.client.0.smithi115.stdout:4/268: dread d1/d16/f26 [0,4194304] 0 2024-03-22T10:42:38.363 INFO:tasks.workunit.client.0.smithi115.stdout:4/269: chown d1/d30 1 1 2024-03-22T10:42:38.364 INFO:tasks.workunit.client.0.smithi115.stdout:6/270: write d4/d11/d12/f31 [303936,128319] 0 2024-03-22T10:42:38.368 INFO:tasks.workunit.client.0.smithi115.stdout:3/219: dread d3/f25 [0,4194304] 0 2024-03-22T10:42:38.368 INFO:tasks.workunit.client.0.smithi115.stdout:7/285: dread d2/f3f [0,4194304] 0 2024-03-22T10:42:38.369 INFO:tasks.workunit.client.0.smithi115.stdout:8/441: chown d6/d49/d68/l71 112149 1 2024-03-22T10:42:38.370 INFO:tasks.workunit.client.0.smithi115.stdout:1/367: symlink d0/d8/l79 0 2024-03-22T10:42:38.376 INFO:tasks.workunit.client.0.smithi115.stdout:1/368: chown d0/d8/l1f 1496962 1 2024-03-22T10:42:38.377 INFO:tasks.workunit.client.0.smithi115.stdout:6/271: creat d4/d11/f54 x:0 0 0 2024-03-22T10:42:38.379 INFO:tasks.workunit.client.0.smithi115.stdout:7/286: link d2/d7/d12/f4b d2/f60 0 2024-03-22T10:42:38.381 INFO:tasks.workunit.client.0.smithi115.stdout:6/272: symlink d4/d11/d12/l55 0 2024-03-22T10:42:38.382 INFO:tasks.workunit.client.0.smithi115.stdout:1/369: stat d0/d58/l78 0 2024-03-22T10:42:38.387 INFO:tasks.workunit.client.0.smithi115.stdout:5/249: dread d2/df/f10 [0,4194304] 0 2024-03-22T10:42:38.390 INFO:tasks.workunit.client.0.smithi115.stdout:3/220: truncate d3/d19/d1b/f22 814910 0 2024-03-22T10:42:38.391 INFO:tasks.workunit.client.0.smithi115.stdout:0/267: dwrite d3/da/f26 [0,4194304] 0 2024-03-22T10:42:38.391 INFO:tasks.workunit.client.0.smithi115.stdout:7/287: dread - d2/d7/d12/d10/d23/d3d/d4a/f5c zero size 2024-03-22T10:42:38.393 INFO:tasks.workunit.client.0.smithi115.stdout:5/250: mknod d2/df/d3b/c4e 0 2024-03-22T10:42:38.394 INFO:tasks.workunit.client.0.smithi115.stdout:3/221: write d3/d19/f1d [291195,58461] 0 2024-03-22T10:42:38.399 INFO:tasks.workunit.client.0.smithi115.stdout:8/442: dread d6/dd/d12/f3b [0,4194304] 0 2024-03-22T10:42:38.404 INFO:tasks.workunit.client.0.smithi115.stdout:5/251: mkdir d2/d15/d1b/d33/d38/d4f 0 2024-03-22T10:42:38.405 INFO:tasks.workunit.client.0.smithi115.stdout:2/221: dread da/f1b [0,4194304] 0 2024-03-22T10:42:38.409 INFO:tasks.workunit.client.0.smithi115.stdout:7/288: mkdir d2/d7/d12/d10/d19/d42/d46/d61 0 2024-03-22T10:42:38.411 INFO:tasks.workunit.client.0.smithi115.stdout:2/222: chown da/d25 1703 1 2024-03-22T10:42:38.411 INFO:tasks.workunit.client.0.smithi115.stdout:3/222: read d3/d19/d1b/f27 [1032690,76976] 0 2024-03-22T10:42:38.417 INFO:tasks.workunit.client.0.smithi115.stdout:5/252: mkdir d2/df/d27/d50 0 2024-03-22T10:42:38.422 INFO:tasks.workunit.client.0.smithi115.stdout:2/223: fsync da/d25/d3d/f3c 0 2024-03-22T10:42:38.423 INFO:tasks.workunit.client.0.smithi115.stdout:6/273: truncate d4/f24 145403 0 2024-03-22T10:42:38.427 INFO:tasks.workunit.client.0.smithi115.stdout:1/370: dwrite d0/d8/de/d1a/f61 [0,4194304] 0 2024-03-22T10:42:38.427 INFO:tasks.workunit.client.0.smithi115.stdout:3/223: fdatasync d3/d19/d1b/f27 0 2024-03-22T10:42:38.428 INFO:tasks.workunit.client.0.smithi115.stdout:4/270: dread d1/dc/d55/f50 [0,4194304] 0 2024-03-22T10:42:38.429 INFO:tasks.workunit.client.0.smithi115.stdout:2/224: fdatasync da/d25/d30/f31 0 2024-03-22T10:42:38.429 INFO:tasks.workunit.client.0.smithi115.stdout:5/253: fdatasync d2/df/d13/f18 0 2024-03-22T10:42:38.437 INFO:tasks.workunit.client.0.smithi115.stdout:6/274: mknod d4/d13/c56 0 2024-03-22T10:42:38.437 INFO:tasks.workunit.client.0.smithi115.stdout:2/225: read da/d25/d3d/f2c [53083,15155] 0 2024-03-22T10:42:38.438 INFO:tasks.workunit.client.0.smithi115.stdout:5/254: stat d2/d15/d1b/d33/c37 0 2024-03-22T10:42:38.440 INFO:tasks.workunit.client.0.smithi115.stdout:0/268: dwrite d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:38.451 INFO:tasks.workunit.client.0.smithi115.stdout:7/289: dwrite d2/d7/d12/d10/d19/d42/f3e [0,4194304] 0 2024-03-22T10:42:38.453 INFO:tasks.workunit.client.0.smithi115.stdout:1/371: chown d0/d12/d17/f3e 19 1 2024-03-22T10:42:38.455 INFO:tasks.workunit.client.0.smithi115.stdout:6/275: symlink d4/d13/d33/l57 0 2024-03-22T10:42:38.464 INFO:tasks.workunit.client.0.smithi115.stdout:5/255: mkdir d2/d51 0 2024-03-22T10:42:38.466 INFO:tasks.workunit.client.0.smithi115.stdout:0/269: symlink d3/d7/d1b/d35/l53 0 2024-03-22T10:42:38.467 INFO:tasks.workunit.client.0.smithi115.stdout:2/226: dread da/d25/f34 [0,4194304] 0 2024-03-22T10:42:38.467 INFO:tasks.workunit.client.0.smithi115.stdout:1/372: fdatasync d0/f7 0 2024-03-22T10:42:38.467 INFO:tasks.workunit.client.0.smithi115.stdout:8/443: dread d6/dd/d12/f37 [0,4194304] 0 2024-03-22T10:42:38.467 INFO:tasks.workunit.client.0.smithi115.stdout:3/224: dread d3/f9 [0,4194304] 0 2024-03-22T10:42:38.473 INFO:tasks.workunit.client.0.smithi115.stdout:2/227: symlink da/d1c/l42 0 2024-03-22T10:42:38.474 INFO:tasks.workunit.client.0.smithi115.stdout:1/373: write d0/d8/de/d1a/f61 [423265,5398] 0 2024-03-22T10:42:38.476 INFO:tasks.workunit.client.0.smithi115.stdout:8/444: creat d6/dd/d24/d4a/f8c x:0 0 0 2024-03-22T10:42:38.476 INFO:tasks.workunit.client.0.smithi115.stdout:2/228: stat da/c1a 0 2024-03-22T10:42:38.477 INFO:tasks.workunit.client.0.smithi115.stdout:0/270: mkdir d3/d4/d14/d54 0 2024-03-22T10:42:38.477 INFO:tasks.workunit.client.0.smithi115.stdout:4/271: link d1/dc/d14/l2c d1/d16/d35/d40/d54/l59 0 2024-03-22T10:42:38.478 INFO:tasks.workunit.client.0.smithi115.stdout:2/229: read da/d1c/f1e [4009934,86029] 0 2024-03-22T10:42:38.481 INFO:tasks.workunit.client.0.smithi115.stdout:4/272: write d1/d30/d42/f3d [1859966,38151] 0 2024-03-22T10:42:38.486 INFO:tasks.workunit.client.0.smithi115.stdout:2/230: creat da/d25/d30/f43 x:0 0 0 2024-03-22T10:42:38.486 INFO:tasks.workunit.client.0.smithi115.stdout:4/273: dread d1/d16/f1c [0,4194304] 0 2024-03-22T10:42:38.486 INFO:tasks.workunit.client.0.smithi115.stdout:1/374: truncate d0/d8/d4d/d56/f6d 132033 0 2024-03-22T10:42:38.550 INFO:tasks.workunit.client.0.smithi115.stdout:0/271: creat d3/d4/d14/d54/f55 x:0 0 0 2024-03-22T10:42:38.551 INFO:tasks.workunit.client.0.smithi115.stdout:4/274: creat d1/dc/d55/f5a x:0 0 0 2024-03-22T10:42:38.553 INFO:tasks.workunit.client.0.smithi115.stdout:1/375: write d0/d12/f1c [5199438,44126] 0 2024-03-22T10:42:38.556 INFO:tasks.workunit.client.0.smithi115.stdout:2/231: truncate da/d25/d3d/f3c 1201445 0 2024-03-22T10:42:38.556 INFO:tasks.workunit.client.0.smithi115.stdout:4/275: chown d1/dc/f25 10578813 1 2024-03-22T10:42:38.557 INFO:tasks.workunit.client.0.smithi115.stdout:7/290: dread d2/d7/fb [0,4194304] 0 2024-03-22T10:42:38.557 INFO:tasks.workunit.client.0.smithi115.stdout:6/276: dwrite d4/f10 [0,4194304] 0 2024-03-22T10:42:38.558 INFO:tasks.workunit.client.0.smithi115.stdout:0/272: creat d3/d7/d1c/f56 x:0 0 0 2024-03-22T10:42:38.565 INFO:tasks.workunit.client.0.smithi115.stdout:6/277: write d4/d13/f3d [988691,100665] 0 2024-03-22T10:42:38.565 INFO:tasks.workunit.client.0.smithi115.stdout:0/273: write d3/d7/f1d [1312079,34339] 0 2024-03-22T10:42:38.567 INFO:tasks.workunit.client.0.smithi115.stdout:8/445: dwrite d6/f44 [0,4194304] 0 2024-03-22T10:42:38.568 INFO:tasks.workunit.client.0.smithi115.stdout:2/232: mknod da/d25/d2a/c44 0 2024-03-22T10:42:38.572 INFO:tasks.workunit.client.0.smithi115.stdout:2/233: chown da/d25/d3d/f2c 419 1 2024-03-22T10:42:38.573 INFO:tasks.workunit.client.0.smithi115.stdout:4/276: link d1/dc/d55/f50 d1/d30/d42/d32/f5b 0 2024-03-22T10:42:38.575 INFO:tasks.workunit.client.0.smithi115.stdout:0/274: truncate d3/d4/d14/f4f 603940 0 2024-03-22T10:42:38.581 INFO:tasks.workunit.client.0.smithi115.stdout:6/278: mkdir d4/d13/d33/d58 0 2024-03-22T10:42:38.581 INFO:tasks.workunit.client.0.smithi115.stdout:8/446: creat d6/dd/d24/d4a/f8d x:0 0 0 2024-03-22T10:42:38.581 INFO:tasks.workunit.client.0.smithi115.stdout:2/234: symlink da/d25/d2a/l45 0 2024-03-22T10:42:38.583 INFO:tasks.workunit.client.0.smithi115.stdout:0/275: rmdir d3/d7/d1b 39 2024-03-22T10:42:38.586 INFO:tasks.workunit.client.0.smithi115.stdout:2/235: mknod da/d1c/c46 0 2024-03-22T10:42:38.586 INFO:tasks.workunit.client.0.smithi115.stdout:6/279: creat d4/d13/d33/f59 x:0 0 0 2024-03-22T10:42:38.587 INFO:tasks.workunit.client.0.smithi115.stdout:5/256: dread d2/df/d13/f16 [0,4194304] 0 2024-03-22T10:42:38.591 INFO:tasks.workunit.client.0.smithi115.stdout:7/291: dwrite d2/d7/d12/d10/d23/f2e [0,4194304] 0 2024-03-22T10:42:38.591 INFO:tasks.workunit.client.0.smithi115.stdout:8/447: link d6/c8 d6/dd/d24/d4a/d7b/c8e 0 2024-03-22T10:42:38.595 INFO:tasks.workunit.client.0.smithi115.stdout:2/236: write da/d25/d3d/f2c [943382,42565] 0 2024-03-22T10:42:38.595 INFO:tasks.workunit.client.0.smithi115.stdout:5/257: creat d2/d15/d1b/d33/d38/d43/d49/f52 x:0 0 0 2024-03-22T10:42:38.595 INFO:tasks.workunit.client.0.smithi115.stdout:4/277: dread d1/d30/d42/d32/f44 [0,4194304] 0 2024-03-22T10:42:38.603 INFO:tasks.workunit.client.0.smithi115.stdout:8/448: mkdir d6/dd/d24/d4a/d7b/d8f 0 2024-03-22T10:42:38.604 INFO:tasks.workunit.client.0.smithi115.stdout:6/280: mknod d4/c5a 0 2024-03-22T10:42:38.607 INFO:tasks.workunit.client.0.smithi115.stdout:4/278: read d1/dc/d14/f58 [4183870,87182] 0 2024-03-22T10:42:38.607 INFO:tasks.workunit.client.0.smithi115.stdout:7/292: rename d2/d3a/d45 to d2/d62 0 2024-03-22T10:42:38.610 INFO:tasks.workunit.client.0.smithi115.stdout:5/258: symlink d2/df/d27/l53 0 2024-03-22T10:42:38.611 INFO:tasks.workunit.client.0.smithi115.stdout:8/449: chown d6/dd/l82 7788 1 2024-03-22T10:42:38.613 INFO:tasks.workunit.client.0.smithi115.stdout:5/259: stat d2/df/d13/c17 0 2024-03-22T10:42:38.615 INFO:tasks.workunit.client.0.smithi115.stdout:8/450: write d6/dd/f26 [3843213,9645] 0 2024-03-22T10:42:38.615 INFO:tasks.workunit.client.0.smithi115.stdout:4/279: truncate d1/d30/d42/f3d 2910078 0 2024-03-22T10:42:38.617 INFO:tasks.workunit.client.0.smithi115.stdout:6/281: write d4/d36/f53 [233590,27881] 0 2024-03-22T10:42:38.619 INFO:tasks.workunit.client.0.smithi115.stdout:0/276: dwrite d3/d7/d1c/f22 [0,4194304] 0 2024-03-22T10:42:38.622 INFO:tasks.workunit.client.0.smithi115.stdout:1/376: dread d0/d12/f5d [0,4194304] 0 2024-03-22T10:42:38.622 INFO:tasks.workunit.client.0.smithi115.stdout:7/293: unlink d2/d7/d12/d10/d23/d3d/d4a/f54 0 2024-03-22T10:42:38.626 INFO:tasks.workunit.client.0.smithi115.stdout:8/451: creat d6/dd/d12/d31/f90 x:0 0 0 2024-03-22T10:42:38.629 INFO:tasks.workunit.client.0.smithi115.stdout:8/452: write d6/dd/f6a [685606,68887] 0 2024-03-22T10:42:38.629 INFO:tasks.workunit.client.0.smithi115.stdout:0/277: creat d3/d52/f57 x:0 0 0 2024-03-22T10:42:38.633 INFO:tasks.workunit.client.0.smithi115.stdout:4/280: rename d1/dc to d1/d16/d35/d40/d5c 0 2024-03-22T10:42:38.634 INFO:tasks.workunit.client.0.smithi115.stdout:5/260: link d2/d15/d1b/d33/l48 d2/d15/d1b/l54 0 2024-03-22T10:42:38.635 INFO:tasks.workunit.client.0.smithi115.stdout:1/377: write d0/d12/f48 [1273668,71230] 0 2024-03-22T10:42:38.638 INFO:tasks.workunit.client.0.smithi115.stdout:2/237: dwrite da/d1c/f2b [0,4194304] 0 2024-03-22T10:42:38.641 INFO:tasks.workunit.client.0.smithi115.stdout:7/294: symlink d2/d7/d12/d10/d19/d42/d46/d61/l63 0 2024-03-22T10:42:38.652 INFO:tasks.workunit.client.0.smithi115.stdout:4/281: mknod d1/d16/d35/d40/d5c/c5d 0 2024-03-22T10:42:38.652 INFO:tasks.workunit.client.0.smithi115.stdout:8/453: creat d6/f91 x:0 0 0 2024-03-22T10:42:38.653 INFO:tasks.workunit.client.0.smithi115.stdout:1/378: creat d0/f7a x:0 0 0 2024-03-22T10:42:38.656 INFO:tasks.workunit.client.0.smithi115.stdout:7/295: truncate d2/d7/d12/d10/d23/d33/f3b 882885 0 2024-03-22T10:42:38.656 INFO:tasks.workunit.client.0.smithi115.stdout:1/379: fdatasync d0/d8/db/f18 0 2024-03-22T10:42:38.658 INFO:tasks.workunit.client.0.smithi115.stdout:5/261: dwrite d2/df/f3c [4194304,4194304] 0 2024-03-22T10:42:38.659 INFO:tasks.workunit.client.0.smithi115.stdout:4/282: mkdir d1/d16/d35/d40/d54/d5e 0 2024-03-22T10:42:38.662 INFO:tasks.workunit.client.0.smithi115.stdout:1/380: read - d0/d8/db/f2d zero size 2024-03-22T10:42:38.664 INFO:tasks.workunit.client.0.smithi115.stdout:5/262: rmdir d2/d15/d1b/d33 39 2024-03-22T10:42:38.666 INFO:tasks.workunit.client.0.smithi115.stdout:7/296: creat d2/d7/d12/d10/d23/f64 x:0 0 0 2024-03-22T10:42:38.668 INFO:tasks.workunit.client.0.smithi115.stdout:0/278: truncate d3/d7/d1b/d1f/f41 8321169 0 2024-03-22T10:42:38.671 INFO:tasks.workunit.client.0.smithi115.stdout:4/283: mknod d1/d16/d35/d40/d54/d5e/c5f 0 2024-03-22T10:42:38.671 INFO:tasks.workunit.client.0.smithi115.stdout:1/381: truncate d0/d12/f48 2079451 0 2024-03-22T10:42:38.674 INFO:tasks.workunit.client.0.smithi115.stdout:2/238: dwrite da/d25/d3d/f2c [0,4194304] 0 2024-03-22T10:42:38.676 INFO:tasks.workunit.client.0.smithi115.stdout:5/263: dread - d2/d15/d1b/d33/d38/d43/d49/f52 zero size 2024-03-22T10:42:38.681 INFO:tasks.workunit.client.0.smithi115.stdout:8/454: dwrite d6/dd/d12/d31/f5b [0,4194304] 0 2024-03-22T10:42:38.686 INFO:tasks.workunit.client.0.smithi115.stdout:8/455: readlink d6/dd/d24/d4a/l50 0 2024-03-22T10:42:38.691 INFO:tasks.workunit.client.0.smithi115.stdout:5/264: creat d2/df/f55 x:0 0 0 2024-03-22T10:42:38.692 INFO:tasks.workunit.client.0.smithi115.stdout:7/297: rename d2/d7/d12/d10/d23/l35 to d2/d7/d12/d10/d19/d42/l65 0 2024-03-22T10:42:38.698 INFO:tasks.workunit.client.0.smithi115.stdout:2/239: dread da/d25/d3d/f2c [0,4194304] 0 2024-03-22T10:42:38.699 INFO:tasks.workunit.client.0.smithi115.stdout:7/298: unlink d2/d7/l1f 0 2024-03-22T10:42:38.701 INFO:tasks.workunit.client.0.smithi115.stdout:4/284: truncate d1/d16/d35/d40/f48 843170 0 2024-03-22T10:42:38.705 INFO:tasks.workunit.client.0.smithi115.stdout:2/240: mknod da/d25/d30/c47 0 2024-03-22T10:42:38.708 INFO:tasks.workunit.client.0.smithi115.stdout:8/456: dwrite d6/f91 [0,4194304] 0 2024-03-22T10:42:38.710 INFO:tasks.workunit.client.0.smithi115.stdout:0/279: dwrite d3/d4/d14/f23 [0,4194304] 0 2024-03-22T10:42:38.710 INFO:tasks.workunit.client.0.smithi115.stdout:4/285: creat d1/d30/d42/d32/f60 x:0 0 0 2024-03-22T10:42:38.713 INFO:tasks.workunit.client.0.smithi115.stdout:0/280: fsync d3/d52/f57 0 2024-03-22T10:42:38.713 INFO:tasks.workunit.client.0.smithi115.stdout:1/382: dread d0/d22/f41 [0,4194304] 0 2024-03-22T10:42:38.713 INFO:tasks.workunit.client.0.smithi115.stdout:4/286: chown d1/d30/d42/d32/f60 4791 1 2024-03-22T10:42:38.715 INFO:tasks.workunit.client.0.smithi115.stdout:2/241: creat da/d1c/f48 x:0 0 0 2024-03-22T10:42:38.717 INFO:tasks.workunit.client.0.smithi115.stdout:0/281: dread - d3/d7/d1c/f56 zero size 2024-03-22T10:42:38.720 INFO:tasks.workunit.client.0.smithi115.stdout:2/242: chown da/d25 8 1 2024-03-22T10:42:38.723 INFO:tasks.workunit.client.0.smithi115.stdout:7/299: link d2/d7/d12/c2c d2/d7/d12/c66 0 2024-03-22T10:42:38.723 INFO:tasks.workunit.client.0.smithi115.stdout:8/457: creat d6/dd/f92 x:0 0 0 2024-03-22T10:42:38.726 INFO:tasks.workunit.client.0.smithi115.stdout:4/287: creat d1/d16/d35/d40/d5c/d14/d33/f61 x:0 0 0 2024-03-22T10:42:38.727 INFO:tasks.workunit.client.0.smithi115.stdout:5/265: dwrite d2/fe [0,4194304] 0 2024-03-22T10:42:38.727 INFO:tasks.workunit.client.0.smithi115.stdout:9/322: sync 2024-03-22T10:42:38.730 INFO:tasks.workunit.client.0.smithi115.stdout:2/243: creat da/d25/f49 x:0 0 0 2024-03-22T10:42:38.730 INFO:tasks.workunit.client.0.smithi115.stdout:0/282: rmdir d3/d7/d1b/d35 39 2024-03-22T10:42:38.734 INFO:tasks.workunit.client.0.smithi115.stdout:8/458: mknod d6/dd/d24/d3a/c93 0 2024-03-22T10:42:38.735 INFO:tasks.workunit.client.0.smithi115.stdout:1/383: getdents d0 0 2024-03-22T10:42:38.737 INFO:tasks.workunit.client.0.smithi115.stdout:9/323: mkdir d6/d7/d48/d70 0 2024-03-22T10:42:38.738 INFO:tasks.workunit.client.0.smithi115.stdout:7/300: rmdir d2/d7/d12/d10/d19 39 2024-03-22T10:42:38.742 INFO:tasks.workunit.client.0.smithi115.stdout:2/244: mknod da/d25/d2a/c4a 0 2024-03-22T10:42:38.743 INFO:tasks.workunit.client.0.smithi115.stdout:0/283: creat d3/d7/d1b/d1f/f58 x:0 0 0 2024-03-22T10:42:38.746 INFO:tasks.workunit.client.0.smithi115.stdout:1/384: unlink d0/d8/db/d25/f54 0 2024-03-22T10:42:38.751 INFO:tasks.workunit.client.0.smithi115.stdout:5/266: dread d2/d15/d1b/d33/d38/f3a [0,4194304] 0 2024-03-22T10:42:38.752 INFO:tasks.workunit.client.0.smithi115.stdout:0/284: rename d3/d7/d1c/f22 to d3/d52/f59 0 2024-03-22T10:42:38.755 INFO:tasks.workunit.client.0.smithi115.stdout:7/301: dread d2/d7/d12/d10/d19/d42/d46/f48 [0,4194304] 0 2024-03-22T10:42:38.757 INFO:tasks.workunit.client.0.smithi115.stdout:2/245: getdents da/d1c 0 2024-03-22T10:42:38.758 INFO:tasks.workunit.client.0.smithi115.stdout:2/246: readlink da/d1c/l42 0 2024-03-22T10:42:38.761 INFO:tasks.workunit.client.0.smithi115.stdout:4/288: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:38.761 INFO:tasks.workunit.client.0.smithi115.stdout:2/247: read da/d1c/f2b [2345601,103933] 0 2024-03-22T10:42:38.764 INFO:tasks.workunit.client.0.smithi115.stdout:8/459: truncate d6/dd/f2e 479839 0 2024-03-22T10:42:38.765 INFO:tasks.workunit.client.0.smithi115.stdout:5/267: mkdir d2/d15/d1b/d33/d38/d56 0 2024-03-22T10:42:38.766 INFO:tasks.workunit.client.0.smithi115.stdout:9/324: truncate d6/d7/d19/f25 3486643 0 2024-03-22T10:42:38.775 INFO:tasks.workunit.client.0.smithi115.stdout:4/289: readlink d1/d16/d35/d40/d54/l59 0 2024-03-22T10:42:38.775 INFO:tasks.workunit.client.0.smithi115.stdout:1/385: truncate d0/d22/f3d 5333301 0 2024-03-22T10:42:38.777 INFO:tasks.workunit.client.0.smithi115.stdout:5/268: fdatasync d2/f4 0 2024-03-22T10:42:38.780 INFO:tasks.workunit.client.0.smithi115.stdout:7/302: link d2/d7/d12/d10/f17 d2/d7/d12/f67 0 2024-03-22T10:42:38.782 INFO:tasks.workunit.client.0.smithi115.stdout:9/325: write d6/d7/d30/d37/f4d [185338,11635] 0 2024-03-22T10:42:38.783 INFO:tasks.workunit.client.0.smithi115.stdout:1/386: write d0/d8/de/d1a/f61 [3297140,126647] 0 2024-03-22T10:42:38.787 INFO:tasks.workunit.client.0.smithi115.stdout:8/460: dwrite d6/f20 [0,4194304] 0 2024-03-22T10:42:38.789 INFO:tasks.workunit.client.0.smithi115.stdout:2/248: dwrite da/d25/d30/f2e [0,4194304] 0 2024-03-22T10:42:38.791 INFO:tasks.workunit.client.0.smithi115.stdout:5/269: truncate d2/d15/d1b/d33/d38/d43/d49/f52 578346 0 2024-03-22T10:42:38.792 INFO:tasks.workunit.client.0.smithi115.stdout:2/249: write da/d25/d30/f2e [2438308,1871] 0 2024-03-22T10:42:38.793 INFO:tasks.workunit.client.0.smithi115.stdout:5/270: read d2/df/f10 [5159682,22636] 0 2024-03-22T10:42:38.794 INFO:tasks.workunit.client.0.smithi115.stdout:9/326: symlink d6/d7/l71 0 2024-03-22T10:42:38.794 INFO:tasks.workunit.client.0.smithi115.stdout:1/387: mkdir d0/d12/d7b 0 2024-03-22T10:42:38.796 INFO:tasks.workunit.client.0.smithi115.stdout:5/271: stat d2/d15/d1b 0 2024-03-22T10:42:38.803 INFO:tasks.workunit.client.0.smithi115.stdout:4/290: getdents d1/d16/d35/d40/d5c/d14/d33 0 2024-03-22T10:42:38.804 INFO:tasks.workunit.client.0.smithi115.stdout:1/388: mknod d0/d58/c7c 0 2024-03-22T10:42:38.806 INFO:tasks.workunit.client.0.smithi115.stdout:5/272: chown d2/df/d13/l25 909 1 2024-03-22T10:42:38.807 INFO:tasks.workunit.client.0.smithi115.stdout:8/461: creat d6/dd/d12/f94 x:0 0 0 2024-03-22T10:42:38.807 INFO:tasks.workunit.client.0.smithi115.stdout:9/327: truncate d6/d7/f1e 873495 0 2024-03-22T10:42:38.808 INFO:tasks.workunit.client.0.smithi115.stdout:5/273: fsync d2/fe 0 2024-03-22T10:42:38.813 INFO:tasks.workunit.client.0.smithi115.stdout:2/250: truncate da/d1c/f1e 2294417 0 2024-03-22T10:42:38.817 INFO:tasks.workunit.client.0.smithi115.stdout:7/303: dwrite d2/d7/d12/d10/f36 [0,4194304] 0 2024-03-22T10:42:38.817 INFO:tasks.workunit.client.0.smithi115.stdout:1/389: creat d0/d12/d17/f7d x:0 0 0 2024-03-22T10:42:38.818 INFO:tasks.workunit.client.0.smithi115.stdout:2/251: fsync da/d25/f37 0 2024-03-22T10:42:38.818 INFO:tasks.workunit.client.0.smithi115.stdout:5/274: write d2/df/d3b/f2a [1502821,52540] 0 2024-03-22T10:42:38.820 INFO:tasks.workunit.client.0.smithi115.stdout:9/328: mknod d6/d7/d19/c72 0 2024-03-22T10:42:38.826 INFO:tasks.workunit.client.0.smithi115.stdout:1/390: mkdir d0/d8/db/d25/d7e 0 2024-03-22T10:42:38.838 INFO:tasks.workunit.client.0.smithi115.stdout:7/304: dread d2/d7/d12/f4b [0,4194304] 0 2024-03-22T10:42:38.844 INFO:tasks.workunit.client.0.smithi115.stdout:2/252: dwrite da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:38.849 INFO:tasks.workunit.client.0.smithi115.stdout:2/253: write da/f15 [1461902,2610] 0 2024-03-22T10:42:38.851 INFO:tasks.workunit.client.0.smithi115.stdout:1/391: dread d0/f1 [0,4194304] 0 2024-03-22T10:42:38.862 INFO:tasks.workunit.client.0.smithi115.stdout:9/329: stat d6/d8/l18 0 2024-03-22T10:42:38.868 INFO:tasks.workunit.client.0.smithi115.stdout:7/305: rmdir d2/d7/d12/d10/d19/d42/d46 39 2024-03-22T10:42:38.871 INFO:tasks.workunit.client.0.smithi115.stdout:4/291: truncate d1/d30/d42/f3d 140193 0 2024-03-22T10:42:38.871 INFO:tasks.workunit.client.0.smithi115.stdout:1/392: dread d0/f52 [0,4194304] 0 2024-03-22T10:42:38.882 INFO:tasks.workunit.client.0.smithi115.stdout:4/292: fdatasync d1/d16/d35/d40/d5c/d55/f50 0 2024-03-22T10:42:38.886 INFO:tasks.workunit.client.0.smithi115.stdout:9/330: creat d6/d7/d48/d70/f73 x:0 0 0 2024-03-22T10:42:38.900 INFO:tasks.workunit.client.0.smithi115.stdout:3/225: sync 2024-03-22T10:42:38.900 INFO:tasks.workunit.client.0.smithi115.stdout:6/282: sync 2024-03-22T10:42:38.906 INFO:tasks.workunit.client.0.smithi115.stdout:6/283: fdatasync d4/d11/f52 0 2024-03-22T10:42:38.907 INFO:tasks.workunit.client.0.smithi115.stdout:3/226: read d3/d19/d1b/f22 [11813,63651] 0 2024-03-22T10:42:38.909 INFO:tasks.workunit.client.0.smithi115.stdout:2/254: write da/fb [529128,23421] 0 2024-03-22T10:42:38.910 INFO:tasks.workunit.client.0.smithi115.stdout:6/284: truncate d4/d11/d12/d1c/f43 957922 0 2024-03-22T10:42:38.911 INFO:tasks.workunit.client.0.smithi115.stdout:3/227: read d3/f30 [1688390,103760] 0 2024-03-22T10:42:38.912 INFO:tasks.workunit.client.0.smithi115.stdout:5/275: truncate d2/f11 3108499 0 2024-03-22T10:42:38.916 INFO:tasks.workunit.client.0.smithi115.stdout:5/276: chown d2/df/d27/l44 517779322 1 2024-03-22T10:42:38.920 INFO:tasks.workunit.client.0.smithi115.stdout:2/255: creat da/d25/d2a/f4b x:0 0 0 2024-03-22T10:42:38.921 INFO:tasks.workunit.client.0.smithi115.stdout:4/293: truncate d1/d30/d42/d32/f5b 3040936 0 2024-03-22T10:42:38.924 INFO:tasks.workunit.client.0.smithi115.stdout:5/277: mknod d2/df/d27/d50/c57 0 2024-03-22T10:42:38.927 INFO:tasks.workunit.client.0.smithi115.stdout:1/393: rmdir d0/d8/d4d 39 2024-03-22T10:42:38.930 INFO:tasks.workunit.client.0.smithi115.stdout:8/462: creat d6/f95 x:0 0 0 2024-03-22T10:42:38.930 INFO:tasks.workunit.client.0.smithi115.stdout:9/331: dread d6/d8/f47 [0,4194304] 0 2024-03-22T10:42:38.932 INFO:tasks.workunit.client.0.smithi115.stdout:5/278: write d2/df/d3b/f2a [3004268,103438] 0 2024-03-22T10:42:38.934 INFO:tasks.workunit.client.0.smithi115.stdout:1/394: chown d0/d8/de/f26 0 1 2024-03-22T10:42:38.936 INFO:tasks.workunit.client.0.smithi115.stdout:7/306: dwrite d2/d7/d12/d10/d19/f37 [0,4194304] 0 2024-03-22T10:42:38.942 INFO:tasks.workunit.client.1.smithi150.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-22T10:42:38.946 INFO:tasks.workunit.client.0.smithi115.stdout:5/279: write d2/df/d3b/f36 [3649044,14172] 0 2024-03-22T10:42:38.946 INFO:tasks.workunit.client.0.smithi115.stdout:6/285: dwrite d4/f30 [0,4194304] 0 2024-03-22T10:42:38.947 INFO:tasks.workunit.client.0.smithi115.stdout:1/395: read d0/d8/de/d1a/f61 [3382780,45149] 0 2024-03-22T10:42:38.949 INFO:tasks.workunit.client.0.smithi115.stdout:8/463: mknod d6/dd/d24/d4a/c96 0 2024-03-22T10:42:38.949 INFO:tasks.workunit.client.0.smithi115.stdout:3/228: dwrite d3/d16/d29/f39 [0,4194304] 0 2024-03-22T10:42:38.953 INFO:tasks.workunit.client.0.smithi115.stdout:4/294: dwrite d1/d30/f41 [0,4194304] 0 2024-03-22T10:42:38.957 INFO:tasks.workunit.client.0.smithi115.stdout:8/464: write d6/dd/d12/d31/f5b [5015885,30604] 0 2024-03-22T10:42:38.962 INFO:tasks.workunit.client.0.smithi115.stdout:7/307: creat d2/d7/d12/d10/d19/d42/d46/f68 x:0 0 0 2024-03-22T10:42:38.967 INFO:tasks.workunit.client.0.smithi115.stdout:4/295: creat d1/d16/d35/d40/d5c/d14/d33/f62 x:0 0 0 2024-03-22T10:42:38.969 INFO:tasks.workunit.client.0.smithi115.stdout:7/308: rename d2/f3f to d2/d4c/f69 0 2024-03-22T10:42:38.972 INFO:tasks.workunit.client.0.smithi115.stdout:4/296: fsync d1/f57 0 2024-03-22T10:42:38.974 INFO:tasks.workunit.client.0.smithi115.stdout:7/309: write d2/d7/d12/d10/d23/f2e [3665547,68522] 0 2024-03-22T10:42:38.974 INFO:tasks.workunit.client.0.smithi115.stdout:8/465: link d6/dd/d24/d4a/c96 d6/d49/d68/c97 0 2024-03-22T10:42:38.978 INFO:tasks.workunit.client.0.smithi115.stdout:6/286: dwrite d4/d11/d12/d1c/f43 [0,4194304] 0 2024-03-22T10:42:38.980 INFO:tasks.workunit.client.1.smithi150.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T10:42:38.981 INFO:tasks.workunit.client.1.smithi150.stderr:+ make 2024-03-22T10:42:38.984 INFO:tasks.workunit.client.0.smithi115.stdout:5/280: dwrite d2/fe [4194304,4194304] 0 2024-03-22T10:42:38.984 INFO:tasks.workunit.client.0.smithi115.stdout:6/287: dread d4/d11/f44 [0,4194304] 0 2024-03-22T10:42:38.985 INFO:tasks.workunit.client.0.smithi115.stdout:4/297: dread d1/f9 [0,4194304] 0 2024-03-22T10:42:38.986 INFO:tasks.workunit.client.0.smithi115.stdout:7/310: read d2/d7/d12/d10/f40 [2099699,95771] 0 2024-03-22T10:42:38.999 INFO:tasks.workunit.client.0.smithi115.stdout:4/298: rename d1/c8 to d1/d16/d35/d40/d5c/d14/c63 0 2024-03-22T10:42:39.007 INFO:tasks.workunit.client.0.smithi115.stdout:4/299: chown d1/d16/d35/d40/d54/l59 8 1 2024-03-22T10:42:39.011 INFO:tasks.workunit.client.0.smithi115.stdout:1/396: dread d0/d8/de/d1a/f32 [0,4194304] 0 2024-03-22T10:42:39.013 INFO:tasks.workunit.client.0.smithi115.stdout:6/288: link d4/ld d4/l5b 0 2024-03-22T10:42:39.015 INFO:tasks.workunit.client.0.smithi115.stdout:9/332: dread d6/d7/ff [0,4194304] 0 2024-03-22T10:42:39.015 INFO:tasks.workunit.client.0.smithi115.stdout:8/466: dwrite d6/d49/d68/f6f [0,4194304] 0 2024-03-22T10:42:39.015 INFO:tasks.workunit.client.0.smithi115.stdout:4/300: fsync d1/d16/f1c 0 2024-03-22T10:42:39.017 INFO:tasks.workunit.client.0.smithi115.stdout:1/397: stat d0/d12/d17/f23 0 2024-03-22T10:42:39.020 INFO:tasks.workunit.client.0.smithi115.stdout:6/289: creat d4/d36/f5c x:0 0 0 2024-03-22T10:42:39.024 INFO:tasks.workunit.client.0.smithi115.stdout:5/281: dwrite d2/df/d13/f16 [0,4194304] 0 2024-03-22T10:42:39.024 INFO:tasks.workunit.client.0.smithi115.stdout:4/301: fdatasync d1/d16/d35/d40/d5c/d14/f1b 0 2024-03-22T10:42:39.024 INFO:tasks.workunit.client.0.smithi115.stdout:9/333: rmdir d6/d7/d30/d3b 39 2024-03-22T10:42:39.027 INFO:tasks.workunit.client.0.smithi115.stdout:3/229: dread d3/f12 [0,4194304] 0 2024-03-22T10:42:39.027 INFO:tasks.workunit.client.0.smithi115.stdout:1/398: creat d0/d68/f7f x:0 0 0 2024-03-22T10:42:39.029 INFO:tasks.workunit.client.0.smithi115.stdout:6/290: rename d4/d11/f44 to d4/d11/d12/f5d 0 2024-03-22T10:42:39.031 INFO:tasks.workunit.client.0.smithi115.stdout:9/334: write d6/d7/d20/d38/f40 [976092,71395] 0 2024-03-22T10:42:39.033 INFO:tasks.workunit.client.0.smithi115.stdout:1/399: stat d0/d8/db/d25 0 2024-03-22T10:42:39.033 INFO:tasks.workunit.client.0.smithi115.stdout:3/230: chown d3/c6 257265419 1 2024-03-22T10:42:39.034 INFO:tasks.workunit.client.0.smithi115.stdout:9/335: write d6/d7/f17 [2860057,107365] 0 2024-03-22T10:42:39.039 INFO:tasks.workunit.client.0.smithi115.stdout:5/282: mknod d2/c58 0 2024-03-22T10:42:39.039 INFO:tasks.workunit.client.0.smithi115.stdout:8/467: dwrite d6/f20 [0,4194304] 0 2024-03-22T10:42:39.041 INFO:tasks.workunit.client.0.smithi115.stdout:1/400: rmdir d0/d22 39 2024-03-22T10:42:39.042 INFO:tasks.workunit.client.0.smithi115.stdout:3/231: unlink d3/f25 0 2024-03-22T10:42:39.044 INFO:tasks.workunit.client.0.smithi115.stdout:6/291: mkdir d4/d5e 0 2024-03-22T10:42:39.045 INFO:tasks.workunit.client.0.smithi115.stdout:9/336: mknod d6/d7/d6a/d6f/c74 0 2024-03-22T10:42:39.046 INFO:tasks.workunit.client.0.smithi115.stdout:3/232: readlink d3/d19/l2e 0 2024-03-22T10:42:39.047 INFO:tasks.workunit.client.0.smithi115.stdout:8/468: dread - d6/dd/d24/d4a/d41/f4b zero size 2024-03-22T10:42:39.050 INFO:tasks.workunit.client.0.smithi115.stdout:4/302: dwrite d1/d30/d42/d32/f60 [0,4194304] 0 2024-03-22T10:42:39.053 INFO:tasks.workunit.client.0.smithi115.stdout:6/292: unlink d4/d13/d33/f34 0 2024-03-22T10:42:39.055 INFO:tasks.workunit.client.0.smithi115.stdout:6/293: fdatasync d4/d11/f52 0 2024-03-22T10:42:39.063 INFO:tasks.workunit.client.0.smithi115.stdout:5/283: link d2/df/d27/c30 d2/d15/d1b/d33/c59 0 2024-03-22T10:42:39.064 INFO:tasks.workunit.client.0.smithi115.stdout:6/294: chown d4/c9 0 1 2024-03-22T10:42:39.064 INFO:tasks.workunit.client.0.smithi115.stdout:8/469: getdents d6/dd/d24/d4a/d7b 0 2024-03-22T10:42:39.066 INFO:tasks.workunit.client.0.smithi115.stdout:6/295: creat d4/d13/f5f x:0 0 0 2024-03-22T10:42:39.068 INFO:tasks.workunit.client.0.smithi115.stdout:9/337: dwrite d6/d8/f1a [0,4194304] 0 2024-03-22T10:42:39.073 INFO:tasks.workunit.client.0.smithi115.stdout:6/296: rename d4/d13/c19 to d4/d36/c60 0 2024-03-22T10:42:39.074 INFO:tasks.workunit.client.0.smithi115.stdout:4/303: dwrite d1/d16/d35/d40/d5c/d14/d33/f61 [0,4194304] 0 2024-03-22T10:42:39.076 INFO:tasks.workunit.client.0.smithi115.stdout:9/338: mkdir d6/d8/d3c/d59/d75 0 2024-03-22T10:42:39.081 INFO:tasks.workunit.client.0.smithi115.stdout:6/297: unlink d4/c28 0 2024-03-22T10:42:39.083 INFO:tasks.workunit.client.0.smithi115.stdout:3/233: dread d3/d19/f1a [0,4194304] 0 2024-03-22T10:42:39.087 INFO:tasks.workunit.client.0.smithi115.stdout:8/470: dwrite d6/dd/d24/d4a/f80 [0,4194304] 0 2024-03-22T10:42:39.087 INFO:tasks.workunit.client.0.smithi115.stdout:9/339: mknod d6/d8/d3c/c76 0 2024-03-22T10:42:39.090 INFO:tasks.workunit.client.0.smithi115.stdout:9/340: chown d6/d7/d19/d4e/f6d 1753519464 1 2024-03-22T10:42:39.092 INFO:tasks.workunit.client.0.smithi115.stdout:3/234: link d3/d36/f3e d3/d16/d29/f43 0 2024-03-22T10:42:39.094 INFO:tasks.workunit.client.0.smithi115.stdout:3/235: creat d3/d36/f44 x:0 0 0 2024-03-22T10:42:39.099 INFO:tasks.workunit.client.0.smithi115.stdout:3/236: rename d3/d19/d1b/f2c to d3/f45 0 2024-03-22T10:42:39.100 INFO:tasks.workunit.client.0.smithi115.stdout:4/304: dwrite d1/d16/d35/d40/d5c/f25 [4194304,4194304] 0 2024-03-22T10:42:39.103 INFO:tasks.workunit.client.0.smithi115.stdout:3/237: symlink d3/l46 0 2024-03-22T10:42:39.105 INFO:tasks.workunit.client.0.smithi115.stdout:6/298: dwrite d4/d11/f23 [0,4194304] 0 2024-03-22T10:42:39.106 INFO:tasks.workunit.client.0.smithi115.stdout:3/238: chown d3/f8 6 1 2024-03-22T10:42:39.108 INFO:tasks.workunit.client.0.smithi115.stdout:6/299: rename d4/d13/c16 to d4/d13/d33/c61 0 2024-03-22T10:42:39.111 INFO:tasks.workunit.client.0.smithi115.stdout:6/300: creat d4/d36/d48/f62 x:0 0 0 2024-03-22T10:42:39.112 INFO:tasks.workunit.client.0.smithi115.stdout:6/301: dread - d4/d13/f5f zero size 2024-03-22T10:42:39.115 INFO:tasks.workunit.client.0.smithi115.stdout:0/285: sync 2024-03-22T10:42:39.118 INFO:tasks.workunit.client.0.smithi115.stdout:6/302: dread d4/d13/f3d [0,4194304] 0 2024-03-22T10:42:39.118 INFO:tasks.workunit.client.0.smithi115.stdout:0/286: truncate d3/d7/d1c/f56 479104 0 2024-03-22T10:42:39.120 INFO:tasks.workunit.client.0.smithi115.stdout:6/303: write d4/f10 [3437146,33894] 0 2024-03-22T10:42:39.121 INFO:tasks.workunit.client.0.smithi115.stdout:0/287: creat d3/da/d4c/f5a x:0 0 0 2024-03-22T10:42:39.122 INFO:tasks.workunit.client.0.smithi115.stdout:6/304: symlink d4/d11/d12/d1c/l63 0 2024-03-22T10:42:39.123 INFO:tasks.workunit.client.0.smithi115.stdout:0/288: mkdir d3/d7/d1b/d1f/d5b 0 2024-03-22T10:42:39.124 INFO:tasks.workunit.client.0.smithi115.stdout:6/305: chown d4/d11/d12/d1c/f46 2737 1 2024-03-22T10:42:39.126 INFO:tasks.workunit.client.0.smithi115.stdout:6/306: mknod d4/d36/c64 0 2024-03-22T10:42:39.127 INFO:tasks.workunit.client.0.smithi115.stdout:0/289: creat d3/d7/f5c x:0 0 0 2024-03-22T10:42:39.128 INFO:tasks.workunit.client.0.smithi115.stdout:6/307: truncate d4/d36/f53 479285 0 2024-03-22T10:42:39.128 INFO:tasks.workunit.client.0.smithi115.stdout:0/290: chown d3/d7/d1c 45 1 2024-03-22T10:42:39.129 INFO:tasks.workunit.client.0.smithi115.stdout:6/308: write d4/d11/f52 [923631,67408] 0 2024-03-22T10:42:39.130 INFO:tasks.workunit.client.0.smithi115.stdout:0/291: creat d3/d7/d1c/f5d x:0 0 0 2024-03-22T10:42:39.131 INFO:tasks.workunit.client.0.smithi115.stdout:0/292: write d3/d7/d1c/f48 [1054734,16891] 0 2024-03-22T10:42:39.133 INFO:tasks.workunit.client.0.smithi115.stdout:0/293: chown d3/d4/c2b 8218 1 2024-03-22T10:42:39.134 INFO:tasks.workunit.client.0.smithi115.stdout:0/294: write d3/d7/d1c/f56 [734296,924] 0 2024-03-22T10:42:39.136 INFO:tasks.workunit.client.0.smithi115.stdout:0/295: write d3/d7/d1c/f5d [54621,97193] 0 2024-03-22T10:42:39.138 INFO:tasks.workunit.client.0.smithi115.stdout:0/296: write d3/f1a [6464873,69306] 0 2024-03-22T10:42:39.140 INFO:tasks.workunit.client.0.smithi115.stdout:0/297: creat d3/d4/d14/d54/f5e x:0 0 0 2024-03-22T10:42:39.142 INFO:tasks.workunit.client.0.smithi115.stdout:0/298: creat d3/d4/d14/d54/f5f x:0 0 0 2024-03-22T10:42:39.143 INFO:tasks.workunit.client.0.smithi115.stdout:6/309: dwrite d4/d11/f23 [0,4194304] 0 2024-03-22T10:42:39.145 INFO:tasks.workunit.client.0.smithi115.stdout:6/310: write d4/d36/f53 [1212606,65243] 0 2024-03-22T10:42:39.148 INFO:tasks.workunit.client.0.smithi115.stdout:6/311: dread d4/f24 [0,4194304] 0 2024-03-22T10:42:39.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:38 smithi150 ceph-mon[34518]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:39.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:38 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1774783633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:42:39.151 INFO:tasks.workunit.client.0.smithi115.stdout:6/312: creat d4/d11/f65 x:0 0 0 2024-03-22T10:42:39.153 INFO:tasks.workunit.client.0.smithi115.stdout:6/313: chown d4/c26 59496162 1 2024-03-22T10:42:39.155 INFO:tasks.workunit.client.0.smithi115.stdout:6/314: mknod d4/d13/d33/c66 0 2024-03-22T10:42:39.163 INFO:tasks.workunit.client.0.smithi115.stdout:2/256: sync 2024-03-22T10:42:39.164 INFO:tasks.workunit.client.0.smithi115.stdout:2/257: dread - da/d25/d30/f43 zero size 2024-03-22T10:42:39.214 INFO:tasks.workunit.client.0.smithi115.stdout:7/311: dwrite d2/f60 [0,4194304] 0 2024-03-22T10:42:39.221 INFO:tasks.workunit.client.0.smithi115.stdout:7/312: unlink d2/d7/d12/d10/d19/f37 0 2024-03-22T10:42:39.225 INFO:tasks.workunit.client.0.smithi115.stdout:1/401: dwrite d0/f14 [4194304,4194304] 0 2024-03-22T10:42:39.230 INFO:tasks.workunit.client.0.smithi115.stdout:8/471: truncate d6/f91 4178295 0 2024-03-22T10:42:39.233 INFO:tasks.workunit.client.0.smithi115.stdout:4/305: dread d1/d16/d35/d40/d5c/d14/f58 [4194304,4194304] 0 2024-03-22T10:42:39.234 INFO:tasks.workunit.client.0.smithi115.stdout:3/239: getdents d3/d19/d1b 0 2024-03-22T10:42:39.237 INFO:tasks.workunit.client.0.smithi115.stdout:3/240: fsync d3/f8 0 2024-03-22T10:42:39.237 INFO:tasks.workunit.client.0.smithi115.stdout:8/472: mknod d6/d49/d88/c98 0 2024-03-22T10:42:39.239 INFO:tasks.workunit.client.0.smithi115.stdout:4/306: dread - d1/d16/d35/d40/d5c/d55/f52 zero size 2024-03-22T10:42:39.241 INFO:tasks.workunit.client.0.smithi115.stdout:3/241: dread - d3/d36/f44 zero size 2024-03-22T10:42:39.244 INFO:tasks.workunit.client.0.smithi115.stdout:1/402: rename d0/f7 to d0/d12/d17/f80 0 2024-03-22T10:42:39.246 INFO:tasks.workunit.client.0.smithi115.stdout:0/299: rmdir d3/da/d4c 39 2024-03-22T10:42:39.247 INFO:tasks.workunit.client.0.smithi115.stdout:6/315: rmdir d4/d11/d12 39 2024-03-22T10:42:39.248 INFO:tasks.workunit.client.0.smithi115.stdout:9/341: dwrite d6/d7/d19/f25 [0,4194304] 0 2024-03-22T10:42:39.248 INFO:tasks.workunit.client.0.smithi115.stdout:8/473: mknod d6/dd/d12/d31/c99 0 2024-03-22T10:42:39.250 INFO:tasks.workunit.client.0.smithi115.stdout:4/307: write d1/d16/d35/d40/d5c/d14/f3e [3724372,47271] 0 2024-03-22T10:42:39.254 INFO:tasks.workunit.client.0.smithi115.stdout:7/313: dwrite d2/d7/d12/d10/d19/d42/f3e [0,4194304] 0 2024-03-22T10:42:39.254 INFO:tasks.workunit.client.0.smithi115.stdout:9/342: readlink d6/l10 0 2024-03-22T10:42:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:38 smithi115 ceph-mon[29111]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:42:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:38 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1774783633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:42:39.269 INFO:tasks.workunit.client.0.smithi115.stdout:5/284: dread d2/df/d13/f18 [0,4194304] 0 2024-03-22T10:42:39.273 INFO:tasks.workunit.client.0.smithi115.stdout:6/316: creat d4/d36/d48/f67 x:0 0 0 2024-03-22T10:42:39.275 INFO:tasks.workunit.client.0.smithi115.stdout:8/474: mknod d6/dd/d24/c9a 0 2024-03-22T10:42:39.279 INFO:tasks.workunit.client.0.smithi115.stdout:0/300: getdents d3/d7/d1b/d1f/d5b 0 2024-03-22T10:42:39.285 INFO:tasks.workunit.client.0.smithi115.stdout:2/258: dwrite da/d1c/f1e [0,4194304] 0 2024-03-22T10:42:39.285 INFO:tasks.workunit.client.0.smithi115.stdout:9/343: link d6/l49 d6/d8/d3c/d59/d68/l77 0 2024-03-22T10:42:39.286 INFO:tasks.workunit.client.0.smithi115.stdout:4/308: dwrite d1/d16/d35/d40/d5c/d14/f3e [0,4194304] 0 2024-03-22T10:42:39.288 INFO:tasks.workunit.client.0.smithi115.stdout:5/285: truncate d2/d15/d1b/d33/d38/f3d 773493 0 2024-03-22T10:42:39.289 INFO:tasks.workunit.client.0.smithi115.stdout:1/403: getdents d0/d12/d17 0 2024-03-22T10:42:39.291 INFO:tasks.workunit.client.0.smithi115.stdout:3/242: dwrite d3/d36/d41/f3f [0,4194304] 0 2024-03-22T10:42:39.292 INFO:tasks.workunit.client.0.smithi115.stdout:6/317: mknod d4/c68 0 2024-03-22T10:42:39.302 INFO:tasks.workunit.client.0.smithi115.stdout:3/243: dread - d3/d19/d2b/f42 zero size 2024-03-22T10:42:39.304 INFO:tasks.workunit.client.0.smithi115.stdout:8/475: read d6/f16 [278083,58607] 0 2024-03-22T10:42:39.306 INFO:tasks.workunit.client.0.smithi115.stdout:5/286: mknod d2/d15/d1b/c5a 0 2024-03-22T10:42:39.306 INFO:tasks.workunit.client.0.smithi115.stdout:6/318: write d4/d13/f3d [450949,53346] 0 2024-03-22T10:42:39.308 INFO:tasks.workunit.client.0.smithi115.stdout:3/244: dread - d3/d16/d29/f43 zero size 2024-03-22T10:42:39.312 INFO:tasks.workunit.client.0.smithi115.stdout:9/344: dread d6/d7/d19/d4e/f6d [0,4194304] 0 2024-03-22T10:42:39.317 INFO:tasks.workunit.client.0.smithi115.stdout:5/287: chown d2/df/d3b/f2a 149537556 1 2024-03-22T10:42:39.327 INFO:tasks.workunit.client.0.smithi115.stdout:9/345: creat d6/d7/f78 x:0 0 0 2024-03-22T10:42:39.328 INFO:tasks.workunit.client.0.smithi115.stdout:6/319: rename d4/d36/f49 to d4/d11/d12/d1c/f69 0 2024-03-22T10:42:39.334 INFO:tasks.workunit.client.0.smithi115.stdout:1/404: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:42:39.334 INFO:tasks.workunit.client.0.smithi115.stdout:2/259: dwrite da/f1b [0,4194304] 0 2024-03-22T10:42:39.337 INFO:tasks.workunit.client.0.smithi115.stdout:5/288: creat d2/d15/d1b/d33/d38/f5b x:0 0 0 2024-03-22T10:42:39.338 INFO:tasks.workunit.client.0.smithi115.stdout:9/346: readlink d6/d7/d19/l36 0 2024-03-22T10:42:39.343 INFO:tasks.workunit.client.0.smithi115.stdout:7/314: dread d2/d62/f4f [0,4194304] 0 2024-03-22T10:42:39.352 INFO:tasks.workunit.client.0.smithi115.stdout:2/260: write da/f14 [1203853,32121] 0 2024-03-22T10:42:39.354 INFO:tasks.workunit.client.0.smithi115.stdout:1/405: mkdir d0/d68/d81 0 2024-03-22T10:42:39.355 INFO:tasks.workunit.client.0.smithi115.stdout:8/476: dwrite d6/dd/f76 [0,4194304] 0 2024-03-22T10:42:39.355 INFO:tasks.workunit.client.0.smithi115.stdout:0/301: dwrite d3/d7/d1b/f2a [0,4194304] 0 2024-03-22T10:42:39.363 INFO:tasks.workunit.client.0.smithi115.stdout:8/477: readlink d6/dd/d12/l2b 0 2024-03-22T10:42:39.365 INFO:tasks.workunit.client.0.smithi115.stdout:3/245: dwrite d3/d19/d1b/f22 [0,4194304] 0 2024-03-22T10:42:39.366 INFO:tasks.workunit.client.0.smithi115.stdout:5/289: symlink d2/d15/l5c 0 2024-03-22T10:42:39.369 INFO:tasks.workunit.client.0.smithi115.stdout:2/261: creat da/d1c/f4c x:0 0 0 2024-03-22T10:42:39.374 INFO:tasks.workunit.client.0.smithi115.stdout:9/347: dwrite d6/d7/d20/f57 [0,4194304] 0 2024-03-22T10:42:39.378 INFO:tasks.workunit.client.0.smithi115.stdout:6/320: getdents d4/d11/d12 0 2024-03-22T10:42:39.382 INFO:tasks.workunit.client.0.smithi115.stdout:5/290: unlink d2/d15/d1b/d33/d38/f3a 0 2024-03-22T10:42:39.385 INFO:tasks.workunit.client.0.smithi115.stdout:0/302: dwrite d3/f1a [4194304,4194304] 0 2024-03-22T10:42:39.389 INFO:tasks.workunit.client.0.smithi115.stdout:3/246: truncate d3/d16/d29/f43 292053 0 2024-03-22T10:42:39.391 INFO:tasks.workunit.client.0.smithi115.stdout:6/321: dread d4/d13/f3d [0,4194304] 0 2024-03-22T10:42:39.394 INFO:tasks.workunit.client.0.smithi115.stdout:3/247: write d3/d19/d2b/f40 [803128,5222] 0 2024-03-22T10:42:39.394 INFO:tasks.workunit.client.0.smithi115.stdout:1/406: mkdir d0/d22/d75/d82 0 2024-03-22T10:42:39.395 INFO:tasks.workunit.client.0.smithi115.stdout:9/348: mknod d6/d7/d48/d70/c79 0 2024-03-22T10:42:39.398 INFO:tasks.workunit.client.0.smithi115.stdout:8/478: dwrite d6/dd/d24/d4a/d41/f4b [0,4194304] 0 2024-03-22T10:42:39.400 INFO:tasks.workunit.client.0.smithi115.stdout:5/291: unlink d2/df/d13/f23 0 2024-03-22T10:42:39.401 INFO:tasks.workunit.client.0.smithi115.stdout:7/315: link d2/d7/d12/d10/d23/d3d/f21 d2/d7/d12/d10/d23/d3d/d4a/f6a 0 2024-03-22T10:42:39.404 INFO:tasks.workunit.client.0.smithi115.stdout:0/303: chown d3/d4/c2c 275423 1 2024-03-22T10:42:39.408 INFO:tasks.workunit.client.0.smithi115.stdout:6/322: creat d4/d11/d12/d1c/f6a x:0 0 0 2024-03-22T10:42:39.408 INFO:tasks.workunit.client.0.smithi115.stdout:3/248: unlink d3/d19/d2b/f40 0 2024-03-22T10:42:39.410 INFO:tasks.workunit.client.0.smithi115.stdout:0/304: truncate d3/d7/f3b 898551 0 2024-03-22T10:42:39.413 INFO:tasks.workunit.client.0.smithi115.stdout:2/262: link da/d25/d2a/l32 da/d25/d2a/l4d 0 2024-03-22T10:42:39.414 INFO:tasks.workunit.client.0.smithi115.stdout:0/305: chown d3/d7/d1b/f24 6307 1 2024-03-22T10:42:39.415 INFO:tasks.workunit.client.0.smithi115.stdout:5/292: rmdir d2/df/d27 39 2024-03-22T10:42:39.422 INFO:tasks.workunit.client.0.smithi115.stdout:1/407: rename d0/d12/d2b/f59 to d0/d12/d17/d37/f83 0 2024-03-22T10:42:39.429 INFO:tasks.workunit.client.0.smithi115.stdout:2/263: creat da/d25/d3d/f4e x:0 0 0 2024-03-22T10:42:39.430 INFO:tasks.workunit.client.0.smithi115.stdout:3/249: dread d3/f12 [0,4194304] 0 2024-03-22T10:42:39.431 INFO:tasks.workunit.client.0.smithi115.stdout:6/323: rename d4/c9 to d4/d5e/c6b 0 2024-03-22T10:42:39.433 INFO:tasks.workunit.client.0.smithi115.stdout:9/349: getdents d6/d7/d20 0 2024-03-22T10:42:39.438 INFO:tasks.workunit.client.0.smithi115.stdout:1/408: creat d0/d8/db/f84 x:0 0 0 2024-03-22T10:42:39.439 INFO:tasks.workunit.client.0.smithi115.stdout:0/306: creat d3/d4/f60 x:0 0 0 2024-03-22T10:42:39.444 INFO:tasks.workunit.client.0.smithi115.stdout:8/479: truncate d6/dd/f6a 363985 0 2024-03-22T10:42:39.444 INFO:tasks.workunit.client.0.smithi115.stdout:3/250: creat d3/d19/d2b/f47 x:0 0 0 2024-03-22T10:42:39.445 INFO:tasks.workunit.client.0.smithi115.stdout:6/324: creat d4/d13/d33/f6c x:0 0 0 2024-03-22T10:42:39.459 INFO:tasks.workunit.client.0.smithi115.stdout:3/251: rename d3/d19/f1a to d3/d36/d41/f48 0 2024-03-22T10:42:39.461 INFO:tasks.workunit.client.0.smithi115.stdout:9/350: dread d6/d8/f47 [0,4194304] 0 2024-03-22T10:42:39.462 INFO:tasks.workunit.client.0.smithi115.stdout:2/264: dwrite da/d1c/f48 [0,4194304] 0 2024-03-22T10:42:39.465 INFO:tasks.workunit.client.0.smithi115.stdout:5/293: dread d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:39.473 INFO:tasks.workunit.client.0.smithi115.stdout:9/351: rmdir d6/d7/d20/d22/d23 39 2024-03-22T10:42:39.473 INFO:tasks.workunit.client.0.smithi115.stdout:2/265: write da/d25/f49 [538940,18405] 0 2024-03-22T10:42:39.479 INFO:tasks.workunit.client.0.smithi115.stdout:9/352: rmdir d6/d7/d20/d38 39 2024-03-22T10:42:39.481 INFO:tasks.workunit.client.0.smithi115.stdout:2/266: write da/f1b [124243,67530] 0 2024-03-22T10:42:39.481 INFO:tasks.workunit.client.0.smithi115.stdout:6/325: dwrite d4/d11/f52 [0,4194304] 0 2024-03-22T10:42:39.495 INFO:tasks.workunit.client.0.smithi115.stdout:8/480: dwrite d6/d49/d68/f6f [0,4194304] 0 2024-03-22T10:42:39.506 INFO:tasks.workunit.client.0.smithi115.stdout:0/307: dread d3/d7/f45 [0,4194304] 0 2024-03-22T10:42:39.506 INFO:tasks.workunit.client.0.smithi115.stdout:5/294: creat d2/d15/d1b/d33/d38/d4f/f5d x:0 0 0 2024-03-22T10:42:39.509 INFO:tasks.workunit.client.0.smithi115.stdout:3/252: truncate d3/d19/f31 82328 0 2024-03-22T10:42:39.509 INFO:tasks.workunit.client.0.smithi115.stdout:9/353: creat d6/d7/d19/f7a x:0 0 0 2024-03-22T10:42:39.513 INFO:tasks.workunit.client.0.smithi115.stdout:2/267: mkdir da/d25/d30/d4f 0 2024-03-22T10:42:39.515 INFO:tasks.workunit.client.0.smithi115.stdout:2/268: chown da/d25/d2a/c38 14016 1 2024-03-22T10:42:39.518 INFO:tasks.workunit.client.0.smithi115.stdout:0/308: mknod d3/d52/c61 0 2024-03-22T10:42:39.519 INFO:tasks.workunit.client.0.smithi115.stdout:3/253: mkdir d3/d19/d2b/d49 0 2024-03-22T10:42:39.521 INFO:tasks.workunit.client.0.smithi115.stdout:0/309: dread d3/d7/f3b [0,4194304] 0 2024-03-22T10:42:39.523 INFO:tasks.workunit.client.0.smithi115.stdout:9/354: rename d6/c5a to d6/d7/d48/c7b 0 2024-03-22T10:42:39.523 INFO:tasks.workunit.client.0.smithi115.stdout:0/310: chown d3/d4/c2c 528648056 1 2024-03-22T10:42:39.529 INFO:tasks.workunit.client.0.smithi115.stdout:0/311: write d3/d4/d14/f4f [1645846,102410] 0 2024-03-22T10:42:39.534 INFO:tasks.workunit.client.0.smithi115.stdout:8/481: mknod d6/c9b 0 2024-03-22T10:42:39.535 INFO:tasks.workunit.client.0.smithi115.stdout:3/254: mknod d3/d36/c4a 0 2024-03-22T10:42:39.536 INFO:tasks.workunit.client.0.smithi115.stdout:6/326: link d4/d36/l4a d4/d5e/l6d 0 2024-03-22T10:42:39.539 INFO:tasks.workunit.client.0.smithi115.stdout:0/312: creat d3/d7/d1b/d1f/f62 x:0 0 0 2024-03-22T10:42:39.540 INFO:tasks.workunit.client.0.smithi115.stdout:8/482: truncate d6/dd/d24/d4a/f80 4977624 0 2024-03-22T10:42:39.542 INFO:tasks.workunit.client.0.smithi115.stdout:8/483: chown d6/f59 6523 1 2024-03-22T10:42:39.542 INFO:tasks.workunit.client.0.smithi115.stdout:5/295: creat d2/f5e x:0 0 0 2024-03-22T10:42:39.547 INFO:tasks.workunit.client.0.smithi115.stdout:2/269: creat da/d25/d30/d4f/f50 x:0 0 0 2024-03-22T10:42:39.548 INFO:tasks.workunit.client.0.smithi115.stdout:8/484: readlink d6/dd/d12/l78 0 2024-03-22T10:42:39.548 INFO:tasks.workunit.client.0.smithi115.stdout:3/255: rmdir d3/d36/d41 39 2024-03-22T10:42:39.548 INFO:tasks.workunit.client.0.smithi115.stdout:2/270: fdatasync da/d25/d3d/f3c 0 2024-03-22T10:42:39.548 INFO:tasks.workunit.client.0.smithi115.stdout:0/313: mknod d3/d7/d1b/d1f/c63 0 2024-03-22T10:42:39.549 INFO:tasks.workunit.client.0.smithi115.stdout:6/327: unlink d4/d11/d12/d1c/l63 0 2024-03-22T10:42:39.549 INFO:tasks.workunit.client.0.smithi115.stdout:2/271: mkdir da/d25/d51 0 2024-03-22T10:42:39.550 INFO:tasks.workunit.client.0.smithi115.stdout:8/485: creat d6/d49/d68/f9c x:0 0 0 2024-03-22T10:42:39.551 INFO:tasks.workunit.client.0.smithi115.stdout:9/355: dwrite d6/d7/d30/d37/f4d [0,4194304] 0 2024-03-22T10:42:39.551 INFO:tasks.workunit.client.0.smithi115.stdout:0/314: mkdir d3/d4/d14/d64 0 2024-03-22T10:42:39.553 INFO:tasks.workunit.client.0.smithi115.stdout:6/328: readlink d4/le 0 2024-03-22T10:42:39.554 INFO:tasks.workunit.client.0.smithi115.stdout:8/486: dread - d6/dd/d24/d4a/f8d zero size 2024-03-22T10:42:39.556 INFO:tasks.workunit.client.0.smithi115.stdout:2/272: creat da/d1c/f52 x:0 0 0 2024-03-22T10:42:39.560 INFO:tasks.workunit.client.0.smithi115.stdout:5/296: rename d2/d15/d1b/d33/l48 to d2/d15/d1b/d33/d38/d4a/l5f 0 2024-03-22T10:42:39.561 INFO:tasks.workunit.client.0.smithi115.stdout:6/329: rmdir d4/d13 39 2024-03-22T10:42:39.566 INFO:tasks.workunit.client.0.smithi115.stdout:0/315: symlink d3/d4/l65 0 2024-03-22T10:42:39.567 INFO:tasks.workunit.client.1.smithi150.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-22T10:42:39.568 INFO:tasks.workunit.client.0.smithi115.stdout:5/297: fsync d2/df/d3b/f36 0 2024-03-22T10:42:39.570 INFO:tasks.workunit.client.0.smithi115.stdout:3/256: link d3/d36/c4a d3/d36/d41/d33/c4b 0 2024-03-22T10:42:39.571 INFO:tasks.workunit.client.0.smithi115.stdout:5/298: readlink d2/df/l26 0 2024-03-22T10:42:39.571 INFO:tasks.workunit.client.0.smithi115.stdout:6/330: write d4/d13/d33/f59 [846068,87338] 0 2024-03-22T10:42:39.573 INFO:tasks.workunit.client.0.smithi115.stdout:5/299: readlink d2/df/d3b/l31 0 2024-03-22T10:42:39.574 INFO:tasks.workunit.client.0.smithi115.stdout:0/316: creat d3/d7/d1b/d1f/d5b/f66 x:0 0 0 2024-03-22T10:42:39.578 INFO:tasks.workunit.client.0.smithi115.stdout:8/487: dwrite d6/dd/d24/d4a/f80 [0,4194304] 0 2024-03-22T10:42:39.581 INFO:tasks.workunit.client.0.smithi115.stdout:9/356: getdents d6/d7/d19 0 2024-03-22T10:42:39.582 INFO:tasks.workunit.client.0.smithi115.stdout:6/331: unlink d4/d5e/l6d 0 2024-03-22T10:42:39.583 INFO:tasks.workunit.client.0.smithi115.stdout:8/488: write d6/dd/d12/f94 [557576,93334] 0 2024-03-22T10:42:39.583 INFO:tasks.workunit.client.0.smithi115.stdout:9/357: fsync d6/d7/d48/f50 0 2024-03-22T10:42:39.585 INFO:tasks.workunit.client.0.smithi115.stdout:8/489: fsync d6/d49/f83 0 2024-03-22T10:42:39.588 INFO:tasks.workunit.client.0.smithi115.stdout:8/490: write d6/dd/f29 [1653096,102214] 0 2024-03-22T10:42:39.588 INFO:tasks.workunit.client.0.smithi115.stdout:9/358: chown d6/d7/d48/d70/c79 686310 1 2024-03-22T10:42:39.590 INFO:tasks.workunit.client.0.smithi115.stdout:0/317: creat d3/da/d4c/f67 x:0 0 0 2024-03-22T10:42:39.593 INFO:tasks.workunit.client.0.smithi115.stdout:9/359: read d6/d7/d19/d4e/f6d [3601,661] 0 2024-03-22T10:42:39.600 INFO:tasks.workunit.client.0.smithi115.stdout:0/318: rename d3/d4/d14/d54 to d3/da/d4c/d68 0 2024-03-22T10:42:39.603 INFO:tasks.workunit.client.0.smithi115.stdout:3/257: getdents d3/d36/d41 0 2024-03-22T10:42:39.603 INFO:tasks.workunit.client.0.smithi115.stdout:2/273: rmdir da/d25/d30/d4f 39 2024-03-22T10:42:39.604 INFO:tasks.workunit.client.0.smithi115.stdout:2/274: chown da/d25/f37 225607927 1 2024-03-22T10:42:39.607 INFO:tasks.workunit.client.0.smithi115.stdout:5/300: link d2/df/l47 d2/df/d13/l60 0 2024-03-22T10:42:39.611 INFO:tasks.workunit.client.0.smithi115.stdout:2/275: mknod da/d1c/c53 0 2024-03-22T10:42:39.613 INFO:tasks.workunit.client.0.smithi115.stdout:8/491: creat d6/dd/d24/f9d x:0 0 0 2024-03-22T10:42:39.615 INFO:tasks.workunit.client.0.smithi115.stdout:2/276: truncate da/d1c/f52 736731 0 2024-03-22T10:42:39.616 INFO:tasks.workunit.client.0.smithi115.stdout:5/301: creat d2/d15/d1b/d33/d38/f61 x:0 0 0 2024-03-22T10:42:39.618 INFO:tasks.workunit.client.0.smithi115.stdout:3/258: creat d3/d16/d29/d35/f4c x:0 0 0 2024-03-22T10:42:39.619 INFO:tasks.workunit.client.0.smithi115.stdout:3/259: chown d3/d36/d41/c37 12033190 1 2024-03-22T10:42:39.624 INFO:tasks.workunit.client.0.smithi115.stdout:0/319: dwrite d3/da/d4c/f5a [0,4194304] 0 2024-03-22T10:42:39.625 INFO:tasks.workunit.client.0.smithi115.stdout:9/360: link d6/l49 d6/d7/d6a/l7c 0 2024-03-22T10:42:39.627 INFO:tasks.workunit.client.0.smithi115.stdout:5/302: mknod d2/c62 0 2024-03-22T10:42:39.628 INFO:tasks.workunit.client.0.smithi115.stdout:9/361: chown d6/f12 6811 1 2024-03-22T10:42:39.628 INFO:tasks.workunit.client.0.smithi115.stdout:8/492: symlink d6/dd/d12/l9e 0 2024-03-22T10:42:39.630 INFO:tasks.workunit.client.0.smithi115.stdout:3/260: creat d3/d19/d2b/f4d x:0 0 0 2024-03-22T10:42:39.631 INFO:tasks.workunit.client.0.smithi115.stdout:6/332: write d4/d11/f2a [669325,57405] 0 2024-03-22T10:42:39.638 INFO:tasks.workunit.client.0.smithi115.stdout:9/362: write d6/d7/d48/d70/f73 [356734,26075] 0 2024-03-22T10:42:39.641 INFO:tasks.workunit.client.0.smithi115.stdout:5/303: creat d2/d15/d1b/d33/f63 x:0 0 0 2024-03-22T10:42:39.643 INFO:tasks.workunit.client.0.smithi115.stdout:3/261: creat d3/d19/f4e x:0 0 0 2024-03-22T10:42:39.651 INFO:tasks.workunit.client.0.smithi115.stdout:9/363: symlink d6/d7/d6a/l7d 0 2024-03-22T10:42:39.652 INFO:tasks.workunit.client.0.smithi115.stdout:6/333: rename d4/d13/c56 to d4/d11/c6e 0 2024-03-22T10:42:39.653 INFO:tasks.workunit.client.0.smithi115.stdout:2/277: dwrite da/d25/d30/d4f/f50 [0,4194304] 0 2024-03-22T10:42:39.655 INFO:tasks.workunit.client.0.smithi115.stdout:3/262: unlink d3/d19/d1b/l34 0 2024-03-22T10:42:39.660 INFO:tasks.workunit.client.0.smithi115.stdout:8/493: dwrite d6/f28 [0,4194304] 0 2024-03-22T10:42:39.661 INFO:tasks.workunit.client.0.smithi115.stdout:6/334: mknod d4/d13/c6f 0 2024-03-22T10:42:39.664 INFO:tasks.workunit.client.0.smithi115.stdout:0/320: dwrite d3/d7/d1b/d1f/d5b/f66 [0,4194304] 0 2024-03-22T10:42:39.666 INFO:tasks.workunit.client.0.smithi115.stdout:5/304: write d2/df/d13/f18 [2929995,128735] 0 2024-03-22T10:42:39.666 INFO:tasks.workunit.client.0.smithi115.stdout:9/364: symlink d6/d8/l7e 0 2024-03-22T10:42:39.668 INFO:tasks.workunit.client.0.smithi115.stdout:6/335: write d4/d11/f23 [2359809,25479] 0 2024-03-22T10:42:39.671 INFO:tasks.workunit.client.0.smithi115.stdout:2/278: mkdir da/d25/d30/d4f/d54 0 2024-03-22T10:42:39.674 INFO:tasks.workunit.client.0.smithi115.stdout:8/494: fdatasync d6/dd/d24/f72 0 2024-03-22T10:42:39.676 INFO:tasks.workunit.client.0.smithi115.stdout:9/365: write d6/d7/f78 [192883,20644] 0 2024-03-22T10:42:39.682 INFO:tasks.workunit.client.0.smithi115.stdout:6/336: rename d4/f5 to d4/d13/f70 0 2024-03-22T10:42:39.683 INFO:tasks.workunit.client.0.smithi115.stdout:2/279: creat da/d25/d51/f55 x:0 0 0 2024-03-22T10:42:39.688 INFO:tasks.workunit.client.0.smithi115.stdout:4/309: sync 2024-03-22T10:42:39.688 INFO:tasks.workunit.client.0.smithi115.stdout:8/495: creat d6/d49/d68/f9f x:0 0 0 2024-03-22T10:42:39.693 INFO:tasks.workunit.client.0.smithi115.stdout:6/337: mknod d4/d36/c71 0 2024-03-22T10:42:39.693 INFO:tasks.workunit.client.0.smithi115.stdout:9/366: creat d6/d8/d3c/d59/d68/f7f x:0 0 0 2024-03-22T10:42:39.696 INFO:tasks.workunit.client.0.smithi115.stdout:3/263: dwrite d3/d19/f4e [0,4194304] 0 2024-03-22T10:42:39.698 INFO:tasks.workunit.client.0.smithi115.stdout:4/310: dread - d1/d16/d35/d40/d5c/d14/d33/f62 zero size 2024-03-22T10:42:39.700 INFO:tasks.workunit.client.0.smithi115.stdout:5/305: rename d2/f11 to d2/d15/d1b/d33/d38/d4f/f64 0 2024-03-22T10:42:39.701 INFO:tasks.workunit.client.0.smithi115.stdout:5/306: dread - d2/f5e zero size 2024-03-22T10:42:39.702 INFO:tasks.workunit.client.0.smithi115.stdout:0/321: dwrite d3/d7/f5c [0,4194304] 0 2024-03-22T10:42:39.705 INFO:tasks.workunit.client.0.smithi115.stdout:2/280: link da/d25/d3d/f3c da/d25/d51/f56 0 2024-03-22T10:42:39.707 INFO:tasks.workunit.client.0.smithi115.stdout:9/367: chown d6/d7/d30/d3b/f58 1612696337 1 2024-03-22T10:42:39.709 INFO:tasks.workunit.client.0.smithi115.stdout:8/496: mkdir d6/dd/d24/d4a/d7b/d8f/da0 0 2024-03-22T10:42:39.713 INFO:tasks.workunit.client.0.smithi115.stdout:3/264: mknod d3/d36/c4f 0 2024-03-22T10:42:39.717 INFO:tasks.workunit.client.0.smithi115.stdout:0/322: mkdir d3/da/d4c/d69 0 2024-03-22T10:42:39.718 INFO:tasks.workunit.client.0.smithi115.stdout:9/368: rename d6/d7/d19/l1f to d6/d8/l80 0 2024-03-22T10:42:39.719 INFO:tasks.workunit.client.0.smithi115.stdout:4/311: creat d1/d16/f64 x:0 0 0 2024-03-22T10:42:39.721 INFO:tasks.workunit.client.0.smithi115.stdout:2/281: dread da/d25/f49 [0,4194304] 0 2024-03-22T10:42:39.722 INFO:tasks.workunit.client.0.smithi115.stdout:3/265: write d3/d19/f1d [670209,25939] 0 2024-03-22T10:42:39.726 INFO:tasks.workunit.client.0.smithi115.stdout:0/323: mkdir d3/da/d4c/d69/d6a 0 2024-03-22T10:42:39.727 INFO:tasks.workunit.client.0.smithi115.stdout:9/369: mknod d6/d8/d3c/d59/d68/c81 0 2024-03-22T10:42:39.727 INFO:tasks.workunit.client.0.smithi115.stdout:3/266: dread - d3/d19/d2b/f42 zero size 2024-03-22T10:42:39.731 INFO:tasks.workunit.client.0.smithi115.stdout:4/312: creat d1/d16/d35/d40/d5c/d14/d33/f65 x:0 0 0 2024-03-22T10:42:39.731 INFO:tasks.workunit.client.0.smithi115.stdout:3/267: truncate d3/d36/f44 378479 0 2024-03-22T10:42:39.731 INFO:tasks.workunit.client.0.smithi115.stdout:0/324: mkdir d3/d7/d6b 0 2024-03-22T10:42:39.733 INFO:tasks.workunit.client.0.smithi115.stdout:4/313: readlink d1/d16/d35/d40/d5c/d14/l2c 0 2024-03-22T10:42:39.734 INFO:tasks.workunit.client.0.smithi115.stdout:3/268: creat d3/f50 x:0 0 0 2024-03-22T10:42:39.737 INFO:tasks.workunit.client.0.smithi115.stdout:3/269: read d3/d16/d29/f39 [2934417,58037] 0 2024-03-22T10:42:39.737 INFO:tasks.workunit.client.0.smithi115.stdout:3/270: fsync d3/d19/d2b/f4d 0 2024-03-22T10:42:39.737 INFO:tasks.workunit.client.0.smithi115.stdout:0/325: mkdir d3/d7/d1c/d6c 0 2024-03-22T10:42:39.740 INFO:tasks.workunit.client.0.smithi115.stdout:9/370: rename d6/d7/d48/c7b to d6/d7/d19/d4e/c82 0 2024-03-22T10:42:39.742 INFO:tasks.workunit.client.0.smithi115.stdout:4/314: mkdir d1/d66 0 2024-03-22T10:42:39.743 INFO:tasks.workunit.client.0.smithi115.stdout:4/315: readlink d1/l4 0 2024-03-22T10:42:39.746 INFO:tasks.workunit.client.0.smithi115.stdout:5/307: truncate d2/d15/d1b/d33/d38/f3d 25332 0 2024-03-22T10:42:39.747 INFO:tasks.workunit.client.0.smithi115.stdout:8/497: truncate d6/dd/d12/f94 460877 0 2024-03-22T10:42:39.749 INFO:tasks.workunit.client.0.smithi115.stdout:4/316: creat d1/d16/d35/d40/d54/f67 x:0 0 0 2024-03-22T10:42:39.750 INFO:tasks.workunit.client.0.smithi115.stdout:0/326: mkdir d3/d4/d6d 0 2024-03-22T10:42:39.755 INFO:tasks.workunit.client.0.smithi115.stdout:8/498: chown d6/dd/d24/d4a/c96 5 1 2024-03-22T10:42:39.756 INFO:tasks.workunit.client.0.smithi115.stdout:3/271: creat d3/d36/f51 x:0 0 0 2024-03-22T10:42:39.757 INFO:tasks.workunit.client.0.smithi115.stdout:5/308: truncate d2/df/d3b/f36 2019036 0 2024-03-22T10:42:39.760 INFO:tasks.workunit.client.0.smithi115.stdout:8/499: read d6/fc [1179709,24655] 0 2024-03-22T10:42:39.760 INFO:tasks.workunit.client.0.smithi115.stdout:9/371: creat d6/d7/d20/d22/d23/f83 x:0 0 0 2024-03-22T10:42:39.761 INFO:tasks.workunit.client.0.smithi115.stdout:5/309: read d2/df/f10 [1798479,118509] 0 2024-03-22T10:42:39.765 INFO:tasks.workunit.client.0.smithi115.stdout:9/372: fsync d6/d7/d20/f57 0 2024-03-22T10:42:39.768 INFO:tasks.workunit.client.0.smithi115.stdout:5/310: creat d2/d51/f65 x:0 0 0 2024-03-22T10:42:39.769 INFO:tasks.workunit.client.0.smithi115.stdout:8/500: mknod d6/dd/d12/ca1 0 2024-03-22T10:42:39.777 INFO:tasks.workunit.client.0.smithi115.stdout:5/311: symlink d2/df/d13/l66 0 2024-03-22T10:42:39.782 INFO:tasks.workunit.client.0.smithi115.stdout:8/501: rmdir d6/dd/d24/d4a/d7b/d8f 39 2024-03-22T10:42:39.784 INFO:tasks.workunit.client.0.smithi115.stdout:4/317: dwrite d1/d16/d35/d40/d5c/d14/d33/f65 [0,4194304] 0 2024-03-22T10:42:39.787 INFO:tasks.workunit.client.0.smithi115.stdout:5/312: unlink d2/d15/d1b/d33/d38/d4f/f5d 0 2024-03-22T10:42:39.791 INFO:tasks.workunit.client.0.smithi115.stdout:4/318: mkdir d1/d16/d68 0 2024-03-22T10:42:39.791 INFO:tasks.workunit.client.0.smithi115.stdout:0/327: dwrite d3/d7/d1b/d1f/d5b/f66 [0,4194304] 0 2024-03-22T10:42:39.792 INFO:tasks.workunit.client.0.smithi115.stdout:3/272: dread d3/f8 [12582912,4194304] 0 2024-03-22T10:42:39.792 INFO:tasks.workunit.client.0.smithi115.stdout:5/313: write d2/df/f10 [5112470,47264] 0 2024-03-22T10:42:39.798 INFO:tasks.workunit.client.0.smithi115.stdout:3/273: truncate d3/d16/d29/f3c 698975 0 2024-03-22T10:42:39.799 INFO:tasks.workunit.client.0.smithi115.stdout:9/373: dwrite d6/d7/d19/d4e/f6d [0,4194304] 0 2024-03-22T10:42:39.801 INFO:tasks.workunit.client.0.smithi115.stdout:9/374: chown d6/d7/d6a/d6f 6470 1 2024-03-22T10:42:39.803 INFO:tasks.workunit.client.0.smithi115.stdout:4/319: rename d1/d16/d35/d40/d5c/d14/d33 to d1/d69 0 2024-03-22T10:42:39.806 INFO:tasks.workunit.client.0.smithi115.stdout:9/375: dread d6/d5e/f5f [0,4194304] 0 2024-03-22T10:42:39.807 INFO:tasks.workunit.client.0.smithi115.stdout:3/274: mkdir d3/d19/d1b/d52 0 2024-03-22T10:42:39.813 INFO:tasks.workunit.client.0.smithi115.stdout:9/376: chown d6/d7/d19/d4e/l69 1435 1 2024-03-22T10:42:39.816 INFO:tasks.workunit.client.0.smithi115.stdout:4/320: rename d1/c2d to d1/d16/d28/c6a 0 2024-03-22T10:42:39.821 INFO:tasks.workunit.client.0.smithi115.stdout:3/275: rmdir d3/d16 39 2024-03-22T10:42:39.821 INFO:tasks.workunit.client.0.smithi115.stdout:0/328: dwrite d3/f46 [0,4194304] 0 2024-03-22T10:42:39.825 INFO:tasks.workunit.client.0.smithi115.stdout:3/276: write d3/d36/d41/f3f [3253163,88441] 0 2024-03-22T10:42:39.827 INFO:tasks.workunit.client.0.smithi115.stdout:5/314: dwrite d2/d15/f39 [0,4194304] 0 2024-03-22T10:42:39.827 INFO:tasks.workunit.client.0.smithi115.stdout:7/316: sync 2024-03-22T10:42:39.827 INFO:tasks.workunit.client.0.smithi115.stdout:1/409: sync 2024-03-22T10:42:39.837 INFO:tasks.workunit.client.0.smithi115.stdout:0/329: creat d3/d52/f6e x:0 0 0 2024-03-22T10:42:39.839 INFO:tasks.workunit.client.0.smithi115.stdout:7/317: mknod d2/c6b 0 2024-03-22T10:42:39.839 INFO:tasks.workunit.client.0.smithi115.stdout:9/377: unlink d6/d7/d20/d22/d23/l43 0 2024-03-22T10:42:39.841 INFO:tasks.workunit.client.0.smithi115.stdout:4/321: mkdir d1/d16/d35/d40/d5c/d6b 0 2024-03-22T10:42:39.842 INFO:tasks.workunit.client.0.smithi115.stdout:1/410: chown d0/d8/d4d/d56/f57 1445026 1 2024-03-22T10:42:39.846 INFO:tasks.workunit.client.0.smithi115.stdout:5/315: truncate d2/d15/d1b/d33/d38/f61 722662 0 2024-03-22T10:42:39.846 INFO:tasks.workunit.client.0.smithi115.stdout:0/330: write d3/f46 [101209,56398] 0 2024-03-22T10:42:39.848 INFO:tasks.workunit.client.0.smithi115.stdout:7/318: write d2/d7/d12/d10/d23/f64 [578764,59237] 0 2024-03-22T10:42:39.852 INFO:tasks.workunit.client.0.smithi115.stdout:3/277: link d3/d36/f51 d3/f53 0 2024-03-22T10:42:39.853 INFO:tasks.workunit.client.0.smithi115.stdout:4/322: creat d1/d16/d68/f6c x:0 0 0 2024-03-22T10:42:39.853 INFO:tasks.workunit.client.0.smithi115.stdout:9/378: symlink d6/d7/d20/d38/l84 0 2024-03-22T10:42:39.857 INFO:tasks.workunit.client.0.smithi115.stdout:9/379: fsync d6/d7/d19/f27 0 2024-03-22T10:42:39.859 INFO:tasks.workunit.client.0.smithi115.stdout:3/278: mkdir d3/d19/d1b/d54 0 2024-03-22T10:42:39.861 INFO:tasks.workunit.client.0.smithi115.stdout:8/502: dread d6/f91 [0,4194304] 0 2024-03-22T10:42:39.868 INFO:tasks.workunit.client.0.smithi115.stdout:3/279: unlink d3/f8 0 2024-03-22T10:42:39.869 INFO:tasks.workunit.client.0.smithi115.stdout:1/411: dwrite d0/d12/d17/f33 [0,4194304] 0 2024-03-22T10:42:39.871 INFO:tasks.workunit.client.0.smithi115.stdout:3/280: readlink d3/l46 0 2024-03-22T10:42:39.874 INFO:tasks.workunit.client.0.smithi115.stdout:4/323: link d1/d16/f26 d1/d66/f6d 0 2024-03-22T10:42:39.878 INFO:tasks.workunit.client.0.smithi115.stdout:7/319: rename d2/d7/d12/l1c to d2/d7/d12/d10/d19/l6c 0 2024-03-22T10:42:39.881 INFO:tasks.workunit.client.0.smithi115.stdout:1/412: mknod d0/d8/d4d/d56/c85 0 2024-03-22T10:42:39.882 INFO:tasks.workunit.client.0.smithi115.stdout:0/331: truncate d3/d7/d1c/f5d 64860 0 2024-03-22T10:42:39.886 INFO:tasks.workunit.client.0.smithi115.stdout:5/316: dread d2/df/d3b/f2a [0,4194304] 0 2024-03-22T10:42:39.888 INFO:tasks.workunit.client.0.smithi115.stdout:0/332: dread - d3/da/d4c/d68/f5f zero size 2024-03-22T10:42:39.889 INFO:tasks.workunit.client.0.smithi115.stdout:8/503: rename d6/f20 to d6/dd/d24/d4a/d7b/fa2 0 2024-03-22T10:42:39.891 INFO:tasks.workunit.client.0.smithi115.stdout:9/380: dwrite d6/d7/d20/d38/f67 [0,4194304] 0 2024-03-22T10:42:39.891 INFO:tasks.workunit.client.0.smithi115.stdout:5/317: read d2/df/d13/f18 [3596984,14971] 0 2024-03-22T10:42:39.896 INFO:tasks.workunit.client.0.smithi115.stdout:8/504: chown d6/dd/d24/d3a/c93 124 1 2024-03-22T10:42:39.897 INFO:tasks.workunit.client.0.smithi115.stdout:1/413: link d0/f55 d0/d22/f86 0 2024-03-22T10:42:39.898 INFO:tasks.workunit.client.0.smithi115.stdout:0/333: chown d3/d4/d14/f15 3724 1 2024-03-22T10:42:39.899 INFO:tasks.workunit.client.0.smithi115.stdout:8/505: stat d6/dd/d12/c14 0 2024-03-22T10:42:39.902 INFO:tasks.workunit.client.0.smithi115.stdout:0/334: dread - d3/d7/d1b/d1f/f58 zero size 2024-03-22T10:42:39.904 INFO:tasks.workunit.client.0.smithi115.stdout:7/320: rename d2/d7/d12/d10/d23/f4e to d2/d7/d12/d10/d19/d42/f6d 0 2024-03-22T10:42:39.906 INFO:tasks.workunit.client.0.smithi115.stdout:3/281: write d3/d16/d29/f39 [769734,8312] 0 2024-03-22T10:42:39.907 INFO:tasks.workunit.client.0.smithi115.stdout:4/324: dwrite d1/d30/d42/d32/f53 [0,4194304] 0 2024-03-22T10:42:39.911 INFO:tasks.workunit.client.0.smithi115.stdout:1/414: creat d0/f87 x:0 0 0 2024-03-22T10:42:39.912 INFO:tasks.workunit.client.0.smithi115.stdout:0/335: mknod d3/d4/d6d/c6f 0 2024-03-22T10:42:39.912 INFO:tasks.workunit.client.0.smithi115.stdout:4/325: write d1/d16/d35/d40/d5c/f23 [4784584,102183] 0 2024-03-22T10:42:39.914 INFO:tasks.workunit.client.0.smithi115.stdout:3/282: mkdir d3/d19/d2b/d55 0 2024-03-22T10:42:39.917 INFO:tasks.workunit.client.0.smithi115.stdout:3/283: truncate d3/d16/d29/f32 246655 0 2024-03-22T10:42:39.918 INFO:tasks.workunit.client.0.smithi115.stdout:7/321: rename d2/d7/d12/d10/d23/d3d/d4a to d2/d3a/d6e 0 2024-03-22T10:42:39.919 INFO:tasks.workunit.client.0.smithi115.stdout:3/284: dread d3/d36/f44 [0,4194304] 0 2024-03-22T10:42:39.919 INFO:tasks.workunit.client.0.smithi115.stdout:3/285: chown d3 16 1 2024-03-22T10:42:39.920 INFO:tasks.workunit.client.0.smithi115.stdout:7/322: chown d2/d7/d12/d10/d19/d42 106 1 2024-03-22T10:42:39.923 INFO:tasks.workunit.client.0.smithi115.stdout:1/415: unlink d0/d8/de/f5f 0 2024-03-22T10:42:39.924 INFO:tasks.workunit.client.0.smithi115.stdout:9/381: truncate d6/d8/f1a 3641192 0 2024-03-22T10:42:39.925 INFO:tasks.workunit.client.0.smithi115.stdout:7/323: write d2/d7/d12/d10/d19/d42/d46/f52 [676537,107231] 0 2024-03-22T10:42:39.927 INFO:tasks.workunit.client.0.smithi115.stdout:6/338: sync 2024-03-22T10:42:39.932 INFO:tasks.workunit.client.0.smithi115.stdout:5/318: truncate d2/df/d3b/f2a 2563449 0 2024-03-22T10:42:39.933 INFO:tasks.workunit.client.0.smithi115.stdout:9/382: chown d6/d7/d30/d3b/c42 28034 1 2024-03-22T10:42:39.933 INFO:tasks.workunit.client.0.smithi115.stdout:4/326: mknod d1/d16/d3b/c6e 0 2024-03-22T10:42:39.934 INFO:tasks.workunit.client.0.smithi115.stdout:6/339: read - d4/d13/f5f zero size 2024-03-22T10:42:39.934 INFO:tasks.workunit.client.0.smithi115.stdout:8/506: link d6/dd/c2f d6/dd/d24/ca3 0 2024-03-22T10:42:39.935 INFO:tasks.workunit.client.0.smithi115.stdout:9/383: chown d6/d7/d30 33012 1 2024-03-22T10:42:39.938 INFO:tasks.workunit.client.0.smithi115.stdout:8/507: chown d6/dd/d12/f37 0 1 2024-03-22T10:42:39.939 INFO:tasks.workunit.client.0.smithi115.stdout:3/286: creat d3/d19/d1b/d52/f56 x:0 0 0 2024-03-22T10:42:39.939 INFO:tasks.workunit.client.0.smithi115.stdout:0/336: creat d3/d4/f70 x:0 0 0 2024-03-22T10:42:39.940 INFO:tasks.workunit.client.0.smithi115.stdout:6/340: mkdir d4/d36/d48/d72 0 2024-03-22T10:42:39.942 INFO:tasks.workunit.client.0.smithi115.stdout:1/416: truncate d0/d8/db/f18 4154671 0 2024-03-22T10:42:39.944 INFO:tasks.workunit.client.0.smithi115.stdout:8/508: read d6/f59 [943042,16157] 0 2024-03-22T10:42:39.944 INFO:tasks.workunit.client.0.smithi115.stdout:9/384: creat d6/d7/d20/d22/d23/f85 x:0 0 0 2024-03-22T10:42:39.945 INFO:tasks.workunit.client.0.smithi115.stdout:5/319: creat d2/df/f67 x:0 0 0 2024-03-22T10:42:39.945 INFO:tasks.workunit.client.0.smithi115.stdout:1/417: chown d0/d8/de/l45 4495004 1 2024-03-22T10:42:39.951 INFO:tasks.workunit.client.0.smithi115.stdout:3/287: mkdir d3/d19/d1b/d57 0 2024-03-22T10:42:39.953 INFO:tasks.workunit.client.0.smithi115.stdout:7/324: dread d2/d62/f4f [0,4194304] 0 2024-03-22T10:42:39.954 INFO:tasks.workunit.client.0.smithi115.stdout:6/341: mknod d4/d11/d12/c73 0 2024-03-22T10:42:39.954 INFO:tasks.workunit.client.0.smithi115.stdout:9/385: stat d6/f2e 0 2024-03-22T10:42:39.958 INFO:tasks.workunit.client.0.smithi115.stdout:7/325: chown d2/d7/d12/d10/f28 14632 1 2024-03-22T10:42:39.959 INFO:tasks.workunit.client.0.smithi115.stdout:3/288: fdatasync d3/f9 0 2024-03-22T10:42:39.961 INFO:tasks.workunit.client.0.smithi115.stdout:6/342: write d4/d13/d33/f59 [799019,105839] 0 2024-03-22T10:42:39.962 INFO:tasks.workunit.client.0.smithi115.stdout:4/327: rename d1/d30/d42/d32/f44 to d1/d16/d35/d40/d5c/f6f 0 2024-03-22T10:42:39.963 INFO:tasks.workunit.client.0.smithi115.stdout:8/509: mknod d6/dd/d12/ca4 0 2024-03-22T10:42:39.964 INFO:tasks.workunit.client.0.smithi115.stdout:9/386: rmdir d6/d8/d3c/d59/d68 39 2024-03-22T10:42:39.967 INFO:tasks.workunit.client.0.smithi115.stdout:9/387: stat d6 0 2024-03-22T10:42:39.967 INFO:tasks.workunit.client.0.smithi115.stdout:8/510: stat d6/dd/d24/d4a/d7b/fa2 0 2024-03-22T10:42:39.967 INFO:tasks.workunit.client.0.smithi115.stdout:1/418: rename d0 to d0/d12/d88 22 2024-03-22T10:42:39.970 INFO:tasks.workunit.client.0.smithi115.stdout:4/328: creat d1/d66/f70 x:0 0 0 2024-03-22T10:42:39.970 INFO:tasks.workunit.client.0.smithi115.stdout:6/343: mknod d4/d36/d48/d72/c74 0 2024-03-22T10:42:39.971 INFO:tasks.workunit.client.0.smithi115.stdout:7/326: creat d2/d7/d12/d10/d23/f6f x:0 0 0 2024-03-22T10:42:39.973 INFO:tasks.workunit.client.0.smithi115.stdout:8/511: unlink d6/d49/d68/c7f 0 2024-03-22T10:42:39.974 INFO:tasks.workunit.client.0.smithi115.stdout:1/419: mkdir d0/d22/d89 0 2024-03-22T10:42:39.975 INFO:tasks.workunit.client.0.smithi115.stdout:3/289: getdents d3/d16/d29/d35 0 2024-03-22T10:42:39.976 INFO:tasks.workunit.client.0.smithi115.stdout:0/337: truncate d3/d7/d1b/f24 948180 0 2024-03-22T10:42:39.977 INFO:tasks.workunit.client.0.smithi115.stdout:7/327: chown d2/d7/d12/d10/d23/f2e 4 1 2024-03-22T10:42:39.977 INFO:tasks.workunit.client.0.smithi115.stdout:5/320: dwrite d2/df/f55 [0,4194304] 0 2024-03-22T10:42:39.984 INFO:tasks.workunit.client.0.smithi115.stdout:5/321: truncate d2/df/f3c 9029547 0 2024-03-22T10:42:39.985 INFO:tasks.workunit.client.0.smithi115.stdout:4/329: fdatasync d1/d16/d35/d40/f48 0 2024-03-22T10:42:39.985 INFO:tasks.workunit.client.0.smithi115.stdout:8/512: rmdir d6/d49/d88 39 2024-03-22T10:42:39.988 INFO:tasks.workunit.client.0.smithi115.stdout:1/420: symlink d0/d22/d89/l8a 0 2024-03-22T10:42:39.992 INFO:tasks.workunit.client.0.smithi115.stdout:1/421: dread - d0/d8/db/f2d zero size 2024-03-22T10:42:39.993 INFO:tasks.workunit.client.0.smithi115.stdout:3/290: link d3/d19/d2b/f42 d3/d16/f58 0 2024-03-22T10:42:39.994 INFO:tasks.workunit.client.0.smithi115.stdout:0/338: creat d3/d4/f71 x:0 0 0 2024-03-22T10:42:39.996 INFO:tasks.workunit.client.0.smithi115.stdout:5/322: mkdir d2/df/d13/d68 0 2024-03-22T10:42:39.997 INFO:tasks.workunit.client.0.smithi115.stdout:0/339: read - d3/da/d4c/d68/f5f zero size 2024-03-22T10:42:39.999 INFO:tasks.workunit.client.0.smithi115.stdout:4/330: readlink d1/d16/d35/d40/d5c/l56 0 2024-03-22T10:42:39.999 INFO:tasks.workunit.client.0.smithi115.stdout:2/282: sync 2024-03-22T10:42:40.000 INFO:tasks.workunit.client.0.smithi115.stdout:4/331: dread - d1/d16/f64 zero size 2024-03-22T10:42:40.001 INFO:tasks.workunit.client.0.smithi115.stdout:5/323: write d2/df/f55 [2570445,58173] 0 2024-03-22T10:42:40.004 INFO:tasks.workunit.client.0.smithi115.stdout:0/340: write d3/d4/d14/f15 [666142,65613] 0 2024-03-22T10:42:40.009 INFO:tasks.workunit.client.0.smithi115.stdout:7/328: link d2/d7/d12/d10/f5b d2/d7/d12/d10/d23/d33/f70 0 2024-03-22T10:42:40.011 INFO:tasks.workunit.client.0.smithi115.stdout:9/388: truncate d6/d8/f47 2315938 0 2024-03-22T10:42:40.016 INFO:tasks.workunit.client.0.smithi115.stdout:9/389: readlink d6/d7/d19/l36 0 2024-03-22T10:42:40.016 INFO:tasks.workunit.client.0.smithi115.stdout:5/324: creat d2/d51/f69 x:0 0 0 2024-03-22T10:42:40.023 INFO:tasks.workunit.client.0.smithi115.stdout:2/283: dread da/fb [0,4194304] 0 2024-03-22T10:42:40.023 INFO:tasks.workunit.client.0.smithi115.stdout:1/422: creat d0/d12/d2b/d77/f8b x:0 0 0 2024-03-22T10:42:40.023 INFO:tasks.workunit.client.0.smithi115.stdout:6/344: dread d4/d11/d12/f21 [0,4194304] 0 2024-03-22T10:42:40.027 INFO:tasks.workunit.client.0.smithi115.stdout:0/341: link d3/d4/f71 d3/d4/d14/d64/f72 0 2024-03-22T10:42:40.028 INFO:tasks.workunit.client.0.smithi115.stdout:6/345: chown d4/d11/f54 92 1 2024-03-22T10:42:40.030 INFO:tasks.workunit.client.0.smithi115.stdout:7/329: rename d2/d7/fb to d2/d7/d12/d10/d23/d33/f71 0 2024-03-22T10:42:40.030 INFO:tasks.workunit.client.0.smithi115.stdout:2/284: mkdir da/d25/d3d/d57 0 2024-03-22T10:42:40.041 INFO:tasks.workunit.client.0.smithi115.stdout:8/513: dwrite d6/dd/d24/d4a/f61 [0,4194304] 0 2024-03-22T10:42:40.042 INFO:tasks.workunit.client.0.smithi115.stdout:6/346: creat d4/d13/d33/d58/f75 x:0 0 0 2024-03-22T10:42:40.044 INFO:tasks.workunit.client.0.smithi115.stdout:0/342: dread - d3/d52/f57 zero size 2024-03-22T10:42:40.047 INFO:tasks.workunit.client.0.smithi115.stdout:7/330: read d2/d7/d12/d10/d19/d42/f6d [8070155,106193] 0 2024-03-22T10:42:40.049 INFO:tasks.workunit.client.0.smithi115.stdout:4/332: dread d1/d16/d35/d40/d5c/f25 [0,4194304] 0 2024-03-22T10:42:40.051 INFO:tasks.workunit.client.0.smithi115.stdout:8/514: stat d6/dd/d24/d4a/f8c 0 2024-03-22T10:42:40.054 INFO:tasks.workunit.client.0.smithi115.stdout:3/291: dwrite d3/d36/f44 [0,4194304] 0 2024-03-22T10:42:40.059 INFO:tasks.workunit.client.0.smithi115.stdout:7/331: mkdir d2/d4c/d72 0 2024-03-22T10:42:40.059 INFO:tasks.workunit.client.0.smithi115.stdout:0/343: truncate d3/d4/f60 215552 0 2024-03-22T10:42:40.060 INFO:tasks.workunit.client.0.smithi115.stdout:9/390: dwrite d6/d7/f66 [0,4194304] 0 2024-03-22T10:42:40.063 INFO:tasks.workunit.client.0.smithi115.stdout:1/423: dwrite d0/d12/d17/f80 [0,4194304] 0 2024-03-22T10:42:40.064 INFO:tasks.workunit.client.0.smithi115.stdout:3/292: read d3/d36/f44 [4093423,101999] 0 2024-03-22T10:42:40.067 INFO:tasks.workunit.client.0.smithi115.stdout:0/344: dread - d3/da/d4c/d68/f55 zero size 2024-03-22T10:42:40.074 INFO:tasks.workunit.client.0.smithi115.stdout:8/515: write d6/dd/d24/d4a/d41/f62 [367965,36945] 0 2024-03-22T10:42:40.079 INFO:tasks.workunit.client.0.smithi115.stdout:1/424: rename d0/d8/db/f84 to d0/d12/d17/d37/d5c/f8c 0 2024-03-22T10:42:40.084 INFO:tasks.workunit.client.0.smithi115.stdout:3/293: creat d3/d19/d1b/d54/f59 x:0 0 0 2024-03-22T10:42:40.084 INFO:tasks.workunit.client.0.smithi115.stdout:4/333: dread d1/d69/f61 [0,4194304] 0 2024-03-22T10:42:40.084 INFO:tasks.workunit.client.0.smithi115.stdout:1/425: read - d0/d12/d17/d37/d5c/f8c zero size 2024-03-22T10:42:40.086 INFO:tasks.workunit.client.0.smithi115.stdout:3/294: readlink d3/l46 0 2024-03-22T10:42:40.086 INFO:tasks.workunit.client.0.smithi115.stdout:6/347: write d4/d13/d33/f42 [349533,98330] 0 2024-03-22T10:42:40.090 INFO:tasks.workunit.client.0.smithi115.stdout:0/345: truncate d3/d7/d1b/f24 873037 0 2024-03-22T10:42:40.098 INFO:tasks.workunit.client.0.smithi115.stdout:1/426: read d0/d8/db/d25/f44 [2468972,35239] 0 2024-03-22T10:42:40.098 INFO:tasks.workunit.client.0.smithi115.stdout:1/427: readlink d0/d8/l1f 0 2024-03-22T10:42:40.098 INFO:tasks.workunit.client.0.smithi115.stdout:6/348: write d4/f10 [3656211,77548] 0 2024-03-22T10:42:40.098 INFO:tasks.workunit.client.0.smithi115.stdout:2/285: dread da/d25/f34 [0,4194304] 0 2024-03-22T10:42:40.098 INFO:tasks.workunit.client.0.smithi115.stdout:2/286: stat da/d1c/f2b 0 2024-03-22T10:42:40.099 INFO:tasks.workunit.client.0.smithi115.stdout:8/516: rename d6/dd/d24/d4a/d41/c63 to d6/dd/d24/d4a/d7b/d8f/da0/ca5 0 2024-03-22T10:42:40.099 INFO:tasks.workunit.client.0.smithi115.stdout:6/349: dread - d4/d36/d48/f62 zero size 2024-03-22T10:42:40.101 INFO:tasks.workunit.client.0.smithi115.stdout:6/350: read - d4/d13/d33/f6c zero size 2024-03-22T10:42:40.108 INFO:tasks.workunit.client.0.smithi115.stdout:4/334: truncate d1/d66/f6d 1632458 0 2024-03-22T10:42:40.109 INFO:tasks.workunit.client.0.smithi115.stdout:1/428: dread d0/d12/d17/d37/f83 [0,4194304] 0 2024-03-22T10:42:40.109 INFO:tasks.workunit.client.0.smithi115.stdout:0/346: symlink d3/d7/l73 0 2024-03-22T10:42:40.110 INFO:tasks.workunit.client.0.smithi115.stdout:3/295: write d3/f53 [448435,87404] 0 2024-03-22T10:42:40.111 INFO:tasks.workunit.client.0.smithi115.stdout:8/517: truncate d6/fc 725796 0 2024-03-22T10:42:40.123 INFO:tasks.workunit.client.0.smithi115.stdout:4/335: unlink d1/l2f 0 2024-03-22T10:42:40.123 INFO:tasks.workunit.client.0.smithi115.stdout:7/332: dwrite d2/d7/d12/d10/d23/d33/f70 [0,4194304] 0 2024-03-22T10:42:40.125 INFO:tasks.workunit.client.0.smithi115.stdout:9/391: dwrite d6/d8/d3c/f54 [0,4194304] 0 2024-03-22T10:42:40.128 INFO:tasks.workunit.client.0.smithi115.stdout:4/336: chown d1/d16/d35/d40/d5c/f23 236122 1 2024-03-22T10:42:40.128 INFO:tasks.workunit.client.0.smithi115.stdout:3/296: write d3/f12 [2410539,71471] 0 2024-03-22T10:42:40.130 INFO:tasks.workunit.client.0.smithi115.stdout:8/518: symlink d6/dd/d12/d31/la6 0 2024-03-22T10:42:40.131 INFO:tasks.workunit.client.0.smithi115.stdout:2/287: write da/f16 [884158,41393] 0 2024-03-22T10:42:40.137 INFO:tasks.workunit.client.0.smithi115.stdout:9/392: mkdir d6/d8/d86 0 2024-03-22T10:42:40.139 INFO:tasks.workunit.client.0.smithi115.stdout:1/429: creat d0/d8/db/d25/d7e/f8d x:0 0 0 2024-03-22T10:42:40.141 INFO:tasks.workunit.client.0.smithi115.stdout:3/297: write d3/f50 [248910,54702] 0 2024-03-22T10:42:40.145 INFO:tasks.workunit.client.0.smithi115.stdout:4/337: symlink d1/d16/d35/l71 0 2024-03-22T10:42:40.145 INFO:tasks.workunit.client.0.smithi115.stdout:2/288: stat da/d25/d3d/l36 0 2024-03-22T10:42:40.147 INFO:tasks.workunit.client.0.smithi115.stdout:3/298: stat d3/d16/d29/f32 0 2024-03-22T10:42:40.148 INFO:tasks.workunit.client.0.smithi115.stdout:7/333: symlink d2/d7/d12/d10/d19/l73 0 2024-03-22T10:42:40.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:39 smithi150 ceph-mon[34518]: pgmap v8: 65 pgs: 65 active+clean; 558 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 77 MiB/s wr, 170 op/s 2024-03-22T10:42:40.150 INFO:tasks.workunit.client.0.smithi115.stdout:6/351: dwrite d4/d36/d48/f67 [0,4194304] 0 2024-03-22T10:42:40.150 INFO:tasks.workunit.client.0.smithi115.stdout:8/519: rmdir d6/d49/d88 39 2024-03-22T10:42:40.150 INFO:tasks.workunit.client.0.smithi115.stdout:4/338: write d1/d30/f4a [277241,124405] 0 2024-03-22T10:42:40.151 INFO:tasks.workunit.client.0.smithi115.stdout:1/430: mkdir d0/d8/db/d8e 0 2024-03-22T10:42:40.152 INFO:tasks.workunit.client.0.smithi115.stdout:9/393: unlink d6/d7/f11 0 2024-03-22T10:42:40.153 INFO:tasks.workunit.client.0.smithi115.stdout:2/289: chown da/d1c/c53 59192206 1 2024-03-22T10:42:40.154 INFO:tasks.workunit.client.0.smithi115.stdout:1/431: chown d0/d12/d17/d67 24 1 2024-03-22T10:42:40.156 INFO:tasks.workunit.client.0.smithi115.stdout:4/339: readlink d1/l4 0 2024-03-22T10:42:40.157 INFO:tasks.workunit.client.0.smithi115.stdout:0/347: dwrite d3/da/d4c/d68/f5f [0,4194304] 0 2024-03-22T10:42:40.162 INFO:tasks.workunit.client.0.smithi115.stdout:1/432: write d0/d8/de/d1a/f2f [1174486,20526] 0 2024-03-22T10:42:40.163 INFO:tasks.workunit.client.0.smithi115.stdout:9/394: chown d6/d7/d30/d3b/c3e 591 1 2024-03-22T10:42:40.163 INFO:tasks.workunit.client.0.smithi115.stdout:4/340: rmdir d1/d69 39 2024-03-22T10:42:40.169 INFO:tasks.workunit.client.0.smithi115.stdout:6/352: creat d4/d36/d48/d72/f76 x:0 0 0 2024-03-22T10:42:40.169 INFO:tasks.workunit.client.0.smithi115.stdout:7/334: mkdir d2/d4c/d72/d74 0 2024-03-22T10:42:40.172 INFO:tasks.workunit.client.0.smithi115.stdout:0/348: mkdir d3/da/d4c/d74 0 2024-03-22T10:42:40.172 INFO:tasks.workunit.client.0.smithi115.stdout:3/299: getdents d3/d19/d2b/d55 0 2024-03-22T10:42:40.176 INFO:tasks.workunit.client.0.smithi115.stdout:8/520: dwrite d6/d49/f83 [0,4194304] 0 2024-03-22T10:42:40.176 INFO:tasks.workunit.client.0.smithi115.stdout:7/335: chown d2/d7/d12/d10/d19/d42/f6d 151 1 2024-03-22T10:42:40.176 INFO:tasks.workunit.client.0.smithi115.stdout:9/395: mkdir d6/d7/d87 0 2024-03-22T10:42:40.178 INFO:tasks.workunit.client.0.smithi115.stdout:4/341: symlink d1/d16/d35/d40/l72 0 2024-03-22T10:42:40.182 INFO:tasks.workunit.client.0.smithi115.stdout:1/433: fsync d0/f1 0 2024-03-22T10:42:40.182 INFO:tasks.workunit.client.0.smithi115.stdout:0/349: mknod d3/d7/d1c/c75 0 2024-03-22T10:42:40.188 INFO:tasks.workunit.client.0.smithi115.stdout:3/300: mknod d3/d16/d29/d35/c5a 0 2024-03-22T10:42:40.192 INFO:tasks.workunit.client.0.smithi115.stdout:5/325: sync 2024-03-22T10:42:40.196 INFO:tasks.workunit.client.0.smithi115.stdout:1/434: stat d0/d12/f48 0 2024-03-22T10:42:40.200 INFO:tasks.workunit.client.0.smithi115.stdout:6/353: dwrite d4/d36/f53 [0,4194304] 0 2024-03-22T10:42:40.204 INFO:tasks.workunit.client.0.smithi115.stdout:3/301: chown d3/d19/f31 1852401 1 2024-03-22T10:42:40.204 INFO:tasks.workunit.client.0.smithi115.stdout:2/290: truncate da/f16 396921 0 2024-03-22T10:42:40.204 INFO:tasks.workunit.client.0.smithi115.stdout:2/291: stat da 0 2024-03-22T10:42:40.204 INFO:tasks.workunit.client.0.smithi115.stdout:6/354: chown d4/d11/d12 4034 1 2024-03-22T10:42:40.206 INFO:tasks.workunit.client.0.smithi115.stdout:0/350: creat d3/da/d4c/d69/f76 x:0 0 0 2024-03-22T10:42:40.210 INFO:tasks.workunit.client.0.smithi115.stdout:3/302: unlink d3/d16/d29/d35/c5a 0 2024-03-22T10:42:40.219 INFO:tasks.workunit.client.0.smithi115.stdout:4/342: dread d1/d69/f61 [0,4194304] 0 2024-03-22T10:42:40.219 INFO:tasks.workunit.client.0.smithi115.stdout:8/521: rename d6/dd/d12/f37 to d6/dd/d24/fa7 0 2024-03-22T10:42:40.221 INFO:tasks.workunit.client.0.smithi115.stdout:6/355: write d4/d13/f70 [7553846,9429] 0 2024-03-22T10:42:40.225 INFO:tasks.workunit.client.0.smithi115.stdout:3/303: truncate d3/d16/d29/f32 617371 0 2024-03-22T10:42:40.226 INFO:tasks.workunit.client.0.smithi115.stdout:3/304: stat d3/f45 0 2024-03-22T10:42:40.228 INFO:tasks.workunit.client.0.smithi115.stdout:5/326: dread d2/df/d13/f18 [0,4194304] 0 2024-03-22T10:42:40.232 INFO:tasks.workunit.client.0.smithi115.stdout:4/343: mkdir d1/d30/d42/d73 0 2024-03-22T10:42:40.233 INFO:tasks.workunit.client.0.smithi115.stdout:3/305: read - d3/d19/d2b/f47 zero size 2024-03-22T10:42:40.233 INFO:tasks.workunit.client.0.smithi115.stdout:1/435: link d0/d8/de/f4c d0/f8f 0 2024-03-22T10:42:40.235 INFO:tasks.workunit.client.0.smithi115.stdout:3/306: stat d3 0 2024-03-22T10:42:40.239 INFO:tasks.workunit.client.0.smithi115.stdout:0/351: dwrite d3/f1a [4194304,4194304] 0 2024-03-22T10:42:40.240 INFO:tasks.workunit.client.0.smithi115.stdout:6/356: dread d4/d11/d12/f31 [0,4194304] 0 2024-03-22T10:42:40.244 INFO:tasks.workunit.client.0.smithi115.stdout:9/396: rename d6/d7/d48/l63 to d6/d7/d19/d4e/l88 0 2024-03-22T10:42:40.244 INFO:tasks.workunit.client.0.smithi115.stdout:3/307: write d3/d16/f1c [3235243,15028] 0 2024-03-22T10:42:40.246 INFO:tasks.workunit.client.0.smithi115.stdout:7/336: dread d2/d7/d12/f4b [0,4194304] 0 2024-03-22T10:42:40.252 INFO:tasks.workunit.client.0.smithi115.stdout:8/522: creat d6/d49/fa8 x:0 0 0 2024-03-22T10:42:40.255 INFO:tasks.workunit.client.0.smithi115.stdout:4/344: mknod d1/d16/c74 0 2024-03-22T10:42:40.255 INFO:tasks.workunit.client.0.smithi115.stdout:2/292: dwrite da/d25/d30/f43 [0,4194304] 0 2024-03-22T10:42:40.256 INFO:tasks.workunit.client.0.smithi115.stdout:7/337: truncate d2/d7/d12/d10/d19/d42/d46/f68 142777 0 2024-03-22T10:42:40.258 INFO:tasks.workunit.client.0.smithi115.stdout:2/293: chown da/c19 0 1 2024-03-22T10:42:40.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:39 smithi115 ceph-mon[29111]: pgmap v8: 65 pgs: 65 active+clean; 558 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 77 MiB/s wr, 170 op/s 2024-03-22T10:42:40.263 INFO:tasks.workunit.client.0.smithi115.stdout:3/308: read d3/d36/f3e [6217,115009] 0 2024-03-22T10:42:40.264 INFO:tasks.workunit.client.0.smithi115.stdout:4/345: write d1/d16/d35/d40/d54/f67 [62580,44144] 0 2024-03-22T10:42:40.264 INFO:tasks.workunit.client.0.smithi115.stdout:5/327: rename d2/d15/d1b/l20 to d2/d51/l6a 0 2024-03-22T10:42:40.265 INFO:tasks.workunit.client.0.smithi115.stdout:9/397: fdatasync d6/d7/d30/d3b/f46 0 2024-03-22T10:42:40.269 INFO:tasks.workunit.client.0.smithi115.stdout:0/352: creat d3/d7/d1c/f77 x:0 0 0 2024-03-22T10:42:40.270 INFO:tasks.workunit.client.0.smithi115.stdout:6/357: symlink d4/d11/l77 0 2024-03-22T10:42:40.277 INFO:tasks.workunit.client.0.smithi115.stdout:4/346: mkdir d1/d30/d42/d75 0 2024-03-22T10:42:40.281 INFO:tasks.workunit.client.0.smithi115.stdout:0/353: chown d3/d4/l65 19756796 1 2024-03-22T10:42:40.285 INFO:tasks.workunit.client.0.smithi115.stdout:8/523: symlink d6/dd/d24/la9 0 2024-03-22T10:42:40.291 INFO:tasks.workunit.client.0.smithi115.stdout:5/328: mknod d2/df/d13/d68/c6b 0 2024-03-22T10:42:40.296 INFO:tasks.workunit.client.0.smithi115.stdout:1/436: rename d0/f70 to d0/d8/db/d25/d7e/f90 0 2024-03-22T10:42:40.296 INFO:tasks.workunit.client.0.smithi115.stdout:0/354: mkdir d3/d7/d1b/d1f/d78 0 2024-03-22T10:42:40.304 INFO:tasks.workunit.client.0.smithi115.stdout:5/329: creat d2/df/d13/d68/f6c x:0 0 0 2024-03-22T10:42:40.305 INFO:tasks.workunit.client.0.smithi115.stdout:6/358: creat d4/d11/f78 x:0 0 0 2024-03-22T10:42:40.306 INFO:tasks.workunit.client.0.smithi115.stdout:7/338: dwrite d2/d7/d12/d10/d23/d33/f71 [0,4194304] 0 2024-03-22T10:42:40.309 INFO:tasks.workunit.client.0.smithi115.stdout:1/437: mkdir d0/d8/de/d1a/d91 0 2024-03-22T10:42:40.313 INFO:tasks.workunit.client.0.smithi115.stdout:0/355: creat d3/d7/d1b/d35/f79 x:0 0 0 2024-03-22T10:42:40.316 INFO:tasks.workunit.client.0.smithi115.stdout:5/330: write d2/d15/d1b/d33/f63 [565144,108322] 0 2024-03-22T10:42:40.321 INFO:tasks.workunit.client.0.smithi115.stdout:0/356: fsync d3/d4/d14/f34 0 2024-03-22T10:42:40.321 INFO:tasks.workunit.client.0.smithi115.stdout:7/339: chown d2/d7/d12/d10/f36 1774 1 2024-03-22T10:42:40.321 INFO:tasks.workunit.client.0.smithi115.stdout:2/294: dread da/d1c/f1e [0,4194304] 0 2024-03-22T10:42:40.322 INFO:tasks.workunit.client.0.smithi115.stdout:1/438: rmdir d0/d8/db 39 2024-03-22T10:42:40.324 INFO:tasks.workunit.client.0.smithi115.stdout:9/398: dread f0 [0,4194304] 0 2024-03-22T10:42:40.324 INFO:tasks.workunit.client.0.smithi115.stdout:4/347: dwrite d1/d30/d42/f21 [0,4194304] 0 2024-03-22T10:42:40.332 INFO:tasks.workunit.client.0.smithi115.stdout:5/331: rmdir d2/df/d13 39 2024-03-22T10:42:40.334 INFO:tasks.workunit.client.0.smithi115.stdout:3/309: dwrite d3/d36/d41/f48 [0,4194304] 0 2024-03-22T10:42:40.335 INFO:tasks.workunit.client.0.smithi115.stdout:4/348: creat d1/d16/d35/d40/d5c/d14/f76 x:0 0 0 2024-03-22T10:42:40.337 INFO:tasks.workunit.client.0.smithi115.stdout:3/310: chown d3/c2d 58 1 2024-03-22T10:42:40.337 INFO:tasks.workunit.client.0.smithi115.stdout:5/332: chown d2/d15/d32 384117 1 2024-03-22T10:42:40.341 INFO:tasks.workunit.client.0.smithi115.stdout:7/340: mknod d2/d7/d12/d10/d19/d42/d46/d61/c75 0 2024-03-22T10:42:40.341 INFO:tasks.workunit.client.0.smithi115.stdout:3/311: chown d3/d36/d41/f3f 660330 1 2024-03-22T10:42:40.341 INFO:tasks.workunit.client.0.smithi115.stdout:7/341: readlink d2/l4 0 2024-03-22T10:42:40.347 INFO:tasks.workunit.client.0.smithi115.stdout:2/295: rename da/d25/l3b to da/d25/d3d/d57/l58 0 2024-03-22T10:42:40.354 INFO:tasks.workunit.client.0.smithi115.stdout:4/349: rename d1/d69/f61 to d1/d66/f77 0 2024-03-22T10:42:40.354 INFO:tasks.workunit.client.1.smithi150.stderr:++ readlink -f fsstress 2024-03-22T10:42:40.360 INFO:tasks.workunit.client.1.smithi150.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-22T10:42:40.360 INFO:tasks.workunit.client.1.smithi150.stderr:+ popd 2024-03-22T10:42:40.362 INFO:tasks.workunit.client.0.smithi115.stdout:0/357: link d3/d4/c2c d3/d7/d1b/c7a 0 2024-03-22T10:42:40.365 INFO:tasks.workunit.client.1.smithi150.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T10:42:40.366 INFO:tasks.workunit.client.1.smithi150.stderr:+ popd 2024-03-22T10:42:40.366 INFO:tasks.workunit.client.0.smithi115.stdout:8/524: dread d6/dd/d12/d31/f5b [0,4194304] 0 2024-03-22T10:42:40.366 INFO:tasks.workunit.client.0.smithi115.stdout:7/342: write d2/d7/d12/d10/f16 [2152610,106801] 0 2024-03-22T10:42:40.367 INFO:tasks.workunit.client.0.smithi115.stdout:1/439: dwrite d0/d12/d2b/f35 [0,4194304] 0 2024-03-22T10:42:40.367 INFO:tasks.workunit.client.1.smithi150.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-22T10:42:40.368 INFO:tasks.workunit.client.1.smithi150.stderr:++ mktemp -d -p . 2024-03-22T10:42:40.369 INFO:tasks.workunit.client.0.smithi115.stdout:2/296: creat da/d25/d30/f59 x:0 0 0 2024-03-22T10:42:40.374 INFO:tasks.workunit.client.0.smithi115.stdout:9/399: dwrite d6/d8/f1a [0,4194304] 0 2024-03-22T10:42:40.374 INFO:tasks.workunit.client.0.smithi115.stdout:5/333: truncate d2/fc 1789677 0 2024-03-22T10:42:40.375 INFO:tasks.workunit.client.1.smithi150.stderr:+ T=./tmp.zi0gx6aUjm 2024-03-22T10:42:40.376 INFO:tasks.workunit.client.1.smithi150.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.zi0gx6aUjm -l 1 -n 1000 -p 10 -v 2024-03-22T10:42:40.384 INFO:tasks.workunit.client.1.smithi150.stdout:seed = 1710973268 2024-03-22T10:42:40.385 INFO:tasks.workunit.client.0.smithi115.stdout:1/440: unlink d0/d12/d2b/l47 0 2024-03-22T10:42:40.385 INFO:tasks.workunit.client.0.smithi115.stdout:4/350: dwrite d1/f9 [0,4194304] 0 2024-03-22T10:42:40.387 INFO:tasks.workunit.client.0.smithi115.stdout:8/525: symlink d6/dd/d24/laa 0 2024-03-22T10:42:40.387 INFO:tasks.workunit.client.0.smithi115.stdout:9/400: symlink d6/d7/d6a/d6f/l89 0 2024-03-22T10:42:40.389 INFO:tasks.workunit.client.0.smithi115.stdout:5/334: symlink d2/df/d3b/l6d 0 2024-03-22T10:42:40.392 INFO:tasks.workunit.client.1.smithi150.stdout:0/0: rename - no filename 2024-03-22T10:42:40.392 INFO:tasks.workunit.client.0.smithi115.stdout:8/526: chown d6/dd/d12/d31/c52 58 1 2024-03-22T10:42:40.393 INFO:tasks.workunit.client.0.smithi115.stdout:9/401: chown d6/d7/d20/f57 3 1 2024-03-22T10:42:40.394 INFO:tasks.workunit.client.1.smithi150.stdout:1/0: write - no filename 2024-03-22T10:42:40.394 INFO:tasks.workunit.client.1.smithi150.stdout:1/1: dread - no filename 2024-03-22T10:42:40.394 INFO:tasks.workunit.client.1.smithi150.stdout:1/2: rename - no filename 2024-03-22T10:42:40.395 INFO:tasks.workunit.client.1.smithi150.stdout:1/3: write - no filename 2024-03-22T10:42:40.395 INFO:tasks.workunit.client.1.smithi150.stdout:1/4: dread - no filename 2024-03-22T10:42:40.396 INFO:tasks.workunit.client.1.smithi150.stdout:0/1: mkdir d0 0 2024-03-22T10:42:40.397 INFO:tasks.workunit.client.0.smithi115.stdout:1/441: truncate d0/d8/d4d/d56/f6b 1039390 0 2024-03-22T10:42:40.397 INFO:tasks.workunit.client.0.smithi115.stdout:7/343: rename d2/d4c/d72 to d2/d7/d12/d10/d19/d42/d76 0 2024-03-22T10:42:40.397 INFO:tasks.workunit.client.0.smithi115.stdout:2/297: link da/d25/d3d/c33 da/d25/d3d/d57/c5a 0 2024-03-22T10:42:40.398 INFO:tasks.workunit.client.0.smithi115.stdout:5/335: mknod d2/d51/c6e 0 2024-03-22T10:42:40.398 INFO:tasks.workunit.client.1.smithi150.stdout:2/0: dread - no filename 2024-03-22T10:42:40.399 INFO:tasks.workunit.client.1.smithi150.stdout:2/1: fdatasync - no filename 2024-03-22T10:42:40.399 INFO:tasks.workunit.client.1.smithi150.stdout:2/2: dread - no filename 2024-03-22T10:42:40.399 INFO:tasks.workunit.client.1.smithi150.stdout:2/3: write - no filename 2024-03-22T10:42:40.399 INFO:tasks.workunit.client.1.smithi150.stdout:2/4: fdatasync - no filename 2024-03-22T10:42:40.401 INFO:tasks.workunit.client.1.smithi150.stdout:3/0: rename - no filename 2024-03-22T10:42:40.401 INFO:tasks.workunit.client.1.smithi150.stdout:3/1: truncate - no filename 2024-03-22T10:42:40.401 INFO:tasks.workunit.client.1.smithi150.stdout:3/2: unlink - no file 2024-03-22T10:42:40.401 INFO:tasks.workunit.client.1.smithi150.stdout:1/5: creat f0 x:0 0 0 2024-03-22T10:42:40.402 INFO:tasks.workunit.client.1.smithi150.stdout:0/2: mkdir d0/d1 0 2024-03-22T10:42:40.402 INFO:tasks.workunit.client.1.smithi150.stdout:0/3: write - no filename 2024-03-22T10:42:40.402 INFO:tasks.workunit.client.0.smithi115.stdout:8/527: unlink c1 0 2024-03-22T10:42:40.404 INFO:tasks.workunit.client.0.smithi115.stdout:9/402: symlink d6/d7/l8a 0 2024-03-22T10:42:40.406 INFO:tasks.workunit.client.1.smithi150.stdout:2/5: symlink l0 0 2024-03-22T10:42:40.406 INFO:tasks.workunit.client.1.smithi150.stdout:1/6: mkdir d1 0 2024-03-22T10:42:40.407 INFO:tasks.workunit.client.1.smithi150.stdout:1/7: fdatasync f0 0 2024-03-22T10:42:40.409 INFO:tasks.workunit.client.1.smithi150.stdout:3/3: mkdir d0 0 2024-03-22T10:42:40.409 INFO:tasks.workunit.client.1.smithi150.stdout:3/4: write - no filename 2024-03-22T10:42:40.409 INFO:tasks.workunit.client.0.smithi115.stdout:2/298: chown da/d25/d3d/d57/l58 10975 1 2024-03-22T10:42:40.410 INFO:tasks.workunit.client.1.smithi150.stdout:3/5: rename d0 to d0/d1 22 2024-03-22T10:42:40.411 INFO:tasks.workunit.client.1.smithi150.stdout:3/6: dwrite - no filename 2024-03-22T10:42:40.411 INFO:tasks.workunit.client.1.smithi150.stdout:3/7: link - no file 2024-03-22T10:42:40.412 INFO:tasks.workunit.client.0.smithi115.stdout:7/344: write d2/d7/d12/f4b [896762,6406] 0 2024-03-22T10:42:40.412 INFO:tasks.workunit.client.0.smithi115.stdout:5/336: symlink d2/d15/d1b/d33/d38/d43/d49/l6f 0 2024-03-22T10:42:40.414 INFO:tasks.workunit.client.0.smithi115.stdout:2/299: write da/d25/d30/f31 [1033379,104376] 0 2024-03-22T10:42:40.419 INFO:tasks.workunit.client.0.smithi115.stdout:3/312: dread d3/d16/f1c [0,4194304] 0 2024-03-22T10:42:40.420 INFO:tasks.workunit.client.0.smithi115.stdout:3/313: chown d3/d16/f58 109 1 2024-03-22T10:42:40.420 INFO:tasks.workunit.client.0.smithi115.stdout:8/528: mknod d6/dd/d24/d4a/d7b/d8f/da0/cab 0 2024-03-22T10:42:40.421 INFO:tasks.workunit.client.1.smithi150.stdout:0/4: mkdir d0/d1/d2 0 2024-03-22T10:42:40.421 INFO:tasks.workunit.client.1.smithi150.stdout:0/5: fsync - no filename 2024-03-22T10:42:40.421 INFO:tasks.workunit.client.1.smithi150.stdout:0/6: dread - no filename 2024-03-22T10:42:40.421 INFO:tasks.workunit.client.1.smithi150.stdout:0/7: truncate - no filename 2024-03-22T10:42:40.421 INFO:tasks.workunit.client.1.smithi150.stdout:0/8: dwrite - no filename 2024-03-22T10:42:40.423 INFO:tasks.workunit.client.0.smithi115.stdout:3/314: write d3/d19/d1b/f27 [2033408,112191] 0 2024-03-22T10:42:40.424 INFO:tasks.workunit.client.1.smithi150.stdout:6/0: dwrite - no filename 2024-03-22T10:42:40.424 INFO:tasks.workunit.client.1.smithi150.stdout:6/1: write - no filename 2024-03-22T10:42:40.425 INFO:tasks.workunit.client.1.smithi150.stdout:2/6: creat f1 x:0 0 0 2024-03-22T10:42:40.426 INFO:tasks.workunit.client.0.smithi115.stdout:9/403: symlink d6/d8/d3c/d59/d75/l8b 0 2024-03-22T10:42:40.427 INFO:tasks.workunit.client.1.smithi150.stdout:4/0: mkdir d0 0 2024-03-22T10:42:40.429 INFO:tasks.workunit.client.1.smithi150.stdout:5/0: mkdir d0 0 2024-03-22T10:42:40.429 INFO:tasks.workunit.client.1.smithi150.stdout:5/1: dread - no filename 2024-03-22T10:42:40.429 INFO:tasks.workunit.client.1.smithi150.stdout:7/0: rmdir - no directory 2024-03-22T10:42:40.429 INFO:tasks.workunit.client.1.smithi150.stdout:7/1: truncate - no filename 2024-03-22T10:42:40.429 INFO:tasks.workunit.client.1.smithi150.stdout:7/2: write - no filename 2024-03-22T10:42:40.429 INFO:tasks.workunit.client.1.smithi150.stdout:7/3: readlink - no filename 2024-03-22T10:42:40.429 INFO:tasks.workunit.client.1.smithi150.stdout:7/4: dread - no filename 2024-03-22T10:42:40.430 INFO:tasks.workunit.client.1.smithi150.stdout:2/7: creat f2 x:0 0 0 2024-03-22T10:42:40.430 INFO:tasks.workunit.client.0.smithi115.stdout:5/337: mkdir d2/d15/d1b/d33/d38/d56/d70 0 2024-03-22T10:42:40.430 INFO:tasks.workunit.client.0.smithi115.stdout:7/345: creat d2/d7/d12/d10/d19/d42/d76/d74/f77 x:0 0 0 2024-03-22T10:42:40.430 INFO:tasks.workunit.client.0.smithi115.stdout:1/442: link d0/d8/de/d1a/l50 d0/d12/d17/l92 0 2024-03-22T10:42:40.431 INFO:tasks.workunit.client.1.smithi150.stdout:0/9: creat d0/d1/d2/f3 x:0 0 0 2024-03-22T10:42:40.431 INFO:tasks.workunit.client.1.smithi150.stdout:6/2: symlink l0 0 2024-03-22T10:42:40.432 INFO:tasks.workunit.client.1.smithi150.stdout:3/8: rmdir d0 0 2024-03-22T10:42:40.432 INFO:tasks.workunit.client.1.smithi150.stdout:3/9: unlink - no file 2024-03-22T10:42:40.432 INFO:tasks.workunit.client.1.smithi150.stdout:3/10: dread - no filename 2024-03-22T10:42:40.432 INFO:tasks.workunit.client.1.smithi150.stdout:3/11: dread - no filename 2024-03-22T10:42:40.432 INFO:tasks.workunit.client.1.smithi150.stdout:3/12: write - no filename 2024-03-22T10:42:40.432 INFO:tasks.workunit.client.1.smithi150.stdout:3/13: rmdir - no directory 2024-03-22T10:42:40.433 INFO:tasks.workunit.client.1.smithi150.stdout:0/10: dread - d0/d1/d2/f3 zero size 2024-03-22T10:42:40.434 INFO:tasks.workunit.client.0.smithi115.stdout:2/300: symlink da/d25/d30/d4f/d54/l5b 0 2024-03-22T10:42:40.434 INFO:tasks.workunit.client.0.smithi115.stdout:8/529: creat d6/dd/d24/d4a/d7b/d8f/da0/fac x:0 0 0 2024-03-22T10:42:40.434 INFO:tasks.workunit.client.1.smithi150.stdout:2/8: mknod c3 0 2024-03-22T10:42:40.435 INFO:tasks.workunit.client.1.smithi150.stdout:8/0: read - no filename 2024-03-22T10:42:40.435 INFO:tasks.workunit.client.1.smithi150.stdout:8/1: readlink - no filename 2024-03-22T10:42:40.436 INFO:tasks.workunit.client.1.smithi150.stdout:4/1: mknod d0/c1 0 2024-03-22T10:42:40.436 INFO:tasks.workunit.client.1.smithi150.stdout:2/9: chown c3 15 1 2024-03-22T10:42:40.436 INFO:tasks.workunit.client.1.smithi150.stdout:3/14: symlink l2 0 2024-03-22T10:42:40.437 INFO:tasks.workunit.client.1.smithi150.stdout:3/15: read - no filename 2024-03-22T10:42:40.438 INFO:tasks.workunit.client.1.smithi150.stdout:5/2: symlink d0/l1 0 2024-03-22T10:42:40.438 INFO:tasks.workunit.client.1.smithi150.stdout:7/5: creat f0 x:0 0 0 2024-03-22T10:42:40.438 INFO:tasks.workunit.client.1.smithi150.stdout:3/16: chown l2 946306 1 2024-03-22T10:42:40.439 INFO:tasks.workunit.client.1.smithi150.stdout:3/17: truncate - no filename 2024-03-22T10:42:40.439 INFO:tasks.workunit.client.1.smithi150.stdout:6/3: rename l0 to l1 0 2024-03-22T10:42:40.439 INFO:tasks.workunit.client.1.smithi150.stdout:6/4: read - no filename 2024-03-22T10:42:40.439 INFO:tasks.workunit.client.1.smithi150.stdout:3/18: stat l2 0 2024-03-22T10:42:40.439 INFO:tasks.workunit.client.1.smithi150.stdout:2/10: dread - f1 zero size 2024-03-22T10:42:40.440 INFO:tasks.workunit.client.0.smithi115.stdout:7/346: mkdir d2/d3a/d6e/d78 0 2024-03-22T10:42:40.440 INFO:tasks.workunit.client.0.smithi115.stdout:1/443: chown d0/c6 918 1 2024-03-22T10:42:40.440 INFO:tasks.workunit.client.0.smithi115.stdout:4/351: dread d1/d16/d35/d40/d5c/d14/f1b [0,4194304] 0 2024-03-22T10:42:40.443 INFO:tasks.workunit.client.1.smithi150.stdout:9/0: write - no filename 2024-03-22T10:42:40.443 INFO:tasks.workunit.client.1.smithi150.stdout:9/1: readlink - no filename 2024-03-22T10:42:40.443 INFO:tasks.workunit.client.1.smithi150.stdout:9/2: dread - no filename 2024-03-22T10:42:40.443 INFO:tasks.workunit.client.1.smithi150.stdout:9/3: fsync - no filename 2024-03-22T10:42:40.443 INFO:tasks.workunit.client.1.smithi150.stdout:9/4: dread - no filename 2024-03-22T10:42:40.443 INFO:tasks.workunit.client.1.smithi150.stdout:4/2: mknod d0/c2 0 2024-03-22T10:42:40.444 INFO:tasks.workunit.client.1.smithi150.stdout:0/11: write d0/d1/d2/f3 [38526,61433] 0 2024-03-22T10:42:40.444 INFO:tasks.workunit.client.1.smithi150.stdout:7/6: mknod c1 0 2024-03-22T10:42:40.444 INFO:tasks.workunit.client.1.smithi150.stdout:2/11: dread - f1 zero size 2024-03-22T10:42:40.444 INFO:tasks.workunit.client.1.smithi150.stdout:3/19: mknod c3 0 2024-03-22T10:42:40.444 INFO:tasks.workunit.client.1.smithi150.stdout:3/20: truncate - no filename 2024-03-22T10:42:40.445 INFO:tasks.workunit.client.1.smithi150.stdout:7/7: dread - f0 zero size 2024-03-22T10:42:40.446 INFO:tasks.workunit.client.0.smithi115.stdout:2/301: write da/d25/f49 [755511,75848] 0 2024-03-22T10:42:40.447 INFO:tasks.workunit.client.1.smithi150.stdout:8/2: mkdir d0 0 2024-03-22T10:42:40.447 INFO:tasks.workunit.client.1.smithi150.stdout:3/21: chown c3 302 1 2024-03-22T10:42:40.447 INFO:tasks.workunit.client.1.smithi150.stdout:3/22: dread - no filename 2024-03-22T10:42:40.447 INFO:tasks.workunit.client.1.smithi150.stdout:2/12: dread - f2 zero size 2024-03-22T10:42:40.447 INFO:tasks.workunit.client.1.smithi150.stdout:0/12: chown d0/d1/d2 2518 1 2024-03-22T10:42:40.449 INFO:tasks.workunit.client.1.smithi150.stdout:4/3: chown d0/c2 3429 1 2024-03-22T10:42:40.449 INFO:tasks.workunit.client.1.smithi150.stdout:4/4: write - no filename 2024-03-22T10:42:40.450 INFO:tasks.workunit.client.1.smithi150.stdout:2/13: truncate f2 214238 0 2024-03-22T10:42:40.450 INFO:tasks.workunit.client.1.smithi150.stdout:9/5: symlink l0 0 2024-03-22T10:42:40.451 INFO:tasks.workunit.client.1.smithi150.stdout:2/14: readlink l0 0 2024-03-22T10:42:40.452 INFO:tasks.workunit.client.1.smithi150.stdout:0/13: fdatasync d0/d1/d2/f3 0 2024-03-22T10:42:40.453 INFO:tasks.workunit.client.1.smithi150.stdout:3/23: mknod c4 0 2024-03-22T10:42:40.454 INFO:tasks.workunit.client.0.smithi115.stdout:1/444: readlink d0/d8/de/d1a/l60 0 2024-03-22T10:42:40.454 INFO:tasks.workunit.client.1.smithi150.stdout:2/15: dread f2 [0,4194304] 0 2024-03-22T10:42:40.457 INFO:tasks.workunit.client.0.smithi115.stdout:9/404: dwrite d6/d7/f66 [0,4194304] 0 2024-03-22T10:42:40.457 INFO:tasks.workunit.client.1.smithi150.stdout:9/6: mkdir d1 0 2024-03-22T10:42:40.457 INFO:tasks.workunit.client.0.smithi115.stdout:3/315: rename d3/d19/d2b/f42 to d3/f5b 0 2024-03-22T10:42:40.459 INFO:tasks.workunit.client.1.smithi150.stdout:3/24: mkdir d5 0 2024-03-22T10:42:40.460 INFO:tasks.workunit.client.1.smithi150.stdout:3/25: stat d5 0 2024-03-22T10:42:40.461 INFO:tasks.workunit.client.0.smithi115.stdout:5/338: creat d2/f71 x:0 0 0 2024-03-22T10:42:40.461 INFO:tasks.workunit.client.0.smithi115.stdout:2/302: unlink da/d1c/l3e 0 2024-03-22T10:42:40.462 INFO:tasks.workunit.client.0.smithi115.stdout:8/530: dwrite d6/dd/d24/d4a/f66 [0,4194304] 0 2024-03-22T10:42:40.463 INFO:tasks.workunit.client.1.smithi150.stdout:8/3: getdents d0 0 2024-03-22T10:42:40.463 INFO:tasks.workunit.client.1.smithi150.stdout:8/4: dwrite - no filename 2024-03-22T10:42:40.463 INFO:tasks.workunit.client.1.smithi150.stdout:2/16: dread f2 [0,4194304] 0 2024-03-22T10:42:40.466 INFO:tasks.workunit.client.0.smithi115.stdout:2/303: fdatasync da/f1b 0 2024-03-22T10:42:40.466 INFO:tasks.workunit.client.0.smithi115.stdout:7/347: mknod d2/d7/d12/d10/d19/d42/c79 0 2024-03-22T10:42:40.471 INFO:tasks.workunit.client.0.smithi115.stdout:1/445: read - d0/f7a zero size 2024-03-22T10:42:40.471 INFO:tasks.workunit.client.1.smithi150.stdout:2/17: creat f4 x:0 0 0 2024-03-22T10:42:40.472 INFO:tasks.workunit.client.0.smithi115.stdout:3/316: write d3/f45 [703453,68518] 0 2024-03-22T10:42:40.475 INFO:tasks.workunit.client.1.smithi150.stdout:9/7: mkdir d1/d2 0 2024-03-22T10:42:40.478 INFO:tasks.workunit.client.0.smithi115.stdout:1/446: write d0/d22/f46 [2125758,34110] 0 2024-03-22T10:42:40.479 INFO:tasks.workunit.client.1.smithi150.stdout:2/18: truncate f1 488792 0 2024-03-22T10:42:40.481 INFO:tasks.workunit.client.0.smithi115.stdout:2/304: rmdir da 39 2024-03-22T10:42:40.483 INFO:tasks.workunit.client.1.smithi150.stdout:8/5: creat d0/f1 x:0 0 0 2024-03-22T10:42:40.484 INFO:tasks.workunit.client.0.smithi115.stdout:7/348: creat d2/d4c/f7a x:0 0 0 2024-03-22T10:42:40.485 INFO:tasks.workunit.client.0.smithi115.stdout:9/405: rename d6/d7/d20/f2f to d6/d7/d87/f8c 0 2024-03-22T10:42:40.487 INFO:tasks.workunit.client.0.smithi115.stdout:5/339: dwrite d2/d15/d1b/d33/d38/f5b [0,4194304] 0 2024-03-22T10:42:40.489 INFO:tasks.workunit.client.1.smithi150.stdout:2/19: mknod c5 0 2024-03-22T10:42:40.492 INFO:tasks.workunit.client.1.smithi150.stdout:7/8: dwrite f0 [0,4194304] 0 2024-03-22T10:42:40.492 INFO:tasks.workunit.client.0.smithi115.stdout:8/531: creat d6/fad x:0 0 0 2024-03-22T10:42:40.493 INFO:tasks.workunit.client.0.smithi115.stdout:5/340: creat d2/df/d3b/f72 x:0 0 0 2024-03-22T10:42:40.495 INFO:tasks.workunit.client.0.smithi115.stdout:3/317: creat d3/d36/f5c x:0 0 0 2024-03-22T10:42:40.496 INFO:tasks.workunit.client.0.smithi115.stdout:9/406: mknod d6/d7/d20/d22/c8d 0 2024-03-22T10:42:40.497 INFO:tasks.workunit.client.0.smithi115.stdout:7/349: creat d2/d7/d12/d10/f7b x:0 0 0 2024-03-22T10:42:40.498 INFO:tasks.workunit.client.1.smithi150.stdout:2/20: creat f6 x:0 0 0 2024-03-22T10:42:40.498 INFO:tasks.workunit.client.1.smithi150.stdout:7/9: mkdir d2 0 2024-03-22T10:42:40.500 INFO:tasks.workunit.client.0.smithi115.stdout:7/350: chown d2/d7/d12/d10/d19/d42/d46/f52 8292475 1 2024-03-22T10:42:40.501 INFO:tasks.workunit.client.0.smithi115.stdout:3/318: symlink d3/d19/d2b/l5d 0 2024-03-22T10:42:40.505 INFO:tasks.workunit.client.1.smithi150.stdout:2/21: dread f1 [0,4194304] 0 2024-03-22T10:42:40.506 INFO:tasks.workunit.client.0.smithi115.stdout:1/447: write d0/d12/d17/f39 [318594,69699] 0 2024-03-22T10:42:40.508 INFO:tasks.workunit.client.0.smithi115.stdout:9/407: mkdir d6/d8/d3c/d59/d68/d8e 0 2024-03-22T10:42:40.509 INFO:tasks.workunit.client.1.smithi150.stdout:0/14: dwrite d0/d1/d2/f3 [0,4194304] 0 2024-03-22T10:42:40.510 INFO:tasks.workunit.client.0.smithi115.stdout:3/319: symlink d3/d19/d1b/d57/l5e 0 2024-03-22T10:42:40.511 INFO:tasks.workunit.client.0.smithi115.stdout:5/341: getdents d2/d51 0 2024-03-22T10:42:40.514 INFO:tasks.workunit.client.0.smithi115.stdout:9/408: chown d6/d7/d20/d22 1707162 1 2024-03-22T10:42:40.516 INFO:tasks.workunit.client.0.smithi115.stdout:1/448: mknod d0/d12/d2b/c93 0 2024-03-22T10:42:40.518 INFO:tasks.workunit.client.0.smithi115.stdout:9/409: chown d6/d7/d30 179393 1 2024-03-22T10:42:40.521 INFO:tasks.workunit.client.0.smithi115.stdout:5/342: write d2/d15/d1b/d33/f63 [131123,36593] 0 2024-03-22T10:42:40.522 INFO:tasks.workunit.client.0.smithi115.stdout:3/320: read d3/d36/d41/f48 [1963562,13001] 0 2024-03-22T10:42:40.522 INFO:tasks.workunit.client.0.smithi115.stdout:1/449: chown d0/d12/f1c 1705 1 2024-03-22T10:42:40.524 INFO:tasks.workunit.client.0.smithi115.stdout:2/305: dwrite da/fb [0,4194304] 0 2024-03-22T10:42:40.531 INFO:tasks.workunit.client.0.smithi115.stdout:5/343: chown d2/f5e 284 1 2024-03-22T10:42:40.531 INFO:tasks.workunit.client.0.smithi115.stdout:9/410: getdents d6/d8 0 2024-03-22T10:42:40.531 INFO:tasks.workunit.client.0.smithi115.stdout:3/321: chown d3/d16/l1e 44 1 2024-03-22T10:42:40.535 INFO:tasks.workunit.client.0.smithi115.stdout:8/532: dwrite d6/dd/d24/d4a/f5f [4194304,4194304] 0 2024-03-22T10:42:40.541 INFO:tasks.workunit.client.1.smithi150.stdout:7/10: dwrite f0 [0,4194304] 0 2024-03-22T10:42:40.542 INFO:tasks.workunit.client.1.smithi150.stdout:8/6: dwrite d0/f1 [0,4194304] 0 2024-03-22T10:42:40.545 INFO:tasks.workunit.client.0.smithi115.stdout:1/450: rename d0/d8/db/l21 to d0/d12/d17/d37/l94 0 2024-03-22T10:42:40.546 INFO:tasks.workunit.client.0.smithi115.stdout:8/533: read d6/f44 [3135203,39805] 0 2024-03-22T10:42:40.550 INFO:tasks.workunit.client.0.smithi115.stdout:5/344: link d2/df/l26 d2/d15/d1b/d33/d38/d4f/l73 0 2024-03-22T10:42:40.552 INFO:tasks.workunit.client.0.smithi115.stdout:8/534: readlink d6/dd/d24/l2d 0 2024-03-22T10:42:40.560 INFO:tasks.workunit.client.1.smithi150.stdout:7/11: link f0 d2/f3 0 2024-03-22T10:42:40.562 INFO:tasks.workunit.client.1.smithi150.stdout:0/15: dread d0/d1/d2/f3 [0,4194304] 0 2024-03-22T10:42:40.563 INFO:tasks.workunit.client.0.smithi115.stdout:1/451: unlink d0/d12/d17/l92 0 2024-03-22T10:42:40.564 INFO:tasks.workunit.client.1.smithi150.stdout:0/16: stat d0/d1/d2/f3 0 2024-03-22T10:42:40.568 INFO:tasks.workunit.client.0.smithi115.stdout:9/411: dwrite d6/d7/d30/d3b/f58 [0,4194304] 0 2024-03-22T10:42:40.571 INFO:tasks.workunit.client.1.smithi150.stdout:7/12: unlink d2/f3 0 2024-03-22T10:42:40.571 INFO:tasks.workunit.client.1.smithi150.stdout:0/17: symlink d0/d1/l4 0 2024-03-22T10:42:40.572 INFO:tasks.workunit.client.0.smithi115.stdout:1/452: truncate d0/d12/f66 813543 0 2024-03-22T10:42:40.576 INFO:tasks.workunit.client.1.smithi150.stdout:0/18: write d0/d1/d2/f3 [608527,31776] 0 2024-03-22T10:42:40.577 INFO:tasks.workunit.client.0.smithi115.stdout:9/412: rmdir d6/d8/d3c 39 2024-03-22T10:42:40.578 INFO:tasks.workunit.client.1.smithi150.stdout:0/19: rename d0 to d0/d5 22 2024-03-22T10:42:40.580 INFO:tasks.workunit.client.0.smithi115.stdout:1/453: getdents d0/d12/d17/d67 0 2024-03-22T10:42:40.584 INFO:tasks.workunit.client.0.smithi115.stdout:8/535: dwrite d6/dd/d24/d4a/f61 [0,4194304] 0 2024-03-22T10:42:40.584 INFO:tasks.workunit.client.1.smithi150.stdout:0/20: read d0/d1/d2/f3 [3115826,77975] 0 2024-03-22T10:42:40.586 INFO:tasks.workunit.client.0.smithi115.stdout:5/345: dwrite d2/d15/d1b/d33/d38/f61 [0,4194304] 0 2024-03-22T10:42:40.588 INFO:tasks.workunit.client.0.smithi115.stdout:9/413: rename d6/d8/f1c to d6/d7/d30/d3b/f8f 0 2024-03-22T10:42:40.590 INFO:tasks.workunit.client.0.smithi115.stdout:8/536: chown d6/dd/d24/d4a/d7b/d8f/da0/cab 220316541 1 2024-03-22T10:42:40.592 INFO:tasks.workunit.client.0.smithi115.stdout:8/537: read - d6/dd/d24/f9d zero size 2024-03-22T10:42:40.592 INFO:tasks.workunit.client.1.smithi150.stdout:4/5: getdents d0 0 2024-03-22T10:42:40.592 INFO:tasks.workunit.client.1.smithi150.stdout:4/6: fdatasync - no filename 2024-03-22T10:42:40.595 INFO:tasks.workunit.client.0.smithi115.stdout:8/538: truncate d6/fad 1020248 0 2024-03-22T10:42:40.601 INFO:tasks.workunit.client.1.smithi150.stdout:7/13: dread f0 [0,4194304] 0 2024-03-22T10:42:40.601 INFO:tasks.workunit.client.1.smithi150.stdout:6/5: link l1 l2 0 2024-03-22T10:42:40.603 INFO:tasks.workunit.client.0.smithi115.stdout:8/539: rmdir d6/dd/d24/d4a/d7b/d8f/da0 39 2024-03-22T10:42:40.606 INFO:tasks.workunit.client.0.smithi115.stdout:5/346: rmdir d2/df/d13/d68 39 2024-03-22T10:42:40.606 INFO:tasks.workunit.client.1.smithi150.stdout:2/22: getdents . 0 2024-03-22T10:42:40.607 INFO:tasks.workunit.client.1.smithi150.stdout:4/7: link d0/c2 d0/c3 0 2024-03-22T10:42:40.607 INFO:tasks.workunit.client.1.smithi150.stdout:4/8: write - no filename 2024-03-22T10:42:40.607 INFO:tasks.workunit.client.1.smithi150.stdout:4/9: write - no filename 2024-03-22T10:42:40.607 INFO:tasks.workunit.client.1.smithi150.stdout:4/10: truncate - no filename 2024-03-22T10:42:40.608 INFO:tasks.workunit.client.0.smithi115.stdout:2/306: dread da/d25/f40 [0,4194304] 0 2024-03-22T10:42:40.609 INFO:tasks.workunit.client.1.smithi150.stdout:2/23: chown f6 2 1 2024-03-22T10:42:40.612 INFO:tasks.workunit.client.0.smithi115.stdout:2/307: write da/f1b [1165688,120868] 0 2024-03-22T10:42:40.614 INFO:tasks.workunit.client.1.smithi150.stdout:2/24: mknod c7 0 2024-03-22T10:42:40.619 INFO:tasks.workunit.client.0.smithi115.stdout:2/308: mknod da/d25/d3d/d57/c5c 0 2024-03-22T10:42:40.619 INFO:tasks.workunit.client.1.smithi150.stdout:2/25: dread f2 [0,4194304] 0 2024-03-22T10:42:40.620 INFO:tasks.workunit.client.0.smithi115.stdout:9/414: creat d6/d8/d3c/d59/f90 x:0 0 0 2024-03-22T10:42:40.621 INFO:tasks.workunit.client.0.smithi115.stdout:8/540: dread d6/dd/d12/f94 [0,4194304] 0 2024-03-22T10:42:40.622 INFO:tasks.workunit.client.1.smithi150.stdout:8/7: truncate d0/f1 2552696 0 2024-03-22T10:42:40.623 INFO:tasks.workunit.client.1.smithi150.stdout:0/21: dwrite d0/d1/d2/f3 [0,4194304] 0 2024-03-22T10:42:40.623 INFO:tasks.workunit.client.1.smithi150.stdout:2/26: fsync f1 0 2024-03-22T10:42:40.623 INFO:tasks.workunit.client.0.smithi115.stdout:3/322: dwrite d3/f7 [0,4194304] 0 2024-03-22T10:42:40.624 INFO:tasks.workunit.client.1.smithi150.stdout:2/27: chown c3 250968660 1 2024-03-22T10:42:40.628 INFO:tasks.workunit.client.1.smithi150.stdout:2/28: dread f1 [0,4194304] 0 2024-03-22T10:42:40.629 INFO:tasks.workunit.client.0.smithi115.stdout:2/309: creat da/d25/d3d/d57/f5d x:0 0 0 2024-03-22T10:42:40.629 INFO:tasks.workunit.client.0.smithi115.stdout:9/415: rename d6/d8/l80 to d6/d5e/l91 0 2024-03-22T10:42:40.629 INFO:tasks.workunit.client.1.smithi150.stdout:0/22: creat d0/f6 x:0 0 0 2024-03-22T10:42:40.630 INFO:tasks.workunit.client.0.smithi115.stdout:8/541: mknod d6/dd/d24/d3a/cae 0 2024-03-22T10:42:40.630 INFO:tasks.workunit.client.0.smithi115.stdout:9/416: stat d6/d7/f5d 0 2024-03-22T10:42:40.631 INFO:tasks.workunit.client.0.smithi115.stdout:9/417: chown d6/d7/d48/f50 0 1 2024-03-22T10:42:40.632 INFO:tasks.workunit.client.0.smithi115.stdout:3/323: mknod d3/d19/d1b/d52/c5f 0 2024-03-22T10:42:40.635 INFO:tasks.workunit.client.0.smithi115.stdout:4/352: sync 2024-03-22T10:42:40.635 INFO:tasks.workunit.client.0.smithi115.stdout:0/358: sync 2024-03-22T10:42:40.635 INFO:tasks.workunit.client.0.smithi115.stdout:6/359: sync 2024-03-22T10:42:40.637 INFO:tasks.workunit.client.1.smithi150.stdout:0/23: unlink d0/f6 0 2024-03-22T10:42:40.639 INFO:tasks.workunit.client.0.smithi115.stdout:2/310: mkdir da/d25/d30/d4f/d54/d5e 0 2024-03-22T10:42:40.640 INFO:tasks.workunit.client.1.smithi150.stdout:4/11: getdents d0 0 2024-03-22T10:42:40.640 INFO:tasks.workunit.client.1.smithi150.stdout:4/12: dwrite - no filename 2024-03-22T10:42:40.641 INFO:tasks.workunit.client.1.smithi150.stdout:4/13: dwrite - no filename 2024-03-22T10:42:40.641 INFO:tasks.workunit.client.1.smithi150.stdout:4/14: dwrite - no filename 2024-03-22T10:42:40.642 INFO:tasks.workunit.client.1.smithi150.stdout:0/24: rmdir d0/d1 39 2024-03-22T10:42:40.642 INFO:tasks.workunit.client.0.smithi115.stdout:9/418: mknod d6/d8/d3c/d59/d75/c92 0 2024-03-22T10:42:40.643 INFO:tasks.workunit.client.0.smithi115.stdout:6/360: dread - d4/d36/d48/d72/f76 zero size 2024-03-22T10:42:40.643 INFO:tasks.workunit.client.1.smithi150.stdout:8/8: write d0/f1 [306853,103764] 0 2024-03-22T10:42:40.644 INFO:tasks.workunit.client.0.smithi115.stdout:8/542: dread - d6/d49/d88/f8a zero size 2024-03-22T10:42:40.646 INFO:tasks.workunit.client.0.smithi115.stdout:2/311: dread - da/d25/d3d/d57/f5d zero size 2024-03-22T10:42:40.647 INFO:tasks.workunit.client.0.smithi115.stdout:3/324: link d3/d19/d2b/f4d d3/d19/d2b/d49/f60 0 2024-03-22T10:42:40.648 INFO:tasks.workunit.client.1.smithi150.stdout:8/9: mkdir d0/d2 0 2024-03-22T10:42:40.651 INFO:tasks.workunit.client.1.smithi150.stdout:4/15: rename d0/c2 to d0/c4 0 2024-03-22T10:42:40.651 INFO:tasks.workunit.client.1.smithi150.stdout:4/16: fdatasync - no filename 2024-03-22T10:42:40.651 INFO:tasks.workunit.client.1.smithi150.stdout:4/17: readlink - no filename 2024-03-22T10:42:40.651 INFO:tasks.workunit.client.1.smithi150.stdout:0/25: truncate d0/d1/d2/f3 4653701 0 2024-03-22T10:42:40.652 INFO:tasks.workunit.client.1.smithi150.stdout:8/10: creat d0/f3 x:0 0 0 2024-03-22T10:42:40.652 INFO:tasks.workunit.client.0.smithi115.stdout:9/419: rename d6/d8/d3c/d59/c64 to d6/d7/d20/d22/d3f/c93 0 2024-03-22T10:42:40.653 INFO:tasks.workunit.client.1.smithi150.stdout:2/29: dwrite f4 [0,4194304] 0 2024-03-22T10:42:40.654 INFO:tasks.workunit.client.0.smithi115.stdout:3/325: mkdir d3/d19/d61 0 2024-03-22T10:42:40.654 INFO:tasks.workunit.client.0.smithi115.stdout:1/454: dread d0/d12/d17/f3e [0,4194304] 0 2024-03-22T10:42:40.655 INFO:tasks.workunit.client.1.smithi150.stdout:4/18: mknod d0/c5 0 2024-03-22T10:42:40.655 INFO:tasks.workunit.client.1.smithi150.stdout:4/19: read - no filename 2024-03-22T10:42:40.657 INFO:tasks.workunit.client.1.smithi150.stdout:2/30: symlink l8 0 2024-03-22T10:42:40.658 INFO:tasks.workunit.client.1.smithi150.stdout:8/11: mknod d0/d2/c4 0 2024-03-22T10:42:40.658 INFO:tasks.workunit.client.0.smithi115.stdout:8/543: rename d6/dd/d12/l9e to d6/dd/d24/d4a/laf 0 2024-03-22T10:42:40.660 INFO:tasks.workunit.client.1.smithi150.stdout:4/20: rename d0/c1 to d0/c6 0 2024-03-22T10:42:40.661 INFO:tasks.workunit.client.0.smithi115.stdout:3/326: write d3/d16/d29/f39 [4187153,874] 0 2024-03-22T10:42:40.663 INFO:tasks.workunit.client.1.smithi150.stdout:8/12: rename d0/f3 to d0/f5 0 2024-03-22T10:42:40.664 INFO:tasks.workunit.client.1.smithi150.stdout:4/21: creat d0/f7 x:0 0 0 2024-03-22T10:42:40.667 INFO:tasks.workunit.client.0.smithi115.stdout:1/455: rmdir d0/d8/de/d1a 39 2024-03-22T10:42:40.668 INFO:tasks.workunit.client.1.smithi150.stdout:8/13: creat d0/f6 x:0 0 0 2024-03-22T10:42:40.668 INFO:tasks.workunit.client.1.smithi150.stdout:4/22: creat d0/f8 x:0 0 0 2024-03-22T10:42:40.670 INFO:tasks.workunit.client.0.smithi115.stdout:3/327: read d3/f7 [3470625,6043] 0 2024-03-22T10:42:40.672 INFO:tasks.workunit.client.1.smithi150.stdout:8/14: rename d0/d2/c4 to d0/c7 0 2024-03-22T10:42:40.673 INFO:tasks.workunit.client.0.smithi115.stdout:2/312: truncate da/d25/d3d/f2c 4681825 0 2024-03-22T10:42:40.674 INFO:tasks.workunit.client.0.smithi115.stdout:0/359: dwrite d3/d7/d1c/f77 [0,4194304] 0 2024-03-22T10:42:40.674 INFO:tasks.workunit.client.1.smithi150.stdout:4/23: unlink d0/c3 0 2024-03-22T10:42:40.677 INFO:tasks.workunit.client.0.smithi115.stdout:1/456: write d0/d8/db/d25/f73 [408382,9527] 0 2024-03-22T10:42:40.677 INFO:tasks.workunit.client.0.smithi115.stdout:5/347: dwrite d2/d15/d1b/d33/d38/f3d [0,4194304] 0 2024-03-22T10:42:40.679 INFO:tasks.workunit.client.0.smithi115.stdout:0/360: fsync d3/d7/d1b/d1f/d5b/f66 0 2024-03-22T10:42:40.680 INFO:tasks.workunit.client.1.smithi150.stdout:4/24: creat d0/f9 x:0 0 0 2024-03-22T10:42:40.682 INFO:tasks.workunit.client.1.smithi150.stdout:2/31: getdents . 0 2024-03-22T10:42:40.682 INFO:tasks.workunit.client.0.smithi115.stdout:6/361: dwrite d4/d11/d12/d1c/f6a [0,4194304] 0 2024-03-22T10:42:40.683 INFO:tasks.workunit.client.0.smithi115.stdout:4/353: dwrite d1/d16/d35/d40/d5c/d14/f58 [0,4194304] 0 2024-03-22T10:42:40.684 INFO:tasks.workunit.client.1.smithi150.stdout:0/26: truncate d0/d1/d2/f3 1465159 0 2024-03-22T10:42:40.684 INFO:tasks.workunit.client.0.smithi115.stdout:6/362: readlink d4/d13/d33/l4d 0 2024-03-22T10:42:40.686 INFO:tasks.workunit.client.0.smithi115.stdout:6/363: fdatasync d4/d13/d33/d58/f75 0 2024-03-22T10:42:40.686 INFO:tasks.workunit.client.1.smithi150.stdout:4/25: symlink d0/la 0 2024-03-22T10:42:40.689 INFO:tasks.workunit.client.1.smithi150.stdout:2/32: rename c5 to c9 0 2024-03-22T10:42:40.690 INFO:tasks.workunit.client.1.smithi150.stdout:4/26: truncate d0/f9 5878 0 2024-03-22T10:42:40.691 INFO:tasks.workunit.client.0.smithi115.stdout:3/328: fdatasync d3/f30 0 2024-03-22T10:42:40.694 INFO:tasks.workunit.client.0.smithi115.stdout:3/329: truncate d3/d36/f51 1250167 0 2024-03-22T10:42:40.695 INFO:tasks.workunit.client.1.smithi150.stdout:4/27: fdatasync d0/f8 0 2024-03-22T10:42:40.696 INFO:tasks.workunit.client.0.smithi115.stdout:7/351: sync 2024-03-22T10:42:40.697 INFO:tasks.workunit.client.0.smithi115.stdout:8/544: dwrite d6/fb [0,4194304] 0 2024-03-22T10:42:40.697 INFO:tasks.workunit.client.0.smithi115.stdout:7/352: stat d2/l3c 0 2024-03-22T10:42:40.697 INFO:tasks.workunit.client.1.smithi150.stdout:2/33: write f6 [1031989,105443] 0 2024-03-22T10:42:40.703 INFO:tasks.workunit.client.1.smithi150.stdout:8/15: dwrite d0/f6 [0,4194304] 0 2024-03-22T10:42:40.704 INFO:tasks.workunit.client.1.smithi150.stdout:4/28: mknod d0/cb 0 2024-03-22T10:42:40.705 INFO:tasks.workunit.client.0.smithi115.stdout:4/354: mkdir d1/d30/d78 0 2024-03-22T10:42:40.712 INFO:tasks.workunit.client.0.smithi115.stdout:9/420: link d6/d7/d20/f4c d6/d8/d3c/d59/d68/f94 0 2024-03-22T10:42:40.712 INFO:tasks.workunit.client.0.smithi115.stdout:4/355: write d1/d16/d35/d40/d5c/f6f [1327948,8463] 0 2024-03-22T10:42:40.714 INFO:tasks.workunit.client.1.smithi150.stdout:0/27: fsync d0/d1/d2/f3 0 2024-03-22T10:42:40.715 INFO:tasks.workunit.client.0.smithi115.stdout:2/313: mknod da/d25/d30/c5f 0 2024-03-22T10:42:40.716 INFO:tasks.workunit.client.1.smithi150.stdout:4/29: dread d0/f9 [0,4194304] 0 2024-03-22T10:42:40.718 INFO:tasks.workunit.client.1.smithi150.stdout:4/30: dread - d0/f8 zero size 2024-03-22T10:42:40.720 INFO:tasks.workunit.client.1.smithi150.stdout:4/31: dread - d0/f7 zero size 2024-03-22T10:42:40.721 INFO:tasks.workunit.client.0.smithi115.stdout:4/356: chown d1/d30/d42/d32/l3c 0 1 2024-03-22T10:42:40.721 INFO:tasks.workunit.client.0.smithi115.stdout:6/364: dwrite d4/d13/f5f [0,4194304] 0 2024-03-22T10:42:40.722 INFO:tasks.workunit.client.1.smithi150.stdout:2/34: getdents . 0 2024-03-22T10:42:40.726 INFO:tasks.workunit.client.1.smithi150.stdout:8/16: dread d0/f1 [0,4194304] 0 2024-03-22T10:42:40.727 INFO:tasks.workunit.client.0.smithi115.stdout:0/361: dread d3/d4/d14/f15 [4194304,4194304] 0 2024-03-22T10:42:40.727 INFO:tasks.workunit.client.0.smithi115.stdout:2/314: chown da/d25/d3d/d57/l58 547857 1 2024-03-22T10:42:40.729 INFO:tasks.workunit.client.0.smithi115.stdout:6/365: mknod d4/d11/d12/d1c/c79 0 2024-03-22T10:42:40.734 INFO:tasks.workunit.client.0.smithi115.stdout:8/545: creat d6/dd/d12/fb0 x:0 0 0 2024-03-22T10:42:40.734 INFO:tasks.workunit.client.0.smithi115.stdout:5/348: getdents d2/d15/d1b/d33/d38/d4a 0 2024-03-22T10:42:40.735 INFO:tasks.workunit.client.0.smithi115.stdout:2/315: mknod da/d25/d2a/c60 0 2024-03-22T10:42:40.735 INFO:tasks.workunit.client.0.smithi115.stdout:3/330: dwrite d3/f53 [0,4194304] 0 2024-03-22T10:42:40.736 INFO:tasks.workunit.client.1.smithi150.stdout:4/32: mknod d0/cc 0 2024-03-22T10:42:40.736 INFO:tasks.workunit.client.1.smithi150.stdout:4/33: readlink d0/la 0 2024-03-22T10:42:40.739 INFO:tasks.workunit.client.1.smithi150.stdout:8/17: fdatasync d0/f1 0 2024-03-22T10:42:40.740 INFO:tasks.workunit.client.0.smithi115.stdout:8/546: write d6/dd/d24/f72 [4558922,92335] 0 2024-03-22T10:42:40.740 INFO:tasks.workunit.client.0.smithi115.stdout:9/421: rename d6/d5e/f5f to d6/d7/d20/d22/f95 0 2024-03-22T10:42:40.741 INFO:tasks.workunit.client.0.smithi115.stdout:9/422: dread - d6/f62 zero size 2024-03-22T10:42:40.742 INFO:tasks.workunit.client.0.smithi115.stdout:5/349: read d2/d15/d1b/d33/d38/d43/d49/f52 [452575,128132] 0 2024-03-22T10:42:40.743 INFO:tasks.workunit.client.1.smithi150.stdout:0/28: dwrite d0/d1/d2/f3 [0,4194304] 0 2024-03-22T10:42:40.743 INFO:tasks.workunit.client.1.smithi150.stdout:4/34: rename d0/la to d0/ld 0 2024-03-22T10:42:40.743 INFO:tasks.workunit.client.1.smithi150.stdout:8/18: dread - d0/f5 zero size 2024-03-22T10:42:40.744 INFO:tasks.workunit.client.0.smithi115.stdout:0/362: read d3/d7/d1b/f24 [871126,72196] 0 2024-03-22T10:42:40.744 INFO:tasks.workunit.client.0.smithi115.stdout:6/366: write d4/d11/d12/f31 [154391,106703] 0 2024-03-22T10:42:40.747 INFO:tasks.workunit.client.0.smithi115.stdout:8/547: dread - d6/d49/d68/f9c zero size 2024-03-22T10:42:40.749 INFO:tasks.workunit.client.0.smithi115.stdout:0/363: chown d3/l21 0 1 2024-03-22T10:42:40.749 INFO:tasks.workunit.client.1.smithi150.stdout:4/35: mkdir d0/de 0 2024-03-22T10:42:40.749 INFO:tasks.workunit.client.1.smithi150.stdout:8/19: creat d0/f8 x:0 0 0 2024-03-22T10:42:40.751 INFO:tasks.workunit.client.1.smithi150.stdout:4/36: chown d0/c5 0 1 2024-03-22T10:42:40.751 INFO:tasks.workunit.client.0.smithi115.stdout:6/367: read d4/d11/d12/d1c/f69 [1330792,61572] 0 2024-03-22T10:42:40.752 INFO:tasks.workunit.client.0.smithi115.stdout:0/364: fdatasync d3/d7/d1b/d1f/f58 0 2024-03-22T10:42:40.752 INFO:tasks.workunit.client.0.smithi115.stdout:4/357: fsync d1/d16/d35/d40/d5c/d14/f1d 0 2024-03-22T10:42:40.752 INFO:tasks.workunit.client.1.smithi150.stdout:8/20: write d0/f5 [598359,48563] 0 2024-03-22T10:42:40.753 INFO:tasks.workunit.client.0.smithi115.stdout:0/365: dread - d3/da/d4c/f67 zero size 2024-03-22T10:42:40.754 INFO:tasks.workunit.client.1.smithi150.stdout:8/21: stat d0/f5 0 2024-03-22T10:42:40.754 INFO:tasks.workunit.client.0.smithi115.stdout:5/350: mkdir d2/d15/d1b/d33/d38/d4a/d74 0 2024-03-22T10:42:40.757 INFO:tasks.workunit.client.1.smithi150.stdout:4/37: rename d0/cb to d0/de/cf 0 2024-03-22T10:42:40.758 INFO:tasks.workunit.client.0.smithi115.stdout:8/548: symlink d6/d49/d68/lb1 0 2024-03-22T10:42:40.758 INFO:tasks.workunit.client.1.smithi150.stdout:8/22: symlink d0/l9 0 2024-03-22T10:42:40.759 INFO:tasks.workunit.client.0.smithi115.stdout:4/358: mkdir d1/d16/d68/d79 0 2024-03-22T10:42:40.759 INFO:tasks.workunit.client.1.smithi150.stdout:4/38: write d0/f8 [974616,123297] 0 2024-03-22T10:42:40.760 INFO:tasks.workunit.client.1.smithi150.stdout:8/23: write d0/f6 [3761334,54259] 0 2024-03-22T10:42:40.763 INFO:tasks.workunit.client.0.smithi115.stdout:0/366: creat d3/d4/d14/f7b x:0 0 0 2024-03-22T10:42:40.767 INFO:tasks.workunit.client.1.smithi150.stdout:2/35: write f1 [365108,126710] 0 2024-03-22T10:42:40.767 INFO:tasks.workunit.client.1.smithi150.stdout:4/39: symlink d0/l10 0 2024-03-22T10:42:40.769 INFO:tasks.workunit.client.0.smithi115.stdout:0/367: write d3/d4/f70 [462821,16888] 0 2024-03-22T10:42:40.772 INFO:tasks.workunit.client.0.smithi115.stdout:1/457: dwrite d0/f8f [0,4194304] 0 2024-03-22T10:42:40.773 INFO:tasks.workunit.client.0.smithi115.stdout:9/423: dwrite d6/d7/f66 [0,4194304] 0 2024-03-22T10:42:40.773 INFO:tasks.workunit.client.0.smithi115.stdout:0/368: dread - d3/d7/d1b/d35/f79 zero size 2024-03-22T10:42:40.774 INFO:tasks.workunit.client.1.smithi150.stdout:4/40: creat d0/f11 x:0 0 0 2024-03-22T10:42:40.775 INFO:tasks.workunit.client.0.smithi115.stdout:9/424: chown d6/c1b 593352 1 2024-03-22T10:42:40.781 INFO:tasks.workunit.client.1.smithi150.stdout:4/41: creat d0/de/f12 x:0 0 0 2024-03-22T10:42:40.783 INFO:tasks.workunit.client.0.smithi115.stdout:0/369: symlink d3/d7/d1b/d1f/d5b/l7c 0 2024-03-22T10:42:40.787 INFO:tasks.workunit.client.0.smithi115.stdout:8/549: getdents d6/dd/d24/d3a 0 2024-03-22T10:42:40.788 INFO:tasks.workunit.client.0.smithi115.stdout:1/458: creat d0/d12/d17/d67/f95 x:0 0 0 2024-03-22T10:42:40.791 INFO:tasks.workunit.client.1.smithi150.stdout:0/29: fsync d0/d1/d2/f3 0 2024-03-22T10:42:40.792 INFO:tasks.workunit.client.1.smithi150.stdout:2/36: fdatasync f1 0 2024-03-22T10:42:40.793 INFO:tasks.workunit.client.1.smithi150.stdout:8/24: dwrite d0/f6 [4194304,4194304] 0 2024-03-22T10:42:40.795 INFO:tasks.workunit.client.0.smithi115.stdout:7/353: dread d2/d7/d12/d10/f17 [0,4194304] 0 2024-03-22T10:42:40.796 INFO:tasks.workunit.client.1.smithi150.stdout:0/30: write d0/d1/d2/f3 [3958904,49100] 0 2024-03-22T10:42:40.797 INFO:tasks.workunit.client.0.smithi115.stdout:5/351: dwrite d2/df/d3b/f72 [0,4194304] 0 2024-03-22T10:42:40.797 INFO:tasks.workunit.client.0.smithi115.stdout:9/425: dread d6/d7/d20/d38/f40 [0,4194304] 0 2024-03-22T10:42:40.798 INFO:tasks.workunit.client.0.smithi115.stdout:3/331: truncate d3/d36/d41/f3f 139204 0 2024-03-22T10:42:40.799 INFO:tasks.workunit.client.1.smithi150.stdout:8/25: creat d0/d2/fa x:0 0 0 2024-03-22T10:42:40.799 INFO:tasks.workunit.client.1.smithi150.stdout:2/37: getdents . 0 2024-03-22T10:42:40.802 INFO:tasks.workunit.client.0.smithi115.stdout:3/332: read d3/f30 [1812777,32241] 0 2024-03-22T10:42:40.803 INFO:tasks.workunit.client.0.smithi115.stdout:9/426: write d6/d7/f2d [1139408,115040] 0 2024-03-22T10:42:40.809 INFO:tasks.workunit.client.0.smithi115.stdout:3/333: mkdir d3/d36/d41/d62 0 2024-03-22T10:42:40.809 INFO:tasks.workunit.client.1.smithi150.stdout:4/42: dread d0/f8 [0,4194304] 0 2024-03-22T10:42:40.811 INFO:tasks.workunit.client.1.smithi150.stdout:8/26: creat d0/fb x:0 0 0 2024-03-22T10:42:40.812 INFO:tasks.workunit.client.0.smithi115.stdout:1/459: dread d0/f8f [0,4194304] 0 2024-03-22T10:42:40.813 INFO:tasks.workunit.client.1.smithi150.stdout:0/31: dread d0/d1/d2/f3 [0,4194304] 0 2024-03-22T10:42:40.815 INFO:tasks.workunit.client.1.smithi150.stdout:4/43: read d0/f9 [2858,115966] 0 2024-03-22T10:42:40.816 INFO:tasks.workunit.client.1.smithi150.stdout:8/27: mkdir d0/dc 0 2024-03-22T10:42:40.817 INFO:tasks.workunit.client.0.smithi115.stdout:6/368: truncate d4/d11/d12/d1c/f43 2642663 0 2024-03-22T10:42:40.821 INFO:tasks.workunit.client.0.smithi115.stdout:0/370: dread d3/da/d4c/d68/f5f [0,4194304] 0 2024-03-22T10:42:40.824 INFO:tasks.workunit.client.1.smithi150.stdout:8/28: creat d0/fd x:0 0 0 2024-03-22T10:42:40.824 INFO:tasks.workunit.client.0.smithi115.stdout:3/334: mknod d3/d19/d1b/d57/c63 0 2024-03-22T10:42:40.825 INFO:tasks.workunit.client.1.smithi150.stdout:0/32: unlink d0/d1/d2/f3 0 2024-03-22T10:42:40.825 INFO:tasks.workunit.client.0.smithi115.stdout:0/371: chown d3/d7/d1c/d6c 100 1 2024-03-22T10:42:40.826 INFO:tasks.workunit.client.1.smithi150.stdout:4/44: creat d0/de/f13 x:0 0 0 2024-03-22T10:42:40.827 INFO:tasks.workunit.client.1.smithi150.stdout:0/33: stat d0/d1/d2 0 2024-03-22T10:42:40.827 INFO:tasks.workunit.client.0.smithi115.stdout:9/427: rename d6/fa to d6/d8/d3c/f96 0 2024-03-22T10:42:40.828 INFO:tasks.workunit.client.1.smithi150.stdout:0/34: chown d0/d1/d2 71245 1 2024-03-22T10:42:40.828 INFO:tasks.workunit.client.1.smithi150.stdout:4/45: write d0/f7 [742570,89178] 0 2024-03-22T10:42:40.829 INFO:tasks.workunit.client.0.smithi115.stdout:6/369: mkdir d4/d36/d48/d7a 0 2024-03-22T10:42:40.831 INFO:tasks.workunit.client.0.smithi115.stdout:5/352: dwrite d2/d15/f39 [0,4194304] 0 2024-03-22T10:42:40.831 INFO:tasks.workunit.client.1.smithi150.stdout:8/29: rename d0/f8 to d0/d2/fe 0 2024-03-22T10:42:40.832 INFO:tasks.workunit.client.1.smithi150.stdout:4/46: creat d0/f14 x:0 0 0 2024-03-22T10:42:40.833 INFO:tasks.workunit.client.0.smithi115.stdout:5/353: chown d2/d15/d1b/d33/d38/d43 443792916 1 2024-03-22T10:42:40.834 INFO:tasks.workunit.client.0.smithi115.stdout:0/372: unlink d3/d7/c6 0 2024-03-22T10:42:40.834 INFO:tasks.workunit.client.1.smithi150.stdout:4/47: chown d0/de 173647 1 2024-03-22T10:42:40.837 INFO:tasks.workunit.client.1.smithi150.stdout:4/48: symlink d0/l15 0 2024-03-22T10:42:40.838 INFO:tasks.workunit.client.0.smithi115.stdout:8/550: truncate d6/dd/d12/f3b 2445268 0 2024-03-22T10:42:40.839 INFO:tasks.workunit.client.0.smithi115.stdout:3/335: creat d3/d19/d1b/f64 x:0 0 0 2024-03-22T10:42:40.841 INFO:tasks.workunit.client.1.smithi150.stdout:4/49: write d0/de/f13 [608118,80214] 0 2024-03-22T10:42:40.843 INFO:tasks.workunit.client.0.smithi115.stdout:1/460: write d0/d8/de/d1a/f32 [3727043,121507] 0 2024-03-22T10:42:40.843 INFO:tasks.workunit.client.0.smithi115.stdout:4/359: dread d1/d30/d42/f34 [0,4194304] 0 2024-03-22T10:42:40.845 INFO:tasks.workunit.client.0.smithi115.stdout:9/428: rename d6/d8/d86 to d6/d8/d3c/d59/d68/d97 0 2024-03-22T10:42:40.846 INFO:tasks.workunit.client.0.smithi115.stdout:3/336: dread d3/d36/f3e [0,4194304] 0 2024-03-22T10:42:40.853 INFO:tasks.workunit.client.0.smithi115.stdout:1/461: mknod d0/d8/d4d/d56/c96 0 2024-03-22T10:42:40.854 INFO:tasks.workunit.client.0.smithi115.stdout:5/354: dread d2/d15/d1b/d33/f63 [0,4194304] 0 2024-03-22T10:42:40.856 INFO:tasks.workunit.client.0.smithi115.stdout:5/355: chown d2/f4 2 1 2024-03-22T10:42:40.856 INFO:tasks.workunit.client.1.smithi150.stdout:2/38: dwrite f2 [0,4194304] 0 2024-03-22T10:42:40.857 INFO:tasks.workunit.client.1.smithi150.stdout:0/35: rename d0/d1/d2 to d0/d1/d7 0 2024-03-22T10:42:40.858 INFO:tasks.workunit.client.0.smithi115.stdout:3/337: symlink d3/d16/d29/d35/l65 0 2024-03-22T10:42:40.858 INFO:tasks.workunit.client.0.smithi115.stdout:4/360: creat d1/d16/d35/d40/d54/f7a x:0 0 0 2024-03-22T10:42:40.861 INFO:tasks.workunit.client.1.smithi150.stdout:8/30: dwrite d0/f1 [0,4194304] 0 2024-03-22T10:42:40.861 INFO:tasks.workunit.client.0.smithi115.stdout:4/361: write d1/f9 [4170111,121810] 0 2024-03-22T10:42:40.864 INFO:tasks.workunit.client.0.smithi115.stdout:5/356: truncate d2/d15/d1b/d33/f63 1076027 0 2024-03-22T10:42:40.864 INFO:tasks.workunit.client.1.smithi150.stdout:2/39: unlink c3 0 2024-03-22T10:42:40.868 INFO:tasks.workunit.client.1.smithi150.stdout:8/31: rmdir d0 39 2024-03-22T10:42:40.872 INFO:tasks.workunit.client.1.smithi150.stdout:2/40: mkdir da 0 2024-03-22T10:42:40.876 INFO:tasks.workunit.client.1.smithi150.stdout:0/36: creat d0/f8 x:0 0 0 2024-03-22T10:42:40.878 INFO:tasks.workunit.client.0.smithi115.stdout:7/354: dread d2/d4c/f69 [0,4194304] 0 2024-03-22T10:42:40.880 INFO:tasks.workunit.client.0.smithi115.stdout:5/357: creat d2/d15/d1b/d33/d38/d4a/f75 x:0 0 0 2024-03-22T10:42:40.881 INFO:tasks.workunit.client.1.smithi150.stdout:8/32: rmdir d0/d2 39 2024-03-22T10:42:40.882 INFO:tasks.workunit.client.0.smithi115.stdout:3/338: getdents d3/d36/d41/d33 0 2024-03-22T10:42:40.882 INFO:tasks.workunit.client.0.smithi115.stdout:6/370: write d4/d13/f3d [1389267,87782] 0 2024-03-22T10:42:40.883 INFO:tasks.workunit.client.0.smithi115.stdout:5/358: dread - d2/d51/f69 zero size 2024-03-22T10:42:40.884 INFO:tasks.workunit.client.1.smithi150.stdout:8/33: chown d0/f1 451920 1 2024-03-22T10:42:40.888 INFO:tasks.workunit.client.1.smithi150.stdout:0/37: rename d0/f8 to d0/f9 0 2024-03-22T10:42:40.889 INFO:tasks.workunit.client.0.smithi115.stdout:8/551: dwrite d6/dd/d12/d31/f5c [0,4194304] 0 2024-03-22T10:42:40.891 INFO:tasks.workunit.client.1.smithi150.stdout:2/41: dread f4 [0,4194304] 0 2024-03-22T10:42:40.891 INFO:tasks.workunit.client.0.smithi115.stdout:1/462: write d0/f55 [288099,90983] 0 2024-03-22T10:42:40.892 INFO:tasks.workunit.client.0.smithi115.stdout:3/339: symlink d3/d16/d29/d35/l66 0 2024-03-22T10:42:40.894 INFO:tasks.workunit.client.0.smithi115.stdout:7/355: mknod d2/d7/c7c 0 2024-03-22T10:42:40.896 INFO:tasks.workunit.client.0.smithi115.stdout:6/371: symlink d4/d13/d33/d58/l7b 0 2024-03-22T10:42:40.897 INFO:tasks.workunit.client.0.smithi115.stdout:0/373: dread d3/f46 [0,4194304] 0 2024-03-22T10:42:40.897 INFO:tasks.workunit.client.1.smithi150.stdout:0/38: link d0/f9 d0/d1/fa 0 2024-03-22T10:42:40.898 INFO:tasks.workunit.client.0.smithi115.stdout:1/463: creat d0/d12/d2b/f97 x:0 0 0 2024-03-22T10:42:40.899 INFO:tasks.workunit.client.0.smithi115.stdout:3/340: unlink d3/d19/d1b/d57/l5e 0 2024-03-22T10:42:40.901 INFO:tasks.workunit.client.0.smithi115.stdout:7/356: write d2/d7/d12/d10/d19/d42/d46/f5e [88843,43495] 0 2024-03-22T10:42:40.901 INFO:tasks.workunit.client.0.smithi115.stdout:4/362: truncate d1/d16/d35/d40/d5c/f23 4469150 0 2024-03-22T10:42:40.902 INFO:tasks.workunit.client.0.smithi115.stdout:1/464: dread - d0/d8/db/f2d zero size 2024-03-22T10:42:40.905 INFO:tasks.workunit.client.1.smithi150.stdout:2/42: rmdir da 0 2024-03-22T10:42:40.905 INFO:tasks.workunit.client.0.smithi115.stdout:6/372: creat d4/d11/d12/d1c/f7c x:0 0 0 2024-03-22T10:42:40.905 INFO:tasks.workunit.client.0.smithi115.stdout:0/374: rmdir d3/d7/d1c 39 2024-03-22T10:42:40.906 INFO:tasks.workunit.client.0.smithi115.stdout:4/363: chown d1/d16/d35/d40/d5c/ce 772 1 2024-03-22T10:42:40.907 INFO:tasks.workunit.client.0.smithi115.stdout:0/375: fsync d3/f33 0 2024-03-22T10:42:40.908 INFO:tasks.workunit.client.1.smithi150.stdout:0/39: unlink d0/f9 0 2024-03-22T10:42:40.911 INFO:tasks.workunit.client.0.smithi115.stdout:5/359: creat d2/df/d13/f76 x:0 0 0 2024-03-22T10:42:40.911 INFO:tasks.workunit.client.1.smithi150.stdout:2/43: write f4 [217519,117351] 0 2024-03-22T10:42:40.912 INFO:tasks.workunit.client.0.smithi115.stdout:1/465: creat d0/d22/f98 x:0 0 0 2024-03-22T10:42:40.913 INFO:tasks.workunit.client.1.smithi150.stdout:0/40: stat d0/d1/fa 0 2024-03-22T10:42:40.914 INFO:tasks.workunit.client.1.smithi150.stdout:8/34: dwrite d0/fb [0,4194304] 0 2024-03-22T10:42:40.915 INFO:tasks.workunit.client.1.smithi150.stdout:2/44: chown c9 5652231 1 2024-03-22T10:42:40.917 INFO:tasks.workunit.client.0.smithi115.stdout:1/466: chown d0/d8/de/l45 25155 1 2024-03-22T10:42:40.918 INFO:tasks.workunit.client.0.smithi115.stdout:4/364: symlink d1/d16/l7b 0 2024-03-22T10:42:40.920 INFO:tasks.workunit.client.0.smithi115.stdout:7/357: truncate d2/d3a/d6e/f6a 697808 0 2024-03-22T10:42:40.921 INFO:tasks.workunit.client.1.smithi150.stdout:8/35: rmdir d0/d2 39 2024-03-22T10:42:40.921 INFO:tasks.workunit.client.0.smithi115.stdout:8/552: dread d6/f28 [0,4194304] 0 2024-03-22T10:42:40.925 INFO:tasks.workunit.client.0.smithi115.stdout:7/358: symlink d2/d62/l7d 0 2024-03-22T10:42:40.927 INFO:tasks.workunit.client.0.smithi115.stdout:1/467: chown d0/d58/l6a 370837150 1 2024-03-22T10:42:40.930 INFO:tasks.workunit.client.0.smithi115.stdout:4/365: link d1/d16/d68/f6c d1/d30/d42/d4f/f7c 0 2024-03-22T10:42:40.931 INFO:tasks.workunit.client.0.smithi115.stdout:7/359: stat d2/d7/d12/d10/d19/d42/d46/f68 0 2024-03-22T10:42:40.932 INFO:tasks.workunit.client.0.smithi115.stdout:0/376: dread d3/d7/d1b/f24 [0,4194304] 0 2024-03-22T10:42:40.934 INFO:tasks.workunit.client.0.smithi115.stdout:6/373: dwrite d4/d36/f5c [0,4194304] 0 2024-03-22T10:42:40.934 INFO:tasks.workunit.client.0.smithi115.stdout:1/468: fsync d0/d22/f3d 0 2024-03-22T10:42:40.938 INFO:tasks.workunit.client.1.smithi150.stdout:0/41: write d0/d1/fa [256295,35243] 0 2024-03-22T10:42:40.938 INFO:tasks.workunit.client.0.smithi115.stdout:0/377: write d3/da/d4c/d69/f76 [1033110,129260] 0 2024-03-22T10:42:40.941 INFO:tasks.workunit.client.0.smithi115.stdout:7/360: symlink d2/d7/d12/d10/d23/d33/l7e 0 2024-03-22T10:42:40.942 INFO:tasks.workunit.client.0.smithi115.stdout:8/553: link d6/dd/d24/d4a/d7b/c7e d6/dd/d24/d4a/d7b/d8f/cb2 0 2024-03-22T10:42:40.943 INFO:tasks.workunit.client.0.smithi115.stdout:6/374: symlink d4/d11/d12/d1c/l7d 0 2024-03-22T10:42:40.945 INFO:tasks.workunit.client.0.smithi115.stdout:1/469: mknod d0/d12/d17/d37/c99 0 2024-03-22T10:42:40.946 INFO:tasks.workunit.client.0.smithi115.stdout:6/375: dread - d4/d36/d48/d72/f76 zero size 2024-03-22T10:42:40.948 INFO:tasks.workunit.client.0.smithi115.stdout:5/360: truncate d2/df/f55 1057589 0 2024-03-22T10:42:40.948 INFO:tasks.workunit.client.0.smithi115.stdout:6/376: fdatasync d4/d11/d12/d1c/f7c 0 2024-03-22T10:42:40.948 INFO:tasks.workunit.client.0.smithi115.stdout:4/366: mknod d1/d30/d42/d73/c7d 0 2024-03-22T10:42:40.950 INFO:tasks.workunit.client.1.smithi150.stdout:8/36: dwrite d0/f5 [0,4194304] 0 2024-03-22T10:42:40.951 INFO:tasks.workunit.client.1.smithi150.stdout:0/42: mkdir d0/d1/d7/db 0 2024-03-22T10:42:40.952 INFO:tasks.workunit.client.1.smithi150.stdout:0/43: chown d0 350 1 2024-03-22T10:42:40.953 INFO:tasks.workunit.client.0.smithi115.stdout:5/361: write d2/d51/f69 [455663,63990] 0 2024-03-22T10:42:40.954 INFO:tasks.workunit.client.0.smithi115.stdout:1/470: creat d0/d22/d75/d82/f9a x:0 0 0 2024-03-22T10:42:40.956 INFO:tasks.workunit.client.0.smithi115.stdout:8/554: read d6/dd/f33 [242312,111876] 0 2024-03-22T10:42:40.957 INFO:tasks.workunit.client.1.smithi150.stdout:0/44: chown d0/d1/l4 165816 1 2024-03-22T10:42:40.958 INFO:tasks.workunit.client.1.smithi150.stdout:8/37: mkdir d0/dc/df 0 2024-03-22T10:42:40.959 INFO:tasks.workunit.client.0.smithi115.stdout:4/367: creat d1/d16/d35/d40/d54/d5e/f7e x:0 0 0 2024-03-22T10:42:40.959 INFO:tasks.workunit.client.0.smithi115.stdout:1/471: rmdir d0/d8/db/d25 39 2024-03-22T10:42:40.962 INFO:tasks.workunit.client.0.smithi115.stdout:6/377: symlink d4/d11/l7e 0 2024-03-22T10:42:40.963 INFO:tasks.workunit.client.1.smithi150.stdout:0/45: dread d0/d1/fa [0,4194304] 0 2024-03-22T10:42:40.965 INFO:tasks.workunit.client.1.smithi150.stdout:8/38: mknod d0/d2/c10 0 2024-03-22T10:42:40.965 INFO:tasks.workunit.client.0.smithi115.stdout:1/472: symlink d0/d22/d75/l9b 0 2024-03-22T10:42:40.968 INFO:tasks.workunit.client.1.smithi150.stdout:0/46: creat d0/d1/d7/db/fc x:0 0 0 2024-03-22T10:42:40.968 INFO:tasks.workunit.client.0.smithi115.stdout:4/368: symlink d1/d16/d35/d40/d54/l7f 0 2024-03-22T10:42:40.970 INFO:tasks.workunit.client.0.smithi115.stdout:4/369: fsync d1/d30/d42/f45 0 2024-03-22T10:42:40.970 INFO:tasks.workunit.client.0.smithi115.stdout:5/362: link d2/df/d13/f16 d2/d15/d1b/d33/d38/f77 0 2024-03-22T10:42:40.970 INFO:tasks.workunit.client.1.smithi150.stdout:8/39: symlink d0/l11 0 2024-03-22T10:42:40.971 INFO:tasks.workunit.client.0.smithi115.stdout:7/361: dread d2/d7/d12/d10/d19/d42/f6d [0,4194304] 0 2024-03-22T10:42:40.974 INFO:tasks.workunit.client.0.smithi115.stdout:0/378: write d3/d4/f71 [981471,112790] 0 2024-03-22T10:42:40.979 INFO:tasks.workunit.client.0.smithi115.stdout:8/555: rename d6/dd/d12/ca4 to d6/cb3 0 2024-03-22T10:42:40.980 INFO:tasks.workunit.client.0.smithi115.stdout:7/362: creat d2/d7/d12/d10/d19/d42/d46/f7f x:0 0 0 2024-03-22T10:42:40.982 INFO:tasks.workunit.client.0.smithi115.stdout:0/379: chown d3/d4/c2c 1086 1 2024-03-22T10:42:40.984 INFO:tasks.workunit.client.0.smithi115.stdout:6/378: dwrite d4/d36/d48/f67 [0,4194304] 0 2024-03-22T10:42:40.994 INFO:tasks.workunit.client.0.smithi115.stdout:5/363: rename d2/df/d13/f16 to d2/d15/d1b/d33/d38/f78 0 2024-03-22T10:42:40.995 INFO:tasks.workunit.client.0.smithi115.stdout:6/379: read d4/f30 [3471614,3235] 0 2024-03-22T10:42:40.995 INFO:tasks.workunit.client.0.smithi115.stdout:4/370: dwrite d1/d16/d35/d40/d5c/d14/f3e [4194304,4194304] 0 2024-03-22T10:42:40.998 INFO:tasks.workunit.client.0.smithi115.stdout:7/363: creat d2/d7/d12/d10/d23/d3d/f80 x:0 0 0 2024-03-22T10:42:40.999 INFO:tasks.workunit.client.0.smithi115.stdout:5/364: read - d2/d15/d1b/d33/d38/d4a/f75 zero size 2024-03-22T10:42:41.001 INFO:tasks.workunit.client.1.smithi150.stdout:0/47: dwrite d0/d1/d7/db/fc [0,4194304] 0 2024-03-22T10:42:41.004 INFO:tasks.workunit.client.1.smithi150.stdout:8/40: dwrite d0/fb [0,4194304] 0 2024-03-22T10:42:41.007 INFO:tasks.workunit.client.1.smithi150.stdout:0/48: mknod d0/d1/cd 0 2024-03-22T10:42:41.009 INFO:tasks.workunit.client.0.smithi115.stdout:7/364: creat d2/f81 x:0 0 0 2024-03-22T10:42:41.009 INFO:tasks.workunit.client.0.smithi115.stdout:6/380: getdents d4/d36/d48/d7a 0 2024-03-22T10:42:41.010 INFO:tasks.workunit.client.1.smithi150.stdout:8/41: link d0/d2/fa d0/d2/f12 0 2024-03-22T10:42:41.011 INFO:tasks.workunit.client.1.smithi150.stdout:8/42: dread - d0/fd zero size 2024-03-22T10:42:41.012 INFO:tasks.workunit.client.0.smithi115.stdout:5/365: write d2/fe [1582717,8083] 0 2024-03-22T10:42:41.014 INFO:tasks.workunit.client.1.smithi150.stdout:0/49: creat d0/d1/d7/fe x:0 0 0 2024-03-22T10:42:41.015 INFO:tasks.workunit.client.0.smithi115.stdout:7/365: write d2/d7/d12/d10/d23/f6f [51712,8603] 0 2024-03-22T10:42:41.017 INFO:tasks.workunit.client.1.smithi150.stdout:0/50: dread - d0/d1/d7/fe zero size 2024-03-22T10:42:41.018 INFO:tasks.workunit.client.0.smithi115.stdout:4/371: getdents d1/d16/d3b 0 2024-03-22T10:42:41.019 INFO:tasks.workunit.client.0.smithi115.stdout:0/380: dwrite d3/d4/f70 [0,4194304] 0 2024-03-22T10:42:41.026 INFO:tasks.workunit.client.0.smithi115.stdout:5/366: rename d2/cd to d2/d15/d1b/d33/c79 0 2024-03-22T10:42:41.030 INFO:tasks.workunit.client.1.smithi150.stdout:8/43: dread d0/f1 [0,4194304] 0 2024-03-22T10:42:41.031 INFO:tasks.workunit.client.0.smithi115.stdout:7/366: symlink d2/d7/d12/d10/d19/d42/l82 0 2024-03-22T10:42:41.031 INFO:tasks.workunit.client.0.smithi115.stdout:4/372: mknod d1/d69/d49/c80 0 2024-03-22T10:42:41.033 INFO:tasks.workunit.client.1.smithi150.stdout:8/44: rmdir d0/d2 39 2024-03-22T10:42:41.036 INFO:tasks.workunit.client.1.smithi150.stdout:8/45: stat d0/d2 0 2024-03-22T10:42:41.039 INFO:tasks.workunit.client.0.smithi115.stdout:0/381: link d3/d7/d1b/l29 d3/da/d4c/l7d 0 2024-03-22T10:42:41.039 INFO:tasks.workunit.client.0.smithi115.stdout:4/373: dread - d1/d16/d35/d40/d54/d5e/f7e zero size 2024-03-22T10:42:41.041 INFO:tasks.workunit.client.0.smithi115.stdout:8/556: dread d6/dd/d24/d4a/f80 [0,4194304] 0 2024-03-22T10:42:41.042 INFO:tasks.workunit.client.0.smithi115.stdout:6/381: dwrite d4/d36/f53 [0,4194304] 0 2024-03-22T10:42:41.043 INFO:tasks.workunit.client.0.smithi115.stdout:7/367: readlink d2/d7/d12/d10/d19/d42/d46/l47 0 2024-03-22T10:42:41.043 INFO:tasks.workunit.client.0.smithi115.stdout:5/367: getdents d2/d15/d1b/d33/d38/d4f 0 2024-03-22T10:42:41.044 INFO:tasks.workunit.client.0.smithi115.stdout:8/557: fdatasync d6/dd/f92 0 2024-03-22T10:42:41.044 INFO:tasks.workunit.client.1.smithi150.stdout:0/51: fsync d0/d1/d7/db/fc 0 2024-03-22T10:42:41.048 INFO:tasks.workunit.client.0.smithi115.stdout:0/382: unlink d3/d4/d14/f7b 0 2024-03-22T10:42:41.059 INFO:tasks.workunit.client.0.smithi115.stdout:6/382: chown d4/d11/d12/d1c/l40 77370 1 2024-03-22T10:42:41.060 INFO:tasks.workunit.client.0.smithi115.stdout:6/383: readlink d4/d11/l77 0 2024-03-22T10:42:41.061 INFO:tasks.workunit.client.0.smithi115.stdout:0/383: rename d3/da/f26 to d3/d7/d1b/d35/f7e 0 2024-03-22T10:42:41.061 INFO:tasks.workunit.client.1.smithi150.stdout:0/52: dwrite d0/d1/d7/fe [0,4194304] 0 2024-03-22T10:42:41.062 INFO:tasks.workunit.client.0.smithi115.stdout:0/384: stat d3/d52/c61 0 2024-03-22T10:42:41.063 INFO:tasks.workunit.client.1.smithi150.stdout:0/53: read d0/d1/d7/db/fc [2253131,7715] 0 2024-03-22T10:42:41.063 INFO:tasks.workunit.client.0.smithi115.stdout:4/374: dwrite d1/d69/f62 [0,4194304] 0 2024-03-22T10:42:41.063 INFO:tasks.workunit.client.0.smithi115.stdout:5/368: mkdir d2/df/d27/d7a 0 2024-03-22T10:42:41.065 INFO:tasks.workunit.client.0.smithi115.stdout:5/369: chown d2/df/d3b 482 1 2024-03-22T10:42:41.066 INFO:tasks.workunit.client.0.smithi115.stdout:0/385: write d3/d4/f70 [2911583,25913] 0 2024-03-22T10:42:41.068 INFO:tasks.workunit.client.0.smithi115.stdout:4/375: write d1/d30/d42/f45 [906695,55167] 0 2024-03-22T10:42:41.077 INFO:tasks.workunit.client.0.smithi115.stdout:6/384: rename d4/f10 to d4/d11/d12/d1c/f7f 0 2024-03-22T10:42:41.080 INFO:tasks.workunit.client.0.smithi115.stdout:6/385: chown d4/le 28651439 1 2024-03-22T10:42:41.082 INFO:tasks.workunit.client.1.smithi150.stdout:0/54: dwrite d0/d1/d7/db/fc [0,4194304] 0 2024-03-22T10:42:41.082 INFO:tasks.workunit.client.0.smithi115.stdout:5/370: creat d2/df/d3b/f7b x:0 0 0 2024-03-22T10:42:41.087 INFO:tasks.workunit.client.0.smithi115.stdout:7/368: link d2/d7/f13 d2/d7/d12/d10/d19/d42/d76/f83 0 2024-03-22T10:42:41.091 INFO:tasks.workunit.client.0.smithi115.stdout:7/369: chown d2/d7/d12/d10/d19/l73 84 1 2024-03-22T10:42:41.091 INFO:tasks.workunit.client.0.smithi115.stdout:8/558: link d6/dd/d12/c38 d6/dd/d24/d4a/d7b/cb4 0 2024-03-22T10:42:41.096 INFO:tasks.workunit.client.0.smithi115.stdout:0/386: rename d3/d4/f70 to d3/f7f 0 2024-03-22T10:42:41.097 INFO:tasks.workunit.client.0.smithi115.stdout:5/371: rename d2/d15/d1b/d33 to d2/d15/d1b/d7c 0 2024-03-22T10:42:41.100 INFO:tasks.workunit.client.0.smithi115.stdout:8/559: mknod d6/dd/d24/d4a/cb5 0 2024-03-22T10:42:41.102 INFO:tasks.workunit.client.0.smithi115.stdout:8/560: chown d6/dd/l82 1057 1 2024-03-22T10:42:41.105 INFO:tasks.workunit.client.0.smithi115.stdout:7/370: rmdir d2/d7/d12/d10/d19 39 2024-03-22T10:42:41.105 INFO:tasks.workunit.client.0.smithi115.stdout:5/372: mknod d2/df/d27/d50/c7d 0 2024-03-22T10:42:41.106 INFO:tasks.workunit.client.0.smithi115.stdout:4/376: truncate d1/d30/d42/d32/f53 2041833 0 2024-03-22T10:42:41.108 INFO:tasks.workunit.client.1.smithi150.stdout:0/55: fsync d0/d1/d7/db/fc 0 2024-03-22T10:42:41.109 INFO:tasks.workunit.client.1.smithi150.stdout:0/56: stat d0/d1/d7/db/fc 0 2024-03-22T10:42:41.111 INFO:tasks.workunit.client.0.smithi115.stdout:8/561: creat d6/dd/fb6 x:0 0 0 2024-03-22T10:42:41.112 INFO:tasks.workunit.client.1.smithi150.stdout:0/57: creat d0/d1/d7/ff x:0 0 0 2024-03-22T10:42:41.112 INFO:tasks.workunit.client.0.smithi115.stdout:6/386: dread d4/d11/f52 [0,4194304] 0 2024-03-22T10:42:41.113 INFO:tasks.workunit.client.1.smithi150.stdout:0/58: fsync d0/d1/d7/db/fc 0 2024-03-22T10:42:41.115 INFO:tasks.workunit.client.0.smithi115.stdout:8/562: creat d6/d49/d68/fb7 x:0 0 0 2024-03-22T10:42:41.117 INFO:tasks.workunit.client.0.smithi115.stdout:7/371: rename d2/d7/d12/d10/d23/d3d/f80 to d2/d7/d12/d10/d23/f84 0 2024-03-22T10:42:41.119 INFO:tasks.workunit.client.0.smithi115.stdout:8/563: stat d6/dd/d24/d4a/f87 0 2024-03-22T10:42:41.122 INFO:tasks.workunit.client.1.smithi150.stdout:0/59: dread d0/d1/d7/db/fc [0,4194304] 0 2024-03-22T10:42:41.122 INFO:tasks.workunit.client.0.smithi115.stdout:0/387: dwrite d3/d7/f3b [0,4194304] 0 2024-03-22T10:42:41.124 INFO:tasks.workunit.client.0.smithi115.stdout:0/388: read d3/d4/d14/f15 [4419453,110017] 0 2024-03-22T10:42:41.125 INFO:tasks.workunit.client.0.smithi115.stdout:5/373: dwrite d2/d15/f39 [4194304,4194304] 0 2024-03-22T10:42:41.125 INFO:tasks.workunit.client.1.smithi150.stdout:0/60: mkdir d0/d1/d7/db/d10 0 2024-03-22T10:42:41.127 INFO:tasks.workunit.client.0.smithi115.stdout:0/389: stat d3/d4/d14/f34 0 2024-03-22T10:42:41.129 INFO:tasks.workunit.client.0.smithi115.stdout:8/564: unlink d6/dd/f29 0 2024-03-22T10:42:41.132 INFO:tasks.workunit.client.1.smithi150.stdout:0/61: mkdir d0/d1/d7/d11 0 2024-03-22T10:42:41.135 INFO:tasks.workunit.client.0.smithi115.stdout:0/390: creat d3/d7/d1b/d35/f80 x:0 0 0 2024-03-22T10:42:41.137 INFO:tasks.workunit.client.0.smithi115.stdout:8/565: rmdir d6/dd/d24/d3a 39 2024-03-22T10:42:41.138 INFO:tasks.workunit.client.1.smithi150.stdout:0/62: write d0/d1/fa [1301122,118925] 0 2024-03-22T10:42:41.138 INFO:tasks.workunit.client.0.smithi115.stdout:6/387: dwrite d4/d11/f23 [0,4194304] 0 2024-03-22T10:42:41.140 INFO:tasks.workunit.client.1.smithi150.stdout:0/63: fsync d0/d1/d7/db/fc 0 2024-03-22T10:42:41.142 INFO:tasks.workunit.client.0.smithi115.stdout:7/372: link d2/d7/d12/d10/d19/f1a d2/d7/d12/d10/d19/d42/f85 0 2024-03-22T10:42:41.145 INFO:tasks.workunit.client.0.smithi115.stdout:7/373: readlink d2/d62/l7d 0 2024-03-22T10:42:41.146 INFO:tasks.workunit.client.1.smithi150.stdout:0/64: mkdir d0/d12 0 2024-03-22T10:42:41.151 INFO:tasks.workunit.client.0.smithi115.stdout:8/566: creat d6/dd/d24/d4a/d7b/d8f/da0/fb8 x:0 0 0 2024-03-22T10:42:41.156 INFO:tasks.workunit.client.0.smithi115.stdout:8/567: rmdir d6/d49/d68 39 2024-03-22T10:42:41.156 INFO:tasks.workunit.client.0.smithi115.stdout:0/391: dwrite d3/d4/d14/d64/f72 [0,4194304] 0 2024-03-22T10:42:41.160 INFO:tasks.workunit.client.0.smithi115.stdout:7/374: dread d2/d7/d12/f67 [4194304,4194304] 0 2024-03-22T10:42:41.161 INFO:tasks.workunit.client.0.smithi115.stdout:0/392: write d3/d7/d1b/d35/f80 [634019,38651] 0 2024-03-22T10:42:41.162 INFO:tasks.workunit.client.1.smithi150.stdout:0/65: truncate d0/d1/fa 331916 0 2024-03-22T10:42:41.162 INFO:tasks.workunit.client.1.smithi150.stdout:0/66: chown d0/d1 5 1 2024-03-22T10:42:41.163 INFO:tasks.workunit.client.0.smithi115.stdout:6/388: dwrite d4/d11/d12/f4e [0,4194304] 0 2024-03-22T10:42:41.165 INFO:tasks.workunit.client.0.smithi115.stdout:5/374: dwrite d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:41.166 INFO:tasks.workunit.client.0.smithi115.stdout:6/389: chown d4/d11/c15 3827 1 2024-03-22T10:42:41.167 INFO:tasks.workunit.client.1.smithi150.stdout:0/67: creat d0/d1/d7/db/d10/f13 x:0 0 0 2024-03-22T10:42:41.170 INFO:tasks.workunit.client.0.smithi115.stdout:7/375: symlink d2/d3a/d6e/d78/l86 0 2024-03-22T10:42:41.174 INFO:tasks.workunit.client.0.smithi115.stdout:5/375: stat d2/df/d13/l25 0 2024-03-22T10:42:41.177 INFO:tasks.workunit.client.0.smithi115.stdout:5/376: fsync d2/d51/f65 0 2024-03-22T10:42:41.180 INFO:tasks.workunit.client.0.smithi115.stdout:6/390: rename d4/d11/d12/f5d to d4/d11/d12/f80 0 2024-03-22T10:42:41.181 INFO:tasks.workunit.client.0.smithi115.stdout:5/377: write d2/d51/f65 [34423,37275] 0 2024-03-22T10:42:41.185 INFO:tasks.workunit.client.0.smithi115.stdout:6/391: write d4/d11/d12/d1c/f2d [2165477,43920] 0 2024-03-22T10:42:41.194 INFO:tasks.workunit.client.0.smithi115.stdout:8/568: dwrite d6/d49/f84 [0,4194304] 0 2024-03-22T10:42:41.198 INFO:tasks.workunit.client.1.smithi150.stdout:0/68: dread d0/d1/d7/fe [0,4194304] 0 2024-03-22T10:42:41.198 INFO:tasks.workunit.client.0.smithi115.stdout:5/378: rename d2/df/f3c to d2/d15/d1b/d7c/d38/d4a/d74/f7e 0 2024-03-22T10:42:41.201 INFO:tasks.workunit.client.0.smithi115.stdout:7/376: dread d2/d7/d12/d10/f17 [0,4194304] 0 2024-03-22T10:42:41.203 INFO:tasks.workunit.client.0.smithi115.stdout:0/393: dwrite d3/d7/d1c/f77 [0,4194304] 0 2024-03-22T10:42:41.207 INFO:tasks.workunit.client.0.smithi115.stdout:6/392: dwrite d4/d13/f25 [4194304,4194304] 0 2024-03-22T10:42:41.209 INFO:tasks.workunit.client.0.smithi115.stdout:7/377: mknod d2/d3a/c87 0 2024-03-22T10:42:41.210 INFO:tasks.workunit.client.0.smithi115.stdout:7/378: chown d2/d7/d12/d10/f26 1908 1 2024-03-22T10:42:41.212 INFO:tasks.workunit.client.0.smithi115.stdout:8/569: getdents d6/dd/d12 0 2024-03-22T10:42:41.214 INFO:tasks.workunit.client.0.smithi115.stdout:6/393: creat d4/d36/d48/d72/f81 x:0 0 0 2024-03-22T10:42:41.217 INFO:tasks.workunit.client.0.smithi115.stdout:7/379: creat d2/d7/d12/d10/d19/d42/d46/d61/f88 x:0 0 0 2024-03-22T10:42:41.221 INFO:tasks.workunit.client.0.smithi115.stdout:6/394: rename d4/d36/c64 to d4/d13/d33/c82 0 2024-03-22T10:42:41.223 INFO:tasks.workunit.client.0.smithi115.stdout:8/570: getdents d6/dd/d12/d31 0 2024-03-22T10:42:41.226 INFO:tasks.workunit.client.0.smithi115.stdout:8/571: fsync d6/dd/d12/f3b 0 2024-03-22T10:42:41.226 INFO:tasks.workunit.client.0.smithi115.stdout:5/379: dwrite d2/df/d3b/f7b [0,4194304] 0 2024-03-22T10:42:41.228 INFO:tasks.workunit.client.0.smithi115.stdout:5/380: fdatasync d2/d51/f65 0 2024-03-22T10:42:41.229 INFO:tasks.workunit.client.0.smithi115.stdout:7/380: rename d2/d7/d12/d10/d23/d3d/c2d to d2/d7/d12/d10/d19/d42/d46/c89 0 2024-03-22T10:42:41.238 INFO:tasks.workunit.client.0.smithi115.stdout:7/381: mknod d2/d3a/d6e/d78/c8a 0 2024-03-22T10:42:41.241 INFO:tasks.workunit.client.0.smithi115.stdout:6/395: link d4/d11/d12/d1c/c41 d4/d36/c83 0 2024-03-22T10:42:41.243 INFO:tasks.workunit.client.0.smithi115.stdout:6/396: write d4/d13/f3d [146456,77868] 0 2024-03-22T10:42:41.244 INFO:tasks.workunit.client.0.smithi115.stdout:2/316: sync 2024-03-22T10:42:41.246 INFO:tasks.workunit.client.0.smithi115.stdout:6/397: write d4/d11/d12/f4e [3471719,99815] 0 2024-03-22T10:42:41.249 INFO:tasks.workunit.client.0.smithi115.stdout:2/317: write da/d25/d3d/f28 [999348,85043] 0 2024-03-22T10:42:41.250 INFO:tasks.workunit.client.0.smithi115.stdout:6/398: read d4/d11/d12/f21 [587961,37468] 0 2024-03-22T10:42:41.255 INFO:tasks.workunit.client.0.smithi115.stdout:6/399: write d4/d11/d12/f21 [3126064,51097] 0 2024-03-22T10:42:41.258 INFO:tasks.workunit.client.0.smithi115.stdout:5/381: dwrite d2/df/d13/f18 [4194304,4194304] 0 2024-03-22T10:42:41.261 INFO:tasks.workunit.client.0.smithi115.stdout:7/382: dwrite d2/d7/d12/d10/f26 [4194304,4194304] 0 2024-03-22T10:42:41.261 INFO:tasks.workunit.client.0.smithi115.stdout:6/400: mkdir d4/d11/d12/d1c/d84 0 2024-03-22T10:42:41.264 INFO:tasks.workunit.client.0.smithi115.stdout:6/401: stat d4/d11/f54 0 2024-03-22T10:42:41.265 INFO:tasks.workunit.client.0.smithi115.stdout:8/572: dread d6/f44 [0,4194304] 0 2024-03-22T10:42:41.267 INFO:tasks.workunit.client.0.smithi115.stdout:6/402: write d4/d36/d48/d72/f81 [175659,15230] 0 2024-03-22T10:42:41.269 INFO:tasks.workunit.client.0.smithi115.stdout:8/573: stat d6/dd/d24/d4a/f80 0 2024-03-22T10:42:41.273 INFO:tasks.workunit.client.0.smithi115.stdout:5/382: creat d2/d15/d1b/f7f x:0 0 0 2024-03-22T10:42:41.275 INFO:tasks.workunit.client.0.smithi115.stdout:8/574: chown d6/dd/d24/d3a 94 1 2024-03-22T10:42:41.282 INFO:tasks.workunit.client.0.smithi115.stdout:7/383: dread d2/d7/d12/d10/d19/d42/f6d [0,4194304] 0 2024-03-22T10:42:41.282 INFO:tasks.workunit.client.0.smithi115.stdout:5/383: unlink d2/df/d27/l44 0 2024-03-22T10:42:41.286 INFO:tasks.workunit.client.0.smithi115.stdout:8/575: rename d6/dd/d12/l3d to d6/dd/d12/d31/lb9 0 2024-03-22T10:42:41.286 INFO:tasks.workunit.client.0.smithi115.stdout:2/318: dwrite da/d25/d3d/f2c [0,4194304] 0 2024-03-22T10:42:41.287 INFO:tasks.workunit.client.0.smithi115.stdout:5/384: fdatasync d2/d15/d1b/f7f 0 2024-03-22T10:42:41.288 INFO:tasks.workunit.client.0.smithi115.stdout:6/403: dwrite d4/d11/d12/f4e [0,4194304] 0 2024-03-22T10:42:41.291 INFO:tasks.workunit.client.0.smithi115.stdout:2/319: write da/f15 [126725,56640] 0 2024-03-22T10:42:41.293 INFO:tasks.workunit.client.0.smithi115.stdout:5/385: mknod d2/d15/d1b/c80 0 2024-03-22T10:42:41.296 INFO:tasks.workunit.client.0.smithi115.stdout:7/384: rename d2/d7/d12/d10/d23/d33/f58 to d2/d7/d12/d10/d19/f8b 0 2024-03-22T10:42:41.299 INFO:tasks.workunit.client.0.smithi115.stdout:6/404: symlink d4/d5e/l85 0 2024-03-22T10:42:41.301 INFO:tasks.workunit.client.0.smithi115.stdout:2/320: rename da/d25/d2a/c38 to da/d25/d51/c61 0 2024-03-22T10:42:41.302 INFO:tasks.workunit.client.0.smithi115.stdout:8/576: creat d6/dd/d24/d4a/d7b/fba x:0 0 0 2024-03-22T10:42:41.306 INFO:tasks.workunit.client.0.smithi115.stdout:2/321: mknod da/d25/d51/c62 0 2024-03-22T10:42:41.308 INFO:tasks.workunit.client.0.smithi115.stdout:8/577: write d6/d49/d68/fb7 [167778,87154] 0 2024-03-22T10:42:41.310 INFO:tasks.workunit.client.0.smithi115.stdout:5/386: dread d2/d15/d1b/d7c/d38/f3d [0,4194304] 0 2024-03-22T10:42:41.313 INFO:tasks.workunit.client.0.smithi115.stdout:8/578: symlink d6/lbb 0 2024-03-22T10:42:41.315 INFO:tasks.workunit.client.0.smithi115.stdout:5/387: rmdir d2/d15/d1b/d7c/d38 39 2024-03-22T10:42:41.318 INFO:tasks.workunit.client.0.smithi115.stdout:6/405: dwrite d4/d11/f78 [0,4194304] 0 2024-03-22T10:42:41.320 INFO:tasks.workunit.client.1.smithi150.stdout:0/69: symlink d0/d12/l14 0 2024-03-22T10:42:41.321 INFO:tasks.workunit.client.0.smithi115.stdout:7/385: write d2/d4c/f69 [4001295,103990] 0 2024-03-22T10:42:41.322 INFO:tasks.workunit.client.0.smithi115.stdout:8/579: mkdir d6/dbc 0 2024-03-22T10:42:41.323 INFO:tasks.workunit.client.1.smithi150.stdout:0/70: dread d0/d1/fa [0,4194304] 0 2024-03-22T10:42:41.325 INFO:tasks.workunit.client.0.smithi115.stdout:8/580: read - d6/dd/d24/d4a/d7b/fba zero size 2024-03-22T10:42:41.328 INFO:tasks.workunit.client.0.smithi115.stdout:2/322: dwrite da/d25/d3d/f4e [0,4194304] 0 2024-03-22T10:42:41.328 INFO:tasks.workunit.client.0.smithi115.stdout:5/388: unlink d2/d15/d1b/d7c/d38/d43/d49/l6f 0 2024-03-22T10:42:41.331 INFO:tasks.workunit.client.0.smithi115.stdout:6/406: mknod d4/d36/d48/d7a/c86 0 2024-03-22T10:42:41.331 INFO:tasks.workunit.client.0.smithi115.stdout:7/386: mknod d2/d4c/c8c 0 2024-03-22T10:42:41.333 INFO:tasks.workunit.client.0.smithi115.stdout:5/389: readlink d2/d15/d1b/d7c/d38/d43/d49/l4b 0 2024-03-22T10:42:41.334 INFO:tasks.workunit.client.0.smithi115.stdout:2/323: rmdir da/d25/d30/d4f 39 2024-03-22T10:42:41.335 INFO:tasks.workunit.client.0.smithi115.stdout:6/407: symlink d4/d13/d33/l87 0 2024-03-22T10:42:41.339 INFO:tasks.workunit.client.0.smithi115.stdout:8/581: symlink d6/dd/d24/d4a/d7b/lbd 0 2024-03-22T10:42:41.345 INFO:tasks.workunit.client.1.smithi150.stdout:0/71: dwrite d0/d1/d7/db/d10/f13 [0,4194304] 0 2024-03-22T10:42:41.350 INFO:tasks.workunit.client.0.smithi115.stdout:5/390: getdents d2/d15/d1b 0 2024-03-22T10:42:41.350 INFO:tasks.workunit.client.0.smithi115.stdout:8/582: rename d6/d49/l73 to d6/d49/d88/lbe 0 2024-03-22T10:42:41.353 INFO:tasks.workunit.client.0.smithi115.stdout:7/387: dwrite d2/f81 [0,4194304] 0 2024-03-22T10:42:41.357 INFO:tasks.workunit.client.0.smithi115.stdout:8/583: fsync d6/fa 0 2024-03-22T10:42:41.357 INFO:tasks.workunit.client.0.smithi115.stdout:7/388: dread - d2/d7/f20 zero size 2024-03-22T10:42:41.365 INFO:tasks.workunit.client.0.smithi115.stdout:6/408: dwrite d4/d11/d12/f21 [0,4194304] 0 2024-03-22T10:42:41.366 INFO:tasks.workunit.client.0.smithi115.stdout:8/584: mknod d6/dd/cbf 0 2024-03-22T10:42:41.367 INFO:tasks.workunit.client.0.smithi115.stdout:7/389: rmdir d2/d7/d12/d10/d23/d33 39 2024-03-22T10:42:41.370 INFO:tasks.workunit.client.1.smithi150.stdout:0/72: dwrite d0/d1/d7/db/fc [0,4194304] 0 2024-03-22T10:42:41.371 INFO:tasks.workunit.client.0.smithi115.stdout:6/409: write d4/d13/d33/f42 [1316843,34018] 0 2024-03-22T10:42:41.374 INFO:tasks.workunit.client.0.smithi115.stdout:5/391: dwrite d2/df/f67 [0,4194304] 0 2024-03-22T10:42:41.375 INFO:tasks.workunit.client.0.smithi115.stdout:8/585: unlink d6/dd/d12/c14 0 2024-03-22T10:42:41.377 INFO:tasks.workunit.client.0.smithi115.stdout:7/390: write d2/d7/d12/d10/d23/f6f [481449,42341] 0 2024-03-22T10:42:41.377 INFO:tasks.workunit.client.0.smithi115.stdout:6/410: rmdir d4/d11/d12 39 2024-03-22T10:42:41.378 INFO:tasks.workunit.client.1.smithi150.stdout:0/73: creat d0/d1/d7/d11/f15 x:0 0 0 2024-03-22T10:42:41.382 INFO:tasks.workunit.client.0.smithi115.stdout:5/392: mkdir d2/d15/d1b/d7c/d38/d43/d49/d81 0 2024-03-22T10:42:41.384 INFO:tasks.workunit.client.0.smithi115.stdout:9/429: sync 2024-03-22T10:42:41.384 INFO:tasks.workunit.client.0.smithi115.stdout:3/341: sync 2024-03-22T10:42:41.385 INFO:tasks.workunit.client.0.smithi115.stdout:7/391: creat d2/d7/d12/f8d x:0 0 0 2024-03-22T10:42:41.386 INFO:tasks.workunit.client.0.smithi115.stdout:8/586: creat d6/dd/d24/fc0 x:0 0 0 2024-03-22T10:42:41.387 INFO:tasks.workunit.client.0.smithi115.stdout:6/411: symlink d4/d11/d12/l88 0 2024-03-22T10:42:41.387 INFO:tasks.workunit.client.0.smithi115.stdout:7/392: stat d2/d62/l7d 0 2024-03-22T10:42:41.388 INFO:tasks.workunit.client.0.smithi115.stdout:9/430: chown d6/d7/d19/c72 13 1 2024-03-22T10:42:41.388 INFO:tasks.workunit.client.0.smithi115.stdout:5/393: readlink d2/df/d13/l60 0 2024-03-22T10:42:41.392 INFO:tasks.workunit.client.0.smithi115.stdout:6/412: symlink d4/d36/l89 0 2024-03-22T10:42:41.392 INFO:tasks.workunit.client.0.smithi115.stdout:9/431: dread - d6/d7/d19/d4e/f6c zero size 2024-03-22T10:42:41.395 INFO:tasks.workunit.client.0.smithi115.stdout:7/393: symlink d2/d7/d12/l8e 0 2024-03-22T10:42:41.396 INFO:tasks.workunit.client.0.smithi115.stdout:8/587: truncate d6/dd/d24/d4a/d7b/d8f/da0/fac 383182 0 2024-03-22T10:42:41.396 INFO:tasks.workunit.client.0.smithi115.stdout:2/324: dread da/d1c/f2b [0,4194304] 0 2024-03-22T10:42:41.397 INFO:tasks.workunit.client.0.smithi115.stdout:6/413: mkdir d4/d11/d12/d1c/d8a 0 2024-03-22T10:42:41.397 INFO:tasks.workunit.client.0.smithi115.stdout:9/432: truncate d6/d7/f5d 879027 0 2024-03-22T10:42:41.399 INFO:tasks.workunit.client.0.smithi115.stdout:7/394: fsync d2/f5 0 2024-03-22T10:42:41.401 INFO:tasks.workunit.client.0.smithi115.stdout:6/414: dread - d4/d36/d48/f62 zero size 2024-03-22T10:42:41.402 INFO:tasks.workunit.client.0.smithi115.stdout:3/342: link d3/d36/d41/c38 d3/d36/c67 0 2024-03-22T10:42:41.405 INFO:tasks.workunit.client.0.smithi115.stdout:8/588: mkdir d6/dd/d24/dc1 0 2024-03-22T10:42:41.408 INFO:tasks.workunit.client.0.smithi115.stdout:7/395: chown d2/d7/d12/d10/d19/d42/c4d 20 1 2024-03-22T10:42:41.409 INFO:tasks.workunit.client.0.smithi115.stdout:6/415: write d4/d36/d48/d72/f81 [527087,83379] 0 2024-03-22T10:42:41.409 INFO:tasks.workunit.client.1.smithi150.stdout:0/74: dwrite d0/d1/d7/d11/f15 [0,4194304] 0 2024-03-22T10:42:41.410 INFO:tasks.workunit.client.0.smithi115.stdout:9/433: link d6/d7/ff d6/d8/d3c/d59/d68/d97/f98 0 2024-03-22T10:42:41.414 INFO:tasks.workunit.client.0.smithi115.stdout:3/343: symlink d3/d19/d2b/l68 0 2024-03-22T10:42:41.415 INFO:tasks.workunit.client.0.smithi115.stdout:8/589: symlink d6/d49/lc2 0 2024-03-22T10:42:41.415 INFO:tasks.workunit.client.0.smithi115.stdout:7/396: dread - d2/d7/d12/d10/d19/d42/d46/f7f zero size 2024-03-22T10:42:41.416 INFO:tasks.workunit.client.0.smithi115.stdout:5/394: dwrite d2/f1f [0,4194304] 0 2024-03-22T10:42:41.418 INFO:tasks.workunit.client.0.smithi115.stdout:6/416: creat d4/d5e/f8b x:0 0 0 2024-03-22T10:42:41.418 INFO:tasks.workunit.client.0.smithi115.stdout:9/434: fdatasync d6/d7/d20/d22/d23/f85 0 2024-03-22T10:42:41.420 INFO:tasks.workunit.client.0.smithi115.stdout:6/417: readlink d4/d13/d33/l57 0 2024-03-22T10:42:41.421 INFO:tasks.workunit.client.1.smithi150.stdout:0/75: symlink d0/l16 0 2024-03-22T10:42:41.422 INFO:tasks.workunit.client.0.smithi115.stdout:7/397: dread - d2/d7/d12/d10/f7b zero size 2024-03-22T10:42:41.428 INFO:tasks.workunit.client.0.smithi115.stdout:9/435: write d6/d7/d20/f57 [437257,96985] 0 2024-03-22T10:42:41.429 INFO:tasks.workunit.client.1.smithi150.stdout:0/76: mknod d0/d1/d7/c17 0 2024-03-22T10:42:41.431 INFO:tasks.workunit.client.1.smithi150.stdout:0/77: read d0/d1/d7/d11/f15 [2387015,115756] 0 2024-03-22T10:42:41.433 INFO:tasks.workunit.client.0.smithi115.stdout:3/344: getdents d3/d19/d2b/d49 0 2024-03-22T10:42:41.436 INFO:tasks.workunit.client.0.smithi115.stdout:9/436: creat d6/d5e/f99 x:0 0 0 2024-03-22T10:42:41.437 INFO:tasks.workunit.client.0.smithi115.stdout:9/437: stat d6/d8/f47 0 2024-03-22T10:42:41.439 INFO:tasks.workunit.client.0.smithi115.stdout:1/473: sync 2024-03-22T10:42:41.439 INFO:tasks.workunit.client.0.smithi115.stdout:4/377: sync 2024-03-22T10:42:41.439 INFO:tasks.workunit.client.0.smithi115.stdout:0/394: sync 2024-03-22T10:42:41.440 INFO:tasks.workunit.client.1.smithi150.stdout:0/78: symlink d0/d12/l18 0 2024-03-22T10:42:41.442 INFO:tasks.workunit.client.0.smithi115.stdout:9/438: read d6/d7/d19/f27 [1448089,28247] 0 2024-03-22T10:42:41.443 INFO:tasks.workunit.client.1.smithi150.stdout:0/79: chown d0/d1/d7/db/d10/f13 5091818 1 2024-03-22T10:42:41.444 INFO:tasks.workunit.client.0.smithi115.stdout:3/345: unlink d3/d19/d2b/f47 0 2024-03-22T10:42:41.445 INFO:tasks.workunit.client.0.smithi115.stdout:1/474: chown d0/d22/f46 5860443 1 2024-03-22T10:42:41.446 INFO:tasks.workunit.client.1.smithi150.stdout:0/80: mknod d0/d12/c19 0 2024-03-22T10:42:41.447 INFO:tasks.workunit.client.0.smithi115.stdout:0/395: read d3/d7/d1c/f48 [2369546,93440] 0 2024-03-22T10:42:41.448 INFO:tasks.workunit.client.0.smithi115.stdout:2/325: truncate da/d25/d30/f43 2327117 0 2024-03-22T10:42:41.454 INFO:tasks.workunit.client.0.smithi115.stdout:0/396: fsync d3/f33 0 2024-03-22T10:42:41.455 INFO:tasks.workunit.client.0.smithi115.stdout:5/395: dwrite d2/d15/d1b/d7c/d38/f5b [0,4194304] 0 2024-03-22T10:42:41.456 INFO:tasks.workunit.client.0.smithi115.stdout:3/346: creat d3/d16/f69 x:0 0 0 2024-03-22T10:42:41.456 INFO:tasks.workunit.client.0.smithi115.stdout:8/590: write d6/dd/d24/d3a/f3c [900837,20522] 0 2024-03-22T10:42:41.458 INFO:tasks.workunit.client.0.smithi115.stdout:8/591: chown d6/d49/d68 25596 1 2024-03-22T10:42:41.462 INFO:tasks.workunit.client.0.smithi115.stdout:4/378: creat d1/d16/f81 x:0 0 0 2024-03-22T10:42:41.464 INFO:tasks.workunit.client.0.smithi115.stdout:2/326: rmdir da/d25/d30/d4f 39 2024-03-22T10:42:41.467 INFO:tasks.workunit.client.0.smithi115.stdout:7/398: write d2/d7/d12/d10/f28 [238988,107045] 0 2024-03-22T10:42:41.468 INFO:tasks.workunit.client.0.smithi115.stdout:3/347: mknod d3/d19/d2b/c6a 0 2024-03-22T10:42:41.475 INFO:tasks.workunit.client.0.smithi115.stdout:0/397: write d3/d7/d1c/f48 [3361428,30154] 0 2024-03-22T10:42:41.475 INFO:tasks.workunit.client.0.smithi115.stdout:8/592: creat d6/dd/d24/d4a/d7b/d8f/da0/fc3 x:0 0 0 2024-03-22T10:42:41.478 INFO:tasks.workunit.client.0.smithi115.stdout:7/399: rename d2/d62/l7d to d2/d62/l8f 0 2024-03-22T10:42:41.482 INFO:tasks.workunit.client.0.smithi115.stdout:8/593: dread - d6/dd/f92 zero size 2024-03-22T10:42:41.483 INFO:tasks.workunit.client.0.smithi115.stdout:3/348: symlink d3/d16/l6b 0 2024-03-22T10:42:41.488 INFO:tasks.workunit.client.0.smithi115.stdout:1/475: dwrite d0/f55 [0,4194304] 0 2024-03-22T10:42:41.493 INFO:tasks.workunit.client.0.smithi115.stdout:0/398: mknod d3/d7/d1b/c81 0 2024-03-22T10:42:41.495 INFO:tasks.workunit.client.0.smithi115.stdout:6/418: dread d4/d11/d12/d1c/f2d [0,4194304] 0 2024-03-22T10:42:41.497 INFO:tasks.workunit.client.0.smithi115.stdout:8/594: mkdir d6/dd/d24/d4a/d41/dc4 0 2024-03-22T10:42:41.501 INFO:tasks.workunit.client.0.smithi115.stdout:7/400: mkdir d2/d3a/d90 0 2024-03-22T10:42:41.501 INFO:tasks.workunit.client.0.smithi115.stdout:1/476: rename d0/d8/d4d/c62 to d0/d8/de/d1a/d91/c9c 0 2024-03-22T10:42:41.502 INFO:tasks.workunit.client.0.smithi115.stdout:2/327: dwrite da/d25/d30/f31 [4194304,4194304] 0 2024-03-22T10:42:41.504 INFO:tasks.workunit.client.0.smithi115.stdout:3/349: rmdir d3/d36/d41/d62 0 2024-03-22T10:42:41.506 INFO:tasks.workunit.client.0.smithi115.stdout:9/439: dread d6/f12 [0,4194304] 0 2024-03-22T10:42:41.507 INFO:tasks.workunit.client.0.smithi115.stdout:1/477: mknod d0/d68/c9d 0 2024-03-22T10:42:41.515 INFO:tasks.workunit.client.0.smithi115.stdout:8/595: creat d6/dd/d24/d4a/d7b/d8f/da0/fc5 x:0 0 0 2024-03-22T10:42:41.515 INFO:tasks.workunit.client.0.smithi115.stdout:3/350: symlink d3/d19/d1b/d52/l6c 0 2024-03-22T10:42:41.515 INFO:tasks.workunit.client.0.smithi115.stdout:1/478: fsync d0/d8/d4d/f6e 0 2024-03-22T10:42:41.517 INFO:tasks.workunit.client.0.smithi115.stdout:2/328: write da/d25/d30/f2e [3252061,112255] 0 2024-03-22T10:42:41.517 INFO:tasks.workunit.client.0.smithi115.stdout:3/351: write d3/d19/f4e [1680951,120683] 0 2024-03-22T10:42:41.519 INFO:tasks.workunit.client.0.smithi115.stdout:8/596: rmdir d6/dd/d12 39 2024-03-22T10:42:41.524 INFO:tasks.workunit.client.0.smithi115.stdout:3/352: chown d3/d36/f3e 296 1 2024-03-22T10:42:41.526 INFO:tasks.workunit.client.0.smithi115.stdout:9/440: dread d6/d7/d20/d38/f40 [0,4194304] 0 2024-03-22T10:42:41.529 INFO:tasks.workunit.client.0.smithi115.stdout:8/597: write d6/d49/fa8 [966440,24467] 0 2024-03-22T10:42:41.530 INFO:tasks.workunit.client.0.smithi115.stdout:3/353: readlink d3/d19/d2b/l5d 0 2024-03-22T10:42:41.535 INFO:tasks.workunit.client.0.smithi115.stdout:4/379: dread d1/d66/f77 [0,4194304] 0 2024-03-22T10:42:41.537 INFO:tasks.workunit.client.0.smithi115.stdout:1/479: write d0/d22/f3d [2143858,25666] 0 2024-03-22T10:42:41.544 INFO:tasks.workunit.client.0.smithi115.stdout:0/399: dwrite d3/d52/f57 [0,4194304] 0 2024-03-22T10:42:41.550 INFO:tasks.workunit.client.0.smithi115.stdout:3/354: mkdir d3/d6d 0 2024-03-22T10:42:41.550 INFO:tasks.workunit.client.0.smithi115.stdout:7/401: dwrite d2/d7/d12/d10/d23/d33/f71 [8388608,4194304] 0 2024-03-22T10:42:41.550 INFO:tasks.workunit.client.0.smithi115.stdout:1/480: fsync d0/d8/f53 0 2024-03-22T10:42:41.551 INFO:tasks.workunit.client.0.smithi115.stdout:5/396: dread d2/fe [4194304,4194304] 0 2024-03-22T10:42:41.552 INFO:tasks.workunit.client.0.smithi115.stdout:6/419: dwrite d4/d11/d12/d1c/f69 [0,4194304] 0 2024-03-22T10:42:41.554 INFO:tasks.workunit.client.0.smithi115.stdout:3/355: chown d3/c6 6 1 2024-03-22T10:42:41.555 INFO:tasks.workunit.client.0.smithi115.stdout:8/598: dread - d6/dd/d12/fb0 zero size 2024-03-22T10:42:41.561 INFO:tasks.workunit.client.0.smithi115.stdout:1/481: chown d0/d8/d4d/d56/c85 0 1 2024-03-22T10:42:41.562 INFO:tasks.workunit.client.0.smithi115.stdout:6/420: fsync d4/d11/d12/d1c/f7f 0 2024-03-22T10:42:41.568 INFO:tasks.workunit.client.0.smithi115.stdout:9/441: truncate d6/d7/d20/d38/f56 811350 0 2024-03-22T10:42:41.575 INFO:tasks.workunit.client.0.smithi115.stdout:2/329: dread da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:41.575 INFO:tasks.workunit.client.0.smithi115.stdout:8/599: truncate d6/f91 4216382 0 2024-03-22T10:42:41.578 INFO:tasks.workunit.client.0.smithi115.stdout:5/397: creat d2/d15/d1b/d7c/d38/d43/f82 x:0 0 0 2024-03-22T10:42:41.580 INFO:tasks.workunit.client.0.smithi115.stdout:0/400: rename d3/d7/d1b/l29 to d3/d52/l82 0 2024-03-22T10:42:41.580 INFO:tasks.workunit.client.0.smithi115.stdout:9/442: readlink d6/d8/l6b 0 2024-03-22T10:42:41.583 INFO:tasks.workunit.client.0.smithi115.stdout:9/443: chown d6/d7 0 1 2024-03-22T10:42:41.588 INFO:tasks.workunit.client.0.smithi115.stdout:8/600: read d6/dd/d12/d31/f5c [2062912,124307] 0 2024-03-22T10:42:41.592 INFO:tasks.workunit.client.0.smithi115.stdout:3/356: link d3/d19/d1b/l26 d3/d19/d1b/l6e 0 2024-03-22T10:42:41.594 INFO:tasks.workunit.client.0.smithi115.stdout:3/357: write d3/f50 [509494,111283] 0 2024-03-22T10:42:41.601 INFO:tasks.workunit.client.0.smithi115.stdout:6/421: dwrite d4/d11/d12/f21 [0,4194304] 0 2024-03-22T10:42:41.602 INFO:tasks.workunit.client.0.smithi115.stdout:7/402: dwrite d2/d7/d12/d10/d23/d3d/f2a [0,4194304] 0 2024-03-22T10:42:41.609 INFO:tasks.workunit.client.0.smithi115.stdout:8/601: mkdir d6/dd/d12/dc6 0 2024-03-22T10:42:41.611 INFO:tasks.workunit.client.0.smithi115.stdout:7/403: truncate d2/d7/d12/d10/f7b 242856 0 2024-03-22T10:42:41.612 INFO:tasks.workunit.client.0.smithi115.stdout:0/401: dwrite d3/f46 [0,4194304] 0 2024-03-22T10:42:41.615 INFO:tasks.workunit.client.0.smithi115.stdout:3/358: creat d3/d19/d61/f6f x:0 0 0 2024-03-22T10:42:41.616 INFO:tasks.workunit.client.0.smithi115.stdout:5/398: dwrite d2/df/d13/f18 [0,4194304] 0 2024-03-22T10:42:41.618 INFO:tasks.workunit.client.0.smithi115.stdout:8/602: mknod d6/dd/d24/d4a/cc7 0 2024-03-22T10:42:41.621 INFO:tasks.workunit.client.0.smithi115.stdout:5/399: fdatasync d2/df/f10 0 2024-03-22T10:42:41.624 INFO:tasks.workunit.client.0.smithi115.stdout:1/482: dread d0/d8/de/d1a/f32 [0,4194304] 0 2024-03-22T10:42:41.627 INFO:tasks.workunit.client.0.smithi115.stdout:9/444: truncate d6/d7/f5d 150899 0 2024-03-22T10:42:41.631 INFO:tasks.workunit.client.0.smithi115.stdout:6/422: dwrite d4/d11/d12/d1c/f6a [0,4194304] 0 2024-03-22T10:42:41.633 INFO:tasks.workunit.client.0.smithi115.stdout:8/603: mkdir d6/dd/d24/dc8 0 2024-03-22T10:42:41.635 INFO:tasks.workunit.client.0.smithi115.stdout:3/359: write d3/d36/f44 [1965317,92202] 0 2024-03-22T10:42:41.646 INFO:tasks.workunit.client.0.smithi115.stdout:1/483: mkdir d0/d12/d17/d37/d9e 0 2024-03-22T10:42:41.646 INFO:tasks.workunit.client.0.smithi115.stdout:5/400: rename d2/d51 to d2/d15/d1b/d7c/d38/d43/d49/d81/d83 0 2024-03-22T10:42:41.649 INFO:tasks.workunit.client.0.smithi115.stdout:3/360: mknod d3/d19/d1b/d57/c70 0 2024-03-22T10:42:41.649 INFO:tasks.workunit.client.0.smithi115.stdout:6/423: creat d4/d13/d33/f8c x:0 0 0 2024-03-22T10:42:41.650 INFO:tasks.workunit.client.0.smithi115.stdout:8/604: unlink d6/dd/d24/d3a/cae 0 2024-03-22T10:42:41.651 INFO:tasks.workunit.client.0.smithi115.stdout:9/445: creat d6/d8/d3c/d59/d68/d8e/f9a x:0 0 0 2024-03-22T10:42:41.651 INFO:tasks.workunit.client.0.smithi115.stdout:6/424: chown d4/d11/l7e 392996133 1 2024-03-22T10:42:41.653 INFO:tasks.workunit.client.0.smithi115.stdout:5/401: unlink d2/df/d27/l53 0 2024-03-22T10:42:41.655 INFO:tasks.workunit.client.0.smithi115.stdout:9/446: dread - d6/d8/d3c/d59/d68/f7f zero size 2024-03-22T10:42:41.659 INFO:tasks.workunit.client.0.smithi115.stdout:1/484: mkdir d0/d8/db/d25/d9f 0 2024-03-22T10:42:41.659 INFO:tasks.workunit.client.0.smithi115.stdout:9/447: fsync d6/d7/f78 0 2024-03-22T10:42:41.659 INFO:tasks.workunit.client.0.smithi115.stdout:3/361: write d3/d36/d41/f48 [3398295,42848] 0 2024-03-22T10:42:41.662 INFO:tasks.workunit.client.0.smithi115.stdout:2/330: dread da/d1c/f48 [0,4194304] 0 2024-03-22T10:42:41.663 INFO:tasks.workunit.client.0.smithi115.stdout:3/362: chown d3/l46 49819063 1 2024-03-22T10:42:41.663 INFO:tasks.workunit.client.0.smithi115.stdout:8/605: truncate d6/dd/d24/d4a/d7b/d8f/da0/fac 595952 0 2024-03-22T10:42:41.666 INFO:tasks.workunit.client.0.smithi115.stdout:6/425: mknod d4/d11/d12/d1c/d8a/c8d 0 2024-03-22T10:42:41.666 INFO:tasks.workunit.client.0.smithi115.stdout:5/402: mkdir d2/d15/d1b/d7c/d84 0 2024-03-22T10:42:41.668 INFO:tasks.workunit.client.0.smithi115.stdout:2/331: write da/d25/d3d/f2c [2868832,86314] 0 2024-03-22T10:42:41.669 INFO:tasks.workunit.client.0.smithi115.stdout:9/448: rename d6/d5e/f99 to d6/d7/d6a/d6f/f9b 0 2024-03-22T10:42:41.672 INFO:tasks.workunit.client.0.smithi115.stdout:3/363: creat d3/d6d/f71 x:0 0 0 2024-03-22T10:42:41.674 INFO:tasks.workunit.client.0.smithi115.stdout:5/403: mknod d2/d15/d1b/d7c/d84/c85 0 2024-03-22T10:42:41.676 INFO:tasks.workunit.client.0.smithi115.stdout:9/449: write d6/d7/f78 [403658,5659] 0 2024-03-22T10:42:41.677 INFO:tasks.workunit.client.0.smithi115.stdout:8/606: truncate d6/fc 425952 0 2024-03-22T10:42:41.680 INFO:tasks.workunit.client.0.smithi115.stdout:3/364: truncate d3/d19/d1b/d52/f56 1014572 0 2024-03-22T10:42:41.683 INFO:tasks.workunit.client.0.smithi115.stdout:5/404: fsync d2/df/f55 0 2024-03-22T10:42:41.683 INFO:tasks.workunit.client.0.smithi115.stdout:1/485: dwrite d0/d12/d17/f39 [0,4194304] 0 2024-03-22T10:42:41.685 INFO:tasks.workunit.client.0.smithi115.stdout:7/404: dread d2/f60 [0,4194304] 0 2024-03-22T10:42:41.689 INFO:tasks.workunit.client.0.smithi115.stdout:5/405: write d2/f5e [640526,34196] 0 2024-03-22T10:42:41.692 INFO:tasks.workunit.client.0.smithi115.stdout:3/365: rmdir d3/d19 39 2024-03-22T10:42:41.694 INFO:tasks.workunit.client.0.smithi115.stdout:2/332: dwrite da/d1c/f4c [0,4194304] 0 2024-03-22T10:42:41.700 INFO:tasks.workunit.client.0.smithi115.stdout:7/405: truncate d2/d7/f20 277723 0 2024-03-22T10:42:41.703 INFO:tasks.workunit.client.0.smithi115.stdout:5/406: creat d2/df/d27/d50/f86 x:0 0 0 2024-03-22T10:42:41.706 INFO:tasks.workunit.client.0.smithi115.stdout:3/366: chown d3/d19/d1b/f27 1034353387 1 2024-03-22T10:42:41.706 INFO:tasks.workunit.client.0.smithi115.stdout:7/406: readlink d2/d7/d12/d10/d23/d33/l7e 0 2024-03-22T10:42:41.706 INFO:tasks.workunit.client.0.smithi115.stdout:6/426: dwrite d4/d11/f78 [4194304,4194304] 0 2024-03-22T10:42:41.708 INFO:tasks.workunit.client.0.smithi115.stdout:3/367: stat d3/d16/f69 0 2024-03-22T10:42:41.708 INFO:tasks.workunit.client.0.smithi115.stdout:1/486: mkdir d0/d12/d17/d37/d9e/da0 0 2024-03-22T10:42:41.709 INFO:tasks.workunit.client.0.smithi115.stdout:6/427: fdatasync d4/d13/f3d 0 2024-03-22T10:42:41.710 INFO:tasks.workunit.client.0.smithi115.stdout:2/333: creat da/d25/d51/f63 x:0 0 0 2024-03-22T10:42:41.712 INFO:tasks.workunit.client.0.smithi115.stdout:7/407: fsync d2/d7/d12/d10/f5b 0 2024-03-22T10:42:41.714 INFO:tasks.workunit.client.0.smithi115.stdout:1/487: write d0/d22/f86 [204608,26416] 0 2024-03-22T10:42:41.718 INFO:tasks.workunit.client.0.smithi115.stdout:3/368: rename d3/l46 to d3/d19/d1b/d52/l72 0 2024-03-22T10:42:41.720 INFO:tasks.workunit.client.0.smithi115.stdout:2/334: symlink da/d25/d3d/d57/l64 0 2024-03-22T10:42:41.721 INFO:tasks.workunit.client.0.smithi115.stdout:1/488: readlink d0/d12/l72 0 2024-03-22T10:42:41.724 INFO:tasks.workunit.client.0.smithi115.stdout:7/408: rmdir d2/d7/d12/d10/d23 39 2024-03-22T10:42:41.725 INFO:tasks.workunit.client.0.smithi115.stdout:3/369: mknod d3/d19/d61/c73 0 2024-03-22T10:42:41.728 INFO:tasks.workunit.client.0.smithi115.stdout:2/335: creat da/d25/d3d/d57/f65 x:0 0 0 2024-03-22T10:42:41.729 INFO:tasks.workunit.client.0.smithi115.stdout:8/607: dwrite d6/dd/d12/f3b [0,4194304] 0 2024-03-22T10:42:41.732 INFO:tasks.workunit.client.0.smithi115.stdout:9/450: dread d6/d8/d3c/f54 [0,4194304] 0 2024-03-22T10:42:41.734 INFO:tasks.workunit.client.0.smithi115.stdout:1/489: rename d0/f3 to d0/d8/de/fa1 0 2024-03-22T10:42:41.737 INFO:tasks.workunit.client.0.smithi115.stdout:2/336: creat da/f66 x:0 0 0 2024-03-22T10:42:41.739 INFO:tasks.workunit.client.0.smithi115.stdout:2/337: write da/f15 [338997,87358] 0 2024-03-22T10:42:41.745 INFO:tasks.workunit.client.0.smithi115.stdout:2/338: rmdir da/d25/d2a 39 2024-03-22T10:42:41.746 INFO:tasks.workunit.client.0.smithi115.stdout:1/490: creat d0/d12/d2b/d77/fa2 x:0 0 0 2024-03-22T10:42:41.746 INFO:tasks.workunit.client.0.smithi115.stdout:8/608: dwrite d6/dd/f26 [0,4194304] 0 2024-03-22T10:42:41.748 INFO:tasks.workunit.client.0.smithi115.stdout:1/491: mkdir d0/d8/de/d1a/d91/da3 0 2024-03-22T10:42:41.750 INFO:tasks.workunit.client.0.smithi115.stdout:8/609: fdatasync d6/dd/d24/d4a/d7b/d8f/da0/fc5 0 2024-03-22T10:42:41.756 INFO:tasks.workunit.client.0.smithi115.stdout:2/339: getdents da/d25/d2a 0 2024-03-22T10:42:41.758 INFO:tasks.workunit.client.0.smithi115.stdout:9/451: dwrite d6/f12 [0,4194304] 0 2024-03-22T10:42:41.759 INFO:tasks.workunit.client.0.smithi115.stdout:2/340: mkdir da/d1c/d67 0 2024-03-22T10:42:41.761 INFO:tasks.workunit.client.0.smithi115.stdout:2/341: dread - da/d25/d51/f63 zero size 2024-03-22T10:42:41.764 INFO:tasks.workunit.client.0.smithi115.stdout:9/452: mkdir d6/d7/d20/d9c 0 2024-03-22T10:42:41.767 INFO:tasks.workunit.client.0.smithi115.stdout:2/342: symlink da/d25/d51/l68 0 2024-03-22T10:42:41.769 INFO:tasks.workunit.client.0.smithi115.stdout:9/453: truncate d6/d8/d3c/d59/f90 14565 0 2024-03-22T10:42:41.772 INFO:tasks.workunit.client.0.smithi115.stdout:2/343: unlink da/l18 0 2024-03-22T10:42:41.777 INFO:tasks.workunit.client.0.smithi115.stdout:1/492: dwrite d0/d12/d17/d37/d5c/f8c [0,4194304] 0 2024-03-22T10:42:41.780 INFO:tasks.workunit.client.0.smithi115.stdout:8/610: dread d6/fb [0,4194304] 0 2024-03-22T10:42:41.791 INFO:tasks.workunit.client.0.smithi115.stdout:9/454: write d6/d7/d19/f31 [585298,130221] 0 2024-03-22T10:42:41.791 INFO:tasks.workunit.client.0.smithi115.stdout:2/344: truncate da/f15 3594975 0 2024-03-22T10:42:41.794 INFO:tasks.workunit.client.0.smithi115.stdout:1/493: dwrite d0/d22/f46 [0,4194304] 0 2024-03-22T10:42:41.794 INFO:tasks.workunit.client.0.smithi115.stdout:9/455: fsync d6/d8/f47 0 2024-03-22T10:42:41.796 INFO:tasks.workunit.client.0.smithi115.stdout:2/345: rmdir da/d25/d3d 39 2024-03-22T10:42:41.798 INFO:tasks.workunit.client.0.smithi115.stdout:2/346: chown da/d25/d2a/c60 109 1 2024-03-22T10:42:41.806 INFO:tasks.workunit.client.0.smithi115.stdout:8/611: truncate d6/f23 1991029 0 2024-03-22T10:42:41.806 INFO:tasks.workunit.client.0.smithi115.stdout:8/612: read d6/d49/d68/fb7 [69247,36577] 0 2024-03-22T10:42:41.808 INFO:tasks.workunit.client.0.smithi115.stdout:9/456: creat d6/d7/d30/f9d x:0 0 0 2024-03-22T10:42:41.810 INFO:tasks.workunit.client.0.smithi115.stdout:2/347: rename da/d25/d30/d4f/f50 to da/d1c/f69 0 2024-03-22T10:42:41.813 INFO:tasks.workunit.client.0.smithi115.stdout:8/613: fsync d6/dd/d12/f94 0 2024-03-22T10:42:41.814 INFO:tasks.workunit.client.0.smithi115.stdout:2/348: mknod da/d25/d30/c6a 0 2024-03-22T10:42:41.816 INFO:tasks.workunit.client.0.smithi115.stdout:2/349: stat da/d25/d2a/c4a 0 2024-03-22T10:42:41.818 INFO:tasks.workunit.client.0.smithi115.stdout:2/350: fdatasync da/fb 0 2024-03-22T10:42:41.820 INFO:tasks.workunit.client.0.smithi115.stdout:8/614: mknod d6/dd/d24/d4a/d41/cc9 0 2024-03-22T10:42:41.820 INFO:tasks.workunit.client.0.smithi115.stdout:1/494: dwrite d0/d8/f53 [0,4194304] 0 2024-03-22T10:42:41.827 INFO:tasks.workunit.client.0.smithi115.stdout:2/351: mknod da/d25/d3d/c6b 0 2024-03-22T10:42:41.828 INFO:tasks.workunit.client.0.smithi115.stdout:1/495: symlink d0/d12/la4 0 2024-03-22T10:42:41.829 INFO:tasks.workunit.client.0.smithi115.stdout:9/457: dwrite d6/d7/d19/f31 [0,4194304] 0 2024-03-22T10:42:41.831 INFO:tasks.workunit.client.0.smithi115.stdout:8/615: creat d6/dbc/fca x:0 0 0 2024-03-22T10:42:41.832 INFO:tasks.workunit.client.0.smithi115.stdout:8/616: stat d6/dd/d24/dc8 0 2024-03-22T10:42:41.835 INFO:tasks.workunit.client.0.smithi115.stdout:8/617: dread - d6/dd/d24/d4a/d7b/d8f/da0/fb8 zero size 2024-03-22T10:42:41.837 INFO:tasks.workunit.client.0.smithi115.stdout:1/496: rmdir d0/d12/d2b 39 2024-03-22T10:42:41.838 INFO:tasks.workunit.client.0.smithi115.stdout:9/458: creat d6/d7/d6a/f9e x:0 0 0 2024-03-22T10:42:41.847 INFO:tasks.workunit.client.0.smithi115.stdout:8/618: dread f4 [0,4194304] 0 2024-03-22T10:42:41.853 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:41 smithi150 ceph-mon[34518]: pgmap v9: 65 pgs: 65 active+clean; 920 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 22 MiB/s rd, 139 MiB/s wr, 308 op/s 2024-03-22T10:42:41.853 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:41 smithi150 ceph-mon[34518]: Standby manager daemon smithi150.fmoumv started 2024-03-22T10:42:41.853 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:41 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/4047041489' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/crt"}]: dispatch 2024-03-22T10:42:41.853 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:41 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/4047041489' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:42:41.853 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:41 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/4047041489' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/key"}]: dispatch 2024-03-22T10:42:41.853 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:41 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/4047041489' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:42:41.855 INFO:tasks.workunit.client.0.smithi115.stdout:1/497: write d0/d12/d17/d37/f49 [566568,123912] 0 2024-03-22T10:42:41.855 INFO:tasks.workunit.client.0.smithi115.stdout:8/619: link d6/dd/d24/d4a/l65 d6/d49/d68/lcb 0 2024-03-22T10:42:41.855 INFO:tasks.workunit.client.0.smithi115.stdout:9/459: write f3 [57542,115300] 0 2024-03-22T10:42:41.858 INFO:tasks.workunit.client.0.smithi115.stdout:9/460: chown d6/d7/d48 1 1 2024-03-22T10:42:41.861 INFO:tasks.workunit.client.0.smithi115.stdout:8/620: symlink d6/dd/d24/dc1/lcc 0 2024-03-22T10:42:41.862 INFO:tasks.workunit.client.0.smithi115.stdout:9/461: rmdir d6/d7/d20/d22/d23 39 2024-03-22T10:42:41.864 INFO:tasks.workunit.client.0.smithi115.stdout:1/498: truncate d0/d8/de/d1a/f61 4228143 0 2024-03-22T10:42:41.866 INFO:tasks.workunit.client.0.smithi115.stdout:1/499: readlink d0/d8/de/l34 0 2024-03-22T10:42:41.866 INFO:tasks.workunit.client.0.smithi115.stdout:8/621: creat d6/dd/d24/d3a/fcd x:0 0 0 2024-03-22T10:42:41.870 INFO:tasks.workunit.client.0.smithi115.stdout:9/462: creat d6/d7/d20/d9c/f9f x:0 0 0 2024-03-22T10:42:41.871 INFO:tasks.workunit.client.0.smithi115.stdout:2/352: dread da/f15 [0,4194304] 0 2024-03-22T10:42:41.871 INFO:tasks.workunit.client.0.smithi115.stdout:9/463: dread - d6/d5e/f60 zero size 2024-03-22T10:42:41.873 INFO:tasks.workunit.client.0.smithi115.stdout:1/500: rename d0/f14 to d0/d22/d75/fa5 0 2024-03-22T10:42:41.873 INFO:tasks.workunit.client.0.smithi115.stdout:2/353: fdatasync da/d25/d3d/f28 0 2024-03-22T10:42:41.875 INFO:tasks.workunit.client.0.smithi115.stdout:2/354: chown da/l23 1 1 2024-03-22T10:42:41.875 INFO:tasks.workunit.client.0.smithi115.stdout:9/464: fsync d6/d7/d20/d38/f40 0 2024-03-22T10:42:41.878 INFO:tasks.workunit.client.0.smithi115.stdout:9/465: creat d6/d7/d30/d37/fa0 x:0 0 0 2024-03-22T10:42:41.879 INFO:tasks.workunit.client.0.smithi115.stdout:2/355: mknod da/c6c 0 2024-03-22T10:42:41.883 INFO:tasks.workunit.client.0.smithi115.stdout:1/501: write d0/d12/d17/f31 [551238,103072] 0 2024-03-22T10:42:41.884 INFO:tasks.workunit.client.0.smithi115.stdout:2/356: mknod da/d25/d3d/c6d 0 2024-03-22T10:42:41.885 INFO:tasks.workunit.client.0.smithi115.stdout:1/502: dread - d0/f7a zero size 2024-03-22T10:42:41.885 INFO:tasks.workunit.client.0.smithi115.stdout:8/622: dwrite d6/dd/d24/f9d [0,4194304] 0 2024-03-22T10:42:41.889 INFO:tasks.workunit.client.0.smithi115.stdout:9/466: rename d6/d7/d20/d22/d23/f51 to d6/d7/d20/d9c/fa1 0 2024-03-22T10:42:41.891 INFO:tasks.workunit.client.0.smithi115.stdout:8/623: write d6/d49/d68/f9f [39668,97232] 0 2024-03-22T10:42:41.891 INFO:tasks.workunit.client.0.smithi115.stdout:1/503: symlink d0/d22/d75/la6 0 2024-03-22T10:42:41.893 INFO:tasks.workunit.client.0.smithi115.stdout:2/357: creat da/d25/d30/f6e x:0 0 0 2024-03-22T10:42:41.898 INFO:tasks.workunit.client.0.smithi115.stdout:1/504: rmdir d0/d12/d2b/d77 39 2024-03-22T10:42:41.900 INFO:tasks.workunit.client.0.smithi115.stdout:1/505: write d0/d12/f1c [571359,14472] 0 2024-03-22T10:42:41.901 INFO:tasks.workunit.client.0.smithi115.stdout:2/358: rmdir da/d25/d30/d4f/d54/d5e 0 2024-03-22T10:42:41.905 INFO:tasks.workunit.client.0.smithi115.stdout:1/506: creat d0/d8/db/d25/fa7 x:0 0 0 2024-03-22T10:42:41.909 INFO:tasks.workunit.client.0.smithi115.stdout:1/507: symlink d0/d12/d17/d37/d9e/la8 0 2024-03-22T10:42:41.909 INFO:tasks.workunit.client.0.smithi115.stdout:8/624: dwrite d6/dd/d24/d4a/d7b/d8f/da0/fb8 [0,4194304] 0 2024-03-22T10:42:41.911 INFO:tasks.workunit.client.0.smithi115.stdout:1/508: read - d0/d12/d17/d67/f95 zero size 2024-03-22T10:42:41.913 INFO:tasks.workunit.client.0.smithi115.stdout:1/509: fsync d0/d12/f66 0 2024-03-22T10:42:41.917 INFO:tasks.workunit.client.0.smithi115.stdout:1/510: symlink d0/d58/la9 0 2024-03-22T10:42:41.918 INFO:tasks.workunit.client.0.smithi115.stdout:1/511: readlink d0/d12/d17/d37/l5b 0 2024-03-22T10:42:41.921 INFO:tasks.workunit.client.0.smithi115.stdout:1/512: read d0/d8/db/f18 [100349,82522] 0 2024-03-22T10:42:41.924 INFO:tasks.workunit.client.0.smithi115.stdout:8/625: write d6/fc [813494,93439] 0 2024-03-22T10:42:41.926 INFO:tasks.workunit.client.0.smithi115.stdout:8/626: dread - d6/dd/fb6 zero size 2024-03-22T10:42:41.930 INFO:tasks.workunit.client.0.smithi115.stdout:8/627: truncate d6/dd/f2e 1413960 0 2024-03-22T10:42:41.941 INFO:tasks.workunit.client.0.smithi115.stdout:2/359: dread da/d25/d30/f31 [0,4194304] 0 2024-03-22T10:42:41.943 INFO:tasks.workunit.client.0.smithi115.stdout:1/513: dwrite d0/f52 [0,4194304] 0 2024-03-22T10:42:41.943 INFO:tasks.workunit.client.0.smithi115.stdout:2/360: read - da/d25/d51/f55 zero size 2024-03-22T10:42:41.952 INFO:tasks.workunit.client.0.smithi115.stdout:1/514: rename d0/f7a to d0/d12/faa 0 2024-03-22T10:42:41.958 INFO:tasks.workunit.client.0.smithi115.stdout:1/515: chown d0/d8/db/d25/fa7 2161 1 2024-03-22T10:42:41.958 INFO:tasks.workunit.client.0.smithi115.stdout:2/361: rename da/fb to da/d1c/d67/f6f 0 2024-03-22T10:42:41.961 INFO:tasks.workunit.client.0.smithi115.stdout:1/516: mknod d0/d12/d7b/cab 0 2024-03-22T10:42:41.964 INFO:tasks.workunit.client.0.smithi115.stdout:1/517: mknod d0/d22/d89/cac 0 2024-03-22T10:42:41.966 INFO:tasks.workunit.client.0.smithi115.stdout:8/628: dread d6/f17 [0,4194304] 0 2024-03-22T10:42:41.968 INFO:tasks.workunit.client.0.smithi115.stdout:8/629: fsync d6/dd/d24/d4a/d41/f4b 0 2024-03-22T10:42:41.970 INFO:tasks.workunit.client.0.smithi115.stdout:2/362: dread da/d25/d3d/f3c [0,4194304] 0 2024-03-22T10:42:41.975 INFO:tasks.workunit.client.0.smithi115.stdout:1/518: truncate d0/d12/d17/d37/d5c/f8c 2999343 0 2024-03-22T10:42:41.976 INFO:tasks.workunit.client.0.smithi115.stdout:8/630: write d6/dd/d24/d4a/f51 [2305677,86737] 0 2024-03-22T10:42:41.977 INFO:tasks.workunit.client.0.smithi115.stdout:8/631: stat d6/dd/d24/d4a/d7b/fa2 0 2024-03-22T10:42:41.981 INFO:tasks.workunit.client.0.smithi115.stdout:8/632: fdatasync d6/f23 0 2024-03-22T10:42:41.985 INFO:tasks.workunit.client.0.smithi115.stdout:8/633: creat d6/dd/fce x:0 0 0 2024-03-22T10:42:41.987 INFO:tasks.workunit.client.0.smithi115.stdout:1/519: truncate d0/d8/d4d/f71 2326530 0 2024-03-22T10:42:41.995 INFO:tasks.workunit.client.0.smithi115.stdout:1/520: link d0/f55 d0/d12/d2b/fad 0 2024-03-22T10:42:41.998 INFO:tasks.workunit.client.0.smithi115.stdout:2/363: dread da/d25/f34 [0,4194304] 0 2024-03-22T10:42:42.001 INFO:tasks.workunit.client.0.smithi115.stdout:8/634: dwrite d6/dd/d12/d31/f5c [0,4194304] 0 2024-03-22T10:42:42.007 INFO:tasks.workunit.client.0.smithi115.stdout:1/521: dread d0/d8/de/d1a/f32 [0,4194304] 0 2024-03-22T10:42:42.010 INFO:tasks.workunit.client.0.smithi115.stdout:1/522: creat d0/d68/fae x:0 0 0 2024-03-22T10:42:42.010 INFO:tasks.workunit.client.0.smithi115.stdout:2/364: dread da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:42.015 INFO:tasks.workunit.client.0.smithi115.stdout:1/523: mknod d0/d12/d17/d37/d9e/caf 0 2024-03-22T10:42:42.015 INFO:tasks.workunit.client.0.smithi115.stdout:2/365: symlink da/d1c/d67/l70 0 2024-03-22T10:42:42.019 INFO:tasks.workunit.client.0.smithi115.stdout:1/524: symlink d0/d8/lb0 0 2024-03-22T10:42:42.020 INFO:tasks.workunit.client.0.smithi115.stdout:8/635: dwrite d6/dd/d24/d3a/fcd [0,4194304] 0 2024-03-22T10:42:42.023 INFO:tasks.workunit.client.0.smithi115.stdout:2/366: creat da/d25/d30/f71 x:0 0 0 2024-03-22T10:42:42.023 INFO:tasks.workunit.client.0.smithi115.stdout:8/636: mkdir d6/d49/d68/dcf 0 2024-03-22T10:42:42.025 INFO:tasks.workunit.client.0.smithi115.stdout:1/525: mknod d0/d12/d2b/cb1 0 2024-03-22T10:42:42.027 INFO:tasks.workunit.client.0.smithi115.stdout:1/526: chown d0/d8/db/f18 26028 1 2024-03-22T10:42:42.028 INFO:tasks.workunit.client.0.smithi115.stdout:2/367: fdatasync da/d25/d30/f43 0 2024-03-22T10:42:42.029 INFO:tasks.workunit.client.0.smithi115.stdout:2/368: chown da/d25/d30/d4f 64529 1 2024-03-22T10:42:42.032 INFO:tasks.workunit.client.0.smithi115.stdout:1/527: rename d0/d8/d4d/f65 to d0/d8/d4d/d56/fb2 0 2024-03-22T10:42:42.034 INFO:tasks.workunit.client.0.smithi115.stdout:2/369: creat da/d25/d2a/f72 x:0 0 0 2024-03-22T10:42:42.035 INFO:tasks.workunit.client.0.smithi115.stdout:1/528: mkdir d0/d12/d17/db3 0 2024-03-22T10:42:42.041 INFO:tasks.workunit.client.0.smithi115.stdout:1/529: link d0/d12/f1c d0/d12/d17/d37/d5c/fb4 0 2024-03-22T10:42:42.043 INFO:tasks.workunit.client.0.smithi115.stdout:8/637: dwrite d6/dd/fb6 [0,4194304] 0 2024-03-22T10:42:42.045 INFO:tasks.workunit.client.0.smithi115.stdout:8/638: stat d6/dd/d24/dc8 0 2024-03-22T10:42:42.048 INFO:tasks.workunit.client.0.smithi115.stdout:8/639: chown d6/dd/l7a 7881756 1 2024-03-22T10:42:42.050 INFO:tasks.workunit.client.0.smithi115.stdout:2/370: dwrite da/d25/d51/f55 [0,4194304] 0 2024-03-22T10:42:42.054 INFO:tasks.workunit.client.0.smithi115.stdout:2/371: creat da/d1c/f73 x:0 0 0 2024-03-22T10:42:42.059 INFO:tasks.workunit.client.0.smithi115.stdout:8/640: dread d6/dd/d12/d31/f5b [4194304,4194304] 0 2024-03-22T10:42:42.062 INFO:tasks.workunit.client.0.smithi115.stdout:8/641: write d6/dd/d12/f1d [2664509,65753] 0 2024-03-22T10:42:42.080 INFO:tasks.workunit.client.0.smithi115.stdout:2/372: dread da/d1c/f1e [0,4194304] 0 2024-03-22T10:42:42.082 INFO:tasks.workunit.client.0.smithi115.stdout:2/373: symlink da/d25/l74 0 2024-03-22T10:42:42.084 INFO:tasks.workunit.client.0.smithi115.stdout:2/374: write da/d25/d3d/f28 [101003,55909] 0 2024-03-22T10:42:42.084 INFO:tasks.workunit.client.0.smithi115.stdout:2/375: stat da/d25/f40 0 2024-03-22T10:42:42.088 INFO:tasks.workunit.client.0.smithi115.stdout:2/376: rename da/l23 to da/d25/d30/l75 0 2024-03-22T10:42:42.091 INFO:tasks.workunit.client.0.smithi115.stdout:2/377: rename da/d25/f40 to da/d25/d3d/f76 0 2024-03-22T10:42:42.106 INFO:tasks.workunit.client.0.smithi115.stdout:2/378: dwrite da/d25/d2a/f72 [0,4194304] 0 2024-03-22T10:42:42.109 INFO:tasks.workunit.client.0.smithi115.stdout:2/379: chown da/d25/d30/f59 767 1 2024-03-22T10:42:42.112 INFO:tasks.workunit.client.0.smithi115.stdout:2/380: symlink da/d25/d30/l77 0 2024-03-22T10:42:42.116 INFO:tasks.workunit.client.0.smithi115.stdout:2/381: write da/d25/f34 [2325652,107239] 0 2024-03-22T10:42:42.118 INFO:tasks.workunit.client.0.smithi115.stdout:2/382: truncate da/f14 4560741 0 2024-03-22T10:42:42.120 INFO:tasks.workunit.client.0.smithi115.stdout:2/383: mkdir da/d1c/d67/d78 0 2024-03-22T10:42:42.133 INFO:tasks.workunit.client.0.smithi115.stdout:2/384: dwrite da/d25/f34 [0,4194304] 0 2024-03-22T10:42:42.136 INFO:tasks.workunit.client.0.smithi115.stdout:2/385: mknod da/d1c/c79 0 2024-03-22T10:42:42.136 INFO:tasks.workunit.client.0.smithi115.stdout:4/380: sync 2024-03-22T10:42:42.138 INFO:tasks.workunit.client.0.smithi115.stdout:4/381: chown d1/d30/d42 1243452 1 2024-03-22T10:42:42.139 INFO:tasks.workunit.client.0.smithi115.stdout:2/386: write da/d1c/f69 [3601396,63353] 0 2024-03-22T10:42:42.143 INFO:tasks.workunit.client.0.smithi115.stdout:4/382: mkdir d1/d66/d82 0 2024-03-22T10:42:42.144 INFO:tasks.workunit.client.0.smithi115.stdout:2/387: creat da/d25/d30/d4f/d54/f7a x:0 0 0 2024-03-22T10:42:42.167 INFO:tasks.workunit.client.0.smithi115.stdout:4/383: dwrite d1/d16/d35/d40/d5c/d14/f58 [4194304,4194304] 0 2024-03-22T10:42:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:41 smithi115 ceph-mon[29111]: pgmap v9: 65 pgs: 65 active+clean; 920 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 22 MiB/s rd, 139 MiB/s wr, 308 op/s 2024-03-22T10:42:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:41 smithi115 ceph-mon[29111]: Standby manager daemon smithi150.fmoumv started 2024-03-22T10:42:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:41 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/4047041489' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/crt"}]: dispatch 2024-03-22T10:42:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:41 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/4047041489' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:42:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:41 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/4047041489' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/key"}]: dispatch 2024-03-22T10:42:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:41 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/4047041489' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:42:42.291 INFO:tasks.workunit.client.0.smithi115.stdout:0/402: sync 2024-03-22T10:42:42.307 INFO:tasks.workunit.client.0.smithi115.stdout:0/403: rename d3/da/d4c/f5a to d3/d7/d1b/d1f/d5b/f83 0 2024-03-22T10:42:42.318 INFO:tasks.workunit.client.0.smithi115.stdout:0/404: mkdir d3/da/d4c/d74/d84 0 2024-03-22T10:42:42.329 INFO:tasks.workunit.client.0.smithi115.stdout:0/405: symlink d3/da/d4c/d74/l85 0 2024-03-22T10:42:42.331 INFO:tasks.workunit.client.0.smithi115.stdout:0/406: stat d3/d7/d1b/d35/l53 0 2024-03-22T10:42:42.337 INFO:tasks.workunit.client.0.smithi115.stdout:0/407: truncate d3/d7/d1b/d35/f79 468641 0 2024-03-22T10:42:42.347 INFO:tasks.workunit.client.0.smithi115.stdout:1/530: write d0/d22/f5e [2878705,96682] 0 2024-03-22T10:42:42.351 INFO:tasks.workunit.client.0.smithi115.stdout:1/531: mkdir d0/d12/d17/d37/db5 0 2024-03-22T10:42:42.355 INFO:tasks.workunit.client.0.smithi115.stdout:1/532: write d0/d12/d17/f80 [6674164,29568] 0 2024-03-22T10:42:42.360 INFO:tasks.workunit.client.0.smithi115.stdout:1/533: mkdir d0/d8/d4d/d56/db6 0 2024-03-22T10:42:42.362 INFO:tasks.workunit.client.0.smithi115.stdout:0/408: rmdir d3/d7/d1b/d1f/d5b 39 2024-03-22T10:42:42.368 INFO:tasks.workunit.client.0.smithi115.stdout:4/384: dwrite d1/d30/d42/d4f/f7c [0,4194304] 0 2024-03-22T10:42:42.370 INFO:tasks.workunit.client.0.smithi115.stdout:0/409: creat d3/da/d4c/d69/d6a/f86 x:0 0 0 2024-03-22T10:42:42.374 INFO:tasks.workunit.client.0.smithi115.stdout:4/385: creat d1/d16/d68/f83 x:0 0 0 2024-03-22T10:42:42.375 INFO:tasks.workunit.client.0.smithi115.stdout:4/386: chown d1/d16/f64 0 1 2024-03-22T10:42:42.377 INFO:tasks.workunit.client.0.smithi115.stdout:4/387: write d1/f9 [3287551,64418] 0 2024-03-22T10:42:42.379 INFO:tasks.workunit.client.0.smithi115.stdout:0/410: mkdir d3/d7/d1c/d6c/d87 0 2024-03-22T10:42:42.380 INFO:tasks.workunit.client.0.smithi115.stdout:5/407: sync 2024-03-22T10:42:42.381 INFO:tasks.workunit.client.0.smithi115.stdout:4/388: readlink d1/d16/d35/d40/l72 0 2024-03-22T10:42:42.382 INFO:tasks.workunit.client.0.smithi115.stdout:0/411: chown d3/d7/d1b/d1f/c4b 970181533 1 2024-03-22T10:42:42.384 INFO:tasks.workunit.client.0.smithi115.stdout:1/534: dwrite d0/d8/de/d1a/f4a [0,4194304] 0 2024-03-22T10:42:42.386 INFO:tasks.workunit.client.0.smithi115.stdout:4/389: truncate d1/d30/d42/f34 4315427 0 2024-03-22T10:42:42.390 INFO:tasks.workunit.client.0.smithi115.stdout:0/412: creat d3/d4/f88 x:0 0 0 2024-03-22T10:42:42.390 INFO:tasks.workunit.client.0.smithi115.stdout:4/390: getdents d1/d30/d78 0 2024-03-22T10:42:42.391 INFO:tasks.workunit.client.0.smithi115.stdout:4/391: readlink d1/d30/l4b 0 2024-03-22T10:42:42.395 INFO:tasks.workunit.client.0.smithi115.stdout:4/392: write d1/d16/d35/d40/d5c/f6f [1344279,95500] 0 2024-03-22T10:42:42.397 INFO:tasks.workunit.client.0.smithi115.stdout:4/393: chown d1/d30/l4b 27 1 2024-03-22T10:42:42.401 INFO:tasks.workunit.client.0.smithi115.stdout:4/394: symlink d1/d16/d35/d40/d5c/d6b/l84 0 2024-03-22T10:42:42.402 INFO:tasks.workunit.client.0.smithi115.stdout:5/408: truncate d2/df/f67 24428 0 2024-03-22T10:42:42.403 INFO:tasks.workunit.client.0.smithi115.stdout:0/413: truncate d3/d4/d14/f23 4326750 0 2024-03-22T10:42:42.404 INFO:tasks.workunit.client.0.smithi115.stdout:4/395: symlink d1/d30/d4c/l85 0 2024-03-22T10:42:42.405 INFO:tasks.workunit.client.0.smithi115.stdout:0/414: read - d3/d7/d1b/d1f/f62 zero size 2024-03-22T10:42:42.406 INFO:tasks.workunit.client.0.smithi115.stdout:5/409: creat d2/d15/d1b/d7c/d38/d56/f87 x:0 0 0 2024-03-22T10:42:42.408 INFO:tasks.workunit.client.0.smithi115.stdout:5/410: rename d2/d15/d1b/d7c to d2/d15/d1b/d7c/d38/d43/d49/d88 22 2024-03-22T10:42:42.410 INFO:tasks.workunit.client.0.smithi115.stdout:0/415: link d3/l4e d3/da/d4c/d74/l89 0 2024-03-22T10:42:42.411 INFO:tasks.workunit.client.0.smithi115.stdout:5/411: chown d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f65 155664 1 2024-03-22T10:42:42.413 INFO:tasks.workunit.client.0.smithi115.stdout:0/416: truncate d3/d7/d1b/d1f/f58 431185 0 2024-03-22T10:42:42.415 INFO:tasks.workunit.client.0.smithi115.stdout:0/417: chown d3/d7/l73 1787177 1 2024-03-22T10:42:42.419 INFO:tasks.workunit.client.0.smithi115.stdout:0/418: creat d3/d7/d1c/d6c/d87/f8a x:0 0 0 2024-03-22T10:42:42.420 INFO:tasks.workunit.client.0.smithi115.stdout:5/412: dread d2/df/f67 [0,4194304] 0 2024-03-22T10:42:42.420 INFO:tasks.workunit.client.0.smithi115.stdout:4/396: dwrite d1/d30/f4a [0,4194304] 0 2024-03-22T10:42:42.423 INFO:tasks.workunit.client.0.smithi115.stdout:5/413: read d2/f5e [193289,13444] 0 2024-03-22T10:42:42.423 INFO:tasks.workunit.client.0.smithi115.stdout:0/419: mknod d3/d7/d1b/d35/c8b 0 2024-03-22T10:42:42.427 INFO:tasks.workunit.client.0.smithi115.stdout:5/414: write d2/d15/d1b/d7c/d38/d4a/f75 [475111,81776] 0 2024-03-22T10:42:42.428 INFO:tasks.workunit.client.0.smithi115.stdout:4/397: truncate d1/d16/d35/d40/f48 149007 0 2024-03-22T10:42:42.430 INFO:tasks.workunit.client.0.smithi115.stdout:0/420: symlink d3/d52/l8c 0 2024-03-22T10:42:42.431 INFO:tasks.workunit.client.0.smithi115.stdout:5/415: chown d2/fc 13041 1 2024-03-22T10:42:42.433 INFO:tasks.workunit.client.0.smithi115.stdout:4/398: unlink d1/d30/d42/c1e 0 2024-03-22T10:42:42.435 INFO:tasks.workunit.client.0.smithi115.stdout:5/416: chown d2/d15/d1b/d7c/d38/d43/d49/d81/d83 27182235 1 2024-03-22T10:42:42.436 INFO:tasks.workunit.client.0.smithi115.stdout:4/399: mkdir d1/d30/d4c/d86 0 2024-03-22T10:42:42.441 INFO:tasks.workunit.client.0.smithi115.stdout:5/417: mkdir d2/d15/d1b/d7c/d84/d89 0 2024-03-22T10:42:42.441 INFO:tasks.workunit.client.0.smithi115.stdout:4/400: mknod d1/d30/d4c/d86/c87 0 2024-03-22T10:42:42.442 INFO:tasks.workunit.client.0.smithi115.stdout:5/418: chown d2/f1f 10588 1 2024-03-22T10:42:42.443 INFO:tasks.workunit.client.0.smithi115.stdout:4/401: readlink d1/d69/l37 0 2024-03-22T10:42:42.444 INFO:tasks.workunit.client.0.smithi115.stdout:0/421: truncate d3/d52/f59 3622343 0 2024-03-22T10:42:42.445 INFO:tasks.workunit.client.0.smithi115.stdout:5/419: fsync d2/d15/d1b/d7c/d38/f3d 0 2024-03-22T10:42:42.446 INFO:tasks.workunit.client.0.smithi115.stdout:4/402: stat d1/d16/d35/d40/d5c/d14/f3e 0 2024-03-22T10:42:42.447 INFO:tasks.workunit.client.0.smithi115.stdout:5/420: chown d2/f71 258020858 1 2024-03-22T10:42:42.447 INFO:tasks.workunit.client.0.smithi115.stdout:0/422: write d3/d7/d1b/f2a [680577,105934] 0 2024-03-22T10:42:42.450 INFO:tasks.workunit.client.0.smithi115.stdout:5/421: write d2/d15/d1b/d7c/d38/f3d [4936748,17063] 0 2024-03-22T10:42:42.453 INFO:tasks.workunit.client.0.smithi115.stdout:5/422: write d2/d15/d1b/f7f [545889,85608] 0 2024-03-22T10:42:42.454 INFO:tasks.workunit.client.0.smithi115.stdout:0/423: link d3/d4/c4d d3/da/d4c/d69/d6a/c8d 0 2024-03-22T10:42:42.460 INFO:tasks.workunit.client.0.smithi115.stdout:5/423: dread - d2/df/d13/d68/f6c zero size 2024-03-22T10:42:42.461 INFO:tasks.workunit.client.0.smithi115.stdout:0/424: write d3/d4/d14/f15 [664403,121154] 0 2024-03-22T10:42:42.467 INFO:tasks.workunit.client.0.smithi115.stdout:0/425: mknod d3/d7/d1b/d35/c8e 0 2024-03-22T10:42:42.468 INFO:tasks.workunit.client.0.smithi115.stdout:4/403: dwrite d1/d16/d35/d40/d5c/d14/f3e [0,4194304] 0 2024-03-22T10:42:42.470 INFO:tasks.workunit.client.0.smithi115.stdout:5/424: creat d2/d15/d1b/d7c/f8a x:0 0 0 2024-03-22T10:42:42.471 INFO:tasks.workunit.client.0.smithi115.stdout:0/426: symlink d3/da/d4c/d68/l8f 0 2024-03-22T10:42:42.472 INFO:tasks.workunit.client.0.smithi115.stdout:0/427: readlink d3/d7/d1b/d1f/l43 0 2024-03-22T10:42:42.473 INFO:tasks.workunit.client.0.smithi115.stdout:0/428: readlink d3/da/l13 0 2024-03-22T10:42:42.476 INFO:tasks.workunit.client.0.smithi115.stdout:4/404: rename d1/d16/d35/d40/d54/l59 to d1/d66/d82/l88 0 2024-03-22T10:42:42.477 INFO:tasks.workunit.client.0.smithi115.stdout:5/425: truncate d2/d15/d1b/d7c/d38/d43/d49/f52 1162774 0 2024-03-22T10:42:42.479 INFO:tasks.workunit.client.0.smithi115.stdout:4/405: creat d1/d16/d68/f89 x:0 0 0 2024-03-22T10:42:42.480 INFO:tasks.workunit.client.0.smithi115.stdout:4/406: fdatasync d1/f9 0 2024-03-22T10:42:42.481 INFO:tasks.workunit.client.0.smithi115.stdout:0/429: link d3/d7/d1b/d1f/f58 d3/da/f90 0 2024-03-22T10:42:42.484 INFO:tasks.workunit.client.0.smithi115.stdout:5/426: mkdir d2/df/d13/d8b 0 2024-03-22T10:42:42.484 INFO:tasks.workunit.client.0.smithi115.stdout:0/430: creat d3/da/d4c/f91 x:0 0 0 2024-03-22T10:42:42.487 INFO:tasks.workunit.client.0.smithi115.stdout:4/407: link d1/f9 d1/d30/d4c/d86/f8a 0 2024-03-22T10:42:42.488 INFO:tasks.workunit.client.0.smithi115.stdout:0/431: read d3/d7/d1b/d35/f80 [118438,115979] 0 2024-03-22T10:42:42.490 INFO:tasks.workunit.client.0.smithi115.stdout:4/408: read d1/d69/f62 [2000548,44465] 0 2024-03-22T10:42:42.493 INFO:tasks.workunit.client.0.smithi115.stdout:4/409: symlink d1/d16/d35/d40/l8b 0 2024-03-22T10:42:42.497 INFO:tasks.workunit.client.0.smithi115.stdout:0/432: rename d3/d7/d1b/d1f/d5b/f83 to d3/d7/d1b/f92 0 2024-03-22T10:42:42.499 INFO:tasks.workunit.client.0.smithi115.stdout:0/433: chown d3/d4/c25 1898 1 2024-03-22T10:42:42.500 INFO:tasks.workunit.client.0.smithi115.stdout:5/427: dwrite d2/d15/d1b/d7c/d38/f61 [0,4194304] 0 2024-03-22T10:42:42.506 INFO:tasks.workunit.client.0.smithi115.stdout:5/428: mkdir d2/d8c 0 2024-03-22T10:42:42.512 INFO:tasks.workunit.client.0.smithi115.stdout:4/410: dwrite d1/d16/d35/d40/d5c/d55/f52 [0,4194304] 0 2024-03-22T10:42:42.513 INFO:tasks.workunit.client.0.smithi115.stdout:5/429: rename d2/d15/l22 to d2/df/l8d 0 2024-03-22T10:42:42.518 INFO:tasks.workunit.client.0.smithi115.stdout:4/411: rename d1/d16/f1c to d1/d16/d68/f8c 0 2024-03-22T10:42:42.521 INFO:tasks.workunit.client.0.smithi115.stdout:4/412: rmdir d1/d16/d35/d40/d5c 39 2024-03-22T10:42:42.528 INFO:tasks.workunit.client.0.smithi115.stdout:4/413: link d1/d16/d35/d40/d5c/l56 d1/d16/d35/d40/d5c/d14/l8d 0 2024-03-22T10:42:42.528 INFO:tasks.workunit.client.0.smithi115.stdout:8/642: sync 2024-03-22T10:42:42.528 INFO:tasks.workunit.client.0.smithi115.stdout:3/370: sync 2024-03-22T10:42:42.528 INFO:tasks.workunit.client.0.smithi115.stdout:2/388: sync 2024-03-22T10:42:42.529 INFO:tasks.workunit.client.0.smithi115.stdout:7/409: sync 2024-03-22T10:42:42.529 INFO:tasks.workunit.client.0.smithi115.stdout:6/428: sync 2024-03-22T10:42:42.529 INFO:tasks.workunit.client.0.smithi115.stdout:9/467: sync 2024-03-22T10:42:42.529 INFO:tasks.workunit.client.0.smithi115.stdout:8/643: readlink d6/l13 0 2024-03-22T10:42:42.530 INFO:tasks.workunit.client.0.smithi115.stdout:5/430: write d2/fc [2600994,69584] 0 2024-03-22T10:42:42.531 INFO:tasks.workunit.client.1.smithi150.stdout:1/8: sync 2024-03-22T10:42:42.532 INFO:tasks.workunit.client.0.smithi115.stdout:2/389: chown da/d25/d2a/f4b 956053814 1 2024-03-22T10:42:42.532 INFO:tasks.workunit.client.1.smithi150.stdout:1/9: dread - f0 zero size 2024-03-22T10:42:42.533 INFO:tasks.workunit.client.0.smithi115.stdout:7/410: fdatasync d2/d3a/d6e/f59 0 2024-03-22T10:42:42.534 INFO:tasks.workunit.client.0.smithi115.stdout:4/414: truncate d1/f7 39873 0 2024-03-22T10:42:42.535 INFO:tasks.workunit.client.0.smithi115.stdout:9/468: chown d6/d7/d48/d70/f73 1 1 2024-03-22T10:42:42.536 INFO:tasks.workunit.client.1.smithi150.stdout:1/10: write f0 [549593,65069] 0 2024-03-22T10:42:42.539 INFO:tasks.workunit.client.0.smithi115.stdout:8/644: creat d6/d49/fd0 x:0 0 0 2024-03-22T10:42:42.540 INFO:tasks.workunit.client.1.smithi150.stdout:1/11: mknod d1/c2 0 2024-03-22T10:42:42.541 INFO:tasks.workunit.client.0.smithi115.stdout:0/434: dread d3/d4/d14/d64/f72 [0,4194304] 0 2024-03-22T10:42:42.543 INFO:tasks.workunit.client.0.smithi115.stdout:2/390: mkdir da/d7b 0 2024-03-22T10:42:42.543 INFO:tasks.workunit.client.0.smithi115.stdout:6/429: truncate d4/d13/f4b 510726 0 2024-03-22T10:42:42.544 INFO:tasks.workunit.client.0.smithi115.stdout:4/415: fdatasync d1/d66/f77 0 2024-03-22T10:42:42.546 INFO:tasks.workunit.client.0.smithi115.stdout:6/430: stat d4/d13/f25 0 2024-03-22T10:42:42.547 INFO:tasks.workunit.client.0.smithi115.stdout:7/411: creat d2/d3a/d90/f91 x:0 0 0 2024-03-22T10:42:42.548 INFO:tasks.workunit.client.0.smithi115.stdout:9/469: mknod d6/d7/d30/ca2 0 2024-03-22T10:42:42.549 INFO:tasks.workunit.client.0.smithi115.stdout:1/535: sync 2024-03-22T10:42:42.550 INFO:tasks.workunit.client.0.smithi115.stdout:4/416: chown d1/d16/l2e 2510 1 2024-03-22T10:42:42.552 INFO:tasks.workunit.client.0.smithi115.stdout:0/435: mkdir d3/d7/d1b/d93 0 2024-03-22T10:42:42.553 INFO:tasks.workunit.client.0.smithi115.stdout:9/470: fdatasync d6/d7/d30/f9d 0 2024-03-22T10:42:42.553 INFO:tasks.workunit.client.0.smithi115.stdout:2/391: rename da/d1c/l42 to da/d1c/d67/l7c 0 2024-03-22T10:42:42.556 INFO:tasks.workunit.client.0.smithi115.stdout:6/431: write d4/d13/d33/f8c [457263,129276] 0 2024-03-22T10:42:42.557 INFO:tasks.workunit.client.0.smithi115.stdout:1/536: chown d0/d8/d4d/f6e 472641794 1 2024-03-22T10:42:42.559 INFO:tasks.workunit.client.0.smithi115.stdout:3/371: dwrite d3/d19/f31 [0,4194304] 0 2024-03-22T10:42:42.560 INFO:tasks.workunit.client.0.smithi115.stdout:6/432: fdatasync d4/d36/d48/f67 0 2024-03-22T10:42:42.562 INFO:tasks.workunit.client.0.smithi115.stdout:4/417: creat d1/d66/d82/f8e x:0 0 0 2024-03-22T10:42:42.564 INFO:tasks.workunit.client.0.smithi115.stdout:8/645: dread d6/dd/f6a [0,4194304] 0 2024-03-22T10:42:42.565 INFO:tasks.workunit.client.0.smithi115.stdout:8/646: fdatasync d6/fad 0 2024-03-22T10:42:42.565 INFO:tasks.workunit.client.0.smithi115.stdout:2/392: mkdir da/d25/d3d/d57/d7d 0 2024-03-22T10:42:42.568 INFO:tasks.workunit.client.0.smithi115.stdout:9/471: mknod d6/d7/d48/d70/ca3 0 2024-03-22T10:42:42.569 INFO:tasks.workunit.client.0.smithi115.stdout:2/393: stat da/d1c/f69 0 2024-03-22T10:42:42.569 INFO:tasks.workunit.client.0.smithi115.stdout:5/431: truncate d2/fe 5499986 0 2024-03-22T10:42:42.569 INFO:tasks.workunit.client.0.smithi115.stdout:0/436: write d3/d7/d1b/f92 [566275,59878] 0 2024-03-22T10:42:42.570 INFO:tasks.workunit.client.1.smithi150.stdout:1/12: link f0 d1/f3 0 2024-03-22T10:42:42.571 INFO:tasks.workunit.client.0.smithi115.stdout:2/394: fsync da/d25/f34 0 2024-03-22T10:42:42.573 INFO:tasks.workunit.client.0.smithi115.stdout:2/395: fdatasync da/d25/d30/f2e 0 2024-03-22T10:42:42.575 INFO:tasks.workunit.client.0.smithi115.stdout:1/537: unlink d0/d12/d7b/cab 0 2024-03-22T10:42:42.576 INFO:tasks.workunit.client.1.smithi150.stdout:1/13: mknod d1/c4 0 2024-03-22T10:42:42.578 INFO:tasks.workunit.client.1.smithi150.stdout:1/14: mknod d1/c5 0 2024-03-22T10:42:42.580 INFO:tasks.workunit.client.0.smithi115.stdout:6/433: rmdir d4/d13/d33 39 2024-03-22T10:42:42.582 INFO:tasks.workunit.client.1.smithi150.stdout:1/15: symlink d1/l6 0 2024-03-22T10:42:42.583 INFO:tasks.workunit.client.1.smithi150.stdout:1/16: stat d1/f3 0 2024-03-22T10:42:42.584 INFO:tasks.workunit.client.1.smithi150.stdout:1/17: rmdir d1 39 2024-03-22T10:42:42.586 INFO:tasks.workunit.client.0.smithi115.stdout:7/412: dwrite d2/d7/d12/d10/d19/f8b [0,4194304] 0 2024-03-22T10:42:42.589 INFO:tasks.workunit.client.1.smithi150.stdout:1/18: write d1/f3 [230739,49968] 0 2024-03-22T10:42:42.590 INFO:tasks.workunit.client.0.smithi115.stdout:9/472: mkdir d6/d8/d3c/d59/d68/d97/da4 0 2024-03-22T10:42:42.591 INFO:tasks.workunit.client.0.smithi115.stdout:2/396: dwrite da/d25/d30/f2e [4194304,4194304] 0 2024-03-22T10:42:42.595 INFO:tasks.workunit.client.0.smithi115.stdout:1/538: chown d0/d12/l4b 598232 1 2024-03-22T10:42:42.595 INFO:tasks.workunit.client.1.smithi150.stdout:1/19: read d1/f3 [148935,38797] 0 2024-03-22T10:42:42.598 INFO:tasks.workunit.client.0.smithi115.stdout:3/372: rename d3/d36/d41/c38 to d3/d16/c74 0 2024-03-22T10:42:42.606 INFO:tasks.workunit.client.1.smithi150.stdout:1/20: dread f0 [0,4194304] 0 2024-03-22T10:42:42.606 INFO:tasks.workunit.client.0.smithi115.stdout:0/437: mkdir d3/d7/d6b/d94 0 2024-03-22T10:42:42.607 INFO:tasks.workunit.client.0.smithi115.stdout:7/413: creat d2/d7/d12/d10/d19/d42/d46/d61/f92 x:0 0 0 2024-03-22T10:42:42.607 INFO:tasks.workunit.client.0.smithi115.stdout:8/647: write d6/dd/f6a [770457,98165] 0 2024-03-22T10:42:42.607 INFO:tasks.workunit.client.0.smithi115.stdout:9/473: symlink d6/d7/d19/d4e/la5 0 2024-03-22T10:42:42.607 INFO:tasks.workunit.client.0.smithi115.stdout:2/397: readlink da/d25/d2a/l4d 0 2024-03-22T10:42:42.608 INFO:tasks.workunit.client.0.smithi115.stdout:9/474: chown d6/d7/d48 25020 1 2024-03-22T10:42:42.610 INFO:tasks.workunit.client.1.smithi150.stdout:1/21: write d1/f3 [571616,14126] 0 2024-03-22T10:42:42.612 INFO:tasks.workunit.client.1.smithi150.stdout:6/6: sync 2024-03-22T10:42:42.612 INFO:tasks.workunit.client.1.smithi150.stdout:6/7: rmdir - no directory 2024-03-22T10:42:42.612 INFO:tasks.workunit.client.1.smithi150.stdout:6/8: write - no filename 2024-03-22T10:42:42.612 INFO:tasks.workunit.client.1.smithi150.stdout:3/26: sync 2024-03-22T10:42:42.613 INFO:tasks.workunit.client.1.smithi150.stdout:3/27: write - no filename 2024-03-22T10:42:42.613 INFO:tasks.workunit.client.1.smithi150.stdout:3/28: dread - no filename 2024-03-22T10:42:42.613 INFO:tasks.workunit.client.1.smithi150.stdout:4/50: sync 2024-03-22T10:42:42.613 INFO:tasks.workunit.client.1.smithi150.stdout:8/46: sync 2024-03-22T10:42:42.613 INFO:tasks.workunit.client.1.smithi150.stdout:5/3: sync 2024-03-22T10:42:42.613 INFO:tasks.workunit.client.1.smithi150.stdout:5/4: write - no filename 2024-03-22T10:42:42.613 INFO:tasks.workunit.client.1.smithi150.stdout:7/14: sync 2024-03-22T10:42:42.615 INFO:tasks.workunit.client.0.smithi115.stdout:5/432: rename d2/d15/d1b/d7c/d38/d4f to d2/df/d3b/d8e 0 2024-03-22T10:42:42.615 INFO:tasks.workunit.client.1.smithi150.stdout:2/45: sync 2024-03-22T10:42:42.615 INFO:tasks.workunit.client.1.smithi150.stdout:9/8: sync 2024-03-22T10:42:42.615 INFO:tasks.workunit.client.1.smithi150.stdout:0/81: sync 2024-03-22T10:42:42.615 INFO:tasks.workunit.client.1.smithi150.stdout:2/46: readlink l8 0 2024-03-22T10:42:42.616 INFO:tasks.workunit.client.0.smithi115.stdout:5/433: readlink d2/d15/d32/l41 0 2024-03-22T10:42:42.616 INFO:tasks.workunit.client.0.smithi115.stdout:6/434: mknod d4/d13/d33/d58/c8e 0 2024-03-22T10:42:42.617 INFO:tasks.workunit.client.1.smithi150.stdout:3/29: chown c4 20670 1 2024-03-22T10:42:42.617 INFO:tasks.workunit.client.1.smithi150.stdout:3/30: write - no filename 2024-03-22T10:42:42.617 INFO:tasks.workunit.client.1.smithi150.stdout:1/22: dread f0 [0,4194304] 0 2024-03-22T10:42:42.618 INFO:tasks.workunit.client.1.smithi150.stdout:8/47: write d0/fd [732570,5050] 0 2024-03-22T10:42:42.619 INFO:tasks.workunit.client.1.smithi150.stdout:5/5: creat d0/f2 x:0 0 0 2024-03-22T10:42:42.621 INFO:tasks.workunit.client.0.smithi115.stdout:8/648: fsync d6/dd/d12/f94 0 2024-03-22T10:42:42.621 INFO:tasks.workunit.client.0.smithi115.stdout:7/414: chown d2/d62/l8f 394 1 2024-03-22T10:42:42.622 INFO:tasks.workunit.client.0.smithi115.stdout:2/398: unlink da/f66 0 2024-03-22T10:42:42.623 INFO:tasks.workunit.client.1.smithi150.stdout:8/48: chown d0/d2/fa 74702 1 2024-03-22T10:42:42.624 INFO:tasks.workunit.client.1.smithi150.stdout:8/49: chown d0/dc 603842336 1 2024-03-22T10:42:42.624 INFO:tasks.workunit.client.1.smithi150.stdout:7/15: write f0 [4130593,20738] 0 2024-03-22T10:42:42.624 INFO:tasks.workunit.client.0.smithi115.stdout:9/475: creat d6/d8/d3c/d59/d75/fa6 x:0 0 0 2024-03-22T10:42:42.626 INFO:tasks.workunit.client.1.smithi150.stdout:6/9: rename l2 to l3 0 2024-03-22T10:42:42.626 INFO:tasks.workunit.client.1.smithi150.stdout:6/10: dwrite - no filename 2024-03-22T10:42:42.626 INFO:tasks.workunit.client.1.smithi150.stdout:6/11: truncate - no filename 2024-03-22T10:42:42.626 INFO:tasks.workunit.client.1.smithi150.stdout:6/12: dread - no filename 2024-03-22T10:42:42.627 INFO:tasks.workunit.client.0.smithi115.stdout:5/434: symlink d2/d15/d1b/d7c/d38/d4a/d74/l8f 0 2024-03-22T10:42:42.628 INFO:tasks.workunit.client.1.smithi150.stdout:8/50: stat d0/d2/c10 0 2024-03-22T10:42:42.629 INFO:tasks.workunit.client.1.smithi150.stdout:1/23: fdatasync d1/f3 0 2024-03-22T10:42:42.629 INFO:tasks.workunit.client.0.smithi115.stdout:0/438: unlink d3/da/d4c/l7d 0 2024-03-22T10:42:42.630 INFO:tasks.workunit.client.1.smithi150.stdout:1/24: chown d1/l6 33262 1 2024-03-22T10:42:42.631 INFO:tasks.workunit.client.0.smithi115.stdout:1/539: dwrite d0/d8/d4d/d56/fb2 [0,4194304] 0 2024-03-22T10:42:42.633 INFO:tasks.workunit.client.0.smithi115.stdout:0/439: readlink d3/da/l31 0 2024-03-22T10:42:42.633 INFO:tasks.workunit.client.0.smithi115.stdout:3/373: write d3/f7 [1409299,117438] 0 2024-03-22T10:42:42.634 INFO:tasks.workunit.client.1.smithi150.stdout:5/6: unlink d0/f2 0 2024-03-22T10:42:42.636 INFO:tasks.workunit.client.1.smithi150.stdout:2/47: dread f4 [0,4194304] 0 2024-03-22T10:42:42.639 INFO:tasks.workunit.client.0.smithi115.stdout:2/399: mkdir da/d25/d3d/d7e 0 2024-03-22T10:42:42.639 INFO:tasks.workunit.client.1.smithi150.stdout:7/16: unlink f0 0 2024-03-22T10:42:42.640 INFO:tasks.workunit.client.0.smithi115.stdout:4/418: dwrite d1/d66/f6d [0,4194304] 0 2024-03-22T10:42:42.645 INFO:tasks.workunit.client.0.smithi115.stdout:3/374: unlink d3/d19/d1b/d52/l6c 0 2024-03-22T10:42:42.645 INFO:tasks.workunit.client.0.smithi115.stdout:5/435: fsync d2/d15/d1b/d7c/d38/f77 0 2024-03-22T10:42:42.646 INFO:tasks.workunit.client.0.smithi115.stdout:6/435: creat d4/d11/f8f x:0 0 0 2024-03-22T10:42:42.647 INFO:tasks.workunit.client.0.smithi115.stdout:2/400: fsync da/d1c/d67/f6f 0 2024-03-22T10:42:42.648 INFO:tasks.workunit.client.1.smithi150.stdout:9/9: rename d1/d2 to d1/d3 0 2024-03-22T10:42:42.650 INFO:tasks.workunit.client.0.smithi115.stdout:8/649: dwrite d6/dd/f6a [0,4194304] 0 2024-03-22T10:42:42.651 INFO:tasks.workunit.client.0.smithi115.stdout:4/419: chown d1/d16/d35/d40/d5c/f23 35 1 2024-03-22T10:42:42.655 INFO:tasks.workunit.client.0.smithi115.stdout:4/420: write d1/d30/d42/d4f/f7c [3858906,110930] 0 2024-03-22T10:42:42.657 INFO:tasks.workunit.client.1.smithi150.stdout:2/48: mknod cb 0 2024-03-22T10:42:42.657 INFO:tasks.workunit.client.0.smithi115.stdout:3/375: creat d3/d19/d1b/d3a/f75 x:0 0 0 2024-03-22T10:42:42.659 INFO:tasks.workunit.client.1.smithi150.stdout:2/49: read f1 [458932,11068] 0 2024-03-22T10:42:42.661 INFO:tasks.workunit.client.0.smithi115.stdout:2/401: write da/d25/d30/f31 [1116499,3821] 0 2024-03-22T10:42:42.662 INFO:tasks.workunit.client.1.smithi150.stdout:6/13: rename l1 to l4 0 2024-03-22T10:42:42.664 INFO:tasks.workunit.client.0.smithi115.stdout:8/650: mknod d6/d49/d88/cd1 0 2024-03-22T10:42:42.664 INFO:tasks.workunit.client.1.smithi150.stdout:3/31: symlink d5/l6 0 2024-03-22T10:42:42.665 INFO:tasks.workunit.client.1.smithi150.stdout:3/32: chown c3 9069565 1 2024-03-22T10:42:42.665 INFO:tasks.workunit.client.1.smithi150.stdout:3/33: dread - no filename 2024-03-22T10:42:42.666 INFO:tasks.workunit.client.1.smithi150.stdout:3/34: chown d5 28487635 1 2024-03-22T10:42:42.666 INFO:tasks.workunit.client.1.smithi150.stdout:3/35: fdatasync - no filename 2024-03-22T10:42:42.666 INFO:tasks.workunit.client.1.smithi150.stdout:3/36: truncate - no filename 2024-03-22T10:42:42.666 INFO:tasks.workunit.client.1.smithi150.stdout:3/37: dread - no filename 2024-03-22T10:42:42.667 INFO:tasks.workunit.client.1.smithi150.stdout:3/38: chown d5 39 1 2024-03-22T10:42:42.667 INFO:tasks.workunit.client.1.smithi150.stdout:3/39: dread - no filename 2024-03-22T10:42:42.667 INFO:tasks.workunit.client.1.smithi150.stdout:3/40: dwrite - no filename 2024-03-22T10:42:42.667 INFO:tasks.workunit.client.1.smithi150.stdout:3/41: fsync - no filename 2024-03-22T10:42:42.669 INFO:tasks.workunit.client.0.smithi115.stdout:0/440: dwrite d3/d7/d1c/f1e [0,4194304] 0 2024-03-22T10:42:42.671 INFO:tasks.workunit.client.0.smithi115.stdout:6/436: dwrite d4/d36/d48/d72/f81 [0,4194304] 0 2024-03-22T10:42:42.671 INFO:tasks.workunit.client.0.smithi115.stdout:5/436: creat d2/d8c/f90 x:0 0 0 2024-03-22T10:42:42.672 INFO:tasks.workunit.client.1.smithi150.stdout:9/10: rmdir d1 39 2024-03-22T10:42:42.672 INFO:tasks.workunit.client.1.smithi150.stdout:9/11: dread - no filename 2024-03-22T10:42:42.672 INFO:tasks.workunit.client.1.smithi150.stdout:9/12: write - no filename 2024-03-22T10:42:42.674 INFO:tasks.workunit.client.1.smithi150.stdout:4/51: getdents d0/de 0 2024-03-22T10:42:42.676 INFO:tasks.workunit.client.1.smithi150.stdout:2/50: symlink lc 0 2024-03-22T10:42:42.679 INFO:tasks.workunit.client.1.smithi150.stdout:0/82: getdents d0/d1 0 2024-03-22T10:42:42.680 INFO:tasks.workunit.client.0.smithi115.stdout:9/476: write d6/d7/d20/f4c [720013,67604] 0 2024-03-22T10:42:42.680 INFO:tasks.workunit.client.0.smithi115.stdout:4/421: mknod d1/d16/d68/d79/c8f 0 2024-03-22T10:42:42.682 INFO:tasks.workunit.client.1.smithi150.stdout:3/42: symlink d5/l7 0 2024-03-22T10:42:42.682 INFO:tasks.workunit.client.1.smithi150.stdout:3/43: dwrite - no filename 2024-03-22T10:42:42.682 INFO:tasks.workunit.client.1.smithi150.stdout:3/44: dwrite - no filename 2024-03-22T10:42:42.682 INFO:tasks.workunit.client.1.smithi150.stdout:3/45: fdatasync - no filename 2024-03-22T10:42:42.684 INFO:tasks.workunit.client.1.smithi150.stdout:8/51: link d0/d2/c10 d0/d2/c13 0 2024-03-22T10:42:42.684 INFO:tasks.workunit.client.0.smithi115.stdout:3/376: link d3/f45 d3/d19/f76 0 2024-03-22T10:42:42.685 INFO:tasks.workunit.client.1.smithi150.stdout:8/52: stat d0/f6 0 2024-03-22T10:42:42.686 INFO:tasks.workunit.client.0.smithi115.stdout:6/437: symlink d4/d5e/l90 0 2024-03-22T10:42:42.687 INFO:tasks.workunit.client.1.smithi150.stdout:4/52: mknod d0/de/c16 0 2024-03-22T10:42:42.688 INFO:tasks.workunit.client.0.smithi115.stdout:2/402: creat da/d25/d3d/d57/d7d/f7f x:0 0 0 2024-03-22T10:42:42.689 INFO:tasks.workunit.client.0.smithi115.stdout:5/437: symlink d2/df/d27/d50/l91 0 2024-03-22T10:42:42.690 INFO:tasks.workunit.client.1.smithi150.stdout:2/51: rename l0 to ld 0 2024-03-22T10:42:42.691 INFO:tasks.workunit.client.0.smithi115.stdout:0/441: rename d3/d7/d1b/d35/c8e to d3/da/d4c/d68/c95 0 2024-03-22T10:42:42.692 INFO:tasks.workunit.client.0.smithi115.stdout:2/403: write da/d25/d30/f31 [996601,104378] 0 2024-03-22T10:42:42.693 INFO:tasks.workunit.client.0.smithi115.stdout:8/651: dwrite d6/d49/d68/f6f [0,4194304] 0 2024-03-22T10:42:42.693 INFO:tasks.workunit.client.0.smithi115.stdout:9/477: mknod d6/d7/d19/ca7 0 2024-03-22T10:42:42.697 INFO:tasks.workunit.client.1.smithi150.stdout:0/83: symlink d0/d1/d7/db/d10/l1a 0 2024-03-22T10:42:42.698 INFO:tasks.workunit.client.0.smithi115.stdout:3/377: mknod d3/d19/d1b/d3a/c77 0 2024-03-22T10:42:42.698 INFO:tasks.workunit.client.1.smithi150.stdout:4/53: dread d0/de/f13 [0,4194304] 0 2024-03-22T10:42:42.699 INFO:tasks.workunit.client.1.smithi150.stdout:0/84: chown d0/d12/l14 221753 1 2024-03-22T10:42:42.700 INFO:tasks.workunit.client.0.smithi115.stdout:5/438: truncate d2/f5e 316125 0 2024-03-22T10:42:42.702 INFO:tasks.workunit.client.1.smithi150.stdout:4/54: read - d0/f11 zero size 2024-03-22T10:42:42.702 INFO:tasks.workunit.client.0.smithi115.stdout:6/438: unlink d4/d5e/c6b 0 2024-03-22T10:42:42.703 INFO:tasks.workunit.client.1.smithi150.stdout:9/13: symlink d1/l4 0 2024-03-22T10:42:42.703 INFO:tasks.workunit.client.0.smithi115.stdout:2/404: creat da/d25/d30/d4f/d54/f80 x:0 0 0 2024-03-22T10:42:42.705 INFO:tasks.workunit.client.1.smithi150.stdout:8/53: creat d0/d2/f14 x:0 0 0 2024-03-22T10:42:42.707 INFO:tasks.workunit.client.0.smithi115.stdout:6/439: fdatasync d4/d13/d33/f42 0 2024-03-22T10:42:42.710 INFO:tasks.workunit.client.0.smithi115.stdout:3/378: symlink d3/d19/d1b/d3a/l78 0 2024-03-22T10:42:42.710 INFO:tasks.workunit.client.0.smithi115.stdout:6/440: fdatasync d4/d11/d12/f21 0 2024-03-22T10:42:42.711 INFO:tasks.workunit.client.1.smithi150.stdout:2/52: rename f4 to fe 0 2024-03-22T10:42:42.712 INFO:tasks.workunit.client.0.smithi115.stdout:2/405: chown da/d1c/f4c 3207956 1 2024-03-22T10:42:42.712 INFO:tasks.workunit.client.1.smithi150.stdout:0/85: fsync d0/d1/fa 0 2024-03-22T10:42:42.713 INFO:tasks.workunit.client.0.smithi115.stdout:2/406: fsync da/f1b 0 2024-03-22T10:42:42.714 INFO:tasks.workunit.client.0.smithi115.stdout:6/441: fdatasync d4/d13/d33/f8c 0 2024-03-22T10:42:42.714 INFO:tasks.workunit.client.0.smithi115.stdout:7/415: dread d2/d7/d12/d10/f40 [0,4194304] 0 2024-03-22T10:42:42.714 INFO:tasks.workunit.client.1.smithi150.stdout:2/53: write f6 [1743336,45238] 0 2024-03-22T10:42:42.716 INFO:tasks.workunit.client.1.smithi150.stdout:4/55: creat d0/f17 x:0 0 0 2024-03-22T10:42:42.717 INFO:tasks.workunit.client.0.smithi115.stdout:3/379: creat d3/d19/d61/f79 x:0 0 0 2024-03-22T10:42:42.718 INFO:tasks.workunit.client.0.smithi115.stdout:2/407: read da/d25/d30/f31 [6990758,9150] 0 2024-03-22T10:42:42.721 INFO:tasks.workunit.client.0.smithi115.stdout:0/442: dwrite d3/d7/d1b/f2a [0,4194304] 0 2024-03-22T10:42:42.723 INFO:tasks.workunit.client.1.smithi150.stdout:9/14: getdents d1/d3 0 2024-03-22T10:42:42.723 INFO:tasks.workunit.client.1.smithi150.stdout:9/15: dwrite - no filename 2024-03-22T10:42:42.723 INFO:tasks.workunit.client.1.smithi150.stdout:9/16: write - no filename 2024-03-22T10:42:42.723 INFO:tasks.workunit.client.1.smithi150.stdout:9/17: dread - no filename 2024-03-22T10:42:42.723 INFO:tasks.workunit.client.0.smithi115.stdout:2/408: stat da/c10 0 2024-03-22T10:42:42.728 INFO:tasks.workunit.client.1.smithi150.stdout:4/56: rename d0/f17 to d0/de/f18 0 2024-03-22T10:42:42.729 INFO:tasks.workunit.client.0.smithi115.stdout:0/443: write d3/f33 [855394,26058] 0 2024-03-22T10:42:42.733 INFO:tasks.workunit.client.0.smithi115.stdout:9/478: dwrite d6/d8/d3c/d59/d75/fa6 [0,4194304] 0 2024-03-22T10:42:42.733 INFO:tasks.workunit.client.0.smithi115.stdout:3/380: mknod d3/d19/d2b/d49/c7a 0 2024-03-22T10:42:42.734 INFO:tasks.workunit.client.1.smithi150.stdout:4/57: write d0/f14 [12632,126394] 0 2024-03-22T10:42:42.734 INFO:tasks.workunit.client.0.smithi115.stdout:0/444: stat d3/d4/d14/f4f 0 2024-03-22T10:42:42.735 INFO:tasks.workunit.client.1.smithi150.stdout:9/18: symlink d1/d3/l5 0 2024-03-22T10:42:42.736 INFO:tasks.workunit.client.0.smithi115.stdout:5/439: dwrite d2/d15/d1b/d7c/f63 [0,4194304] 0 2024-03-22T10:42:42.737 INFO:tasks.workunit.client.0.smithi115.stdout:0/445: readlink d3/d7/d1c/l30 0 2024-03-22T10:42:42.739 INFO:tasks.workunit.client.0.smithi115.stdout:8/652: truncate d6/dd/d24/d3a/f3c 2166182 0 2024-03-22T10:42:42.741 INFO:tasks.workunit.client.0.smithi115.stdout:0/446: chown d3/da/d4c/d68/l8f 23482 1 2024-03-22T10:42:42.742 INFO:tasks.workunit.client.1.smithi150.stdout:9/19: mknod d1/d3/c6 0 2024-03-22T10:42:42.742 INFO:tasks.workunit.client.1.smithi150.stdout:9/20: write - no filename 2024-03-22T10:42:42.746 INFO:tasks.workunit.client.1.smithi150.stdout:9/21: mknod d1/c7 0 2024-03-22T10:42:42.746 INFO:tasks.workunit.client.1.smithi150.stdout:9/22: truncate - no filename 2024-03-22T10:42:42.749 INFO:tasks.workunit.client.0.smithi115.stdout:4/422: dread d1/d30/d42/f45 [0,4194304] 0 2024-03-22T10:42:42.749 INFO:tasks.workunit.client.0.smithi115.stdout:5/440: stat d2/df/d27/c30 0 2024-03-22T10:42:42.752 INFO:tasks.workunit.client.1.smithi150.stdout:2/54: dwrite fe [4194304,4194304] 0 2024-03-22T10:42:42.752 INFO:tasks.workunit.client.1.smithi150.stdout:6/14: readlink l3 0 2024-03-22T10:42:42.752 INFO:tasks.workunit.client.1.smithi150.stdout:6/15: fsync - no filename 2024-03-22T10:42:42.753 INFO:tasks.workunit.client.1.smithi150.stdout:4/58: dread d0/f14 [0,4194304] 0 2024-03-22T10:42:42.753 INFO:tasks.workunit.client.1.smithi150.stdout:2/55: truncate f2 5028158 0 2024-03-22T10:42:42.755 INFO:tasks.workunit.client.1.smithi150.stdout:9/23: mkdir d1/d8 0 2024-03-22T10:42:42.758 INFO:tasks.workunit.client.0.smithi115.stdout:4/423: truncate d1/d16/d35/d40/d5c/d14/f1b 8497964 0 2024-03-22T10:42:42.758 INFO:tasks.workunit.client.1.smithi150.stdout:6/16: creat f5 x:0 0 0 2024-03-22T10:42:42.761 INFO:tasks.workunit.client.1.smithi150.stdout:5/7: rmdir d0 39 2024-03-22T10:42:42.761 INFO:tasks.workunit.client.1.smithi150.stdout:5/8: dread - no filename 2024-03-22T10:42:42.761 INFO:tasks.workunit.client.1.smithi150.stdout:6/17: write f5 [881405,88873] 0 2024-03-22T10:42:42.761 INFO:tasks.workunit.client.1.smithi150.stdout:4/59: write d0/f14 [831994,85283] 0 2024-03-22T10:42:42.767 INFO:tasks.workunit.client.0.smithi115.stdout:5/441: creat d2/df/d3b/f92 x:0 0 0 2024-03-22T10:42:42.769 INFO:tasks.workunit.client.1.smithi150.stdout:9/24: creat d1/d3/f9 x:0 0 0 2024-03-22T10:42:42.769 INFO:tasks.workunit.client.1.smithi150.stdout:4/60: chown d0/de/cf 397 1 2024-03-22T10:42:42.770 INFO:tasks.workunit.client.0.smithi115.stdout:7/416: dread d2/f5 [0,4194304] 0 2024-03-22T10:42:42.775 INFO:tasks.workunit.client.1.smithi150.stdout:4/61: chown d0/cc 6 1 2024-03-22T10:42:42.777 INFO:tasks.workunit.client.0.smithi115.stdout:0/447: link d3/l12 d3/da/d4c/d69/l96 0 2024-03-22T10:42:42.779 INFO:tasks.workunit.client.1.smithi150.stdout:3/46: getdents d5 0 2024-03-22T10:42:42.780 INFO:tasks.workunit.client.0.smithi115.stdout:5/442: mkdir d2/df/d13/d8b/d93 0 2024-03-22T10:42:42.781 INFO:tasks.workunit.client.1.smithi150.stdout:2/56: dwrite f2 [0,4194304] 0 2024-03-22T10:42:42.787 INFO:tasks.workunit.client.1.smithi150.stdout:3/47: creat d5/f8 x:0 0 0 2024-03-22T10:42:42.789 INFO:tasks.workunit.client.0.smithi115.stdout:5/443: mknod d2/df/d13/c94 0 2024-03-22T10:42:42.789 INFO:tasks.workunit.client.0.smithi115.stdout:7/417: link d2/d7/d12/d10/d23/l34 d2/d7/d12/d10/d19/d42/d46/l93 0 2024-03-22T10:42:42.792 INFO:tasks.workunit.client.0.smithi115.stdout:6/442: dread d4/d11/d12/d1c/f69 [0,4194304] 0 2024-03-22T10:42:42.792 INFO:tasks.workunit.client.0.smithi115.stdout:5/444: symlink d2/d15/d1b/d7c/d38/l95 0 2024-03-22T10:42:42.793 INFO:tasks.workunit.client.1.smithi150.stdout:3/48: write d5/f8 [362741,60575] 0 2024-03-22T10:42:42.795 INFO:tasks.workunit.client.1.smithi150.stdout:8/54: write d0/d2/f12 [862626,111094] 0 2024-03-22T10:42:42.796 INFO:tasks.workunit.client.0.smithi115.stdout:8/653: truncate d6/dd/d24/fa7 2376987 0 2024-03-22T10:42:42.796 INFO:tasks.workunit.client.0.smithi115.stdout:7/418: mkdir d2/d7/d12/d10/d19/d42/d46/d94 0 2024-03-22T10:42:42.797 INFO:tasks.workunit.client.1.smithi150.stdout:6/18: dwrite f5 [0,4194304] 0 2024-03-22T10:42:42.802 INFO:tasks.workunit.client.0.smithi115.stdout:6/443: mkdir d4/d11/d91 0 2024-03-22T10:42:42.803 INFO:tasks.workunit.client.0.smithi115.stdout:3/381: dread d3/f12 [0,4194304] 0 2024-03-22T10:42:42.803 INFO:tasks.workunit.client.1.smithi150.stdout:6/19: write f5 [2169119,12462] 0 2024-03-22T10:42:42.804 INFO:tasks.workunit.client.0.smithi115.stdout:7/419: fdatasync d2/d7/d12/d10/f25 0 2024-03-22T10:42:42.806 INFO:tasks.workunit.client.1.smithi150.stdout:9/25: rename d1/d3 to d1/d8/da 0 2024-03-22T10:42:42.806 INFO:tasks.workunit.client.0.smithi115.stdout:5/445: getdents d2/d15/d1b/d7c/d38/d4a/d74 0 2024-03-22T10:42:42.807 INFO:tasks.workunit.client.0.smithi115.stdout:8/654: creat d6/fd2 x:0 0 0 2024-03-22T10:42:42.808 INFO:tasks.workunit.client.0.smithi115.stdout:4/424: dwrite d1/d16/d35/d40/f48 [0,4194304] 0 2024-03-22T10:42:42.808 INFO:tasks.workunit.client.0.smithi115.stdout:3/382: rename d3/c24 to d3/d16/c7b 0 2024-03-22T10:42:42.809 INFO:tasks.workunit.client.0.smithi115.stdout:6/444: creat d4/f92 x:0 0 0 2024-03-22T10:42:42.809 INFO:tasks.workunit.client.1.smithi150.stdout:6/20: chown f5 33 1 2024-03-22T10:42:42.810 INFO:tasks.workunit.client.0.smithi115.stdout:8/655: write d6/d49/f84 [4208529,113876] 0 2024-03-22T10:42:42.813 INFO:tasks.workunit.client.0.smithi115.stdout:8/656: stat d6/d49/d68/f6f 0 2024-03-22T10:42:42.814 INFO:tasks.workunit.client.0.smithi115.stdout:3/383: chown d3/d19/d1b/d54/f59 251474281 1 2024-03-22T10:42:42.816 INFO:tasks.workunit.client.0.smithi115.stdout:5/446: write d2/d15/d1b/d7c/d38/d4a/d74/f7e [1643556,92786] 0 2024-03-22T10:42:42.816 INFO:tasks.workunit.client.0.smithi115.stdout:7/420: truncate d2/d7/d12/d10/d19/d42/d46/f68 357307 0 2024-03-22T10:42:42.817 INFO:tasks.workunit.client.0.smithi115.stdout:4/425: creat d1/d16/d35/d40/d5c/d14/f90 x:0 0 0 2024-03-22T10:42:42.819 INFO:tasks.workunit.client.1.smithi150.stdout:8/55: getdents d0/dc/df 0 2024-03-22T10:42:42.820 INFO:tasks.workunit.client.0.smithi115.stdout:5/447: stat d2/df/d3b/f2a 0 2024-03-22T10:42:42.823 INFO:tasks.workunit.client.1.smithi150.stdout:6/21: mknod c6 0 2024-03-22T10:42:42.826 INFO:tasks.workunit.client.0.smithi115.stdout:4/426: unlink d1/d16/d68/d79/c8f 0 2024-03-22T10:42:42.827 INFO:tasks.workunit.client.0.smithi115.stdout:5/448: dread d2/df/f67 [0,4194304] 0 2024-03-22T10:42:42.833 INFO:tasks.workunit.client.1.smithi150.stdout:6/22: symlink l7 0 2024-03-22T10:42:42.834 INFO:tasks.workunit.client.0.smithi115.stdout:6/445: dwrite d4/d13/d33/f42 [0,4194304] 0 2024-03-22T10:42:42.835 INFO:tasks.workunit.client.1.smithi150.stdout:8/56: fsync d0/f5 0 2024-03-22T10:42:42.835 INFO:tasks.workunit.client.0.smithi115.stdout:8/657: rename d6/d49/d68/l71 to d6/dd/ld3 0 2024-03-22T10:42:42.844 INFO:tasks.workunit.client.0.smithi115.stdout:8/658: symlink d6/dd/d24/d4a/d41/ld4 0 2024-03-22T10:42:42.847 INFO:tasks.workunit.client.1.smithi150.stdout:4/62: dwrite d0/de/f18 [0,4194304] 0 2024-03-22T10:42:42.850 INFO:tasks.workunit.client.1.smithi150.stdout:2/57: dwrite f2 [0,4194304] 0 2024-03-22T10:42:42.851 INFO:tasks.workunit.client.0.smithi115.stdout:3/384: dwrite d3/f12 [0,4194304] 0 2024-03-22T10:42:42.855 INFO:tasks.workunit.client.0.smithi115.stdout:3/385: readlink d3/d19/d2b/l68 0 2024-03-22T10:42:42.856 INFO:tasks.workunit.client.1.smithi150.stdout:2/58: fsync fe 0 2024-03-22T10:42:42.857 INFO:tasks.workunit.client.1.smithi150.stdout:3/49: dwrite d5/f8 [0,4194304] 0 2024-03-22T10:42:42.857 INFO:tasks.workunit.client.1.smithi150.stdout:1/25: sync 2024-03-22T10:42:42.858 INFO:tasks.workunit.client.1.smithi150.stdout:8/57: creat d0/d2/f15 x:0 0 0 2024-03-22T10:42:42.860 INFO:tasks.workunit.client.0.smithi115.stdout:8/659: chown d6/dd/d24/d4a/cc7 1014554 1 2024-03-22T10:42:42.860 INFO:tasks.workunit.client.0.smithi115.stdout:7/421: dwrite d2/d7/d12/d10/f25 [8388608,4194304] 0 2024-03-22T10:42:42.862 INFO:tasks.workunit.client.1.smithi150.stdout:8/58: chown d0/f1 1791722 1 2024-03-22T10:42:42.863 INFO:tasks.workunit.client.1.smithi150.stdout:3/50: chown d5/l7 870 1 2024-03-22T10:42:42.863 INFO:tasks.workunit.client.1.smithi150.stdout:4/63: unlink d0/de/c16 0 2024-03-22T10:42:42.863 INFO:tasks.workunit.client.1.smithi150.stdout:2/59: chown f2 0 1 2024-03-22T10:42:42.865 INFO:tasks.workunit.client.1.smithi150.stdout:1/26: rmdir d1 39 2024-03-22T10:42:42.867 INFO:tasks.workunit.client.0.smithi115.stdout:7/422: creat d2/d3a/d6e/f95 x:0 0 0 2024-03-22T10:42:42.867 INFO:tasks.workunit.client.1.smithi150.stdout:1/27: stat f0 0 2024-03-22T10:42:42.867 INFO:tasks.workunit.client.0.smithi115.stdout:4/427: dwrite d1/d16/d68/f89 [0,4194304] 0 2024-03-22T10:42:42.870 INFO:tasks.workunit.client.1.smithi150.stdout:4/64: write d0/de/f12 [251804,57997] 0 2024-03-22T10:42:42.870 INFO:tasks.workunit.client.1.smithi150.stdout:2/60: write f1 [1239852,119801] 0 2024-03-22T10:42:42.871 INFO:tasks.workunit.client.1.smithi150.stdout:3/51: symlink d5/l9 0 2024-03-22T10:42:42.873 INFO:tasks.workunit.client.0.smithi115.stdout:8/660: rename d6/dd/d12/d31/c52 to d6/dd/d12/dc6/cd5 0 2024-03-22T10:42:42.873 INFO:tasks.workunit.client.1.smithi150.stdout:9/26: dwrite d1/d8/da/f9 [0,4194304] 0 2024-03-22T10:42:42.873 INFO:tasks.workunit.client.1.smithi150.stdout:6/23: fsync f5 0 2024-03-22T10:42:42.876 INFO:tasks.workunit.client.0.smithi115.stdout:3/386: link d3/d16/d29/c3d d3/d19/d2b/c7c 0 2024-03-22T10:42:42.880 INFO:tasks.workunit.client.1.smithi150.stdout:3/52: mkdir d5/da 0 2024-03-22T10:42:42.883 INFO:tasks.workunit.client.1.smithi150.stdout:4/65: symlink d0/de/l19 0 2024-03-22T10:42:42.885 INFO:tasks.workunit.client.1.smithi150.stdout:2/61: creat ff x:0 0 0 2024-03-22T10:42:42.887 INFO:tasks.workunit.client.0.smithi115.stdout:6/446: dwrite d4/d11/d12/f31 [0,4194304] 0 2024-03-22T10:42:42.888 INFO:tasks.workunit.client.1.smithi150.stdout:6/24: chown l4 2993 1 2024-03-22T10:42:42.888 INFO:tasks.workunit.client.1.smithi150.stdout:9/27: mknod d1/d8/cb 0 2024-03-22T10:42:42.890 INFO:tasks.workunit.client.0.smithi115.stdout:7/423: creat d2/d7/d12/d10/f96 x:0 0 0 2024-03-22T10:42:42.890 INFO:tasks.workunit.client.0.smithi115.stdout:3/387: symlink d3/d16/d29/d35/l7d 0 2024-03-22T10:42:42.890 INFO:tasks.workunit.client.1.smithi150.stdout:8/59: creat d0/f16 x:0 0 0 2024-03-22T10:42:42.892 INFO:tasks.workunit.client.1.smithi150.stdout:1/28: creat d1/f7 x:0 0 0 2024-03-22T10:42:42.894 INFO:tasks.workunit.client.1.smithi150.stdout:3/53: creat d5/fb x:0 0 0 2024-03-22T10:42:42.898 INFO:tasks.workunit.client.1.smithi150.stdout:2/62: unlink f1 0 2024-03-22T10:42:42.900 INFO:tasks.workunit.client.1.smithi150.stdout:4/66: dread d0/de/f13 [0,4194304] 0 2024-03-22T10:42:42.906 INFO:tasks.workunit.client.1.smithi150.stdout:9/28: symlink d1/d8/lc 0 2024-03-22T10:42:42.907 INFO:tasks.workunit.client.1.smithi150.stdout:6/25: rename l4 to l8 0 2024-03-22T10:42:42.907 INFO:tasks.workunit.client.0.smithi115.stdout:5/449: dwrite d2/d15/d1b/d7c/d38/d43/d49/f52 [0,4194304] 0 2024-03-22T10:42:42.908 INFO:tasks.workunit.client.1.smithi150.stdout:6/26: chown l7 3 1 2024-03-22T10:42:42.908 INFO:tasks.workunit.client.1.smithi150.stdout:8/60: write d0/f1 [4699560,15474] 0 2024-03-22T10:42:42.909 INFO:tasks.workunit.client.1.smithi150.stdout:1/29: mknod d1/c8 0 2024-03-22T10:42:42.909 INFO:tasks.workunit.client.0.smithi115.stdout:6/447: mknod d4/d11/d12/d1c/c93 0 2024-03-22T10:42:42.910 INFO:tasks.workunit.client.0.smithi115.stdout:7/424: dread d2/d7/d12/d10/f7b [0,4194304] 0 2024-03-22T10:42:42.911 INFO:tasks.workunit.client.1.smithi150.stdout:1/30: truncate f0 1142537 0 2024-03-22T10:42:42.912 INFO:tasks.workunit.client.0.smithi115.stdout:3/388: dread d3/d19/d1b/d52/f56 [0,4194304] 0 2024-03-22T10:42:42.916 INFO:tasks.workunit.client.1.smithi150.stdout:2/63: creat f10 x:0 0 0 2024-03-22T10:42:42.916 INFO:tasks.workunit.client.0.smithi115.stdout:5/450: mkdir d2/df/d27/d50/d96 0 2024-03-22T10:42:42.917 INFO:tasks.workunit.client.0.smithi115.stdout:6/448: creat d4/d36/d48/f94 x:0 0 0 2024-03-22T10:42:42.917 INFO:tasks.workunit.client.0.smithi115.stdout:7/425: fdatasync d2/f5 0 2024-03-22T10:42:42.917 INFO:tasks.workunit.client.1.smithi150.stdout:6/27: mknod c9 0 2024-03-22T10:42:42.921 INFO:tasks.workunit.client.1.smithi150.stdout:6/28: stat c9 0 2024-03-22T10:42:42.921 INFO:tasks.workunit.client.1.smithi150.stdout:9/29: mknod d1/d8/cd 0 2024-03-22T10:42:42.922 INFO:tasks.workunit.client.1.smithi150.stdout:6/29: write f5 [625312,17582] 0 2024-03-22T10:42:42.922 INFO:tasks.workunit.client.0.smithi115.stdout:3/389: mkdir d3/d19/d1b/d3a/d7e 0 2024-03-22T10:42:42.926 INFO:tasks.workunit.client.0.smithi115.stdout:5/451: creat d2/d15/d1b/d7c/d84/f97 x:0 0 0 2024-03-22T10:42:42.928 INFO:tasks.workunit.client.0.smithi115.stdout:6/449: getdents d4/d11/d12/d1c/d84 0 2024-03-22T10:42:42.930 INFO:tasks.workunit.client.0.smithi115.stdout:6/450: chown d4/d11/d12/d1c/f7c 236608984 1 2024-03-22T10:42:42.934 INFO:tasks.workunit.client.0.smithi115.stdout:5/452: rename d2/f2e to d2/df/d27/d7a/f98 0 2024-03-22T10:42:42.935 INFO:tasks.workunit.client.1.smithi150.stdout:9/30: mkdir d1/d8/de 0 2024-03-22T10:42:42.941 INFO:tasks.workunit.client.0.smithi115.stdout:3/390: dread d3/f23 [0,4194304] 0 2024-03-22T10:42:42.942 INFO:tasks.workunit.client.0.smithi115.stdout:3/391: fsync d3/d19/f31 0 2024-03-22T10:42:42.943 INFO:tasks.workunit.client.1.smithi150.stdout:8/61: creat d0/f17 x:0 0 0 2024-03-22T10:42:42.943 INFO:tasks.workunit.client.0.smithi115.stdout:4/428: dwrite d1/d16/d35/d40/d5c/d14/f1d [0,4194304] 0 2024-03-22T10:42:42.946 INFO:tasks.workunit.client.1.smithi150.stdout:9/31: unlink d1/l4 0 2024-03-22T10:42:42.948 INFO:tasks.workunit.client.1.smithi150.stdout:1/31: dwrite f0 [0,4194304] 0 2024-03-22T10:42:42.951 INFO:tasks.workunit.client.0.smithi115.stdout:5/453: dread d2/d15/d1b/d7c/d38/d43/d49/f52 [0,4194304] 0 2024-03-22T10:42:42.952 INFO:tasks.workunit.client.1.smithi150.stdout:2/64: dread fe [4194304,4194304] 0 2024-03-22T10:42:42.952 INFO:tasks.workunit.client.1.smithi150.stdout:9/32: readlink d1/d8/da/l5 0 2024-03-22T10:42:42.952 INFO:tasks.workunit.client.1.smithi150.stdout:6/30: dread f5 [0,4194304] 0 2024-03-22T10:42:42.952 INFO:tasks.workunit.client.1.smithi150.stdout:6/31: rmdir - no directory 2024-03-22T10:42:42.955 INFO:tasks.workunit.client.0.smithi115.stdout:5/454: write d2/d15/d1b/d7c/d38/d4a/f75 [1003771,90130] 0 2024-03-22T10:42:42.956 INFO:tasks.workunit.client.1.smithi150.stdout:2/65: write f10 [555948,10359] 0 2024-03-22T10:42:42.958 INFO:tasks.workunit.client.0.smithi115.stdout:5/455: chown d2/df/f55 743979 1 2024-03-22T10:42:42.959 INFO:tasks.workunit.client.1.smithi150.stdout:9/33: mkdir d1/d8/de/df 0 2024-03-22T10:42:42.960 INFO:tasks.workunit.client.0.smithi115.stdout:5/456: stat d2/d15/l35 0 2024-03-22T10:42:42.961 INFO:tasks.workunit.client.1.smithi150.stdout:2/66: unlink f6 0 2024-03-22T10:42:42.964 INFO:tasks.workunit.client.1.smithi150.stdout:2/67: truncate f10 1157471 0 2024-03-22T10:42:42.964 INFO:tasks.workunit.client.0.smithi115.stdout:5/457: write d2/f71 [736860,127659] 0 2024-03-22T10:42:42.966 INFO:tasks.workunit.client.0.smithi115.stdout:4/429: dwrite d1/d66/d82/f8e [0,4194304] 0 2024-03-22T10:42:42.969 INFO:tasks.workunit.client.1.smithi150.stdout:2/68: dread - ff zero size 2024-03-22T10:42:42.978 INFO:tasks.workunit.client.0.smithi115.stdout:5/458: symlink d2/df/d27/d50/d96/l99 0 2024-03-22T10:42:42.978 INFO:tasks.workunit.client.0.smithi115.stdout:7/426: dread d2/f81 [0,4194304] 0 2024-03-22T10:42:42.978 INFO:tasks.workunit.client.0.smithi115.stdout:5/459: chown d2/df/d27/l42 47445 1 2024-03-22T10:42:42.979 INFO:tasks.workunit.client.0.smithi115.stdout:7/427: stat d2/d3a/d6e/f95 0 2024-03-22T10:42:42.979 INFO:tasks.workunit.client.1.smithi150.stdout:2/69: stat ld 0 2024-03-22T10:42:42.979 INFO:tasks.workunit.client.1.smithi150.stdout:8/62: dwrite d0/f5 [0,4194304] 0 2024-03-22T10:42:42.980 INFO:tasks.workunit.client.0.smithi115.stdout:5/460: readlink d2/d15/d1b/d7c/d38/d4a/d74/l8f 0 2024-03-22T10:42:42.981 INFO:tasks.workunit.client.1.smithi150.stdout:2/70: readlink ld 0 2024-03-22T10:42:42.982 INFO:tasks.workunit.client.1.smithi150.stdout:9/34: link d1/d8/da/l5 d1/d8/de/df/l10 0 2024-03-22T10:42:42.983 INFO:tasks.workunit.client.1.smithi150.stdout:6/32: dread f5 [0,4194304] 0 2024-03-22T10:42:42.991 INFO:tasks.workunit.client.1.smithi150.stdout:2/71: rename f2 to f11 0 2024-03-22T10:42:42.991 INFO:tasks.workunit.client.1.smithi150.stdout:1/32: dwrite f0 [0,4194304] 0 2024-03-22T10:42:42.991 INFO:tasks.workunit.client.0.smithi115.stdout:6/451: dread d4/d36/f5c [0,4194304] 0 2024-03-22T10:42:42.991 INFO:tasks.workunit.client.0.smithi115.stdout:4/430: getdents d1/d30/d4c 0 2024-03-22T10:42:42.992 INFO:tasks.workunit.client.1.smithi150.stdout:7/17: sync 2024-03-22T10:42:42.992 INFO:tasks.workunit.client.1.smithi150.stdout:7/18: dread - no filename 2024-03-22T10:42:42.993 INFO:tasks.workunit.client.1.smithi150.stdout:7/19: dwrite - no filename 2024-03-22T10:42:42.993 INFO:tasks.workunit.client.1.smithi150.stdout:7/20: write - no filename 2024-03-22T10:42:42.996 INFO:tasks.workunit.client.0.smithi115.stdout:7/428: creat d2/d7/d12/d10/d23/f97 x:0 0 0 2024-03-22T10:42:42.996 INFO:tasks.workunit.client.1.smithi150.stdout:6/33: write f5 [2336238,35949] 0 2024-03-22T10:42:42.998 INFO:tasks.workunit.client.0.smithi115.stdout:6/452: mknod d4/d11/d91/c95 0 2024-03-22T10:42:42.999 INFO:tasks.workunit.client.1.smithi150.stdout:2/72: mkdir d12 0 2024-03-22T10:42:43.001 INFO:tasks.workunit.client.1.smithi150.stdout:7/21: symlink d2/l4 0 2024-03-22T10:42:43.002 INFO:tasks.workunit.client.0.smithi115.stdout:4/431: link d1/d16/d35/d40/d5c/ce d1/d16/d35/d40/d5c/d6b/c91 0 2024-03-22T10:42:43.003 INFO:tasks.workunit.client.1.smithi150.stdout:2/73: read f10 [683463,125076] 0 2024-03-22T10:42:43.004 INFO:tasks.workunit.client.1.smithi150.stdout:8/63: link d0/fd d0/d2/f18 0 2024-03-22T10:42:43.006 INFO:tasks.workunit.client.1.smithi150.stdout:2/74: chown lc 40264385 1 2024-03-22T10:42:43.006 INFO:tasks.workunit.client.0.smithi115.stdout:3/392: write d3/d36/d41/f3f [360757,62857] 0 2024-03-22T10:42:43.007 INFO:tasks.workunit.client.0.smithi115.stdout:4/432: rmdir d1/d16/d35/d40/d5c/d55 39 2024-03-22T10:42:43.007 INFO:tasks.workunit.client.1.smithi150.stdout:6/34: mknod ca 0 2024-03-22T10:42:43.008 INFO:tasks.workunit.client.1.smithi150.stdout:9/35: dread d1/d8/da/f9 [0,4194304] 0 2024-03-22T10:42:43.009 INFO:tasks.workunit.client.0.smithi115.stdout:7/429: rename d2/d7/d12/f4b to d2/d7/d12/d10/d19/d42/d76/f98 0 2024-03-22T10:42:43.009 INFO:tasks.workunit.client.0.smithi115.stdout:5/461: dwrite d2/df/f67 [0,4194304] 0 2024-03-22T10:42:43.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:42 smithi115 ceph-mon[29111]: mgrmap e30: smithi115.iqzwsz(active, since 13s), standbys: smithi150.fmoumv 2024-03-22T10:42:43.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:42 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi150.fmoumv", "id": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:42:43.011 INFO:tasks.workunit.client.1.smithi150.stdout:3/54: getdents d5 0 2024-03-22T10:42:43.012 INFO:tasks.workunit.client.0.smithi115.stdout:4/433: write d1/d30/d42/d32/f60 [3388552,5601] 0 2024-03-22T10:42:43.012 INFO:tasks.workunit.client.0.smithi115.stdout:3/393: creat d3/d36/d41/d33/f7f x:0 0 0 2024-03-22T10:42:43.013 INFO:tasks.workunit.client.1.smithi150.stdout:7/22: mkdir d2/d5 0 2024-03-22T10:42:43.013 INFO:tasks.workunit.client.1.smithi150.stdout:7/23: dread - no filename 2024-03-22T10:42:43.013 INFO:tasks.workunit.client.1.smithi150.stdout:7/24: dread - no filename 2024-03-22T10:42:43.013 INFO:tasks.workunit.client.1.smithi150.stdout:7/25: dread - no filename 2024-03-22T10:42:43.014 INFO:tasks.workunit.client.0.smithi115.stdout:7/430: unlink d2/d7/d12/d10/f28 0 2024-03-22T10:42:43.015 INFO:tasks.workunit.client.0.smithi115.stdout:4/434: write d1/d16/d35/d40/d54/f7a [674255,23878] 0 2024-03-22T10:42:43.015 INFO:tasks.workunit.client.0.smithi115.stdout:5/462: creat d2/d15/d1b/d7c/d84/f9a x:0 0 0 2024-03-22T10:42:43.016 INFO:tasks.workunit.client.1.smithi150.stdout:5/9: sync 2024-03-22T10:42:43.016 INFO:tasks.workunit.client.1.smithi150.stdout:5/10: write - no filename 2024-03-22T10:42:43.016 INFO:tasks.workunit.client.1.smithi150.stdout:4/67: sync 2024-03-22T10:42:43.017 INFO:tasks.workunit.client.1.smithi150.stdout:0/86: sync 2024-03-22T10:42:43.019 INFO:tasks.workunit.client.0.smithi115.stdout:5/463: stat d2/df/d3b/l6d 0 2024-03-22T10:42:43.019 INFO:tasks.workunit.client.0.smithi115.stdout:7/431: creat d2/d7/d12/f99 x:0 0 0 2024-03-22T10:42:43.022 INFO:tasks.workunit.client.0.smithi115.stdout:4/435: write d1/d16/d35/d40/d5c/f25 [925152,90092] 0 2024-03-22T10:42:43.024 INFO:tasks.workunit.client.0.smithi115.stdout:5/464: creat d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f9b x:0 0 0 2024-03-22T10:42:43.025 INFO:tasks.workunit.client.1.smithi150.stdout:7/26: creat d2/f6 x:0 0 0 2024-03-22T10:42:43.025 INFO:tasks.workunit.client.0.smithi115.stdout:4/436: fsync d1/d30/d4c/d86/f8a 0 2024-03-22T10:42:43.025 INFO:tasks.workunit.client.0.smithi115.stdout:7/432: mknod d2/d7/d12/d10/d19/d42/d76/d74/c9a 0 2024-03-22T10:42:43.026 INFO:tasks.workunit.client.1.smithi150.stdout:8/64: unlink d0/c7 0 2024-03-22T10:42:43.027 INFO:tasks.workunit.client.0.smithi115.stdout:4/437: fdatasync d1/d16/f26 0 2024-03-22T10:42:43.028 INFO:tasks.workunit.client.0.smithi115.stdout:5/465: symlink d2/d15/d1b/d7c/d38/l9c 0 2024-03-22T10:42:43.029 INFO:tasks.workunit.client.1.smithi150.stdout:0/87: symlink d0/d1/l1b 0 2024-03-22T10:42:43.030 INFO:tasks.workunit.client.0.smithi115.stdout:4/438: chown d1/d16/d35/d40/d5c/d6b/l84 4073486 1 2024-03-22T10:42:43.031 INFO:tasks.workunit.client.0.smithi115.stdout:5/466: rename d2/df/d13/l66 to d2/d15/d1b/l9d 0 2024-03-22T10:42:43.033 INFO:tasks.workunit.client.1.smithi150.stdout:9/36: dread d1/d8/da/f9 [0,4194304] 0 2024-03-22T10:42:43.033 INFO:tasks.workunit.client.1.smithi150.stdout:2/75: creat d12/f13 x:0 0 0 2024-03-22T10:42:43.033 INFO:tasks.workunit.client.1.smithi150.stdout:4/68: unlink d0/de/cf 0 2024-03-22T10:42:43.035 INFO:tasks.workunit.client.1.smithi150.stdout:3/55: symlink d5/da/lc 0 2024-03-22T10:42:43.035 INFO:tasks.workunit.client.1.smithi150.stdout:2/76: dread - d12/f13 zero size 2024-03-22T10:42:43.036 INFO:tasks.workunit.client.1.smithi150.stdout:1/33: fsync f0 0 2024-03-22T10:42:43.039 INFO:tasks.workunit.client.1.smithi150.stdout:6/35: dread f5 [0,4194304] 0 2024-03-22T10:42:43.039 INFO:tasks.workunit.client.1.smithi150.stdout:4/69: dread - d0/f11 zero size 2024-03-22T10:42:43.039 INFO:tasks.workunit.client.1.smithi150.stdout:1/34: chown d1 6372 1 2024-03-22T10:42:43.041 INFO:tasks.workunit.client.1.smithi150.stdout:8/65: mkdir d0/d2/d19 0 2024-03-22T10:42:43.041 INFO:tasks.workunit.client.1.smithi150.stdout:5/11: unlink d0/l1 0 2024-03-22T10:42:43.042 INFO:tasks.workunit.client.0.smithi115.stdout:7/433: dwrite d2/d7/d12/d10/f36 [0,4194304] 0 2024-03-22T10:42:43.046 INFO:tasks.workunit.client.0.smithi115.stdout:7/434: fdatasync d2/d7/d12/d10/d19/d42/d46/f5e 0 2024-03-22T10:42:43.046 INFO:tasks.workunit.client.0.smithi115.stdout:5/467: dwrite d2/df/d3b/f7b [0,4194304] 0 2024-03-22T10:42:43.047 INFO:tasks.workunit.client.1.smithi150.stdout:2/77: fdatasync f11 0 2024-03-22T10:42:43.053 INFO:tasks.workunit.client.1.smithi150.stdout:1/35: creat d1/f9 x:0 0 0 2024-03-22T10:42:43.053 INFO:tasks.workunit.client.1.smithi150.stdout:6/36: mkdir db 0 2024-03-22T10:42:43.053 INFO:tasks.workunit.client.1.smithi150.stdout:1/36: readlink d1/l6 0 2024-03-22T10:42:43.053 INFO:tasks.workunit.client.1.smithi150.stdout:2/78: write d12/f13 [221546,3661] 0 2024-03-22T10:42:43.054 INFO:tasks.workunit.client.1.smithi150.stdout:0/88: symlink d0/l1c 0 2024-03-22T10:42:43.056 INFO:tasks.workunit.client.1.smithi150.stdout:1/37: dread - d1/f9 zero size 2024-03-22T10:42:43.059 INFO:tasks.workunit.client.1.smithi150.stdout:5/12: creat d0/f3 x:0 0 0 2024-03-22T10:42:43.061 INFO:tasks.workunit.client.0.smithi115.stdout:3/394: truncate d3/d19/d1b/f27 712700 0 2024-03-22T10:42:43.061 INFO:tasks.workunit.client.1.smithi150.stdout:9/37: creat d1/d8/de/df/f11 x:0 0 0 2024-03-22T10:42:43.064 INFO:tasks.workunit.client.1.smithi150.stdout:8/66: creat d0/dc/df/f1a x:0 0 0 2024-03-22T10:42:43.065 INFO:tasks.workunit.client.1.smithi150.stdout:7/27: dwrite d2/f6 [0,4194304] 0 2024-03-22T10:42:43.065 INFO:tasks.workunit.client.1.smithi150.stdout:2/79: write f10 [1355859,61821] 0 2024-03-22T10:42:43.065 INFO:tasks.workunit.client.1.smithi150.stdout:1/38: mkdir d1/da 0 2024-03-22T10:42:43.066 INFO:tasks.workunit.client.0.smithi115.stdout:7/435: rename d2/d7/d12/d10/d23/d33/f71 to d2/d7/d12/d10/d23/d33/f9b 0 2024-03-22T10:42:43.067 INFO:tasks.workunit.client.1.smithi150.stdout:5/13: mknod d0/c4 0 2024-03-22T10:42:43.068 INFO:tasks.workunit.client.1.smithi150.stdout:6/37: creat db/fc x:0 0 0 2024-03-22T10:42:43.069 INFO:tasks.workunit.client.0.smithi115.stdout:7/436: chown d2/f60 1593 1 2024-03-22T10:42:43.070 INFO:tasks.workunit.client.1.smithi150.stdout:8/67: readlink d0/l11 0 2024-03-22T10:42:43.071 INFO:tasks.workunit.client.1.smithi150.stdout:6/38: dread - db/fc zero size 2024-03-22T10:42:43.073 INFO:tasks.workunit.client.0.smithi115.stdout:3/395: rename d3/d16/d29/d35/f4c to d3/d36/f80 0 2024-03-22T10:42:43.074 INFO:tasks.workunit.client.0.smithi115.stdout:5/468: dwrite d2/df/f67 [0,4194304] 0 2024-03-22T10:42:43.074 INFO:tasks.workunit.client.0.smithi115.stdout:7/437: write d2/d7/d12/d10/d19/d42/d76/d74/f77 [188931,91987] 0 2024-03-22T10:42:43.075 INFO:tasks.workunit.client.0.smithi115.stdout:3/396: fdatasync d3/d16/f69 0 2024-03-22T10:42:43.077 INFO:tasks.workunit.client.0.smithi115.stdout:7/438: dread - d2/d7/d12/d10/d23/d3d/f5a zero size 2024-03-22T10:42:43.079 INFO:tasks.workunit.client.0.smithi115.stdout:3/397: read d3/d36/f44 [4082309,52912] 0 2024-03-22T10:42:43.080 INFO:tasks.workunit.client.1.smithi150.stdout:7/28: mkdir d2/d5/d7 0 2024-03-22T10:42:43.080 INFO:tasks.workunit.client.1.smithi150.stdout:6/39: mkdir db/dd 0 2024-03-22T10:42:43.083 INFO:tasks.workunit.client.0.smithi115.stdout:6/453: dread d4/d13/f25 [0,4194304] 0 2024-03-22T10:42:43.084 INFO:tasks.workunit.client.1.smithi150.stdout:5/14: link d0/f3 d0/f5 0 2024-03-22T10:42:43.087 INFO:tasks.workunit.client.1.smithi150.stdout:6/40: mknod db/ce 0 2024-03-22T10:42:43.091 INFO:tasks.workunit.client.0.smithi115.stdout:4/439: dread d1/d30/d42/d32/f60 [0,4194304] 0 2024-03-22T10:42:43.091 INFO:tasks.workunit.client.0.smithi115.stdout:4/440: chown d1 814865977 1 2024-03-22T10:42:43.092 INFO:tasks.workunit.client.0.smithi115.stdout:1/540: sync 2024-03-22T10:42:43.093 INFO:tasks.workunit.client.0.smithi115.stdout:4/441: read d1/f57 [2920280,112201] 0 2024-03-22T10:42:43.094 INFO:tasks.workunit.client.0.smithi115.stdout:3/398: link d3/d19/d1b/l26 d3/d19/d1b/l81 0 2024-03-22T10:42:43.100 INFO:tasks.workunit.client.0.smithi115.stdout:4/442: chown d1/d16/d35/d40/d54/d5e/f7e 24 1 2024-03-22T10:42:43.100 INFO:tasks.workunit.client.0.smithi115.stdout:7/439: rename d2/d7/d12/d10/d23/f2e to d2/d7/d12/d10/d19/d42/f9c 0 2024-03-22T10:42:43.102 INFO:tasks.workunit.client.0.smithi115.stdout:5/469: dwrite d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f69 [0,4194304] 0 2024-03-22T10:42:43.104 INFO:tasks.workunit.client.0.smithi115.stdout:7/440: dread - d2/d7/d12/f8d zero size 2024-03-22T10:42:43.106 INFO:tasks.workunit.client.1.smithi150.stdout:6/41: rename f5 to db/ff 0 2024-03-22T10:42:43.106 INFO:tasks.workunit.client.1.smithi150.stdout:6/42: readlink l7 0 2024-03-22T10:42:43.106 INFO:tasks.workunit.client.1.smithi150.stdout:5/15: write d0/f3 [26591,15063] 0 2024-03-22T10:42:43.107 INFO:tasks.workunit.client.0.smithi115.stdout:4/443: symlink d1/d16/d35/d40/d54/l92 0 2024-03-22T10:42:43.107 INFO:tasks.workunit.client.1.smithi150.stdout:5/16: readlink - no filename 2024-03-22T10:42:43.110 INFO:tasks.workunit.client.0.smithi115.stdout:4/444: readlink d1/d16/l7b 0 2024-03-22T10:42:43.111 INFO:tasks.workunit.client.1.smithi150.stdout:2/80: getdents d12 0 2024-03-22T10:42:43.111 INFO:tasks.workunit.client.0.smithi115.stdout:7/441: write d2/d7/d12/d10/d19/d42/d46/f68 [1140612,48255] 0 2024-03-22T10:42:43.111 INFO:tasks.workunit.client.0.smithi115.stdout:6/454: dwrite d4/d11/d12/f21 [0,4194304] 0 2024-03-22T10:42:43.113 INFO:tasks.workunit.client.0.smithi115.stdout:6/455: stat d4/le 0 2024-03-22T10:42:43.114 INFO:tasks.workunit.client.1.smithi150.stdout:4/70: rmdir d0/de 39 2024-03-22T10:42:43.115 INFO:tasks.workunit.client.0.smithi115.stdout:5/470: link d2/d15/d1b/c34 d2/d15/d1b/d7c/d84/d89/c9e 0 2024-03-22T10:42:43.116 INFO:tasks.workunit.client.0.smithi115.stdout:6/456: write d4/d11/d12/d1c/f7c [746341,13853] 0 2024-03-22T10:42:43.117 INFO:tasks.workunit.client.0.smithi115.stdout:7/442: getdents d2/d4c 0 2024-03-22T10:42:43.118 INFO:tasks.workunit.client.0.smithi115.stdout:5/471: dread - d2/df/d3b/f92 zero size 2024-03-22T10:42:43.119 INFO:tasks.workunit.client.0.smithi115.stdout:6/457: fsync d4/d11/d12/d1c/f6a 0 2024-03-22T10:42:43.119 INFO:tasks.workunit.client.1.smithi150.stdout:0/89: chown d0/d1/d7/d11 124094 1 2024-03-22T10:42:43.120 INFO:tasks.workunit.client.1.smithi150.stdout:1/39: dwrite d1/f3 [0,4194304] 0 2024-03-22T10:42:43.121 INFO:tasks.workunit.client.1.smithi150.stdout:9/38: dwrite d1/d8/da/f9 [0,4194304] 0 2024-03-22T10:42:43.125 INFO:tasks.workunit.client.0.smithi115.stdout:7/443: write d2/d4c/f69 [4988962,107639] 0 2024-03-22T10:42:43.126 INFO:tasks.workunit.client.1.smithi150.stdout:7/29: dwrite d2/f6 [0,4194304] 0 2024-03-22T10:42:43.126 INFO:tasks.workunit.client.1.smithi150.stdout:2/81: symlink d12/l14 0 2024-03-22T10:42:43.127 INFO:tasks.workunit.client.1.smithi150.stdout:6/43: mknod db/dd/c10 0 2024-03-22T10:42:43.128 INFO:tasks.workunit.client.1.smithi150.stdout:5/17: creat d0/f6 x:0 0 0 2024-03-22T10:42:43.128 INFO:tasks.workunit.client.1.smithi150.stdout:7/30: stat d2/d5 0 2024-03-22T10:42:43.133 INFO:tasks.workunit.client.1.smithi150.stdout:6/44: write db/fc [339163,92102] 0 2024-03-22T10:42:43.133 INFO:tasks.workunit.client.1.smithi150.stdout:0/90: mknod d0/d1/c1d 0 2024-03-22T10:42:43.136 INFO:tasks.workunit.client.1.smithi150.stdout:5/18: mknod d0/c7 0 2024-03-22T10:42:43.139 INFO:tasks.workunit.client.0.smithi115.stdout:3/399: rename d3/d19/d1b to d3/d82 0 2024-03-22T10:42:43.143 INFO:tasks.workunit.client.0.smithi115.stdout:5/472: dwrite d2/d15/d1b/d7c/d38/f61 [0,4194304] 0 2024-03-22T10:42:43.143 INFO:tasks.workunit.client.1.smithi150.stdout:1/40: dread d1/f3 [0,4194304] 0 2024-03-22T10:42:43.143 INFO:tasks.workunit.client.1.smithi150.stdout:4/71: creat d0/f1a x:0 0 0 2024-03-22T10:42:43.144 INFO:tasks.workunit.client.0.smithi115.stdout:4/445: rename d1/d16/d35/d40/d5c/d6b to d1/d30/d42/d4f/d93 0 2024-03-22T10:42:43.145 INFO:tasks.workunit.client.0.smithi115.stdout:4/446: stat d1/c15 0 2024-03-22T10:42:43.147 INFO:tasks.workunit.client.1.smithi150.stdout:2/82: creat d12/f15 x:0 0 0 2024-03-22T10:42:43.148 INFO:tasks.workunit.client.0.smithi115.stdout:5/473: write d2/d15/d1b/d7c/d38/d43/f82 [293894,101045] 0 2024-03-22T10:42:43.148 INFO:tasks.workunit.client.1.smithi150.stdout:7/31: mkdir d2/d5/d7/d8 0 2024-03-22T10:42:43.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:42 smithi150 ceph-mon[34518]: mgrmap e30: smithi115.iqzwsz(active, since 13s), standbys: smithi150.fmoumv 2024-03-22T10:42:43.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:42 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi150.fmoumv", "id": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:42:43.149 INFO:tasks.workunit.client.1.smithi150.stdout:2/83: fdatasync f10 0 2024-03-22T10:42:43.150 INFO:tasks.workunit.client.0.smithi115.stdout:5/474: write d2/df/f10 [990077,78713] 0 2024-03-22T10:42:43.151 INFO:tasks.workunit.client.1.smithi150.stdout:0/91: dread d0/d1/fa [0,4194304] 0 2024-03-22T10:42:43.154 INFO:tasks.workunit.client.1.smithi150.stdout:0/92: stat d0/d1/l1b 0 2024-03-22T10:42:43.154 INFO:tasks.workunit.client.0.smithi115.stdout:7/444: rename d2/d7/d12/d10/f36 to d2/d7/d12/d10/d19/f9d 0 2024-03-22T10:42:43.160 INFO:tasks.workunit.client.0.smithi115.stdout:3/400: dwrite d3/d16/f1c [4194304,4194304] 0 2024-03-22T10:42:43.161 INFO:tasks.workunit.client.0.smithi115.stdout:3/401: chown d3/d6d 218913455 1 2024-03-22T10:42:43.161 INFO:tasks.workunit.client.1.smithi150.stdout:4/72: mkdir d0/d1b 0 2024-03-22T10:42:43.162 INFO:tasks.workunit.client.1.smithi150.stdout:7/32: mkdir d2/d5/d7/d9 0 2024-03-22T10:42:43.168 INFO:tasks.workunit.client.0.smithi115.stdout:3/402: truncate d3/f30 343698 0 2024-03-22T10:42:43.172 INFO:tasks.workunit.client.0.smithi115.stdout:3/403: creat d3/d82/f83 x:0 0 0 2024-03-22T10:42:43.175 INFO:tasks.workunit.client.0.smithi115.stdout:3/404: creat d3/d16/f84 x:0 0 0 2024-03-22T10:42:43.177 INFO:tasks.workunit.client.0.smithi115.stdout:3/405: mkdir d3/d36/d41/d85 0 2024-03-22T10:42:43.182 INFO:tasks.workunit.client.1.smithi150.stdout:4/73: stat d0/c5 0 2024-03-22T10:42:43.183 INFO:tasks.workunit.client.1.smithi150.stdout:9/39: dwrite d1/d8/da/f9 [4194304,4194304] 0 2024-03-22T10:42:43.183 INFO:tasks.workunit.client.1.smithi150.stdout:7/33: write d2/f6 [1908286,78130] 0 2024-03-22T10:42:43.183 INFO:tasks.workunit.client.0.smithi115.stdout:3/406: rename d3/d19/d2b/d49/c7a to d3/d82/d57/c86 0 2024-03-22T10:42:43.183 INFO:tasks.workunit.client.0.smithi115.stdout:3/407: chown d3/d16/f69 2190 1 2024-03-22T10:42:43.186 INFO:tasks.workunit.client.1.smithi150.stdout:0/93: mkdir d0/d1/d7/db/d10/d1e 0 2024-03-22T10:42:43.187 INFO:tasks.workunit.client.1.smithi150.stdout:6/45: dwrite db/ff [0,4194304] 0 2024-03-22T10:42:43.191 INFO:tasks.workunit.client.1.smithi150.stdout:3/56: sync 2024-03-22T10:42:43.193 INFO:tasks.workunit.client.1.smithi150.stdout:1/41: dwrite d1/f9 [0,4194304] 0 2024-03-22T10:42:43.197 INFO:tasks.workunit.client.0.smithi115.stdout:3/408: dwrite d3/d36/f51 [0,4194304] 0 2024-03-22T10:42:43.197 INFO:tasks.workunit.client.1.smithi150.stdout:2/84: dwrite d12/f13 [0,4194304] 0 2024-03-22T10:42:43.215 INFO:tasks.workunit.client.1.smithi150.stdout:3/57: creat d5/fd x:0 0 0 2024-03-22T10:42:43.216 INFO:tasks.workunit.client.1.smithi150.stdout:0/94: unlink d0/d1/d7/d11/f15 0 2024-03-22T10:42:43.217 INFO:tasks.workunit.client.1.smithi150.stdout:2/85: chown c7 17430 1 2024-03-22T10:42:43.225 INFO:tasks.workunit.client.0.smithi115.stdout:4/447: truncate d1/d30/d42/f21 624870 0 2024-03-22T10:42:43.228 INFO:tasks.workunit.client.0.smithi115.stdout:4/448: symlink d1/d69/l94 0 2024-03-22T10:42:43.229 INFO:tasks.workunit.client.1.smithi150.stdout:5/19: fsync d0/f6 0 2024-03-22T10:42:43.230 INFO:tasks.workunit.client.0.smithi115.stdout:1/541: dread d0/d8/db/d25/f73 [0,4194304] 0 2024-03-22T10:42:43.233 INFO:tasks.workunit.client.0.smithi115.stdout:4/449: mkdir d1/d16/d35/d40/d95 0 2024-03-22T10:42:43.234 INFO:tasks.workunit.client.1.smithi150.stdout:9/40: dread d1/d8/da/f9 [4194304,4194304] 0 2024-03-22T10:42:43.235 INFO:tasks.workunit.client.0.smithi115.stdout:1/542: creat d0/d8/d4d/d56/db6/fb7 x:0 0 0 2024-03-22T10:42:43.236 INFO:tasks.workunit.client.0.smithi115.stdout:7/445: dwrite d2/d7/d12/d10/f7b [0,4194304] 0 2024-03-22T10:42:43.236 INFO:tasks.workunit.client.0.smithi115.stdout:4/450: dread - d1/d16/d35/d40/d5c/d14/f90 zero size 2024-03-22T10:42:43.238 INFO:tasks.workunit.client.1.smithi150.stdout:6/46: dread db/ff [0,4194304] 0 2024-03-22T10:42:43.239 INFO:tasks.workunit.client.1.smithi150.stdout:7/34: dwrite d2/f6 [0,4194304] 0 2024-03-22T10:42:43.241 INFO:tasks.workunit.client.1.smithi150.stdout:8/68: sync 2024-03-22T10:42:43.242 INFO:tasks.workunit.client.0.smithi115.stdout:4/451: creat d1/d16/d35/d40/d5c/d14/f96 x:0 0 0 2024-03-22T10:42:43.247 INFO:tasks.workunit.client.1.smithi150.stdout:1/42: dwrite d1/f3 [0,4194304] 0 2024-03-22T10:42:43.250 INFO:tasks.workunit.client.1.smithi150.stdout:8/69: write d0/f5 [3953429,92107] 0 2024-03-22T10:42:43.259 INFO:tasks.workunit.client.0.smithi115.stdout:1/543: dwrite d0/d68/fae [0,4194304] 0 2024-03-22T10:42:43.267 INFO:tasks.workunit.client.0.smithi115.stdout:7/446: write d2/d3a/d6e/f6a [358406,110615] 0 2024-03-22T10:42:43.270 INFO:tasks.workunit.client.0.smithi115.stdout:1/544: mknod d0/d68/cb8 0 2024-03-22T10:42:43.273 INFO:tasks.workunit.client.0.smithi115.stdout:1/545: write d0/d12/d17/f3e [3648760,87413] 0 2024-03-22T10:42:43.273 INFO:tasks.workunit.client.1.smithi150.stdout:4/74: dwrite d0/de/f13 [0,4194304] 0 2024-03-22T10:42:43.273 INFO:tasks.workunit.client.0.smithi115.stdout:7/447: creat d2/d7/d12/d10/d23/f9e x:0 0 0 2024-03-22T10:42:43.275 INFO:tasks.workunit.client.0.smithi115.stdout:7/448: dread - d2/d3a/d90/f91 zero size 2024-03-22T10:42:43.275 INFO:tasks.workunit.client.0.smithi115.stdout:1/546: fsync d0/d8/d4d/d56/f6d 0 2024-03-22T10:42:43.278 INFO:tasks.workunit.client.1.smithi150.stdout:4/75: write d0/de/f18 [2584892,65140] 0 2024-03-22T10:42:43.278 INFO:tasks.workunit.client.0.smithi115.stdout:7/449: fsync d2/d7/d12/d10/d23/f6f 0 2024-03-22T10:42:43.281 INFO:tasks.workunit.client.0.smithi115.stdout:7/450: readlink d2/l3c 0 2024-03-22T10:42:43.284 INFO:tasks.workunit.client.0.smithi115.stdout:7/451: mknod d2/d4c/c9f 0 2024-03-22T10:42:43.287 INFO:tasks.workunit.client.0.smithi115.stdout:3/409: dread d3/d36/f44 [0,4194304] 0 2024-03-22T10:42:43.287 INFO:tasks.workunit.client.0.smithi115.stdout:7/452: read d2/d3a/d6e/f6a [647030,7047] 0 2024-03-22T10:42:43.289 INFO:tasks.workunit.client.0.smithi115.stdout:7/453: stat d2/d3a/c87 0 2024-03-22T10:42:43.292 INFO:tasks.workunit.client.0.smithi115.stdout:1/547: write d0/d12/d2b/fad [2133997,17981] 0 2024-03-22T10:42:43.295 INFO:tasks.workunit.client.0.smithi115.stdout:1/548: mkdir d0/d12/d17/d37/d9e/da0/db9 0 2024-03-22T10:42:43.300 INFO:tasks.workunit.client.0.smithi115.stdout:4/452: dread d1/d30/d42/f21 [0,4194304] 0 2024-03-22T10:42:43.301 INFO:tasks.workunit.client.0.smithi115.stdout:4/453: fdatasync d1/d66/f6d 0 2024-03-22T10:42:43.305 INFO:tasks.workunit.client.0.smithi115.stdout:7/454: dwrite d2/d3a/f5f [0,4194304] 0 2024-03-22T10:42:43.305 INFO:tasks.workunit.client.0.smithi115.stdout:4/454: rmdir d1/d16/d35/d40/d54 39 2024-03-22T10:42:43.307 INFO:tasks.workunit.client.0.smithi115.stdout:7/455: write d2/d7/d12/d10/d23/d33/f70 [4095655,67413] 0 2024-03-22T10:42:43.308 INFO:tasks.workunit.client.0.smithi115.stdout:2/409: sync 2024-03-22T10:42:43.309 INFO:tasks.workunit.client.0.smithi115.stdout:9/479: sync 2024-03-22T10:42:43.309 INFO:tasks.workunit.client.0.smithi115.stdout:0/448: sync 2024-03-22T10:42:43.311 INFO:tasks.workunit.client.0.smithi115.stdout:4/455: creat d1/d16/d35/d40/d95/f97 x:0 0 0 2024-03-22T10:42:43.312 INFO:tasks.workunit.client.0.smithi115.stdout:7/456: creat d2/d62/fa0 x:0 0 0 2024-03-22T10:42:43.317 INFO:tasks.workunit.client.0.smithi115.stdout:2/410: mknod da/d1c/d67/c81 0 2024-03-22T10:42:43.321 INFO:tasks.workunit.client.1.smithi150.stdout:3/58: mknod d5/ce 0 2024-03-22T10:42:43.321 INFO:tasks.workunit.client.0.smithi115.stdout:4/456: mkdir d1/d16/d35/d40/d95/d98 0 2024-03-22T10:42:43.322 INFO:tasks.workunit.client.1.smithi150.stdout:2/86: mkdir d12/d16 0 2024-03-22T10:42:43.323 INFO:tasks.workunit.client.0.smithi115.stdout:4/457: readlink d1/d30/d42/d32/l3c 0 2024-03-22T10:42:43.323 INFO:tasks.workunit.client.1.smithi150.stdout:5/20: mkdir d0/d8 0 2024-03-22T10:42:43.324 INFO:tasks.workunit.client.0.smithi115.stdout:2/411: getdents da/d25/d30/d4f 0 2024-03-22T10:42:43.324 INFO:tasks.workunit.client.1.smithi150.stdout:9/41: creat d1/d8/de/df/f12 x:0 0 0 2024-03-22T10:42:43.325 INFO:tasks.workunit.client.1.smithi150.stdout:9/42: chown d1/d8 96 1 2024-03-22T10:42:43.326 INFO:tasks.workunit.client.0.smithi115.stdout:3/410: dread d3/d19/f4e [0,4194304] 0 2024-03-22T10:42:43.326 INFO:tasks.workunit.client.1.smithi150.stdout:6/47: rename l7 to db/dd/l11 0 2024-03-22T10:42:43.327 INFO:tasks.workunit.client.1.smithi150.stdout:9/43: write d1/d8/de/df/f11 [761815,105479] 0 2024-03-22T10:42:43.327 INFO:tasks.workunit.client.0.smithi115.stdout:2/412: write da/d25/d30/f59 [56568,64959] 0 2024-03-22T10:42:43.328 INFO:tasks.workunit.client.1.smithi150.stdout:8/70: mkdir d0/dc/df/d1b 0 2024-03-22T10:42:43.330 INFO:tasks.workunit.client.1.smithi150.stdout:6/48: read db/fc [7175,92171] 0 2024-03-22T10:42:43.330 INFO:tasks.workunit.client.0.smithi115.stdout:3/411: unlink d3/f7 0 2024-03-22T10:42:43.332 INFO:tasks.workunit.client.1.smithi150.stdout:0/95: mknod d0/c1f 0 2024-03-22T10:42:43.332 INFO:tasks.workunit.client.0.smithi115.stdout:2/413: stat da/d25/d3d/d57/d7d 0 2024-03-22T10:42:43.333 INFO:tasks.workunit.client.1.smithi150.stdout:3/59: creat d5/da/ff x:0 0 0 2024-03-22T10:42:43.334 INFO:tasks.workunit.client.0.smithi115.stdout:3/412: write d3/d82/d3a/f75 [1043944,3130] 0 2024-03-22T10:42:43.334 INFO:tasks.workunit.client.1.smithi150.stdout:5/21: symlink d0/l9 0 2024-03-22T10:42:43.335 INFO:tasks.workunit.client.1.smithi150.stdout:7/35: mkdir d2/d5/d7/d8/da 0 2024-03-22T10:42:43.337 INFO:tasks.workunit.client.1.smithi150.stdout:1/43: rename d1/f9 to d1/da/fb 0 2024-03-22T10:42:43.339 INFO:tasks.workunit.client.1.smithi150.stdout:9/44: rmdir d1/d8 39 2024-03-22T10:42:43.339 INFO:tasks.workunit.client.0.smithi115.stdout:0/449: dwrite d3/d7/d1c/f48 [0,4194304] 0 2024-03-22T10:42:43.339 INFO:tasks.workunit.client.0.smithi115.stdout:3/413: chown d3/d82/f64 14482 1 2024-03-22T10:42:43.343 INFO:tasks.workunit.client.0.smithi115.stdout:2/414: rename da/d25/d2a/f4b to da/f82 0 2024-03-22T10:42:43.344 INFO:tasks.workunit.client.0.smithi115.stdout:3/414: rmdir d3/d19/d2b/d49 39 2024-03-22T10:42:43.344 INFO:tasks.workunit.client.1.smithi150.stdout:2/87: symlink d12/d16/l17 0 2024-03-22T10:42:43.345 INFO:tasks.workunit.client.0.smithi115.stdout:7/457: dwrite d2/d7/d12/d10/f40 [0,4194304] 0 2024-03-22T10:42:43.346 INFO:tasks.workunit.client.0.smithi115.stdout:0/450: mknod d3/d7/d1b/d93/c97 0 2024-03-22T10:42:43.347 INFO:tasks.workunit.client.0.smithi115.stdout:3/415: mknod d3/d16/c87 0 2024-03-22T10:42:43.348 INFO:tasks.workunit.client.1.smithi150.stdout:7/36: dread d2/f6 [0,4194304] 0 2024-03-22T10:42:43.348 INFO:tasks.workunit.client.0.smithi115.stdout:2/415: mkdir da/d25/d83 0 2024-03-22T10:42:43.349 INFO:tasks.workunit.client.0.smithi115.stdout:3/416: chown d3/d19/d2b/l68 3037503 1 2024-03-22T10:42:43.349 INFO:tasks.workunit.client.0.smithi115.stdout:9/480: dread d6/d7/d19/f31 [0,4194304] 0 2024-03-22T10:42:43.350 INFO:tasks.workunit.client.0.smithi115.stdout:3/417: dread - d3/d82/f83 zero size 2024-03-22T10:42:43.350 INFO:tasks.workunit.client.1.smithi150.stdout:7/37: readlink d2/l4 0 2024-03-22T10:42:43.352 INFO:tasks.workunit.client.0.smithi115.stdout:9/481: mkdir d6/d8/d3c/d59/d68/da8 0 2024-03-22T10:42:43.354 INFO:tasks.workunit.client.0.smithi115.stdout:3/418: unlink d3/d19/f76 0 2024-03-22T10:42:43.355 INFO:tasks.workunit.client.1.smithi150.stdout:7/38: write d2/f6 [3880094,115785] 0 2024-03-22T10:42:43.356 INFO:tasks.workunit.client.0.smithi115.stdout:3/419: readlink d3/d16/d29/d35/l65 0 2024-03-22T10:42:43.359 INFO:tasks.workunit.client.1.smithi150.stdout:1/44: symlink d1/da/lc 0 2024-03-22T10:42:43.359 INFO:tasks.workunit.client.0.smithi115.stdout:3/420: creat d3/d19/d2b/d55/f88 x:0 0 0 2024-03-22T10:42:43.361 INFO:tasks.workunit.client.0.smithi115.stdout:3/421: fsync d3/f50 0 2024-03-22T10:42:43.363 INFO:tasks.workunit.client.0.smithi115.stdout:7/458: dwrite d2/d7/d12/d10/f16 [4194304,4194304] 0 2024-03-22T10:42:43.365 INFO:tasks.workunit.client.1.smithi150.stdout:6/49: dwrite db/ff [0,4194304] 0 2024-03-22T10:42:43.365 INFO:tasks.workunit.client.0.smithi115.stdout:3/422: chown d3/d82/d54/f59 11 1 2024-03-22T10:42:43.365 INFO:tasks.workunit.client.1.smithi150.stdout:0/96: link d0/d1/fa d0/d1/d7/f20 0 2024-03-22T10:42:43.367 INFO:tasks.workunit.client.1.smithi150.stdout:2/88: symlink d12/d16/l18 0 2024-03-22T10:42:43.368 INFO:tasks.workunit.client.0.smithi115.stdout:7/459: rmdir d2/d7/d12/d10/d23/d3d 39 2024-03-22T10:42:43.368 INFO:tasks.workunit.client.1.smithi150.stdout:2/89: fdatasync f10 0 2024-03-22T10:42:43.368 INFO:tasks.workunit.client.1.smithi150.stdout:0/97: fsync d0/d1/d7/ff 0 2024-03-22T10:42:43.369 INFO:tasks.workunit.client.0.smithi115.stdout:2/416: dwrite da/d25/d30/f31 [0,4194304] 0 2024-03-22T10:42:43.370 INFO:tasks.workunit.client.0.smithi115.stdout:9/482: dwrite d6/d7/d19/d4e/f6c [0,4194304] 0 2024-03-22T10:42:43.372 INFO:tasks.workunit.client.0.smithi115.stdout:9/483: chown d6/d8/l13 3662081 1 2024-03-22T10:42:43.373 INFO:tasks.workunit.client.1.smithi150.stdout:5/22: mknod d0/d8/ca 0 2024-03-22T10:42:43.375 INFO:tasks.workunit.client.1.smithi150.stdout:9/45: rename d1/d8/lc to d1/d8/da/l13 0 2024-03-22T10:42:43.375 INFO:tasks.workunit.client.0.smithi115.stdout:2/417: creat da/d25/f84 x:0 0 0 2024-03-22T10:42:43.375 INFO:tasks.workunit.client.0.smithi115.stdout:9/484: mkdir d6/d7/d48/d70/da9 0 2024-03-22T10:42:43.376 INFO:tasks.workunit.client.0.smithi115.stdout:2/418: chown da/d25/f34 26 1 2024-03-22T10:42:43.377 INFO:tasks.workunit.client.0.smithi115.stdout:9/485: fsync d6/d7/d48/d70/f73 0 2024-03-22T10:42:43.377 INFO:tasks.workunit.client.1.smithi150.stdout:9/46: stat d1/d8/cd 0 2024-03-22T10:42:43.378 INFO:tasks.workunit.client.0.smithi115.stdout:2/419: readlink da/d1c/l21 0 2024-03-22T10:42:43.381 INFO:tasks.workunit.client.0.smithi115.stdout:9/486: mkdir d6/d8/daa 0 2024-03-22T10:42:43.382 INFO:tasks.workunit.client.0.smithi115.stdout:2/420: creat da/d25/d3d/d57/f85 x:0 0 0 2024-03-22T10:42:43.384 INFO:tasks.workunit.client.0.smithi115.stdout:2/421: stat da/d25/d30/f6e 0 2024-03-22T10:42:43.385 INFO:tasks.workunit.client.1.smithi150.stdout:9/47: fsync d1/d8/de/df/f11 0 2024-03-22T10:42:43.385 INFO:tasks.workunit.client.0.smithi115.stdout:9/487: write d6/d7/d30/d3b/f46 [2516655,26126] 0 2024-03-22T10:42:43.386 INFO:tasks.workunit.client.1.smithi150.stdout:5/23: symlink d0/lb 0 2024-03-22T10:42:43.389 INFO:tasks.workunit.client.0.smithi115.stdout:9/488: chown d6/d8/d3c/d59/d68/f7f 763941627 1 2024-03-22T10:42:43.389 INFO:tasks.workunit.client.0.smithi115.stdout:7/460: dwrite d2/d7/d12/d10/f25 [12582912,4194304] 0 2024-03-22T10:42:43.390 INFO:tasks.workunit.client.1.smithi150.stdout:1/45: dread d1/da/fb [0,4194304] 0 2024-03-22T10:42:43.391 INFO:tasks.workunit.client.1.smithi150.stdout:7/39: rename d2/d5/d7/d8/da to d2/d5/d7/d9/db 0 2024-03-22T10:42:43.394 INFO:tasks.workunit.client.0.smithi115.stdout:9/489: write d6/d7/d30/d3b/f46 [3075403,53259] 0 2024-03-22T10:42:43.394 INFO:tasks.workunit.client.0.smithi115.stdout:7/461: write d2/d7/d12/d10/f96 [193755,46842] 0 2024-03-22T10:42:43.395 INFO:tasks.workunit.client.0.smithi115.stdout:2/422: mkdir da/d1c/d67/d78/d86 0 2024-03-22T10:42:43.396 INFO:tasks.workunit.client.1.smithi150.stdout:9/48: mknod d1/d8/da/c14 0 2024-03-22T10:42:43.397 INFO:tasks.workunit.client.0.smithi115.stdout:7/462: truncate d2/d3a/d90/f91 194345 0 2024-03-22T10:42:43.399 INFO:tasks.workunit.client.1.smithi150.stdout:5/24: mkdir d0/d8/dc 0 2024-03-22T10:42:43.401 INFO:tasks.workunit.client.1.smithi150.stdout:2/90: dwrite ff [0,4194304] 0 2024-03-22T10:42:43.401 INFO:tasks.workunit.client.0.smithi115.stdout:9/490: unlink d6/d7/d19/ca7 0 2024-03-22T10:42:43.403 INFO:tasks.workunit.client.1.smithi150.stdout:5/25: truncate d0/f6 338954 0 2024-03-22T10:42:43.404 INFO:tasks.workunit.client.0.smithi115.stdout:9/491: chown d6/d7/d6a/d6f/l89 38 1 2024-03-22T10:42:43.405 INFO:tasks.workunit.client.0.smithi115.stdout:7/463: mknod d2/d3a/d6e/ca1 0 2024-03-22T10:42:43.408 INFO:tasks.workunit.client.0.smithi115.stdout:9/492: write d6/d8/d3c/d59/d68/d8e/f9a [960166,93405] 0 2024-03-22T10:42:43.409 INFO:tasks.workunit.client.0.smithi115.stdout:7/464: creat d2/d3a/d6e/fa2 x:0 0 0 2024-03-22T10:42:43.411 INFO:tasks.workunit.client.1.smithi150.stdout:1/46: write d1/da/fb [5072972,7100] 0 2024-03-22T10:42:43.411 INFO:tasks.workunit.client.1.smithi150.stdout:9/49: rmdir d1/d8 39 2024-03-22T10:42:43.412 INFO:tasks.workunit.client.1.smithi150.stdout:2/91: unlink cb 0 2024-03-22T10:42:43.422 INFO:tasks.workunit.client.1.smithi150.stdout:1/47: creat d1/da/fd x:0 0 0 2024-03-22T10:42:43.422 INFO:tasks.workunit.client.1.smithi150.stdout:1/48: stat d1 0 2024-03-22T10:42:43.425 INFO:tasks.workunit.client.1.smithi150.stdout:5/26: dread d0/f3 [0,4194304] 0 2024-03-22T10:42:43.425 INFO:tasks.workunit.client.1.smithi150.stdout:7/40: dread d2/f6 [0,4194304] 0 2024-03-22T10:42:43.434 INFO:tasks.workunit.client.0.smithi115.stdout:2/423: write da/f15 [4635310,124740] 0 2024-03-22T10:42:43.435 INFO:tasks.workunit.client.1.smithi150.stdout:1/49: creat d1/da/fe x:0 0 0 2024-03-22T10:42:43.436 INFO:tasks.workunit.client.0.smithi115.stdout:2/424: fsync da/d25/d30/d4f/d54/f80 0 2024-03-22T10:42:43.437 INFO:tasks.workunit.client.0.smithi115.stdout:2/425: chown da/d25/d3d 122894 1 2024-03-22T10:42:43.438 INFO:tasks.workunit.client.1.smithi150.stdout:7/41: write d2/f6 [339422,71696] 0 2024-03-22T10:42:43.439 INFO:tasks.workunit.client.0.smithi115.stdout:2/426: fdatasync da/d1c/f1e 0 2024-03-22T10:42:43.440 INFO:tasks.workunit.client.0.smithi115.stdout:2/427: chown da/f1d 28569624 1 2024-03-22T10:42:43.442 INFO:tasks.workunit.client.0.smithi115.stdout:2/428: readlink da/d1c/d67/l7c 0 2024-03-22T10:42:43.442 INFO:tasks.workunit.client.0.smithi115.stdout:2/429: chown da 37 1 2024-03-22T10:42:43.443 INFO:tasks.workunit.client.1.smithi150.stdout:1/50: mknod d1/cf 0 2024-03-22T10:42:43.443 INFO:tasks.workunit.client.0.smithi115.stdout:2/430: read - da/d25/f84 zero size 2024-03-22T10:42:43.444 INFO:tasks.workunit.client.1.smithi150.stdout:2/92: dwrite f11 [4194304,4194304] 0 2024-03-22T10:42:43.445 INFO:tasks.workunit.client.1.smithi150.stdout:7/42: write d2/f6 [4477531,108155] 0 2024-03-22T10:42:43.447 INFO:tasks.workunit.client.0.smithi115.stdout:7/465: dread d2/d7/d12/d10/d19/d42/f9c [0,4194304] 0 2024-03-22T10:42:43.449 INFO:tasks.workunit.client.1.smithi150.stdout:2/93: dread - d12/f15 zero size 2024-03-22T10:42:43.452 INFO:tasks.workunit.client.0.smithi115.stdout:7/466: mkdir d2/d7/d12/d10/d19/d42/d46/d61/da3 0 2024-03-22T10:42:43.453 INFO:tasks.workunit.client.0.smithi115.stdout:2/431: dread da/d25/d51/f56 [0,4194304] 0 2024-03-22T10:42:43.453 INFO:tasks.workunit.client.1.smithi150.stdout:1/51: creat d1/f10 x:0 0 0 2024-03-22T10:42:43.455 INFO:tasks.workunit.client.0.smithi115.stdout:7/467: dread - d2/d7/d12/d10/d23/f84 zero size 2024-03-22T10:42:43.456 INFO:tasks.workunit.client.0.smithi115.stdout:2/432: write da/d25/d2a/f72 [2589387,15646] 0 2024-03-22T10:42:43.457 INFO:tasks.workunit.client.1.smithi150.stdout:2/94: mkdir d12/d19 0 2024-03-22T10:42:43.459 INFO:tasks.workunit.client.1.smithi150.stdout:9/50: dwrite d1/d8/da/f9 [4194304,4194304] 0 2024-03-22T10:42:43.459 INFO:tasks.workunit.client.0.smithi115.stdout:2/433: rename da/f1b to da/d7b/f87 0 2024-03-22T10:42:43.459 INFO:tasks.workunit.client.1.smithi150.stdout:1/52: chown d1/da 2011 1 2024-03-22T10:42:43.462 INFO:tasks.workunit.client.1.smithi150.stdout:1/53: rename d1 to d1/d11 22 2024-03-22T10:42:43.463 INFO:tasks.workunit.client.0.smithi115.stdout:7/468: getdents d2/d7/d12/d10/d23 0 2024-03-22T10:42:43.463 INFO:tasks.workunit.client.0.smithi115.stdout:2/434: rename da/d1c/f2b to da/d1c/f88 0 2024-03-22T10:42:43.463 INFO:tasks.workunit.client.0.smithi115.stdout:7/469: chown d2/d62/l8f 1762854 1 2024-03-22T10:42:43.465 INFO:tasks.workunit.client.1.smithi150.stdout:1/54: readlink d1/da/lc 0 2024-03-22T10:42:43.465 INFO:tasks.workunit.client.0.smithi115.stdout:2/435: write da/d25/d3d/d57/f65 [416811,100336] 0 2024-03-22T10:42:43.468 INFO:tasks.workunit.client.1.smithi150.stdout:9/51: symlink d1/l15 0 2024-03-22T10:42:43.469 INFO:tasks.workunit.client.1.smithi150.stdout:5/27: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:42:43.470 INFO:tasks.workunit.client.1.smithi150.stdout:1/55: dread - d1/da/fd zero size 2024-03-22T10:42:43.471 INFO:tasks.workunit.client.1.smithi150.stdout:5/28: chown d0/l9 25 1 2024-03-22T10:42:43.476 INFO:tasks.workunit.client.1.smithi150.stdout:1/56: truncate d1/f7 569182 0 2024-03-22T10:42:43.476 INFO:tasks.workunit.client.1.smithi150.stdout:7/43: dread d2/f6 [0,4194304] 0 2024-03-22T10:42:43.476 INFO:tasks.workunit.client.1.smithi150.stdout:9/52: creat d1/d8/de/f16 x:0 0 0 2024-03-22T10:42:43.477 INFO:tasks.workunit.client.1.smithi150.stdout:1/57: fsync d1/f7 0 2024-03-22T10:42:43.481 INFO:tasks.workunit.client.0.smithi115.stdout:7/470: dwrite d2/d7/d12/d10/d23/f84 [0,4194304] 0 2024-03-22T10:42:43.481 INFO:tasks.workunit.client.1.smithi150.stdout:2/95: truncate f10 1080324 0 2024-03-22T10:42:43.481 INFO:tasks.workunit.client.1.smithi150.stdout:1/58: write d1/f7 [1310225,71056] 0 2024-03-22T10:42:43.481 INFO:tasks.workunit.client.1.smithi150.stdout:3/60: getdents d5 0 2024-03-22T10:42:43.482 INFO:tasks.workunit.client.0.smithi115.stdout:2/436: dwrite da/d25/d3d/d57/f85 [0,4194304] 0 2024-03-22T10:42:43.484 INFO:tasks.workunit.client.0.smithi115.stdout:7/471: write d2/d7/d12/d10/d23/f6f [95806,6642] 0 2024-03-22T10:42:43.487 INFO:tasks.workunit.client.1.smithi150.stdout:7/44: write d2/f6 [1495590,75858] 0 2024-03-22T10:42:43.488 INFO:tasks.workunit.client.1.smithi150.stdout:3/61: fdatasync d5/fd 0 2024-03-22T10:42:43.489 INFO:tasks.workunit.client.1.smithi150.stdout:1/59: fdatasync d1/f10 0 2024-03-22T10:42:43.489 INFO:tasks.workunit.client.0.smithi115.stdout:7/472: mkdir d2/d7/d12/d10/d23/d3d/da4 0 2024-03-22T10:42:43.491 INFO:tasks.workunit.client.0.smithi115.stdout:7/473: creat d2/fa5 x:0 0 0 2024-03-22T10:42:43.492 INFO:tasks.workunit.client.1.smithi150.stdout:2/96: mkdir d12/d19/d1a 0 2024-03-22T10:42:43.498 INFO:tasks.workunit.client.0.smithi115.stdout:2/437: dwrite da/d1c/f1e [0,4194304] 0 2024-03-22T10:42:43.504 INFO:tasks.workunit.client.1.smithi150.stdout:5/29: dwrite d0/f5 [0,4194304] 0 2024-03-22T10:42:43.505 INFO:tasks.workunit.client.1.smithi150.stdout:3/62: creat d5/f10 x:0 0 0 2024-03-22T10:42:43.506 INFO:tasks.workunit.client.1.smithi150.stdout:1/60: mknod d1/da/c12 0 2024-03-22T10:42:43.506 INFO:tasks.workunit.client.1.smithi150.stdout:3/63: readlink d5/l9 0 2024-03-22T10:42:43.508 INFO:tasks.workunit.client.1.smithi150.stdout:4/76: truncate d0/de/f13 89096 0 2024-03-22T10:42:43.508 INFO:tasks.workunit.client.1.smithi150.stdout:9/53: dread d1/d8/da/f9 [0,4194304] 0 2024-03-22T10:42:43.509 INFO:tasks.workunit.client.0.smithi115.stdout:8/661: sync 2024-03-22T10:42:43.509 INFO:tasks.workunit.client.1.smithi150.stdout:5/30: creat d0/d8/fd x:0 0 0 2024-03-22T10:42:43.511 INFO:tasks.workunit.client.1.smithi150.stdout:5/31: readlink d0/l9 0 2024-03-22T10:42:43.512 INFO:tasks.workunit.client.1.smithi150.stdout:6/50: truncate db/fc 70332 0 2024-03-22T10:42:43.512 INFO:tasks.workunit.client.1.smithi150.stdout:3/64: truncate d5/da/ff 540338 0 2024-03-22T10:42:43.514 INFO:tasks.workunit.client.1.smithi150.stdout:4/77: mknod d0/de/c1c 0 2024-03-22T10:42:43.516 INFO:tasks.workunit.client.1.smithi150.stdout:3/65: write d5/da/ff [1107001,23464] 0 2024-03-22T10:42:43.516 INFO:tasks.workunit.client.1.smithi150.stdout:7/45: dread d2/f6 [0,4194304] 0 2024-03-22T10:42:43.518 INFO:tasks.workunit.client.1.smithi150.stdout:8/71: truncate d0/fb 3187726 0 2024-03-22T10:42:43.520 INFO:tasks.workunit.client.0.smithi115.stdout:2/438: dwrite da/d25/d3d/d57/f85 [0,4194304] 0 2024-03-22T10:42:43.520 INFO:tasks.workunit.client.0.smithi115.stdout:8/662: mkdir d6/dd/d24/d4a/d7b/d8f/dd6 0 2024-03-22T10:42:43.521 INFO:tasks.workunit.client.1.smithi150.stdout:9/54: creat d1/d8/de/f17 x:0 0 0 2024-03-22T10:42:43.524 INFO:tasks.workunit.client.0.smithi115.stdout:8/663: chown d6/dd/d24/d3a/c93 290510 1 2024-03-22T10:42:43.524 INFO:tasks.workunit.client.1.smithi150.stdout:4/78: read d0/de/f12 [27900,113812] 0 2024-03-22T10:42:43.526 INFO:tasks.workunit.client.0.smithi115.stdout:8/664: write d6/fd2 [730301,97416] 0 2024-03-22T10:42:43.528 INFO:tasks.workunit.client.0.smithi115.stdout:2/439: mknod da/d25/d3d/d57/d7d/c89 0 2024-03-22T10:42:43.529 INFO:tasks.workunit.client.1.smithi150.stdout:3/66: mknod d5/da/c11 0 2024-03-22T10:42:43.531 INFO:tasks.workunit.client.1.smithi150.stdout:9/55: truncate d1/d8/de/f17 31472 0 2024-03-22T10:42:43.532 INFO:tasks.workunit.client.1.smithi150.stdout:5/32: mkdir d0/d8/dc/de 0 2024-03-22T10:42:43.533 INFO:tasks.workunit.client.0.smithi115.stdout:2/440: write da/d1c/f4c [280977,55423] 0 2024-03-22T10:42:43.534 INFO:tasks.workunit.client.0.smithi115.stdout:8/665: fsync d6/f91 0 2024-03-22T10:42:43.535 INFO:tasks.workunit.client.1.smithi150.stdout:0/98: truncate d0/d1/d7/db/d10/f13 3650021 0 2024-03-22T10:42:43.537 INFO:tasks.workunit.client.0.smithi115.stdout:8/666: chown d6/dd/d24/d4a/d41/l6d 70484 1 2024-03-22T10:42:43.537 INFO:tasks.workunit.client.0.smithi115.stdout:2/441: creat da/d25/d83/f8a x:0 0 0 2024-03-22T10:42:43.538 INFO:tasks.workunit.client.0.smithi115.stdout:2/442: chown da/d1c/c79 112107 1 2024-03-22T10:42:43.540 INFO:tasks.workunit.client.1.smithi150.stdout:6/51: dread db/ff [0,4194304] 0 2024-03-22T10:42:43.541 INFO:tasks.workunit.client.0.smithi115.stdout:2/443: write da/d25/d3d/f28 [1905732,109916] 0 2024-03-22T10:42:43.543 INFO:tasks.workunit.client.1.smithi150.stdout:5/33: symlink d0/d8/dc/lf 0 2024-03-22T10:42:43.543 INFO:tasks.workunit.client.0.smithi115.stdout:2/444: chown da/d25/f49 1 1 2024-03-22T10:42:43.544 INFO:tasks.workunit.client.1.smithi150.stdout:0/99: rmdir d0/d1 39 2024-03-22T10:42:43.545 INFO:tasks.workunit.client.0.smithi115.stdout:8/667: rename d6/dd/d12/l75 to d6/dd/d12/ld7 0 2024-03-22T10:42:43.548 INFO:tasks.workunit.client.1.smithi150.stdout:1/61: dwrite d1/f7 [0,4194304] 0 2024-03-22T10:42:43.549 INFO:tasks.workunit.client.0.smithi115.stdout:2/445: creat da/d25/d3d/d57/d7d/f8b x:0 0 0 2024-03-22T10:42:43.550 INFO:tasks.workunit.client.1.smithi150.stdout:5/34: read d0/f3 [1776427,121661] 0 2024-03-22T10:42:43.553 INFO:tasks.workunit.client.0.smithi115.stdout:2/446: write da/d25/d51/f63 [725775,57072] 0 2024-03-22T10:42:43.554 INFO:tasks.workunit.client.0.smithi115.stdout:8/668: link d6/fd2 d6/dd/d24/d4a/d7b/d8f/fd8 0 2024-03-22T10:42:43.560 INFO:tasks.workunit.client.0.smithi115.stdout:2/447: chown da/d25/d2a/l4d 27486386 1 2024-03-22T10:42:43.560 INFO:tasks.workunit.client.0.smithi115.stdout:8/669: dread - d6/dbc/fca zero size 2024-03-22T10:42:43.561 INFO:tasks.workunit.client.0.smithi115.stdout:7/474: write d2/d7/d12/d10/d23/f38 [1430246,53136] 0 2024-03-22T10:42:43.565 INFO:tasks.workunit.client.1.smithi150.stdout:4/79: getdents d0/de 0 2024-03-22T10:42:43.568 INFO:tasks.workunit.client.1.smithi150.stdout:7/46: truncate d2/f6 2273037 0 2024-03-22T10:42:43.570 INFO:tasks.workunit.client.0.smithi115.stdout:2/448: truncate da/d1c/f88 60568 0 2024-03-22T10:42:43.570 INFO:tasks.workunit.client.0.smithi115.stdout:7/475: rename d2/fa5 to d2/d7/d12/d10/d19/d42/d46/d94/fa6 0 2024-03-22T10:42:43.570 INFO:tasks.workunit.client.1.smithi150.stdout:0/100: dread d0/d1/fa [0,4194304] 0 2024-03-22T10:42:43.571 INFO:tasks.workunit.client.1.smithi150.stdout:5/35: rmdir d0/d8/dc 39 2024-03-22T10:42:43.571 INFO:tasks.workunit.client.0.smithi115.stdout:2/449: chown da/d25/d2a 10011821 1 2024-03-22T10:42:43.572 INFO:tasks.workunit.client.1.smithi150.stdout:8/72: dwrite d0/d2/fa [0,4194304] 0 2024-03-22T10:42:43.574 INFO:tasks.workunit.client.0.smithi115.stdout:7/476: dread - d2/d7/d12/d10/d19/d42/d46/d61/f92 zero size 2024-03-22T10:42:43.574 INFO:tasks.workunit.client.0.smithi115.stdout:2/450: truncate da/d1c/f52 1381801 0 2024-03-22T10:42:43.578 INFO:tasks.workunit.client.0.smithi115.stdout:7/477: creat d2/d7/d12/fa7 x:0 0 0 2024-03-22T10:42:43.580 INFO:tasks.workunit.client.0.smithi115.stdout:7/478: fdatasync d2/d7/d12/d10/d23/d3d/f5a 0 2024-03-22T10:42:43.583 INFO:tasks.workunit.client.1.smithi150.stdout:5/36: creat d0/f10 x:0 0 0 2024-03-22T10:42:43.584 INFO:tasks.workunit.client.1.smithi150.stdout:4/80: fdatasync d0/f8 0 2024-03-22T10:42:43.585 INFO:tasks.workunit.client.1.smithi150.stdout:9/56: dwrite d1/d8/de/df/f11 [0,4194304] 0 2024-03-22T10:42:43.586 INFO:tasks.workunit.client.0.smithi115.stdout:2/451: unlink da/d25/d30/d4f/d54/f80 0 2024-03-22T10:42:43.589 INFO:tasks.workunit.client.0.smithi115.stdout:7/479: mkdir d2/d7/d12/d10/d23/d3d/da8 0 2024-03-22T10:42:43.596 INFO:tasks.workunit.client.0.smithi115.stdout:8/670: dwrite d6/f91 [0,4194304] 0 2024-03-22T10:42:43.596 INFO:tasks.workunit.client.0.smithi115.stdout:7/480: fsync d2/d3a/f5f 0 2024-03-22T10:42:43.596 INFO:tasks.workunit.client.0.smithi115.stdout:2/452: symlink da/d1c/l8c 0 2024-03-22T10:42:43.597 INFO:tasks.workunit.client.1.smithi150.stdout:0/101: write d0/d1/d7/f20 [518650,42149] 0 2024-03-22T10:42:43.599 INFO:tasks.workunit.client.0.smithi115.stdout:7/481: chown d2/d7/d12/d10/d19/l6c 5 1 2024-03-22T10:42:43.599 INFO:tasks.workunit.client.0.smithi115.stdout:2/453: fsync da/f16 0 2024-03-22T10:42:43.600 INFO:tasks.workunit.client.0.smithi115.stdout:8/671: symlink d6/dd/d24/d4a/d7b/d8f/ld9 0 2024-03-22T10:42:43.601 INFO:tasks.workunit.client.1.smithi150.stdout:4/81: mkdir d0/d1b/d1d 0 2024-03-22T10:42:43.604 INFO:tasks.workunit.client.0.smithi115.stdout:8/672: truncate d6/f23 2532674 0 2024-03-22T10:42:43.605 INFO:tasks.workunit.client.1.smithi150.stdout:4/82: readlink d0/l15 0 2024-03-22T10:42:43.606 INFO:tasks.workunit.client.1.smithi150.stdout:1/62: dwrite d1/da/fd [0,4194304] 0 2024-03-22T10:42:43.606 INFO:tasks.workunit.client.1.smithi150.stdout:0/102: mkdir d0/d1/d7/d21 0 2024-03-22T10:42:43.612 INFO:tasks.workunit.client.1.smithi150.stdout:4/83: dread - d0/f11 zero size 2024-03-22T10:42:43.615 INFO:tasks.workunit.client.0.smithi115.stdout:2/454: link da/d1c/f4c da/d25/d30/d4f/f8d 0 2024-03-22T10:42:43.617 INFO:tasks.workunit.client.1.smithi150.stdout:1/63: write f0 [838729,60579] 0 2024-03-22T10:42:43.617 INFO:tasks.workunit.client.0.smithi115.stdout:2/455: fsync da/d25/d3d/d57/f65 0 2024-03-22T10:42:43.618 INFO:tasks.workunit.client.1.smithi150.stdout:4/84: chown d0/de/l19 3879 1 2024-03-22T10:42:43.620 INFO:tasks.workunit.client.0.smithi115.stdout:2/456: stat da/d25/d30/d4f 0 2024-03-22T10:42:43.623 INFO:tasks.workunit.client.1.smithi150.stdout:1/64: creat d1/da/f13 x:0 0 0 2024-03-22T10:42:43.628 INFO:tasks.workunit.client.1.smithi150.stdout:5/37: dread d0/f5 [0,4194304] 0 2024-03-22T10:42:43.628 INFO:tasks.workunit.client.1.smithi150.stdout:6/52: dwrite db/fc [0,4194304] 0 2024-03-22T10:42:43.631 INFO:tasks.workunit.client.1.smithi150.stdout:1/65: truncate d1/da/f13 880104 0 2024-03-22T10:42:43.632 INFO:tasks.workunit.client.1.smithi150.stdout:9/57: dread d1/d8/de/df/f11 [0,4194304] 0 2024-03-22T10:42:43.634 INFO:tasks.workunit.client.1.smithi150.stdout:7/47: dread d2/f6 [0,4194304] 0 2024-03-22T10:42:43.636 INFO:tasks.workunit.client.0.smithi115.stdout:5/475: sync 2024-03-22T10:42:43.636 INFO:tasks.workunit.client.0.smithi115.stdout:6/458: sync 2024-03-22T10:42:43.636 INFO:tasks.workunit.client.1.smithi150.stdout:6/53: write db/ff [2430705,45056] 0 2024-03-22T10:42:43.637 INFO:tasks.workunit.client.1.smithi150.stdout:9/58: read - d1/d8/de/df/f12 zero size 2024-03-22T10:42:43.638 INFO:tasks.workunit.client.1.smithi150.stdout:1/66: unlink d1/c5 0 2024-03-22T10:42:43.639 INFO:tasks.workunit.client.0.smithi115.stdout:5/476: dread - d2/df/d13/f4c zero size 2024-03-22T10:42:43.640 INFO:tasks.workunit.client.1.smithi150.stdout:1/67: chown d1 54641 1 2024-03-22T10:42:43.643 INFO:tasks.workunit.client.1.smithi150.stdout:7/48: fsync d2/f6 0 2024-03-22T10:42:43.647 INFO:tasks.workunit.client.0.smithi115.stdout:6/459: dread d4/d11/d12/f80 [0,4194304] 0 2024-03-22T10:42:43.647 INFO:tasks.workunit.client.0.smithi115.stdout:2/457: write da/d1c/f88 [159311,26897] 0 2024-03-22T10:42:43.648 INFO:tasks.workunit.client.0.smithi115.stdout:6/460: fsync d4/d11/d12/f31 0 2024-03-22T10:42:43.649 INFO:tasks.workunit.client.1.smithi150.stdout:5/38: truncate d0/f3 3895945 0 2024-03-22T10:42:43.652 INFO:tasks.workunit.client.0.smithi115.stdout:2/458: mknod da/d25/d83/c8e 0 2024-03-22T10:42:43.652 INFO:tasks.workunit.client.0.smithi115.stdout:6/461: rmdir d4/d13/d33/d58 39 2024-03-22T10:42:43.655 INFO:tasks.workunit.client.1.smithi150.stdout:7/49: mknod d2/cc 0 2024-03-22T10:42:43.660 INFO:tasks.workunit.client.0.smithi115.stdout:6/462: write d4/d11/f65 [524773,104720] 0 2024-03-22T10:42:43.662 INFO:tasks.workunit.client.0.smithi115.stdout:2/459: write da/f16 [308657,96285] 0 2024-03-22T10:42:43.663 INFO:tasks.workunit.client.0.smithi115.stdout:8/673: dread d6/dd/f76 [0,4194304] 0 2024-03-22T10:42:43.664 INFO:tasks.workunit.client.1.smithi150.stdout:6/54: dread db/ff [0,4194304] 0 2024-03-22T10:42:43.668 INFO:tasks.workunit.client.1.smithi150.stdout:7/50: chown d2/f6 0 1 2024-03-22T10:42:43.668 INFO:tasks.workunit.client.1.smithi150.stdout:8/73: dwrite d0/fd [0,4194304] 0 2024-03-22T10:42:43.675 INFO:tasks.workunit.client.0.smithi115.stdout:5/477: dread d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:43.676 INFO:tasks.workunit.client.1.smithi150.stdout:6/55: mknod db/dd/c12 0 2024-03-22T10:42:43.676 INFO:tasks.workunit.client.0.smithi115.stdout:8/674: dwrite d6/dd/d12/f1d [0,4194304] 0 2024-03-22T10:42:43.678 INFO:tasks.workunit.client.0.smithi115.stdout:5/478: symlink d2/d15/d32/l9f 0 2024-03-22T10:42:43.681 INFO:tasks.workunit.client.1.smithi150.stdout:9/59: dwrite d1/d8/de/f17 [0,4194304] 0 2024-03-22T10:42:43.685 INFO:tasks.workunit.client.1.smithi150.stdout:1/68: dwrite d1/f3 [0,4194304] 0 2024-03-22T10:42:43.685 INFO:tasks.workunit.client.1.smithi150.stdout:7/51: rename c1 to d2/d5/d7/d8/cd 0 2024-03-22T10:42:43.689 INFO:tasks.workunit.client.1.smithi150.stdout:5/39: dread d0/f5 [0,4194304] 0 2024-03-22T10:42:43.693 INFO:tasks.workunit.client.1.smithi150.stdout:6/56: symlink db/dd/l13 0 2024-03-22T10:42:43.702 INFO:tasks.workunit.client.1.smithi150.stdout:8/74: dread d0/d2/f18 [0,4194304] 0 2024-03-22T10:42:43.706 INFO:tasks.workunit.client.1.smithi150.stdout:6/57: creat db/dd/f14 x:0 0 0 2024-03-22T10:42:43.714 INFO:tasks.workunit.client.1.smithi150.stdout:6/58: chown db/ce 0 1 2024-03-22T10:42:43.717 INFO:tasks.workunit.client.1.smithi150.stdout:8/75: rename d0/d2/f15 to d0/dc/df/d1b/f1c 0 2024-03-22T10:42:43.725 INFO:tasks.workunit.client.1.smithi150.stdout:6/59: unlink l3 0 2024-03-22T10:42:43.738 INFO:tasks.workunit.client.1.smithi150.stdout:5/40: dwrite d0/f6 [0,4194304] 0 2024-03-22T10:42:43.739 INFO:tasks.workunit.client.0.smithi115.stdout:5/479: rmdir d2/d15/d1b/d7c/d38/d43/d49/d81 39 2024-03-22T10:42:43.739 INFO:tasks.workunit.client.0.smithi115.stdout:8/675: creat d6/dd/d24/d4a/d7b/d8f/dd6/fda x:0 0 0 2024-03-22T10:42:43.740 INFO:tasks.workunit.client.0.smithi115.stdout:5/480: stat d2/d15/d1b/d7c/d38/d4a/d74 0 2024-03-22T10:42:43.741 INFO:tasks.workunit.client.1.smithi150.stdout:9/60: dwrite d1/d8/de/f17 [4194304,4194304] 0 2024-03-22T10:42:43.742 INFO:tasks.workunit.client.1.smithi150.stdout:6/60: read db/ff [3691682,128326] 0 2024-03-22T10:42:43.743 INFO:tasks.workunit.client.1.smithi150.stdout:7/52: dwrite d2/f6 [0,4194304] 0 2024-03-22T10:42:43.749 INFO:tasks.workunit.client.1.smithi150.stdout:1/69: dwrite d1/da/fe [0,4194304] 0 2024-03-22T10:42:43.750 INFO:tasks.workunit.client.1.smithi150.stdout:5/41: chown d0/f10 475 1 2024-03-22T10:42:43.751 INFO:tasks.workunit.client.1.smithi150.stdout:9/61: stat d1/d8/de/df 0 2024-03-22T10:42:43.753 INFO:tasks.workunit.client.1.smithi150.stdout:7/53: mkdir d2/d5/d7/de 0 2024-03-22T10:42:43.754 INFO:tasks.workunit.client.0.smithi115.stdout:8/676: dread d6/dd/d12/f27 [0,4194304] 0 2024-03-22T10:42:43.756 INFO:tasks.workunit.client.0.smithi115.stdout:8/677: chown d6/f91 638629 1 2024-03-22T10:42:43.760 INFO:tasks.workunit.client.1.smithi150.stdout:5/42: creat d0/f11 x:0 0 0 2024-03-22T10:42:43.761 INFO:tasks.workunit.client.0.smithi115.stdout:5/481: dwrite d2/df/d13/f18 [8388608,4194304] 0 2024-03-22T10:42:43.763 INFO:tasks.workunit.client.0.smithi115.stdout:8/678: getdents d6/dd/d12/d31 0 2024-03-22T10:42:43.764 INFO:tasks.workunit.client.0.smithi115.stdout:5/482: write d2/df/f67 [1647201,37959] 0 2024-03-22T10:42:43.765 INFO:tasks.workunit.client.1.smithi150.stdout:5/43: read d0/f6 [400473,40903] 0 2024-03-22T10:42:43.765 INFO:tasks.workunit.client.1.smithi150.stdout:8/76: dread d0/fd [0,4194304] 0 2024-03-22T10:42:43.767 INFO:tasks.workunit.client.1.smithi150.stdout:9/62: rename d1/d8/da/f9 to d1/d8/de/df/f18 0 2024-03-22T10:42:43.769 INFO:tasks.workunit.client.0.smithi115.stdout:5/483: truncate d2/d15/d1b/d7c/d38/f77 1131145 0 2024-03-22T10:42:43.770 INFO:tasks.workunit.client.0.smithi115.stdout:8/679: mkdir d6/dd/d24/d4a/d7b/d8f/dd6/ddb 0 2024-03-22T10:42:43.773 INFO:tasks.workunit.client.0.smithi115.stdout:8/680: mknod d6/dd/d24/d4a/d7b/d8f/dd6/ddb/cdc 0 2024-03-22T10:42:43.773 INFO:tasks.workunit.client.1.smithi150.stdout:8/77: write d0/d2/fa [3302935,111638] 0 2024-03-22T10:42:43.775 INFO:tasks.workunit.client.0.smithi115.stdout:8/681: symlink d6/dd/d24/dc1/ldd 0 2024-03-22T10:42:43.777 INFO:tasks.workunit.client.0.smithi115.stdout:8/682: read d6/dd/d12/f94 [158738,33321] 0 2024-03-22T10:42:43.780 INFO:tasks.workunit.client.1.smithi150.stdout:7/54: rename d2/l4 to d2/d5/d7/d9/lf 0 2024-03-22T10:42:43.782 INFO:tasks.workunit.client.0.smithi115.stdout:8/683: link d6/dd/d12/d31/f5b d6/dd/d12/dc6/fde 0 2024-03-22T10:42:43.784 INFO:tasks.workunit.client.0.smithi115.stdout:8/684: creat d6/dd/d12/d31/fdf x:0 0 0 2024-03-22T10:42:43.788 INFO:tasks.workunit.client.0.smithi115.stdout:8/685: creat d6/dd/d24/d4a/d7b/fe0 x:0 0 0 2024-03-22T10:42:43.788 INFO:tasks.workunit.client.1.smithi150.stdout:1/70: dread d1/da/fe [0,4194304] 0 2024-03-22T10:42:43.792 INFO:tasks.workunit.client.1.smithi150.stdout:6/61: dwrite db/dd/f14 [0,4194304] 0 2024-03-22T10:42:43.792 INFO:tasks.workunit.client.1.smithi150.stdout:7/55: mknod d2/c10 0 2024-03-22T10:42:43.792 INFO:tasks.workunit.client.1.smithi150.stdout:8/78: creat d0/dc/df/d1b/f1d x:0 0 0 2024-03-22T10:42:43.794 INFO:tasks.workunit.client.1.smithi150.stdout:6/62: readlink db/dd/l13 0 2024-03-22T10:42:43.798 INFO:tasks.workunit.client.1.smithi150.stdout:1/71: write f0 [1403850,91307] 0 2024-03-22T10:42:43.800 INFO:tasks.workunit.client.1.smithi150.stdout:6/63: write db/fc [2364919,50481] 0 2024-03-22T10:42:43.800 INFO:tasks.workunit.client.1.smithi150.stdout:8/79: chown d0/dc/df/d1b/f1d 1 1 2024-03-22T10:42:43.801 INFO:tasks.workunit.client.1.smithi150.stdout:7/56: symlink d2/d5/d7/d9/db/l11 0 2024-03-22T10:42:43.804 INFO:tasks.workunit.client.1.smithi150.stdout:6/64: read db/ff [3024501,17603] 0 2024-03-22T10:42:43.806 INFO:tasks.workunit.client.1.smithi150.stdout:9/63: dread d1/d8/de/df/f18 [4194304,4194304] 0 2024-03-22T10:42:43.807 INFO:tasks.workunit.client.1.smithi150.stdout:1/72: unlink d1/f10 0 2024-03-22T10:42:43.810 INFO:tasks.workunit.client.1.smithi150.stdout:5/44: dwrite d0/f11 [0,4194304] 0 2024-03-22T10:42:43.811 INFO:tasks.workunit.client.0.smithi115.stdout:5/484: dread d2/f4 [0,4194304] 0 2024-03-22T10:42:43.814 INFO:tasks.workunit.client.1.smithi150.stdout:6/65: truncate db/dd/f14 4939353 0 2024-03-22T10:42:43.815 INFO:tasks.workunit.client.0.smithi115.stdout:5/485: creat d2/df/d3b/fa0 x:0 0 0 2024-03-22T10:42:43.819 INFO:tasks.workunit.client.1.smithi150.stdout:7/57: rename d2/d5/d7/d9/lf to d2/d5/d7/d9/l12 0 2024-03-22T10:42:43.821 INFO:tasks.workunit.client.1.smithi150.stdout:9/64: mkdir d1/d19 0 2024-03-22T10:42:43.821 INFO:tasks.workunit.client.0.smithi115.stdout:0/451: sync 2024-03-22T10:42:43.821 INFO:tasks.workunit.client.0.smithi115.stdout:4/458: sync 2024-03-22T10:42:43.821 INFO:tasks.workunit.client.0.smithi115.stdout:3/423: sync 2024-03-22T10:42:43.821 INFO:tasks.workunit.client.0.smithi115.stdout:1/549: sync 2024-03-22T10:42:43.822 INFO:tasks.workunit.client.0.smithi115.stdout:5/486: rename d2/d15/d1b/d7c/c59 to d2/d15/d1b/d7c/d38/d56/ca1 0 2024-03-22T10:42:43.824 INFO:tasks.workunit.client.1.smithi150.stdout:8/80: creat d0/f1e x:0 0 0 2024-03-22T10:42:43.825 INFO:tasks.workunit.client.0.smithi115.stdout:1/550: creat d0/d22/d89/fba x:0 0 0 2024-03-22T10:42:43.827 INFO:tasks.workunit.client.0.smithi115.stdout:1/551: write d0/d8/de/d1a/f4a [1548509,64118] 0 2024-03-22T10:42:43.827 INFO:tasks.workunit.client.1.smithi150.stdout:7/58: creat d2/d5/d7/d9/f13 x:0 0 0 2024-03-22T10:42:43.829 INFO:tasks.workunit.client.1.smithi150.stdout:9/65: chown d1/d8/da/l5 3434 1 2024-03-22T10:42:43.829 INFO:tasks.workunit.client.1.smithi150.stdout:5/45: creat d0/d8/dc/f12 x:0 0 0 2024-03-22T10:42:43.830 INFO:tasks.workunit.client.0.smithi115.stdout:1/552: chown d0/d12/d17/f80 816 1 2024-03-22T10:42:43.831 INFO:tasks.workunit.client.1.smithi150.stdout:7/59: write d2/f6 [4747437,97102] 0 2024-03-22T10:42:43.832 INFO:tasks.workunit.client.0.smithi115.stdout:1/553: readlink d0/d12/d17/d37/d5c/l74 0 2024-03-22T10:42:43.838 INFO:tasks.workunit.client.1.smithi150.stdout:8/81: mknod d0/c1f 0 2024-03-22T10:42:43.839 INFO:tasks.workunit.client.0.smithi115.stdout:8/686: dread d6/dd/f6a [0,4194304] 0 2024-03-22T10:42:43.840 INFO:tasks.workunit.client.1.smithi150.stdout:1/73: getdents d1/da 0 2024-03-22T10:42:43.841 INFO:tasks.workunit.client.0.smithi115.stdout:0/452: dwrite d3/da/d4c/f67 [0,4194304] 0 2024-03-22T10:42:43.841 INFO:tasks.workunit.client.0.smithi115.stdout:8/687: write d6/dd/f26 [4525798,109916] 0 2024-03-22T10:42:43.844 INFO:tasks.workunit.client.0.smithi115.stdout:0/453: dread - d3/d52/f6e zero size 2024-03-22T10:42:43.846 INFO:tasks.workunit.client.1.smithi150.stdout:5/46: creat d0/d8/dc/de/f13 x:0 0 0 2024-03-22T10:42:43.849 INFO:tasks.workunit.client.1.smithi150.stdout:9/66: mknod d1/c1a 0 2024-03-22T10:42:43.850 INFO:tasks.workunit.client.1.smithi150.stdout:5/47: fsync d0/f11 0 2024-03-22T10:42:43.854 INFO:tasks.workunit.client.1.smithi150.stdout:1/74: unlink d1/da/c12 0 2024-03-22T10:42:43.854 INFO:tasks.workunit.client.1.smithi150.stdout:6/66: truncate db/dd/f14 4652714 0 2024-03-22T10:42:43.856 INFO:tasks.workunit.client.1.smithi150.stdout:9/67: write d1/d8/de/df/f12 [884527,49547] 0 2024-03-22T10:42:43.859 INFO:tasks.workunit.client.1.smithi150.stdout:8/82: rename d0/f6 to d0/d2/d19/f20 0 2024-03-22T10:42:43.865 INFO:tasks.workunit.client.1.smithi150.stdout:8/83: write d0/f1 [2432976,91647] 0 2024-03-22T10:42:43.867 INFO:tasks.workunit.client.1.smithi150.stdout:6/67: truncate db/ff 4921428 0 2024-03-22T10:42:43.867 INFO:tasks.workunit.client.1.smithi150.stdout:9/68: creat d1/d8/de/f1b x:0 0 0 2024-03-22T10:42:43.868 INFO:tasks.workunit.client.0.smithi115.stdout:1/554: chown d0/d12/d2b/d77/fa2 569473 1 2024-03-22T10:42:43.871 INFO:tasks.workunit.client.0.smithi115.stdout:0/454: creat d3/d4/d6d/f98 x:0 0 0 2024-03-22T10:42:43.875 INFO:tasks.workunit.client.1.smithi150.stdout:7/60: dwrite d2/d5/d7/d9/f13 [0,4194304] 0 2024-03-22T10:42:43.879 INFO:tasks.workunit.client.1.smithi150.stdout:2/97: sync 2024-03-22T10:42:43.880 INFO:tasks.workunit.client.0.smithi115.stdout:5/487: link d2/df/c40 d2/df/d27/d7a/ca2 0 2024-03-22T10:42:43.882 INFO:tasks.workunit.client.0.smithi115.stdout:1/555: symlink d0/d8/db/lbb 0 2024-03-22T10:42:43.884 INFO:tasks.workunit.client.0.smithi115.stdout:8/688: getdents d6/d49/d88 0 2024-03-22T10:42:43.885 INFO:tasks.workunit.client.0.smithi115.stdout:0/455: dread d3/d7/d1c/f48 [0,4194304] 0 2024-03-22T10:42:43.887 INFO:tasks.workunit.client.0.smithi115.stdout:1/556: write d0/d12/faa [127457,107457] 0 2024-03-22T10:42:43.891 INFO:tasks.workunit.client.1.smithi150.stdout:5/48: dread d0/f5 [0,4194304] 0 2024-03-22T10:42:43.892 INFO:tasks.workunit.client.0.smithi115.stdout:8/689: fsync d6/dd/d24/fa7 0 2024-03-22T10:42:43.893 INFO:tasks.workunit.client.0.smithi115.stdout:1/557: creat d0/d22/d75/fbc x:0 0 0 2024-03-22T10:42:43.893 INFO:tasks.workunit.client.0.smithi115.stdout:9/493: sync 2024-03-22T10:42:43.896 INFO:tasks.workunit.client.0.smithi115.stdout:8/690: fsync d6/dd/d24/d4a/f81 0 2024-03-22T10:42:43.898 INFO:tasks.workunit.client.0.smithi115.stdout:9/494: truncate d6/d7/d20/d22/d23/f83 670048 0 2024-03-22T10:42:43.900 INFO:tasks.workunit.client.0.smithi115.stdout:8/691: creat d6/dd/d24/d4a/d41/fe1 x:0 0 0 2024-03-22T10:42:43.901 INFO:tasks.workunit.client.0.smithi115.stdout:9/495: write d6/d7/d20/f4c [1720730,114042] 0 2024-03-22T10:42:43.903 INFO:tasks.workunit.client.0.smithi115.stdout:8/692: write d6/d49/f83 [3945363,100404] 0 2024-03-22T10:42:43.904 INFO:tasks.workunit.client.0.smithi115.stdout:0/456: creat d3/d4/d14/f99 x:0 0 0 2024-03-22T10:42:43.909 INFO:tasks.workunit.client.0.smithi115.stdout:0/457: mkdir d3/d7/d1c/d6c/d9a 0 2024-03-22T10:42:43.911 INFO:tasks.workunit.client.1.smithi150.stdout:8/84: dread d0/d2/d19/f20 [0,4194304] 0 2024-03-22T10:42:43.911 INFO:tasks.workunit.client.1.smithi150.stdout:1/75: dwrite d1/da/fd [0,4194304] 0 2024-03-22T10:42:43.915 INFO:tasks.workunit.client.0.smithi115.stdout:0/458: mkdir d3/d7/d1b/d9b 0 2024-03-22T10:42:43.916 INFO:tasks.workunit.client.1.smithi150.stdout:9/69: dwrite d1/d8/de/f1b [0,4194304] 0 2024-03-22T10:42:43.917 INFO:tasks.workunit.client.1.smithi150.stdout:6/68: dread db/dd/f14 [0,4194304] 0 2024-03-22T10:42:43.918 INFO:tasks.workunit.client.1.smithi150.stdout:8/85: dread - d0/dc/df/f1a zero size 2024-03-22T10:42:43.920 INFO:tasks.workunit.client.0.smithi115.stdout:3/424: truncate d3/f23 1563476 0 2024-03-22T10:42:43.921 INFO:tasks.workunit.client.0.smithi115.stdout:1/558: rmdir d0/d12/d17/db3 0 2024-03-22T10:42:43.922 INFO:tasks.workunit.client.1.smithi150.stdout:7/61: dwrite d2/f6 [0,4194304] 0 2024-03-22T10:42:43.922 INFO:tasks.workunit.client.1.smithi150.stdout:2/98: dread fe [0,4194304] 0 2024-03-22T10:42:43.923 INFO:tasks.workunit.client.1.smithi150.stdout:9/70: unlink d1/d8/de/f1b 0 2024-03-22T10:42:43.924 INFO:tasks.workunit.client.0.smithi115.stdout:3/425: rmdir d3/d19 39 2024-03-22T10:42:43.924 INFO:tasks.workunit.client.0.smithi115.stdout:0/459: creat d3/d4/d6d/f9c x:0 0 0 2024-03-22T10:42:43.926 INFO:tasks.workunit.client.1.smithi150.stdout:9/71: stat d1/d8/cd 0 2024-03-22T10:42:43.927 INFO:tasks.workunit.client.0.smithi115.stdout:9/496: dwrite d6/d8/d3c/d59/d68/d8e/f9a [0,4194304] 0 2024-03-22T10:42:43.928 INFO:tasks.workunit.client.0.smithi115.stdout:1/559: mkdir d0/d8/d4d/d56/dbd 0 2024-03-22T10:42:43.928 INFO:tasks.workunit.client.1.smithi150.stdout:2/99: creat d12/d19/f1b x:0 0 0 2024-03-22T10:42:43.928 INFO:tasks.workunit.client.1.smithi150.stdout:8/86: creat d0/f21 x:0 0 0 2024-03-22T10:42:43.932 INFO:tasks.workunit.client.0.smithi115.stdout:4/459: rename d1/d16/d35/d40/d95 to d1/d30/d42/d75/d99 0 2024-03-22T10:42:43.935 INFO:tasks.workunit.client.0.smithi115.stdout:0/460: symlink d3/da/d4c/l9d 0 2024-03-22T10:42:43.936 INFO:tasks.workunit.client.1.smithi150.stdout:8/87: symlink d0/l22 0 2024-03-22T10:42:43.938 INFO:tasks.workunit.client.0.smithi115.stdout:3/426: mkdir d3/d89 0 2024-03-22T10:42:43.942 INFO:tasks.workunit.client.0.smithi115.stdout:9/497: chown d6/d7/d20/f55 12883 1 2024-03-22T10:42:43.944 INFO:tasks.workunit.client.0.smithi115.stdout:8/693: rename d6/dd/d24/d4a/f66 to d6/d49/d88/fe2 0 2024-03-22T10:42:43.944 INFO:tasks.workunit.client.0.smithi115.stdout:4/460: creat d1/d66/d82/f9a x:0 0 0 2024-03-22T10:42:43.944 INFO:tasks.workunit.client.1.smithi150.stdout:1/76: dread d1/da/fe [0,4194304] 0 2024-03-22T10:42:43.946 INFO:tasks.workunit.client.1.smithi150.stdout:7/62: dread d2/d5/d7/d9/f13 [0,4194304] 0 2024-03-22T10:42:43.947 INFO:tasks.workunit.client.0.smithi115.stdout:5/488: unlink d2/d15/d1b/d7c/d38/f5b 0 2024-03-22T10:42:43.947 INFO:tasks.workunit.client.0.smithi115.stdout:9/498: write d6/d7/d30/d3b/f58 [1492322,3171] 0 2024-03-22T10:42:43.947 INFO:tasks.workunit.client.0.smithi115.stdout:7/482: sync 2024-03-22T10:42:43.947 INFO:tasks.workunit.client.0.smithi115.stdout:6/463: sync 2024-03-22T10:42:43.947 INFO:tasks.workunit.client.0.smithi115.stdout:2/460: sync 2024-03-22T10:42:43.948 INFO:tasks.workunit.client.0.smithi115.stdout:1/560: dwrite d0/d12/d17/f80 [8388608,4194304] 0 2024-03-22T10:42:43.949 INFO:tasks.workunit.client.1.smithi150.stdout:2/100: rename fe to d12/d19/f1c 0 2024-03-22T10:42:43.954 INFO:tasks.workunit.client.0.smithi115.stdout:3/427: fdatasync d3/d19/d2b/d55/f88 0 2024-03-22T10:42:43.958 INFO:tasks.workunit.client.1.smithi150.stdout:9/72: dwrite d1/d8/de/f16 [0,4194304] 0 2024-03-22T10:42:43.958 INFO:tasks.workunit.client.0.smithi115.stdout:3/428: chown d3/d36/c4f 19 1 2024-03-22T10:42:43.959 INFO:tasks.workunit.client.0.smithi115.stdout:5/489: rename d2/df/d3b/d8e/f64 to d2/df/d27/d7a/fa3 0 2024-03-22T10:42:43.959 INFO:tasks.workunit.client.1.smithi150.stdout:9/73: readlink d1/l15 0 2024-03-22T10:42:43.961 INFO:tasks.workunit.client.0.smithi115.stdout:1/561: creat d0/d22/d75/fbe x:0 0 0 2024-03-22T10:42:43.962 INFO:tasks.workunit.client.0.smithi115.stdout:3/429: chown d3/d36/d41/d85 7 1 2024-03-22T10:42:43.963 INFO:tasks.workunit.client.0.smithi115.stdout:9/499: creat d6/d7/d20/d22/d3f/fab x:0 0 0 2024-03-22T10:42:43.964 INFO:tasks.workunit.client.0.smithi115.stdout:5/490: chown d2/f1f 225 1 2024-03-22T10:42:43.964 INFO:tasks.workunit.client.1.smithi150.stdout:2/101: creat d12/d19/f1d x:0 0 0 2024-03-22T10:42:43.965 INFO:tasks.workunit.client.0.smithi115.stdout:1/562: chown d0/c6 2 1 2024-03-22T10:42:43.966 INFO:tasks.workunit.client.1.smithi150.stdout:8/88: creat d0/f23 x:0 0 0 2024-03-22T10:42:43.968 INFO:tasks.workunit.client.0.smithi115.stdout:8/694: mkdir d6/dd/d12/de3 0 2024-03-22T10:42:43.970 INFO:tasks.workunit.client.0.smithi115.stdout:1/563: dread - d0/d12/d17/f7d zero size 2024-03-22T10:42:43.973 INFO:tasks.workunit.client.1.smithi150.stdout:7/63: dread d2/f6 [0,4194304] 0 2024-03-22T10:42:43.974 INFO:tasks.workunit.client.0.smithi115.stdout:8/695: dread - d6/d49/fd0 zero size 2024-03-22T10:42:43.975 INFO:tasks.workunit.client.0.smithi115.stdout:3/430: symlink d3/d19/d61/l8a 0 2024-03-22T10:42:43.976 INFO:tasks.workunit.client.1.smithi150.stdout:1/77: dwrite f0 [0,4194304] 0 2024-03-22T10:42:43.978 INFO:tasks.workunit.client.0.smithi115.stdout:8/696: stat d6/d49/d68/c97 0 2024-03-22T10:42:43.980 INFO:tasks.workunit.client.1.smithi150.stdout:1/78: write d1/da/fd [3312140,111622] 0 2024-03-22T10:42:43.980 INFO:tasks.workunit.client.0.smithi115.stdout:2/461: dwrite da/d25/f34 [0,4194304] 0 2024-03-22T10:42:43.984 INFO:tasks.workunit.client.1.smithi150.stdout:2/102: dread f11 [4194304,4194304] 0 2024-03-22T10:42:43.989 INFO:tasks.workunit.client.1.smithi150.stdout:1/79: rename d1/f3 to d1/da/f14 0 2024-03-22T10:42:43.989 INFO:tasks.workunit.client.0.smithi115.stdout:9/500: rename d6/d7/f5d to d6/d7/d48/d70/da9/fac 0 2024-03-22T10:42:43.992 INFO:tasks.workunit.client.1.smithi150.stdout:2/103: write d12/d19/f1c [4398750,27991] 0 2024-03-22T10:42:43.992 INFO:tasks.workunit.client.1.smithi150.stdout:1/80: chown d1/f7 5315301 1 2024-03-22T10:42:43.993 INFO:tasks.workunit.client.1.smithi150.stdout:7/64: dread d2/f6 [0,4194304] 0 2024-03-22T10:42:43.993 INFO:tasks.workunit.client.1.smithi150.stdout:9/74: dwrite d1/d8/de/df/f18 [0,4194304] 0 2024-03-22T10:42:43.994 INFO:tasks.workunit.client.1.smithi150.stdout:2/104: fdatasync d12/f13 0 2024-03-22T10:42:43.996 INFO:tasks.workunit.client.0.smithi115.stdout:4/461: truncate d1/d16/d68/f8c 128536 0 2024-03-22T10:42:43.997 INFO:tasks.workunit.client.1.smithi150.stdout:9/75: read d1/d8/de/df/f18 [4222464,84555] 0 2024-03-22T10:42:44.002 INFO:tasks.workunit.client.1.smithi150.stdout:9/76: fdatasync d1/d8/de/f17 0 2024-03-22T10:42:44.002 INFO:tasks.workunit.client.0.smithi115.stdout:6/464: dwrite d4/d36/d48/d72/f76 [0,4194304] 0 2024-03-22T10:42:44.002 INFO:tasks.workunit.client.0.smithi115.stdout:2/462: fsync da/d25/d51/f56 0 2024-03-22T10:42:44.003 INFO:tasks.workunit.client.0.smithi115.stdout:7/483: truncate d2/d7/d12/d10/d19/d42/d76/d74/f77 36324 0 2024-03-22T10:42:44.004 INFO:tasks.workunit.client.1.smithi150.stdout:9/77: stat d1/c7 0 2024-03-22T10:42:44.004 INFO:tasks.workunit.client.1.smithi150.stdout:2/105: symlink d12/l1e 0 2024-03-22T10:42:44.007 INFO:tasks.workunit.client.0.smithi115.stdout:5/491: write d2/fe [1516603,78011] 0 2024-03-22T10:42:44.007 INFO:tasks.workunit.client.1.smithi150.stdout:2/106: dread - d12/d19/f1b zero size 2024-03-22T10:42:44.010 INFO:tasks.workunit.client.1.smithi150.stdout:7/65: link d2/cc d2/d5/d7/c14 0 2024-03-22T10:42:44.010 INFO:tasks.workunit.client.1.smithi150.stdout:9/78: write d1/d8/de/f16 [2322603,51917] 0 2024-03-22T10:42:44.011 INFO:tasks.workunit.client.0.smithi115.stdout:0/461: dwrite d3/d7/d1b/f47 [0,4194304] 0 2024-03-22T10:42:44.014 INFO:tasks.workunit.client.0.smithi115.stdout:3/431: truncate d3/f50 522573 0 2024-03-22T10:42:44.015 INFO:tasks.workunit.client.1.smithi150.stdout:2/107: symlink d12/d19/l1f 0 2024-03-22T10:42:44.016 INFO:tasks.workunit.client.0.smithi115.stdout:4/462: mkdir d1/d30/d42/d75/d99/d98/d9b 0 2024-03-22T10:42:44.019 INFO:tasks.workunit.client.0.smithi115.stdout:4/463: stat d1/d69/l94 0 2024-03-22T10:42:44.021 INFO:tasks.workunit.client.1.smithi150.stdout:7/66: write d2/d5/d7/d9/f13 [645398,94764] 0 2024-03-22T10:42:44.022 INFO:tasks.workunit.client.1.smithi150.stdout:2/108: symlink d12/d19/l20 0 2024-03-22T10:42:44.026 INFO:tasks.workunit.client.1.smithi150.stdout:1/81: dwrite d1/f7 [0,4194304] 0 2024-03-22T10:42:44.027 INFO:tasks.workunit.client.1.smithi150.stdout:2/109: chown f11 30394 1 2024-03-22T10:42:44.027 INFO:tasks.workunit.client.0.smithi115.stdout:1/564: dwrite d0/d8/de/f4c [4194304,4194304] 0 2024-03-22T10:42:44.028 INFO:tasks.workunit.client.0.smithi115.stdout:0/462: dread d3/d7/d1b/d1f/f58 [0,4194304] 0 2024-03-22T10:42:44.031 INFO:tasks.workunit.client.0.smithi115.stdout:9/501: creat d6/d7/d20/d22/fad x:0 0 0 2024-03-22T10:42:44.031 INFO:tasks.workunit.client.0.smithi115.stdout:4/464: creat d1/d16/d35/f9c x:0 0 0 2024-03-22T10:42:44.035 INFO:tasks.workunit.client.1.smithi150.stdout:1/82: unlink d1/c4 0 2024-03-22T10:42:44.039 INFO:tasks.workunit.client.1.smithi150.stdout:1/83: mkdir d1/da/d15 0 2024-03-22T10:42:44.040 INFO:tasks.workunit.client.0.smithi115.stdout:1/565: chown d0/d12/d17/f33 775 1 2024-03-22T10:42:44.040 INFO:tasks.workunit.client.0.smithi115.stdout:7/484: creat d2/d7/d12/d10/d19/fa9 x:0 0 0 2024-03-22T10:42:44.041 INFO:tasks.workunit.client.0.smithi115.stdout:9/502: fsync d6/d7/d20/f57 0 2024-03-22T10:42:44.043 INFO:tasks.workunit.client.1.smithi150.stdout:9/79: dwrite d1/d8/de/df/f12 [0,4194304] 0 2024-03-22T10:42:44.045 INFO:tasks.workunit.client.1.smithi150.stdout:1/84: link d1/da/f13 d1/da/f16 0 2024-03-22T10:42:44.047 INFO:tasks.workunit.client.1.smithi150.stdout:9/80: symlink d1/d8/de/l1c 0 2024-03-22T10:42:44.047 INFO:tasks.workunit.client.0.smithi115.stdout:2/463: dwrite da/d25/d30/f59 [0,4194304] 0 2024-03-22T10:42:44.047 INFO:tasks.workunit.client.0.smithi115.stdout:3/432: dwrite d3/d82/f22 [0,4194304] 0 2024-03-22T10:42:44.048 INFO:tasks.workunit.client.1.smithi150.stdout:2/110: dread d12/f13 [0,4194304] 0 2024-03-22T10:42:44.050 INFO:tasks.workunit.client.1.smithi150.stdout:7/67: dread d2/d5/d7/d9/f13 [0,4194304] 0 2024-03-22T10:42:44.052 INFO:tasks.workunit.client.1.smithi150.stdout:1/85: dread d1/da/f16 [0,4194304] 0 2024-03-22T10:42:44.053 INFO:tasks.workunit.client.1.smithi150.stdout:9/81: write d1/d8/de/df/f18 [7906974,81327] 0 2024-03-22T10:42:44.053 INFO:tasks.workunit.client.1.smithi150.stdout:2/111: mkdir d12/d21 0 2024-03-22T10:42:44.054 INFO:tasks.workunit.client.1.smithi150.stdout:7/68: fdatasync d2/f6 0 2024-03-22T10:42:44.057 INFO:tasks.workunit.client.1.smithi150.stdout:1/86: mknod d1/da/d15/c17 0 2024-03-22T10:42:44.061 INFO:tasks.workunit.client.0.smithi115.stdout:1/566: chown d0/d22/d75/fa5 10 1 2024-03-22T10:42:44.062 INFO:tasks.workunit.client.0.smithi115.stdout:0/463: rename d3/d4/d14/f99 to d3/d4/d6d/f9e 0 2024-03-22T10:42:44.063 INFO:tasks.workunit.client.1.smithi150.stdout:7/69: unlink d2/f6 0 2024-03-22T10:42:44.063 INFO:tasks.workunit.client.1.smithi150.stdout:9/82: mknod d1/d19/c1d 0 2024-03-22T10:42:44.064 INFO:tasks.workunit.client.1.smithi150.stdout:7/70: chown d2/d5/d7/d8 129468632 1 2024-03-22T10:42:44.064 INFO:tasks.workunit.client.0.smithi115.stdout:2/464: symlink da/d1c/l8f 0 2024-03-22T10:42:44.065 INFO:tasks.workunit.client.0.smithi115.stdout:9/503: mknod d6/d7/d19/d4e/cae 0 2024-03-22T10:42:44.067 INFO:tasks.workunit.client.1.smithi150.stdout:3/67: sync 2024-03-22T10:42:44.067 INFO:tasks.workunit.client.1.smithi150.stdout:0/103: sync 2024-03-22T10:42:44.069 INFO:tasks.workunit.client.1.smithi150.stdout:3/68: readlink d5/l7 0 2024-03-22T10:42:44.070 INFO:tasks.workunit.client.0.smithi115.stdout:6/465: dwrite d4/d11/f52 [0,4194304] 0 2024-03-22T10:42:44.071 INFO:tasks.workunit.client.1.smithi150.stdout:1/87: mkdir d1/da/d15/d18 0 2024-03-22T10:42:44.078 INFO:tasks.workunit.client.1.smithi150.stdout:9/83: rmdir d1/d19 39 2024-03-22T10:42:44.080 INFO:tasks.workunit.client.1.smithi150.stdout:7/71: unlink d2/d5/d7/d9/f13 0 2024-03-22T10:42:44.080 INFO:tasks.workunit.client.1.smithi150.stdout:9/84: chown l0 617 1 2024-03-22T10:42:44.081 INFO:tasks.workunit.client.1.smithi150.stdout:3/69: mknod d5/c12 0 2024-03-22T10:42:44.081 INFO:tasks.workunit.client.0.smithi115.stdout:4/465: creat d1/d16/f9d x:0 0 0 2024-03-22T10:42:44.083 INFO:tasks.workunit.client.0.smithi115.stdout:4/466: stat d1/d16 0 2024-03-22T10:42:44.084 INFO:tasks.workunit.client.0.smithi115.stdout:3/433: unlink d3/d19/d2b/f4d 0 2024-03-22T10:42:44.086 INFO:tasks.workunit.client.0.smithi115.stdout:0/464: symlink d3/da/d4c/d74/l9f 0 2024-03-22T10:42:44.086 INFO:tasks.workunit.client.1.smithi150.stdout:1/88: mkdir d1/d19 0 2024-03-22T10:42:44.087 INFO:tasks.workunit.client.0.smithi115.stdout:7/485: dwrite d2/d7/d12/d10/f25 [4194304,4194304] 0 2024-03-22T10:42:44.087 INFO:tasks.workunit.client.0.smithi115.stdout:5/492: dwrite d2/d15/d1b/d7c/d84/f9a [0,4194304] 0 2024-03-22T10:42:44.089 INFO:tasks.workunit.client.1.smithi150.stdout:3/70: write d5/f8 [4690809,124223] 0 2024-03-22T10:42:44.090 INFO:tasks.workunit.client.0.smithi115.stdout:0/465: chown d3/d7/d1c/d6c/d9a 8 1 2024-03-22T10:42:44.092 INFO:tasks.workunit.client.0.smithi115.stdout:0/466: fsync d3/d4/d6d/f9c 0 2024-03-22T10:42:44.092 INFO:tasks.workunit.client.1.smithi150.stdout:7/72: symlink d2/d5/d7/de/l15 0 2024-03-22T10:42:44.093 INFO:tasks.workunit.client.0.smithi115.stdout:5/493: readlink d2/df/d27/l42 0 2024-03-22T10:42:44.096 INFO:tasks.workunit.client.0.smithi115.stdout:8/697: dread d6/dd/d24/d4a/f61 [0,4194304] 0 2024-03-22T10:42:44.098 INFO:tasks.workunit.client.0.smithi115.stdout:1/567: creat d0/d8/d4d/d56/dbd/fbf x:0 0 0 2024-03-22T10:42:44.099 INFO:tasks.workunit.client.1.smithi150.stdout:7/73: rename d2 to d2/d5/d7/d9/d16 22 2024-03-22T10:42:44.099 INFO:tasks.workunit.client.1.smithi150.stdout:7/74: fdatasync - no filename 2024-03-22T10:42:44.099 INFO:tasks.workunit.client.1.smithi150.stdout:7/75: dwrite - no filename 2024-03-22T10:42:44.099 INFO:tasks.workunit.client.1.smithi150.stdout:7/76: dread - no filename 2024-03-22T10:42:44.100 INFO:tasks.workunit.client.0.smithi115.stdout:2/465: mkdir da/d25/d3d/d7e/d90 0 2024-03-22T10:42:44.103 INFO:tasks.workunit.client.1.smithi150.stdout:2/112: getdents d12/d19 0 2024-03-22T10:42:44.110 INFO:tasks.workunit.client.0.smithi115.stdout:6/466: dread d4/d11/d12/f31 [0,4194304] 0 2024-03-22T10:42:44.111 INFO:tasks.workunit.client.0.smithi115.stdout:0/467: mkdir d3/da/d4c/d68/da0 0 2024-03-22T10:42:44.112 INFO:tasks.workunit.client.1.smithi150.stdout:3/71: creat d5/da/f13 x:0 0 0 2024-03-22T10:42:44.113 INFO:tasks.workunit.client.0.smithi115.stdout:3/434: link d3/d16/f84 d3/d89/f8b 0 2024-03-22T10:42:44.114 INFO:tasks.workunit.client.0.smithi115.stdout:2/466: rmdir da 39 2024-03-22T10:42:44.116 INFO:tasks.workunit.client.1.smithi150.stdout:0/104: dwrite d0/d1/d7/f20 [0,4194304] 0 2024-03-22T10:42:44.117 INFO:tasks.workunit.client.1.smithi150.stdout:7/77: mknod d2/c17 0 2024-03-22T10:42:44.117 INFO:tasks.workunit.client.1.smithi150.stdout:3/72: chown c4 9 1 2024-03-22T10:42:44.117 INFO:tasks.workunit.client.0.smithi115.stdout:1/568: creat d0/d12/d2b/d77/fc0 x:0 0 0 2024-03-22T10:42:44.118 INFO:tasks.workunit.client.1.smithi150.stdout:2/113: creat d12/d16/f22 x:0 0 0 2024-03-22T10:42:44.120 INFO:tasks.workunit.client.0.smithi115.stdout:3/435: dread - d3/d82/d54/f59 zero size 2024-03-22T10:42:44.123 INFO:tasks.workunit.client.1.smithi150.stdout:3/73: rename d5 to d5/da/d14 22 2024-03-22T10:42:44.125 INFO:tasks.workunit.client.0.smithi115.stdout:9/504: getdents d6/d7/d20/d22/d23 0 2024-03-22T10:42:44.128 INFO:tasks.workunit.client.1.smithi150.stdout:2/114: fsync f10 0 2024-03-22T10:42:44.130 INFO:tasks.workunit.client.0.smithi115.stdout:5/494: dread d2/df/d13/f18 [8388608,4194304] 0 2024-03-22T10:42:44.131 INFO:tasks.workunit.client.1.smithi150.stdout:7/78: creat d2/d5/d7/d9/f18 x:0 0 0 2024-03-22T10:42:44.131 INFO:tasks.workunit.client.1.smithi150.stdout:9/85: dwrite d1/d8/de/df/f18 [4194304,4194304] 0 2024-03-22T10:42:44.132 INFO:tasks.workunit.client.1.smithi150.stdout:0/105: symlink d0/d1/d7/db/d10/d1e/l22 0 2024-03-22T10:42:44.132 INFO:tasks.workunit.client.0.smithi115.stdout:7/486: dwrite d2/d7/d12/d10/d19/d42/f6d [12582912,4194304] 0 2024-03-22T10:42:44.135 INFO:tasks.workunit.client.0.smithi115.stdout:0/468: truncate d3/d7/d1b/d1f/f58 274942 0 2024-03-22T10:42:44.136 INFO:tasks.workunit.client.1.smithi150.stdout:1/89: dwrite d1/f7 [0,4194304] 0 2024-03-22T10:42:44.136 INFO:tasks.workunit.client.0.smithi115.stdout:3/436: chown d3/d82/f27 3092 1 2024-03-22T10:42:44.139 INFO:tasks.workunit.client.0.smithi115.stdout:9/505: creat d6/d7/d48/faf x:0 0 0 2024-03-22T10:42:44.139 INFO:tasks.workunit.client.0.smithi115.stdout:8/698: dwrite d6/d49/d68/f9f [0,4194304] 0 2024-03-22T10:42:44.139 INFO:tasks.workunit.client.0.smithi115.stdout:2/467: symlink da/d25/d3d/d57/l91 0 2024-03-22T10:42:44.139 INFO:tasks.workunit.client.1.smithi150.stdout:9/86: fsync d1/d8/de/f16 0 2024-03-22T10:42:44.140 INFO:tasks.workunit.client.1.smithi150.stdout:7/79: readlink d2/d5/d7/d9/db/l11 0 2024-03-22T10:42:44.141 INFO:tasks.workunit.client.0.smithi115.stdout:6/467: getdents d4/d36/d48/d72 0 2024-03-22T10:42:44.143 INFO:tasks.workunit.client.1.smithi150.stdout:3/74: link d5/l9 d5/da/l15 0 2024-03-22T10:42:44.144 INFO:tasks.workunit.client.0.smithi115.stdout:2/468: chown da/d25/d2a/f72 29 1 2024-03-22T10:42:44.145 INFO:tasks.workunit.client.0.smithi115.stdout:2/469: chown da/d25/d30/d4f/d54 899 1 2024-03-22T10:42:44.146 INFO:tasks.workunit.client.1.smithi150.stdout:3/75: chown d5/da/ff 21840 1 2024-03-22T10:42:44.147 INFO:tasks.workunit.client.1.smithi150.stdout:3/76: truncate d5/fb 429357 0 2024-03-22T10:42:44.147 INFO:tasks.workunit.client.0.smithi115.stdout:2/470: write da/d1c/f69 [549426,1436] 0 2024-03-22T10:42:44.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:43 smithi150 ceph-mon[34518]: pgmap v10: 65 pgs: 65 active+clean; 920 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 126 MiB/s wr, 280 op/s 2024-03-22T10:42:44.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:43 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:44.150 INFO:tasks.workunit.client.1.smithi150.stdout:2/115: rename ld to d12/d16/l23 0 2024-03-22T10:42:44.155 INFO:tasks.workunit.client.0.smithi115.stdout:5/495: mknod d2/d15/d1b/d7c/d38/d43/ca4 0 2024-03-22T10:42:44.157 INFO:tasks.workunit.client.1.smithi150.stdout:0/106: mknod d0/d1/d7/db/c23 0 2024-03-22T10:42:44.162 INFO:tasks.workunit.client.1.smithi150.stdout:1/90: dread d1/da/fd [0,4194304] 0 2024-03-22T10:42:44.167 INFO:tasks.workunit.client.0.smithi115.stdout:2/471: rmdir da/d25/d30 39 2024-03-22T10:42:44.167 INFO:tasks.workunit.client.0.smithi115.stdout:3/437: dwrite d3/d16/d29/f3c [0,4194304] 0 2024-03-22T10:42:44.168 INFO:tasks.workunit.client.1.smithi150.stdout:7/80: symlink d2/d5/d7/l19 0 2024-03-22T10:42:44.168 INFO:tasks.workunit.client.1.smithi150.stdout:3/77: symlink d5/l16 0 2024-03-22T10:42:44.168 INFO:tasks.workunit.client.1.smithi150.stdout:2/116: fsync f10 0 2024-03-22T10:42:44.174 INFO:tasks.workunit.client.1.smithi150.stdout:1/91: mkdir d1/d19/d1a 0 2024-03-22T10:42:44.174 INFO:tasks.workunit.client.1.smithi150.stdout:3/78: mknod d5/c17 0 2024-03-22T10:42:44.174 INFO:tasks.workunit.client.1.smithi150.stdout:7/81: unlink d2/d5/d7/l19 0 2024-03-22T10:42:44.175 INFO:tasks.workunit.client.1.smithi150.stdout:9/87: link d1/d8/da/l13 d1/d8/l1e 0 2024-03-22T10:42:44.176 INFO:tasks.workunit.client.1.smithi150.stdout:1/92: stat d1/l6 0 2024-03-22T10:42:44.179 INFO:tasks.workunit.client.1.smithi150.stdout:3/79: unlink d5/da/f13 0 2024-03-22T10:42:44.182 INFO:tasks.workunit.client.1.smithi150.stdout:0/107: dwrite d0/d1/fa [0,4194304] 0 2024-03-22T10:42:44.182 INFO:tasks.workunit.client.0.smithi115.stdout:8/699: creat d6/dd/fe4 x:0 0 0 2024-03-22T10:42:44.183 INFO:tasks.workunit.client.1.smithi150.stdout:3/80: write d5/fb [233914,89696] 0 2024-03-22T10:42:44.183 INFO:tasks.workunit.client.1.smithi150.stdout:1/93: unlink f0 0 2024-03-22T10:42:44.185 INFO:tasks.workunit.client.0.smithi115.stdout:3/438: creat d3/d82/f8c x:0 0 0 2024-03-22T10:42:44.187 INFO:tasks.workunit.client.1.smithi150.stdout:0/108: read d0/d1/d7/fe [2404811,100051] 0 2024-03-22T10:42:44.188 INFO:tasks.workunit.client.0.smithi115.stdout:1/569: rename d0/d12/d2b/d77 to d0/d8/db/d25/d7e/dc1 0 2024-03-22T10:42:44.192 INFO:tasks.workunit.client.0.smithi115.stdout:7/487: dread d2/d7/d12/d10/d19/d42/f3e [0,4194304] 0 2024-03-22T10:42:44.196 INFO:tasks.workunit.client.0.smithi115.stdout:0/469: link d3/d4/cf d3/da/ca1 0 2024-03-22T10:42:44.197 INFO:tasks.workunit.client.0.smithi115.stdout:5/496: symlink d2/la5 0 2024-03-22T10:42:44.199 INFO:tasks.workunit.client.0.smithi115.stdout:8/700: chown d6/dd/d24/d4a/d7b/c7e 34680677 1 2024-03-22T10:42:44.201 INFO:tasks.workunit.client.1.smithi150.stdout:3/81: dread d5/da/ff [0,4194304] 0 2024-03-22T10:42:44.202 INFO:tasks.workunit.client.1.smithi150.stdout:3/82: chown l2 859088434 1 2024-03-22T10:42:44.206 INFO:tasks.workunit.client.1.smithi150.stdout:3/83: chown d5/da/ff 45 1 2024-03-22T10:42:44.206 INFO:tasks.workunit.client.1.smithi150.stdout:9/88: truncate d1/d8/de/f16 1066325 0 2024-03-22T10:42:44.207 INFO:tasks.workunit.client.0.smithi115.stdout:3/439: unlink d3/d36/d41/c37 0 2024-03-22T10:42:44.207 INFO:tasks.workunit.client.0.smithi115.stdout:9/506: dread d6/f2e [0,4194304] 0 2024-03-22T10:42:44.207 INFO:tasks.workunit.client.0.smithi115.stdout:3/440: chown d3 4607 1 2024-03-22T10:42:44.207 INFO:tasks.workunit.client.1.smithi150.stdout:9/89: chown d1/l15 180961582 1 2024-03-22T10:42:44.210 INFO:tasks.workunit.client.0.smithi115.stdout:8/701: dread d6/dd/d24/d4a/d41/f62 [0,4194304] 0 2024-03-22T10:42:44.210 INFO:tasks.workunit.client.0.smithi115.stdout:9/507: fdatasync d6/d7/d19/d4e/f6d 0 2024-03-22T10:42:44.211 INFO:tasks.workunit.client.0.smithi115.stdout:0/470: mknod d3/d7/d6b/ca2 0 2024-03-22T10:42:44.211 INFO:tasks.workunit.client.0.smithi115.stdout:2/472: chown da/d25/d30/l75 7 1 2024-03-22T10:42:44.211 INFO:tasks.workunit.client.1.smithi150.stdout:7/82: getdents d2/d5/d7/d9 0 2024-03-22T10:42:44.212 INFO:tasks.workunit.client.1.smithi150.stdout:2/117: link d12/d16/l23 d12/d19/l24 0 2024-03-22T10:42:44.214 INFO:tasks.workunit.client.1.smithi150.stdout:4/85: sync 2024-03-22T10:42:44.215 INFO:tasks.workunit.client.0.smithi115.stdout:0/471: chown d3/da/d4c/d68/f5e 94639823 1 2024-03-22T10:42:44.216 INFO:tasks.workunit.client.0.smithi115.stdout:2/473: read da/d25/d30/f43 [1507932,41622] 0 2024-03-22T10:42:44.216 INFO:tasks.workunit.client.0.smithi115.stdout:5/497: rename d2/d8c to d2/df/d13/d8b/d93/da6 0 2024-03-22T10:42:44.219 INFO:tasks.workunit.client.0.smithi115.stdout:1/570: creat d0/d8/db/d25/d9f/fc2 x:0 0 0 2024-03-22T10:42:44.221 INFO:tasks.workunit.client.0.smithi115.stdout:1/571: write d0/d12/faa [925574,80144] 0 2024-03-22T10:42:44.221 INFO:tasks.workunit.client.1.smithi150.stdout:7/83: chown d2/d5/d7/d9/db 507120540 1 2024-03-22T10:42:44.222 INFO:tasks.workunit.client.1.smithi150.stdout:0/109: write d0/d1/d7/db/d10/f13 [2314058,49550] 0 2024-03-22T10:42:44.226 INFO:tasks.workunit.client.1.smithi150.stdout:2/118: write d12/d19/f1d [951705,55662] 0 2024-03-22T10:42:44.232 INFO:tasks.workunit.client.1.smithi150.stdout:7/84: dread - d2/d5/d7/d9/f18 zero size 2024-03-22T10:42:44.232 INFO:tasks.workunit.client.1.smithi150.stdout:0/110: rename d0/d1/d7 to d0/d1/d7/db/d10/d24 22 2024-03-22T10:42:44.232 INFO:tasks.workunit.client.1.smithi150.stdout:3/84: dwrite d5/fb [0,4194304] 0 2024-03-22T10:42:44.233 INFO:tasks.workunit.client.0.smithi115.stdout:3/441: creat d3/d36/d41/d33/f8d x:0 0 0 2024-03-22T10:42:44.233 INFO:tasks.workunit.client.0.smithi115.stdout:7/488: mknod d2/d7/caa 0 2024-03-22T10:42:44.234 INFO:tasks.workunit.client.0.smithi115.stdout:9/508: creat d6/d7/d20/d9c/fb0 x:0 0 0 2024-03-22T10:42:44.235 INFO:tasks.workunit.client.0.smithi115.stdout:2/474: mknod da/d25/d3d/d7e/c92 0 2024-03-22T10:42:44.236 INFO:tasks.workunit.client.1.smithi150.stdout:9/90: dwrite d1/d8/de/f17 [0,4194304] 0 2024-03-22T10:42:44.237 INFO:tasks.workunit.client.0.smithi115.stdout:5/498: rmdir d2/d15/d1b/d7c/d84 39 2024-03-22T10:42:44.240 INFO:tasks.workunit.client.0.smithi115.stdout:0/472: rename d3/da/d4c/d68 to d3/da/d4c/da3 0 2024-03-22T10:42:44.240 INFO:tasks.workunit.client.0.smithi115.stdout:2/475: write da/d25/d30/d4f/d54/f7a [827939,130296] 0 2024-03-22T10:42:44.241 INFO:tasks.workunit.client.0.smithi115.stdout:9/509: truncate d6/d7/d30/d37/fa0 724356 0 2024-03-22T10:42:44.243 INFO:tasks.workunit.client.0.smithi115.stdout:1/572: symlink d0/d12/d17/d37/d9e/da0/lc3 0 2024-03-22T10:42:44.250 INFO:tasks.workunit.client.0.smithi115.stdout:1/573: creat d0/d22/fc4 x:0 0 0 2024-03-22T10:42:44.251 INFO:tasks.workunit.client.1.smithi150.stdout:4/86: dwrite d0/f7 [0,4194304] 0 2024-03-22T10:42:44.253 INFO:tasks.workunit.client.0.smithi115.stdout:7/489: creat d2/d7/d12/d10/d19/d42/d46/d61/da3/fab x:0 0 0 2024-03-22T10:42:44.253 INFO:tasks.workunit.client.1.smithi150.stdout:4/87: readlink d0/l15 0 2024-03-22T10:42:44.254 INFO:tasks.workunit.client.0.smithi115.stdout:9/510: unlink d6/l49 0 2024-03-22T10:42:44.257 INFO:tasks.workunit.client.0.smithi115.stdout:3/442: rename d3/d16/d29/f39 to d3/f8e 0 2024-03-22T10:42:44.258 INFO:tasks.workunit.client.1.smithi150.stdout:4/88: write d0/de/f18 [636454,98966] 0 2024-03-22T10:42:44.260 INFO:tasks.workunit.client.0.smithi115.stdout:0/473: read d3/d7/d1b/d35/f7e [848628,131033] 0 2024-03-22T10:42:44.260 INFO:tasks.workunit.client.0.smithi115.stdout:9/511: readlink d6/d7/d20/l35 0 2024-03-22T10:42:44.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:43 smithi115 ceph-mon[29111]: pgmap v10: 65 pgs: 65 active+clean; 920 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 126 MiB/s wr, 280 op/s 2024-03-22T10:42:44.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:43 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:44.263 INFO:tasks.workunit.client.0.smithi115.stdout:2/476: dwrite da/d25/d51/f55 [0,4194304] 0 2024-03-22T10:42:44.264 INFO:tasks.workunit.client.0.smithi115.stdout:7/490: creat d2/d7/d12/d10/d19/d42/d46/d61/da3/fac x:0 0 0 2024-03-22T10:42:44.267 INFO:tasks.workunit.client.0.smithi115.stdout:3/443: chown d3/f23 116 1 2024-03-22T10:42:44.268 INFO:tasks.workunit.client.0.smithi115.stdout:9/512: creat d6/d7/d20/d22/d3f/fb1 x:0 0 0 2024-03-22T10:42:44.269 INFO:tasks.workunit.client.0.smithi115.stdout:7/491: chown d2/d7/d12/d10/d19/d42/f9c 28 1 2024-03-22T10:42:44.273 INFO:tasks.workunit.client.0.smithi115.stdout:0/474: write d3/d7/d1b/d35/f7e [2719314,15114] 0 2024-03-22T10:42:44.274 INFO:tasks.workunit.client.0.smithi115.stdout:3/444: symlink d3/d19/l8f 0 2024-03-22T10:42:44.275 INFO:tasks.workunit.client.0.smithi115.stdout:2/477: creat da/d25/d3d/d57/f93 x:0 0 0 2024-03-22T10:42:44.277 INFO:tasks.workunit.client.0.smithi115.stdout:9/513: mkdir d6/d7/d20/d38/db2 0 2024-03-22T10:42:44.283 INFO:tasks.workunit.client.0.smithi115.stdout:3/445: write d3/d82/d3a/f75 [1938879,116287] 0 2024-03-22T10:42:44.286 INFO:tasks.workunit.client.0.smithi115.stdout:0/475: creat d3/da/d4c/d69/d6a/fa4 x:0 0 0 2024-03-22T10:42:44.289 INFO:tasks.workunit.client.0.smithi115.stdout:2/478: rename da/c6c to da/d25/d30/d4f/d54/c94 0 2024-03-22T10:42:44.292 INFO:tasks.workunit.client.0.smithi115.stdout:0/476: chown d3/d4/c25 53392270 1 2024-03-22T10:42:44.297 INFO:tasks.workunit.client.0.smithi115.stdout:8/702: dwrite f4 [0,4194304] 0 2024-03-22T10:42:44.300 INFO:tasks.workunit.client.0.smithi115.stdout:0/477: chown d3/d52/f59 170 1 2024-03-22T10:42:44.305 INFO:tasks.workunit.client.0.smithi115.stdout:7/492: dwrite d2/d7/d12/d10/d19/d42/d46/d61/f88 [0,4194304] 0 2024-03-22T10:42:44.305 INFO:tasks.workunit.client.0.smithi115.stdout:8/703: stat d6/dd/c2f 0 2024-03-22T10:42:44.309 INFO:tasks.workunit.client.1.smithi150.stdout:0/111: mknod d0/d12/c25 0 2024-03-22T10:42:44.310 INFO:tasks.workunit.client.1.smithi150.stdout:3/85: fsync d5/da/ff 0 2024-03-22T10:42:44.311 INFO:tasks.workunit.client.0.smithi115.stdout:2/479: getdents da/d25/d2a 0 2024-03-22T10:42:44.312 INFO:tasks.workunit.client.1.smithi150.stdout:3/86: fsync d5/da/ff 0 2024-03-22T10:42:44.313 INFO:tasks.workunit.client.1.smithi150.stdout:7/85: creat d2/d5/d7/d9/db/f1a x:0 0 0 2024-03-22T10:42:44.313 INFO:tasks.workunit.client.1.smithi150.stdout:9/91: readlink d1/d8/da/l13 0 2024-03-22T10:42:44.315 INFO:tasks.workunit.client.1.smithi150.stdout:2/119: creat d12/d21/f25 x:0 0 0 2024-03-22T10:42:44.316 INFO:tasks.workunit.client.0.smithi115.stdout:9/514: dwrite d6/d7/d30/d3b/f46 [4194304,4194304] 0 2024-03-22T10:42:44.316 INFO:tasks.workunit.client.0.smithi115.stdout:1/574: dread d0/d12/f1c [0,4194304] 0 2024-03-22T10:42:44.316 INFO:tasks.workunit.client.0.smithi115.stdout:8/704: mknod d6/dd/d24/d4a/ce5 0 2024-03-22T10:42:44.317 INFO:tasks.workunit.client.1.smithi150.stdout:4/89: dread d0/de/f13 [0,4194304] 0 2024-03-22T10:42:44.317 INFO:tasks.workunit.client.1.smithi150.stdout:0/112: mkdir d0/d1/d7/d11/d26 0 2024-03-22T10:42:44.317 INFO:tasks.workunit.client.1.smithi150.stdout:7/86: fdatasync d2/d5/d7/d9/db/f1a 0 2024-03-22T10:42:44.318 INFO:tasks.workunit.client.1.smithi150.stdout:3/87: readlink d5/l9 0 2024-03-22T10:42:44.319 INFO:tasks.workunit.client.1.smithi150.stdout:9/92: chown d1/d8/da/l5 5 1 2024-03-22T10:42:44.323 INFO:tasks.workunit.client.1.smithi150.stdout:3/88: rmdir d5 39 2024-03-22T10:42:44.323 INFO:tasks.workunit.client.1.smithi150.stdout:9/93: write d1/d8/de/f17 [2548017,35877] 0 2024-03-22T10:42:44.324 INFO:tasks.workunit.client.0.smithi115.stdout:8/705: dread - d6/d49/d68/f9c zero size 2024-03-22T10:42:44.324 INFO:tasks.workunit.client.1.smithi150.stdout:0/113: rmdir d0/d1/d7/db/d10 39 2024-03-22T10:42:44.325 INFO:tasks.workunit.client.1.smithi150.stdout:0/114: chown d0/l16 1465065 1 2024-03-22T10:42:44.326 INFO:tasks.workunit.client.0.smithi115.stdout:0/478: dwrite d3/da/d4c/d69/d6a/f86 [0,4194304] 0 2024-03-22T10:42:44.326 INFO:tasks.workunit.client.0.smithi115.stdout:7/493: creat d2/d7/d12/d10/d19/fad x:0 0 0 2024-03-22T10:42:44.327 INFO:tasks.workunit.client.1.smithi150.stdout:9/94: fsync d1/d8/de/df/f18 0 2024-03-22T10:42:44.328 INFO:tasks.workunit.client.0.smithi115.stdout:3/446: write d3/d89/f8b [626977,89782] 0 2024-03-22T10:42:44.330 INFO:tasks.workunit.client.0.smithi115.stdout:0/479: stat d3/d7/d1b/f24 0 2024-03-22T10:42:44.330 INFO:tasks.workunit.client.1.smithi150.stdout:0/115: write d0/d1/d7/db/fc [2123819,115286] 0 2024-03-22T10:42:44.332 INFO:tasks.workunit.client.0.smithi115.stdout:3/447: chown d3/d16/d29/f43 124836744 1 2024-03-22T10:42:44.336 INFO:tasks.workunit.client.0.smithi115.stdout:0/480: rmdir d3/d4 39 2024-03-22T10:42:44.340 INFO:tasks.workunit.client.0.smithi115.stdout:8/706: truncate d6/dd/d12/f94 253365 0 2024-03-22T10:42:44.340 INFO:tasks.workunit.client.1.smithi150.stdout:4/90: creat d0/f1e x:0 0 0 2024-03-22T10:42:44.341 INFO:tasks.workunit.client.1.smithi150.stdout:9/95: creat d1/d8/de/df/f1f x:0 0 0 2024-03-22T10:42:44.344 INFO:tasks.workunit.client.0.smithi115.stdout:3/448: write d3/d19/f4e [3783360,79678] 0 2024-03-22T10:42:44.347 INFO:tasks.workunit.client.1.smithi150.stdout:2/120: dwrite f10 [0,4194304] 0 2024-03-22T10:42:44.350 INFO:tasks.workunit.client.1.smithi150.stdout:2/121: readlink d12/l1e 0 2024-03-22T10:42:44.350 INFO:tasks.workunit.client.1.smithi150.stdout:3/89: symlink d5/da/l18 0 2024-03-22T10:42:44.352 INFO:tasks.workunit.client.1.smithi150.stdout:3/90: fsync d5/fd 0 2024-03-22T10:42:44.352 INFO:tasks.workunit.client.1.smithi150.stdout:7/87: link d2/d5/d7/de/l15 d2/d5/l1b 0 2024-03-22T10:42:44.354 INFO:tasks.workunit.client.1.smithi150.stdout:9/96: rename d1/d8/da to d1/d8/de/df/d20 0 2024-03-22T10:42:44.357 INFO:tasks.workunit.client.1.smithi150.stdout:3/91: read d5/f8 [3714948,80846] 0 2024-03-22T10:42:44.357 INFO:tasks.workunit.client.0.smithi115.stdout:3/449: symlink d3/d36/d41/d33/l90 0 2024-03-22T10:42:44.361 INFO:tasks.workunit.client.0.smithi115.stdout:3/450: chown d3 2 1 2024-03-22T10:42:44.363 INFO:tasks.workunit.client.1.smithi150.stdout:5/49: sync 2024-03-22T10:42:44.363 INFO:tasks.workunit.client.1.smithi150.stdout:0/116: dread d0/d1/d7/db/d10/f13 [0,4194304] 0 2024-03-22T10:42:44.366 INFO:tasks.workunit.client.1.smithi150.stdout:5/50: read d0/f5 [2824846,36182] 0 2024-03-22T10:42:44.366 INFO:tasks.workunit.client.0.smithi115.stdout:2/480: dwrite da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:44.366 INFO:tasks.workunit.client.0.smithi115.stdout:1/575: dwrite d0/f87 [0,4194304] 0 2024-03-22T10:42:44.367 INFO:tasks.workunit.client.0.smithi115.stdout:9/515: dwrite d6/d7/d20/d22/d3f/fab [0,4194304] 0 2024-03-22T10:42:44.370 INFO:tasks.workunit.client.1.smithi150.stdout:0/117: write d0/d1/fa [4308521,65946] 0 2024-03-22T10:42:44.372 INFO:tasks.workunit.client.0.smithi115.stdout:7/494: dwrite d2/d3a/d6e/fa2 [0,4194304] 0 2024-03-22T10:42:44.373 INFO:tasks.workunit.client.1.smithi150.stdout:7/88: rename d2/d5/l1b to d2/d5/d7/d8/l1c 0 2024-03-22T10:42:44.377 INFO:tasks.workunit.client.0.smithi115.stdout:9/516: symlink d6/lb3 0 2024-03-22T10:42:44.378 INFO:tasks.workunit.client.1.smithi150.stdout:9/97: dread d1/d8/de/df/f12 [0,4194304] 0 2024-03-22T10:42:44.380 INFO:tasks.workunit.client.0.smithi115.stdout:9/517: fdatasync d6/d8/d3c/d59/d68/d8e/f9a 0 2024-03-22T10:42:44.382 INFO:tasks.workunit.client.0.smithi115.stdout:2/481: link da/f14 da/d1c/d67/d78/f95 0 2024-03-22T10:42:44.382 INFO:tasks.workunit.client.1.smithi150.stdout:5/51: dread d0/f3 [0,4194304] 0 2024-03-22T10:42:44.383 INFO:tasks.workunit.client.1.smithi150.stdout:5/52: dread - d0/f10 zero size 2024-03-22T10:42:44.384 INFO:tasks.workunit.client.0.smithi115.stdout:7/495: mkdir d2/d3a/d90/dae 0 2024-03-22T10:42:44.384 INFO:tasks.workunit.client.0.smithi115.stdout:3/451: mkdir d3/d19/d2b/d49/d91 0 2024-03-22T10:42:44.387 INFO:tasks.workunit.client.0.smithi115.stdout:2/482: readlink da/d1c/d67/l7c 0 2024-03-22T10:42:44.389 INFO:tasks.workunit.client.0.smithi115.stdout:7/496: chown d2/d7/d12/d10/d23/d3d/c22 3 1 2024-03-22T10:42:44.391 INFO:tasks.workunit.client.0.smithi115.stdout:9/518: creat d6/d8/d3c/d59/d68/da8/fb4 x:0 0 0 2024-03-22T10:42:44.392 INFO:tasks.workunit.client.0.smithi115.stdout:7/497: write d2/d4c/f69 [1255608,23934] 0 2024-03-22T10:42:44.394 INFO:tasks.workunit.client.1.smithi150.stdout:5/53: mkdir d0/d8/dc/de/d14 0 2024-03-22T10:42:44.395 INFO:tasks.workunit.client.0.smithi115.stdout:1/576: dwrite d0/d12/d2b/f35 [0,4194304] 0 2024-03-22T10:42:44.396 INFO:tasks.workunit.client.1.smithi150.stdout:5/54: stat d0/d8/dc/de 0 2024-03-22T10:42:44.396 INFO:tasks.workunit.client.0.smithi115.stdout:9/519: write d6/d7/d20/d38/f40 [1067537,60417] 0 2024-03-22T10:42:44.398 INFO:tasks.workunit.client.1.smithi150.stdout:9/98: symlink d1/d19/l21 0 2024-03-22T10:42:44.399 INFO:tasks.workunit.client.0.smithi115.stdout:3/452: creat d3/d89/f92 x:0 0 0 2024-03-22T10:42:44.399 INFO:tasks.workunit.client.0.smithi115.stdout:1/577: chown d0/d12/d17/d37/db5 122 1 2024-03-22T10:42:44.404 INFO:tasks.workunit.client.0.smithi115.stdout:1/578: write d0/d8/d4d/f6e [919224,9703] 0 2024-03-22T10:42:44.404 INFO:tasks.workunit.client.1.smithi150.stdout:0/118: rmdir d0/d1/d7/d21 0 2024-03-22T10:42:44.411 INFO:tasks.workunit.client.1.smithi150.stdout:0/119: mknod d0/d12/c27 0 2024-03-22T10:42:44.411 INFO:tasks.workunit.client.1.smithi150.stdout:9/99: truncate d1/d8/de/df/f1f 337079 0 2024-03-22T10:42:44.411 INFO:tasks.workunit.client.0.smithi115.stdout:1/579: write d0/f87 [3150589,77935] 0 2024-03-22T10:42:44.412 INFO:tasks.workunit.client.0.smithi115.stdout:2/483: mkdir da/d25/d3d/d7e/d90/d96 0 2024-03-22T10:42:44.412 INFO:tasks.workunit.client.0.smithi115.stdout:9/520: write d6/d8/d3c/f54 [3359760,11951] 0 2024-03-22T10:42:44.414 INFO:tasks.workunit.client.1.smithi150.stdout:0/120: dread - d0/d1/d7/ff zero size 2024-03-22T10:42:44.417 INFO:tasks.workunit.client.0.smithi115.stdout:7/498: dread d2/d7/d12/d10/d19/d42/f9c [0,4194304] 0 2024-03-22T10:42:44.421 INFO:tasks.workunit.client.1.smithi150.stdout:0/121: write d0/d1/d7/db/fc [4795051,32152] 0 2024-03-22T10:42:44.421 INFO:tasks.workunit.client.0.smithi115.stdout:1/580: creat d0/d8/db/fc5 x:0 0 0 2024-03-22T10:42:44.424 INFO:tasks.workunit.client.0.smithi115.stdout:1/581: chown d0/d22/f46 772 1 2024-03-22T10:42:44.425 INFO:tasks.workunit.client.1.smithi150.stdout:0/122: chown d0/d1/d7/c17 6 1 2024-03-22T10:42:44.427 INFO:tasks.workunit.client.1.smithi150.stdout:9/100: link d1/d19/l21 d1/d8/de/df/d20/l22 0 2024-03-22T10:42:44.428 INFO:tasks.workunit.client.1.smithi150.stdout:5/55: dwrite d0/f11 [0,4194304] 0 2024-03-22T10:42:44.428 INFO:tasks.workunit.client.0.smithi115.stdout:7/499: dread d2/d7/d12/d10/d23/d3d/f21 [0,4194304] 0 2024-03-22T10:42:44.429 INFO:tasks.workunit.client.0.smithi115.stdout:1/582: creat d0/d22/fc6 x:0 0 0 2024-03-22T10:42:44.431 INFO:tasks.workunit.client.0.smithi115.stdout:2/484: dread da/d25/d51/f55 [0,4194304] 0 2024-03-22T10:42:44.431 INFO:tasks.workunit.client.0.smithi115.stdout:7/500: fsync d2/d3a/d90/f91 0 2024-03-22T10:42:44.434 INFO:tasks.workunit.client.1.smithi150.stdout:0/123: truncate d0/d1/d7/ff 987663 0 2024-03-22T10:42:44.434 INFO:tasks.workunit.client.0.smithi115.stdout:2/485: read da/d1c/f1f [1897476,52794] 0 2024-03-22T10:42:44.435 INFO:tasks.workunit.client.0.smithi115.stdout:2/486: stat c1 0 2024-03-22T10:42:44.435 INFO:tasks.workunit.client.0.smithi115.stdout:9/521: dwrite d6/d8/d3c/d59/f90 [0,4194304] 0 2024-03-22T10:42:44.436 INFO:tasks.workunit.client.0.smithi115.stdout:7/501: chown d2/d7/d12/d10/f16 10354535 1 2024-03-22T10:42:44.437 INFO:tasks.workunit.client.1.smithi150.stdout:0/124: chown d0/d1/d7/db/d10/f13 17800566 1 2024-03-22T10:42:44.440 INFO:tasks.workunit.client.1.smithi150.stdout:0/125: write d0/d1/d7/f20 [4076602,101953] 0 2024-03-22T10:42:44.441 INFO:tasks.workunit.client.1.smithi150.stdout:5/56: link d0/f3 d0/d8/dc/de/f15 0 2024-03-22T10:42:44.442 INFO:tasks.workunit.client.0.smithi115.stdout:3/453: rmdir d3/d36/d41/d33 39 2024-03-22T10:42:44.443 INFO:tasks.workunit.client.0.smithi115.stdout:0/481: dwrite d3/d52/f59 [0,4194304] 0 2024-03-22T10:42:44.443 INFO:tasks.workunit.client.1.smithi150.stdout:1/94: sync 2024-03-22T10:42:44.443 INFO:tasks.workunit.client.1.smithi150.stdout:8/89: sync 2024-03-22T10:42:44.443 INFO:tasks.workunit.client.1.smithi150.stdout:6/69: sync 2024-03-22T10:42:44.445 INFO:tasks.workunit.client.1.smithi150.stdout:0/126: chown d0/d1/d7/d11 167897 1 2024-03-22T10:42:44.446 INFO:tasks.workunit.client.1.smithi150.stdout:9/101: link d1/d8/de/df/d20/l13 d1/l23 0 2024-03-22T10:42:44.446 INFO:tasks.workunit.client.1.smithi150.stdout:1/95: write d1/f7 [3833857,98816] 0 2024-03-22T10:42:44.449 INFO:tasks.workunit.client.0.smithi115.stdout:9/522: getdents d6/d8/daa 0 2024-03-22T10:42:44.450 INFO:tasks.workunit.client.0.smithi115.stdout:0/482: rmdir d3/d7/d1c 39 2024-03-22T10:42:44.452 INFO:tasks.workunit.client.1.smithi150.stdout:6/70: mkdir db/dd/d15 0 2024-03-22T10:42:44.453 INFO:tasks.workunit.client.0.smithi115.stdout:3/454: dread - d3/d36/d41/d33/f8d zero size 2024-03-22T10:42:44.453 INFO:tasks.workunit.client.1.smithi150.stdout:8/90: creat d0/dc/df/d1b/f24 x:0 0 0 2024-03-22T10:42:44.455 INFO:tasks.workunit.client.1.smithi150.stdout:9/102: mkdir d1/d8/de/df/d24 0 2024-03-22T10:42:44.456 INFO:tasks.workunit.client.0.smithi115.stdout:9/523: write d6/d7/f66 [1530212,23513] 0 2024-03-22T10:42:44.460 INFO:tasks.workunit.client.1.smithi150.stdout:8/91: creat d0/d2/d19/f25 x:0 0 0 2024-03-22T10:42:44.466 INFO:tasks.workunit.client.1.smithi150.stdout:5/57: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:42:44.466 INFO:tasks.workunit.client.1.smithi150.stdout:8/92: read d0/d2/fa [3971184,73053] 0 2024-03-22T10:42:44.467 INFO:tasks.workunit.client.0.smithi115.stdout:7/502: dwrite d2/d7/d12/f99 [0,4194304] 0 2024-03-22T10:42:44.470 INFO:tasks.workunit.client.1.smithi150.stdout:9/103: write d1/d8/de/df/f12 [3084276,60626] 0 2024-03-22T10:42:44.475 INFO:tasks.workunit.client.0.smithi115.stdout:9/524: dread d6/d7/d19/f31 [0,4194304] 0 2024-03-22T10:42:44.475 INFO:tasks.workunit.client.0.smithi115.stdout:7/503: write d2/d7/d12/d10/d19/fad [162584,47246] 0 2024-03-22T10:42:44.480 INFO:tasks.workunit.client.1.smithi150.stdout:9/104: mkdir d1/d8/de/df/d25 0 2024-03-22T10:42:44.482 INFO:tasks.workunit.client.0.smithi115.stdout:7/504: stat d2/d7/d12/d10/d19/d42/d46/f5e 0 2024-03-22T10:42:44.482 INFO:tasks.workunit.client.0.smithi115.stdout:3/455: write d3/d16/f58 [686377,45467] 0 2024-03-22T10:42:44.484 INFO:tasks.workunit.client.1.smithi150.stdout:5/58: creat d0/d8/dc/de/d14/f16 x:0 0 0 2024-03-22T10:42:44.485 INFO:tasks.workunit.client.0.smithi115.stdout:3/456: fdatasync d3/d16/d29/f3c 0 2024-03-22T10:42:44.487 INFO:tasks.workunit.client.1.smithi150.stdout:5/59: creat d0/d8/dc/de/d14/f17 x:0 0 0 2024-03-22T10:42:44.488 INFO:tasks.workunit.client.1.smithi150.stdout:5/60: chown d0/d8/dc 6943 1 2024-03-22T10:42:44.489 INFO:tasks.workunit.client.1.smithi150.stdout:5/61: readlink d0/d8/dc/lf 0 2024-03-22T10:42:44.490 INFO:tasks.workunit.client.1.smithi150.stdout:9/105: getdents d1/d8/de 0 2024-03-22T10:42:44.494 INFO:tasks.workunit.client.1.smithi150.stdout:5/62: mkdir d0/d8/dc/de/d18 0 2024-03-22T10:42:44.497 INFO:tasks.workunit.client.0.smithi115.stdout:3/457: symlink d3/d82/d52/l93 0 2024-03-22T10:42:44.498 INFO:tasks.workunit.client.1.smithi150.stdout:9/106: truncate d1/d8/de/f16 196445 0 2024-03-22T10:42:44.499 INFO:tasks.workunit.client.0.smithi115.stdout:2/487: dwrite da/f1d [0,4194304] 0 2024-03-22T10:42:44.500 INFO:tasks.workunit.client.1.smithi150.stdout:1/96: creat d1/da/d15/f1b x:0 0 0 2024-03-22T10:42:44.502 INFO:tasks.workunit.client.0.smithi115.stdout:9/525: dwrite d6/d7/d20/d22/fad [0,4194304] 0 2024-03-22T10:42:44.505 INFO:tasks.workunit.client.0.smithi115.stdout:9/526: fdatasync d6/d7/d19/f25 0 2024-03-22T10:42:44.507 INFO:tasks.workunit.client.1.smithi150.stdout:3/92: getdents d5/da 0 2024-03-22T10:42:44.509 INFO:tasks.workunit.client.1.smithi150.stdout:4/91: write d0/de/f13 [876039,109430] 0 2024-03-22T10:42:44.512 INFO:tasks.workunit.client.0.smithi115.stdout:9/527: readlink d6/d8/d3c/d59/d68/l77 0 2024-03-22T10:42:44.513 INFO:tasks.workunit.client.0.smithi115.stdout:2/488: creat da/d25/d3d/f97 x:0 0 0 2024-03-22T10:42:44.514 INFO:tasks.workunit.client.1.smithi150.stdout:4/92: write d0/de/f13 [521457,79019] 0 2024-03-22T10:42:44.514 INFO:tasks.workunit.client.1.smithi150.stdout:1/97: mknod d1/da/d15/d18/c1c 0 2024-03-22T10:42:44.515 INFO:tasks.workunit.client.0.smithi115.stdout:7/505: dread d2/d7/d12/d10/d23/f84 [0,4194304] 0 2024-03-22T10:42:44.518 INFO:tasks.workunit.client.0.smithi115.stdout:9/528: write d6/d7/d20/d22/fad [4124916,98913] 0 2024-03-22T10:42:44.520 INFO:tasks.workunit.client.1.smithi150.stdout:2/122: truncate ff 954028 0 2024-03-22T10:42:44.520 INFO:tasks.workunit.client.1.smithi150.stdout:5/63: dwrite d0/f11 [0,4194304] 0 2024-03-22T10:42:44.530 INFO:tasks.workunit.client.0.smithi115.stdout:7/506: creat d2/d7/d12/d10/d19/d42/d46/d61/da3/faf x:0 0 0 2024-03-22T10:42:44.531 INFO:tasks.workunit.client.0.smithi115.stdout:3/458: dwrite d3/d89/f8b [0,4194304] 0 2024-03-22T10:42:44.531 INFO:tasks.workunit.client.1.smithi150.stdout:7/89: rename d2/d5/d7/d8/l1c to d2/d5/d7/de/l1d 0 2024-03-22T10:42:44.534 INFO:tasks.workunit.client.0.smithi115.stdout:3/459: write d3/d82/d54/f59 [841263,64515] 0 2024-03-22T10:42:44.534 INFO:tasks.workunit.client.0.smithi115.stdout:7/507: symlink d2/d4c/lb0 0 2024-03-22T10:42:44.535 INFO:tasks.workunit.client.1.smithi150.stdout:5/64: mknod d0/d8/c19 0 2024-03-22T10:42:44.536 INFO:tasks.workunit.client.0.smithi115.stdout:9/529: creat d6/d8/d3c/d59/fb5 x:0 0 0 2024-03-22T10:42:44.538 INFO:tasks.workunit.client.0.smithi115.stdout:7/508: write d2/d62/fa0 [4501,126791] 0 2024-03-22T10:42:44.539 INFO:tasks.workunit.client.1.smithi150.stdout:2/123: link d12/d21/f25 d12/f26 0 2024-03-22T10:42:44.539 INFO:tasks.workunit.client.0.smithi115.stdout:9/530: read - d6/f62 zero size 2024-03-22T10:42:44.540 INFO:tasks.workunit.client.1.smithi150.stdout:3/93: dread d5/f8 [0,4194304] 0 2024-03-22T10:42:44.540 INFO:tasks.workunit.client.1.smithi150.stdout:1/98: dwrite d1/f7 [0,4194304] 0 2024-03-22T10:42:44.542 INFO:tasks.workunit.client.1.smithi150.stdout:5/65: write d0/f11 [1847950,83691] 0 2024-03-22T10:42:44.542 INFO:tasks.workunit.client.1.smithi150.stdout:8/93: rename d0/f1e to d0/dc/df/f26 0 2024-03-22T10:42:44.547 INFO:tasks.workunit.client.1.smithi150.stdout:1/99: stat d1/da/d15/f1b 0 2024-03-22T10:42:44.547 INFO:tasks.workunit.client.1.smithi150.stdout:2/124: write f10 [1051775,65751] 0 2024-03-22T10:42:44.551 INFO:tasks.workunit.client.0.smithi115.stdout:9/531: link d6/d7/d20/d38/f67 d6/d7/d20/d22/fb6 0 2024-03-22T10:42:44.551 INFO:tasks.workunit.client.1.smithi150.stdout:3/94: creat d5/da/f19 x:0 0 0 2024-03-22T10:42:44.552 INFO:tasks.workunit.client.1.smithi150.stdout:8/94: dread - d0/f21 zero size 2024-03-22T10:42:44.557 INFO:tasks.workunit.client.1.smithi150.stdout:8/95: write d0/f1 [3562983,7372] 0 2024-03-22T10:42:44.559 INFO:tasks.workunit.client.1.smithi150.stdout:6/71: getdents db/dd 0 2024-03-22T10:42:44.559 INFO:tasks.workunit.client.0.smithi115.stdout:9/532: creat d6/d8/d3c/d59/d68/d97/da4/fb7 x:0 0 0 2024-03-22T10:42:44.559 INFO:tasks.workunit.client.1.smithi150.stdout:5/66: mknod d0/d8/dc/c1a 0 2024-03-22T10:42:44.561 INFO:tasks.workunit.client.1.smithi150.stdout:4/93: rename d0/c6 to d0/d1b/c1f 0 2024-03-22T10:42:44.563 INFO:tasks.workunit.client.1.smithi150.stdout:0/127: truncate d0/d1/d7/ff 790420 0 2024-03-22T10:42:44.568 INFO:tasks.workunit.client.1.smithi150.stdout:0/128: stat d0/d1/d7/db/fc 0 2024-03-22T10:42:44.571 INFO:tasks.workunit.client.0.smithi115.stdout:3/460: truncate d3/f5b 316530 0 2024-03-22T10:42:44.573 INFO:tasks.workunit.client.1.smithi150.stdout:8/96: mknod d0/c27 0 2024-03-22T10:42:44.575 INFO:tasks.workunit.client.0.smithi115.stdout:3/461: chown d3/d16/d29/d35/l7d 6062 1 2024-03-22T10:42:44.578 INFO:tasks.workunit.client.1.smithi150.stdout:9/107: dread d1/d8/de/f16 [0,4194304] 0 2024-03-22T10:42:44.578 INFO:tasks.workunit.client.1.smithi150.stdout:4/94: link d0/c5 d0/d1b/c20 0 2024-03-22T10:42:44.578 INFO:tasks.workunit.client.0.smithi115.stdout:7/509: dwrite d2/d7/d12/d10/d19/d42/d76/f83 [4194304,4194304] 0 2024-03-22T10:42:44.586 INFO:tasks.workunit.client.1.smithi150.stdout:8/97: fsync d0/d2/f18 0 2024-03-22T10:42:44.588 INFO:tasks.workunit.client.0.smithi115.stdout:9/533: dwrite d6/d8/d3c/d59/fb5 [0,4194304] 0 2024-03-22T10:42:44.590 INFO:tasks.workunit.client.1.smithi150.stdout:7/90: dwrite d2/d5/d7/d9/f18 [0,4194304] 0 2024-03-22T10:42:44.590 INFO:tasks.workunit.client.0.smithi115.stdout:2/489: dread da/d25/d3d/f4e [0,4194304] 0 2024-03-22T10:42:44.594 INFO:tasks.workunit.client.1.smithi150.stdout:9/108: read d1/d8/de/df/f1f [101707,66986] 0 2024-03-22T10:42:44.594 INFO:tasks.workunit.client.1.smithi150.stdout:2/125: dwrite d12/d19/f1d [0,4194304] 0 2024-03-22T10:42:44.595 INFO:tasks.workunit.client.0.smithi115.stdout:2/490: chown da/d25/d2a/c44 593829382 1 2024-03-22T10:42:44.597 INFO:tasks.workunit.client.0.smithi115.stdout:7/510: rename d2/d7/d12/d10/f5b to d2/d7/d12/d10/d19/d42/d76/fb1 0 2024-03-22T10:42:44.598 INFO:tasks.workunit.client.0.smithi115.stdout:3/462: dwrite d3/d82/f8c [0,4194304] 0 2024-03-22T10:42:44.602 INFO:tasks.workunit.client.1.smithi150.stdout:6/72: dwrite db/dd/f14 [0,4194304] 0 2024-03-22T10:42:44.607 INFO:tasks.workunit.client.0.smithi115.stdout:9/534: rename d6/d7/d20/l26 to d6/d7/d48/lb8 0 2024-03-22T10:42:44.609 INFO:tasks.workunit.client.1.smithi150.stdout:4/95: creat d0/d1b/f21 x:0 0 0 2024-03-22T10:42:44.609 INFO:tasks.workunit.client.0.smithi115.stdout:7/511: mknod d2/d7/d12/d10/d19/d42/d46/d61/cb2 0 2024-03-22T10:42:44.611 INFO:tasks.workunit.client.0.smithi115.stdout:2/491: dread da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:44.613 INFO:tasks.workunit.client.0.smithi115.stdout:2/492: readlink da/d1c/l21 0 2024-03-22T10:42:44.613 INFO:tasks.workunit.client.0.smithi115.stdout:7/512: symlink d2/d7/d12/d10/d23/lb3 0 2024-03-22T10:42:44.618 INFO:tasks.workunit.client.1.smithi150.stdout:8/98: mknod d0/d2/c28 0 2024-03-22T10:42:44.619 INFO:tasks.workunit.client.1.smithi150.stdout:3/95: dwrite d5/f8 [0,4194304] 0 2024-03-22T10:42:44.619 INFO:tasks.workunit.client.0.smithi115.stdout:2/493: chown da/d25/d51/l68 11 1 2024-03-22T10:42:44.619 INFO:tasks.workunit.client.0.smithi115.stdout:3/463: dread d3/f50 [0,4194304] 0 2024-03-22T10:42:44.619 INFO:tasks.workunit.client.0.smithi115.stdout:9/535: write d6/d7/d30/d3b/f58 [4487384,124462] 0 2024-03-22T10:42:44.624 INFO:tasks.workunit.client.1.smithi150.stdout:3/96: chown d5/ce 1630564 1 2024-03-22T10:42:44.624 INFO:tasks.workunit.client.1.smithi150.stdout:2/126: dread d12/d19/f1c [4194304,4194304] 0 2024-03-22T10:42:44.625 INFO:tasks.workunit.client.0.smithi115.stdout:7/513: mknod d2/d7/d12/d10/d19/d42/d46/cb4 0 2024-03-22T10:42:44.625 INFO:tasks.workunit.client.0.smithi115.stdout:9/536: readlink d6/d7/d19/d4e/la5 0 2024-03-22T10:42:44.626 INFO:tasks.workunit.client.1.smithi150.stdout:5/67: truncate d0/f3 2924167 0 2024-03-22T10:42:44.627 INFO:tasks.workunit.client.1.smithi150.stdout:9/109: write d1/d8/de/df/f11 [564314,101167] 0 2024-03-22T10:42:44.627 INFO:tasks.workunit.client.1.smithi150.stdout:6/73: symlink db/l16 0 2024-03-22T10:42:44.631 INFO:tasks.workunit.client.1.smithi150.stdout:5/68: read d0/f11 [1404218,18299] 0 2024-03-22T10:42:44.633 INFO:tasks.workunit.client.1.smithi150.stdout:7/91: dwrite d2/d5/d7/d9/f18 [0,4194304] 0 2024-03-22T10:42:44.635 INFO:tasks.workunit.client.0.smithi115.stdout:9/537: symlink d6/d7/d20/d9c/lb9 0 2024-03-22T10:42:44.639 INFO:tasks.workunit.client.0.smithi115.stdout:7/514: creat d2/d7/d12/d10/d19/d42/d46/fb5 x:0 0 0 2024-03-22T10:42:44.639 INFO:tasks.workunit.client.1.smithi150.stdout:9/110: read d1/d8/de/df/f12 [479592,81339] 0 2024-03-22T10:42:44.639 INFO:tasks.workunit.client.1.smithi150.stdout:0/129: getdents d0/d1/d7/db 0 2024-03-22T10:42:44.640 INFO:tasks.workunit.client.1.smithi150.stdout:3/97: mknod d5/da/c1a 0 2024-03-22T10:42:44.641 INFO:tasks.workunit.client.0.smithi115.stdout:3/464: creat d3/d36/d41/f94 x:0 0 0 2024-03-22T10:42:44.642 INFO:tasks.workunit.client.1.smithi150.stdout:4/96: mknod d0/c22 0 2024-03-22T10:42:44.642 INFO:tasks.workunit.client.1.smithi150.stdout:3/98: truncate d5/fd 938369 0 2024-03-22T10:42:44.643 INFO:tasks.workunit.client.0.smithi115.stdout:9/538: mkdir d6/d7/d20/d22/d3f/dba 0 2024-03-22T10:42:44.643 INFO:tasks.workunit.client.1.smithi150.stdout:0/130: stat d0/d1/d7/d11 0 2024-03-22T10:42:44.647 INFO:tasks.workunit.client.1.smithi150.stdout:0/131: write d0/d1/d7/f20 [5003109,13463] 0 2024-03-22T10:42:44.649 INFO:tasks.workunit.client.0.smithi115.stdout:9/539: symlink d6/d8/d3c/d59/d68/d97/lbb 0 2024-03-22T10:42:44.652 INFO:tasks.workunit.client.1.smithi150.stdout:6/74: mknod db/c17 0 2024-03-22T10:42:44.660 INFO:tasks.workunit.client.0.smithi115.stdout:7/515: write d2/d7/d12/d10/d23/f9e [613020,91604] 0 2024-03-22T10:42:44.660 INFO:tasks.workunit.client.1.smithi150.stdout:4/97: creat d0/d1b/f23 x:0 0 0 2024-03-22T10:42:44.663 INFO:tasks.workunit.client.0.smithi115.stdout:7/516: write d2/d3a/f5f [5048285,17188] 0 2024-03-22T10:42:44.664 INFO:tasks.workunit.client.1.smithi150.stdout:8/99: dwrite d0/dc/df/f26 [0,4194304] 0 2024-03-22T10:42:44.664 INFO:tasks.workunit.client.0.smithi115.stdout:3/465: dread d3/f23 [0,4194304] 0 2024-03-22T10:42:44.665 INFO:tasks.workunit.client.0.smithi115.stdout:2/494: dwrite da/f14 [0,4194304] 0 2024-03-22T10:42:44.670 INFO:tasks.workunit.client.1.smithi150.stdout:2/127: mknod d12/d19/d1a/c27 0 2024-03-22T10:42:44.673 INFO:tasks.workunit.client.0.smithi115.stdout:3/466: symlink d3/d6d/l95 0 2024-03-22T10:42:44.673 INFO:tasks.workunit.client.0.smithi115.stdout:9/540: dwrite d6/d7/d30/f9d [0,4194304] 0 2024-03-22T10:42:44.675 INFO:tasks.workunit.client.1.smithi150.stdout:9/111: dwrite d1/d8/de/df/f11 [0,4194304] 0 2024-03-22T10:42:44.675 INFO:tasks.workunit.client.0.smithi115.stdout:7/517: read d2/d7/d12/d10/d19/d42/d46/f52 [529869,8517] 0 2024-03-22T10:42:44.675 INFO:tasks.workunit.client.1.smithi150.stdout:8/100: mkdir d0/d2/d19/d29 0 2024-03-22T10:42:44.678 INFO:tasks.workunit.client.0.smithi115.stdout:3/467: write d3/d6d/f71 [908089,117572] 0 2024-03-22T10:42:44.679 INFO:tasks.workunit.client.0.smithi115.stdout:3/468: stat d3/f8e 0 2024-03-22T10:42:44.682 INFO:tasks.workunit.client.1.smithi150.stdout:5/69: write d0/d8/dc/de/f15 [1970427,120216] 0 2024-03-22T10:42:44.683 INFO:tasks.workunit.client.0.smithi115.stdout:7/518: write d2/f81 [4825098,80964] 0 2024-03-22T10:42:44.684 INFO:tasks.workunit.client.1.smithi150.stdout:3/99: link d5/c12 d5/c1b 0 2024-03-22T10:42:44.684 INFO:tasks.workunit.client.0.smithi115.stdout:2/495: unlink da/d25/d3d/c33 0 2024-03-22T10:42:44.687 INFO:tasks.workunit.client.0.smithi115.stdout:3/469: mknod d3/d82/d52/c96 0 2024-03-22T10:42:44.689 INFO:tasks.workunit.client.1.smithi150.stdout:0/132: read d0/d1/d7/ff [20510,82825] 0 2024-03-22T10:42:44.689 INFO:tasks.workunit.client.1.smithi150.stdout:9/112: creat d1/d19/f26 x:0 0 0 2024-03-22T10:42:44.689 INFO:tasks.workunit.client.0.smithi115.stdout:3/470: dread - d3/d19/d61/f6f zero size 2024-03-22T10:42:44.690 INFO:tasks.workunit.client.1.smithi150.stdout:8/101: mknod d0/dc/c2a 0 2024-03-22T10:42:44.693 INFO:tasks.workunit.client.0.smithi115.stdout:7/519: mknod d2/d3a/cb6 0 2024-03-22T10:42:44.694 INFO:tasks.workunit.client.0.smithi115.stdout:3/471: symlink d3/d82/l97 0 2024-03-22T10:42:44.695 INFO:tasks.workunit.client.0.smithi115.stdout:9/541: link d6/d7/d19/d4e/c82 d6/d7/d48/d70/da9/cbc 0 2024-03-22T10:42:44.695 INFO:tasks.workunit.client.1.smithi150.stdout:6/75: link ca db/dd/c18 0 2024-03-22T10:42:44.697 INFO:tasks.workunit.client.0.smithi115.stdout:3/472: symlink d3/d82/d3a/l98 0 2024-03-22T10:42:44.698 INFO:tasks.workunit.client.1.smithi150.stdout:7/92: getdents d2/d5/d7/d9/db 0 2024-03-22T10:42:44.702 INFO:tasks.workunit.client.0.smithi115.stdout:9/542: mkdir d6/d7/d48/d70/da9/dbd 0 2024-03-22T10:42:44.705 INFO:tasks.workunit.client.1.smithi150.stdout:0/133: creat d0/d1/d7/f28 x:0 0 0 2024-03-22T10:42:44.705 INFO:tasks.workunit.client.1.smithi150.stdout:4/98: dwrite d0/d1b/f21 [0,4194304] 0 2024-03-22T10:42:44.706 INFO:tasks.workunit.client.0.smithi115.stdout:3/473: creat d3/d82/d3a/d7e/f99 x:0 0 0 2024-03-22T10:42:44.707 INFO:tasks.workunit.client.1.smithi150.stdout:7/93: rename d2/d5/d7/d9/db to d2/d5/d7/d9/db/d1e 22 2024-03-22T10:42:44.708 INFO:tasks.workunit.client.0.smithi115.stdout:7/520: dread d2/d7/d12/d10/f25 [12582912,4194304] 0 2024-03-22T10:42:44.710 INFO:tasks.workunit.client.1.smithi150.stdout:7/94: chown d2/d5/d7/d9/db 59 1 2024-03-22T10:42:44.711 INFO:tasks.workunit.client.1.smithi150.stdout:9/113: symlink d1/d19/l27 0 2024-03-22T10:42:44.712 INFO:tasks.workunit.client.0.smithi115.stdout:7/521: creat d2/fb7 x:0 0 0 2024-03-22T10:42:44.714 INFO:tasks.workunit.client.1.smithi150.stdout:4/99: creat d0/d1b/f24 x:0 0 0 2024-03-22T10:42:44.716 INFO:tasks.workunit.client.0.smithi115.stdout:3/474: rename d3/d82/d3a/d7e/f99 to d3/d36/d41/f9a 0 2024-03-22T10:42:44.717 INFO:tasks.workunit.client.0.smithi115.stdout:3/475: stat d3/d82/f27 0 2024-03-22T10:42:44.719 INFO:tasks.workunit.client.0.smithi115.stdout:3/476: chown d3/f5b 173345169 1 2024-03-22T10:42:44.721 INFO:tasks.workunit.client.1.smithi150.stdout:6/76: write db/ff [2388692,108854] 0 2024-03-22T10:42:44.721 INFO:tasks.workunit.client.1.smithi150.stdout:0/134: symlink d0/d1/d7/l29 0 2024-03-22T10:42:44.721 INFO:tasks.workunit.client.1.smithi150.stdout:7/95: write d2/d5/d7/d9/f18 [3364274,5983] 0 2024-03-22T10:42:44.721 INFO:tasks.workunit.client.0.smithi115.stdout:3/477: chown d3/d19/f31 3 1 2024-03-22T10:42:44.723 INFO:tasks.workunit.client.1.smithi150.stdout:4/100: dread d0/f8 [0,4194304] 0 2024-03-22T10:42:44.726 INFO:tasks.workunit.client.1.smithi150.stdout:3/100: dwrite d5/f8 [0,4194304] 0 2024-03-22T10:42:44.731 INFO:tasks.workunit.client.1.smithi150.stdout:9/114: symlink d1/d8/de/df/d24/l28 0 2024-03-22T10:42:44.734 INFO:tasks.workunit.client.1.smithi150.stdout:7/96: read - d2/d5/d7/d9/db/f1a zero size 2024-03-22T10:42:44.735 INFO:tasks.workunit.client.1.smithi150.stdout:3/101: symlink d5/l1c 0 2024-03-22T10:42:44.736 INFO:tasks.workunit.client.1.smithi150.stdout:4/101: mknod d0/d1b/c25 0 2024-03-22T10:42:44.742 INFO:tasks.workunit.client.1.smithi150.stdout:8/102: dwrite d0/dc/df/d1b/f1c [0,4194304] 0 2024-03-22T10:42:44.743 INFO:tasks.workunit.client.0.smithi115.stdout:3/478: dwrite d3/d16/f69 [0,4194304] 0 2024-03-22T10:42:44.744 INFO:tasks.workunit.client.1.smithi150.stdout:0/135: rename d0/d1/cd to d0/d1/d7/d11/d26/c2a 0 2024-03-22T10:42:44.746 INFO:tasks.workunit.client.1.smithi150.stdout:2/128: truncate f10 2289093 0 2024-03-22T10:42:44.748 INFO:tasks.workunit.client.1.smithi150.stdout:7/97: creat d2/f1f x:0 0 0 2024-03-22T10:42:44.749 INFO:tasks.workunit.client.1.smithi150.stdout:3/102: symlink d5/da/l1d 0 2024-03-22T10:42:44.750 INFO:tasks.workunit.client.0.smithi115.stdout:9/543: dread d6/d8/d3c/d59/d75/fa6 [0,4194304] 0 2024-03-22T10:42:44.752 INFO:tasks.workunit.client.1.smithi150.stdout:8/103: unlink d0/dc/df/f1a 0 2024-03-22T10:42:44.752 INFO:tasks.workunit.client.0.smithi115.stdout:3/479: dread - d3/d19/d2b/d49/f60 zero size 2024-03-22T10:42:44.753 INFO:tasks.workunit.client.1.smithi150.stdout:1/100: sync 2024-03-22T10:42:44.757 INFO:tasks.workunit.client.0.smithi115.stdout:3/480: mkdir d3/d16/d29/d9b 0 2024-03-22T10:42:44.759 INFO:tasks.workunit.client.1.smithi150.stdout:6/77: dread db/dd/f14 [0,4194304] 0 2024-03-22T10:42:44.763 INFO:tasks.workunit.client.1.smithi150.stdout:3/103: unlink c3 0 2024-03-22T10:42:44.763 INFO:tasks.workunit.client.1.smithi150.stdout:6/78: write db/fc [620068,68854] 0 2024-03-22T10:42:44.764 INFO:tasks.workunit.client.1.smithi150.stdout:0/136: creat d0/d1/d7/db/f2b x:0 0 0 2024-03-22T10:42:44.764 INFO:tasks.workunit.client.1.smithi150.stdout:4/102: creat d0/f26 x:0 0 0 2024-03-22T10:42:44.764 INFO:tasks.workunit.client.1.smithi150.stdout:5/70: dwrite d0/f11 [0,4194304] 0 2024-03-22T10:42:44.771 INFO:tasks.workunit.client.0.smithi115.stdout:3/481: truncate d3/d19/f4e 4091560 0 2024-03-22T10:42:44.771 INFO:tasks.workunit.client.1.smithi150.stdout:3/104: dread d5/fd [0,4194304] 0 2024-03-22T10:42:44.772 INFO:tasks.workunit.client.1.smithi150.stdout:1/101: creat d1/f1d x:0 0 0 2024-03-22T10:42:44.773 INFO:tasks.workunit.client.1.smithi150.stdout:3/105: write d5/fb [2721711,26482] 0 2024-03-22T10:42:44.773 INFO:tasks.workunit.client.0.smithi115.stdout:3/482: stat d3/d16/d29/f32 0 2024-03-22T10:42:44.775 INFO:tasks.workunit.client.1.smithi150.stdout:3/106: dread - d5/f10 zero size 2024-03-22T10:42:44.777 INFO:tasks.workunit.client.0.smithi115.stdout:3/483: write d3/d6d/f71 [432677,93190] 0 2024-03-22T10:42:44.778 INFO:tasks.workunit.client.1.smithi150.stdout:6/79: symlink db/dd/l19 0 2024-03-22T10:42:44.778 INFO:tasks.workunit.client.1.smithi150.stdout:5/71: mknod d0/d8/dc/de/c1b 0 2024-03-22T10:42:44.778 INFO:tasks.workunit.client.1.smithi150.stdout:9/115: write d1/d8/de/f16 [662427,26093] 0 2024-03-22T10:42:44.780 INFO:tasks.workunit.client.0.smithi115.stdout:3/484: stat d3/d82/d3a/d7e 0 2024-03-22T10:42:44.780 INFO:tasks.workunit.client.0.smithi115.stdout:9/544: dwrite d6/d7/d20/d38/f67 [0,4194304] 0 2024-03-22T10:42:44.783 INFO:tasks.workunit.client.1.smithi150.stdout:5/72: dread - d0/d8/fd zero size 2024-03-22T10:42:44.785 INFO:tasks.workunit.client.1.smithi150.stdout:7/98: link d2/d5/d7/de/l15 d2/d5/d7/de/l20 0 2024-03-22T10:42:44.787 INFO:tasks.workunit.client.1.smithi150.stdout:3/107: truncate d5/da/ff 2089445 0 2024-03-22T10:42:44.788 INFO:tasks.workunit.client.0.smithi115.stdout:9/545: mkdir d6/d7/d30/d37/dbe 0 2024-03-22T10:42:44.790 INFO:tasks.workunit.client.0.smithi115.stdout:3/485: rename d3/d36/d41/d33/c4b to d3/d19/d2b/d55/c9c 0 2024-03-22T10:42:44.791 INFO:tasks.workunit.client.1.smithi150.stdout:8/104: getdents d0/d2/d19 0 2024-03-22T10:42:44.791 INFO:tasks.workunit.client.1.smithi150.stdout:9/116: mkdir d1/d8/de/df/d20/d29 0 2024-03-22T10:42:44.792 INFO:tasks.workunit.client.1.smithi150.stdout:8/105: truncate d0/f5 4725983 0 2024-03-22T10:42:44.793 INFO:tasks.workunit.client.1.smithi150.stdout:9/117: chown d1/d8/de/f17 16 1 2024-03-22T10:42:44.794 INFO:tasks.workunit.client.0.smithi115.stdout:3/486: mknod d3/d16/c9d 0 2024-03-22T10:42:44.796 INFO:tasks.workunit.client.1.smithi150.stdout:4/103: dwrite d0/d1b/f24 [0,4194304] 0 2024-03-22T10:42:44.801 INFO:tasks.workunit.client.0.smithi115.stdout:3/487: creat d3/d6d/f9e x:0 0 0 2024-03-22T10:42:44.803 INFO:tasks.workunit.client.0.smithi115.stdout:3/488: dread - d3/d19/d61/f6f zero size 2024-03-22T10:42:44.808 INFO:tasks.workunit.client.0.smithi115.stdout:9/546: dread d6/d8/d3c/d59/fb5 [0,4194304] 0 2024-03-22T10:42:44.809 INFO:tasks.workunit.client.0.smithi115.stdout:3/489: rmdir d3/d16/d29 39 2024-03-22T10:42:44.811 INFO:tasks.workunit.client.0.smithi115.stdout:3/490: dread - d3/d6d/f9e zero size 2024-03-22T10:42:44.813 INFO:tasks.workunit.client.0.smithi115.stdout:3/491: chown d3/f50 437445 1 2024-03-22T10:42:44.825 INFO:tasks.workunit.client.1.smithi150.stdout:9/118: dread d1/d8/de/f16 [0,4194304] 0 2024-03-22T10:42:44.830 INFO:tasks.workunit.client.1.smithi150.stdout:6/80: truncate db/fc 2710283 0 2024-03-22T10:42:44.833 INFO:tasks.workunit.client.1.smithi150.stdout:4/104: mkdir d0/de/d27 0 2024-03-22T10:42:44.834 INFO:tasks.workunit.client.1.smithi150.stdout:7/99: link d2/d5/d7/d9/l12 d2/l21 0 2024-03-22T10:42:44.837 INFO:tasks.workunit.client.0.smithi115.stdout:3/492: dwrite d3/d36/d41/d33/f7f [0,4194304] 0 2024-03-22T10:42:44.841 INFO:tasks.workunit.client.1.smithi150.stdout:9/119: creat d1/d8/de/df/d25/f2a x:0 0 0 2024-03-22T10:42:44.844 INFO:tasks.workunit.client.1.smithi150.stdout:2/129: dwrite f10 [0,4194304] 0 2024-03-22T10:42:44.846 INFO:tasks.workunit.client.1.smithi150.stdout:8/106: dwrite d0/f17 [0,4194304] 0 2024-03-22T10:42:44.846 INFO:tasks.workunit.client.1.smithi150.stdout:3/108: dwrite d5/fd [0,4194304] 0 2024-03-22T10:42:44.847 INFO:tasks.workunit.client.1.smithi150.stdout:6/81: fdatasync db/fc 0 2024-03-22T10:42:44.847 INFO:tasks.workunit.client.1.smithi150.stdout:4/105: creat d0/de/f28 x:0 0 0 2024-03-22T10:42:44.853 INFO:tasks.workunit.client.1.smithi150.stdout:1/102: dwrite d1/da/fd [4194304,4194304] 0 2024-03-22T10:42:44.855 INFO:tasks.workunit.client.0.smithi115.stdout:9/547: dread d6/d7/d30/d37/f4d [0,4194304] 0 2024-03-22T10:42:44.855 INFO:tasks.workunit.client.1.smithi150.stdout:8/107: fsync d0/dc/df/d1b/f24 0 2024-03-22T10:42:44.861 INFO:tasks.workunit.client.0.smithi115.stdout:9/548: mknod d6/d8/d3c/d59/d68/d97/da4/cbf 0 2024-03-22T10:42:44.864 INFO:tasks.workunit.client.0.smithi115.stdout:9/549: chown d6/d7/d20/d22/f95 36 1 2024-03-22T10:42:44.867 INFO:tasks.workunit.client.1.smithi150.stdout:7/100: creat d2/d5/d7/de/f22 x:0 0 0 2024-03-22T10:42:44.870 INFO:tasks.workunit.client.0.smithi115.stdout:9/550: creat d6/d7/d48/d70/fc0 x:0 0 0 2024-03-22T10:42:44.871 INFO:tasks.workunit.client.1.smithi150.stdout:2/130: rename ff to d12/d16/f28 0 2024-03-22T10:42:44.876 INFO:tasks.workunit.client.1.smithi150.stdout:1/103: unlink d1/da/d15/d18/c1c 0 2024-03-22T10:42:44.877 INFO:tasks.workunit.client.0.smithi115.stdout:9/551: creat d6/d7/d30/d37/dbe/fc1 x:0 0 0 2024-03-22T10:42:44.880 INFO:tasks.workunit.client.1.smithi150.stdout:7/101: read - d2/d5/d7/d9/db/f1a zero size 2024-03-22T10:42:44.881 INFO:tasks.workunit.client.0.smithi115.stdout:6/468: sync 2024-03-22T10:42:44.881 INFO:tasks.workunit.client.0.smithi115.stdout:8/707: sync 2024-03-22T10:42:44.882 INFO:tasks.workunit.client.0.smithi115.stdout:4/467: sync 2024-03-22T10:42:44.882 INFO:tasks.workunit.client.0.smithi115.stdout:1/583: sync 2024-03-22T10:42:44.882 INFO:tasks.workunit.client.0.smithi115.stdout:7/522: sync 2024-03-22T10:42:44.882 INFO:tasks.workunit.client.0.smithi115.stdout:0/483: sync 2024-03-22T10:42:44.882 INFO:tasks.workunit.client.0.smithi115.stdout:5/499: sync 2024-03-22T10:42:44.882 INFO:tasks.workunit.client.0.smithi115.stdout:2/496: sync 2024-03-22T10:42:44.883 INFO:tasks.workunit.client.1.smithi150.stdout:8/108: link d0/d2/f18 d0/d2/f2b 0 2024-03-22T10:42:44.884 INFO:tasks.workunit.client.1.smithi150.stdout:3/109: dwrite d5/fd [0,4194304] 0 2024-03-22T10:42:44.884 INFO:tasks.workunit.client.1.smithi150.stdout:9/120: write d1/d8/de/df/f1f [1186458,9994] 0 2024-03-22T10:42:44.887 INFO:tasks.workunit.client.1.smithi150.stdout:3/110: fdatasync d5/f8 0 2024-03-22T10:42:44.887 INFO:tasks.workunit.client.0.smithi115.stdout:0/484: dread - d3/da/d4c/f91 zero size 2024-03-22T10:42:44.887 INFO:tasks.workunit.client.0.smithi115.stdout:2/497: fdatasync da/d25/d30/f6e 0 2024-03-22T10:42:44.889 INFO:tasks.workunit.client.0.smithi115.stdout:3/493: dread d3/d19/f4e [0,4194304] 0 2024-03-22T10:42:44.891 INFO:tasks.workunit.client.0.smithi115.stdout:7/523: rmdir d2/d3a/d6e/d78 39 2024-03-22T10:42:44.893 INFO:tasks.workunit.client.0.smithi115.stdout:1/584: mkdir d0/d8/de/d1a/d91/dc7 0 2024-03-22T10:42:44.894 INFO:tasks.workunit.client.0.smithi115.stdout:3/494: readlink d3/d19/l2e 0 2024-03-22T10:42:44.894 INFO:tasks.workunit.client.1.smithi150.stdout:7/102: dread - d2/d5/d7/de/f22 zero size 2024-03-22T10:42:44.895 INFO:tasks.workunit.client.1.smithi150.stdout:9/121: readlink d1/d8/de/df/l10 0 2024-03-22T10:42:44.896 INFO:tasks.workunit.client.1.smithi150.stdout:4/106: dwrite d0/f1e [0,4194304] 0 2024-03-22T10:42:44.897 INFO:tasks.workunit.client.1.smithi150.stdout:0/137: dwrite d0/d1/d7/db/fc [0,4194304] 0 2024-03-22T10:42:44.899 INFO:tasks.workunit.client.1.smithi150.stdout:8/109: write d0/d2/f14 [762521,87589] 0 2024-03-22T10:42:44.901 INFO:tasks.workunit.client.1.smithi150.stdout:2/131: rename d12/d16/l23 to d12/l29 0 2024-03-22T10:42:44.905 INFO:tasks.workunit.client.1.smithi150.stdout:6/82: truncate db/dd/f14 3999326 0 2024-03-22T10:42:44.905 INFO:tasks.workunit.client.1.smithi150.stdout:7/103: rmdir d2/d5/d7/d8 39 2024-03-22T10:42:44.906 INFO:tasks.workunit.client.0.smithi115.stdout:5/500: dread d2/f5e [0,4194304] 0 2024-03-22T10:42:44.907 INFO:tasks.workunit.client.0.smithi115.stdout:7/524: creat d2/d7/d12/fb8 x:0 0 0 2024-03-22T10:42:44.908 INFO:tasks.workunit.client.0.smithi115.stdout:2/498: dread - da/d25/d83/f8a zero size 2024-03-22T10:42:44.909 INFO:tasks.workunit.client.0.smithi115.stdout:9/552: dwrite d6/d8/d3c/d59/d68/da8/fb4 [0,4194304] 0 2024-03-22T10:42:44.910 INFO:tasks.workunit.client.1.smithi150.stdout:2/132: creat d12/d16/f2a x:0 0 0 2024-03-22T10:42:44.911 INFO:tasks.workunit.client.0.smithi115.stdout:3/495: getdents d3/d19/d2b/d49/d91 0 2024-03-22T10:42:44.915 INFO:tasks.workunit.client.1.smithi150.stdout:8/110: creat d0/d2/d19/d29/f2c x:0 0 0 2024-03-22T10:42:44.916 INFO:tasks.workunit.client.1.smithi150.stdout:4/107: symlink d0/l29 0 2024-03-22T10:42:44.921 INFO:tasks.workunit.client.0.smithi115.stdout:3/496: chown d3/d19/d2b/d55/f88 237208 1 2024-03-22T10:42:44.923 INFO:tasks.workunit.client.1.smithi150.stdout:1/104: rename d1/da/f14 to d1/d19/d1a/f1e 0 2024-03-22T10:42:44.924 INFO:tasks.workunit.client.0.smithi115.stdout:1/585: rename d0/d8/db/d25/l2e to d0/d12/d17/d37/d9e/lc8 0 2024-03-22T10:42:44.926 INFO:tasks.workunit.client.1.smithi150.stdout:2/133: stat d12/d16 0 2024-03-22T10:42:44.930 INFO:tasks.workunit.client.1.smithi150.stdout:0/138: write d0/d1/d7/db/d10/f13 [4392953,47220] 0 2024-03-22T10:42:44.931 INFO:tasks.workunit.client.0.smithi115.stdout:1/586: write d0/d8/f53 [4219613,75541] 0 2024-03-22T10:42:44.933 INFO:tasks.workunit.client.1.smithi150.stdout:9/122: dwrite d1/d19/f26 [0,4194304] 0 2024-03-22T10:42:44.935 INFO:tasks.workunit.client.1.smithi150.stdout:2/134: dread - d12/f26 zero size 2024-03-22T10:42:44.936 INFO:tasks.workunit.client.1.smithi150.stdout:8/111: mknod d0/c2d 0 2024-03-22T10:42:44.941 INFO:tasks.workunit.client.1.smithi150.stdout:5/73: sync 2024-03-22T10:42:44.944 INFO:tasks.workunit.client.1.smithi150.stdout:1/105: rmdir d1/d19/d1a 39 2024-03-22T10:42:44.944 INFO:tasks.workunit.client.1.smithi150.stdout:3/111: dwrite d5/fd [0,4194304] 0 2024-03-22T10:42:44.948 INFO:tasks.workunit.client.1.smithi150.stdout:9/123: mknod d1/d8/de/df/d25/c2b 0 2024-03-22T10:42:44.949 INFO:tasks.workunit.client.0.smithi115.stdout:1/587: write d0/d12/faa [436079,69882] 0 2024-03-22T10:42:44.950 INFO:tasks.workunit.client.0.smithi115.stdout:5/501: dwrite d2/df/f10 [4194304,4194304] 0 2024-03-22T10:42:44.953 INFO:tasks.workunit.client.1.smithi150.stdout:2/135: creat d12/d19/d1a/f2b x:0 0 0 2024-03-22T10:42:44.956 INFO:tasks.workunit.client.0.smithi115.stdout:3/497: dwrite d3/d82/f83 [0,4194304] 0 2024-03-22T10:42:44.956 INFO:tasks.workunit.client.1.smithi150.stdout:8/112: mkdir d0/d2/d19/d29/d2e 0 2024-03-22T10:42:44.956 INFO:tasks.workunit.client.0.smithi115.stdout:6/469: dread d4/d11/d12/f4e [0,4194304] 0 2024-03-22T10:42:44.957 INFO:tasks.workunit.client.1.smithi150.stdout:6/83: dread db/ff [0,4194304] 0 2024-03-22T10:42:44.957 INFO:tasks.workunit.client.1.smithi150.stdout:1/106: write d1/f7 [2079705,62037] 0 2024-03-22T10:42:44.958 INFO:tasks.workunit.client.0.smithi115.stdout:0/485: dwrite d3/d4/d14/f34 [0,4194304] 0 2024-03-22T10:42:44.959 INFO:tasks.workunit.client.1.smithi150.stdout:0/139: unlink d0/d1/d7/db/c23 0 2024-03-22T10:42:44.959 INFO:tasks.workunit.client.1.smithi150.stdout:3/112: symlink d5/da/l1e 0 2024-03-22T10:42:44.960 INFO:tasks.workunit.client.0.smithi115.stdout:1/588: fsync d0/d22/d75/d82/f9a 0 2024-03-22T10:42:44.961 INFO:tasks.workunit.client.1.smithi150.stdout:5/74: truncate d0/f10 746630 0 2024-03-22T10:42:44.963 INFO:tasks.workunit.client.0.smithi115.stdout:9/553: dread d6/d7/d20/f4c [0,4194304] 0 2024-03-22T10:42:44.964 INFO:tasks.workunit.client.1.smithi150.stdout:9/124: readlink d1/d8/de/df/d20/l5 0 2024-03-22T10:42:44.969 INFO:tasks.workunit.client.1.smithi150.stdout:7/104: dwrite d2/d5/d7/de/f22 [0,4194304] 0 2024-03-22T10:42:44.974 INFO:tasks.workunit.client.0.smithi115.stdout:5/502: symlink d2/df/d27/la7 0 2024-03-22T10:42:44.975 INFO:tasks.workunit.client.0.smithi115.stdout:1/589: mkdir d0/d12/d17/d37/d5c/dc9 0 2024-03-22T10:42:44.975 INFO:tasks.workunit.client.1.smithi150.stdout:1/107: read - d1/f1d zero size 2024-03-22T10:42:44.978 INFO:tasks.workunit.client.1.smithi150.stdout:0/140: mknod d0/d12/c2c 0 2024-03-22T10:42:44.978 INFO:tasks.workunit.client.1.smithi150.stdout:8/113: rename d0/dc to d0/d2/d19/d29/d2f 0 2024-03-22T10:42:44.979 INFO:tasks.workunit.client.0.smithi115.stdout:9/554: creat d6/d7/d48/fc2 x:0 0 0 2024-03-22T10:42:44.979 INFO:tasks.workunit.client.0.smithi115.stdout:7/525: read d2/d7/d12/d10/d23/f6f [432057,130010] 0 2024-03-22T10:42:44.981 INFO:tasks.workunit.client.1.smithi150.stdout:5/75: write d0/f3 [2672602,123287] 0 2024-03-22T10:42:44.982 INFO:tasks.workunit.client.1.smithi150.stdout:9/125: write d1/d8/de/df/f18 [2175003,85811] 0 2024-03-22T10:42:44.986 INFO:tasks.workunit.client.1.smithi150.stdout:1/108: stat d1/da/d15/d18 0 2024-03-22T10:42:44.986 INFO:tasks.workunit.client.1.smithi150.stdout:8/114: dread - d0/f16 zero size 2024-03-22T10:42:44.986 INFO:tasks.workunit.client.1.smithi150.stdout:4/108: dwrite d0/de/f13 [0,4194304] 0 2024-03-22T10:42:44.987 INFO:tasks.workunit.client.1.smithi150.stdout:5/76: fsync d0/d8/fd 0 2024-03-22T10:42:44.990 INFO:tasks.workunit.client.0.smithi115.stdout:3/498: creat d3/d19/d2b/d49/d91/f9f x:0 0 0 2024-03-22T10:42:44.990 INFO:tasks.workunit.client.0.smithi115.stdout:4/468: dread d1/d66/f6d [0,4194304] 0 2024-03-22T10:42:44.991 INFO:tasks.workunit.client.0.smithi115.stdout:2/499: dwrite da/d25/d3d/d57/d7d/f8b [0,4194304] 0 2024-03-22T10:42:44.993 INFO:tasks.workunit.client.1.smithi150.stdout:8/115: write d0/f17 [5085050,5558] 0 2024-03-22T10:42:44.994 INFO:tasks.workunit.client.1.smithi150.stdout:5/77: write d0/f5 [3269749,33126] 0 2024-03-22T10:42:45.000 INFO:tasks.workunit.client.1.smithi150.stdout:7/105: rename d2/l21 to d2/d5/d7/d9/l23 0 2024-03-22T10:42:45.000 INFO:tasks.workunit.client.1.smithi150.stdout:4/109: mknod d0/d1b/c2a 0 2024-03-22T10:42:45.002 INFO:tasks.workunit.client.0.smithi115.stdout:5/503: stat d2/d15/d1b/c34 0 2024-03-22T10:42:45.003 INFO:tasks.workunit.client.1.smithi150.stdout:6/84: mknod db/dd/d15/c1a 0 2024-03-22T10:42:45.007 INFO:tasks.workunit.client.1.smithi150.stdout:0/141: mknod d0/d1/d7/db/c2d 0 2024-03-22T10:42:45.008 INFO:tasks.workunit.client.0.smithi115.stdout:6/470: dwrite d4/d13/f5f [0,4194304] 0 2024-03-22T10:42:45.008 INFO:tasks.workunit.client.0.smithi115.stdout:0/486: creat d3/d7/d1c/d6c/d9a/fa5 x:0 0 0 2024-03-22T10:42:45.010 INFO:tasks.workunit.client.1.smithi150.stdout:1/109: mkdir d1/d19/d1a/d1f 0 2024-03-22T10:42:45.012 INFO:tasks.workunit.client.0.smithi115.stdout:8/708: dread d6/f91 [0,4194304] 0 2024-03-22T10:42:45.015 INFO:tasks.workunit.client.1.smithi150.stdout:9/126: dread d1/d8/de/df/f18 [0,4194304] 0 2024-03-22T10:42:45.016 INFO:tasks.workunit.client.1.smithi150.stdout:7/106: readlink d2/d5/d7/de/l20 0 2024-03-22T10:42:45.022 INFO:tasks.workunit.client.1.smithi150.stdout:3/113: getdents d5 0 2024-03-22T10:42:45.022 INFO:tasks.workunit.client.0.smithi115.stdout:0/487: mkdir d3/d7/d1c/d6c/da6 0 2024-03-22T10:42:45.026 INFO:tasks.workunit.client.1.smithi150.stdout:1/110: mknod d1/da/d15/d18/c20 0 2024-03-22T10:42:45.027 INFO:tasks.workunit.client.1.smithi150.stdout:8/116: rmdir d0/d2/d19/d29 39 2024-03-22T10:42:45.027 INFO:tasks.workunit.client.0.smithi115.stdout:8/709: fsync d6/dd/d24/d4a/d7b/d8f/da0/fc5 0 2024-03-22T10:42:45.027 INFO:tasks.workunit.client.0.smithi115.stdout:7/526: dwrite d2/d7/d12/d10/f16 [4194304,4194304] 0 2024-03-22T10:42:45.030 INFO:tasks.workunit.client.1.smithi150.stdout:8/117: write d0/f1 [1455290,10471] 0 2024-03-22T10:42:45.032 INFO:tasks.workunit.client.1.smithi150.stdout:9/127: creat d1/d8/de/f2c x:0 0 0 2024-03-22T10:42:45.032 INFO:tasks.workunit.client.1.smithi150.stdout:3/114: symlink d5/da/l1f 0 2024-03-22T10:42:45.034 INFO:tasks.workunit.client.0.smithi115.stdout:3/499: mknod d3/d36/d41/ca0 0 2024-03-22T10:42:45.044 INFO:tasks.workunit.client.1.smithi150.stdout:3/115: write d5/da/ff [1842852,39692] 0 2024-03-22T10:42:45.044 INFO:tasks.workunit.client.1.smithi150.stdout:9/128: mknod d1/d8/de/df/d24/c2d 0 2024-03-22T10:42:45.047 INFO:tasks.workunit.client.1.smithi150.stdout:8/118: write d0/d2/d19/f20 [3913303,129332] 0 2024-03-22T10:42:45.054 INFO:tasks.workunit.client.1.smithi150.stdout:3/116: unlink d5/l1c 0 2024-03-22T10:42:45.054 INFO:tasks.workunit.client.0.smithi115.stdout:8/710: rename d6/dd/d12/d31/c6e to d6/dd/d12/d31/ce6 0 2024-03-22T10:42:45.057 INFO:tasks.workunit.client.0.smithi115.stdout:9/555: rename d6/d7/d20/d22/d3f/c93 to d6/d7/d20/d22/cc3 0 2024-03-22T10:42:45.059 INFO:tasks.workunit.client.0.smithi115.stdout:2/500: write da/d25/d30/f43 [765425,57774] 0 2024-03-22T10:42:45.060 INFO:tasks.workunit.client.1.smithi150.stdout:2/136: dwrite d12/d16/f28 [0,4194304] 0 2024-03-22T10:42:45.063 INFO:tasks.workunit.client.0.smithi115.stdout:5/504: write d2/df/f55 [68345,21205] 0 2024-03-22T10:42:45.065 INFO:tasks.workunit.client.0.smithi115.stdout:0/488: dwrite d3/f46 [0,4194304] 0 2024-03-22T10:42:45.065 INFO:tasks.workunit.client.0.smithi115.stdout:7/527: read d2/d7/d12/d10/d19/d42/d76/d74/f77 [1739,72715] 0 2024-03-22T10:42:45.066 INFO:tasks.workunit.client.0.smithi115.stdout:4/469: getdents d1/d16/d35/d40/d5c 0 2024-03-22T10:42:45.078 INFO:tasks.workunit.client.0.smithi115.stdout:3/500: rename d3/d19/d2b/c6a to d3/d16/d29/d35/ca1 0 2024-03-22T10:42:45.079 INFO:tasks.workunit.client.0.smithi115.stdout:6/471: dread d4/d11/d12/d1c/f7f [0,4194304] 0 2024-03-22T10:42:45.082 INFO:tasks.workunit.client.1.smithi150.stdout:6/85: dwrite db/dd/f14 [4194304,4194304] 0 2024-03-22T10:42:45.082 INFO:tasks.workunit.client.0.smithi115.stdout:1/590: dread d0/d68/fae [0,4194304] 0 2024-03-22T10:42:45.082 INFO:tasks.workunit.client.0.smithi115.stdout:3/501: write d3/d36/f5c [493276,82732] 0 2024-03-22T10:42:45.085 INFO:tasks.workunit.client.0.smithi115.stdout:2/501: symlink da/d1c/d67/d78/l98 0 2024-03-22T10:42:45.085 INFO:tasks.workunit.client.0.smithi115.stdout:4/470: creat d1/d16/d68/f9e x:0 0 0 2024-03-22T10:42:45.086 INFO:tasks.workunit.client.0.smithi115.stdout:3/502: write d3/d89/f8b [2486369,38880] 0 2024-03-22T10:42:45.088 INFO:tasks.workunit.client.0.smithi115.stdout:9/556: creat d6/d7/d20/d22/d23/fc4 x:0 0 0 2024-03-22T10:42:45.089 INFO:tasks.workunit.client.1.smithi150.stdout:7/107: fsync d2/d5/d7/de/f22 0 2024-03-22T10:42:45.092 INFO:tasks.workunit.client.0.smithi115.stdout:5/505: rmdir d2/d15/d1b/d7c/d38/d4a/d74 39 2024-03-22T10:42:45.094 INFO:tasks.workunit.client.1.smithi150.stdout:8/119: dread d0/d2/f18 [0,4194304] 0 2024-03-22T10:42:45.094 INFO:tasks.workunit.client.0.smithi115.stdout:8/711: creat d6/dd/d12/de3/fe7 x:0 0 0 2024-03-22T10:42:45.094 INFO:tasks.workunit.client.0.smithi115.stdout:1/591: dread d0/d12/f1c [4194304,4194304] 0 2024-03-22T10:42:45.095 INFO:tasks.workunit.client.1.smithi150.stdout:6/86: fsync db/ff 0 2024-03-22T10:42:45.095 INFO:tasks.workunit.client.1.smithi150.stdout:9/129: dwrite d1/d19/f26 [0,4194304] 0 2024-03-22T10:42:45.097 INFO:tasks.workunit.client.0.smithi115.stdout:5/506: write d2/df/f10 [2651193,96344] 0 2024-03-22T10:42:45.098 INFO:tasks.workunit.client.0.smithi115.stdout:1/592: write d0/d22/fc6 [807733,49534] 0 2024-03-22T10:42:45.100 INFO:tasks.workunit.client.1.smithi150.stdout:6/87: readlink db/l16 0 2024-03-22T10:42:45.105 INFO:tasks.workunit.client.0.smithi115.stdout:2/502: unlink da/c1a 0 2024-03-22T10:42:45.106 INFO:tasks.workunit.client.1.smithi150.stdout:3/117: dwrite d5/da/f19 [0,4194304] 0 2024-03-22T10:42:45.106 INFO:tasks.workunit.client.0.smithi115.stdout:1/593: truncate d0/d8/db/d25/f73 567229 0 2024-03-22T10:42:45.108 INFO:tasks.workunit.client.0.smithi115.stdout:2/503: readlink da/d25/l74 0 2024-03-22T10:42:45.113 INFO:tasks.workunit.client.1.smithi150.stdout:9/130: mkdir d1/d8/de/df/d25/d2e 0 2024-03-22T10:42:45.113 INFO:tasks.workunit.client.1.smithi150.stdout:8/120: fdatasync d0/d2/d19/f20 0 2024-03-22T10:42:45.114 INFO:tasks.workunit.client.0.smithi115.stdout:2/504: write da/d25/d3d/f97 [856299,126978] 0 2024-03-22T10:42:45.114 INFO:tasks.workunit.client.0.smithi115.stdout:2/505: stat da/f14 0 2024-03-22T10:42:45.116 INFO:tasks.workunit.client.1.smithi150.stdout:6/88: creat db/dd/f1b x:0 0 0 2024-03-22T10:42:45.119 INFO:tasks.workunit.client.0.smithi115.stdout:5/507: creat d2/df/d13/d8b/d93/da6/fa8 x:0 0 0 2024-03-22T10:42:45.122 INFO:tasks.workunit.client.1.smithi150.stdout:9/131: creat d1/d19/f2f x:0 0 0 2024-03-22T10:42:45.124 INFO:tasks.workunit.client.1.smithi150.stdout:2/137: dwrite d12/d21/f25 [0,4194304] 0 2024-03-22T10:42:45.125 INFO:tasks.workunit.client.0.smithi115.stdout:8/712: creat d6/dd/d24/d4a/d41/dc4/fe8 x:0 0 0 2024-03-22T10:42:45.127 INFO:tasks.workunit.client.1.smithi150.stdout:1/111: dwrite d1/da/f16 [0,4194304] 0 2024-03-22T10:42:45.127 INFO:tasks.workunit.client.0.smithi115.stdout:2/506: read da/d25/d30/d4f/f8d [3951834,25561] 0 2024-03-22T10:42:45.127 INFO:tasks.workunit.client.0.smithi115.stdout:7/528: getdents d2/d7/d12/d10/d19 0 2024-03-22T10:42:45.127 INFO:tasks.workunit.client.0.smithi115.stdout:3/503: rename d3/d36/d41/d33/f7f to d3/d36/d41/fa2 0 2024-03-22T10:42:45.129 INFO:tasks.workunit.client.0.smithi115.stdout:6/472: getdents d4/d36 0 2024-03-22T10:42:45.130 INFO:tasks.workunit.client.1.smithi150.stdout:9/132: chown d1/d19/f2f 429 1 2024-03-22T10:42:45.132 INFO:tasks.workunit.client.1.smithi150.stdout:6/89: mknod db/dd/d15/c1c 0 2024-03-22T10:42:45.135 INFO:tasks.workunit.client.0.smithi115.stdout:5/508: rmdir d2/d15/d1b/d7c/d38/d56 39 2024-03-22T10:42:45.136 INFO:tasks.workunit.client.1.smithi150.stdout:6/90: readlink db/dd/l19 0 2024-03-22T10:42:45.137 INFO:tasks.workunit.client.1.smithi150.stdout:7/108: dread d2/d5/d7/d9/f18 [0,4194304] 0 2024-03-22T10:42:45.137 INFO:tasks.workunit.client.1.smithi150.stdout:1/112: creat d1/da/d15/f21 x:0 0 0 2024-03-22T10:42:45.137 INFO:tasks.workunit.client.0.smithi115.stdout:8/713: rename d6/f17 to d6/d49/d88/fe9 0 2024-03-22T10:42:45.138 INFO:tasks.workunit.client.1.smithi150.stdout:8/121: creat d0/d2/d19/d29/d2f/df/d1b/f30 x:0 0 0 2024-03-22T10:42:45.138 INFO:tasks.workunit.client.0.smithi115.stdout:1/594: creat d0/d12/d2b/fca x:0 0 0 2024-03-22T10:42:45.139 INFO:tasks.workunit.client.1.smithi150.stdout:4/110: sync 2024-03-22T10:42:45.140 INFO:tasks.workunit.client.1.smithi150.stdout:3/118: link d5/da/c1a d5/c20 0 2024-03-22T10:42:45.140 INFO:tasks.workunit.client.1.smithi150.stdout:6/91: write db/dd/f14 [4691650,10612] 0 2024-03-22T10:42:45.140 INFO:tasks.workunit.client.0.smithi115.stdout:1/595: fdatasync d0/d12/d17/f7d 0 2024-03-22T10:42:45.141 INFO:tasks.workunit.client.0.smithi115.stdout:7/529: symlink d2/d3a/d90/dae/lb9 0 2024-03-22T10:42:45.142 INFO:tasks.workunit.client.1.smithi150.stdout:4/111: stat d0/f1e 0 2024-03-22T10:42:45.143 INFO:tasks.workunit.client.0.smithi115.stdout:3/504: creat d3/d19/d2b/d49/d91/fa3 x:0 0 0 2024-03-22T10:42:45.144 INFO:tasks.workunit.client.1.smithi150.stdout:3/119: write d5/f8 [1131284,72215] 0 2024-03-22T10:42:45.146 INFO:tasks.workunit.client.1.smithi150.stdout:8/122: write d0/d2/d19/d29/d2f/df/d1b/f1d [968037,75390] 0 2024-03-22T10:42:45.149 INFO:tasks.workunit.client.1.smithi150.stdout:1/113: mknod d1/d19/d1a/c22 0 2024-03-22T10:42:45.149 INFO:tasks.workunit.client.0.smithi115.stdout:3/505: truncate d3/d19/d2b/d49/d91/fa3 441208 0 2024-03-22T10:42:45.149 INFO:tasks.workunit.client.1.smithi150.stdout:6/92: symlink db/l1d 0 2024-03-22T10:42:45.151 INFO:tasks.workunit.client.1.smithi150.stdout:7/109: symlink d2/d5/d7/d8/l24 0 2024-03-22T10:42:45.151 INFO:tasks.workunit.client.1.smithi150.stdout:1/114: fsync d1/da/fd 0 2024-03-22T10:42:45.153 INFO:tasks.workunit.client.1.smithi150.stdout:3/120: creat d5/f21 x:0 0 0 2024-03-22T10:42:45.154 INFO:tasks.workunit.client.1.smithi150.stdout:2/138: dread d12/f26 [0,4194304] 0 2024-03-22T10:42:45.154 INFO:tasks.workunit.client.0.smithi115.stdout:8/714: dread - d6/dd/d12/d31/f90 zero size 2024-03-22T10:42:45.155 INFO:tasks.workunit.client.1.smithi150.stdout:3/121: readlink d5/da/l18 0 2024-03-22T10:42:45.159 INFO:tasks.workunit.client.1.smithi150.stdout:1/115: write d1/da/f13 [805862,65885] 0 2024-03-22T10:42:45.160 INFO:tasks.workunit.client.1.smithi150.stdout:2/139: chown d12/d19/f1d 92 1 2024-03-22T10:42:45.162 INFO:tasks.workunit.client.0.smithi115.stdout:3/506: creat d3/d82/fa4 x:0 0 0 2024-03-22T10:42:45.165 INFO:tasks.workunit.client.0.smithi115.stdout:6/473: getdents d4/d11/d12 0 2024-03-22T10:42:45.166 INFO:tasks.workunit.client.1.smithi150.stdout:2/140: write d12/d16/f22 [862248,105998] 0 2024-03-22T10:42:45.170 INFO:tasks.workunit.client.1.smithi150.stdout:6/93: link ca db/dd/c1e 0 2024-03-22T10:42:45.170 INFO:tasks.workunit.client.0.smithi115.stdout:5/509: creat d2/d15/d1b/d7c/d38/d43/d49/fa9 x:0 0 0 2024-03-22T10:42:45.172 INFO:tasks.workunit.client.0.smithi115.stdout:6/474: write d4/d36/d48/f67 [3654986,14371] 0 2024-03-22T10:42:45.173 INFO:tasks.workunit.client.1.smithi150.stdout:9/133: write d1/d8/de/df/f12 [2508008,10664] 0 2024-03-22T10:42:45.173 INFO:tasks.workunit.client.1.smithi150.stdout:4/112: dwrite d0/f14 [0,4194304] 0 2024-03-22T10:42:45.174 INFO:tasks.workunit.client.0.smithi115.stdout:4/471: dread d1/d16/d35/d40/d5c/d55/f52 [0,4194304] 0 2024-03-22T10:42:45.174 INFO:tasks.workunit.client.0.smithi115.stdout:2/507: dwrite da/f1d [0,4194304] 0 2024-03-22T10:42:45.180 INFO:tasks.workunit.client.0.smithi115.stdout:0/489: truncate d3/d7/d1b/d35/f42 654920 0 2024-03-22T10:42:45.183 INFO:tasks.workunit.client.1.smithi150.stdout:6/94: mknod db/dd/d15/c1f 0 2024-03-22T10:42:45.184 INFO:tasks.workunit.client.0.smithi115.stdout:5/510: write d2/df/d13/f18 [3978195,43035] 0 2024-03-22T10:42:45.185 INFO:tasks.workunit.client.0.smithi115.stdout:7/530: dwrite d2/d7/d12/d10/d19/fad [0,4194304] 0 2024-03-22T10:42:45.188 INFO:tasks.workunit.client.1.smithi150.stdout:4/113: getdents d0/de/d27 0 2024-03-22T10:42:45.189 INFO:tasks.workunit.client.1.smithi150.stdout:8/123: dwrite d0/d2/d19/d29/d2f/df/d1b/f1c [0,4194304] 0 2024-03-22T10:42:45.190 INFO:tasks.workunit.client.1.smithi150.stdout:7/110: dread d2/d5/d7/de/f22 [0,4194304] 0 2024-03-22T10:42:45.191 INFO:tasks.workunit.client.0.smithi115.stdout:8/715: link d6/dd/fe4 d6/dd/d12/fea 0 2024-03-22T10:42:45.196 INFO:tasks.workunit.client.0.smithi115.stdout:8/716: readlink l3 0 2024-03-22T10:42:45.196 INFO:tasks.workunit.client.0.smithi115.stdout:2/508: creat da/d25/d3d/d7e/f99 x:0 0 0 2024-03-22T10:42:45.198 INFO:tasks.workunit.client.1.smithi150.stdout:6/95: creat db/dd/d15/f20 x:0 0 0 2024-03-22T10:42:45.199 INFO:tasks.workunit.client.1.smithi150.stdout:8/124: dread - d0/d2/d19/f25 zero size 2024-03-22T10:42:45.202 INFO:tasks.workunit.client.0.smithi115.stdout:8/717: stat d6/dd/d12/dc6 0 2024-03-22T10:42:45.203 INFO:tasks.workunit.client.1.smithi150.stdout:3/122: dwrite d5/f8 [4194304,4194304] 0 2024-03-22T10:42:45.204 INFO:tasks.workunit.client.1.smithi150.stdout:7/111: creat d2/f25 x:0 0 0 2024-03-22T10:42:45.205 INFO:tasks.workunit.client.1.smithi150.stdout:4/114: getdents d0/de/d27 0 2024-03-22T10:42:45.206 INFO:tasks.workunit.client.1.smithi150.stdout:8/125: chown d0/f1 5500 1 2024-03-22T10:42:45.206 INFO:tasks.workunit.client.0.smithi115.stdout:7/531: creat d2/d7/d12/d10/d19/d42/d46/d61/fba x:0 0 0 2024-03-22T10:42:45.207 INFO:tasks.workunit.client.1.smithi150.stdout:2/141: dwrite f11 [4194304,4194304] 0 2024-03-22T10:42:45.210 INFO:tasks.workunit.client.0.smithi115.stdout:2/509: truncate da/d25/d3d/f97 1783470 0 2024-03-22T10:42:45.210 INFO:tasks.workunit.client.0.smithi115.stdout:8/718: truncate d6/fad 1671699 0 2024-03-22T10:42:45.210 INFO:tasks.workunit.client.1.smithi150.stdout:2/142: fdatasync d12/f15 0 2024-03-22T10:42:45.210 INFO:tasks.workunit.client.1.smithi150.stdout:1/116: dwrite d1/da/fe [0,4194304] 0 2024-03-22T10:42:45.213 INFO:tasks.workunit.client.1.smithi150.stdout:7/112: creat d2/d5/d7/de/f26 x:0 0 0 2024-03-22T10:42:45.213 INFO:tasks.workunit.client.1.smithi150.stdout:2/143: chown d12/d19/d1a/c27 1406 1 2024-03-22T10:42:45.213 INFO:tasks.workunit.client.1.smithi150.stdout:4/115: link d0/f1a d0/d1b/f2b 0 2024-03-22T10:42:45.214 INFO:tasks.workunit.client.1.smithi150.stdout:1/117: write d1/f1d [232764,128371] 0 2024-03-22T10:42:45.217 INFO:tasks.workunit.client.1.smithi150.stdout:1/118: chown d1/c8 6893 1 2024-03-22T10:42:45.218 INFO:tasks.workunit.client.1.smithi150.stdout:2/144: write d12/d19/d1a/f2b [1039683,34614] 0 2024-03-22T10:42:45.220 INFO:tasks.workunit.client.1.smithi150.stdout:7/113: mknod d2/d5/c27 0 2024-03-22T10:42:45.221 INFO:tasks.workunit.client.0.smithi115.stdout:5/511: mknod d2/d15/d1b/d7c/d38/d43/d49/d81/d83/caa 0 2024-03-22T10:42:45.223 INFO:tasks.workunit.client.0.smithi115.stdout:3/507: dwrite d3/d36/f5c [0,4194304] 0 2024-03-22T10:42:45.224 INFO:tasks.workunit.client.1.smithi150.stdout:4/116: link d0/f14 d0/d1b/f2c 0 2024-03-22T10:42:45.225 INFO:tasks.workunit.client.1.smithi150.stdout:2/145: read - d12/d16/f2a zero size 2024-03-22T10:42:45.226 INFO:tasks.workunit.client.0.smithi115.stdout:2/510: write da/d25/d3d/f28 [1546970,111204] 0 2024-03-22T10:42:45.229 INFO:tasks.workunit.client.1.smithi150.stdout:4/117: mknod d0/c2d 0 2024-03-22T10:42:45.230 INFO:tasks.workunit.client.1.smithi150.stdout:2/146: write d12/f15 [266598,34751] 0 2024-03-22T10:42:45.231 INFO:tasks.workunit.client.0.smithi115.stdout:1/596: dread d0/f55 [0,4194304] 0 2024-03-22T10:42:45.232 INFO:tasks.workunit.client.1.smithi150.stdout:6/96: dwrite db/ff [0,4194304] 0 2024-03-22T10:42:45.232 INFO:tasks.workunit.client.0.smithi115.stdout:0/490: dwrite d3/d7/f5c [0,4194304] 0 2024-03-22T10:42:45.235 INFO:tasks.workunit.client.0.smithi115.stdout:0/491: stat d3/da/d4c/l9d 0 2024-03-22T10:42:45.238 INFO:tasks.workunit.client.1.smithi150.stdout:7/114: rename d2/d5/d7/de/f26 to d2/d5/d7/d9/f28 0 2024-03-22T10:42:45.239 INFO:tasks.workunit.client.1.smithi150.stdout:2/147: mkdir d12/d16/d2c 0 2024-03-22T10:42:45.241 INFO:tasks.workunit.client.0.smithi115.stdout:0/492: truncate d3/d4/d14/d64/f72 5240004 0 2024-03-22T10:42:45.242 INFO:tasks.workunit.client.1.smithi150.stdout:1/119: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:45.243 INFO:tasks.workunit.client.0.smithi115.stdout:0/493: chown d3/d7/d1c/d6c/da6 2321 1 2024-03-22T10:42:45.250 INFO:tasks.workunit.client.1.smithi150.stdout:1/120: rename d1/c8 to d1/d19/d1a/c23 0 2024-03-22T10:42:45.253 INFO:tasks.workunit.client.1.smithi150.stdout:4/118: dread d0/f1e [0,4194304] 0 2024-03-22T10:42:45.255 INFO:tasks.workunit.client.1.smithi150.stdout:1/121: write d1/f7 [4347495,71161] 0 2024-03-22T10:42:45.261 INFO:tasks.workunit.client.1.smithi150.stdout:4/119: rename d0/de/f13 to d0/d1b/f2e 0 2024-03-22T10:42:45.262 INFO:tasks.workunit.client.0.smithi115.stdout:8/719: dwrite d6/d49/f84 [0,4194304] 0 2024-03-22T10:42:45.262 INFO:tasks.workunit.client.1.smithi150.stdout:1/122: symlink d1/d19/d1a/l24 0 2024-03-22T10:42:45.263 INFO:tasks.workunit.client.1.smithi150.stdout:2/148: dread d12/d19/f1c [0,4194304] 0 2024-03-22T10:42:45.264 INFO:tasks.workunit.client.1.smithi150.stdout:6/97: dwrite db/dd/f1b [0,4194304] 0 2024-03-22T10:42:45.265 INFO:tasks.workunit.client.0.smithi115.stdout:1/597: dwrite d0/d8/d4d/d56/f6d [0,4194304] 0 2024-03-22T10:42:45.267 INFO:tasks.workunit.client.0.smithi115.stdout:6/475: dwrite d4/d11/f54 [0,4194304] 0 2024-03-22T10:42:45.271 INFO:tasks.workunit.client.1.smithi150.stdout:2/149: stat d12/d19/l24 0 2024-03-22T10:42:45.272 INFO:tasks.workunit.client.0.smithi115.stdout:7/532: rename d2/d4c to d2/d3a/d90/dae/dbb 0 2024-03-22T10:42:45.276 INFO:tasks.workunit.client.0.smithi115.stdout:3/508: chown d3/d16/c7b 67099770 1 2024-03-22T10:42:45.277 INFO:tasks.workunit.client.1.smithi150.stdout:2/150: mkdir d12/d19/d2d 0 2024-03-22T10:42:45.278 INFO:tasks.workunit.client.0.smithi115.stdout:5/512: write d2/d15/d1b/d7c/d38/d56/f87 [218237,3605] 0 2024-03-22T10:42:45.278 INFO:tasks.workunit.client.0.smithi115.stdout:0/494: rmdir d3/da/d4c/da3 39 2024-03-22T10:42:45.282 INFO:tasks.workunit.client.1.smithi150.stdout:7/115: getdents d2 0 2024-03-22T10:42:45.286 INFO:tasks.workunit.client.1.smithi150.stdout:2/151: unlink d12/d21/f25 0 2024-03-22T10:42:45.287 INFO:tasks.workunit.client.0.smithi115.stdout:8/720: mkdir d6/d49/d68/deb 0 2024-03-22T10:42:45.287 INFO:tasks.workunit.client.0.smithi115.stdout:6/476: write d4/d11/f23 [2464574,69901] 0 2024-03-22T10:42:45.289 INFO:tasks.workunit.client.1.smithi150.stdout:1/123: getdents d1/da/d15 0 2024-03-22T10:42:45.291 INFO:tasks.workunit.client.1.smithi150.stdout:1/124: fsync d1/da/f16 0 2024-03-22T10:42:45.292 INFO:tasks.workunit.client.0.smithi115.stdout:5/513: symlink d2/df/d27/d50/d96/lab 0 2024-03-22T10:42:45.295 INFO:tasks.workunit.client.0.smithi115.stdout:5/514: chown d2/d15/c2f 2 1 2024-03-22T10:42:45.298 INFO:tasks.workunit.client.0.smithi115.stdout:6/477: rmdir d4/d5e 39 2024-03-22T10:42:45.298 INFO:tasks.workunit.client.1.smithi150.stdout:5/78: sync 2024-03-22T10:42:45.299 INFO:tasks.workunit.client.1.smithi150.stdout:0/142: sync 2024-03-22T10:42:45.302 INFO:tasks.workunit.client.0.smithi115.stdout:4/472: dread d1/d16/d35/d40/d5c/f6f [0,4194304] 0 2024-03-22T10:42:45.304 INFO:tasks.workunit.client.0.smithi115.stdout:8/721: mknod d6/dd/d24/d4a/d41/dc4/cec 0 2024-03-22T10:42:45.306 INFO:tasks.workunit.client.0.smithi115.stdout:9/557: sync 2024-03-22T10:42:45.307 INFO:tasks.workunit.client.0.smithi115.stdout:1/598: creat d0/d12/d2b/fcb x:0 0 0 2024-03-22T10:42:45.307 INFO:tasks.workunit.client.1.smithi150.stdout:7/116: getdents d2/d5/d7/d9/db 0 2024-03-22T10:42:45.308 INFO:tasks.workunit.client.0.smithi115.stdout:0/495: truncate d3/d7/d1c/f5d 545256 0 2024-03-22T10:42:45.310 INFO:tasks.workunit.client.1.smithi150.stdout:2/152: truncate d12/f13 4107212 0 2024-03-22T10:42:45.310 INFO:tasks.workunit.client.1.smithi150.stdout:4/120: write d0/de/f12 [435482,41395] 0 2024-03-22T10:42:45.314 INFO:tasks.workunit.client.1.smithi150.stdout:2/153: stat d12/d16/l18 0 2024-03-22T10:42:45.314 INFO:tasks.workunit.client.1.smithi150.stdout:7/117: unlink d2/f25 0 2024-03-22T10:42:45.314 INFO:tasks.workunit.client.0.smithi115.stdout:0/496: write d3/d52/f57 [4337719,44573] 0 2024-03-22T10:42:45.316 INFO:tasks.workunit.client.1.smithi150.stdout:4/121: dread d0/f9 [0,4194304] 0 2024-03-22T10:42:45.316 INFO:tasks.workunit.client.0.smithi115.stdout:0/497: chown d3/d52 6121940 1 2024-03-22T10:42:45.317 INFO:tasks.workunit.client.0.smithi115.stdout:3/509: dwrite d3/d36/d41/f48 [0,4194304] 0 2024-03-22T10:42:45.319 INFO:tasks.workunit.client.0.smithi115.stdout:5/515: rename d2/d15/f39 to d2/d15/fac 0 2024-03-22T10:42:45.320 INFO:tasks.workunit.client.0.smithi115.stdout:7/533: dwrite d2/d7/d12/d10/d19/d42/f9c [0,4194304] 0 2024-03-22T10:42:45.323 INFO:tasks.workunit.client.1.smithi150.stdout:0/143: link d0/d1/d7/db/d10/f13 d0/d1/d7/d11/d26/f2e 0 2024-03-22T10:42:45.326 INFO:tasks.workunit.client.1.smithi150.stdout:4/122: write d0/d1b/f24 [1071878,31401] 0 2024-03-22T10:42:45.328 INFO:tasks.workunit.client.0.smithi115.stdout:4/473: truncate d1/d16/d35/d40/d54/f67 812135 0 2024-03-22T10:42:45.329 INFO:tasks.workunit.client.1.smithi150.stdout:5/79: dwrite d0/d8/fd [0,4194304] 0 2024-03-22T10:42:45.329 INFO:tasks.workunit.client.1.smithi150.stdout:6/98: truncate db/ff 1305846 0 2024-03-22T10:42:45.329 INFO:tasks.workunit.client.0.smithi115.stdout:2/511: dread da/d1c/f1e [0,4194304] 0 2024-03-22T10:42:45.330 INFO:tasks.workunit.client.1.smithi150.stdout:7/118: truncate d2/d5/d7/d9/f28 676899 0 2024-03-22T10:42:45.330 INFO:tasks.workunit.client.1.smithi150.stdout:4/123: readlink d0/l29 0 2024-03-22T10:42:45.332 INFO:tasks.workunit.client.1.smithi150.stdout:6/99: mkdir db/dd/d21 0 2024-03-22T10:42:45.333 INFO:tasks.workunit.client.0.smithi115.stdout:7/534: truncate d2/d7/d12/d10/d19/d42/d46/d61/fba 621304 0 2024-03-22T10:42:45.333 INFO:tasks.workunit.client.1.smithi150.stdout:0/144: readlink d0/d1/d7/db/d10/l1a 0 2024-03-22T10:42:45.334 INFO:tasks.workunit.client.0.smithi115.stdout:4/474: chown d1/d30/d42/f34 7138084 1 2024-03-22T10:42:45.336 INFO:tasks.workunit.client.0.smithi115.stdout:9/558: mknod d6/d8/d3c/d59/d75/cc5 0 2024-03-22T10:42:45.337 INFO:tasks.workunit.client.1.smithi150.stdout:2/154: dread f10 [0,4194304] 0 2024-03-22T10:42:45.344 INFO:tasks.workunit.client.1.smithi150.stdout:1/125: dwrite d1/d19/d1a/f1e [0,4194304] 0 2024-03-22T10:42:45.344 INFO:tasks.workunit.client.1.smithi150.stdout:4/124: rename d0/d1b/d1d to d0/d2f 0 2024-03-22T10:42:45.346 INFO:tasks.workunit.client.0.smithi115.stdout:1/599: dwrite d0/d8/de/f4c [8388608,4194304] 0 2024-03-22T10:42:45.350 INFO:tasks.workunit.client.1.smithi150.stdout:1/126: read d1/da/fd [4427138,3273] 0 2024-03-22T10:42:45.350 INFO:tasks.workunit.client.0.smithi115.stdout:8/722: truncate d6/dd/d24/fa7 1015518 0 2024-03-22T10:42:45.357 INFO:tasks.workunit.client.1.smithi150.stdout:5/80: getdents d0/d8/dc 0 2024-03-22T10:42:45.358 INFO:tasks.workunit.client.1.smithi150.stdout:1/127: write d1/da/f13 [2182488,120171] 0 2024-03-22T10:42:45.359 INFO:tasks.workunit.client.1.smithi150.stdout:0/145: rename d0/d1/fa to d0/d1/d7/d11/f2f 0 2024-03-22T10:42:45.362 INFO:tasks.workunit.client.0.smithi115.stdout:2/512: mknod da/d1c/d67/d78/c9a 0 2024-03-22T10:42:45.362 INFO:tasks.workunit.client.0.smithi115.stdout:0/498: read d3/d4/d14/f23 [2552603,56540] 0 2024-03-22T10:42:45.363 INFO:tasks.workunit.client.1.smithi150.stdout:4/125: creat d0/d2f/f30 x:0 0 0 2024-03-22T10:42:45.363 INFO:tasks.workunit.client.0.smithi115.stdout:1/600: dread d0/f87 [0,4194304] 0 2024-03-22T10:42:45.365 INFO:tasks.workunit.client.1.smithi150.stdout:5/81: write d0/f10 [171901,41150] 0 2024-03-22T10:42:45.365 INFO:tasks.workunit.client.0.smithi115.stdout:7/535: mknod d2/d62/cbc 0 2024-03-22T10:42:45.366 INFO:tasks.workunit.client.0.smithi115.stdout:5/516: truncate d2/d15/d1b/d7c/d84/f97 231059 0 2024-03-22T10:42:45.367 INFO:tasks.workunit.client.0.smithi115.stdout:2/513: write da/d25/f34 [2984303,94438] 0 2024-03-22T10:42:45.369 INFO:tasks.workunit.client.0.smithi115.stdout:1/601: readlink d0/d8/de/l2a 0 2024-03-22T10:42:45.369 INFO:tasks.workunit.client.0.smithi115.stdout:7/536: truncate d2/f60 4880083 0 2024-03-22T10:42:45.369 INFO:tasks.workunit.client.1.smithi150.stdout:0/146: chown d0/d1/d7/l29 239042306 1 2024-03-22T10:42:45.370 INFO:tasks.workunit.client.0.smithi115.stdout:7/537: stat d2/c6b 0 2024-03-22T10:42:45.371 INFO:tasks.workunit.client.0.smithi115.stdout:4/475: rename d1/d30/d42 to d1/d30/d4c/d86/d9f 0 2024-03-22T10:42:45.371 INFO:tasks.workunit.client.1.smithi150.stdout:1/128: write d1/f7 [872978,93958] 0 2024-03-22T10:42:45.371 INFO:tasks.workunit.client.1.smithi150.stdout:7/119: dwrite d2/d5/d7/de/f22 [4194304,4194304] 0 2024-03-22T10:42:45.373 INFO:tasks.workunit.client.1.smithi150.stdout:6/100: rename db/dd/l19 to db/dd/d15/l22 0 2024-03-22T10:42:45.374 INFO:tasks.workunit.client.0.smithi115.stdout:7/538: chown d2/d3a/d6e/f59 88 1 2024-03-22T10:42:45.375 INFO:tasks.workunit.client.1.smithi150.stdout:5/82: dread - d0/d8/dc/de/d14/f17 zero size 2024-03-22T10:42:45.376 INFO:tasks.workunit.client.1.smithi150.stdout:1/129: readlink d1/d19/d1a/l24 0 2024-03-22T10:42:45.377 INFO:tasks.workunit.client.1.smithi150.stdout:4/126: mknod d0/d2f/c31 0 2024-03-22T10:42:45.383 INFO:tasks.workunit.client.0.smithi115.stdout:8/723: fsync d6/dd/f70 0 2024-03-22T10:42:45.383 INFO:tasks.workunit.client.1.smithi150.stdout:0/147: write d0/d1/d7/f28 [458572,60266] 0 2024-03-22T10:42:45.384 INFO:tasks.workunit.client.1.smithi150.stdout:7/120: unlink d2/c17 0 2024-03-22T10:42:45.388 INFO:tasks.workunit.client.0.smithi115.stdout:6/478: truncate d4/d11/d12/f31 4151803 0 2024-03-22T10:42:45.390 INFO:tasks.workunit.client.0.smithi115.stdout:9/559: dwrite d6/d7/d87/f8c [0,4194304] 0 2024-03-22T10:42:45.392 INFO:tasks.workunit.client.0.smithi115.stdout:0/499: read d3/d4/f60 [65991,95693] 0 2024-03-22T10:42:45.392 INFO:tasks.workunit.client.1.smithi150.stdout:6/101: rename db/dd/c18 to db/dd/d21/c23 0 2024-03-22T10:42:45.392 INFO:tasks.workunit.client.1.smithi150.stdout:7/121: mknod d2/d5/d7/d8/c29 0 2024-03-22T10:42:45.397 INFO:tasks.workunit.client.1.smithi150.stdout:0/148: symlink d0/d1/d7/d11/d26/l30 0 2024-03-22T10:42:45.399 INFO:tasks.workunit.client.0.smithi115.stdout:1/602: mknod d0/d8/d4d/d56/ccc 0 2024-03-22T10:42:45.401 INFO:tasks.workunit.client.0.smithi115.stdout:7/539: mkdir d2/d7/d12/d10/d19/dbd 0 2024-03-22T10:42:45.402 INFO:tasks.workunit.client.1.smithi150.stdout:6/102: dread db/ff [0,4194304] 0 2024-03-22T10:42:45.402 INFO:tasks.workunit.client.0.smithi115.stdout:3/510: write d3/d36/f3e [904260,1552] 0 2024-03-22T10:42:45.406 INFO:tasks.workunit.client.1.smithi150.stdout:0/149: fdatasync d0/d1/d7/db/fc 0 2024-03-22T10:42:45.408 INFO:tasks.workunit.client.1.smithi150.stdout:0/150: readlink d0/d1/l1b 0 2024-03-22T10:42:45.409 INFO:tasks.workunit.client.1.smithi150.stdout:4/127: dread d0/d1b/f2c [0,4194304] 0 2024-03-22T10:42:45.409 INFO:tasks.workunit.client.0.smithi115.stdout:9/560: creat d6/d7/d30/d37/dbe/fc6 x:0 0 0 2024-03-22T10:42:45.410 INFO:tasks.workunit.client.0.smithi115.stdout:5/517: unlink d2/df/d27/d7a/fa3 0 2024-03-22T10:42:45.416 INFO:tasks.workunit.client.0.smithi115.stdout:4/476: creat d1/d30/d4c/d86/d9f/d75/d99/d98/d9b/fa0 x:0 0 0 2024-03-22T10:42:45.421 INFO:tasks.workunit.client.1.smithi150.stdout:5/83: dwrite d0/d8/dc/de/f15 [0,4194304] 0 2024-03-22T10:42:45.422 INFO:tasks.workunit.client.1.smithi150.stdout:7/122: dread d2/d5/d7/d9/f18 [0,4194304] 0 2024-03-22T10:42:45.422 INFO:tasks.workunit.client.0.smithi115.stdout:3/511: rename d3/d82/d3a to d3/d19/d2b/d49/d91/da5 0 2024-03-22T10:42:45.425 INFO:tasks.workunit.client.1.smithi150.stdout:5/84: read - d0/d8/dc/de/d14/f17 zero size 2024-03-22T10:42:45.432 INFO:tasks.workunit.client.1.smithi150.stdout:7/123: fdatasync d2/d5/d7/d9/f28 0 2024-03-22T10:42:45.432 INFO:tasks.workunit.client.0.smithi115.stdout:6/479: mknod d4/d11/c96 0 2024-03-22T10:42:45.434 INFO:tasks.workunit.client.0.smithi115.stdout:3/512: stat d3/d16/d29/d35/ca1 0 2024-03-22T10:42:45.435 INFO:tasks.workunit.client.1.smithi150.stdout:7/124: chown d2/d5/d7/d9/f18 86140 1 2024-03-22T10:42:45.436 INFO:tasks.workunit.client.1.smithi150.stdout:6/103: dwrite db/dd/f1b [0,4194304] 0 2024-03-22T10:42:45.439 INFO:tasks.workunit.client.0.smithi115.stdout:5/518: mknod d2/df/d13/cad 0 2024-03-22T10:42:45.440 INFO:tasks.workunit.client.1.smithi150.stdout:0/151: link d0/d1/d7/d11/d26/f2e d0/d1/d7/f31 0 2024-03-22T10:42:45.440 INFO:tasks.workunit.client.0.smithi115.stdout:5/519: readlink d2/df/d13/l25 0 2024-03-22T10:42:45.442 INFO:tasks.workunit.client.1.smithi150.stdout:7/125: symlink d2/d5/d7/d9/l2a 0 2024-03-22T10:42:45.442 INFO:tasks.workunit.client.1.smithi150.stdout:0/152: chown d0/d1/d7/l29 240226828 1 2024-03-22T10:42:45.444 INFO:tasks.workunit.client.1.smithi150.stdout:9/134: sync 2024-03-22T10:42:45.444 INFO:tasks.workunit.client.0.smithi115.stdout:1/603: dread d0/d12/f5d [0,4194304] 0 2024-03-22T10:42:45.445 INFO:tasks.workunit.client.0.smithi115.stdout:5/520: write d2/d15/d1b/d7c/d38/d43/d49/fa9 [375876,129023] 0 2024-03-22T10:42:45.447 INFO:tasks.workunit.client.1.smithi150.stdout:9/135: fsync d1/d8/de/f2c 0 2024-03-22T10:42:45.448 INFO:tasks.workunit.client.1.smithi150.stdout:7/126: write d2/d5/d7/d9/f28 [1216547,9985] 0 2024-03-22T10:42:45.448 INFO:tasks.workunit.client.1.smithi150.stdout:6/104: mknod db/dd/d21/c24 0 2024-03-22T10:42:45.448 INFO:tasks.workunit.client.0.smithi115.stdout:6/480: readlink d4/d11/d12/d1c/l40 0 2024-03-22T10:42:45.450 INFO:tasks.workunit.client.1.smithi150.stdout:0/153: rmdir d0/d1/d7/d11/d26 39 2024-03-22T10:42:45.451 INFO:tasks.workunit.client.0.smithi115.stdout:5/521: write d2/d15/d1b/d7c/f63 [933025,129970] 0 2024-03-22T10:42:45.452 INFO:tasks.workunit.client.0.smithi115.stdout:3/513: readlink d3/d82/l81 0 2024-03-22T10:42:45.453 INFO:tasks.workunit.client.1.smithi150.stdout:0/154: chown d0/d1/d7 16 1 2024-03-22T10:42:45.453 INFO:tasks.workunit.client.1.smithi150.stdout:6/105: mkdir db/dd/d25 0 2024-03-22T10:42:45.453 INFO:tasks.workunit.client.1.smithi150.stdout:7/127: stat d2/d5/d7/d9/db/l11 0 2024-03-22T10:42:45.456 INFO:tasks.workunit.client.1.smithi150.stdout:6/106: creat db/dd/f26 x:0 0 0 2024-03-22T10:42:45.457 INFO:tasks.workunit.client.0.smithi115.stdout:3/514: stat d3/d19/d2b/d49/d91/da5/l78 0 2024-03-22T10:42:45.457 INFO:tasks.workunit.client.0.smithi115.stdout:7/540: chown d2/d7/d12/d10/d19/d42/d46/d61 10 1 2024-03-22T10:42:45.463 INFO:tasks.workunit.client.0.smithi115.stdout:4/477: rmdir d1/d16/d68/d79 0 2024-03-22T10:42:45.464 INFO:tasks.workunit.client.1.smithi150.stdout:7/128: creat d2/d5/d7/f2b x:0 0 0 2024-03-22T10:42:45.465 INFO:tasks.workunit.client.0.smithi115.stdout:8/724: truncate d6/dd/d12/f27 3905276 0 2024-03-22T10:42:45.465 INFO:tasks.workunit.client.1.smithi150.stdout:4/128: dwrite d0/f1a [0,4194304] 0 2024-03-22T10:42:45.465 INFO:tasks.workunit.client.1.smithi150.stdout:6/107: creat db/dd/d15/f27 x:0 0 0 2024-03-22T10:42:45.467 INFO:tasks.workunit.client.0.smithi115.stdout:1/604: dwrite d0/d22/fc4 [0,4194304] 0 2024-03-22T10:42:45.467 INFO:tasks.workunit.client.0.smithi115.stdout:6/481: chown d4/d5e/f8b 578 1 2024-03-22T10:42:45.471 INFO:tasks.workunit.client.1.smithi150.stdout:6/108: mkdir db/dd/d15/d28 0 2024-03-22T10:42:45.473 INFO:tasks.workunit.client.1.smithi150.stdout:4/129: mkdir d0/d32 0 2024-03-22T10:42:45.475 INFO:tasks.workunit.client.1.smithi150.stdout:9/136: dwrite d1/d8/de/f17 [0,4194304] 0 2024-03-22T10:42:45.475 INFO:tasks.workunit.client.0.smithi115.stdout:5/522: symlink d2/d15/lae 0 2024-03-22T10:42:45.476 INFO:tasks.workunit.client.0.smithi115.stdout:0/500: truncate d3/d7/d1b/d35/f7e 3811436 0 2024-03-22T10:42:45.479 INFO:tasks.workunit.client.1.smithi150.stdout:6/109: unlink db/dd/d15/c1c 0 2024-03-22T10:42:45.482 INFO:tasks.workunit.client.1.smithi150.stdout:9/137: fsync d1/d8/de/df/f18 0 2024-03-22T10:42:45.482 INFO:tasks.workunit.client.1.smithi150.stdout:6/110: stat db/dd/c12 0 2024-03-22T10:42:45.483 INFO:tasks.workunit.client.0.smithi115.stdout:6/482: creat d4/d5e/f97 x:0 0 0 2024-03-22T10:42:45.485 INFO:tasks.workunit.client.0.smithi115.stdout:1/605: dread d0/d8/d4d/d56/f6d [0,4194304] 0 2024-03-22T10:42:45.485 INFO:tasks.workunit.client.1.smithi150.stdout:7/129: dread d2/d5/d7/d9/f18 [0,4194304] 0 2024-03-22T10:42:45.489 INFO:tasks.workunit.client.1.smithi150.stdout:6/111: symlink db/dd/d15/d28/l29 0 2024-03-22T10:42:45.489 INFO:tasks.workunit.client.0.smithi115.stdout:9/561: truncate d6/d7/ff 3853583 0 2024-03-22T10:42:45.492 INFO:tasks.workunit.client.0.smithi115.stdout:0/501: unlink d3/l3a 0 2024-03-22T10:42:45.492 INFO:tasks.workunit.client.1.smithi150.stdout:9/138: rename d1/d8/cb to d1/d8/de/df/d24/c30 0 2024-03-22T10:42:45.493 INFO:tasks.workunit.client.1.smithi150.stdout:6/112: write db/fc [255823,37705] 0 2024-03-22T10:42:45.493 INFO:tasks.workunit.client.0.smithi115.stdout:6/483: read d4/d36/d48/d72/f76 [1834516,101177] 0 2024-03-22T10:42:45.494 INFO:tasks.workunit.client.1.smithi150.stdout:4/130: dread d0/f1e [0,4194304] 0 2024-03-22T10:42:45.495 INFO:tasks.workunit.client.0.smithi115.stdout:0/502: write d3/d7/f3b [4088385,83195] 0 2024-03-22T10:42:45.498 INFO:tasks.workunit.client.1.smithi150.stdout:4/131: read d0/d1b/f2b [1382088,127674] 0 2024-03-22T10:42:45.499 INFO:tasks.workunit.client.0.smithi115.stdout:8/725: truncate d6/dd/d12/dc6/fde 3886768 0 2024-03-22T10:42:45.499 INFO:tasks.workunit.client.1.smithi150.stdout:6/113: dread db/ff [0,4194304] 0 2024-03-22T10:42:45.500 INFO:tasks.workunit.client.0.smithi115.stdout:1/606: rename d0/d8/d4d/d56/f6b to d0/d8/de/d1a/d91/fcd 0 2024-03-22T10:42:45.501 INFO:tasks.workunit.client.0.smithi115.stdout:1/607: chown d0/d12/d2b/fcb 8 1 2024-03-22T10:42:45.502 INFO:tasks.workunit.client.0.smithi115.stdout:5/523: mkdir d2/d15/d1b/d7c/daf 0 2024-03-22T10:42:45.502 INFO:tasks.workunit.client.1.smithi150.stdout:4/132: mknod d0/d2f/c33 0 2024-03-22T10:42:45.503 INFO:tasks.workunit.client.0.smithi115.stdout:7/541: creat d2/d7/fbe x:0 0 0 2024-03-22T10:42:45.504 INFO:tasks.workunit.client.1.smithi150.stdout:6/114: chown db/dd/d21/c24 7 1 2024-03-22T10:42:45.505 INFO:tasks.workunit.client.0.smithi115.stdout:5/524: write d2/df/f10 [7606990,114258] 0 2024-03-22T10:42:45.507 INFO:tasks.workunit.client.1.smithi150.stdout:4/133: symlink d0/d2f/l34 0 2024-03-22T10:42:45.507 INFO:tasks.workunit.client.0.smithi115.stdout:6/484: mkdir d4/d11/d12/d1c/d8a/d98 0 2024-03-22T10:42:45.508 INFO:tasks.workunit.client.1.smithi150.stdout:6/115: creat db/f2a x:0 0 0 2024-03-22T10:42:45.508 INFO:tasks.workunit.client.1.smithi150.stdout:7/130: dread d2/d5/d7/d9/f18 [0,4194304] 0 2024-03-22T10:42:45.508 INFO:tasks.workunit.client.0.smithi115.stdout:0/503: fsync d3/d4/d6d/f9e 0 2024-03-22T10:42:45.509 INFO:tasks.workunit.client.0.smithi115.stdout:6/485: chown d4/d36/c71 375 1 2024-03-22T10:42:45.510 INFO:tasks.workunit.client.1.smithi150.stdout:4/134: write d0/f1e [2987772,20875] 0 2024-03-22T10:42:45.510 INFO:tasks.workunit.client.0.smithi115.stdout:8/726: fsync d6/dd/d24/f3f 0 2024-03-22T10:42:45.513 INFO:tasks.workunit.client.1.smithi150.stdout:7/131: mkdir d2/d5/d7/d8/d2c 0 2024-03-22T10:42:45.515 INFO:tasks.workunit.client.0.smithi115.stdout:3/515: getdents d3/d16/d29 0 2024-03-22T10:42:45.516 INFO:tasks.workunit.client.1.smithi150.stdout:4/135: symlink d0/d2f/l35 0 2024-03-22T10:42:45.517 INFO:tasks.workunit.client.0.smithi115.stdout:3/516: chown d3/d82 11 1 2024-03-22T10:42:45.518 INFO:tasks.workunit.client.1.smithi150.stdout:7/132: creat d2/d5/d7/d8/f2d x:0 0 0 2024-03-22T10:42:45.519 INFO:tasks.workunit.client.1.smithi150.stdout:4/136: symlink d0/d1b/l36 0 2024-03-22T10:42:45.520 INFO:tasks.workunit.client.0.smithi115.stdout:3/517: write d3/f53 [4974680,36880] 0 2024-03-22T10:42:45.520 INFO:tasks.workunit.client.1.smithi150.stdout:4/137: fsync d0/f26 0 2024-03-22T10:42:45.521 INFO:tasks.workunit.client.1.smithi150.stdout:7/133: truncate d2/d5/d7/d8/f2d 1001188 0 2024-03-22T10:42:45.522 INFO:tasks.workunit.client.0.smithi115.stdout:4/478: write d1/d30/f4a [4298735,99377] 0 2024-03-22T10:42:45.522 INFO:tasks.workunit.client.0.smithi115.stdout:3/518: fsync d3/d82/f8c 0 2024-03-22T10:42:45.523 INFO:tasks.workunit.client.1.smithi150.stdout:9/139: unlink d1/d8/de/df/d24/c30 0 2024-03-22T10:42:45.523 INFO:tasks.workunit.client.1.smithi150.stdout:4/138: truncate d0/d1b/f23 82862 0 2024-03-22T10:42:45.525 INFO:tasks.workunit.client.0.smithi115.stdout:0/504: rename d3/l9 to d3/d7/d1c/d6c/d9a/la7 0 2024-03-22T10:42:45.525 INFO:tasks.workunit.client.1.smithi150.stdout:4/139: write d0/f7 [941146,68347] 0 2024-03-22T10:42:45.525 INFO:tasks.workunit.client.1.smithi150.stdout:7/134: mkdir d2/d5/d7/d2e 0 2024-03-22T10:42:45.529 INFO:tasks.workunit.client.0.smithi115.stdout:1/608: truncate d0/d12/d17/d37/d5c/f8c 1141926 0 2024-03-22T10:42:45.530 INFO:tasks.workunit.client.1.smithi150.stdout:7/135: mknod d2/d5/d7/d9/c2f 0 2024-03-22T10:42:45.530 INFO:tasks.workunit.client.1.smithi150.stdout:4/140: dread d0/f8 [0,4194304] 0 2024-03-22T10:42:45.532 INFO:tasks.workunit.client.0.smithi115.stdout:6/486: mkdir d4/d11/d99 0 2024-03-22T10:42:45.533 INFO:tasks.workunit.client.1.smithi150.stdout:7/136: rename d2/d5 to d2/d5/d7/de/d30 22 2024-03-22T10:42:45.536 INFO:tasks.workunit.client.1.smithi150.stdout:7/137: chown d2/d5/d7/de/f22 13 1 2024-03-22T10:42:45.537 INFO:tasks.workunit.client.0.smithi115.stdout:3/519: mkdir d3/d19/d61/da6 0 2024-03-22T10:42:45.538 INFO:tasks.workunit.client.1.smithi150.stdout:9/140: truncate d1/d8/de/f17 2264183 0 2024-03-22T10:42:45.539 INFO:tasks.workunit.client.0.smithi115.stdout:3/520: stat d3/d6d/f9e 0 2024-03-22T10:42:45.540 INFO:tasks.workunit.client.0.smithi115.stdout:6/487: read d4/d11/d12/d1c/f46 [2255803,9324] 0 2024-03-22T10:42:45.540 INFO:tasks.workunit.client.0.smithi115.stdout:3/521: dread - d3/d19/d2b/d55/f88 zero size 2024-03-22T10:42:45.541 INFO:tasks.workunit.client.0.smithi115.stdout:7/542: rename d2/d3a/d90 to d2/d3a/d6e/d78/dbf 0 2024-03-22T10:42:45.544 INFO:tasks.workunit.client.1.smithi150.stdout:9/141: creat d1/d8/de/df/d25/d2e/f31 x:0 0 0 2024-03-22T10:42:45.544 INFO:tasks.workunit.client.0.smithi115.stdout:8/727: truncate d6/dd/d24/fa7 280236 0 2024-03-22T10:42:45.546 INFO:tasks.workunit.client.1.smithi150.stdout:9/142: write d1/d8/de/f2c [694358,122640] 0 2024-03-22T10:42:45.548 INFO:tasks.workunit.client.1.smithi150.stdout:4/141: dwrite d0/f1e [0,4194304] 0 2024-03-22T10:42:45.550 INFO:tasks.workunit.client.1.smithi150.stdout:7/138: dread d2/d5/d7/d9/f18 [0,4194304] 0 2024-03-22T10:42:45.551 INFO:tasks.workunit.client.0.smithi115.stdout:3/522: fdatasync d3/d16/f58 0 2024-03-22T10:42:45.553 INFO:tasks.workunit.client.0.smithi115.stdout:8/728: dread d6/fad [0,4194304] 0 2024-03-22T10:42:45.554 INFO:tasks.workunit.client.0.smithi115.stdout:3/523: truncate d3/d82/f22 5201938 0 2024-03-22T10:42:45.556 INFO:tasks.workunit.client.0.smithi115.stdout:5/525: rename d2/d15/d1b/d7c/d38/d4a/d74/f7e to d2/df/d3b/fb0 0 2024-03-22T10:42:45.557 INFO:tasks.workunit.client.1.smithi150.stdout:7/139: getdents d2/d5/d7/de 0 2024-03-22T10:42:45.559 INFO:tasks.workunit.client.1.smithi150.stdout:7/140: chown d2/d5/d7/d8/d2c 301372 1 2024-03-22T10:42:45.562 INFO:tasks.workunit.client.1.smithi150.stdout:9/143: rename d1/d8/de/df/d25 to d1/d8/de/df/d20/d32 0 2024-03-22T10:42:45.563 INFO:tasks.workunit.client.0.smithi115.stdout:0/505: dread d3/d7/d1b/d1f/d5b/f66 [0,4194304] 0 2024-03-22T10:42:45.565 INFO:tasks.workunit.client.0.smithi115.stdout:0/506: dread - d3/da/d4c/f91 zero size 2024-03-22T10:42:45.566 INFO:tasks.workunit.client.1.smithi150.stdout:7/141: dread d2/d5/d7/d8/f2d [0,4194304] 0 2024-03-22T10:42:45.566 INFO:tasks.workunit.client.0.smithi115.stdout:7/543: truncate d2/d7/d12/d10/d19/f9d 4651393 0 2024-03-22T10:42:45.567 INFO:tasks.workunit.client.1.smithi150.stdout:4/142: dwrite d0/de/f28 [0,4194304] 0 2024-03-22T10:42:45.569 INFO:tasks.workunit.client.1.smithi150.stdout:7/142: dread - d2/d5/d7/d9/db/f1a zero size 2024-03-22T10:42:45.570 INFO:tasks.workunit.client.1.smithi150.stdout:4/143: chown d0/d2f/c31 1861116 1 2024-03-22T10:42:45.572 INFO:tasks.workunit.client.0.smithi115.stdout:1/609: mknod d0/d68/d81/cce 0 2024-03-22T10:42:45.573 INFO:tasks.workunit.client.1.smithi150.stdout:7/143: write d2/d5/d7/d9/f28 [1872213,22377] 0 2024-03-22T10:42:45.576 INFO:tasks.workunit.client.1.smithi150.stdout:7/144: mknod d2/d5/d7/d9/c31 0 2024-03-22T10:42:45.577 INFO:tasks.workunit.client.1.smithi150.stdout:9/144: dread d1/d8/de/df/f18 [0,4194304] 0 2024-03-22T10:42:45.579 INFO:tasks.workunit.client.0.smithi115.stdout:4/479: rename d1/d69/d49 to d1/d66/d82/da1 0 2024-03-22T10:42:45.579 INFO:tasks.workunit.client.0.smithi115.stdout:8/729: symlink d6/dd/d24/d3a/led 0 2024-03-22T10:42:45.579 INFO:tasks.workunit.client.1.smithi150.stdout:4/144: link d0/de/f18 d0/d1b/f37 0 2024-03-22T10:42:45.580 INFO:tasks.workunit.client.1.smithi150.stdout:4/145: chown d0/d1b/f2c 2 1 2024-03-22T10:42:45.581 INFO:tasks.workunit.client.1.smithi150.stdout:4/146: readlink d0/l15 0 2024-03-22T10:42:45.583 INFO:tasks.workunit.client.1.smithi150.stdout:4/147: rmdir d0/d2f 39 2024-03-22T10:42:45.584 INFO:tasks.workunit.client.0.smithi115.stdout:7/544: fdatasync d2/d7/d12/d10/d23/f6f 0 2024-03-22T10:42:45.585 INFO:tasks.workunit.client.0.smithi115.stdout:6/488: creat d4/d11/f9a x:0 0 0 2024-03-22T10:42:45.587 INFO:tasks.workunit.client.0.smithi115.stdout:7/545: read d2/d7/d12/d10/d19/d42/d46/d61/fba [509664,113191] 0 2024-03-22T10:42:45.588 INFO:tasks.workunit.client.0.smithi115.stdout:1/610: unlink d0/d12/l72 0 2024-03-22T10:42:45.593 INFO:tasks.workunit.client.1.smithi150.stdout:4/148: rmdir d0/d1b 39 2024-03-22T10:42:45.594 INFO:tasks.workunit.client.1.smithi150.stdout:9/145: dwrite d1/d8/de/df/f1f [0,4194304] 0 2024-03-22T10:42:45.600 INFO:tasks.workunit.client.1.smithi150.stdout:9/146: symlink d1/d8/de/df/d20/d32/d2e/l33 0 2024-03-22T10:42:45.602 INFO:tasks.workunit.client.1.smithi150.stdout:4/149: creat d0/f38 x:0 0 0 2024-03-22T10:42:45.603 INFO:tasks.workunit.client.0.smithi115.stdout:3/524: link d3/d19/f1d d3/d19/d2b/d49/d91/da5/fa7 0 2024-03-22T10:42:45.603 INFO:tasks.workunit.client.1.smithi150.stdout:4/150: chown d0/de/f18 448678 1 2024-03-22T10:42:45.606 INFO:tasks.workunit.client.0.smithi115.stdout:0/507: creat d3/d7/d1b/fa8 x:0 0 0 2024-03-22T10:42:45.610 INFO:tasks.workunit.client.1.smithi150.stdout:4/151: write d0/f9 [245105,96441] 0 2024-03-22T10:42:45.610 INFO:tasks.workunit.client.0.smithi115.stdout:4/480: symlink d1/d16/d35/d40/la2 0 2024-03-22T10:42:45.613 INFO:tasks.workunit.client.1.smithi150.stdout:9/147: dread d1/d8/de/df/f18 [4194304,4194304] 0 2024-03-22T10:42:45.613 INFO:tasks.workunit.client.0.smithi115.stdout:6/489: dread d4/d11/d12/d1c/f69 [0,4194304] 0 2024-03-22T10:42:45.614 INFO:tasks.workunit.client.1.smithi150.stdout:4/152: rmdir d0/de/d27 0 2024-03-22T10:42:45.616 INFO:tasks.workunit.client.0.smithi115.stdout:3/525: symlink d3/d82/la8 0 2024-03-22T10:42:45.617 INFO:tasks.workunit.client.1.smithi150.stdout:3/123: sync 2024-03-22T10:42:45.617 INFO:tasks.workunit.client.1.smithi150.stdout:8/126: sync 2024-03-22T10:42:45.617 INFO:tasks.workunit.client.1.smithi150.stdout:8/127: fdatasync d0/d2/f14 0 2024-03-22T10:42:45.618 INFO:tasks.workunit.client.1.smithi150.stdout:4/153: creat d0/d1b/f39 x:0 0 0 2024-03-22T10:42:45.618 INFO:tasks.workunit.client.0.smithi115.stdout:7/546: mkdir d2/d7/d12/d10/d23/d3d/da4/dc0 0 2024-03-22T10:42:45.618 INFO:tasks.workunit.client.1.smithi150.stdout:3/124: mkdir d5/d22 0 2024-03-22T10:42:45.619 INFO:tasks.workunit.client.0.smithi115.stdout:4/481: rmdir d1/d30/d4c 39 2024-03-22T10:42:45.621 INFO:tasks.workunit.client.0.smithi115.stdout:1/611: dwrite d0/d12/d17/d37/d5c/fb4 [0,4194304] 0 2024-03-22T10:42:45.622 INFO:tasks.workunit.client.1.smithi150.stdout:8/128: unlink d0/d2/d19/d29/f2c 0 2024-03-22T10:42:45.624 INFO:tasks.workunit.client.0.smithi115.stdout:3/526: write d3/d82/f8c [3519990,68962] 0 2024-03-22T10:42:45.625 INFO:tasks.workunit.client.1.smithi150.stdout:8/129: fdatasync d0/f21 0 2024-03-22T10:42:45.625 INFO:tasks.workunit.client.0.smithi115.stdout:6/490: fdatasync d4/d13/f4b 0 2024-03-22T10:42:45.627 INFO:tasks.workunit.client.0.smithi115.stdout:7/547: stat d2/d7/d12/d10/d23/d3d/c22 0 2024-03-22T10:42:45.628 INFO:tasks.workunit.client.0.smithi115.stdout:3/527: readlink d3/d82/l26 0 2024-03-22T10:42:45.629 INFO:tasks.workunit.client.0.smithi115.stdout:4/482: read d1/d16/d35/d40/d5c/d14/f1b [4715788,97774] 0 2024-03-22T10:42:45.629 INFO:tasks.workunit.client.0.smithi115.stdout:7/548: read - d2/d3a/d6e/f95 zero size 2024-03-22T10:42:45.631 INFO:tasks.workunit.client.1.smithi150.stdout:8/130: mkdir d0/d2/d19/d29/d31 0 2024-03-22T10:42:45.633 INFO:tasks.workunit.client.0.smithi115.stdout:4/483: readlink d1/d16/d35/d40/d5c/d14/l4e 0 2024-03-22T10:42:45.634 INFO:tasks.workunit.client.0.smithi115.stdout:1/612: mkdir d0/d8/d4d/d56/dcf 0 2024-03-22T10:42:45.634 INFO:tasks.workunit.client.0.smithi115.stdout:3/528: fdatasync d3/d19/d2b/d49/d91/da5/f75 0 2024-03-22T10:42:45.635 INFO:tasks.workunit.client.1.smithi150.stdout:9/148: dwrite d1/d8/de/df/f18 [4194304,4194304] 0 2024-03-22T10:42:45.635 INFO:tasks.workunit.client.1.smithi150.stdout:8/131: write d0/d2/d19/f25 [207385,36077] 0 2024-03-22T10:42:45.636 INFO:tasks.workunit.client.0.smithi115.stdout:6/491: mknod d4/d5e/c9b 0 2024-03-22T10:42:45.637 INFO:tasks.workunit.client.0.smithi115.stdout:3/529: read d3/f23 [82695,106372] 0 2024-03-22T10:42:45.639 INFO:tasks.workunit.client.1.smithi150.stdout:3/125: dread d5/f8 [0,4194304] 0 2024-03-22T10:42:45.639 INFO:tasks.workunit.client.0.smithi115.stdout:4/484: write d1/d66/f6d [2194079,85945] 0 2024-03-22T10:42:45.643 INFO:tasks.workunit.client.1.smithi150.stdout:3/126: mknod d5/da/c23 0 2024-03-22T10:42:45.643 INFO:tasks.workunit.client.0.smithi115.stdout:6/492: creat d4/d11/d12/f9c x:0 0 0 2024-03-22T10:42:45.644 INFO:tasks.workunit.client.1.smithi150.stdout:3/127: chown d5/da/l1f 343892 1 2024-03-22T10:42:45.645 INFO:tasks.workunit.client.0.smithi115.stdout:1/613: creat d0/d8/d4d/d56/dbd/fd0 x:0 0 0 2024-03-22T10:42:45.647 INFO:tasks.workunit.client.0.smithi115.stdout:1/614: write d0/d22/d89/fba [944709,8021] 0 2024-03-22T10:42:45.648 INFO:tasks.workunit.client.0.smithi115.stdout:0/508: truncate d3/d7/d1c/f77 2086657 0 2024-03-22T10:42:45.648 INFO:tasks.workunit.client.0.smithi115.stdout:4/485: symlink d1/d16/d68/la3 0 2024-03-22T10:42:45.650 INFO:tasks.workunit.client.1.smithi150.stdout:3/128: creat d5/f24 x:0 0 0 2024-03-22T10:42:45.654 INFO:tasks.workunit.client.1.smithi150.stdout:3/129: creat d5/f25 x:0 0 0 2024-03-22T10:42:45.656 INFO:tasks.workunit.client.0.smithi115.stdout:1/615: rmdir d0/d8/db 39 2024-03-22T10:42:45.657 INFO:tasks.workunit.client.1.smithi150.stdout:3/130: mknod d5/d22/c26 0 2024-03-22T10:42:45.659 INFO:tasks.workunit.client.1.smithi150.stdout:8/132: dwrite d0/d2/d19/f25 [0,4194304] 0 2024-03-22T10:42:45.659 INFO:tasks.workunit.client.1.smithi150.stdout:3/131: write d5/fd [1879207,52787] 0 2024-03-22T10:42:45.661 INFO:tasks.workunit.client.1.smithi150.stdout:8/133: fsync d0/d2/f14 0 2024-03-22T10:42:45.663 INFO:tasks.workunit.client.0.smithi115.stdout:0/509: creat d3/fa9 x:0 0 0 2024-03-22T10:42:45.663 INFO:tasks.workunit.client.0.smithi115.stdout:7/549: getdents d2/d7/d12/d10/d19/d42/d46 0 2024-03-22T10:42:45.666 INFO:tasks.workunit.client.1.smithi150.stdout:8/134: symlink d0/d2/d19/d29/d2f/df/d1b/l32 0 2024-03-22T10:42:45.667 INFO:tasks.workunit.client.0.smithi115.stdout:8/730: dwrite d6/dd/d12/dc6/fde [0,4194304] 0 2024-03-22T10:42:45.670 INFO:tasks.workunit.client.0.smithi115.stdout:1/616: chown d0/d12/d17/d37/l94 893 1 2024-03-22T10:42:45.671 INFO:tasks.workunit.client.1.smithi150.stdout:4/154: dread d0/d1b/f37 [0,4194304] 0 2024-03-22T10:42:45.671 INFO:tasks.workunit.client.1.smithi150.stdout:8/135: rename d0/l9 to d0/d2/d19/l33 0 2024-03-22T10:42:45.672 INFO:tasks.workunit.client.1.smithi150.stdout:4/155: getdents d0/d32 0 2024-03-22T10:42:45.674 INFO:tasks.workunit.client.1.smithi150.stdout:8/136: creat d0/f34 x:0 0 0 2024-03-22T10:42:45.676 INFO:tasks.workunit.client.0.smithi115.stdout:4/486: unlink d1/d30/d4c/d86/d9f/f3d 0 2024-03-22T10:42:45.677 INFO:tasks.workunit.client.1.smithi150.stdout:8/137: write d0/d2/d19/d29/d2f/df/d1b/f24 [728440,76787] 0 2024-03-22T10:42:45.679 INFO:tasks.workunit.client.1.smithi150.stdout:4/156: link d0/d1b/f24 d0/d1b/f3a 0 2024-03-22T10:42:45.681 INFO:tasks.workunit.client.0.smithi115.stdout:4/487: fdatasync d1/d16/d35/d40/d5c/d14/f3e 0 2024-03-22T10:42:45.681 INFO:tasks.workunit.client.0.smithi115.stdout:6/493: dwrite d4/d11/f54 [0,4194304] 0 2024-03-22T10:42:45.684 INFO:tasks.workunit.client.1.smithi150.stdout:3/132: getdents d5 0 2024-03-22T10:42:45.686 INFO:tasks.workunit.client.0.smithi115.stdout:3/530: dwrite d3/d36/d41/f9a [0,4194304] 0 2024-03-22T10:42:45.691 INFO:tasks.workunit.client.0.smithi115.stdout:1/617: chown d0/d12/f30 2836369 1 2024-03-22T10:42:45.692 INFO:tasks.workunit.client.1.smithi150.stdout:3/133: rename d5/c1b to d5/d22/c27 0 2024-03-22T10:42:45.693 INFO:tasks.workunit.client.0.smithi115.stdout:1/618: readlink d0/d12/la4 0 2024-03-22T10:42:45.693 INFO:tasks.workunit.client.0.smithi115.stdout:6/494: write d4/d36/f53 [2265683,21226] 0 2024-03-22T10:42:45.694 INFO:tasks.workunit.client.1.smithi150.stdout:3/134: creat d5/da/f28 x:0 0 0 2024-03-22T10:42:45.696 INFO:tasks.workunit.client.1.smithi150.stdout:3/135: dread - d5/da/f28 zero size 2024-03-22T10:42:45.699 INFO:tasks.workunit.client.1.smithi150.stdout:8/138: dwrite d0/d2/d19/f25 [0,4194304] 0 2024-03-22T10:42:45.701 INFO:tasks.workunit.client.0.smithi115.stdout:4/488: symlink d1/d30/d4c/d86/d9f/la4 0 2024-03-22T10:42:45.702 INFO:tasks.workunit.client.1.smithi150.stdout:8/139: fsync d0/f34 0 2024-03-22T10:42:45.706 INFO:tasks.workunit.client.0.smithi115.stdout:7/550: dread d2/d7/f13 [4194304,4194304] 0 2024-03-22T10:42:45.708 INFO:tasks.workunit.client.1.smithi150.stdout:8/140: mknod d0/d2/d19/d29/d31/c35 0 2024-03-22T10:42:45.709 INFO:tasks.workunit.client.0.smithi115.stdout:3/531: symlink d3/d19/d2b/d49/d91/da5/la9 0 2024-03-22T10:42:45.710 INFO:tasks.workunit.client.0.smithi115.stdout:1/619: rmdir d0/d22/d89 39 2024-03-22T10:42:45.712 INFO:tasks.workunit.client.1.smithi150.stdout:8/141: creat d0/d2/d19/d29/d2f/f36 x:0 0 0 2024-03-22T10:42:45.713 INFO:tasks.workunit.client.1.smithi150.stdout:3/136: dread d5/da/f19 [0,4194304] 0 2024-03-22T10:42:45.715 INFO:tasks.workunit.client.0.smithi115.stdout:8/731: rmdir d6/dd/d24/dc8 0 2024-03-22T10:42:45.715 INFO:tasks.workunit.client.0.smithi115.stdout:7/551: dread - d2/d7/d12/d10/d19/d42/d46/f7f zero size 2024-03-22T10:42:45.716 INFO:tasks.workunit.client.1.smithi150.stdout:8/142: mkdir d0/d2/d19/d29/d37 0 2024-03-22T10:42:45.716 INFO:tasks.workunit.client.0.smithi115.stdout:0/510: dwrite d3/d4/d6d/f98 [0,4194304] 0 2024-03-22T10:42:45.719 INFO:tasks.workunit.client.1.smithi150.stdout:8/143: rename d0/f5 to d0/d2/d19/d29/d2e/f38 0 2024-03-22T10:42:45.721 INFO:tasks.workunit.client.0.smithi115.stdout:8/732: chown d6/dd/d24/d3a/led 3150377 1 2024-03-22T10:42:45.722 INFO:tasks.workunit.client.1.smithi150.stdout:8/144: creat d0/d2/d19/d29/d31/f39 x:0 0 0 2024-03-22T10:42:45.724 INFO:tasks.workunit.client.0.smithi115.stdout:6/495: symlink d4/d13/l9d 0 2024-03-22T10:42:45.724 INFO:tasks.workunit.client.0.smithi115.stdout:3/532: mkdir d3/d19/d61/daa 0 2024-03-22T10:42:45.731 INFO:tasks.workunit.client.0.smithi115.stdout:3/533: write d3/d36/d41/f3f [515477,119678] 0 2024-03-22T10:42:45.731 INFO:tasks.workunit.client.0.smithi115.stdout:2/514: sync 2024-03-22T10:42:45.732 INFO:tasks.workunit.client.0.smithi115.stdout:8/733: write d6/d49/d88/f8a [771537,2379] 0 2024-03-22T10:42:45.734 INFO:tasks.workunit.client.1.smithi150.stdout:8/145: dread d0/d2/d19/d29/d2f/df/f26 [0,4194304] 0 2024-03-22T10:42:45.736 INFO:tasks.workunit.client.1.smithi150.stdout:8/146: chown d0/d2/d19/d29/d2f/df/d1b/f24 54 1 2024-03-22T10:42:45.739 INFO:tasks.workunit.client.0.smithi115.stdout:3/534: mknod d3/d19/d2b/d49/d91/da5/d7e/cab 0 2024-03-22T10:42:45.740 INFO:tasks.workunit.client.0.smithi115.stdout:2/515: mknod da/d25/d3d/c9b 0 2024-03-22T10:42:45.741 INFO:tasks.workunit.client.1.smithi150.stdout:8/147: link d0/f34 d0/d2/d19/d29/d2e/f3a 0 2024-03-22T10:42:45.742 INFO:tasks.workunit.client.0.smithi115.stdout:1/620: dwrite d0/d8/d4d/d56/dbd/fbf [0,4194304] 0 2024-03-22T10:42:45.750 INFO:tasks.workunit.client.0.smithi115.stdout:2/516: unlink da/d25/d30/c6a 0 2024-03-22T10:42:45.751 INFO:tasks.workunit.client.0.smithi115.stdout:4/489: dread d1/d16/d35/d40/d54/f67 [0,4194304] 0 2024-03-22T10:42:45.752 INFO:tasks.workunit.client.0.smithi115.stdout:0/511: dwrite d3/d4/d6d/f98 [0,4194304] 0 2024-03-22T10:42:45.756 INFO:tasks.workunit.client.1.smithi150.stdout:8/148: dwrite d0/f16 [0,4194304] 0 2024-03-22T10:42:45.764 INFO:tasks.workunit.client.0.smithi115.stdout:6/496: dwrite d4/d13/d33/f8c [0,4194304] 0 2024-03-22T10:42:45.765 INFO:tasks.workunit.client.0.smithi115.stdout:3/535: creat d3/d19/d61/fac x:0 0 0 2024-03-22T10:42:45.769 INFO:tasks.workunit.client.0.smithi115.stdout:0/512: truncate d3/d4/d14/f4f 2392641 0 2024-03-22T10:42:45.771 INFO:tasks.workunit.client.0.smithi115.stdout:9/562: sync 2024-03-22T10:42:45.771 INFO:tasks.workunit.client.0.smithi115.stdout:5/526: sync 2024-03-22T10:42:45.775 INFO:tasks.workunit.client.0.smithi115.stdout:8/734: truncate d6/d49/d68/f6f 3621419 0 2024-03-22T10:42:45.778 INFO:tasks.workunit.client.0.smithi115.stdout:2/517: dread da/d25/d3d/d57/d7d/f8b [0,4194304] 0 2024-03-22T10:42:45.779 INFO:tasks.workunit.client.0.smithi115.stdout:6/497: write d4/d11/f65 [223253,84152] 0 2024-03-22T10:42:45.781 INFO:tasks.workunit.client.0.smithi115.stdout:3/536: mkdir d3/d19/d2b/d49/d91/dad 0 2024-03-22T10:42:45.781 INFO:tasks.workunit.client.0.smithi115.stdout:4/490: dwrite d1/d66/f77 [0,4194304] 0 2024-03-22T10:42:45.782 INFO:tasks.workunit.client.0.smithi115.stdout:6/498: write d4/d5e/f97 [498561,31366] 0 2024-03-22T10:42:45.783 INFO:tasks.workunit.client.0.smithi115.stdout:2/518: chown da/d25/d51/f56 406378 1 2024-03-22T10:42:45.785 INFO:tasks.workunit.client.1.smithi150.stdout:2/155: sync 2024-03-22T10:42:45.787 INFO:tasks.workunit.client.0.smithi115.stdout:0/513: symlink d3/da/d4c/d69/laa 0 2024-03-22T10:42:45.790 INFO:tasks.workunit.client.0.smithi115.stdout:0/514: stat d3/d4/d14/f34 0 2024-03-22T10:42:45.793 INFO:tasks.workunit.client.0.smithi115.stdout:8/735: dread d6/d49/fa8 [0,4194304] 0 2024-03-22T10:42:45.798 INFO:tasks.workunit.client.0.smithi115.stdout:5/527: mkdir d2/df/d27/d50/db1 0 2024-03-22T10:42:45.801 INFO:tasks.workunit.client.0.smithi115.stdout:0/515: truncate d3/da/d4c/f91 1044896 0 2024-03-22T10:42:45.801 INFO:tasks.workunit.client.1.smithi150.stdout:2/156: read d12/d16/f22 [800525,25667] 0 2024-03-22T10:42:45.804 INFO:tasks.workunit.client.1.smithi150.stdout:8/149: dwrite d0/d2/d19/f20 [4194304,4194304] 0 2024-03-22T10:42:45.807 INFO:tasks.workunit.client.1.smithi150.stdout:2/157: creat d12/d19/f2e x:0 0 0 2024-03-22T10:42:45.807 INFO:tasks.workunit.client.0.smithi115.stdout:1/621: link d0/d8/db/d25/f73 d0/d8/de/d1a/d91/fd1 0 2024-03-22T10:42:45.808 INFO:tasks.workunit.client.1.smithi150.stdout:8/150: creat d0/d2/d19/d29/d2f/f3b x:0 0 0 2024-03-22T10:42:45.808 INFO:tasks.workunit.client.0.smithi115.stdout:2/519: symlink da/d25/d3d/d57/l9c 0 2024-03-22T10:42:45.808 INFO:tasks.workunit.client.0.smithi115.stdout:9/563: dwrite d6/d7/d19/f27 [4194304,4194304] 0 2024-03-22T10:42:45.810 INFO:tasks.workunit.client.1.smithi150.stdout:2/158: symlink d12/d21/l2f 0 2024-03-22T10:42:45.811 INFO:tasks.workunit.client.1.smithi150.stdout:8/151: truncate d0/fb 2476906 0 2024-03-22T10:42:45.813 INFO:tasks.workunit.client.1.smithi150.stdout:2/159: creat d12/d19/d1a/f30 x:0 0 0 2024-03-22T10:42:45.814 INFO:tasks.workunit.client.1.smithi150.stdout:8/152: mkdir d0/d3c 0 2024-03-22T10:42:45.818 INFO:tasks.workunit.client.0.smithi115.stdout:8/736: mkdir d6/dd/d12/dc6/dee 0 2024-03-22T10:42:45.823 INFO:tasks.workunit.client.0.smithi115.stdout:8/737: chown d6/dd/f76 24379 1 2024-03-22T10:42:45.824 INFO:tasks.workunit.client.0.smithi115.stdout:2/520: write da/d25/d30/d4f/d54/f7a [1585291,81462] 0 2024-03-22T10:42:45.825 INFO:tasks.workunit.client.0.smithi115.stdout:6/499: dread d4/d11/f54 [0,4194304] 0 2024-03-22T10:42:45.829 INFO:tasks.workunit.client.0.smithi115.stdout:5/528: creat d2/df/d27/fb2 x:0 0 0 2024-03-22T10:42:45.830 INFO:tasks.workunit.client.0.smithi115.stdout:3/537: dread d3/d36/f5c [0,4194304] 0 2024-03-22T10:42:45.836 INFO:tasks.workunit.client.1.smithi150.stdout:2/160: dwrite d12/d16/f28 [4194304,4194304] 0 2024-03-22T10:42:45.838 INFO:tasks.workunit.client.1.smithi150.stdout:8/153: dwrite d0/fd [0,4194304] 0 2024-03-22T10:42:45.840 INFO:tasks.workunit.client.0.smithi115.stdout:4/491: dwrite d1/d30/d4c/d86/d9f/d4f/f7c [0,4194304] 0 2024-03-22T10:42:45.842 INFO:tasks.workunit.client.0.smithi115.stdout:2/521: mkdir da/d1c/d67/d78/d86/d9d 0 2024-03-22T10:42:45.842 INFO:tasks.workunit.client.1.smithi150.stdout:2/161: mknod d12/d19/c31 0 2024-03-22T10:42:45.843 INFO:tasks.workunit.client.0.smithi115.stdout:5/529: fsync d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f69 0 2024-03-22T10:42:45.843 INFO:tasks.workunit.client.1.smithi150.stdout:2/162: chown f10 24180 1 2024-03-22T10:42:45.844 INFO:tasks.workunit.client.1.smithi150.stdout:2/163: chown d12/d19 1 1 2024-03-22T10:42:45.846 INFO:tasks.workunit.client.0.smithi115.stdout:3/538: truncate d3/d82/f27 1488541 0 2024-03-22T10:42:45.847 INFO:tasks.workunit.client.0.smithi115.stdout:3/539: readlink d3/l10 0 2024-03-22T10:42:45.852 INFO:tasks.workunit.client.0.smithi115.stdout:2/522: write da/d25/d30/d4f/d54/f7a [842375,80320] 0 2024-03-22T10:42:45.854 INFO:tasks.workunit.client.0.smithi115.stdout:7/552: dread d2/d7/d12/d10/d23/f84 [0,4194304] 0 2024-03-22T10:42:45.855 INFO:tasks.workunit.client.0.smithi115.stdout:3/540: creat d3/d82/d54/fae x:0 0 0 2024-03-22T10:42:45.856 INFO:tasks.workunit.client.0.smithi115.stdout:4/492: chown d1/d16/d35/d40/d5c/c13 3346 1 2024-03-22T10:42:45.868 INFO:tasks.workunit.client.0.smithi115.stdout:6/500: truncate d4/d13/f25 2255366 0 2024-03-22T10:42:45.870 INFO:tasks.workunit.client.0.smithi115.stdout:5/530: creat d2/df/d13/fb3 x:0 0 0 2024-03-22T10:42:45.872 INFO:tasks.workunit.client.0.smithi115.stdout:1/622: write d0/d8/de/d1a/f24 [2316011,6845] 0 2024-03-22T10:42:45.875 INFO:tasks.workunit.client.0.smithi115.stdout:1/623: chown d0/d22/d75/d82 73303298 1 2024-03-22T10:42:45.876 INFO:tasks.workunit.client.0.smithi115.stdout:7/553: write d2/d7/d12/d10/d19/d42/f9c [251967,98747] 0 2024-03-22T10:42:45.876 INFO:tasks.workunit.client.0.smithi115.stdout:8/738: dwrite d6/dd/d24/d4a/d7b/fba [0,4194304] 0 2024-03-22T10:42:45.879 INFO:tasks.workunit.client.0.smithi115.stdout:9/564: dread d6/d7/d48/d70/f73 [0,4194304] 0 2024-03-22T10:42:45.879 INFO:tasks.workunit.client.0.smithi115.stdout:2/523: symlink da/d1c/d67/d78/l9e 0 2024-03-22T10:42:45.884 INFO:tasks.workunit.client.1.smithi150.stdout:2/164: rmdir d12/d19 39 2024-03-22T10:42:45.884 INFO:tasks.workunit.client.0.smithi115.stdout:7/554: dread - d2/d3a/d6e/d78/dbf/dae/dbb/f7a zero size 2024-03-22T10:42:45.885 INFO:tasks.workunit.client.0.smithi115.stdout:2/524: chown da/c10 28402773 1 2024-03-22T10:42:45.886 INFO:tasks.workunit.client.1.smithi150.stdout:2/165: chown d12/l1e 65945 1 2024-03-22T10:42:45.890 INFO:tasks.workunit.client.1.smithi150.stdout:2/166: unlink d12/d21/l2f 0 2024-03-22T10:42:45.893 INFO:tasks.workunit.client.0.smithi115.stdout:2/525: write da/d25/d3d/f3c [158278,32367] 0 2024-03-22T10:42:45.893 INFO:tasks.workunit.client.0.smithi115.stdout:5/531: dread d2/d15/d1b/d7c/d38/d43/d49/fa9 [0,4194304] 0 2024-03-22T10:42:45.896 INFO:tasks.workunit.client.0.smithi115.stdout:8/739: creat d6/d49/d88/fef x:0 0 0 2024-03-22T10:42:45.898 INFO:tasks.workunit.client.0.smithi115.stdout:9/565: rename d6/d7/d19/f7a to d6/d7/d20/d22/d23/fc7 0 2024-03-22T10:42:45.901 INFO:tasks.workunit.client.1.smithi150.stdout:2/167: dread f10 [0,4194304] 0 2024-03-22T10:42:45.903 INFO:tasks.workunit.client.1.smithi150.stdout:1/130: sync 2024-03-22T10:42:45.903 INFO:tasks.workunit.client.1.smithi150.stdout:2/168: chown d12/d19/c31 562 1 2024-03-22T10:42:45.905 INFO:tasks.workunit.client.0.smithi115.stdout:5/532: write d2/d15/d1b/d7c/d38/d4a/f75 [3772,44475] 0 2024-03-22T10:42:45.905 INFO:tasks.workunit.client.1.smithi150.stdout:2/169: mknod d12/d16/c32 0 2024-03-22T10:42:45.906 INFO:tasks.workunit.client.1.smithi150.stdout:1/131: symlink d1/da/l25 0 2024-03-22T10:42:45.907 INFO:tasks.workunit.client.0.smithi115.stdout:5/533: stat d2/df/d13/c17 0 2024-03-22T10:42:45.907 INFO:tasks.workunit.client.0.smithi115.stdout:4/493: unlink d1/d16/d35/d40/d5c/ce 0 2024-03-22T10:42:45.908 INFO:tasks.workunit.client.0.smithi115.stdout:1/624: dwrite d0/d22/d75/d82/f9a [0,4194304] 0 2024-03-22T10:42:45.910 INFO:tasks.workunit.client.1.smithi150.stdout:2/170: creat d12/d16/d2c/f33 x:0 0 0 2024-03-22T10:42:45.912 INFO:tasks.workunit.client.1.smithi150.stdout:2/171: truncate d12/d16/f22 1622964 0 2024-03-22T10:42:45.914 INFO:tasks.workunit.client.1.smithi150.stdout:2/172: write d12/d16/f22 [86728,33315] 0 2024-03-22T10:42:45.915 INFO:tasks.workunit.client.0.smithi115.stdout:7/555: dwrite d2/d7/d12/d10/f40 [0,4194304] 0 2024-03-22T10:42:45.916 INFO:tasks.workunit.client.1.smithi150.stdout:2/173: creat d12/d19/f34 x:0 0 0 2024-03-22T10:42:45.918 INFO:tasks.workunit.client.1.smithi150.stdout:1/132: dread d1/da/fe [0,4194304] 0 2024-03-22T10:42:45.919 INFO:tasks.workunit.client.1.smithi150.stdout:2/174: mkdir d12/d35 0 2024-03-22T10:42:45.919 INFO:tasks.workunit.client.1.smithi150.stdout:1/133: chown d1/f7 57018 1 2024-03-22T10:42:45.920 INFO:tasks.workunit.client.0.smithi115.stdout:2/526: truncate da/d25/d3d/d57/f5d 640857 0 2024-03-22T10:42:45.920 INFO:tasks.workunit.client.0.smithi115.stdout:3/541: getdents d3/d36/d41 0 2024-03-22T10:42:45.921 INFO:tasks.workunit.client.0.smithi115.stdout:8/740: creat d6/dd/d12/de3/ff0 x:0 0 0 2024-03-22T10:42:45.923 INFO:tasks.workunit.client.1.smithi150.stdout:2/175: rename d12/d19/f1c to d12/d19/d2d/f36 0 2024-03-22T10:42:45.924 INFO:tasks.workunit.client.0.smithi115.stdout:7/556: dread - d2/d7/d12/d10/d19/d42/d46/f7f zero size 2024-03-22T10:42:45.925 INFO:tasks.workunit.client.1.smithi150.stdout:1/134: rmdir d1/d19/d1a/d1f 0 2024-03-22T10:42:45.927 INFO:tasks.workunit.client.0.smithi115.stdout:6/501: truncate d4/d11/d12/d1c/f7c 501176 0 2024-03-22T10:42:45.932 INFO:tasks.workunit.client.0.smithi115.stdout:6/502: truncate d4/d11/d12/f9c 975379 0 2024-03-22T10:42:45.933 INFO:tasks.workunit.client.0.smithi115.stdout:1/625: creat d0/d8/de/d1a/d91/fd2 x:0 0 0 2024-03-22T10:42:45.933 INFO:tasks.workunit.client.0.smithi115.stdout:8/741: read d6/dd/d12/f94 [81903,3410] 0 2024-03-22T10:42:45.934 INFO:tasks.workunit.client.0.smithi115.stdout:8/742: chown d6/d49/d88/f8a 15834096 1 2024-03-22T10:42:45.936 INFO:tasks.workunit.client.0.smithi115.stdout:2/527: truncate da/d25/f37 854696 0 2024-03-22T10:42:45.937 INFO:tasks.workunit.client.0.smithi115.stdout:8/743: write d6/dd/d12/de3/fe7 [998489,90984] 0 2024-03-22T10:42:45.939 INFO:tasks.workunit.client.0.smithi115.stdout:8/744: readlink d6/dd/d24/laa 0 2024-03-22T10:42:45.941 INFO:tasks.workunit.client.0.smithi115.stdout:2/528: read da/f15 [2444173,8220] 0 2024-03-22T10:42:45.941 INFO:tasks.workunit.client.1.smithi150.stdout:2/176: dwrite f11 [0,4194304] 0 2024-03-22T10:42:45.942 INFO:tasks.workunit.client.1.smithi150.stdout:1/135: dread d1/da/fe [0,4194304] 0 2024-03-22T10:42:45.944 INFO:tasks.workunit.client.0.smithi115.stdout:9/566: unlink d6/d7/d48/lb8 0 2024-03-22T10:42:45.947 INFO:tasks.workunit.client.1.smithi150.stdout:2/177: link d12/f26 d12/f37 0 2024-03-22T10:42:45.949 INFO:tasks.workunit.client.0.smithi115.stdout:8/745: truncate d6/fad 226677 0 2024-03-22T10:42:45.950 INFO:tasks.workunit.client.1.smithi150.stdout:2/178: mknod d12/d35/c38 0 2024-03-22T10:42:45.951 INFO:tasks.workunit.client.0.smithi115.stdout:6/503: rename d4/d11/f52 to d4/d36/d48/f9e 0 2024-03-22T10:42:45.953 INFO:tasks.workunit.client.0.smithi115.stdout:4/494: dwrite d1/d16/f64 [0,4194304] 0 2024-03-22T10:42:45.954 INFO:tasks.workunit.client.1.smithi150.stdout:2/179: write d12/f37 [4754222,41087] 0 2024-03-22T10:42:45.956 INFO:tasks.workunit.client.1.smithi150.stdout:2/180: fdatasync d12/d19/d1a/f2b 0 2024-03-22T10:42:45.959 INFO:tasks.workunit.client.1.smithi150.stdout:2/181: chown d12/d19/d1a/f2b 6 1 2024-03-22T10:42:45.959 INFO:tasks.workunit.client.1.smithi150.stdout:1/136: truncate d1/da/f13 1719871 0 2024-03-22T10:42:45.960 INFO:tasks.workunit.client.0.smithi115.stdout:9/567: unlink d6/d8/l18 0 2024-03-22T10:42:45.961 INFO:tasks.workunit.client.1.smithi150.stdout:2/182: stat d12/d19/f34 0 2024-03-22T10:42:45.964 INFO:tasks.workunit.client.0.smithi115.stdout:7/557: creat d2/d7/d12/d10/d19/fc1 x:0 0 0 2024-03-22T10:42:45.966 INFO:tasks.workunit.client.0.smithi115.stdout:5/534: link d2/d15/c1d d2/df/cb4 0 2024-03-22T10:42:45.969 INFO:tasks.workunit.client.1.smithi150.stdout:2/183: link d12/l1e d12/d16/l39 0 2024-03-22T10:42:45.970 INFO:tasks.workunit.client.1.smithi150.stdout:2/184: stat d12/d19/l1f 0 2024-03-22T10:42:45.972 INFO:tasks.workunit.client.0.smithi115.stdout:5/535: truncate d2/d15/d1b/d7c/d38/d43/f82 497796 0 2024-03-22T10:42:45.973 INFO:tasks.workunit.client.1.smithi150.stdout:1/137: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:45.976 INFO:tasks.workunit.client.0.smithi115.stdout:4/495: mknod d1/d16/d28/ca5 0 2024-03-22T10:42:45.977 INFO:tasks.workunit.client.0.smithi115.stdout:1/626: write d0/d12/d17/d37/d5c/f8c [101575,111011] 0 2024-03-22T10:42:45.978 INFO:tasks.workunit.client.0.smithi115.stdout:9/568: creat d6/d7/d20/d22/d3f/fc8 x:0 0 0 2024-03-22T10:42:45.980 INFO:tasks.workunit.client.0.smithi115.stdout:9/569: write d6/d7/f78 [1397207,69488] 0 2024-03-22T10:42:45.981 INFO:tasks.workunit.client.1.smithi150.stdout:1/138: rename d1/da/d15/f1b to d1/d19/f26 0 2024-03-22T10:42:45.981 INFO:tasks.workunit.client.0.smithi115.stdout:7/558: symlink d2/lc2 0 2024-03-22T10:42:45.982 INFO:tasks.workunit.client.1.smithi150.stdout:1/139: fsync d1/da/fe 0 2024-03-22T10:42:45.984 INFO:tasks.workunit.client.0.smithi115.stdout:5/536: mkdir d2/d15/db5 0 2024-03-22T10:42:45.984 INFO:tasks.workunit.client.1.smithi150.stdout:2/185: dread d12/f26 [0,4194304] 0 2024-03-22T10:42:45.985 INFO:tasks.workunit.client.0.smithi115.stdout:9/570: write d6/d8/d3c/f54 [1298349,88932] 0 2024-03-22T10:42:45.985 INFO:tasks.workunit.client.1.smithi150.stdout:2/186: readlink d12/d16/l17 0 2024-03-22T10:42:45.986 INFO:tasks.workunit.client.1.smithi150.stdout:5/85: sync 2024-03-22T10:42:45.986 INFO:tasks.workunit.client.1.smithi150.stdout:0/155: sync 2024-03-22T10:42:45.988 INFO:tasks.workunit.client.1.smithi150.stdout:2/187: write d12/d16/d2c/f33 [122594,23145] 0 2024-03-22T10:42:45.989 INFO:tasks.workunit.client.0.smithi115.stdout:1/627: fsync d0/d68/f7f 0 2024-03-22T10:42:45.990 INFO:tasks.workunit.client.1.smithi150.stdout:5/86: write d0/d8/dc/de/f13 [820440,76136] 0 2024-03-22T10:42:45.990 INFO:tasks.workunit.client.0.smithi115.stdout:9/571: write d6/d7/d20/f57 [4507146,32592] 0 2024-03-22T10:42:45.991 INFO:tasks.workunit.client.1.smithi150.stdout:0/156: symlink d0/d1/l32 0 2024-03-22T10:42:45.991 INFO:tasks.workunit.client.1.smithi150.stdout:2/188: symlink d12/d19/d2d/l3a 0 2024-03-22T10:42:45.992 INFO:tasks.workunit.client.0.smithi115.stdout:2/529: getdents da/d1c/d67/d78 0 2024-03-22T10:42:45.993 INFO:tasks.workunit.client.0.smithi115.stdout:8/746: dwrite d6/dd/d24/d4a/d7b/fa2 [0,4194304] 0 2024-03-22T10:42:45.995 INFO:tasks.workunit.client.0.smithi115.stdout:7/559: write d2/d7/d12/d10/f25 [3353668,58596] 0 2024-03-22T10:42:45.995 INFO:tasks.workunit.client.1.smithi150.stdout:5/87: getdents d0/d8/dc 0 2024-03-22T10:42:45.996 INFO:tasks.workunit.client.1.smithi150.stdout:5/88: chown d0/d8/dc/de/d14 6931312 1 2024-03-22T10:42:46.000 INFO:tasks.workunit.client.1.smithi150.stdout:5/89: unlink d0/d8/dc/de/d14/f17 0 2024-03-22T10:42:46.006 INFO:tasks.workunit.client.0.smithi115.stdout:1/628: creat d0/d8/db/d25/d9f/fd3 x:0 0 0 2024-03-22T10:42:46.007 INFO:tasks.workunit.client.0.smithi115.stdout:6/504: write d4/d13/d33/f59 [1921668,100787] 0 2024-03-22T10:42:46.010 INFO:tasks.workunit.client.0.smithi115.stdout:4/496: rmdir d1/d16 39 2024-03-22T10:42:46.011 INFO:tasks.workunit.client.1.smithi150.stdout:2/189: dwrite d12/d19/d1a/f30 [0,4194304] 0 2024-03-22T10:42:46.013 INFO:tasks.workunit.client.0.smithi115.stdout:9/572: mkdir d6/d7/d48/d53/dc9 0 2024-03-22T10:42:46.013 INFO:tasks.workunit.client.0.smithi115.stdout:8/747: creat d6/dd/d24/d4a/d7b/d8f/dd6/ddb/ff1 x:0 0 0 2024-03-22T10:42:46.014 INFO:tasks.workunit.client.1.smithi150.stdout:0/157: dwrite d0/d1/d7/f20 [0,4194304] 0 2024-03-22T10:42:46.014 INFO:tasks.workunit.client.1.smithi150.stdout:2/190: readlink d12/d19/l20 0 2024-03-22T10:42:46.024 INFO:tasks.workunit.client.1.smithi150.stdout:2/191: truncate d12/f26 4949103 0 2024-03-22T10:42:46.025 INFO:tasks.workunit.client.0.smithi115.stdout:1/629: dwrite d0/d22/f5e [0,4194304] 0 2024-03-22T10:42:46.025 INFO:tasks.workunit.client.1.smithi150.stdout:5/90: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:42:46.030 INFO:tasks.workunit.client.0.smithi115.stdout:5/537: dwrite d2/df/d3b/f2a [0,4194304] 0 2024-03-22T10:42:46.032 INFO:tasks.workunit.client.1.smithi150.stdout:5/91: symlink d0/d8/l1c 0 2024-03-22T10:42:46.032 INFO:tasks.workunit.client.0.smithi115.stdout:7/560: rename d2/d7/d12/d10/d19/d42/d46/d61 to d2/d62/dc3 0 2024-03-22T10:42:46.034 INFO:tasks.workunit.client.1.smithi150.stdout:5/92: chown d0/f6 0 1 2024-03-22T10:42:46.035 INFO:tasks.workunit.client.0.smithi115.stdout:7/561: write d2/d7/d12/d10/d19/d42/d46/f5e [829093,52488] 0 2024-03-22T10:42:46.037 INFO:tasks.workunit.client.0.smithi115.stdout:4/497: creat d1/d66/d82/fa6 x:0 0 0 2024-03-22T10:42:46.039 INFO:tasks.workunit.client.1.smithi150.stdout:0/158: truncate d0/d1/d7/db/d10/f13 2852675 0 2024-03-22T10:42:46.039 INFO:tasks.workunit.client.1.smithi150.stdout:5/93: mkdir d0/d8/dc/d1d 0 2024-03-22T10:42:46.040 INFO:tasks.workunit.client.0.smithi115.stdout:7/562: chown d2/d3a/d6e/d78/dbf/dae/lb9 566598 1 2024-03-22T10:42:46.041 INFO:tasks.workunit.client.1.smithi150.stdout:0/159: stat d0/d1/d7/ff 0 2024-03-22T10:42:46.047 INFO:tasks.workunit.client.1.smithi150.stdout:0/160: mknod d0/d12/c33 0 2024-03-22T10:42:46.048 INFO:tasks.workunit.client.1.smithi150.stdout:5/94: rename d0/d8/dc/c1a to d0/d8/dc/de/d18/c1e 0 2024-03-22T10:42:46.050 INFO:tasks.workunit.client.0.smithi115.stdout:9/573: symlink d6/d7/d48/d70/da9/lca 0 2024-03-22T10:42:46.050 INFO:tasks.workunit.client.0.smithi115.stdout:9/574: chown d6/f2e 0 1 2024-03-22T10:42:46.051 INFO:tasks.workunit.client.1.smithi150.stdout:2/192: dwrite d12/d19/f1b [0,4194304] 0 2024-03-22T10:42:46.053 INFO:tasks.workunit.client.1.smithi150.stdout:2/193: chown d12/d35 12535 1 2024-03-22T10:42:46.055 INFO:tasks.workunit.client.1.smithi150.stdout:2/194: readlink d12/d19/l24 0 2024-03-22T10:42:46.056 INFO:tasks.workunit.client.0.smithi115.stdout:6/505: mkdir d4/d11/d12/d1c/d8a/d98/d9f 0 2024-03-22T10:42:46.057 INFO:tasks.workunit.client.1.smithi150.stdout:2/195: write f11 [6594499,90775] 0 2024-03-22T10:42:46.058 INFO:tasks.workunit.client.0.smithi115.stdout:1/630: unlink d0/d8/de/l45 0 2024-03-22T10:42:46.060 INFO:tasks.workunit.client.0.smithi115.stdout:6/506: truncate d4/d13/d33/f59 2834100 0 2024-03-22T10:42:46.061 INFO:tasks.workunit.client.1.smithi150.stdout:0/161: symlink d0/d1/d7/db/d10/l34 0 2024-03-22T10:42:46.061 INFO:tasks.workunit.client.0.smithi115.stdout:5/538: symlink d2/d15/d1b/d7c/d84/lb6 0 2024-03-22T10:42:46.066 INFO:tasks.workunit.client.0.smithi115.stdout:4/498: mknod d1/d66/d82/ca7 0 2024-03-22T10:42:46.066 INFO:tasks.workunit.client.0.smithi115.stdout:5/539: write d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f9b [724724,70966] 0 2024-03-22T10:42:46.066 INFO:tasks.workunit.client.0.smithi115.stdout:1/631: readlink d0/d12/l29 0 2024-03-22T10:42:46.068 INFO:tasks.workunit.client.1.smithi150.stdout:9/149: sync 2024-03-22T10:42:46.069 INFO:tasks.workunit.client.1.smithi150.stdout:4/157: sync 2024-03-22T10:42:46.069 INFO:tasks.workunit.client.1.smithi150.stdout:7/145: sync 2024-03-22T10:42:46.069 INFO:tasks.workunit.client.1.smithi150.stdout:3/137: sync 2024-03-22T10:42:46.069 INFO:tasks.workunit.client.1.smithi150.stdout:8/154: sync 2024-03-22T10:42:46.069 INFO:tasks.workunit.client.1.smithi150.stdout:6/116: sync 2024-03-22T10:42:46.069 INFO:tasks.workunit.client.1.smithi150.stdout:2/196: chown d12/l14 412700 1 2024-03-22T10:42:46.070 INFO:tasks.workunit.client.0.smithi115.stdout:1/632: chown d0/d8/de/d1a/d91/dc7 151 1 2024-03-22T10:42:46.071 INFO:tasks.workunit.client.1.smithi150.stdout:0/162: mkdir d0/d1/d7/db/d10/d1e/d35 0 2024-03-22T10:42:46.072 INFO:tasks.workunit.client.0.smithi115.stdout:4/499: mkdir d1/d30/d4c/d86/d9f/d75/d99/da8 0 2024-03-22T10:42:46.073 INFO:tasks.workunit.client.1.smithi150.stdout:2/197: dread - d12/d19/f34 zero size 2024-03-22T10:42:46.074 INFO:tasks.workunit.client.1.smithi150.stdout:3/138: rename l2 to d5/l29 0 2024-03-22T10:42:46.077 INFO:tasks.workunit.client.1.smithi150.stdout:7/146: dread - d2/d5/d7/d9/db/f1a zero size 2024-03-22T10:42:46.077 INFO:tasks.workunit.client.0.smithi115.stdout:1/633: creat d0/d12/d17/d37/db5/fd4 x:0 0 0 2024-03-22T10:42:46.080 INFO:tasks.workunit.client.0.smithi115.stdout:1/634: write d0/d22/f46 [1758878,70979] 0 2024-03-22T10:42:46.082 INFO:tasks.workunit.client.0.smithi115.stdout:1/635: chown d0/d8/ld 10 1 2024-03-22T10:42:46.083 INFO:tasks.workunit.client.1.smithi150.stdout:4/158: unlink d0/c5 0 2024-03-22T10:42:46.084 INFO:tasks.workunit.client.1.smithi150.stdout:8/155: fdatasync d0/d2/d19/d29/d2e/f38 0 2024-03-22T10:42:46.087 INFO:tasks.workunit.client.1.smithi150.stdout:0/163: stat d0/d1/d7/d11/d26 0 2024-03-22T10:42:46.087 INFO:tasks.workunit.client.1.smithi150.stdout:3/139: unlink d5/da/l1d 0 2024-03-22T10:42:46.087 INFO:tasks.workunit.client.0.smithi115.stdout:8/748: dwrite d6/f59 [0,4194304] 0 2024-03-22T10:42:46.089 INFO:tasks.workunit.client.1.smithi150.stdout:6/117: rename db/fc to db/dd/d15/d28/f2b 0 2024-03-22T10:42:46.091 INFO:tasks.workunit.client.1.smithi150.stdout:5/95: dwrite d0/f6 [0,4194304] 0 2024-03-22T10:42:46.097 INFO:tasks.workunit.client.1.smithi150.stdout:7/147: mknod d2/d5/d7/c32 0 2024-03-22T10:42:46.098 INFO:tasks.workunit.client.1.smithi150.stdout:3/140: chown d5/da/c11 465296 1 2024-03-22T10:42:46.100 INFO:tasks.workunit.client.1.smithi150.stdout:0/164: chown d0/d1/d7/d11/d26/c2a 84790 1 2024-03-22T10:42:46.101 INFO:tasks.workunit.client.1.smithi150.stdout:8/156: mkdir d0/d2/d19/d29/d2f/df/d1b/d3d 0 2024-03-22T10:42:46.102 INFO:tasks.workunit.client.1.smithi150.stdout:9/150: dwrite d1/d19/f26 [0,4194304] 0 2024-03-22T10:42:46.103 INFO:tasks.workunit.client.0.smithi115.stdout:8/749: getdents d6/dd/d12/dc6 0 2024-03-22T10:42:46.103 INFO:tasks.workunit.client.1.smithi150.stdout:5/96: write d0/f3 [3938106,105731] 0 2024-03-22T10:42:46.109 INFO:tasks.workunit.client.0.smithi115.stdout:6/507: dread d4/d13/d33/f42 [0,4194304] 0 2024-03-22T10:42:46.110 INFO:tasks.workunit.client.0.smithi115.stdout:5/540: dread d2/d15/fac [4194304,4194304] 0 2024-03-22T10:42:46.110 INFO:tasks.workunit.client.1.smithi150.stdout:7/148: chown d2/d5/d7/d8/cd 206149 1 2024-03-22T10:42:46.112 INFO:tasks.workunit.client.1.smithi150.stdout:0/165: write d0/d1/d7/db/f2b [559867,119863] 0 2024-03-22T10:42:46.112 INFO:tasks.workunit.client.0.smithi115.stdout:1/636: dwrite d0/d12/d17/f33 [0,4194304] 0 2024-03-22T10:42:46.114 INFO:tasks.workunit.client.0.smithi115.stdout:1/637: fsync d0/d8/d4d/d56/dbd/fbf 0 2024-03-22T10:42:46.118 INFO:tasks.workunit.client.1.smithi150.stdout:3/141: unlink d5/da/l1f 0 2024-03-22T10:42:46.119 INFO:tasks.workunit.client.0.smithi115.stdout:6/508: rename d4/d11/d12/d1c/f7c to d4/d36/d48/fa0 0 2024-03-22T10:42:46.122 INFO:tasks.workunit.client.1.smithi150.stdout:6/118: dwrite db/dd/f14 [0,4194304] 0 2024-03-22T10:42:46.123 INFO:tasks.workunit.client.1.smithi150.stdout:4/159: rmdir d0 39 2024-03-22T10:42:46.127 INFO:tasks.workunit.client.0.smithi115.stdout:1/638: symlink d0/d8/db/d8e/ld5 0 2024-03-22T10:42:46.127 INFO:tasks.workunit.client.1.smithi150.stdout:8/157: symlink d0/d2/d19/d29/d2f/df/d1b/l3e 0 2024-03-22T10:42:46.134 INFO:tasks.workunit.client.1.smithi150.stdout:5/97: dread d0/f5 [0,4194304] 0 2024-03-22T10:42:46.137 INFO:tasks.workunit.client.1.smithi150.stdout:7/149: dwrite d2/d5/d7/d9/f28 [0,4194304] 0 2024-03-22T10:42:46.138 INFO:tasks.workunit.client.1.smithi150.stdout:5/98: read d0/d8/dc/de/f15 [1276270,53936] 0 2024-03-22T10:42:46.141 INFO:tasks.workunit.client.1.smithi150.stdout:9/151: symlink d1/d8/de/df/d20/l34 0 2024-03-22T10:42:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:45 smithi150 ceph-mon[34518]: pgmap v11: 65 pgs: 65 active+clean; 920 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 126 MiB/s wr, 280 op/s 2024-03-22T10:42:46.149 INFO:tasks.workunit.client.1.smithi150.stdout:3/142: rename d5/da/l15 to d5/d22/l2a 0 2024-03-22T10:42:46.151 INFO:tasks.workunit.client.1.smithi150.stdout:3/143: write d5/da/ff [450293,3402] 0 2024-03-22T10:42:46.151 INFO:tasks.workunit.client.0.smithi115.stdout:8/750: dread d6/dd/f76 [0,4194304] 0 2024-03-22T10:42:46.152 INFO:tasks.workunit.client.1.smithi150.stdout:6/119: rename db/dd/f26 to db/dd/f2c 0 2024-03-22T10:42:46.152 INFO:tasks.workunit.client.1.smithi150.stdout:8/158: rmdir d0/d2/d19/d29/d2f/df 39 2024-03-22T10:42:46.155 INFO:tasks.workunit.client.1.smithi150.stdout:8/159: write d0/d2/d19/f25 [2686613,63804] 0 2024-03-22T10:42:46.158 INFO:tasks.workunit.client.0.smithi115.stdout:1/639: dwrite d0/d8/de/f26 [0,4194304] 0 2024-03-22T10:42:46.158 INFO:tasks.workunit.client.0.smithi115.stdout:5/541: dread d2/d15/fac [0,4194304] 0 2024-03-22T10:42:46.159 INFO:tasks.workunit.client.1.smithi150.stdout:7/150: creat d2/d5/d7/de/f33 x:0 0 0 2024-03-22T10:42:46.161 INFO:tasks.workunit.client.0.smithi115.stdout:5/542: stat d2/d15/d32 0 2024-03-22T10:42:46.161 INFO:tasks.workunit.client.1.smithi150.stdout:9/152: symlink d1/d8/de/df/d24/l35 0 2024-03-22T10:42:46.162 INFO:tasks.workunit.client.1.smithi150.stdout:0/166: truncate d0/d1/d7/d11/d26/f2e 1633333 0 2024-03-22T10:42:46.163 INFO:tasks.workunit.client.0.smithi115.stdout:1/640: symlink d0/d8/d4d/d56/db6/ld6 0 2024-03-22T10:42:46.165 INFO:tasks.workunit.client.0.smithi115.stdout:8/751: link d6/dd/d24/d4a/d7b/d8f/da0/fc3 d6/dd/d24/ff2 0 2024-03-22T10:42:46.166 INFO:tasks.workunit.client.1.smithi150.stdout:3/144: creat d5/d22/f2b x:0 0 0 2024-03-22T10:42:46.167 INFO:tasks.workunit.client.1.smithi150.stdout:9/153: chown d1/d8/de/df/d20/d32/f2a 116793 1 2024-03-22T10:42:46.167 INFO:tasks.workunit.client.0.smithi115.stdout:8/752: write d6/d49/d88/f8a [1295504,77098] 0 2024-03-22T10:42:46.169 INFO:tasks.workunit.client.1.smithi150.stdout:6/120: symlink db/l2d 0 2024-03-22T10:42:46.172 INFO:tasks.workunit.client.1.smithi150.stdout:8/160: readlink d0/d2/d19/l33 0 2024-03-22T10:42:46.174 INFO:tasks.workunit.client.1.smithi150.stdout:8/161: truncate d0/d2/fe 1025795 0 2024-03-22T10:42:46.175 INFO:tasks.workunit.client.0.smithi115.stdout:1/641: mkdir d0/d8/db/dd7 0 2024-03-22T10:42:46.176 INFO:tasks.workunit.client.1.smithi150.stdout:7/151: unlink d2/d5/d7/d8/c29 0 2024-03-22T10:42:46.176 INFO:tasks.workunit.client.1.smithi150.stdout:5/99: creat d0/d8/dc/d1d/f1f x:0 0 0 2024-03-22T10:42:46.177 INFO:tasks.workunit.client.0.smithi115.stdout:5/543: creat d2/d15/d1b/d7c/d38/d43/d49/fb7 x:0 0 0 2024-03-22T10:42:46.177 INFO:tasks.workunit.client.0.smithi115.stdout:8/753: symlink d6/dd/d24/d4a/d7b/d8f/lf3 0 2024-03-22T10:42:46.179 INFO:tasks.workunit.client.0.smithi115.stdout:1/642: mkdir d0/d12/d17/d67/dd8 0 2024-03-22T10:42:46.180 INFO:tasks.workunit.client.1.smithi150.stdout:4/160: rename d0/f1e to d0/d2f/f3b 0 2024-03-22T10:42:46.180 INFO:tasks.workunit.client.0.smithi115.stdout:5/544: write d2/df/d13/f4c [781480,35319] 0 2024-03-22T10:42:46.181 INFO:tasks.workunit.client.0.smithi115.stdout:6/509: dread d4/d11/d12/f4e [0,4194304] 0 2024-03-22T10:42:46.182 INFO:tasks.workunit.client.1.smithi150.stdout:4/161: fdatasync d0/f9 0 2024-03-22T10:42:46.183 INFO:tasks.workunit.client.0.smithi115.stdout:8/754: rename d6/f28 to d6/ff4 0 2024-03-22T10:42:46.184 INFO:tasks.workunit.client.1.smithi150.stdout:3/145: write d5/da/f19 [128707,55023] 0 2024-03-22T10:42:46.186 INFO:tasks.workunit.client.0.smithi115.stdout:8/755: dread - d6/dd/d24/d4a/d7b/fe0 zero size 2024-03-22T10:42:46.186 INFO:tasks.workunit.client.0.smithi115.stdout:5/545: fdatasync d2/d15/d1b/d7c/d38/f77 0 2024-03-22T10:42:46.187 INFO:tasks.workunit.client.0.smithi115.stdout:8/756: readlink d6/d49/l67 0 2024-03-22T10:42:46.187 INFO:tasks.workunit.client.1.smithi150.stdout:6/121: mkdir db/dd/d2e 0 2024-03-22T10:42:46.188 INFO:tasks.workunit.client.0.smithi115.stdout:6/510: truncate d4/d36/d48/d72/f76 1956879 0 2024-03-22T10:42:46.190 INFO:tasks.workunit.client.1.smithi150.stdout:6/122: fsync db/dd/d15/f20 0 2024-03-22T10:42:46.191 INFO:tasks.workunit.client.1.smithi150.stdout:7/152: read d2/d5/d7/d8/f2d [995011,124192] 0 2024-03-22T10:42:46.192 INFO:tasks.workunit.client.1.smithi150.stdout:0/167: mknod d0/d1/d7/d11/c36 0 2024-03-22T10:42:46.193 INFO:tasks.workunit.client.0.smithi115.stdout:6/511: mknod d4/d13/d33/d58/ca1 0 2024-03-22T10:42:46.195 INFO:tasks.workunit.client.1.smithi150.stdout:8/162: dwrite d0/fd [0,4194304] 0 2024-03-22T10:42:46.195 INFO:tasks.workunit.client.0.smithi115.stdout:5/546: rename d2/d15/d1b/d7c/d84/f9a to d2/d15/d1b/d7c/d84/d89/fb8 0 2024-03-22T10:42:46.196 INFO:tasks.workunit.client.0.smithi115.stdout:8/757: creat d6/ff5 x:0 0 0 2024-03-22T10:42:46.197 INFO:tasks.workunit.client.1.smithi150.stdout:4/162: unlink d0/de/f12 0 2024-03-22T10:42:46.200 INFO:tasks.workunit.client.0.smithi115.stdout:1/643: truncate d0/d22/f46 1359133 0 2024-03-22T10:42:46.202 INFO:tasks.workunit.client.1.smithi150.stdout:3/146: symlink d5/l2c 0 2024-03-22T10:42:46.204 INFO:tasks.workunit.client.1.smithi150.stdout:0/168: mkdir d0/d1/d7/db/d10/d1e/d37 0 2024-03-22T10:42:46.204 INFO:tasks.workunit.client.1.smithi150.stdout:4/163: creat d0/de/f3c x:0 0 0 2024-03-22T10:42:46.205 INFO:tasks.workunit.client.1.smithi150.stdout:3/147: mknod d5/d22/c2d 0 2024-03-22T10:42:46.206 INFO:tasks.workunit.client.0.smithi115.stdout:1/644: rename d0/f1 to d0/d12/d17/d37/db5/fd9 0 2024-03-22T10:42:46.207 INFO:tasks.workunit.client.1.smithi150.stdout:3/148: rmdir d5 39 2024-03-22T10:42:46.208 INFO:tasks.workunit.client.0.smithi115.stdout:6/512: dread d4/d11/f54 [0,4194304] 0 2024-03-22T10:42:46.209 INFO:tasks.workunit.client.1.smithi150.stdout:1/140: sync 2024-03-22T10:42:46.211 INFO:tasks.workunit.client.0.smithi115.stdout:1/645: unlink d0/d8/d4d/d56/db6/fb7 0 2024-03-22T10:42:46.215 INFO:tasks.workunit.client.1.smithi150.stdout:9/154: dread d1/d8/de/df/f12 [0,4194304] 0 2024-03-22T10:42:46.216 INFO:tasks.workunit.client.0.smithi115.stdout:8/758: dwrite d6/dd/f92 [0,4194304] 0 2024-03-22T10:42:46.217 INFO:tasks.workunit.client.1.smithi150.stdout:1/141: mknod d1/da/d15/d18/c27 0 2024-03-22T10:42:46.217 INFO:tasks.workunit.client.0.smithi115.stdout:6/513: getdents d4/d13 0 2024-03-22T10:42:46.219 INFO:tasks.workunit.client.1.smithi150.stdout:6/123: dwrite db/dd/d15/f20 [0,4194304] 0 2024-03-22T10:42:46.220 INFO:tasks.workunit.client.1.smithi150.stdout:1/142: chown d1/c2 11210 1 2024-03-22T10:42:46.221 INFO:tasks.workunit.client.1.smithi150.stdout:6/124: chown db 4771 1 2024-03-22T10:42:46.222 INFO:tasks.workunit.client.1.smithi150.stdout:9/155: mkdir d1/d8/de/df/d36 0 2024-03-22T10:42:46.224 INFO:tasks.workunit.client.1.smithi150.stdout:1/143: chown d1/d19 240702 1 2024-03-22T10:42:46.226 INFO:tasks.workunit.client.0.smithi115.stdout:8/759: chown d6/dd/fce 29 1 2024-03-22T10:42:46.231 INFO:tasks.workunit.client.1.smithi150.stdout:9/156: symlink d1/d8/l37 0 2024-03-22T10:42:46.234 INFO:tasks.workunit.client.0.smithi115.stdout:5/547: truncate d2/df/d3b/f2a 1583629 0 2024-03-22T10:42:46.235 INFO:tasks.workunit.client.0.smithi115.stdout:5/548: readlink d2/df/d27/d50/l91 0 2024-03-22T10:42:46.237 INFO:tasks.workunit.client.0.smithi115.stdout:5/549: chown d2/df/d13/fb3 0 1 2024-03-22T10:42:46.241 INFO:tasks.workunit.client.1.smithi150.stdout:6/125: dread - db/dd/d15/f27 zero size 2024-03-22T10:42:46.242 INFO:tasks.workunit.client.0.smithi115.stdout:6/514: read - d4/d13/d33/f6c zero size 2024-03-22T10:42:46.244 INFO:tasks.workunit.client.1.smithi150.stdout:1/144: write d1/f1d [561298,95831] 0 2024-03-22T10:42:46.244 INFO:tasks.workunit.client.1.smithi150.stdout:8/163: dwrite d0/d2/fe [0,4194304] 0 2024-03-22T10:42:46.245 INFO:tasks.workunit.client.0.smithi115.stdout:6/515: symlink d4/d13/d33/d58/la2 0 2024-03-22T10:42:46.248 INFO:tasks.workunit.client.1.smithi150.stdout:9/157: symlink d1/d8/de/df/d20/l38 0 2024-03-22T10:42:46.249 INFO:tasks.workunit.client.0.smithi115.stdout:6/516: write d4/d13/d33/f42 [4242497,64115] 0 2024-03-22T10:42:46.256 INFO:tasks.workunit.client.0.smithi115.stdout:5/550: dwrite d2/df/f55 [0,4194304] 0 2024-03-22T10:42:46.258 INFO:tasks.workunit.client.1.smithi150.stdout:6/126: creat db/dd/d21/f2f x:0 0 0 2024-03-22T10:42:46.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:45 smithi115 ceph-mon[29111]: pgmap v11: 65 pgs: 65 active+clean; 920 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 126 MiB/s wr, 280 op/s 2024-03-22T10:42:46.260 INFO:tasks.workunit.client.1.smithi150.stdout:9/158: dread - d1/d19/f2f zero size 2024-03-22T10:42:46.261 INFO:tasks.workunit.client.0.smithi115.stdout:5/551: write d2/d15/d1b/d7c/d38/f3d [4074729,18170] 0 2024-03-22T10:42:46.262 INFO:tasks.workunit.client.1.smithi150.stdout:3/149: dwrite d5/f25 [0,4194304] 0 2024-03-22T10:42:46.266 INFO:tasks.workunit.client.1.smithi150.stdout:1/145: rename d1/da/fe to d1/d19/d1a/f28 0 2024-03-22T10:42:46.267 INFO:tasks.workunit.client.1.smithi150.stdout:0/169: dread d0/d1/d7/d11/d26/f2e [0,4194304] 0 2024-03-22T10:42:46.268 INFO:tasks.workunit.client.1.smithi150.stdout:4/164: dwrite d0/d1b/f2e [4194304,4194304] 0 2024-03-22T10:42:46.268 INFO:tasks.workunit.client.1.smithi150.stdout:6/127: mkdir db/dd/d15/d30 0 2024-03-22T10:42:46.270 INFO:tasks.workunit.client.0.smithi115.stdout:5/552: creat d2/d15/d1b/d7c/d38/d43/d49/fb9 x:0 0 0 2024-03-22T10:42:46.272 INFO:tasks.workunit.client.1.smithi150.stdout:3/150: dread - d5/d22/f2b zero size 2024-03-22T10:42:46.272 INFO:tasks.workunit.client.1.smithi150.stdout:7/153: dwrite d2/d5/d7/d9/f28 [4194304,4194304] 0 2024-03-22T10:42:46.276 INFO:tasks.workunit.client.1.smithi150.stdout:8/164: getdents d0/d2/d19/d29 0 2024-03-22T10:42:46.276 INFO:tasks.workunit.client.0.smithi115.stdout:5/553: symlink d2/d15/db5/lba 0 2024-03-22T10:42:46.278 INFO:tasks.workunit.client.1.smithi150.stdout:4/165: truncate d0/de/f18 4713932 0 2024-03-22T10:42:46.279 INFO:tasks.workunit.client.1.smithi150.stdout:9/159: creat d1/d8/de/df/d36/f39 x:0 0 0 2024-03-22T10:42:46.279 INFO:tasks.workunit.client.1.smithi150.stdout:3/151: write d5/da/f28 [478574,96856] 0 2024-03-22T10:42:46.292 INFO:tasks.workunit.client.0.smithi115.stdout:6/517: dread d4/d11/d12/d1c/f2d [0,4194304] 0 2024-03-22T10:42:46.309 INFO:tasks.workunit.client.0.smithi115.stdout:6/518: dwrite d4/d11/f54 [0,4194304] 0 2024-03-22T10:42:46.312 INFO:tasks.workunit.client.0.smithi115.stdout:6/519: mknod d4/d11/d12/d1c/d8a/ca3 0 2024-03-22T10:42:46.318 INFO:tasks.workunit.client.0.smithi115.stdout:6/520: rename d4/l5b to d4/d5e/la4 0 2024-03-22T10:42:46.319 INFO:tasks.workunit.client.0.smithi115.stdout:5/554: dwrite d2/d15/d1b/d7c/d38/d43/d49/fa9 [0,4194304] 0 2024-03-22T10:42:46.322 INFO:tasks.workunit.client.0.smithi115.stdout:5/555: stat d2/d15/d1b/d7c/d38/f3d 0 2024-03-22T10:42:46.324 INFO:tasks.workunit.client.0.smithi115.stdout:6/521: creat d4/d11/d12/d1c/fa5 x:0 0 0 2024-03-22T10:42:46.329 INFO:tasks.workunit.client.1.smithi150.stdout:7/154: symlink d2/d5/l34 0 2024-03-22T10:42:46.329 INFO:tasks.workunit.client.1.smithi150.stdout:8/165: write d0/d2/d19/d29/d2f/f36 [139557,2217] 0 2024-03-22T10:42:46.330 INFO:tasks.workunit.client.0.smithi115.stdout:6/522: creat d4/d11/d12/d1c/d8a/fa6 x:0 0 0 2024-03-22T10:42:46.332 INFO:tasks.workunit.client.0.smithi115.stdout:6/523: read d4/d11/d12/f4e [553151,71860] 0 2024-03-22T10:42:46.332 INFO:tasks.workunit.client.1.smithi150.stdout:9/160: symlink d1/l3a 0 2024-03-22T10:42:46.333 INFO:tasks.workunit.client.1.smithi150.stdout:3/152: write d5/f8 [770257,37853] 0 2024-03-22T10:42:46.334 INFO:tasks.workunit.client.0.smithi115.stdout:6/524: chown d4/d11/d12/l27 1 1 2024-03-22T10:42:46.337 INFO:tasks.workunit.client.1.smithi150.stdout:9/161: symlink d1/d8/de/df/d24/l3b 0 2024-03-22T10:42:46.337 INFO:tasks.workunit.client.0.smithi115.stdout:3/542: sync 2024-03-22T10:42:46.337 INFO:tasks.workunit.client.0.smithi115.stdout:0/516: sync 2024-03-22T10:42:46.340 INFO:tasks.workunit.client.0.smithi115.stdout:6/525: creat d4/d11/d12/d1c/fa7 x:0 0 0 2024-03-22T10:42:46.340 INFO:tasks.workunit.client.0.smithi115.stdout:5/556: dwrite d2/df/f55 [0,4194304] 0 2024-03-22T10:42:46.341 INFO:tasks.workunit.client.1.smithi150.stdout:9/162: creat d1/d8/de/f3c x:0 0 0 2024-03-22T10:42:46.341 INFO:tasks.workunit.client.1.smithi150.stdout:4/166: write d0/d1b/f3a [3040565,62325] 0 2024-03-22T10:42:46.341 INFO:tasks.workunit.client.0.smithi115.stdout:3/543: chown d3/d16/d29/f43 185 1 2024-03-22T10:42:46.345 INFO:tasks.workunit.client.1.smithi150.stdout:0/170: dwrite d0/d1/d7/db/d10/f13 [0,4194304] 0 2024-03-22T10:42:46.345 INFO:tasks.workunit.client.1.smithi150.stdout:4/167: write d0/f26 [333638,102336] 0 2024-03-22T10:42:46.347 INFO:tasks.workunit.client.0.smithi115.stdout:6/526: fsync d4/d13/d33/d58/f75 0 2024-03-22T10:42:46.348 INFO:tasks.workunit.client.1.smithi150.stdout:9/163: read d1/d19/f26 [4151807,126363] 0 2024-03-22T10:42:46.354 INFO:tasks.workunit.client.0.smithi115.stdout:6/527: write d4/d11/d12/d1c/f2d [520526,130256] 0 2024-03-22T10:42:46.357 INFO:tasks.workunit.client.0.smithi115.stdout:5/557: dread d2/d15/d1b/d7c/d38/d43/d49/fa9 [0,4194304] 0 2024-03-22T10:42:46.359 INFO:tasks.workunit.client.0.smithi115.stdout:3/544: link d3/d16/c74 d3/d82/d57/caf 0 2024-03-22T10:42:46.360 INFO:tasks.workunit.client.0.smithi115.stdout:6/528: symlink d4/d11/d99/la8 0 2024-03-22T10:42:46.360 INFO:tasks.workunit.client.0.smithi115.stdout:5/558: write d2/fc [2175639,56793] 0 2024-03-22T10:42:46.362 INFO:tasks.workunit.client.1.smithi150.stdout:0/171: mkdir d0/d1/d7/db/d10/d38 0 2024-03-22T10:42:46.366 INFO:tasks.workunit.client.0.smithi115.stdout:5/559: write d2/df/d3b/f7b [1981287,50737] 0 2024-03-22T10:42:46.367 INFO:tasks.workunit.client.0.smithi115.stdout:3/545: rename d3/d82/d57/c63 to d3/d19/d2b/d49/d91/da5/cb0 0 2024-03-22T10:42:46.368 INFO:tasks.workunit.client.1.smithi150.stdout:8/166: dwrite d0/d2/fa [0,4194304] 0 2024-03-22T10:42:46.368 INFO:tasks.workunit.client.1.smithi150.stdout:4/168: link d0/de/f28 d0/de/f3d 0 2024-03-22T10:42:46.368 INFO:tasks.workunit.client.1.smithi150.stdout:8/167: fdatasync d0/d2/f2b 0 2024-03-22T10:42:46.370 INFO:tasks.workunit.client.1.smithi150.stdout:3/153: dwrite d5/d22/f2b [0,4194304] 0 2024-03-22T10:42:46.371 INFO:tasks.workunit.client.0.smithi115.stdout:3/546: symlink d3/d19/d2b/d49/d91/da5/lb1 0 2024-03-22T10:42:46.374 INFO:tasks.workunit.client.0.smithi115.stdout:3/547: dread - d3/d36/d41/d33/f8d zero size 2024-03-22T10:42:46.374 INFO:tasks.workunit.client.0.smithi115.stdout:0/517: fsync d3/d4/d14/d64/f72 0 2024-03-22T10:42:46.376 INFO:tasks.workunit.client.1.smithi150.stdout:3/154: write d5/da/f19 [1545206,115416] 0 2024-03-22T10:42:46.378 INFO:tasks.workunit.client.1.smithi150.stdout:4/169: creat d0/d2f/f3e x:0 0 0 2024-03-22T10:42:46.380 INFO:tasks.workunit.client.1.smithi150.stdout:8/168: rename d0/d2/d19/d29/d2e/f38 to d0/d2/f3f 0 2024-03-22T10:42:46.381 INFO:tasks.workunit.client.0.smithi115.stdout:0/518: readlink d3/d52/l82 0 2024-03-22T10:42:46.382 INFO:tasks.workunit.client.1.smithi150.stdout:8/169: dread - d0/f34 zero size 2024-03-22T10:42:46.382 INFO:tasks.workunit.client.1.smithi150.stdout:0/172: dread d0/d1/d7/db/f2b [0,4194304] 0 2024-03-22T10:42:46.383 INFO:tasks.workunit.client.0.smithi115.stdout:6/529: truncate d4/d11/f78 2733315 0 2024-03-22T10:42:46.384 INFO:tasks.workunit.client.1.smithi150.stdout:4/170: chown d0/d1b/c1f 220406500 1 2024-03-22T10:42:46.385 INFO:tasks.workunit.client.0.smithi115.stdout:5/560: dread d2/d15/d1b/d7c/d38/f3d [4194304,4194304] 0 2024-03-22T10:42:46.387 INFO:tasks.workunit.client.1.smithi150.stdout:8/170: rename d0/d2/d19/d29/d31 to d0/d2/d19/d29/d31/d40 22 2024-03-22T10:42:46.389 INFO:tasks.workunit.client.1.smithi150.stdout:4/171: read d0/d1b/f37 [323845,28857] 0 2024-03-22T10:42:46.392 INFO:tasks.workunit.client.0.smithi115.stdout:0/519: creat d3/d4/d14/d64/fab x:0 0 0 2024-03-22T10:42:46.393 INFO:tasks.workunit.client.0.smithi115.stdout:6/530: chown d4/d11/l77 2971 1 2024-03-22T10:42:46.400 INFO:tasks.workunit.client.0.smithi115.stdout:6/531: chown d4/d11/c47 57 1 2024-03-22T10:42:46.400 INFO:tasks.workunit.client.1.smithi150.stdout:9/164: dread d1/d8/de/f16 [0,4194304] 0 2024-03-22T10:42:46.400 INFO:tasks.workunit.client.1.smithi150.stdout:8/171: symlink d0/d2/d19/d29/d2f/l41 0 2024-03-22T10:42:46.401 INFO:tasks.workunit.client.1.smithi150.stdout:4/172: symlink d0/d32/l3f 0 2024-03-22T10:42:46.401 INFO:tasks.workunit.client.0.smithi115.stdout:3/548: dwrite d3/d36/d41/fa2 [0,4194304] 0 2024-03-22T10:42:46.402 INFO:tasks.workunit.client.0.smithi115.stdout:5/561: creat d2/df/d13/fbb x:0 0 0 2024-03-22T10:42:46.403 INFO:tasks.workunit.client.0.smithi115.stdout:6/532: dread - d4/d36/d48/f94 zero size 2024-03-22T10:42:46.404 INFO:tasks.workunit.client.0.smithi115.stdout:6/533: readlink d4/le 0 2024-03-22T10:42:46.409 INFO:tasks.workunit.client.1.smithi150.stdout:2/198: sync 2024-03-22T10:42:46.411 INFO:tasks.workunit.client.1.smithi150.stdout:3/155: dwrite d5/f25 [4194304,4194304] 0 2024-03-22T10:42:46.413 INFO:tasks.workunit.client.0.smithi115.stdout:6/534: rename d4/d36/d48/fa0 to d4/d36/d48/d72/fa9 0 2024-03-22T10:42:46.414 INFO:tasks.workunit.client.1.smithi150.stdout:8/172: unlink d0/d2/d19/d29/d2f/f36 0 2024-03-22T10:42:46.416 INFO:tasks.workunit.client.1.smithi150.stdout:3/156: fdatasync d5/da/f28 0 2024-03-22T10:42:46.420 INFO:tasks.workunit.client.1.smithi150.stdout:8/173: chown d0/d2/f18 12224 1 2024-03-22T10:42:46.420 INFO:tasks.workunit.client.1.smithi150.stdout:2/199: truncate d12/d16/d2c/f33 375979 0 2024-03-22T10:42:46.421 INFO:tasks.workunit.client.0.smithi115.stdout:5/562: rename d2/df/d27 to d2/df/d13/d8b/d93/da6/dbc 0 2024-03-22T10:42:46.425 INFO:tasks.workunit.client.0.smithi115.stdout:2/530: sync 2024-03-22T10:42:46.425 INFO:tasks.workunit.client.1.smithi150.stdout:0/173: dwrite d0/d1/d7/f28 [0,4194304] 0 2024-03-22T10:42:46.428 INFO:tasks.workunit.client.0.smithi115.stdout:2/531: chown da/d1c/f73 104817064 1 2024-03-22T10:42:46.432 INFO:tasks.workunit.client.1.smithi150.stdout:8/174: read - d0/d2/d19/d29/d2f/f3b zero size 2024-03-22T10:42:46.432 INFO:tasks.workunit.client.0.smithi115.stdout:5/563: write d2/d15/d1b/d7c/d38/d43/d49/f52 [3119908,10650] 0 2024-03-22T10:42:46.432 INFO:tasks.workunit.client.1.smithi150.stdout:2/200: write d12/f26 [2177619,12604] 0 2024-03-22T10:42:46.435 INFO:tasks.workunit.client.1.smithi150.stdout:0/174: write d0/d1/d7/d11/f2f [4151271,9490] 0 2024-03-22T10:42:46.440 INFO:tasks.workunit.client.1.smithi150.stdout:0/175: readlink d0/d1/l32 0 2024-03-22T10:42:46.441 INFO:tasks.workunit.client.0.smithi115.stdout:0/520: dwrite d3/d7/d1b/f24 [0,4194304] 0 2024-03-22T10:42:46.441 INFO:tasks.workunit.client.0.smithi115.stdout:2/532: getdents da/d25/d30/d4f/d54 0 2024-03-22T10:42:46.442 INFO:tasks.workunit.client.1.smithi150.stdout:8/175: write d0/f17 [5187038,74093] 0 2024-03-22T10:42:46.442 INFO:tasks.workunit.client.1.smithi150.stdout:2/201: mkdir d12/d19/d2d/d3b 0 2024-03-22T10:42:46.443 INFO:tasks.workunit.client.1.smithi150.stdout:9/165: dwrite d1/d8/de/df/d20/d32/f2a [0,4194304] 0 2024-03-22T10:42:46.444 INFO:tasks.workunit.client.0.smithi115.stdout:5/564: mkdir d2/d15/d1b/d7c/d38/d43/d49/d81/dbd 0 2024-03-22T10:42:46.445 INFO:tasks.workunit.client.0.smithi115.stdout:2/533: chown da/d1c/l21 832761418 1 2024-03-22T10:42:46.446 INFO:tasks.workunit.client.0.smithi115.stdout:3/549: dwrite d3/d82/d52/f56 [0,4194304] 0 2024-03-22T10:42:46.448 INFO:tasks.workunit.client.0.smithi115.stdout:2/534: stat da/d7b 0 2024-03-22T10:42:46.449 INFO:tasks.workunit.client.0.smithi115.stdout:5/565: truncate d2/df/d13/fb3 727617 0 2024-03-22T10:42:46.450 INFO:tasks.workunit.client.1.smithi150.stdout:4/173: dwrite d0/d1b/f23 [0,4194304] 0 2024-03-22T10:42:46.452 INFO:tasks.workunit.client.1.smithi150.stdout:9/166: unlink d1/d19/f2f 0 2024-03-22T10:42:46.453 INFO:tasks.workunit.client.0.smithi115.stdout:0/521: mkdir d3/da/d4c/d74/d84/dac 0 2024-03-22T10:42:46.455 INFO:tasks.workunit.client.1.smithi150.stdout:8/176: getdents d0/d3c 0 2024-03-22T10:42:46.460 INFO:tasks.workunit.client.0.smithi115.stdout:5/566: getdents d2/d15/d1b/d7c/d38/d4a/d74 0 2024-03-22T10:42:46.460 INFO:tasks.workunit.client.1.smithi150.stdout:0/176: creat d0/d1/f39 x:0 0 0 2024-03-22T10:42:46.463 INFO:tasks.workunit.client.1.smithi150.stdout:9/167: mknod d1/d8/de/df/d20/d29/c3d 0 2024-03-22T10:42:46.465 INFO:tasks.workunit.client.0.smithi115.stdout:5/567: symlink d2/d15/d1b/d7c/daf/lbe 0 2024-03-22T10:42:46.468 INFO:tasks.workunit.client.1.smithi150.stdout:3/157: dread d5/fd [0,4194304] 0 2024-03-22T10:42:46.469 INFO:tasks.workunit.client.1.smithi150.stdout:8/177: rename d0/d2/d19/f25 to d0/d2/d19/d29/d2f/df/f42 0 2024-03-22T10:42:46.470 INFO:tasks.workunit.client.0.smithi115.stdout:3/550: dwrite d3/d82/f83 [0,4194304] 0 2024-03-22T10:42:46.474 INFO:tasks.workunit.client.1.smithi150.stdout:9/168: symlink d1/d8/de/df/l3e 0 2024-03-22T10:42:46.474 INFO:tasks.workunit.client.0.smithi115.stdout:5/568: rename d2/df/d13/d8b/d93/da6/dbc/la7 to d2/df/d13/d8b/d93/lbf 0 2024-03-22T10:42:46.478 INFO:tasks.workunit.client.1.smithi150.stdout:3/158: rename d5/da/lc to d5/d22/l2e 0 2024-03-22T10:42:46.478 INFO:tasks.workunit.client.0.smithi115.stdout:0/522: dwrite d3/d7/d1c/f1e [0,4194304] 0 2024-03-22T10:42:46.478 INFO:tasks.workunit.client.0.smithi115.stdout:5/569: symlink d2/d15/d1b/d7c/d38/d43/lc0 0 2024-03-22T10:42:46.479 INFO:tasks.workunit.client.0.smithi115.stdout:3/551: getdents d3/d82/d52 0 2024-03-22T10:42:46.482 INFO:tasks.workunit.client.1.smithi150.stdout:8/178: symlink d0/d2/d19/d29/l43 0 2024-03-22T10:42:46.484 INFO:tasks.workunit.client.1.smithi150.stdout:9/169: symlink d1/d8/de/df/d20/d32/d2e/l3f 0 2024-03-22T10:42:46.484 INFO:tasks.workunit.client.1.smithi150.stdout:8/179: dread - d0/d2/d19/d29/d2f/f3b zero size 2024-03-22T10:42:46.485 INFO:tasks.workunit.client.0.smithi115.stdout:4/500: sync 2024-03-22T10:42:46.485 INFO:tasks.workunit.client.0.smithi115.stdout:8/760: sync 2024-03-22T10:42:46.485 INFO:tasks.workunit.client.0.smithi115.stdout:1/646: sync 2024-03-22T10:42:46.485 INFO:tasks.workunit.client.0.smithi115.stdout:7/563: sync 2024-03-22T10:42:46.486 INFO:tasks.workunit.client.1.smithi150.stdout:3/159: mkdir d5/d22/d2f 0 2024-03-22T10:42:46.487 INFO:tasks.workunit.client.0.smithi115.stdout:9/575: sync 2024-03-22T10:42:46.487 INFO:tasks.workunit.client.0.smithi115.stdout:5/570: rename d2/df/d3b/f72 to d2/d15/d1b/d7c/d38/d43/d49/d81/dbd/fc1 0 2024-03-22T10:42:46.487 INFO:tasks.workunit.client.1.smithi150.stdout:8/180: write d0/d2/f14 [325136,85339] 0 2024-03-22T10:42:46.489 INFO:tasks.workunit.client.1.smithi150.stdout:3/160: read d5/f8 [5673375,3584] 0 2024-03-22T10:42:46.491 INFO:tasks.workunit.client.1.smithi150.stdout:0/177: creat d0/f3a x:0 0 0 2024-03-22T10:42:46.491 INFO:tasks.workunit.client.0.smithi115.stdout:9/576: chown d6/d7/d48/d70 6908 1 2024-03-22T10:42:46.492 INFO:tasks.workunit.client.0.smithi115.stdout:8/761: creat d6/d49/d68/deb/ff6 x:0 0 0 2024-03-22T10:42:46.493 INFO:tasks.workunit.client.0.smithi115.stdout:4/501: write d1/d30/d4c/d86/d9f/f45 [3220250,98007] 0 2024-03-22T10:42:46.494 INFO:tasks.workunit.client.0.smithi115.stdout:5/571: read d2/df/d13/f18 [4154527,12831] 0 2024-03-22T10:42:46.494 INFO:tasks.workunit.client.1.smithi150.stdout:9/170: mkdir d1/d8/de/df/d36/d40 0 2024-03-22T10:42:46.494 INFO:tasks.workunit.client.0.smithi115.stdout:9/577: readlink d6/d7/d6a/d6f/l89 0 2024-03-22T10:42:46.495 INFO:tasks.workunit.client.0.smithi115.stdout:6/535: truncate d4/d36/d48/f9e 840387 0 2024-03-22T10:42:46.498 INFO:tasks.workunit.client.0.smithi115.stdout:5/572: readlink d2/d15/l5c 0 2024-03-22T10:42:46.501 INFO:tasks.workunit.client.1.smithi150.stdout:2/202: write d12/f13 [2832634,77764] 0 2024-03-22T10:42:46.502 INFO:tasks.workunit.client.0.smithi115.stdout:7/564: truncate d2/d7/d12/d10/d19/fa9 394139 0 2024-03-22T10:42:46.504 INFO:tasks.workunit.client.0.smithi115.stdout:7/565: fsync d2/d62/f4f 0 2024-03-22T10:42:46.506 INFO:tasks.workunit.client.1.smithi150.stdout:4/174: truncate d0/d1b/f2b 1099894 0 2024-03-22T10:42:46.507 INFO:tasks.workunit.client.1.smithi150.stdout:8/181: creat d0/d3c/f44 x:0 0 0 2024-03-22T10:42:46.508 INFO:tasks.workunit.client.0.smithi115.stdout:6/536: mkdir d4/d36/d48/d72/daa 0 2024-03-22T10:42:46.509 INFO:tasks.workunit.client.0.smithi115.stdout:1/647: rename d0/d12/d17/d37/f6f to d0/d22/fda 0 2024-03-22T10:42:46.510 INFO:tasks.workunit.client.1.smithi150.stdout:8/182: write d0/f1 [5710995,96904] 0 2024-03-22T10:42:46.511 INFO:tasks.workunit.client.0.smithi115.stdout:6/537: fdatasync d4/d11/f54 0 2024-03-22T10:42:46.511 INFO:tasks.workunit.client.1.smithi150.stdout:3/161: creat d5/d22/d2f/f30 x:0 0 0 2024-03-22T10:42:46.512 INFO:tasks.workunit.client.1.smithi150.stdout:9/171: mkdir d1/d8/de/df/d36/d40/d41 0 2024-03-22T10:42:46.514 INFO:tasks.workunit.client.0.smithi115.stdout:5/573: mknod d2/d15/d1b/d7c/d84/d89/cc2 0 2024-03-22T10:42:46.515 INFO:tasks.workunit.client.1.smithi150.stdout:9/172: dread - d1/d8/de/f3c zero size 2024-03-22T10:42:46.516 INFO:tasks.workunit.client.1.smithi150.stdout:3/162: truncate d5/da/f19 4609711 0 2024-03-22T10:42:46.516 INFO:tasks.workunit.client.1.smithi150.stdout:8/183: creat d0/d2/d19/d29/d2f/df/f45 x:0 0 0 2024-03-22T10:42:46.519 INFO:tasks.workunit.client.1.smithi150.stdout:3/163: dread - d5/f10 zero size 2024-03-22T10:42:46.520 INFO:tasks.workunit.client.1.smithi150.stdout:8/184: mkdir d0/d2/d46 0 2024-03-22T10:42:46.522 INFO:tasks.workunit.client.0.smithi115.stdout:6/538: write d4/d36/f53 [3594287,90970] 0 2024-03-22T10:42:46.527 INFO:tasks.workunit.client.1.smithi150.stdout:4/175: dread d0/f14 [0,4194304] 0 2024-03-22T10:42:46.527 INFO:tasks.workunit.client.0.smithi115.stdout:8/762: dwrite d6/dd/d12/f1d [0,4194304] 0 2024-03-22T10:42:46.529 INFO:tasks.workunit.client.1.smithi150.stdout:8/185: read d0/d2/d19/d29/d2f/df/d1b/f1d [825450,94744] 0 2024-03-22T10:42:46.529 INFO:tasks.workunit.client.1.smithi150.stdout:9/173: dread d1/d8/de/f2c [0,4194304] 0 2024-03-22T10:42:46.530 INFO:tasks.workunit.client.0.smithi115.stdout:4/502: creat d1/d30/d4c/d86/d9f/d75/d99/da8/fa9 x:0 0 0 2024-03-22T10:42:46.531 INFO:tasks.workunit.client.0.smithi115.stdout:3/552: dread d3/d36/f3e [0,4194304] 0 2024-03-22T10:42:46.533 INFO:tasks.workunit.client.1.smithi150.stdout:4/176: symlink d0/d1b/l40 0 2024-03-22T10:42:46.535 INFO:tasks.workunit.client.1.smithi150.stdout:8/186: write d0/d2/d19/d29/d2f/df/d1b/f24 [844133,94137] 0 2024-03-22T10:42:46.535 INFO:tasks.workunit.client.0.smithi115.stdout:2/535: dread da/d25/f37 [0,4194304] 0 2024-03-22T10:42:46.535 INFO:tasks.workunit.client.0.smithi115.stdout:5/574: truncate d2/f5e 253352 0 2024-03-22T10:42:46.536 INFO:tasks.workunit.client.1.smithi150.stdout:9/174: symlink d1/d8/de/df/d20/d32/l42 0 2024-03-22T10:42:46.539 INFO:tasks.workunit.client.0.smithi115.stdout:6/539: mknod d4/d36/d48/cab 0 2024-03-22T10:42:46.542 INFO:tasks.workunit.client.0.smithi115.stdout:0/523: dwrite d3/d4/d14/f4f [0,4194304] 0 2024-03-22T10:42:46.543 INFO:tasks.workunit.client.1.smithi150.stdout:4/177: unlink d0/c2d 0 2024-03-22T10:42:46.545 INFO:tasks.workunit.client.1.smithi150.stdout:4/178: write d0/f38 [792375,18800] 0 2024-03-22T10:42:46.545 INFO:tasks.workunit.client.0.smithi115.stdout:0/524: readlink d3/da/l3c 0 2024-03-22T10:42:46.545 INFO:tasks.workunit.client.0.smithi115.stdout:6/540: write d4/d11/f54 [3310583,2057] 0 2024-03-22T10:42:46.545 INFO:tasks.workunit.client.0.smithi115.stdout:9/578: dread d6/d8/d3c/d59/f90 [0,4194304] 0 2024-03-22T10:42:46.545 INFO:tasks.workunit.client.1.smithi150.stdout:9/175: creat d1/d8/de/df/d36/f43 x:0 0 0 2024-03-22T10:42:46.546 INFO:tasks.workunit.client.0.smithi115.stdout:4/503: dread - d1/d16/f9d zero size 2024-03-22T10:42:46.548 INFO:tasks.workunit.client.1.smithi150.stdout:4/179: fdatasync d0/d2f/f3e 0 2024-03-22T10:42:46.550 INFO:tasks.workunit.client.1.smithi150.stdout:0/178: truncate d0/d1/d7/d11/d26/f2e 1755925 0 2024-03-22T10:42:46.551 INFO:tasks.workunit.client.0.smithi115.stdout:2/536: creat da/d25/d3d/f9f x:0 0 0 2024-03-22T10:42:46.551 INFO:tasks.workunit.client.0.smithi115.stdout:0/525: write d3/d7/d1b/f92 [1995120,60227] 0 2024-03-22T10:42:46.551 INFO:tasks.workunit.client.1.smithi150.stdout:2/203: truncate f11 5263801 0 2024-03-22T10:42:46.552 INFO:tasks.workunit.client.1.smithi150.stdout:9/176: write d1/d8/de/df/d36/f43 [953836,26868] 0 2024-03-22T10:42:46.552 INFO:tasks.workunit.client.0.smithi115.stdout:1/648: creat d0/d8/de/d1a/d91/da3/fdb x:0 0 0 2024-03-22T10:42:46.553 INFO:tasks.workunit.client.1.smithi150.stdout:2/204: fdatasync d12/f26 0 2024-03-22T10:42:46.553 INFO:tasks.workunit.client.1.smithi150.stdout:5/100: sync 2024-03-22T10:42:46.555 INFO:tasks.workunit.client.0.smithi115.stdout:0/526: truncate d3/fa9 231494 0 2024-03-22T10:42:46.555 INFO:tasks.workunit.client.1.smithi150.stdout:9/177: readlink d1/d8/de/df/d24/l28 0 2024-03-22T10:42:46.556 INFO:tasks.workunit.client.0.smithi115.stdout:2/537: chown da/d25/d30/d4f/d54/l5b 24705581 1 2024-03-22T10:42:46.557 INFO:tasks.workunit.client.1.smithi150.stdout:4/180: fdatasync d0/de/f18 0 2024-03-22T10:42:46.558 INFO:tasks.workunit.client.1.smithi150.stdout:8/187: rmdir d0/d2 39 2024-03-22T10:42:46.559 INFO:tasks.workunit.client.1.smithi150.stdout:0/179: creat d0/d12/f3b x:0 0 0 2024-03-22T10:42:46.561 INFO:tasks.workunit.client.1.smithi150.stdout:0/180: read - d0/d12/f3b zero size 2024-03-22T10:42:46.563 INFO:tasks.workunit.client.1.smithi150.stdout:3/164: dread d5/fb [0,4194304] 0 2024-03-22T10:42:46.563 INFO:tasks.workunit.client.0.smithi115.stdout:7/566: creat d2/d7/d12/d10/d19/d42/fc4 x:0 0 0 2024-03-22T10:42:46.563 INFO:tasks.workunit.client.0.smithi115.stdout:7/567: write d2/d7/d12/d10/f26 [4918500,32016] 0 2024-03-22T10:42:46.564 INFO:tasks.workunit.client.1.smithi150.stdout:3/165: stat d5 0 2024-03-22T10:42:46.567 INFO:tasks.workunit.client.0.smithi115.stdout:9/579: write d6/d7/d6a/d6f/f9b [203955,67399] 0 2024-03-22T10:42:46.567 INFO:tasks.workunit.client.1.smithi150.stdout:8/188: stat d0/d2/f3f 0 2024-03-22T10:42:46.569 INFO:tasks.workunit.client.0.smithi115.stdout:9/580: chown d6/d7/d48/d53/dc9 0 1 2024-03-22T10:42:46.569 INFO:tasks.workunit.client.1.smithi150.stdout:8/189: fdatasync d0/f1 0 2024-03-22T10:42:46.571 INFO:tasks.workunit.client.0.smithi115.stdout:5/575: fdatasync d2/df/d3b/f2a 0 2024-03-22T10:42:46.574 INFO:tasks.workunit.client.1.smithi150.stdout:5/101: dread d0/f6 [0,4194304] 0 2024-03-22T10:42:46.575 INFO:tasks.workunit.client.0.smithi115.stdout:3/553: creat d3/fb2 x:0 0 0 2024-03-22T10:42:46.576 INFO:tasks.workunit.client.0.smithi115.stdout:5/576: dread - d2/df/d13/d8b/d93/da6/dbc/fb2 zero size 2024-03-22T10:42:46.576 INFO:tasks.workunit.client.1.smithi150.stdout:2/205: getdents d12/d19/d2d 0 2024-03-22T10:42:46.579 INFO:tasks.workunit.client.1.smithi150.stdout:4/181: getdents d0/d1b 0 2024-03-22T10:42:46.579 INFO:tasks.workunit.client.0.smithi115.stdout:4/504: rename d1/d16/d68/f8c to d1/d30/d4c/d86/d9f/d75/d99/d98/faa 0 2024-03-22T10:42:46.584 INFO:tasks.workunit.client.0.smithi115.stdout:0/527: dwrite d3/d52/f59 [0,4194304] 0 2024-03-22T10:42:46.585 INFO:tasks.workunit.client.0.smithi115.stdout:5/577: write d2/df/d3b/f36 [1284065,51581] 0 2024-03-22T10:42:46.586 INFO:tasks.workunit.client.0.smithi115.stdout:9/581: symlink d6/d8/d3c/d59/lcb 0 2024-03-22T10:42:46.588 INFO:tasks.workunit.client.1.smithi150.stdout:4/182: write d0/f26 [162824,13285] 0 2024-03-22T10:42:46.589 INFO:tasks.workunit.client.1.smithi150.stdout:1/146: sync 2024-03-22T10:42:46.592 INFO:tasks.workunit.client.1.smithi150.stdout:8/190: creat d0/d2/d19/f47 x:0 0 0 2024-03-22T10:42:46.595 INFO:tasks.workunit.client.0.smithi115.stdout:1/649: rename d0/d22/d75 to d0/d8/d4d/d56/db6/ddc 0 2024-03-22T10:42:46.597 INFO:tasks.workunit.client.1.smithi150.stdout:4/183: write d0/f26 [607439,11876] 0 2024-03-22T10:42:46.599 INFO:tasks.workunit.client.0.smithi115.stdout:4/505: mknod d1/d66/d82/da1/cab 0 2024-03-22T10:42:46.599 INFO:tasks.workunit.client.1.smithi150.stdout:9/178: dwrite d1/d8/de/df/f12 [4194304,4194304] 0 2024-03-22T10:42:46.605 INFO:tasks.workunit.client.0.smithi115.stdout:5/578: mkdir d2/df/d13/d8b/d93/da6/dc3 0 2024-03-22T10:42:46.606 INFO:tasks.workunit.client.1.smithi150.stdout:8/191: chown d0/d2/fe 364 1 2024-03-22T10:42:46.607 INFO:tasks.workunit.client.1.smithi150.stdout:0/181: dwrite d0/d1/d7/fe [0,4194304] 0 2024-03-22T10:42:46.607 INFO:tasks.workunit.client.1.smithi150.stdout:4/184: rmdir d0 39 2024-03-22T10:42:46.608 INFO:tasks.workunit.client.0.smithi115.stdout:9/582: truncate d6/d8/f1a 752993 0 2024-03-22T10:42:46.610 INFO:tasks.workunit.client.0.smithi115.stdout:9/583: chown d6/d7/d30/d3b/f8f 4 1 2024-03-22T10:42:46.611 INFO:tasks.workunit.client.0.smithi115.stdout:9/584: symlink d6/d8/d3c/lcc 2 2024-03-22T10:42:46.614 INFO:tasks.workunit.client.0.smithi115.stdout:7/568: rename d2/d7/d12/d10/d23/d33/l7e to d2/d7/d12/lc5 0 2024-03-22T10:42:46.615 INFO:tasks.workunit.client.1.smithi150.stdout:5/102: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:42:46.618 INFO:tasks.workunit.client.0.smithi115.stdout:3/554: truncate d3/d19/d2b/d49/d91/da5/f75 1149612 0 2024-03-22T10:42:46.622 INFO:tasks.workunit.client.1.smithi150.stdout:9/179: mkdir d1/d8/de/df/d36/d40/d44 0 2024-03-22T10:42:46.622 INFO:tasks.workunit.client.1.smithi150.stdout:5/103: stat d0/l9 0 2024-03-22T10:42:46.622 INFO:tasks.workunit.client.1.smithi150.stdout:4/185: write d0/d1b/f23 [4362357,28073] 0 2024-03-22T10:42:46.624 INFO:tasks.workunit.client.1.smithi150.stdout:2/206: dwrite d12/f13 [0,4194304] 0 2024-03-22T10:42:46.625 INFO:tasks.workunit.client.1.smithi150.stdout:1/147: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:46.626 INFO:tasks.workunit.client.0.smithi115.stdout:5/579: rmdir d2/d15 39 2024-03-22T10:42:46.627 INFO:tasks.workunit.client.0.smithi115.stdout:0/528: dread d3/d7/d1b/f2a [0,4194304] 0 2024-03-22T10:42:46.628 INFO:tasks.workunit.client.1.smithi150.stdout:4/186: readlink d0/de/l19 0 2024-03-22T10:42:46.628 INFO:tasks.workunit.client.0.smithi115.stdout:7/569: rmdir d2/d3a/d6e/d78/dbf/dae 39 2024-03-22T10:42:46.629 INFO:tasks.workunit.client.1.smithi150.stdout:3/166: dwrite d5/f8 [0,4194304] 0 2024-03-22T10:42:46.629 INFO:tasks.workunit.client.1.smithi150.stdout:1/148: creat d1/f29 x:0 0 0 2024-03-22T10:42:46.630 INFO:tasks.workunit.client.1.smithi150.stdout:4/187: chown d0/d2f/c33 6306 1 2024-03-22T10:42:46.631 INFO:tasks.workunit.client.1.smithi150.stdout:0/182: symlink d0/d1/d7/db/d10/d1e/d35/l3c 0 2024-03-22T10:42:46.631 INFO:tasks.workunit.client.0.smithi115.stdout:6/541: write d4/d13/f70 [560388,22366] 0 2024-03-22T10:42:46.632 INFO:tasks.workunit.client.1.smithi150.stdout:4/188: readlink d0/d2f/l35 0 2024-03-22T10:42:46.634 INFO:tasks.workunit.client.0.smithi115.stdout:6/542: stat d4/c68 0 2024-03-22T10:42:46.634 INFO:tasks.workunit.client.0.smithi115.stdout:3/555: mkdir d3/d19/d61/db3 0 2024-03-22T10:42:46.634 INFO:tasks.workunit.client.1.smithi150.stdout:5/104: write d0/d8/dc/d1d/f1f [461962,71870] 0 2024-03-22T10:42:46.636 INFO:tasks.workunit.client.1.smithi150.stdout:9/180: symlink d1/d8/de/df/d20/d32/d2e/l45 0 2024-03-22T10:42:46.639 INFO:tasks.workunit.client.0.smithi115.stdout:4/506: link d1/d66/d82/f9a d1/d30/d4c/d86/fac 0 2024-03-22T10:42:46.639 INFO:tasks.workunit.client.1.smithi150.stdout:1/149: mkdir d1/da/d2a 0 2024-03-22T10:42:46.640 INFO:tasks.workunit.client.0.smithi115.stdout:8/763: dwrite d6/f95 [0,4194304] 0 2024-03-22T10:42:46.640 INFO:tasks.workunit.client.0.smithi115.stdout:9/585: dwrite d6/d7/d20/d22/d23/fc4 [0,4194304] 0 2024-03-22T10:42:46.641 INFO:tasks.workunit.client.0.smithi115.stdout:2/538: truncate da/d25/f34 888533 0 2024-03-22T10:42:46.643 INFO:tasks.workunit.client.1.smithi150.stdout:2/207: truncate d12/f37 2360645 0 2024-03-22T10:42:46.644 INFO:tasks.workunit.client.1.smithi150.stdout:9/181: chown d1/d8/de/df/d20/l5 1045516705 1 2024-03-22T10:42:46.649 INFO:tasks.workunit.client.0.smithi115.stdout:9/586: write d6/d7/d48/faf [137109,21863] 0 2024-03-22T10:42:46.651 INFO:tasks.workunit.client.0.smithi115.stdout:5/580: chown d2/df/d3b/l21 2 1 2024-03-22T10:42:46.654 INFO:tasks.workunit.client.0.smithi115.stdout:9/587: chown d6/d8/d3c/d59 0 1 2024-03-22T10:42:46.654 INFO:tasks.workunit.client.0.smithi115.stdout:3/556: rmdir d3/d19/d2b/d49 39 2024-03-22T10:42:46.660 INFO:tasks.workunit.client.1.smithi150.stdout:0/183: link d0/d1/d7/db/f2b d0/d1/d7/db/d10/d1e/d37/f3d 0 2024-03-22T10:42:46.660 INFO:tasks.workunit.client.1.smithi150.stdout:6/128: sync 2024-03-22T10:42:46.660 INFO:tasks.workunit.client.1.smithi150.stdout:7/155: sync 2024-03-22T10:42:46.661 INFO:tasks.workunit.client.1.smithi150.stdout:9/182: mknod d1/d8/de/df/d36/d40/c46 0 2024-03-22T10:42:46.662 INFO:tasks.workunit.client.1.smithi150.stdout:3/167: dread d5/da/ff [0,4194304] 0 2024-03-22T10:42:46.663 INFO:tasks.workunit.client.1.smithi150.stdout:1/150: read d1/da/f13 [771395,17770] 0 2024-03-22T10:42:46.664 INFO:tasks.workunit.client.1.smithi150.stdout:8/192: dwrite d0/d2/d19/d29/d2e/f3a [0,4194304] 0 2024-03-22T10:42:46.666 INFO:tasks.workunit.client.0.smithi115.stdout:1/650: write d0/d8/d4d/d56/db6/ddc/fa5 [372270,71662] 0 2024-03-22T10:42:46.672 INFO:tasks.workunit.client.0.smithi115.stdout:8/764: symlink d6/dd/d24/d3a/lf7 0 2024-03-22T10:42:46.674 INFO:tasks.workunit.client.0.smithi115.stdout:2/539: dread da/d1c/f88 [0,4194304] 0 2024-03-22T10:42:46.675 INFO:tasks.workunit.client.0.smithi115.stdout:4/507: rename d1/d16/d35/d40/d54/l92 to d1/d16/d35/lad 0 2024-03-22T10:42:46.681 INFO:tasks.workunit.client.1.smithi150.stdout:9/183: unlink d1/l23 0 2024-03-22T10:42:46.684 INFO:tasks.workunit.client.1.smithi150.stdout:4/189: dwrite d0/d1b/f3a [0,4194304] 0 2024-03-22T10:42:46.684 INFO:tasks.workunit.client.0.smithi115.stdout:2/540: dread da/d25/f37 [0,4194304] 0 2024-03-22T10:42:46.685 INFO:tasks.workunit.client.1.smithi150.stdout:7/156: symlink d2/d5/d7/d8/d2c/l35 0 2024-03-22T10:42:46.685 INFO:tasks.workunit.client.1.smithi150.stdout:5/105: dwrite d0/f10 [0,4194304] 0 2024-03-22T10:42:46.690 INFO:tasks.workunit.client.1.smithi150.stdout:0/184: mknod d0/d1/d7/db/d10/d38/c3e 0 2024-03-22T10:42:46.692 INFO:tasks.workunit.client.0.smithi115.stdout:1/651: dread d0/d22/f41 [0,4194304] 0 2024-03-22T10:42:46.695 INFO:tasks.workunit.client.0.smithi115.stdout:0/529: write d3/d7/d1b/d1f/f41 [653186,18293] 0 2024-03-22T10:42:46.696 INFO:tasks.workunit.client.1.smithi150.stdout:5/106: chown d0/d8/dc/lf 250353 1 2024-03-22T10:42:46.696 INFO:tasks.workunit.client.0.smithi115.stdout:6/543: link d4/d11/d12/f9c d4/d36/d48/d72/daa/fac 0 2024-03-22T10:42:46.700 INFO:tasks.workunit.client.1.smithi150.stdout:0/185: mknod d0/d12/c3f 0 2024-03-22T10:42:46.701 INFO:tasks.workunit.client.0.smithi115.stdout:6/544: write d4/d36/d48/f94 [861120,4880] 0 2024-03-22T10:42:46.701 INFO:tasks.workunit.client.1.smithi150.stdout:7/157: mknod d2/d5/d7/d8/d2c/c36 0 2024-03-22T10:42:46.702 INFO:tasks.workunit.client.0.smithi115.stdout:0/530: chown d3/da/d4c/d69/f76 21112 1 2024-03-22T10:42:46.703 INFO:tasks.workunit.client.1.smithi150.stdout:5/107: write d0/f6 [4226053,74693] 0 2024-03-22T10:42:46.704 INFO:tasks.workunit.client.0.smithi115.stdout:4/508: creat d1/d16/d68/fae x:0 0 0 2024-03-22T10:42:46.706 INFO:tasks.workunit.client.1.smithi150.stdout:9/184: link d1/d19/f26 d1/d8/de/df/d36/d40/d41/f47 0 2024-03-22T10:42:46.707 INFO:tasks.workunit.client.0.smithi115.stdout:0/531: write d3/d7/d1c/f1e [3886986,74502] 0 2024-03-22T10:42:46.709 INFO:tasks.workunit.client.0.smithi115.stdout:5/581: mknod d2/df/d13/d8b/d93/da6/dc3/cc4 0 2024-03-22T10:42:46.709 INFO:tasks.workunit.client.1.smithi150.stdout:3/168: dwrite d5/d22/d2f/f30 [0,4194304] 0 2024-03-22T10:42:46.710 INFO:tasks.workunit.client.0.smithi115.stdout:0/532: readlink d3/d7/l73 0 2024-03-22T10:42:46.714 INFO:tasks.workunit.client.0.smithi115.stdout:5/582: write d2/f71 [264147,1316] 0 2024-03-22T10:42:46.714 INFO:tasks.workunit.client.1.smithi150.stdout:4/190: dread d0/f14 [0,4194304] 0 2024-03-22T10:42:46.714 INFO:tasks.workunit.client.1.smithi150.stdout:0/186: symlink d0/d1/d7/db/d10/d38/l40 0 2024-03-22T10:42:46.715 INFO:tasks.workunit.client.0.smithi115.stdout:1/652: creat d0/d68/fdd x:0 0 0 2024-03-22T10:42:46.718 INFO:tasks.workunit.client.1.smithi150.stdout:5/108: rename d0/c7 to d0/c20 0 2024-03-22T10:42:46.718 INFO:tasks.workunit.client.1.smithi150.stdout:3/169: creat d5/f31 x:0 0 0 2024-03-22T10:42:46.718 INFO:tasks.workunit.client.1.smithi150.stdout:9/185: symlink d1/d8/de/df/d36/d40/l48 0 2024-03-22T10:42:46.719 INFO:tasks.workunit.client.0.smithi115.stdout:9/588: dwrite d6/d8/d3c/d59/d75/fa6 [0,4194304] 0 2024-03-22T10:42:46.720 INFO:tasks.workunit.client.0.smithi115.stdout:2/541: creat da/d1c/d67/d78/d86/d9d/fa0 x:0 0 0 2024-03-22T10:42:46.721 INFO:tasks.workunit.client.1.smithi150.stdout:4/191: creat d0/d32/f41 x:0 0 0 2024-03-22T10:42:46.723 INFO:tasks.workunit.client.0.smithi115.stdout:2/542: dread - da/d25/d83/f8a zero size 2024-03-22T10:42:46.723 INFO:tasks.workunit.client.0.smithi115.stdout:0/533: rmdir d3/da/d4c/d74 39 2024-03-22T10:42:46.724 INFO:tasks.workunit.client.1.smithi150.stdout:7/158: rename d2/d5/d7/d9/l2a to d2/d5/d7/d2e/l37 0 2024-03-22T10:42:46.724 INFO:tasks.workunit.client.1.smithi150.stdout:3/170: mknod d5/d22/c32 0 2024-03-22T10:42:46.725 INFO:tasks.workunit.client.0.smithi115.stdout:8/765: creat d6/dd/ff8 x:0 0 0 2024-03-22T10:42:46.727 INFO:tasks.workunit.client.1.smithi150.stdout:5/109: mknod d0/d8/dc/de/d18/c21 0 2024-03-22T10:42:46.728 INFO:tasks.workunit.client.0.smithi115.stdout:5/583: mknod d2/df/d13/d8b/d93/da6/dbc/cc5 0 2024-03-22T10:42:46.729 INFO:tasks.workunit.client.0.smithi115.stdout:1/653: mkdir d0/d8/db/d25/d7e/dde 0 2024-03-22T10:42:46.729 INFO:tasks.workunit.client.0.smithi115.stdout:6/545: read d4/d36/d48/d72/f76 [1572043,80721] 0 2024-03-22T10:42:46.730 INFO:tasks.workunit.client.1.smithi150.stdout:4/192: fsync d0/d1b/f2c 0 2024-03-22T10:42:46.732 INFO:tasks.workunit.client.0.smithi115.stdout:9/589: symlink d6/d8/d3c/d59/d75/lcd 0 2024-03-22T10:42:46.733 INFO:tasks.workunit.client.0.smithi115.stdout:1/654: chown d0/d12/d17/f7d 459295299 1 2024-03-22T10:42:46.734 INFO:tasks.workunit.client.0.smithi115.stdout:2/543: mkdir da/d25/d3d/d57/d7d/da1 0 2024-03-22T10:42:46.739 INFO:tasks.workunit.client.1.smithi150.stdout:7/159: chown d2/d5/d7/c14 9 1 2024-03-22T10:42:46.746 INFO:tasks.workunit.client.1.smithi150.stdout:1/151: dread d1/da/fb [0,4194304] 0 2024-03-22T10:42:46.748 INFO:tasks.workunit.client.1.smithi150.stdout:4/193: mkdir d0/d42 0 2024-03-22T10:42:46.749 INFO:tasks.workunit.client.1.smithi150.stdout:2/208: truncate d12/f37 3165165 0 2024-03-22T10:42:46.751 INFO:tasks.workunit.client.1.smithi150.stdout:9/186: rename d1/d8/de/df/d20/d32/d2e to d1/d49 0 2024-03-22T10:42:46.753 INFO:tasks.workunit.client.1.smithi150.stdout:5/110: dwrite d0/d8/dc/d1d/f1f [0,4194304] 0 2024-03-22T10:42:46.753 INFO:tasks.workunit.client.0.smithi115.stdout:3/557: dwrite d3/d19/f4e [4194304,4194304] 0 2024-03-22T10:42:46.753 INFO:tasks.workunit.client.0.smithi115.stdout:7/570: dwrite d2/d7/d12/d10/d19/d42/f85 [4194304,4194304] 0 2024-03-22T10:42:46.755 INFO:tasks.workunit.client.0.smithi115.stdout:0/534: dwrite d3/d7/d1c/d6c/d87/f8a [0,4194304] 0 2024-03-22T10:42:46.756 INFO:tasks.workunit.client.1.smithi150.stdout:6/129: truncate db/dd/d15/d28/f2b 1422911 0 2024-03-22T10:42:46.757 INFO:tasks.workunit.client.0.smithi115.stdout:3/558: chown d3/d82/fa4 1990980 1 2024-03-22T10:42:46.758 INFO:tasks.workunit.client.1.smithi150.stdout:1/152: creat d1/d19/d1a/f2b x:0 0 0 2024-03-22T10:42:46.759 INFO:tasks.workunit.client.1.smithi150.stdout:5/111: write d0/f6 [2979869,27157] 0 2024-03-22T10:42:46.759 INFO:tasks.workunit.client.0.smithi115.stdout:3/559: chown d3/d19/d61/f6f 52097 1 2024-03-22T10:42:46.764 INFO:tasks.workunit.client.1.smithi150.stdout:9/187: creat d1/d8/de/df/d24/f4a x:0 0 0 2024-03-22T10:42:46.771 INFO:tasks.workunit.client.0.smithi115.stdout:4/509: write d1/d69/f65 [3534344,68214] 0 2024-03-22T10:42:46.773 INFO:tasks.workunit.client.1.smithi150.stdout:2/209: mkdir d12/d19/d2d/d3b/d3c 0 2024-03-22T10:42:46.778 INFO:tasks.workunit.client.1.smithi150.stdout:1/153: unlink d1/da/d15/f21 0 2024-03-22T10:42:46.780 INFO:tasks.workunit.client.1.smithi150.stdout:1/154: fsync d1/d19/d1a/f1e 0 2024-03-22T10:42:46.782 INFO:tasks.workunit.client.0.smithi115.stdout:0/535: dread d3/d7/d1b/f2a [0,4194304] 0 2024-03-22T10:42:46.783 INFO:tasks.workunit.client.1.smithi150.stdout:5/112: unlink d0/d8/c19 0 2024-03-22T10:42:46.784 INFO:tasks.workunit.client.0.smithi115.stdout:3/560: rename d3/d19/d2b/d49/d91/da5/d7e to d3/d6d/db4 0 2024-03-22T10:42:46.785 INFO:tasks.workunit.client.1.smithi150.stdout:5/113: read d0/f3 [1891117,126710] 0 2024-03-22T10:42:46.786 INFO:tasks.workunit.client.0.smithi115.stdout:7/571: dread d2/d7/d12/d10/d23/f38 [0,4194304] 0 2024-03-22T10:42:46.786 INFO:tasks.workunit.client.1.smithi150.stdout:9/188: rename d1/d8/de/df/d24/l3b to d1/d8/de/df/l4b 0 2024-03-22T10:42:46.787 INFO:tasks.workunit.client.0.smithi115.stdout:3/561: read - d3/d82/fa4 zero size 2024-03-22T10:42:46.789 INFO:tasks.workunit.client.1.smithi150.stdout:6/130: creat db/dd/d2e/f31 x:0 0 0 2024-03-22T10:42:46.790 INFO:tasks.workunit.client.0.smithi115.stdout:2/544: link da/d1c/d67/d78/f95 da/d25/d3d/d7e/fa2 0 2024-03-22T10:42:46.791 INFO:tasks.workunit.client.1.smithi150.stdout:2/210: creat d12/d19/d2d/d3b/f3d x:0 0 0 2024-03-22T10:42:46.791 INFO:tasks.workunit.client.1.smithi150.stdout:4/194: dread d0/f1a [0,4194304] 0 2024-03-22T10:42:46.792 INFO:tasks.workunit.client.1.smithi150.stdout:2/211: readlink d12/d19/d2d/l3a 0 2024-03-22T10:42:46.792 INFO:tasks.workunit.client.0.smithi115.stdout:9/590: truncate d6/d8/d3c/d59/fb5 553528 0 2024-03-22T10:42:46.796 INFO:tasks.workunit.client.1.smithi150.stdout:0/187: write d0/d1/d7/ff [276281,65890] 0 2024-03-22T10:42:46.796 INFO:tasks.workunit.client.0.smithi115.stdout:2/545: stat da/d25/d30/c24 0 2024-03-22T10:42:46.796 INFO:tasks.workunit.client.0.smithi115.stdout:1/655: dread d0/d22/fc4 [0,4194304] 0 2024-03-22T10:42:46.796 INFO:tasks.workunit.client.0.smithi115.stdout:0/536: symlink d3/da/d4c/d74/d84/dac/lad 0 2024-03-22T10:42:46.798 INFO:tasks.workunit.client.1.smithi150.stdout:6/131: fsync db/ff 0 2024-03-22T10:42:46.801 INFO:tasks.workunit.client.1.smithi150.stdout:2/212: symlink d12/d16/l3e 0 2024-03-22T10:42:46.802 INFO:tasks.workunit.client.0.smithi115.stdout:4/510: dwrite d1/d66/d82/f8e [0,4194304] 0 2024-03-22T10:42:46.802 INFO:tasks.workunit.client.1.smithi150.stdout:4/195: mknod d0/d1b/c43 0 2024-03-22T10:42:46.802 INFO:tasks.workunit.client.0.smithi115.stdout:9/591: write d6/d7/d20/d22/d23/f85 [397650,61841] 0 2024-03-22T10:42:46.803 INFO:tasks.workunit.client.0.smithi115.stdout:0/537: rmdir d3/d7/d6b 39 2024-03-22T10:42:46.804 INFO:tasks.workunit.client.1.smithi150.stdout:4/196: chown d0/f9 40663 1 2024-03-22T10:42:46.805 INFO:tasks.workunit.client.1.smithi150.stdout:6/132: creat db/dd/d21/f32 x:0 0 0 2024-03-22T10:42:46.806 INFO:tasks.workunit.client.1.smithi150.stdout:3/171: truncate d5/da/ff 1508235 0 2024-03-22T10:42:46.806 INFO:tasks.workunit.client.0.smithi115.stdout:1/656: fsync d0/d68/f7f 0 2024-03-22T10:42:46.808 INFO:tasks.workunit.client.1.smithi150.stdout:8/193: sync 2024-03-22T10:42:46.808 INFO:tasks.workunit.client.1.smithi150.stdout:2/213: symlink d12/d16/d2c/l3f 0 2024-03-22T10:42:46.810 INFO:tasks.workunit.client.1.smithi150.stdout:2/214: stat lc 0 2024-03-22T10:42:46.810 INFO:tasks.workunit.client.1.smithi150.stdout:4/197: chown d0/f7 90 1 2024-03-22T10:42:46.811 INFO:tasks.workunit.client.1.smithi150.stdout:5/114: dwrite d0/d8/dc/de/f15 [4194304,4194304] 0 2024-03-22T10:42:46.811 INFO:tasks.workunit.client.1.smithi150.stdout:6/133: creat db/dd/f33 x:0 0 0 2024-03-22T10:42:46.812 INFO:tasks.workunit.client.1.smithi150.stdout:3/172: read - d5/f10 zero size 2024-03-22T10:42:46.815 INFO:tasks.workunit.client.1.smithi150.stdout:7/160: truncate d2/d5/d7/d9/f18 429724 0 2024-03-22T10:42:46.816 INFO:tasks.workunit.client.0.smithi115.stdout:0/538: symlink d3/d4/d14/lae 0 2024-03-22T10:42:46.818 INFO:tasks.workunit.client.0.smithi115.stdout:7/572: dwrite d2/d7/d12/f8d [0,4194304] 0 2024-03-22T10:42:46.818 INFO:tasks.workunit.client.0.smithi115.stdout:1/657: symlink d0/d8/d4d/d56/dbd/ldf 0 2024-03-22T10:42:46.820 INFO:tasks.workunit.client.1.smithi150.stdout:5/115: chown d0/f10 3586 1 2024-03-22T10:42:46.822 INFO:tasks.workunit.client.1.smithi150.stdout:5/116: write d0/f11 [4706724,48474] 0 2024-03-22T10:42:46.824 INFO:tasks.workunit.client.1.smithi150.stdout:4/198: creat d0/d2f/f44 x:0 0 0 2024-03-22T10:42:46.825 INFO:tasks.workunit.client.0.smithi115.stdout:4/511: symlink d1/d30/d4c/d86/d9f/d75/d99/d98/laf 0 2024-03-22T10:42:46.826 INFO:tasks.workunit.client.1.smithi150.stdout:3/173: creat d5/da/f33 x:0 0 0 2024-03-22T10:42:46.827 INFO:tasks.workunit.client.0.smithi115.stdout:2/546: dwrite da/f1d [0,4194304] 0 2024-03-22T10:42:46.828 INFO:tasks.workunit.client.0.smithi115.stdout:3/562: dwrite d3/f53 [0,4194304] 0 2024-03-22T10:42:46.828 INFO:tasks.workunit.client.1.smithi150.stdout:9/189: dwrite d1/d8/de/df/f1f [4194304,4194304] 0 2024-03-22T10:42:46.830 INFO:tasks.workunit.client.1.smithi150.stdout:0/188: truncate d0/d1/d7/db/d10/d1e/d37/f3d 296600 0 2024-03-22T10:42:46.835 INFO:tasks.workunit.client.1.smithi150.stdout:8/194: symlink d0/d2/d19/d29/d2e/l48 0 2024-03-22T10:42:46.840 INFO:tasks.workunit.client.0.smithi115.stdout:2/547: read - da/d25/d3d/d57/f93 zero size 2024-03-22T10:42:46.846 INFO:tasks.workunit.client.1.smithi150.stdout:6/134: link db/dd/f1b db/dd/d25/f34 0 2024-03-22T10:42:46.846 INFO:tasks.workunit.client.1.smithi150.stdout:8/195: write d0/fd [2648737,30969] 0 2024-03-22T10:42:46.847 INFO:tasks.workunit.client.0.smithi115.stdout:7/573: mkdir d2/d7/dc6 0 2024-03-22T10:42:46.848 INFO:tasks.workunit.client.1.smithi150.stdout:1/155: truncate d1/da/fb 195436 0 2024-03-22T10:42:46.849 INFO:tasks.workunit.client.1.smithi150.stdout:0/189: write d0/d12/f3b [470395,9288] 0 2024-03-22T10:42:46.850 INFO:tasks.workunit.client.0.smithi115.stdout:4/512: dwrite d1/f57 [4194304,4194304] 0 2024-03-22T10:42:46.850 INFO:tasks.workunit.client.1.smithi150.stdout:9/190: write d1/d8/de/df/f11 [957197,71175] 0 2024-03-22T10:42:46.850 INFO:tasks.workunit.client.1.smithi150.stdout:3/174: creat d5/da/f34 x:0 0 0 2024-03-22T10:42:46.851 INFO:tasks.workunit.client.0.smithi115.stdout:3/563: dread d3/d82/f83 [0,4194304] 0 2024-03-22T10:42:46.851 INFO:tasks.workunit.client.0.smithi115.stdout:0/539: symlink d3/da/d4c/da3/laf 0 2024-03-22T10:42:46.853 INFO:tasks.workunit.client.1.smithi150.stdout:2/215: dwrite d12/d16/f28 [8388608,4194304] 0 2024-03-22T10:42:46.854 INFO:tasks.workunit.client.1.smithi150.stdout:7/161: mkdir d2/d5/d7/d38 0 2024-03-22T10:42:46.854 INFO:tasks.workunit.client.0.smithi115.stdout:4/513: stat d1/d16/d28/ca5 0 2024-03-22T10:42:46.855 INFO:tasks.workunit.client.1.smithi150.stdout:3/175: readlink d5/l7 0 2024-03-22T10:42:46.856 INFO:tasks.workunit.client.0.smithi115.stdout:2/548: unlink da/d25/d30/f71 0 2024-03-22T10:42:46.858 INFO:tasks.workunit.client.1.smithi150.stdout:2/216: read d12/f13 [1847366,3099] 0 2024-03-22T10:42:46.859 INFO:tasks.workunit.client.1.smithi150.stdout:3/176: fdatasync d5/da/f33 0 2024-03-22T10:42:46.860 INFO:tasks.workunit.client.0.smithi115.stdout:4/514: write d1/d16/d35/d40/d54/f7a [235307,88164] 0 2024-03-22T10:42:46.863 INFO:tasks.workunit.client.0.smithi115.stdout:4/515: write d1/d16/d68/f6c [1793225,124795] 0 2024-03-22T10:42:46.865 INFO:tasks.workunit.client.0.smithi115.stdout:2/549: creat da/d25/d83/fa3 x:0 0 0 2024-03-22T10:42:46.866 INFO:tasks.workunit.client.1.smithi150.stdout:4/199: dwrite d0/f7 [0,4194304] 0 2024-03-22T10:42:46.869 INFO:tasks.workunit.client.1.smithi150.stdout:8/196: chown d0/d2/c10 28775023 1 2024-03-22T10:42:46.870 INFO:tasks.workunit.client.0.smithi115.stdout:2/550: chown da/d25/d30/d4f 799500 1 2024-03-22T10:42:46.870 INFO:tasks.workunit.client.0.smithi115.stdout:1/658: getdents d0/d12/d17/d37 0 2024-03-22T10:42:46.874 INFO:tasks.workunit.client.0.smithi115.stdout:0/540: dwrite d3/d4/d6d/f9c [0,4194304] 0 2024-03-22T10:42:46.876 INFO:tasks.workunit.client.1.smithi150.stdout:6/135: creat db/dd/d15/f35 x:0 0 0 2024-03-22T10:42:46.878 INFO:tasks.workunit.client.0.smithi115.stdout:1/659: mkdir d0/d12/d17/d67/dd8/de0 0 2024-03-22T10:42:46.881 INFO:tasks.workunit.client.0.smithi115.stdout:0/541: write d3/d7/d1c/d6c/d9a/fa5 [167140,102479] 0 2024-03-22T10:42:46.883 INFO:tasks.workunit.client.0.smithi115.stdout:1/660: mknod d0/d8/de/ce1 0 2024-03-22T10:42:46.886 INFO:tasks.workunit.client.1.smithi150.stdout:9/191: symlink d1/d8/de/df/d36/l4c 0 2024-03-22T10:42:46.886 INFO:tasks.workunit.client.0.smithi115.stdout:0/542: creat d3/d4/d6d/fb0 x:0 0 0 2024-03-22T10:42:46.887 INFO:tasks.workunit.client.0.smithi115.stdout:1/661: chown d0/d12/d17/f80 314553173 1 2024-03-22T10:42:46.890 INFO:tasks.workunit.client.1.smithi150.stdout:7/162: creat d2/d5/d7/de/f39 x:0 0 0 2024-03-22T10:42:46.894 INFO:tasks.workunit.client.0.smithi115.stdout:0/543: write d3/d7/d1b/d35/f79 [566342,68536] 0 2024-03-22T10:42:46.898 INFO:tasks.workunit.client.1.smithi150.stdout:1/156: mknod d1/da/d2a/c2c 0 2024-03-22T10:42:46.901 INFO:tasks.workunit.client.1.smithi150.stdout:8/197: creat d0/d2/d19/d29/d31/f49 x:0 0 0 2024-03-22T10:42:46.906 INFO:tasks.workunit.client.1.smithi150.stdout:6/136: symlink db/dd/d2e/l36 0 2024-03-22T10:42:46.910 INFO:tasks.workunit.client.1.smithi150.stdout:7/163: readlink d2/d5/d7/d9/l12 0 2024-03-22T10:42:46.912 INFO:tasks.workunit.client.1.smithi150.stdout:4/200: dread d0/de/f28 [0,4194304] 0 2024-03-22T10:42:46.915 INFO:tasks.workunit.client.0.smithi115.stdout:9/592: link d6/d8/d3c/d59/fb5 d6/d7/d20/d22/d23/fce 0 2024-03-22T10:42:46.916 INFO:tasks.workunit.client.0.smithi115.stdout:9/593: chown d6/d7/d20/d38 1993504 1 2024-03-22T10:42:46.918 INFO:tasks.workunit.client.0.smithi115.stdout:9/594: chown d6/d7/d20/d22/c8d 3 1 2024-03-22T10:42:46.918 INFO:tasks.workunit.client.0.smithi115.stdout:7/574: write d2/d7/d12/d10/d23/f64 [511158,20265] 0 2024-03-22T10:42:46.919 INFO:tasks.workunit.client.1.smithi150.stdout:1/157: mknod d1/d19/d1a/c2d 0 2024-03-22T10:42:46.919 INFO:tasks.workunit.client.0.smithi115.stdout:3/564: write d3/f45 [114517,82212] 0 2024-03-22T10:42:46.926 INFO:tasks.workunit.client.1.smithi150.stdout:3/177: rmdir d5 39 2024-03-22T10:42:46.926 INFO:tasks.workunit.client.0.smithi115.stdout:9/595: rename d6/d7/f1e to d6/fcf 0 2024-03-22T10:42:46.927 INFO:tasks.workunit.client.0.smithi115.stdout:7/575: mkdir d2/d3a/d6e/dc7 0 2024-03-22T10:42:46.929 INFO:tasks.workunit.client.0.smithi115.stdout:3/565: mknod d3/d19/d2b/d49/d91/dad/cb5 0 2024-03-22T10:42:46.931 INFO:tasks.workunit.client.1.smithi150.stdout:0/190: rename d0/d1/l32 to d0/d1/l41 0 2024-03-22T10:42:46.931 INFO:tasks.workunit.client.0.smithi115.stdout:9/596: symlink d6/d7/d20/d22/d3f/ld0 0 2024-03-22T10:42:46.932 INFO:tasks.workunit.client.0.smithi115.stdout:7/576: symlink d2/d62/dc3/lc8 0 2024-03-22T10:42:46.933 INFO:tasks.workunit.client.1.smithi150.stdout:6/137: creat db/dd/f37 x:0 0 0 2024-03-22T10:42:46.933 INFO:tasks.workunit.client.0.smithi115.stdout:3/566: dread - d3/d36/d41/d33/f8d zero size 2024-03-22T10:42:46.935 INFO:tasks.workunit.client.0.smithi115.stdout:4/516: dwrite d1/d30/d4c/d86/d9f/d32/f53 [0,4194304] 0 2024-03-22T10:42:46.935 INFO:tasks.workunit.client.1.smithi150.stdout:5/117: truncate d0/d8/dc/de/f15 1480003 0 2024-03-22T10:42:46.936 INFO:tasks.workunit.client.0.smithi115.stdout:7/577: chown d2/d7/d12/d10/d19/d42/c4d 123893 1 2024-03-22T10:42:46.937 INFO:tasks.workunit.client.1.smithi150.stdout:7/164: dwrite d2/d5/d7/de/f39 [0,4194304] 0 2024-03-22T10:42:46.937 INFO:tasks.workunit.client.0.smithi115.stdout:1/662: dwrite d0/d8/db/d25/f44 [0,4194304] 0 2024-03-22T10:42:46.938 INFO:tasks.workunit.client.1.smithi150.stdout:1/158: mknod d1/da/d15/d18/c2e 0 2024-03-22T10:42:46.938 INFO:tasks.workunit.client.1.smithi150.stdout:6/138: read - db/dd/d21/f2f zero size 2024-03-22T10:42:46.939 INFO:tasks.workunit.client.0.smithi115.stdout:3/567: chown d3/d16/d29/d35/l7d 78 1 2024-03-22T10:42:46.942 INFO:tasks.workunit.client.0.smithi115.stdout:4/517: readlink d1/d16/d35/d40/d5c/d14/l2c 0 2024-03-22T10:42:46.942 INFO:tasks.workunit.client.1.smithi150.stdout:8/198: link d0/f16 d0/d3c/f4a 0 2024-03-22T10:42:46.944 INFO:tasks.workunit.client.1.smithi150.stdout:6/139: truncate db/dd/f2c 534328 0 2024-03-22T10:42:46.945 INFO:tasks.workunit.client.1.smithi150.stdout:4/201: dwrite d0/d2f/f3e [0,4194304] 0 2024-03-22T10:42:46.945 INFO:tasks.workunit.client.1.smithi150.stdout:6/140: chown db/l1d 71031508 1 2024-03-22T10:42:46.947 INFO:tasks.workunit.client.0.smithi115.stdout:3/568: symlink d3/d89/lb6 0 2024-03-22T10:42:46.948 INFO:tasks.workunit.client.1.smithi150.stdout:2/217: truncate d12/d19/f1b 3676037 0 2024-03-22T10:42:46.949 INFO:tasks.workunit.client.1.smithi150.stdout:6/141: write db/dd/f37 [688411,14118] 0 2024-03-22T10:42:46.950 INFO:tasks.workunit.client.0.smithi115.stdout:0/544: dread d3/d7/d1b/f47 [0,4194304] 0 2024-03-22T10:42:46.953 INFO:tasks.workunit.client.1.smithi150.stdout:2/218: fdatasync d12/d19/d2d/d3b/f3d 0 2024-03-22T10:42:46.955 INFO:tasks.workunit.client.0.smithi115.stdout:7/578: rename d2/d7/d12/d10/d23/d3d/da8 to d2/d3a/d6e/d78/dbf/dc9 0 2024-03-22T10:42:46.955 INFO:tasks.workunit.client.0.smithi115.stdout:3/569: getdents d3/d19/d61/daa 0 2024-03-22T10:42:46.955 INFO:tasks.workunit.client.0.smithi115.stdout:0/545: write d3/f7f [5028301,66702] 0 2024-03-22T10:42:46.963 INFO:tasks.workunit.client.1.smithi150.stdout:0/191: rename d0/d1/d7/db to d0/d1/d7/d42 0 2024-03-22T10:42:46.964 INFO:tasks.workunit.client.0.smithi115.stdout:3/570: link d3/d16/f69 d3/d19/d61/da6/fb7 0 2024-03-22T10:42:46.964 INFO:tasks.workunit.client.0.smithi115.stdout:9/597: truncate d6/d7/f17 3405250 0 2024-03-22T10:42:46.964 INFO:tasks.workunit.client.1.smithi150.stdout:9/192: dwrite d1/d8/de/df/d36/d40/d41/f47 [0,4194304] 0 2024-03-22T10:42:46.964 INFO:tasks.workunit.client.1.smithi150.stdout:5/118: creat d0/f22 x:0 0 0 2024-03-22T10:42:46.966 INFO:tasks.workunit.client.0.smithi115.stdout:9/598: fsync d6/d7/d48/d70/f73 0 2024-03-22T10:42:46.967 INFO:tasks.workunit.client.1.smithi150.stdout:5/119: write d0/f22 [216597,67947] 0 2024-03-22T10:42:46.968 INFO:tasks.workunit.client.0.smithi115.stdout:7/579: creat d2/d7/d12/d10/d19/d42/fca x:0 0 0 2024-03-22T10:42:46.968 INFO:tasks.workunit.client.0.smithi115.stdout:1/663: dwrite d0/d8/de/d1a/f32 [0,4194304] 0 2024-03-22T10:42:46.969 INFO:tasks.workunit.client.1.smithi150.stdout:0/192: write d0/d1/d7/d42/fc [207101,56567] 0 2024-03-22T10:42:46.970 INFO:tasks.workunit.client.0.smithi115.stdout:0/546: creat d3/da/fb1 x:0 0 0 2024-03-22T10:42:46.971 INFO:tasks.workunit.client.0.smithi115.stdout:3/571: mkdir d3/d36/d41/d85/db8 0 2024-03-22T10:42:46.972 INFO:tasks.workunit.client.1.smithi150.stdout:1/159: creat d1/d19/d1a/f2f x:0 0 0 2024-03-22T10:42:46.973 INFO:tasks.workunit.client.1.smithi150.stdout:6/142: dwrite db/f2a [0,4194304] 0 2024-03-22T10:42:46.974 INFO:tasks.workunit.client.1.smithi150.stdout:7/165: symlink d2/d5/d7/d8/l3a 0 2024-03-22T10:42:46.974 INFO:tasks.workunit.client.1.smithi150.stdout:5/120: truncate d0/d8/dc/f12 843628 0 2024-03-22T10:42:46.976 INFO:tasks.workunit.client.0.smithi115.stdout:1/664: mkdir d0/d12/d7b/de2 0 2024-03-22T10:42:46.976 INFO:tasks.workunit.client.1.smithi150.stdout:8/199: creat d0/d2/d19/d29/d31/f4b x:0 0 0 2024-03-22T10:42:46.979 INFO:tasks.workunit.client.0.smithi115.stdout:7/580: write d2/d7/d12/d10/d19/f9d [4935367,125749] 0 2024-03-22T10:42:46.980 INFO:tasks.workunit.client.1.smithi150.stdout:3/178: creat d5/f35 x:0 0 0 2024-03-22T10:42:46.980 INFO:tasks.workunit.client.1.smithi150.stdout:2/219: mkdir d12/d19/d2d/d40 0 2024-03-22T10:42:46.981 INFO:tasks.workunit.client.0.smithi115.stdout:9/599: creat d6/d8/d3c/fd1 x:0 0 0 2024-03-22T10:42:46.983 INFO:tasks.workunit.client.0.smithi115.stdout:4/518: dread d1/d30/f4a [4194304,4194304] 0 2024-03-22T10:42:46.983 INFO:tasks.workunit.client.0.smithi115.stdout:7/581: write d2/d62/fa0 [716904,61734] 0 2024-03-22T10:42:46.983 INFO:tasks.workunit.client.0.smithi115.stdout:3/572: truncate d3/f23 944120 0 2024-03-22T10:42:46.984 INFO:tasks.workunit.client.1.smithi150.stdout:9/193: creat d1/d8/de/df/f4d x:0 0 0 2024-03-22T10:42:46.984 INFO:tasks.workunit.client.0.smithi115.stdout:9/600: chown d6/d7/f16 7 1 2024-03-22T10:42:46.985 INFO:tasks.workunit.client.1.smithi150.stdout:3/179: write d5/f21 [1021368,18522] 0 2024-03-22T10:42:46.985 INFO:tasks.workunit.client.0.smithi115.stdout:1/665: fdatasync d0/d8/de/d1a/d91/fcd 0 2024-03-22T10:42:46.987 INFO:tasks.workunit.client.0.smithi115.stdout:7/582: chown d2/d62/dc3/c75 0 1 2024-03-22T10:42:46.987 INFO:tasks.workunit.client.1.smithi150.stdout:2/220: truncate d12/d16/f2a 664571 0 2024-03-22T10:42:46.988 INFO:tasks.workunit.client.1.smithi150.stdout:3/180: write d5/f8 [683251,96263] 0 2024-03-22T10:42:46.988 INFO:tasks.workunit.client.0.smithi115.stdout:3/573: truncate d3/f9 552954 0 2024-03-22T10:42:46.988 INFO:tasks.workunit.client.1.smithi150.stdout:7/166: read d2/d5/d7/de/f22 [542556,73366] 0 2024-03-22T10:42:46.990 INFO:tasks.workunit.client.1.smithi150.stdout:1/160: rmdir d1/da 39 2024-03-22T10:42:46.992 INFO:tasks.workunit.client.1.smithi150.stdout:1/161: dread - d1/d19/d1a/f2f zero size 2024-03-22T10:42:46.993 INFO:tasks.workunit.client.0.smithi115.stdout:9/601: unlink d6/d5e/l91 0 2024-03-22T10:42:46.995 INFO:tasks.workunit.client.1.smithi150.stdout:1/162: dread - d1/d19/d1a/f2b zero size 2024-03-22T10:42:46.996 INFO:tasks.workunit.client.0.smithi115.stdout:1/666: symlink d0/d12/d17/d67/le3 0 2024-03-22T10:42:46.998 INFO:tasks.workunit.client.0.smithi115.stdout:4/519: unlink d1/d30/d4c/d86/d9f/d75/d99/d98/faa 0 2024-03-22T10:42:46.999 INFO:tasks.workunit.client.1.smithi150.stdout:5/121: rmdir d0/d8/dc/de/d14 39 2024-03-22T10:42:47.000 INFO:tasks.workunit.client.0.smithi115.stdout:3/574: creat d3/d16/d29/d9b/fb9 x:0 0 0 2024-03-22T10:42:47.000 INFO:tasks.workunit.client.0.smithi115.stdout:9/602: mknod d6/d7/d48/cd2 0 2024-03-22T10:42:47.001 INFO:tasks.workunit.client.0.smithi115.stdout:4/520: chown d1/d30/l4b 72 1 2024-03-22T10:42:47.002 INFO:tasks.workunit.client.1.smithi150.stdout:1/163: write d1/d19/d1a/f2f [942651,49556] 0 2024-03-22T10:42:47.003 INFO:tasks.workunit.client.1.smithi150.stdout:0/193: dread d0/d1/d7/fe [0,4194304] 0 2024-03-22T10:42:47.003 INFO:tasks.workunit.client.0.smithi115.stdout:7/583: link d2/d7/f20 d2/d62/fcb 0 2024-03-22T10:42:47.005 INFO:tasks.workunit.client.1.smithi150.stdout:6/143: rename db/dd/d15/d28/l29 to db/dd/l38 0 2024-03-22T10:42:47.006 INFO:tasks.workunit.client.0.smithi115.stdout:1/667: link d0/d12/d17/f80 d0/d8/d4d/d56/db6/ddc/fe4 0 2024-03-22T10:42:47.010 INFO:tasks.workunit.client.1.smithi150.stdout:6/144: write db/dd/d21/f2f [806849,67925] 0 2024-03-22T10:42:47.010 INFO:tasks.workunit.client.0.smithi115.stdout:4/521: creat d1/d16/d35/fb0 x:0 0 0 2024-03-22T10:42:47.012 INFO:tasks.workunit.client.0.smithi115.stdout:9/603: creat d6/d7/d87/fd3 x:0 0 0 2024-03-22T10:42:47.014 INFO:tasks.workunit.client.1.smithi150.stdout:9/194: mkdir d1/d8/de/df/d24/d4e 0 2024-03-22T10:42:47.014 INFO:tasks.workunit.client.0.smithi115.stdout:7/584: write d2/d7/d12/d10/d19/fad [578917,40884] 0 2024-03-22T10:42:47.015 INFO:tasks.workunit.client.0.smithi115.stdout:1/668: unlink d0/d12/d17/d37/d9e/caf 0 2024-03-22T10:42:47.018 INFO:tasks.workunit.client.1.smithi150.stdout:3/181: chown d5/d22/c27 5 1 2024-03-22T10:42:47.019 INFO:tasks.workunit.client.1.smithi150.stdout:4/202: dwrite d0/d1b/f2e [4194304,4194304] 0 2024-03-22T10:42:47.020 INFO:tasks.workunit.client.1.smithi150.stdout:3/182: stat d5/da/f33 0 2024-03-22T10:42:47.022 INFO:tasks.workunit.client.0.smithi115.stdout:0/547: write d3/d7/d1b/d1f/f58 [478549,39077] 0 2024-03-22T10:42:47.024 INFO:tasks.workunit.client.0.smithi115.stdout:4/522: mknod d1/d30/d4c/d86/d9f/d75/d99/d98/d9b/cb1 0 2024-03-22T10:42:47.027 INFO:tasks.workunit.client.1.smithi150.stdout:1/164: write d1/d19/d1a/f28 [1788587,103450] 0 2024-03-22T10:42:47.028 INFO:tasks.workunit.client.0.smithi115.stdout:3/575: getdents d3/d6d/db4 0 2024-03-22T10:42:47.029 INFO:tasks.workunit.client.1.smithi150.stdout:2/221: dread d12/f26 [0,4194304] 0 2024-03-22T10:42:47.031 INFO:tasks.workunit.client.0.smithi115.stdout:9/604: write d6/d7/d30/d37/dbe/fc6 [131103,105513] 0 2024-03-22T10:42:47.031 INFO:tasks.workunit.client.0.smithi115.stdout:4/523: write d1/d16/d35/d40/f48 [3292408,99478] 0 2024-03-22T10:42:47.032 INFO:tasks.workunit.client.1.smithi150.stdout:2/222: chown d12/d19/l24 57 1 2024-03-22T10:42:47.033 INFO:tasks.workunit.client.1.smithi150.stdout:1/165: write d1/d19/d1a/f2f [377069,26763] 0 2024-03-22T10:42:47.034 INFO:tasks.workunit.client.1.smithi150.stdout:7/167: getdents d2/d5/d7/d9/db 0 2024-03-22T10:42:47.038 INFO:tasks.workunit.client.0.smithi115.stdout:3/576: creat d3/d36/d41/fba x:0 0 0 2024-03-22T10:42:47.040 INFO:tasks.workunit.client.1.smithi150.stdout:9/195: creat d1/d8/de/df/d20/d32/f4f x:0 0 0 2024-03-22T10:42:47.041 INFO:tasks.workunit.client.0.smithi115.stdout:9/605: creat d6/d7/d30/d37/dbe/fd4 x:0 0 0 2024-03-22T10:42:47.041 INFO:tasks.workunit.client.0.smithi115.stdout:4/524: mknod d1/d16/d35/d40/cb2 0 2024-03-22T10:42:47.042 INFO:tasks.workunit.client.1.smithi150.stdout:4/203: write d0/d2f/f3b [791950,81116] 0 2024-03-22T10:42:47.044 INFO:tasks.workunit.client.1.smithi150.stdout:4/204: readlink d0/l15 0 2024-03-22T10:42:47.046 INFO:tasks.workunit.client.1.smithi150.stdout:3/183: dwrite d5/f10 [0,4194304] 0 2024-03-22T10:42:47.047 INFO:tasks.workunit.client.1.smithi150.stdout:9/196: read d1/d8/de/df/d36/d40/d41/f47 [2710965,41860] 0 2024-03-22T10:42:47.047 INFO:tasks.workunit.client.1.smithi150.stdout:5/122: creat d0/d8/f23 x:0 0 0 2024-03-22T10:42:47.051 INFO:tasks.workunit.client.1.smithi150.stdout:8/200: getdents d0/d2/d19/d29/d31 0 2024-03-22T10:42:47.051 INFO:tasks.workunit.client.1.smithi150.stdout:2/223: creat d12/d19/d2d/d40/f41 x:0 0 0 2024-03-22T10:42:47.054 INFO:tasks.workunit.client.1.smithi150.stdout:9/197: write d1/d8/de/df/f11 [2779888,64812] 0 2024-03-22T10:42:47.054 INFO:tasks.workunit.client.1.smithi150.stdout:8/201: chown d0/d2/fa 58124438 1 2024-03-22T10:42:47.055 INFO:tasks.workunit.client.1.smithi150.stdout:2/224: read d12/d19/d1a/f2b [788056,52548] 0 2024-03-22T10:42:47.056 INFO:tasks.workunit.client.1.smithi150.stdout:8/202: chown d0/l11 423 1 2024-03-22T10:42:47.057 INFO:tasks.workunit.client.1.smithi150.stdout:1/166: symlink d1/da/d15/l30 0 2024-03-22T10:42:47.061 INFO:tasks.workunit.client.1.smithi150.stdout:8/203: write d0/d2/fa [3122831,9429] 0 2024-03-22T10:42:47.061 INFO:tasks.workunit.client.1.smithi150.stdout:0/194: getdents d0/d12 0 2024-03-22T10:42:47.061 INFO:tasks.workunit.client.1.smithi150.stdout:3/184: rmdir d5/d22/d2f 39 2024-03-22T10:42:47.065 INFO:tasks.workunit.client.1.smithi150.stdout:0/195: write d0/f3a [227284,74575] 0 2024-03-22T10:42:47.067 INFO:tasks.workunit.client.1.smithi150.stdout:7/168: link d2/d5/d7/d9/c2f d2/d5/d7/d9/c3b 0 2024-03-22T10:42:47.067 INFO:tasks.workunit.client.1.smithi150.stdout:5/123: creat d0/d8/dc/d1d/f24 x:0 0 0 2024-03-22T10:42:47.067 INFO:tasks.workunit.client.1.smithi150.stdout:8/204: write d0/d2/f12 [2067314,127831] 0 2024-03-22T10:42:47.069 INFO:tasks.workunit.client.1.smithi150.stdout:7/169: dread - d2/f1f zero size 2024-03-22T10:42:47.069 INFO:tasks.workunit.client.1.smithi150.stdout:8/205: readlink d0/d2/d19/d29/l43 0 2024-03-22T10:42:47.071 INFO:tasks.workunit.client.1.smithi150.stdout:9/198: unlink d1/d49/l45 0 2024-03-22T10:42:47.072 INFO:tasks.workunit.client.1.smithi150.stdout:6/145: truncate db/dd/d15/f20 3741816 0 2024-03-22T10:42:47.073 INFO:tasks.workunit.client.1.smithi150.stdout:9/199: readlink d1/l15 0 2024-03-22T10:42:47.078 INFO:tasks.workunit.client.1.smithi150.stdout:5/124: fsync d0/d8/f23 0 2024-03-22T10:42:47.078 INFO:tasks.workunit.client.1.smithi150.stdout:4/205: dread d0/d1b/f2c [0,4194304] 0 2024-03-22T10:42:47.078 INFO:tasks.workunit.client.1.smithi150.stdout:1/167: mknod d1/d19/d1a/c31 0 2024-03-22T10:42:47.079 INFO:tasks.workunit.client.1.smithi150.stdout:0/196: dread d0/f3a [0,4194304] 0 2024-03-22T10:42:47.080 INFO:tasks.workunit.client.1.smithi150.stdout:3/185: unlink d5/l7 0 2024-03-22T10:42:47.082 INFO:tasks.workunit.client.1.smithi150.stdout:8/206: mknod d0/d2/c4c 0 2024-03-22T10:42:47.085 INFO:tasks.workunit.client.1.smithi150.stdout:2/225: link f10 d12/d16/f42 0 2024-03-22T10:42:47.085 INFO:tasks.workunit.client.1.smithi150.stdout:9/200: creat d1/d8/de/df/d20/f50 x:0 0 0 2024-03-22T10:42:47.086 INFO:tasks.workunit.client.1.smithi150.stdout:6/146: rename db/dd/d2e/l36 to db/dd/d15/l39 0 2024-03-22T10:42:47.090 INFO:tasks.workunit.client.1.smithi150.stdout:1/168: symlink d1/d19/d1a/l32 0 2024-03-22T10:42:47.091 INFO:tasks.workunit.client.1.smithi150.stdout:7/170: dwrite d2/f1f [0,4194304] 0 2024-03-22T10:42:47.101 INFO:tasks.workunit.client.1.smithi150.stdout:8/207: dread d0/d2/d19/d29/d2e/f3a [0,4194304] 0 2024-03-22T10:42:47.101 INFO:tasks.workunit.client.1.smithi150.stdout:2/226: chown d12/f15 43978 1 2024-03-22T10:42:47.104 INFO:tasks.workunit.client.1.smithi150.stdout:4/206: unlink d0/d1b/f3a 0 2024-03-22T10:42:47.104 INFO:tasks.workunit.client.1.smithi150.stdout:2/227: chown d12/d19/f2e 59029 1 2024-03-22T10:42:47.105 INFO:tasks.workunit.client.1.smithi150.stdout:8/208: stat d0/d2/d19/l33 0 2024-03-22T10:42:47.105 INFO:tasks.workunit.client.1.smithi150.stdout:0/197: rename d0/d1/d7/f20 to d0/d1/d7/d42/d10/f43 0 2024-03-22T10:42:47.106 INFO:tasks.workunit.client.1.smithi150.stdout:0/198: chown d0 12175889 1 2024-03-22T10:42:47.112 INFO:tasks.workunit.client.1.smithi150.stdout:1/169: write d1/f7 [5084139,125967] 0 2024-03-22T10:42:47.112 INFO:tasks.workunit.client.1.smithi150.stdout:6/147: dwrite db/dd/d21/f2f [0,4194304] 0 2024-03-22T10:42:47.116 INFO:tasks.workunit.client.1.smithi150.stdout:6/148: truncate db/dd/d15/f27 986466 0 2024-03-22T10:42:47.119 INFO:tasks.workunit.client.1.smithi150.stdout:3/186: rename d5/d22/c26 to d5/d22/d2f/c36 0 2024-03-22T10:42:47.119 INFO:tasks.workunit.client.1.smithi150.stdout:1/170: dread - d1/d19/f26 zero size 2024-03-22T10:42:47.121 INFO:tasks.workunit.client.1.smithi150.stdout:5/125: link d0/d8/dc/de/f13 d0/d8/dc/de/d14/f25 0 2024-03-22T10:42:47.124 INFO:tasks.workunit.client.1.smithi150.stdout:5/126: fsync d0/d8/fd 0 2024-03-22T10:42:47.128 INFO:tasks.workunit.client.1.smithi150.stdout:2/228: rename d12/d16/f28 to d12/d16/f43 0 2024-03-22T10:42:47.128 INFO:tasks.workunit.client.1.smithi150.stdout:0/199: rmdir d0/d1/d7/d42/d10/d1e 39 2024-03-22T10:42:47.129 INFO:tasks.workunit.client.1.smithi150.stdout:9/201: write d1/d8/de/df/d36/d40/d41/f47 [1868661,61999] 0 2024-03-22T10:42:47.132 INFO:tasks.workunit.client.1.smithi150.stdout:1/171: rmdir d1/da/d2a 39 2024-03-22T10:42:47.133 INFO:tasks.workunit.client.1.smithi150.stdout:4/207: mknod d0/d42/c45 0 2024-03-22T10:42:47.139 INFO:tasks.workunit.client.1.smithi150.stdout:9/202: dread - d1/d8/de/df/d20/d32/f4f zero size 2024-03-22T10:42:47.140 INFO:tasks.workunit.client.1.smithi150.stdout:2/229: truncate d12/d19/d2d/d40/f41 512220 0 2024-03-22T10:42:47.141 INFO:tasks.workunit.client.1.smithi150.stdout:3/187: mkdir d5/d22/d37 0 2024-03-22T10:42:47.142 INFO:tasks.workunit.client.1.smithi150.stdout:8/209: write d0/d3c/f4a [82604,84529] 0 2024-03-22T10:42:47.144 INFO:tasks.workunit.client.1.smithi150.stdout:6/149: dwrite db/dd/f37 [0,4194304] 0 2024-03-22T10:42:47.144 INFO:tasks.workunit.client.1.smithi150.stdout:4/208: mkdir d0/d2f/d46 0 2024-03-22T10:42:47.148 INFO:tasks.workunit.client.1.smithi150.stdout:8/210: chown d0/d2/d19/d29/d2e 41 1 2024-03-22T10:42:47.149 INFO:tasks.workunit.client.1.smithi150.stdout:7/171: dwrite d2/d5/d7/d8/f2d [0,4194304] 0 2024-03-22T10:42:47.149 INFO:tasks.workunit.client.1.smithi150.stdout:6/150: chown db/dd/d21 7 1 2024-03-22T10:42:47.149 INFO:tasks.workunit.client.1.smithi150.stdout:9/203: symlink d1/d8/de/df/l51 0 2024-03-22T10:42:47.158 INFO:tasks.workunit.client.1.smithi150.stdout:5/127: truncate d0/f10 3059167 0 2024-03-22T10:42:47.159 INFO:tasks.workunit.client.0.smithi115.stdout:3/577: mkdir d3/d19/dbb 0 2024-03-22T10:42:47.160 INFO:tasks.workunit.client.1.smithi150.stdout:0/200: fdatasync d0/d1/d7/f31 0 2024-03-22T10:42:47.161 INFO:tasks.workunit.client.1.smithi150.stdout:9/204: read d1/d8/de/df/f1f [5673221,79795] 0 2024-03-22T10:42:47.162 INFO:tasks.workunit.client.0.smithi115.stdout:7/585: write d2/f5 [2821721,130690] 0 2024-03-22T10:42:47.162 INFO:tasks.workunit.client.0.smithi115.stdout:1/669: truncate d0/d8/de/d1a/f24 1394217 0 2024-03-22T10:42:47.163 INFO:tasks.workunit.client.1.smithi150.stdout:7/172: creat d2/d5/d7/d2e/f3c x:0 0 0 2024-03-22T10:42:47.165 INFO:tasks.workunit.client.0.smithi115.stdout:4/525: creat d1/d30/d4c/d86/d9f/d32/fb3 x:0 0 0 2024-03-22T10:42:47.167 INFO:tasks.workunit.client.0.smithi115.stdout:3/578: rmdir d3/d19/dbb 0 2024-03-22T10:42:47.167 INFO:tasks.workunit.client.1.smithi150.stdout:1/172: getdents d1/da/d15 0 2024-03-22T10:42:47.168 INFO:tasks.workunit.client.1.smithi150.stdout:6/151: mknod db/dd/d25/c3a 0 2024-03-22T10:42:47.170 INFO:tasks.workunit.client.0.smithi115.stdout:1/670: creat d0/d8/db/fe5 x:0 0 0 2024-03-22T10:42:47.171 INFO:tasks.workunit.client.1.smithi150.stdout:2/230: dread d12/d19/d1a/f30 [0,4194304] 0 2024-03-22T10:42:47.172 INFO:tasks.workunit.client.1.smithi150.stdout:4/209: rmdir d0/d2f/d46 0 2024-03-22T10:42:47.172 INFO:tasks.workunit.client.1.smithi150.stdout:1/173: stat d1/da/f16 0 2024-03-22T10:42:47.174 INFO:tasks.workunit.client.0.smithi115.stdout:1/671: mkdir d0/d12/d17/d67/de6 0 2024-03-22T10:42:47.174 INFO:tasks.workunit.client.0.smithi115.stdout:7/586: read d2/d3a/f5f [4986713,125518] 0 2024-03-22T10:42:47.177 INFO:tasks.workunit.client.0.smithi115.stdout:1/672: rmdir d0/d12/d17/d37 39 2024-03-22T10:42:47.178 INFO:tasks.workunit.client.1.smithi150.stdout:4/210: chown d0/d2f/c33 30784099 1 2024-03-22T10:42:47.178 INFO:tasks.workunit.client.0.smithi115.stdout:7/587: creat d2/d62/dc3/fcc x:0 0 0 2024-03-22T10:42:47.179 INFO:tasks.workunit.client.1.smithi150.stdout:5/128: dread d0/d8/dc/de/d14/f25 [0,4194304] 0 2024-03-22T10:42:47.180 INFO:tasks.workunit.client.0.smithi115.stdout:1/673: readlink d0/d12/l43 0 2024-03-22T10:42:47.181 INFO:tasks.workunit.client.1.smithi150.stdout:6/152: chown db/dd/d21/c24 5 1 2024-03-22T10:42:47.182 INFO:tasks.workunit.client.1.smithi150.stdout:3/188: dwrite d5/f24 [0,4194304] 0 2024-03-22T10:42:47.182 INFO:tasks.workunit.client.1.smithi150.stdout:9/205: rename d1/d8/l37 to d1/d49/l52 0 2024-03-22T10:42:47.182 INFO:tasks.workunit.client.0.smithi115.stdout:0/548: dread d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:47.183 INFO:tasks.workunit.client.1.smithi150.stdout:2/231: creat d12/d19/d2d/d40/f44 x:0 0 0 2024-03-22T10:42:47.184 INFO:tasks.workunit.client.0.smithi115.stdout:4/526: dwrite d1/d66/d82/fa6 [0,4194304] 0 2024-03-22T10:42:47.184 INFO:tasks.workunit.client.0.smithi115.stdout:1/674: symlink d0/d8/db/d25/d7e/le7 0 2024-03-22T10:42:47.184 INFO:tasks.workunit.client.1.smithi150.stdout:0/201: dread d0/d1/d7/d42/f2b [0,4194304] 0 2024-03-22T10:42:47.187 INFO:tasks.workunit.client.1.smithi150.stdout:4/211: chown d0/f26 1572 1 2024-03-22T10:42:47.190 INFO:tasks.workunit.client.1.smithi150.stdout:8/211: dwrite d0/f21 [0,4194304] 0 2024-03-22T10:42:47.196 INFO:tasks.workunit.client.1.smithi150.stdout:6/153: rmdir db 39 2024-03-22T10:42:47.200 INFO:tasks.workunit.client.1.smithi150.stdout:7/173: rename d2/d5/d7/d9/l12 to d2/d5/d7/d9/db/l3d 0 2024-03-22T10:42:47.201 INFO:tasks.workunit.client.0.smithi115.stdout:7/588: dwrite d2/d62/dc3/f92 [0,4194304] 0 2024-03-22T10:42:47.203 INFO:tasks.workunit.client.1.smithi150.stdout:5/129: mkdir d0/d26 0 2024-03-22T10:42:47.205 INFO:tasks.workunit.client.0.smithi115.stdout:7/589: fsync d2/d7/d12/d10/d19/fad 0 2024-03-22T10:42:47.205 INFO:tasks.workunit.client.0.smithi115.stdout:0/549: dread d3/d4/d14/f4f [0,4194304] 0 2024-03-22T10:42:47.206 INFO:tasks.workunit.client.0.smithi115.stdout:1/675: link d0/d12/d17/d37/d5c/f8c d0/d8/d4d/d56/db6/ddc/d82/fe8 0 2024-03-22T10:42:47.206 INFO:tasks.workunit.client.1.smithi150.stdout:1/174: truncate d1/da/fb 924884 0 2024-03-22T10:42:47.209 INFO:tasks.workunit.client.1.smithi150.stdout:3/189: mkdir d5/d22/d2f/d38 0 2024-03-22T10:42:47.210 INFO:tasks.workunit.client.0.smithi115.stdout:0/550: write d3/d4/d6d/f98 [3259865,30402] 0 2024-03-22T10:42:47.210 INFO:tasks.workunit.client.0.smithi115.stdout:7/590: chown d2/d7/d12/d10/d19/d42/f85 1 1 2024-03-22T10:42:47.210 INFO:tasks.workunit.client.1.smithi150.stdout:8/212: mkdir d0/d2/d4d 0 2024-03-22T10:42:47.211 INFO:tasks.workunit.client.0.smithi115.stdout:7/591: stat d2/d62/dc3/lc8 0 2024-03-22T10:42:47.213 INFO:tasks.workunit.client.1.smithi150.stdout:9/206: write d1/d8/de/f2c [844119,104138] 0 2024-03-22T10:42:47.214 INFO:tasks.workunit.client.0.smithi115.stdout:0/551: truncate d3/da/d4c/f67 4460970 0 2024-03-22T10:42:47.214 INFO:tasks.workunit.client.0.smithi115.stdout:7/592: dread - d2/d7/d12/fa7 zero size 2024-03-22T10:42:47.214 INFO:tasks.workunit.client.1.smithi150.stdout:7/174: rename d2/d5/d7/d8/l24 to d2/l3e 0 2024-03-22T10:42:47.217 INFO:tasks.workunit.client.0.smithi115.stdout:4/527: write d1/d66/d82/f9a [473296,69020] 0 2024-03-22T10:42:47.218 INFO:tasks.workunit.client.1.smithi150.stdout:2/232: write d12/f13 [3223631,125042] 0 2024-03-22T10:42:47.218 INFO:tasks.workunit.client.1.smithi150.stdout:1/175: symlink d1/da/d15/d18/l33 0 2024-03-22T10:42:47.218 INFO:tasks.workunit.client.0.smithi115.stdout:3/579: write d3/f9 [875873,97161] 0 2024-03-22T10:42:47.219 INFO:tasks.workunit.client.0.smithi115.stdout:0/552: mknod d3/d7/d1b/d35/cb2 0 2024-03-22T10:42:47.220 INFO:tasks.workunit.client.1.smithi150.stdout:4/212: mkdir d0/d47 0 2024-03-22T10:42:47.221 INFO:tasks.workunit.client.1.smithi150.stdout:1/176: readlink d1/d19/d1a/l32 0 2024-03-22T10:42:47.223 INFO:tasks.workunit.client.0.smithi115.stdout:4/528: chown d1/d16/d35/d40/la2 28340367 1 2024-03-22T10:42:47.226 INFO:tasks.workunit.client.1.smithi150.stdout:9/207: rename d1/d8/de/df/f12 to d1/d8/de/df/d24/f53 0 2024-03-22T10:42:47.228 INFO:tasks.workunit.client.1.smithi150.stdout:2/233: mknod d12/d19/d2d/d40/c45 0 2024-03-22T10:42:47.229 INFO:tasks.workunit.client.0.smithi115.stdout:4/529: dread - d1/d16/d68/fae zero size 2024-03-22T10:42:47.229 INFO:tasks.workunit.client.0.smithi115.stdout:7/593: rename d2/d7/d12/d10/d19/d42/c79 to d2/d7/d12/d10/d19/d42/d46/d94/ccd 0 2024-03-22T10:42:47.232 INFO:tasks.workunit.client.1.smithi150.stdout:3/190: creat d5/d22/d2f/d38/f39 x:0 0 0 2024-03-22T10:42:47.232 INFO:tasks.workunit.client.1.smithi150.stdout:6/154: link db/dd/d21/f2f db/dd/d25/f3b 0 2024-03-22T10:42:47.237 INFO:tasks.workunit.client.1.smithi150.stdout:6/155: read db/dd/f14 [4301977,2987] 0 2024-03-22T10:42:47.239 INFO:tasks.workunit.client.1.smithi150.stdout:9/208: creat d1/d8/de/df/d20/d29/f54 x:0 0 0 2024-03-22T10:42:47.245 INFO:tasks.workunit.client.1.smithi150.stdout:4/213: link d0/f9 d0/d42/f48 0 2024-03-22T10:42:47.246 INFO:tasks.workunit.client.1.smithi150.stdout:8/213: dwrite d0/d2/d19/d29/d2f/df/d1b/f24 [0,4194304] 0 2024-03-22T10:42:47.247 INFO:tasks.workunit.client.0.smithi115.stdout:0/553: dwrite d3/d7/d1b/d35/f42 [0,4194304] 0 2024-03-22T10:42:47.254 INFO:tasks.workunit.client.0.smithi115.stdout:4/530: rename d1/d16/f26 to d1/d30/d78/fb4 0 2024-03-22T10:42:47.257 INFO:tasks.workunit.client.0.smithi115.stdout:3/580: getdents d3/d36/d41 0 2024-03-22T10:42:47.257 INFO:tasks.workunit.client.1.smithi150.stdout:2/234: dread d12/f13 [0,4194304] 0 2024-03-22T10:42:47.258 INFO:tasks.workunit.client.0.smithi115.stdout:7/594: mkdir d2/d7/d12/d10/d19/dbd/dce 0 2024-03-22T10:42:47.259 INFO:tasks.workunit.client.1.smithi150.stdout:1/177: write d1/da/f16 [1826935,79363] 0 2024-03-22T10:42:47.260 INFO:tasks.workunit.client.1.smithi150.stdout:5/130: dwrite d0/d8/dc/de/f13 [0,4194304] 0 2024-03-22T10:42:47.260 INFO:tasks.workunit.client.0.smithi115.stdout:0/554: symlink d3/da/lb3 0 2024-03-22T10:42:47.261 INFO:tasks.workunit.client.0.smithi115.stdout:4/531: read d1/d16/d68/f89 [2914899,108460] 0 2024-03-22T10:42:47.261 INFO:tasks.workunit.client.0.smithi115.stdout:7/595: chown d2/d7/d12/d10/f40 77004150 1 2024-03-22T10:42:47.261 INFO:tasks.workunit.client.0.smithi115.stdout:3/581: mknod d3/d82/d54/cbc 0 2024-03-22T10:42:47.262 INFO:tasks.workunit.client.1.smithi150.stdout:0/202: link d0/d1/d7/d42/d10/d1e/d37/f3d d0/d1/d7/d42/f44 0 2024-03-22T10:42:47.263 INFO:tasks.workunit.client.1.smithi150.stdout:3/191: dwrite d5/da/f28 [0,4194304] 0 2024-03-22T10:42:47.264 INFO:tasks.workunit.client.0.smithi115.stdout:0/555: symlink d3/da/d4c/da3/lb4 0 2024-03-22T10:42:47.266 INFO:tasks.workunit.client.1.smithi150.stdout:9/209: unlink d1/d8/de/f3c 0 2024-03-22T10:42:47.266 INFO:tasks.workunit.client.1.smithi150.stdout:7/175: link d2/d5/d7/d9/c3b d2/d5/d7/d9/db/c3f 0 2024-03-22T10:42:47.267 INFO:tasks.workunit.client.1.smithi150.stdout:2/235: read d12/d19/d1a/f30 [3807488,86868] 0 2024-03-22T10:42:47.270 INFO:tasks.workunit.client.0.smithi115.stdout:4/532: getdents d1/d16/d35/d40/d54 0 2024-03-22T10:42:47.270 INFO:tasks.workunit.client.1.smithi150.stdout:5/131: chown d0/d8/dc/de/d14/f25 3 1 2024-03-22T10:42:47.270 INFO:tasks.workunit.client.0.smithi115.stdout:0/556: symlink d3/d7/d1b/d1f/lb5 0 2024-03-22T10:42:47.272 INFO:tasks.workunit.client.0.smithi115.stdout:0/557: chown d3/d7/d1b/d93 38219955 1 2024-03-22T10:42:47.274 INFO:tasks.workunit.client.1.smithi150.stdout:3/192: fdatasync d5/f10 0 2024-03-22T10:42:47.276 INFO:tasks.workunit.client.1.smithi150.stdout:0/203: write d0/d1/d7/d42/fc [4906667,39665] 0 2024-03-22T10:42:47.279 INFO:tasks.workunit.client.0.smithi115.stdout:0/558: creat d3/da/d4c/d74/fb6 x:0 0 0 2024-03-22T10:42:47.280 INFO:tasks.workunit.client.1.smithi150.stdout:9/210: dread - d1/d8/de/df/d24/f4a zero size 2024-03-22T10:42:47.281 INFO:tasks.workunit.client.1.smithi150.stdout:1/178: read d1/f1d [221444,93350] 0 2024-03-22T10:42:47.281 INFO:tasks.workunit.client.0.smithi115.stdout:0/559: fdatasync d3/d7/d1b/d1f/f41 0 2024-03-22T10:42:47.282 INFO:tasks.workunit.client.0.smithi115.stdout:4/533: symlink d1/d30/d4c/d86/d9f/d75/d99/d98/lb5 0 2024-03-22T10:42:47.284 INFO:tasks.workunit.client.0.smithi115.stdout:3/582: dwrite d3/d36/d41/fa2 [0,4194304] 0 2024-03-22T10:42:47.285 INFO:tasks.workunit.client.0.smithi115.stdout:0/560: write d3/d7/f5c [4918289,33409] 0 2024-03-22T10:42:47.285 INFO:tasks.workunit.client.1.smithi150.stdout:7/176: creat d2/d5/d7/d8/d2c/f40 x:0 0 0 2024-03-22T10:42:47.286 INFO:tasks.workunit.client.0.smithi115.stdout:4/534: write d1/d66/d82/f9a [585580,11311] 0 2024-03-22T10:42:47.286 INFO:tasks.workunit.client.1.smithi150.stdout:3/193: write d5/d22/f2b [101462,100422] 0 2024-03-22T10:42:47.287 INFO:tasks.workunit.client.1.smithi150.stdout:3/194: dread - d5/da/f34 zero size 2024-03-22T10:42:47.287 INFO:tasks.workunit.client.1.smithi150.stdout:7/177: readlink d2/d5/d7/d9/db/l11 0 2024-03-22T10:42:47.291 INFO:tasks.workunit.client.0.smithi115.stdout:4/535: creat d1/d30/d4c/d86/d9f/d73/fb6 x:0 0 0 2024-03-22T10:42:47.294 INFO:tasks.workunit.client.1.smithi150.stdout:3/195: write d5/d22/d2f/f30 [2179267,119716] 0 2024-03-22T10:42:47.296 INFO:tasks.workunit.client.1.smithi150.stdout:8/214: dwrite d0/f1 [4194304,4194304] 0 2024-03-22T10:42:47.298 INFO:tasks.workunit.client.1.smithi150.stdout:6/156: dread db/dd/f1b [0,4194304] 0 2024-03-22T10:42:47.299 INFO:tasks.workunit.client.1.smithi150.stdout:8/215: read d0/fd [1015361,121890] 0 2024-03-22T10:42:47.301 INFO:tasks.workunit.client.1.smithi150.stdout:6/157: write db/dd/d15/f27 [1016993,10556] 0 2024-03-22T10:42:47.308 INFO:tasks.workunit.client.0.smithi115.stdout:7/596: truncate d2/d3a/d6e/fa2 3491790 0 2024-03-22T10:42:47.310 INFO:tasks.workunit.client.0.smithi115.stdout:7/597: fdatasync d2/d7/d12/d10/d23/f6f 0 2024-03-22T10:42:47.311 INFO:tasks.workunit.client.0.smithi115.stdout:4/536: truncate d1/f9 798371 0 2024-03-22T10:42:47.314 INFO:tasks.workunit.client.0.smithi115.stdout:7/598: creat d2/d3a/d6e/d78/dbf/dc9/fcf x:0 0 0 2024-03-22T10:42:47.317 INFO:tasks.workunit.client.0.smithi115.stdout:7/599: creat d2/d7/d12/d10/d19/d42/d76/d74/fd0 x:0 0 0 2024-03-22T10:42:47.321 INFO:tasks.workunit.client.0.smithi115.stdout:7/600: write d2/d7/d12/d10/d19/d42/d76/f83 [781340,128896] 0 2024-03-22T10:42:47.323 INFO:tasks.workunit.client.0.smithi115.stdout:4/537: write d1/d16/f81 [753787,32952] 0 2024-03-22T10:42:47.325 INFO:tasks.workunit.client.0.smithi115.stdout:4/538: write d1/d69/f65 [641209,46387] 0 2024-03-22T10:42:47.327 INFO:tasks.workunit.client.0.smithi115.stdout:7/601: creat d2/d7/d12/d10/d19/dbd/dce/fd1 x:0 0 0 2024-03-22T10:42:47.330 INFO:tasks.workunit.client.1.smithi150.stdout:9/211: symlink d1/d8/de/df/d24/l55 0 2024-03-22T10:42:47.332 INFO:tasks.workunit.client.0.smithi115.stdout:7/602: rename d2/d62/dc3 to d2/d7/d12/d10/d23/d3d/dd2 0 2024-03-22T10:42:47.334 INFO:tasks.workunit.client.0.smithi115.stdout:4/539: write d1/f7 [610894,92148] 0 2024-03-22T10:42:47.335 INFO:tasks.workunit.client.0.smithi115.stdout:7/603: truncate d2/d7/d12/d10/d19/dbd/dce/fd1 474573 0 2024-03-22T10:42:47.336 INFO:tasks.workunit.client.0.smithi115.stdout:7/604: chown d2/d3a 34671 1 2024-03-22T10:42:47.337 INFO:tasks.workunit.client.0.smithi115.stdout:4/540: creat d1/d16/d3b/fb7 x:0 0 0 2024-03-22T10:42:47.337 INFO:tasks.workunit.client.1.smithi150.stdout:6/158: readlink db/dd/d15/l39 0 2024-03-22T10:42:47.338 INFO:tasks.workunit.client.0.smithi115.stdout:7/605: mknod d2/d7/d12/d10/cd3 0 2024-03-22T10:42:47.339 INFO:tasks.workunit.client.0.smithi115.stdout:4/541: mkdir d1/d30/d4c/d86/d9f/db8 0 2024-03-22T10:42:47.340 INFO:tasks.workunit.client.1.smithi150.stdout:8/216: rename d0/d2/d19/d29/d31/f39 to d0/d2/d46/f4e 0 2024-03-22T10:42:47.341 INFO:tasks.workunit.client.0.smithi115.stdout:4/542: symlink d1/d66/d82/lb9 0 2024-03-22T10:42:47.342 INFO:tasks.workunit.client.0.smithi115.stdout:7/606: rmdir d2/d3a/d6e/d78/dbf/dae 39 2024-03-22T10:42:47.342 INFO:tasks.workunit.client.0.smithi115.stdout:4/543: stat d1/d16/d35/l71 0 2024-03-22T10:42:47.342 INFO:tasks.workunit.client.1.smithi150.stdout:1/179: dwrite d1/f1d [0,4194304] 0 2024-03-22T10:42:47.343 INFO:tasks.workunit.client.0.smithi115.stdout:5/584: sync 2024-03-22T10:42:47.343 INFO:tasks.workunit.client.1.smithi150.stdout:0/204: creat d0/f45 x:0 0 0 2024-03-22T10:42:47.344 INFO:tasks.workunit.client.0.smithi115.stdout:7/607: readlink d2/d7/d12/d10/d23/d3d/dd2/lc8 0 2024-03-22T10:42:47.344 INFO:tasks.workunit.client.1.smithi150.stdout:3/196: mknod d5/d22/c3a 0 2024-03-22T10:42:47.345 INFO:tasks.workunit.client.0.smithi115.stdout:4/544: mknod d1/d30/cba 0 2024-03-22T10:42:47.347 INFO:tasks.workunit.client.0.smithi115.stdout:7/608: write d2/d7/d12/d10/d23/f38 [413940,22314] 0 2024-03-22T10:42:47.348 INFO:tasks.workunit.client.1.smithi150.stdout:7/178: link d2/d5/c27 d2/d5/d7/d9/db/c41 0 2024-03-22T10:42:47.349 INFO:tasks.workunit.client.0.smithi115.stdout:4/545: mkdir d1/dbb 0 2024-03-22T10:42:47.350 INFO:tasks.workunit.client.1.smithi150.stdout:9/212: symlink d1/d8/de/df/d36/d40/d41/l56 0 2024-03-22T10:42:47.351 INFO:tasks.workunit.client.0.smithi115.stdout:5/585: symlink d2/d15/d1b/d7c/d38/lc6 0 2024-03-22T10:42:47.353 INFO:tasks.workunit.client.1.smithi150.stdout:0/205: fsync d0/d1/d7/d42/f44 0 2024-03-22T10:42:47.353 INFO:tasks.workunit.client.0.smithi115.stdout:7/609: chown d2/d3a/d6e/d78/dbf/dae/dbb/c8c 153 1 2024-03-22T10:42:47.354 INFO:tasks.workunit.client.1.smithi150.stdout:8/217: write d0/d2/d19/d29/d2f/f3b [664208,17013] 0 2024-03-22T10:42:47.354 INFO:tasks.workunit.client.1.smithi150.stdout:3/197: write d5/da/f33 [1006013,80117] 0 2024-03-22T10:42:47.355 INFO:tasks.workunit.client.0.smithi115.stdout:7/610: stat d2/d7/d12/d10/d19/d42/d76/fb1 0 2024-03-22T10:42:47.355 INFO:tasks.workunit.client.0.smithi115.stdout:5/586: creat d2/fc7 x:0 0 0 2024-03-22T10:42:47.356 INFO:tasks.workunit.client.1.smithi150.stdout:2/236: dwrite d12/d16/f42 [0,4194304] 0 2024-03-22T10:42:47.357 INFO:tasks.workunit.client.0.smithi115.stdout:4/546: link d1/c5 d1/d30/d4c/d86/d9f/d75/d99/da8/cbc 0 2024-03-22T10:42:47.357 INFO:tasks.workunit.client.0.smithi115.stdout:5/587: chown d2/df/d13/c3f 28 1 2024-03-22T10:42:47.359 INFO:tasks.workunit.client.0.smithi115.stdout:5/588: readlink d2/d15/d32/l41 0 2024-03-22T10:42:47.359 INFO:tasks.workunit.client.0.smithi115.stdout:7/611: write d2/d7/d12/d10/d19/d42/d76/f98 [570494,58916] 0 2024-03-22T10:42:47.359 INFO:tasks.workunit.client.1.smithi150.stdout:1/180: symlink d1/da/d15/d18/l34 0 2024-03-22T10:42:47.360 INFO:tasks.workunit.client.0.smithi115.stdout:4/547: creat d1/d16/d35/fbd x:0 0 0 2024-03-22T10:42:47.362 INFO:tasks.workunit.client.1.smithi150.stdout:3/198: write d5/f10 [3650412,26924] 0 2024-03-22T10:42:47.364 INFO:tasks.workunit.client.1.smithi150.stdout:1/181: dread - d1/f29 zero size 2024-03-22T10:42:47.364 INFO:tasks.workunit.client.1.smithi150.stdout:7/179: creat d2/d5/d7/d8/f42 x:0 0 0 2024-03-22T10:42:47.367 INFO:tasks.workunit.client.1.smithi150.stdout:9/213: mkdir d1/d8/de/df/d20/d32/d57 0 2024-03-22T10:42:47.375 INFO:tasks.workunit.client.1.smithi150.stdout:6/159: dwrite db/dd/d2e/f31 [0,4194304] 0 2024-03-22T10:42:47.375 INFO:tasks.workunit.client.1.smithi150.stdout:3/199: symlink d5/l3b 0 2024-03-22T10:42:47.382 INFO:tasks.workunit.client.1.smithi150.stdout:7/180: symlink d2/d5/d7/d8/l43 0 2024-03-22T10:42:47.383 INFO:tasks.workunit.client.1.smithi150.stdout:9/214: chown d1/d8/l1e 10035 1 2024-03-22T10:42:47.384 INFO:tasks.workunit.client.1.smithi150.stdout:1/182: creat d1/da/d2a/f35 x:0 0 0 2024-03-22T10:42:47.385 INFO:tasks.workunit.client.1.smithi150.stdout:3/200: rmdir d5/d22/d2f 39 2024-03-22T10:42:47.392 INFO:tasks.workunit.client.1.smithi150.stdout:0/206: link d0/d1/f39 d0/d1/d7/d42/d10/d38/f46 0 2024-03-22T10:42:47.393 INFO:tasks.workunit.client.1.smithi150.stdout:7/181: mknod d2/d5/d7/d8/c44 0 2024-03-22T10:42:47.394 INFO:tasks.workunit.client.1.smithi150.stdout:6/160: write db/dd/d2e/f31 [4748447,25409] 0 2024-03-22T10:42:47.400 INFO:tasks.workunit.client.1.smithi150.stdout:9/215: mkdir d1/d8/de/df/d24/d58 0 2024-03-22T10:42:47.401 INFO:tasks.workunit.client.1.smithi150.stdout:3/201: creat d5/f3c x:0 0 0 2024-03-22T10:42:47.401 INFO:tasks.workunit.client.1.smithi150.stdout:1/183: write d1/d19/f26 [383742,106632] 0 2024-03-22T10:42:47.402 INFO:tasks.workunit.client.1.smithi150.stdout:2/237: link d12/d19/d2d/l3a d12/l46 0 2024-03-22T10:42:47.406 INFO:tasks.workunit.client.1.smithi150.stdout:7/182: chown d2/l3e 121231 1 2024-03-22T10:42:47.407 INFO:tasks.workunit.client.1.smithi150.stdout:0/207: creat d0/d1/d7/d42/d10/f47 x:0 0 0 2024-03-22T10:42:47.408 INFO:tasks.workunit.client.1.smithi150.stdout:2/238: mknod d12/c47 0 2024-03-22T10:42:47.408 INFO:tasks.workunit.client.1.smithi150.stdout:8/218: dread d0/d2/d19/d29/d2f/df/d1b/f1c [0,4194304] 0 2024-03-22T10:42:47.410 INFO:tasks.workunit.client.1.smithi150.stdout:1/184: unlink d1/d19/d1a/f28 0 2024-03-22T10:42:47.412 INFO:tasks.workunit.client.1.smithi150.stdout:1/185: dread - d1/da/d2a/f35 zero size 2024-03-22T10:42:47.418 INFO:tasks.workunit.client.1.smithi150.stdout:8/219: creat d0/d2/d19/d29/d2f/df/f4f x:0 0 0 2024-03-22T10:42:47.418 INFO:tasks.workunit.client.1.smithi150.stdout:3/202: creat d5/d22/d37/f3d x:0 0 0 2024-03-22T10:42:47.420 INFO:tasks.workunit.client.1.smithi150.stdout:3/203: fdatasync d5/d22/f2b 0 2024-03-22T10:42:47.421 INFO:tasks.workunit.client.1.smithi150.stdout:3/204: chown d5/d22 6309985 1 2024-03-22T10:42:47.423 INFO:tasks.workunit.client.1.smithi150.stdout:2/239: rename d12/d19/d2d/d40 to d12/d48 0 2024-03-22T10:42:47.424 INFO:tasks.workunit.client.1.smithi150.stdout:8/220: rename d0/d2/d19/d29/d2f/df to d0/d2/d19/d29/d2f/df/d50 22 2024-03-22T10:42:47.425 INFO:tasks.workunit.client.1.smithi150.stdout:9/216: dread d1/d8/de/f17 [0,4194304] 0 2024-03-22T10:42:47.426 INFO:tasks.workunit.client.1.smithi150.stdout:0/208: rmdir d0/d1/d7/d42/d10 39 2024-03-22T10:42:47.431 INFO:tasks.workunit.client.1.smithi150.stdout:1/186: getdents d1/da/d15/d18 0 2024-03-22T10:42:47.431 INFO:tasks.workunit.client.1.smithi150.stdout:7/183: write d2/d5/d7/de/f22 [9292348,41763] 0 2024-03-22T10:42:47.434 INFO:tasks.workunit.client.1.smithi150.stdout:7/184: dread - d2/d5/d7/f2b zero size 2024-03-22T10:42:47.434 INFO:tasks.workunit.client.1.smithi150.stdout:9/217: symlink d1/d49/l59 0 2024-03-22T10:42:47.435 INFO:tasks.workunit.client.1.smithi150.stdout:3/205: mkdir d5/d22/d2f/d38/d3e 0 2024-03-22T10:42:47.438 INFO:tasks.workunit.client.1.smithi150.stdout:7/185: stat d2/d5/d7/de/l15 0 2024-03-22T10:42:47.438 INFO:tasks.workunit.client.1.smithi150.stdout:9/218: fsync d1/d8/de/df/d20/f50 0 2024-03-22T10:42:47.440 INFO:tasks.workunit.client.1.smithi150.stdout:1/187: link d1/d19/d1a/c2d d1/da/d2a/c36 0 2024-03-22T10:42:47.442 INFO:tasks.workunit.client.1.smithi150.stdout:0/209: link d0/d1/d7/f28 d0/d1/f48 0 2024-03-22T10:42:47.443 INFO:tasks.workunit.client.1.smithi150.stdout:7/186: mkdir d2/d5/d7/d8/d2c/d45 0 2024-03-22T10:42:47.444 INFO:tasks.workunit.client.1.smithi150.stdout:2/240: dwrite d12/f15 [0,4194304] 0 2024-03-22T10:42:47.445 INFO:tasks.workunit.client.1.smithi150.stdout:1/188: symlink d1/d19/d1a/l37 0 2024-03-22T10:42:47.447 INFO:tasks.workunit.client.1.smithi150.stdout:3/206: truncate d5/da/ff 1731043 0 2024-03-22T10:42:47.447 INFO:tasks.workunit.client.1.smithi150.stdout:7/187: write d2/f1f [530483,103990] 0 2024-03-22T10:42:47.452 INFO:tasks.workunit.client.1.smithi150.stdout:2/241: symlink d12/d19/d1a/l49 0 2024-03-22T10:42:47.452 INFO:tasks.workunit.client.1.smithi150.stdout:1/189: read d1/da/f13 [262140,57762] 0 2024-03-22T10:42:47.453 INFO:tasks.workunit.client.1.smithi150.stdout:8/221: dwrite d0/d2/d19/d29/d2f/df/f26 [0,4194304] 0 2024-03-22T10:42:47.454 INFO:tasks.workunit.client.1.smithi150.stdout:1/190: fsync d1/d19/f26 0 2024-03-22T10:42:47.456 INFO:tasks.workunit.client.1.smithi150.stdout:1/191: readlink d1/l6 0 2024-03-22T10:42:47.536 INFO:tasks.workunit.client.0.smithi115.stdout:4/548: creat d1/dbb/fbe x:0 0 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:5/589: dread - d2/df/d13/d8b/d93/da6/dbc/d50/f86 zero size 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:7/612: creat d2/d7/d12/d10/d23/fd4 x:0 0 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:5/590: fdatasync d2/df/f55 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:4/549: mknod d1/d16/d35/d40/d54/cbf 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:4/550: mknod d1/d30/d4c/d86/d9f/cc0 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:7/613: read d2/d7/d12/d10/d23/d3d/dd2/fba [608425,37255] 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:4/551: fsync d1/d30/f4a 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:4/552: readlink d1/d66/d82/lb9 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:4/553: chown d1/d16/d35/d40/d5c/f25 1398462 1 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:7/614: rename d2/d7/d12/d10/d19/d42/d76/fb1 to d2/d3a/d6e/fd5 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:4/554: creat d1/d66/fc1 x:0 0 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:4/555: stat d1/d30/d4c/d86/f8a 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:5/591: dread d2/d15/d1b/d7c/d38/f78 [0,4194304] 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:7/615: getdents d2/d7/d12/d10/d19/dbd/dce 0 2024-03-22T10:42:47.537 INFO:tasks.workunit.client.0.smithi115.stdout:7/616: fsync d2/d7/d12/d10/d19/fad 0 2024-03-22T10:42:47.538 INFO:tasks.workunit.client.0.smithi115.stdout:7/617: symlink d2/d3a/ld6 0 2024-03-22T10:42:47.538 INFO:tasks.workunit.client.0.smithi115.stdout:7/618: fsync d2/d7/d12/d10/d19/d42/f85 0 2024-03-22T10:42:47.538 INFO:tasks.workunit.client.0.smithi115.stdout:4/556: dwrite d1/d16/f64 [0,4194304] 0 2024-03-22T10:42:47.538 INFO:tasks.workunit.client.0.smithi115.stdout:7/619: dread - d2/d7/d12/d10/d23/d3d/dd2/da3/faf zero size 2024-03-22T10:42:47.539 INFO:tasks.workunit.client.1.smithi150.stdout:7/188: mkdir d2/d5/d7/d8/d46 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:7/189: write d2/d5/d7/f2b [598156,78840] 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:0/210: unlink d0/d1/d7/d42/d10/d38/l40 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:3/207: creat d5/d22/d2f/d38/d3e/f3f x:0 0 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:2/242: rename d12/d16/l39 to d12/d19/d2d/l4a 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:8/222: getdents d0/d2/d19/d29/d2e 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:9/219: dwrite d1/d8/de/df/d20/d32/f4f [0,4194304] 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:8/223: write d0/d2/fe [1200029,27848] 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:1/192: dwrite d1/d19/d1a/f1e [0,4194304] 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:0/211: rename d0/l1c to d0/d1/d7/d42/d10/d1e/d35/l49 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:0/212: chown d0/d1/d7/d42 1420594 1 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:1/193: chown d1/da/d15/d18/l34 44 1 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:7/190: write d2/d5/d7/d9/f18 [1220615,94888] 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:1/194: unlink d1/d19/d1a/f2f 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:9/220: dread d1/d19/f26 [0,4194304] 0 2024-03-22T10:42:47.540 INFO:tasks.workunit.client.1.smithi150.stdout:2/243: link d12/f26 d12/d21/f4b 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:8/224: getdents d0/d3c 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:7/191: creat d2/d5/d7/d8/d2c/f47 x:0 0 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:2/244: chown d12/d19/d2d/f36 410047196 1 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:3/208: truncate d5/f10 4051484 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:9/221: rename d1/d8/de/df/d24/l28 to d1/d8/de/df/d24/l5a 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:7/192: read d2/f1f [4058070,83617] 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:3/209: fsync d5/da/f19 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:7/193: stat d2/d5/d7/d9/db/l11 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:0/213: dwrite d0/f45 [0,4194304] 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:8/225: creat d0/d2/d19/d29/d2f/df/f51 x:0 0 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:9/222: unlink d1/d8/de/df/d24/l55 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:8/226: dread - d0/d2/d19/f47 zero size 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:0/214: chown d0/d1/d7/d42/d10/d1e/d35/l49 4970590 1 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:8/227: creat d0/d2/d46/f52 x:0 0 0 2024-03-22T10:42:47.541 INFO:tasks.workunit.client.1.smithi150.stdout:8/228: chown d0/d3c 115837 1 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:1/195: dwrite d1/da/f13 [0,4194304] 0 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:8/229: chown d0/c1f 27473 1 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:1/196: chown d1/da/l25 1 1 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:0/215: creat d0/d1/d7/d42/f4a x:0 0 0 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:1/197: truncate d1/d19/d1a/f2b 273974 0 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:8/230: mknod d0/d2/d19/d29/d2f/df/d1b/d3d/c53 0 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:0/216: dread d0/d1/d7/d42/f2b [0,4194304] 0 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:1/198: unlink d1/c2 0 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:8/231: dread - d0/d2/d19/d29/d2f/df/f45 zero size 2024-03-22T10:42:47.542 INFO:tasks.workunit.client.1.smithi150.stdout:0/217: creat d0/d1/d7/d42/d10/d38/f4b x:0 0 0 2024-03-22T10:42:47.543 INFO:tasks.workunit.client.0.smithi115.stdout:7/620: symlink d2/d62/ld7 0 2024-03-22T10:42:47.543 INFO:tasks.workunit.client.1.smithi150.stdout:8/232: write d0/d2/d46/f52 [845457,51937] 0 2024-03-22T10:42:47.544 INFO:tasks.workunit.client.0.smithi115.stdout:6/546: sync 2024-03-22T10:42:47.544 INFO:tasks.workunit.client.0.smithi115.stdout:8/766: sync 2024-03-22T10:42:47.545 INFO:tasks.workunit.client.1.smithi150.stdout:8/233: chown d0/d2/d19/f20 56211814 1 2024-03-22T10:42:47.548 INFO:tasks.workunit.client.1.smithi150.stdout:8/234: write d0/d2/fa [195857,29393] 0 2024-03-22T10:42:47.548 INFO:tasks.workunit.client.1.smithi150.stdout:7/194: read d2/d5/d7/d9/f28 [17899,86402] 0 2024-03-22T10:42:47.557 INFO:tasks.workunit.client.0.smithi115.stdout:6/547: symlink d4/d13/d33/lad 0 2024-03-22T10:42:47.559 INFO:tasks.workunit.client.0.smithi115.stdout:5/592: write d2/df/d3b/f2a [2288637,53755] 0 2024-03-22T10:42:47.563 INFO:tasks.workunit.client.0.smithi115.stdout:5/593: readlink d2/df/l26 0 2024-03-22T10:42:47.564 INFO:tasks.workunit.client.0.smithi115.stdout:4/557: write d1/d16/d35/d40/d5c/f23 [623934,36245] 0 2024-03-22T10:42:47.566 INFO:tasks.workunit.client.0.smithi115.stdout:8/767: dwrite d6/dd/ff8 [0,4194304] 0 2024-03-22T10:42:47.569 INFO:tasks.workunit.client.1.smithi150.stdout:3/210: rmdir d5 39 2024-03-22T10:42:47.569 INFO:tasks.workunit.client.0.smithi115.stdout:5/594: mkdir d2/d15/dc8 0 2024-03-22T10:42:47.572 INFO:tasks.workunit.client.0.smithi115.stdout:4/558: unlink d1/d69/l4d 0 2024-03-22T10:42:47.572 INFO:tasks.workunit.client.1.smithi150.stdout:7/195: unlink d2/f1f 0 2024-03-22T10:42:47.577 INFO:tasks.workunit.client.1.smithi150.stdout:8/235: symlink d0/d2/d19/d29/l54 0 2024-03-22T10:42:47.582 INFO:tasks.workunit.client.1.smithi150.stdout:8/236: mknod d0/d2/d19/d29/d2e/c55 0 2024-03-22T10:42:47.583 INFO:tasks.workunit.client.0.smithi115.stdout:5/595: creat d2/d15/d1b/fc9 x:0 0 0 2024-03-22T10:42:47.584 INFO:tasks.workunit.client.0.smithi115.stdout:5/596: chown d2/df/d3b/c4e 888 1 2024-03-22T10:42:47.585 INFO:tasks.workunit.client.0.smithi115.stdout:7/621: dread d2/d7/d12/d10/d19/d42/f9c [0,4194304] 0 2024-03-22T10:42:47.585 INFO:tasks.workunit.client.0.smithi115.stdout:8/768: write d6/dd/d24/d3a/fcd [2576240,93448] 0 2024-03-22T10:42:47.587 INFO:tasks.workunit.client.0.smithi115.stdout:7/622: dread - d2/d7/d12/fb8 zero size 2024-03-22T10:42:47.588 INFO:tasks.workunit.client.1.smithi150.stdout:8/237: write d0/d2/d19/d29/d2f/df/f26 [1319027,34481] 0 2024-03-22T10:42:47.591 INFO:tasks.workunit.client.0.smithi115.stdout:8/769: symlink d6/dd/d24/d4a/d7b/d8f/dd6/ddb/lf9 0 2024-03-22T10:42:47.593 INFO:tasks.workunit.client.1.smithi150.stdout:7/196: dwrite d2/d5/d7/de/f33 [0,4194304] 0 2024-03-22T10:42:47.594 INFO:tasks.workunit.client.0.smithi115.stdout:7/623: symlink d2/d7/d12/d10/d23/d3d/dd2/da3/ld8 0 2024-03-22T10:42:47.594 INFO:tasks.workunit.client.1.smithi150.stdout:1/199: truncate d1/da/f16 2855590 0 2024-03-22T10:42:47.599 INFO:tasks.workunit.client.1.smithi150.stdout:7/197: write d2/d5/d7/d9/f18 [1526926,105693] 0 2024-03-22T10:42:47.599 INFO:tasks.workunit.client.0.smithi115.stdout:7/624: creat d2/d3a/d6e/d78/dbf/dc9/fd9 x:0 0 0 2024-03-22T10:42:47.601 INFO:tasks.workunit.client.1.smithi150.stdout:7/198: chown d2/d5/d7/d8/d46 204 1 2024-03-22T10:42:47.602 INFO:tasks.workunit.client.1.smithi150.stdout:9/223: dwrite d1/d8/de/f17 [0,4194304] 0 2024-03-22T10:42:47.602 INFO:tasks.workunit.client.0.smithi115.stdout:7/625: chown d2/d7/d12/d10/d23/d3d/dd2/fcc 0 1 2024-03-22T10:42:47.603 INFO:tasks.workunit.client.1.smithi150.stdout:7/199: chown d2/d5/l34 1325073488 1 2024-03-22T10:42:47.605 INFO:tasks.workunit.client.0.smithi115.stdout:5/597: dread d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:47.606 INFO:tasks.workunit.client.0.smithi115.stdout:7/626: dread - d2/d7/d12/d10/d23/f97 zero size 2024-03-22T10:42:47.609 INFO:tasks.workunit.client.0.smithi115.stdout:5/598: write d2/df/d3b/f7b [1045374,33955] 0 2024-03-22T10:42:47.613 INFO:tasks.workunit.client.1.smithi150.stdout:1/200: creat d1/da/d15/d18/f38 x:0 0 0 2024-03-22T10:42:47.614 INFO:tasks.workunit.client.1.smithi150.stdout:9/224: write d1/d8/de/df/d24/f4a [249727,69697] 0 2024-03-22T10:42:47.616 INFO:tasks.workunit.client.0.smithi115.stdout:5/599: write d2/d15/d1b/d7c/d38/f3d [5215456,9860] 0 2024-03-22T10:42:47.616 INFO:tasks.workunit.client.0.smithi115.stdout:4/559: dread d1/d30/d4c/d86/d9f/d4f/f7c [0,4194304] 0 2024-03-22T10:42:47.617 INFO:tasks.workunit.client.0.smithi115.stdout:7/627: rename d2/d7/d12/d10/d19/d42/f9c to d2/d7/d12/d10/d23/d3d/dd2/da3/fda 0 2024-03-22T10:42:47.620 INFO:tasks.workunit.client.0.smithi115.stdout:5/600: write d2/df/d13/f4c [1337225,115861] 0 2024-03-22T10:42:47.623 INFO:tasks.workunit.client.0.smithi115.stdout:4/560: symlink d1/d30/d4c/d86/d9f/d73/lc2 0 2024-03-22T10:42:47.624 INFO:tasks.workunit.client.0.smithi115.stdout:7/628: creat d2/d7/d12/d10/d19/dbd/fdb x:0 0 0 2024-03-22T10:42:47.625 INFO:tasks.workunit.client.1.smithi150.stdout:8/238: dwrite d0/d2/d19/d29/d2f/df/f51 [0,4194304] 0 2024-03-22T10:42:47.625 INFO:tasks.workunit.client.1.smithi150.stdout:9/225: write d1/d8/de/df/d20/f50 [715096,31276] 0 2024-03-22T10:42:47.626 INFO:tasks.workunit.client.0.smithi115.stdout:5/601: creat d2/d15/d1b/d7c/d38/d43/fca x:0 0 0 2024-03-22T10:42:47.628 INFO:tasks.workunit.client.0.smithi115.stdout:7/629: creat d2/d7/d12/fdc x:0 0 0 2024-03-22T10:42:47.629 INFO:tasks.workunit.client.0.smithi115.stdout:4/561: fsync d1/d16/d35/d40/d5c/f6f 0 2024-03-22T10:42:47.630 INFO:tasks.workunit.client.1.smithi150.stdout:0/218: dwrite d0/d1/d7/d42/d10/d1e/d37/f3d [0,4194304] 0 2024-03-22T10:42:47.633 INFO:tasks.workunit.client.0.smithi115.stdout:7/630: write d2/d7/d12/d10/d23/d3d/dd2/fcc [388231,95206] 0 2024-03-22T10:42:47.634 INFO:tasks.workunit.client.1.smithi150.stdout:9/226: creat d1/d8/de/df/f5b x:0 0 0 2024-03-22T10:42:47.634 INFO:tasks.workunit.client.1.smithi150.stdout:7/200: dread d2/d5/d7/de/f33 [0,4194304] 0 2024-03-22T10:42:47.634 INFO:tasks.workunit.client.1.smithi150.stdout:3/211: link d5/f10 d5/da/f40 0 2024-03-22T10:42:47.638 INFO:tasks.workunit.client.1.smithi150.stdout:0/219: fdatasync d0/d1/d7/d11/d26/f2e 0 2024-03-22T10:42:47.638 INFO:tasks.workunit.client.0.smithi115.stdout:4/562: dread d1/d16/d35/d40/d54/f67 [0,4194304] 0 2024-03-22T10:42:47.639 INFO:tasks.workunit.client.0.smithi115.stdout:8/770: dread d6/f23 [0,4194304] 0 2024-03-22T10:42:47.640 INFO:tasks.workunit.client.1.smithi150.stdout:3/212: creat d5/d22/d2f/f41 x:0 0 0 2024-03-22T10:42:47.641 INFO:tasks.workunit.client.0.smithi115.stdout:8/771: chown d6/dbc/fca 32 1 2024-03-22T10:42:47.642 INFO:tasks.workunit.client.1.smithi150.stdout:1/201: rmdir d1/da/d15/d18 39 2024-03-22T10:42:47.642 INFO:tasks.workunit.client.0.smithi115.stdout:4/563: dread - d1/d30/d4c/d86/d9f/d75/d99/da8/fa9 zero size 2024-03-22T10:42:47.644 INFO:tasks.workunit.client.0.smithi115.stdout:8/772: mknod d6/dd/d12/d31/cfa 0 2024-03-22T10:42:47.645 INFO:tasks.workunit.client.0.smithi115.stdout:4/564: symlink d1/d30/d78/lc3 0 2024-03-22T10:42:47.645 INFO:tasks.workunit.client.1.smithi150.stdout:8/239: dread d0/f1 [4194304,4194304] 0 2024-03-22T10:42:47.648 INFO:tasks.workunit.client.0.smithi115.stdout:4/565: write d1/d69/f62 [885132,33903] 0 2024-03-22T10:42:47.649 INFO:tasks.workunit.client.0.smithi115.stdout:8/773: symlink d6/dd/d24/d4a/d7b/lfb 0 2024-03-22T10:42:47.651 INFO:tasks.workunit.client.0.smithi115.stdout:8/774: read - d6/d49/d68/f9c zero size 2024-03-22T10:42:47.651 INFO:tasks.workunit.client.0.smithi115.stdout:7/631: dwrite d2/d7/d12/fa7 [0,4194304] 0 2024-03-22T10:42:47.652 INFO:tasks.workunit.client.1.smithi150.stdout:7/201: rename d2/d5/d7/d8/l43 to d2/d5/d7/l48 0 2024-03-22T10:42:47.653 INFO:tasks.workunit.client.0.smithi115.stdout:4/566: write d1/d16/d68/f83 [288106,41433] 0 2024-03-22T10:42:47.656 INFO:tasks.workunit.client.1.smithi150.stdout:7/202: read d2/d5/d7/d9/f28 [1765112,67711] 0 2024-03-22T10:42:47.658 INFO:tasks.workunit.client.1.smithi150.stdout:8/240: creat d0/d3c/f56 x:0 0 0 2024-03-22T10:42:47.659 INFO:tasks.workunit.client.0.smithi115.stdout:8/775: mkdir d6/dd/d24/d4a/dfc 0 2024-03-22T10:42:47.660 INFO:tasks.workunit.client.0.smithi115.stdout:8/776: dread - d6/dd/f70 zero size 2024-03-22T10:42:47.661 INFO:tasks.workunit.client.1.smithi150.stdout:0/220: getdents d0/d1/d7/d42/d10/d1e 0 2024-03-22T10:42:47.662 INFO:tasks.workunit.client.0.smithi115.stdout:9/606: sync 2024-03-22T10:42:47.662 INFO:tasks.workunit.client.0.smithi115.stdout:1/676: sync 2024-03-22T10:42:47.662 INFO:tasks.workunit.client.0.smithi115.stdout:2/551: sync 2024-03-22T10:42:47.662 INFO:tasks.workunit.client.0.smithi115.stdout:4/567: symlink d1/d30/d4c/d86/d9f/d32/lc4 0 2024-03-22T10:42:47.664 INFO:tasks.workunit.client.1.smithi150.stdout:9/227: truncate d1/d8/de/df/d20/d32/f2a 2585326 0 2024-03-22T10:42:47.664 INFO:tasks.workunit.client.0.smithi115.stdout:9/607: stat d6/d8/d3c/d59/d75/lcd 0 2024-03-22T10:42:47.665 INFO:tasks.workunit.client.0.smithi115.stdout:4/568: chown d1/d16/d35/d40 3 1 2024-03-22T10:42:47.665 INFO:tasks.workunit.client.0.smithi115.stdout:2/552: readlink da/d25/d3d/l36 0 2024-03-22T10:42:47.666 INFO:tasks.workunit.client.1.smithi150.stdout:7/203: mkdir d2/d5/d7/d8/d46/d49 0 2024-03-22T10:42:47.667 INFO:tasks.workunit.client.1.smithi150.stdout:0/221: rmdir d0/d12 39 2024-03-22T10:42:47.668 INFO:tasks.workunit.client.1.smithi150.stdout:3/213: dwrite d5/d22/f2b [0,4194304] 0 2024-03-22T10:42:47.669 INFO:tasks.workunit.client.0.smithi115.stdout:9/608: chown d6/d7/ff 417524229 1 2024-03-22T10:42:47.670 INFO:tasks.workunit.client.0.smithi115.stdout:8/777: getdents d6/dd/d12/de3 0 2024-03-22T10:42:47.671 INFO:tasks.workunit.client.1.smithi150.stdout:9/228: creat d1/d8/de/df/d36/f5c x:0 0 0 2024-03-22T10:42:47.672 INFO:tasks.workunit.client.0.smithi115.stdout:1/677: truncate d0/d8/d4d/d56/f57 4992246 0 2024-03-22T10:42:47.673 INFO:tasks.workunit.client.0.smithi115.stdout:7/632: dwrite d2/d7/d12/d10/d19/d42/d76/f83 [0,4194304] 0 2024-03-22T10:42:47.676 INFO:tasks.workunit.client.1.smithi150.stdout:3/214: creat d5/d22/d2f/d38/f42 x:0 0 0 2024-03-22T10:42:47.676 INFO:tasks.workunit.client.0.smithi115.stdout:8/778: mkdir d6/dd/d12/dc6/dfd 0 2024-03-22T10:42:47.679 INFO:tasks.workunit.client.1.smithi150.stdout:9/229: creat d1/d8/de/df/d20/f5d x:0 0 0 2024-03-22T10:42:47.681 INFO:tasks.workunit.client.0.smithi115.stdout:7/633: chown d2/d7/d12/d10/d19/l73 39478 1 2024-03-22T10:42:47.683 INFO:tasks.workunit.client.0.smithi115.stdout:5/602: truncate d2/d15/d1b/d7c/d38/d43/d49/fa9 1574144 0 2024-03-22T10:42:47.683 INFO:tasks.workunit.client.0.smithi115.stdout:1/678: read d0/d12/d17/f3e [2006284,119845] 0 2024-03-22T10:42:47.684 INFO:tasks.workunit.client.0.smithi115.stdout:8/779: symlink d6/dd/d24/d3a/lfe 0 2024-03-22T10:42:47.686 INFO:tasks.workunit.client.1.smithi150.stdout:3/215: symlink d5/d22/l43 0 2024-03-22T10:42:47.687 INFO:tasks.workunit.client.1.smithi150.stdout:0/222: write d0/d1/d7/f28 [330969,108371] 0 2024-03-22T10:42:47.687 INFO:tasks.workunit.client.0.smithi115.stdout:4/569: write d1/d16/d35/d40/d5c/f6f [1844137,79578] 0 2024-03-22T10:42:47.695 INFO:tasks.workunit.client.1.smithi150.stdout:8/241: dwrite d0/d2/d19/d29/d2f/f3b [0,4194304] 0 2024-03-22T10:42:47.696 INFO:tasks.workunit.client.0.smithi115.stdout:3/583: sync 2024-03-22T10:42:47.696 INFO:tasks.workunit.client.0.smithi115.stdout:0/561: sync 2024-03-22T10:42:47.698 INFO:tasks.workunit.client.0.smithi115.stdout:5/603: symlink d2/d15/d1b/d7c/d38/d56/lcb 0 2024-03-22T10:42:47.699 INFO:tasks.workunit.client.1.smithi150.stdout:7/204: dwrite d2/d5/d7/d8/f42 [0,4194304] 0 2024-03-22T10:42:47.700 INFO:tasks.workunit.client.0.smithi115.stdout:7/634: truncate d2/d7/d12/d10/d23/d3d/dd2/da3/fab 577432 0 2024-03-22T10:42:47.700 INFO:tasks.workunit.client.0.smithi115.stdout:4/570: creat d1/d30/d4c/d86/d9f/d75/d99/d98/fc5 x:0 0 0 2024-03-22T10:42:47.703 INFO:tasks.workunit.client.0.smithi115.stdout:8/780: rename d6/dd/d24/d4a/f81 to d6/dd/d24/d4a/d7b/fff 0 2024-03-22T10:42:47.705 INFO:tasks.workunit.client.0.smithi115.stdout:9/609: dwrite d6/d7/d30/d37/dbe/fc1 [0,4194304] 0 2024-03-22T10:42:47.705 INFO:tasks.workunit.client.1.smithi150.stdout:3/216: write d5/d22/d2f/d38/f39 [244463,113060] 0 2024-03-22T10:42:47.707 INFO:tasks.workunit.client.1.smithi150.stdout:8/242: read d0/d2/d19/d29/d2f/df/f51 [3900509,101804] 0 2024-03-22T10:42:47.710 INFO:tasks.workunit.client.0.smithi115.stdout:4/571: unlink d1/d16/d35/l71 0 2024-03-22T10:42:47.710 INFO:tasks.workunit.client.0.smithi115.stdout:2/553: dread da/d1c/f48 [0,4194304] 0 2024-03-22T10:42:47.712 INFO:tasks.workunit.client.1.smithi150.stdout:8/243: write d0/f21 [4694126,69756] 0 2024-03-22T10:42:47.715 INFO:tasks.workunit.client.1.smithi150.stdout:8/244: write d0/d2/d19/f47 [482149,118371] 0 2024-03-22T10:42:47.715 INFO:tasks.workunit.client.0.smithi115.stdout:4/572: chown d1/d30/d4c/d86/d9f/d75/d99/d98/lb5 101459180 1 2024-03-22T10:42:47.717 INFO:tasks.workunit.client.0.smithi115.stdout:7/635: creat d2/d3a/d6e/d78/dbf/dae/dbb/fdd x:0 0 0 2024-03-22T10:42:47.717 INFO:tasks.workunit.client.0.smithi115.stdout:1/679: rename d0/d8/db/d25/d7e to d0/d12/d17/d37/d9e/de9 0 2024-03-22T10:42:47.717 INFO:tasks.workunit.client.0.smithi115.stdout:4/573: readlink d1/d16/d68/la3 0 2024-03-22T10:42:47.718 INFO:tasks.workunit.client.1.smithi150.stdout:3/217: unlink d5/f24 0 2024-03-22T10:42:47.724 INFO:tasks.workunit.client.0.smithi115.stdout:9/610: creat d6/d7/d6a/fd5 x:0 0 0 2024-03-22T10:42:47.727 INFO:tasks.workunit.client.0.smithi115.stdout:2/554: mkdir da/d7b/da4 0 2024-03-22T10:42:47.728 INFO:tasks.workunit.client.1.smithi150.stdout:8/245: unlink d0/f16 0 2024-03-22T10:42:47.728 INFO:tasks.workunit.client.1.smithi150.stdout:3/218: fsync d5/f25 0 2024-03-22T10:42:47.728 INFO:tasks.workunit.client.1.smithi150.stdout:7/205: unlink d2/d5/d7/d9/l23 0 2024-03-22T10:42:47.728 INFO:tasks.workunit.client.0.smithi115.stdout:7/636: fsync d2/d7/d12/d10/d19/f1a 0 2024-03-22T10:42:47.731 INFO:tasks.workunit.client.1.smithi150.stdout:9/230: dwrite d1/d8/de/df/f1f [0,4194304] 0 2024-03-22T10:42:47.731 INFO:tasks.workunit.client.1.smithi150.stdout:8/246: rename d0/d2/d19 to d0/d2/d19/d29/d2f/d57 22 2024-03-22T10:42:47.731 INFO:tasks.workunit.client.0.smithi115.stdout:7/637: stat d2/d3a/d6e/f95 0 2024-03-22T10:42:47.733 INFO:tasks.workunit.client.0.smithi115.stdout:0/562: dread d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:47.733 INFO:tasks.workunit.client.0.smithi115.stdout:5/604: dread d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:47.735 INFO:tasks.workunit.client.0.smithi115.stdout:2/555: mkdir da/da5 0 2024-03-22T10:42:47.737 INFO:tasks.workunit.client.0.smithi115.stdout:7/638: rmdir d2/d7/d12/d10/d23/d3d/dd2 39 2024-03-22T10:42:47.738 INFO:tasks.workunit.client.1.smithi150.stdout:7/206: rename d2/d5/d7/de to d2/d5/d7/d2e/d4a 0 2024-03-22T10:42:47.739 INFO:tasks.workunit.client.0.smithi115.stdout:2/556: rmdir da/d1c 39 2024-03-22T10:42:47.739 INFO:tasks.workunit.client.1.smithi150.stdout:0/223: write d0/d1/d7/d42/d10/d38/f46 [21382,8870] 0 2024-03-22T10:42:47.740 INFO:tasks.workunit.client.0.smithi115.stdout:4/574: dwrite d1/d16/d35/d40/d5c/d14/f96 [0,4194304] 0 2024-03-22T10:42:47.742 INFO:tasks.workunit.client.1.smithi150.stdout:9/231: creat d1/d8/de/df/d20/d32/f5e x:0 0 0 2024-03-22T10:42:47.743 INFO:tasks.workunit.client.0.smithi115.stdout:0/563: mkdir d3/d7/db7 0 2024-03-22T10:42:47.744 INFO:tasks.workunit.client.1.smithi150.stdout:7/207: stat d2/d5/d7/d2e/d4a/l15 0 2024-03-22T10:42:47.744 INFO:tasks.workunit.client.0.smithi115.stdout:4/575: mkdir d1/dbb/dc6 0 2024-03-22T10:42:47.749 INFO:tasks.workunit.client.0.smithi115.stdout:2/557: symlink da/d25/d30/d4f/d54/la6 0 2024-03-22T10:42:47.751 INFO:tasks.workunit.client.0.smithi115.stdout:2/558: chown da/d25/d30 26492 1 2024-03-22T10:42:47.752 INFO:tasks.workunit.client.1.smithi150.stdout:0/224: creat d0/d1/d7/d42/d10/d38/f4c x:0 0 0 2024-03-22T10:42:47.753 INFO:tasks.workunit.client.1.smithi150.stdout:7/208: read - d2/d5/d7/d8/d2c/f47 zero size 2024-03-22T10:42:47.753 INFO:tasks.workunit.client.0.smithi115.stdout:9/611: write d6/d7/d30/d37/dbe/fc1 [4672318,106496] 0 2024-03-22T10:42:47.753 INFO:tasks.workunit.client.0.smithi115.stdout:2/559: chown da/d25/d51/f63 3 1 2024-03-22T10:42:47.754 INFO:tasks.workunit.client.1.smithi150.stdout:9/232: creat d1/d8/de/df/d24/f5f x:0 0 0 2024-03-22T10:42:47.757 INFO:tasks.workunit.client.1.smithi150.stdout:7/209: dread - d2/d5/d7/d9/db/f1a zero size 2024-03-22T10:42:47.758 INFO:tasks.workunit.client.0.smithi115.stdout:7/639: link d2/d3a/d6e/d78/dbf/dae/dbb/fdd d2/d7/d12/d10/d19/d42/d46/d94/fde 0 2024-03-22T10:42:47.759 INFO:tasks.workunit.client.1.smithi150.stdout:0/225: read d0/d1/f48 [2338500,87350] 0 2024-03-22T10:42:47.760 INFO:tasks.workunit.client.1.smithi150.stdout:9/233: symlink d1/d8/de/df/d36/l60 0 2024-03-22T10:42:47.760 INFO:tasks.workunit.client.1.smithi150.stdout:3/219: dwrite d5/da/f33 [0,4194304] 0 2024-03-22T10:42:47.764 INFO:tasks.workunit.client.0.smithi115.stdout:4/576: mkdir d1/d30/d4c/d86/d9f/d4f/d93/dc7 0 2024-03-22T10:42:47.768 INFO:tasks.workunit.client.0.smithi115.stdout:4/577: dread - d1/d66/fc1 zero size 2024-03-22T10:42:47.768 INFO:tasks.workunit.client.1.smithi150.stdout:7/210: creat d2/d5/d7/d38/f4b x:0 0 0 2024-03-22T10:42:47.768 INFO:tasks.workunit.client.1.smithi150.stdout:0/226: mknod d0/c4d 0 2024-03-22T10:42:47.769 INFO:tasks.workunit.client.0.smithi115.stdout:9/612: creat d6/d8/d3c/fd6 x:0 0 0 2024-03-22T10:42:47.770 INFO:tasks.workunit.client.1.smithi150.stdout:4/214: sync 2024-03-22T10:42:47.771 INFO:tasks.workunit.client.1.smithi150.stdout:9/234: creat d1/d8/de/df/d36/d40/d41/f61 x:0 0 0 2024-03-22T10:42:47.772 INFO:tasks.workunit.client.0.smithi115.stdout:2/560: mkdir da/d25/d3d/d57/da7 0 2024-03-22T10:42:47.774 INFO:tasks.workunit.client.1.smithi150.stdout:9/235: rmdir d1/d8 39 2024-03-22T10:42:47.774 INFO:tasks.workunit.client.1.smithi150.stdout:4/215: write d0/f11 [327243,64991] 0 2024-03-22T10:42:47.776 INFO:tasks.workunit.client.0.smithi115.stdout:3/584: dwrite d3/f30 [0,4194304] 0 2024-03-22T10:42:47.782 INFO:tasks.workunit.client.0.smithi115.stdout:8/781: dwrite d6/fc [0,4194304] 0 2024-03-22T10:42:47.782 INFO:tasks.workunit.client.1.smithi150.stdout:9/236: mknod d1/d8/de/df/d24/c62 0 2024-03-22T10:42:47.784 INFO:tasks.workunit.client.1.smithi150.stdout:4/216: symlink d0/l49 0 2024-03-22T10:42:47.785 INFO:tasks.workunit.client.1.smithi150.stdout:0/227: dread d0/d1/d7/fe [0,4194304] 0 2024-03-22T10:42:47.787 INFO:tasks.workunit.client.0.smithi115.stdout:7/640: truncate d2/d7/d12/d10/d23/d33/f3b 1178526 0 2024-03-22T10:42:47.788 INFO:tasks.workunit.client.1.smithi150.stdout:0/228: fsync d0/d1/d7/d42/f44 0 2024-03-22T10:42:47.791 INFO:tasks.workunit.client.0.smithi115.stdout:8/782: mkdir d6/dbc/d100 0 2024-03-22T10:42:47.794 INFO:tasks.workunit.client.0.smithi115.stdout:2/561: symlink da/d1c/d67/d78/d86/la8 0 2024-03-22T10:42:47.795 INFO:tasks.workunit.client.0.smithi115.stdout:9/613: getdents d6/d7/d48 0 2024-03-22T10:42:47.796 INFO:tasks.workunit.client.1.smithi150.stdout:0/229: symlink d0/d1/d7/d42/l4e 0 2024-03-22T10:42:47.796 INFO:tasks.workunit.client.1.smithi150.stdout:9/237: getdents d1/d19 0 2024-03-22T10:42:47.796 INFO:tasks.workunit.client.0.smithi115.stdout:0/564: dwrite d3/d7/d1c/f77 [0,4194304] 0 2024-03-22T10:42:47.796 INFO:tasks.workunit.client.0.smithi115.stdout:8/783: chown d6/dd/d24/d4a/f51 1997 1 2024-03-22T10:42:47.797 INFO:tasks.workunit.client.0.smithi115.stdout:5/605: dread d2/df/d3b/f7b [0,4194304] 0 2024-03-22T10:42:47.801 INFO:tasks.workunit.client.0.smithi115.stdout:5/606: chown d2/df/d13/d8b/d93/da6/dbc/d50/db1 106 1 2024-03-22T10:42:47.803 INFO:tasks.workunit.client.1.smithi150.stdout:9/238: chown d1/d8/de/df/d20/l5 112263 1 2024-03-22T10:42:47.803 INFO:tasks.workunit.client.1.smithi150.stdout:0/230: getdents d0/d1/d7/d11 0 2024-03-22T10:42:47.805 INFO:tasks.workunit.client.0.smithi115.stdout:2/562: dread da/d25/f37 [0,4194304] 0 2024-03-22T10:42:47.805 INFO:tasks.workunit.client.0.smithi115.stdout:7/641: symlink d2/d3a/d6e/ldf 0 2024-03-22T10:42:47.805 INFO:tasks.workunit.client.1.smithi150.stdout:0/231: chown d0/d1/d7/d42/d10 1440 1 2024-03-22T10:42:47.806 INFO:tasks.workunit.client.0.smithi115.stdout:7/642: chown d2/d7/d12/d10/d19/dbd 54121926 1 2024-03-22T10:42:47.807 INFO:tasks.workunit.client.1.smithi150.stdout:9/239: fsync d1/d8/de/df/d36/f39 0 2024-03-22T10:42:47.807 INFO:tasks.workunit.client.0.smithi115.stdout:9/614: rename d6/d7/d6a/d6f/f9b to d6/d7/d19/d4e/fd7 0 2024-03-22T10:42:47.808 INFO:tasks.workunit.client.1.smithi150.stdout:4/217: dwrite d0/d1b/f39 [0,4194304] 0 2024-03-22T10:42:47.808 INFO:tasks.workunit.client.0.smithi115.stdout:9/615: readlink d6/d7/d20/l35 0 2024-03-22T10:42:47.809 INFO:tasks.workunit.client.0.smithi115.stdout:7/643: write d2/d7/d12/d10/d19/dbd/fdb [487726,40343] 0 2024-03-22T10:42:47.812 INFO:tasks.workunit.client.0.smithi115.stdout:9/616: write d6/d7/d30/d37/dbe/fc6 [1140559,80941] 0 2024-03-22T10:42:47.813 INFO:tasks.workunit.client.1.smithi150.stdout:3/220: write d5/f10 [4294268,46900] 0 2024-03-22T10:42:47.814 INFO:tasks.workunit.client.0.smithi115.stdout:8/784: symlink d6/d49/d68/dcf/l101 0 2024-03-22T10:42:47.815 INFO:tasks.workunit.client.1.smithi150.stdout:7/211: truncate d2/d5/d7/d2e/d4a/f22 5119649 0 2024-03-22T10:42:47.816 INFO:tasks.workunit.client.0.smithi115.stdout:5/607: creat d2/df/d13/d8b/d93/da6/dbc/d50/d96/fcc x:0 0 0 2024-03-22T10:42:47.817 INFO:tasks.workunit.client.1.smithi150.stdout:9/240: mknod d1/d8/de/df/d20/c63 0 2024-03-22T10:42:47.817 INFO:tasks.workunit.client.1.smithi150.stdout:4/218: rmdir d0/d1b 39 2024-03-22T10:42:47.819 INFO:tasks.workunit.client.0.smithi115.stdout:0/565: dread d3/d7/d1b/d35/f80 [0,4194304] 0 2024-03-22T10:42:47.821 INFO:tasks.workunit.client.1.smithi150.stdout:3/221: creat d5/f44 x:0 0 0 2024-03-22T10:42:47.822 INFO:tasks.workunit.client.1.smithi150.stdout:7/212: symlink d2/d5/d7/d9/l4c 0 2024-03-22T10:42:47.822 INFO:tasks.workunit.client.0.smithi115.stdout:5/608: fdatasync d2/d15/d1b/d7c/d38/f3d 0 2024-03-22T10:42:47.823 INFO:tasks.workunit.client.1.smithi150.stdout:9/241: rmdir d1/d8 39 2024-03-22T10:42:47.823 INFO:tasks.workunit.client.0.smithi115.stdout:3/585: link d3/d19/d2b/d49/f60 d3/d19/d2b/d49/fbd 0 2024-03-22T10:42:47.824 INFO:tasks.workunit.client.1.smithi150.stdout:3/222: read - d5/d22/d2f/d38/f42 zero size 2024-03-22T10:42:47.826 INFO:tasks.workunit.client.1.smithi150.stdout:7/213: dread - d2/d5/d7/d8/d2c/f40 zero size 2024-03-22T10:42:47.827 INFO:tasks.workunit.client.1.smithi150.stdout:9/242: readlink d1/d8/de/df/d20/l5 0 2024-03-22T10:42:47.829 INFO:tasks.workunit.client.1.smithi150.stdout:3/223: rmdir d5/d22/d37 39 2024-03-22T10:42:47.831 INFO:tasks.workunit.client.0.smithi115.stdout:8/785: mkdir d6/dd/d24/d4a/dfc/d102 0 2024-03-22T10:42:47.832 INFO:tasks.workunit.client.1.smithi150.stdout:7/214: chown d2/d5/d7/d9/db/f1a 6366318 1 2024-03-22T10:42:47.832 INFO:tasks.workunit.client.1.smithi150.stdout:0/232: rename d0/d1/d7 to d0/d12/d4f 0 2024-03-22T10:42:47.832 INFO:tasks.workunit.client.1.smithi150.stdout:9/243: write d1/d8/de/df/d36/f5c [918565,104481] 0 2024-03-22T10:42:47.833 INFO:tasks.workunit.client.0.smithi115.stdout:3/586: mknod d3/d16/d29/cbe 0 2024-03-22T10:42:47.833 INFO:tasks.workunit.client.0.smithi115.stdout:9/617: creat d6/d8/d3c/d59/fd8 x:0 0 0 2024-03-22T10:42:47.833 INFO:tasks.workunit.client.0.smithi115.stdout:0/566: rename d3/da/ca1 to d3/d7/d1b/d9b/cb8 0 2024-03-22T10:42:47.835 INFO:tasks.workunit.client.1.smithi150.stdout:7/215: read - d2/d5/d7/d8/d2c/f40 zero size 2024-03-22T10:42:47.837 INFO:tasks.workunit.client.0.smithi115.stdout:4/578: dread d1/d66/f77 [0,4194304] 0 2024-03-22T10:42:47.838 INFO:tasks.workunit.client.1.smithi150.stdout:0/233: write d0/d12/d4f/d11/f2f [919057,66476] 0 2024-03-22T10:42:47.839 INFO:tasks.workunit.client.0.smithi115.stdout:5/609: rename d2/d15/d1b/d7c/d38/f77 to d2/d15/d1b/d7c/d38/d56/fcd 0 2024-03-22T10:42:47.840 INFO:tasks.workunit.client.0.smithi115.stdout:0/567: mkdir d3/d7/d1b/d1f/db9 0 2024-03-22T10:42:47.844 INFO:tasks.workunit.client.0.smithi115.stdout:3/587: unlink d3/d19/d2b/d49/d91/da5/l78 0 2024-03-22T10:42:47.846 INFO:tasks.workunit.client.0.smithi115.stdout:2/563: dread da/d25/d2a/f72 [0,4194304] 0 2024-03-22T10:42:47.847 INFO:tasks.workunit.client.0.smithi115.stdout:4/579: fsync d1/d30/d4c/d86/d9f/d32/f60 0 2024-03-22T10:42:47.848 INFO:tasks.workunit.client.0.smithi115.stdout:7/644: dwrite d2/d7/d12/d10/d23/d3d/dd2/da3/faf [0,4194304] 0 2024-03-22T10:42:47.849 INFO:tasks.workunit.client.0.smithi115.stdout:5/610: creat d2/d15/d1b/d7c/d38/d43/fce x:0 0 0 2024-03-22T10:42:47.849 INFO:tasks.workunit.client.0.smithi115.stdout:0/568: mkdir d3/d7/d1b/d1f/d5b/dba 0 2024-03-22T10:42:47.850 INFO:tasks.workunit.client.0.smithi115.stdout:3/588: mkdir d3/d36/d41/d33/dbf 0 2024-03-22T10:42:47.851 INFO:tasks.workunit.client.0.smithi115.stdout:5/611: chown d2/df 1 1 2024-03-22T10:42:47.852 INFO:tasks.workunit.client.0.smithi115.stdout:6/548: sync 2024-03-22T10:42:47.855 INFO:tasks.workunit.client.0.smithi115.stdout:7/645: chown d2/d7/d12/d10/d23/d3d/dd2/f88 931922756 1 2024-03-22T10:42:47.858 INFO:tasks.workunit.client.0.smithi115.stdout:0/569: dread - d3/d4/d14/d64/fab zero size 2024-03-22T10:42:47.859 INFO:tasks.workunit.client.0.smithi115.stdout:2/564: getdents da/d25/d3d/d7e/d90/d96 0 2024-03-22T10:42:47.861 INFO:tasks.workunit.client.0.smithi115.stdout:9/618: dwrite d6/d7/d20/d9c/fb0 [0,4194304] 0 2024-03-22T10:42:47.863 INFO:tasks.workunit.client.0.smithi115.stdout:9/619: stat d6/d8/d3c/d59/d75/lcd 0 2024-03-22T10:42:47.864 INFO:tasks.workunit.client.0.smithi115.stdout:6/549: symlink d4/d11/d12/d1c/d8a/lae 0 2024-03-22T10:42:47.865 INFO:tasks.workunit.client.0.smithi115.stdout:7/646: truncate d2/d7/d12/d10/d19/d42/d76/f98 5219078 0 2024-03-22T10:42:47.867 INFO:tasks.workunit.client.0.smithi115.stdout:2/565: readlink da/d25/d3d/d57/l91 0 2024-03-22T10:42:47.868 INFO:tasks.workunit.client.0.smithi115.stdout:6/550: mknod d4/d13/caf 0 2024-03-22T10:42:47.872 INFO:tasks.workunit.client.0.smithi115.stdout:9/620: creat d6/d7/d6a/fd9 x:0 0 0 2024-03-22T10:42:47.874 INFO:tasks.workunit.client.0.smithi115.stdout:6/551: creat d4/d11/d91/fb0 x:0 0 0 2024-03-22T10:42:47.876 INFO:tasks.workunit.client.0.smithi115.stdout:7/647: link d2/d7/d12/d10/d23/d3d/dd2/f92 d2/d3a/d6e/dc7/fe0 0 2024-03-22T10:42:47.877 INFO:tasks.workunit.client.0.smithi115.stdout:9/621: mknod d6/d8/d3c/d59/d68/da8/cda 0 2024-03-22T10:42:47.880 INFO:tasks.workunit.client.0.smithi115.stdout:2/566: getdents da/d25/d3d 0 2024-03-22T10:42:47.882 INFO:tasks.workunit.client.0.smithi115.stdout:7/648: mknod d2/d3a/d6e/dc7/ce1 0 2024-03-22T10:42:47.884 INFO:tasks.workunit.client.0.smithi115.stdout:2/567: read da/d25/d51/f56 [293183,80065] 0 2024-03-22T10:42:47.884 INFO:tasks.workunit.client.0.smithi115.stdout:7/649: dread - d2/d3a/d6e/d78/dbf/dc9/fd9 zero size 2024-03-22T10:42:47.887 INFO:tasks.workunit.client.0.smithi115.stdout:2/568: fdatasync da/d1c/f88 0 2024-03-22T10:42:47.889 INFO:tasks.workunit.client.0.smithi115.stdout:2/569: write da/d25/d83/fa3 [434440,56640] 0 2024-03-22T10:42:47.891 INFO:tasks.workunit.client.0.smithi115.stdout:7/650: write d2/d7/d12/d10/d23/f64 [873093,121857] 0 2024-03-22T10:42:47.894 INFO:tasks.workunit.client.0.smithi115.stdout:2/570: mknod da/ca9 0 2024-03-22T10:42:47.897 INFO:tasks.workunit.client.0.smithi115.stdout:4/580: read d1/d30/d4c/d86/fac [268762,75025] 0 2024-03-22T10:42:47.898 INFO:tasks.workunit.client.0.smithi115.stdout:0/570: dread d3/d7/f3b [0,4194304] 0 2024-03-22T10:42:47.899 INFO:tasks.workunit.client.0.smithi115.stdout:7/651: rename d2/d7/d12/d10/d19/d42/d46/f7f to d2/d3a/d6e/d78/dbf/dae/fe2 0 2024-03-22T10:42:47.900 INFO:tasks.workunit.client.0.smithi115.stdout:4/581: write d1/d16/d35/d40/f48 [4397863,106905] 0 2024-03-22T10:42:47.905 INFO:tasks.workunit.client.0.smithi115.stdout:4/582: unlink d1/d16/d68/la3 0 2024-03-22T10:42:47.906 INFO:tasks.workunit.client.0.smithi115.stdout:0/571: mknod d3/d7/db7/cbb 0 2024-03-22T10:42:47.910 INFO:tasks.workunit.client.0.smithi115.stdout:0/572: fdatasync d3/da/d4c/da3/f5e 0 2024-03-22T10:42:47.912 INFO:tasks.workunit.client.0.smithi115.stdout:0/573: write d3/da/f90 [666582,55257] 0 2024-03-22T10:42:47.912 INFO:tasks.workunit.client.0.smithi115.stdout:4/583: rename d1/d16/d35/d40/d5c/c5d to d1/d16/d35/d40/d5c/d14/cc8 0 2024-03-22T10:42:47.914 INFO:tasks.workunit.client.0.smithi115.stdout:0/574: fsync d3/d4/d6d/fb0 0 2024-03-22T10:42:47.916 INFO:tasks.workunit.client.0.smithi115.stdout:0/575: chown d3/d7/d1b/f92 606949 1 2024-03-22T10:42:47.921 INFO:tasks.workunit.client.0.smithi115.stdout:0/576: symlink d3/d7/d1b/d1f/d78/lbc 0 2024-03-22T10:42:47.923 INFO:tasks.workunit.client.0.smithi115.stdout:5/612: rmdir d2/d15/d1b 39 2024-03-22T10:42:47.925 INFO:tasks.workunit.client.0.smithi115.stdout:4/584: dread d1/d16/d35/d40/d5c/f6f [0,4194304] 0 2024-03-22T10:42:47.928 INFO:tasks.workunit.client.0.smithi115.stdout:5/613: creat d2/df/d13/d8b/d93/da6/fcf x:0 0 0 2024-03-22T10:42:47.929 INFO:tasks.workunit.client.1.smithi150.stdout:9/244: write d1/d8/de/df/d36/d40/d41/f47 [1808437,75364] 0 2024-03-22T10:42:47.930 INFO:tasks.workunit.client.0.smithi115.stdout:4/585: creat d1/d66/d82/da1/fc9 x:0 0 0 2024-03-22T10:42:47.930 INFO:tasks.workunit.client.1.smithi150.stdout:4/219: dwrite d0/d2f/f30 [0,4194304] 0 2024-03-22T10:42:47.930 INFO:tasks.workunit.client.0.smithi115.stdout:5/614: write d2/df/f67 [4649292,115391] 0 2024-03-22T10:42:47.933 INFO:tasks.workunit.client.1.smithi150.stdout:4/220: chown d0/d2f/l34 531 1 2024-03-22T10:42:47.933 INFO:tasks.workunit.client.1.smithi150.stdout:0/234: write d0/d12/d4f/d42/f4a [10039,41449] 0 2024-03-22T10:42:47.934 INFO:tasks.workunit.client.1.smithi150.stdout:9/245: stat d1/d8/de/df/d20/c63 0 2024-03-22T10:42:47.934 INFO:tasks.workunit.client.0.smithi115.stdout:4/586: chown d1/d16/d35 27653375 1 2024-03-22T10:42:47.936 INFO:tasks.workunit.client.0.smithi115.stdout:3/589: dwrite d3/f23 [0,4194304] 0 2024-03-22T10:42:47.936 INFO:tasks.workunit.client.1.smithi150.stdout:9/246: stat d1/d8/de/df/d24 0 2024-03-22T10:42:47.937 INFO:tasks.workunit.client.1.smithi150.stdout:4/221: creat d0/d32/f4a x:0 0 0 2024-03-22T10:42:47.939 INFO:tasks.workunit.client.1.smithi150.stdout:9/247: write d1/d19/f26 [3873203,70716] 0 2024-03-22T10:42:47.939 INFO:tasks.workunit.client.1.smithi150.stdout:7/216: chown d2/d5/d7/d2e/d4a/f22 1 1 2024-03-22T10:42:47.941 INFO:tasks.workunit.client.1.smithi150.stdout:0/235: truncate d0/d12/d4f/d42/d10/d38/f46 275536 0 2024-03-22T10:42:47.943 INFO:tasks.workunit.client.1.smithi150.stdout:4/222: rename d0/f38 to d0/de/f4b 0 2024-03-22T10:42:47.944 INFO:tasks.workunit.client.0.smithi115.stdout:5/615: creat d2/df/d13/d8b/d93/da6/dbc/fd0 x:0 0 0 2024-03-22T10:42:47.944 INFO:tasks.workunit.client.0.smithi115.stdout:4/587: creat d1/d30/d4c/d86/d9f/d75/d99/da8/fca x:0 0 0 2024-03-22T10:42:47.946 INFO:tasks.workunit.client.0.smithi115.stdout:4/588: readlink d1/d66/d82/l88 0 2024-03-22T10:42:47.946 INFO:tasks.workunit.client.0.smithi115.stdout:3/590: creat d3/d82/d52/fc0 x:0 0 0 2024-03-22T10:42:47.949 INFO:tasks.workunit.client.0.smithi115.stdout:6/552: rename d4/d13/c3e to d4/d5e/cb1 0 2024-03-22T10:42:47.950 INFO:tasks.workunit.client.0.smithi115.stdout:2/571: write da/d25/d3d/d7e/fa2 [1609629,87261] 0 2024-03-22T10:42:47.951 INFO:tasks.workunit.client.1.smithi150.stdout:9/248: symlink d1/d8/de/df/d24/d4e/l64 0 2024-03-22T10:42:47.951 INFO:tasks.workunit.client.1.smithi150.stdout:9/249: creat d1/d8/f65 x:0 0 0 2024-03-22T10:42:47.952 INFO:tasks.workunit.client.1.smithi150.stdout:4/223: link d0/f1a d0/de/f4c 0 2024-03-22T10:42:47.952 INFO:tasks.workunit.client.1.smithi150.stdout:4/224: read d0/d2f/f30 [2106655,73501] 0 2024-03-22T10:42:47.952 INFO:tasks.workunit.client.1.smithi150.stdout:4/225: chown d0/d42 130485059 1 2024-03-22T10:42:47.953 INFO:tasks.workunit.client.0.smithi115.stdout:2/572: read da/f1d [2066865,5097] 0 2024-03-22T10:42:47.955 INFO:tasks.workunit.client.0.smithi115.stdout:9/622: dwrite d6/d7/d6a/f9e [0,4194304] 0 2024-03-22T10:42:47.958 INFO:tasks.workunit.client.1.smithi150.stdout:4/226: mkdir d0/d1b/d4d 0 2024-03-22T10:42:47.959 INFO:tasks.workunit.client.0.smithi115.stdout:3/591: mkdir d3/d16/d29/dc1 0 2024-03-22T10:42:47.959 INFO:tasks.workunit.client.0.smithi115.stdout:7/652: truncate d2/d7/d12/d10/d19/f9d 4196491 0 2024-03-22T10:42:47.962 INFO:tasks.workunit.client.0.smithi115.stdout:4/589: creat d1/d16/d35/d40/d5c/d55/fcb x:0 0 0 2024-03-22T10:42:47.968 INFO:tasks.workunit.client.1.smithi150.stdout:7/217: dwrite d2/d5/d7/d2e/d4a/f33 [4194304,4194304] 0 2024-03-22T10:42:47.969 INFO:tasks.workunit.client.0.smithi115.stdout:2/573: rename da/d25/d83/fa3 to da/d1c/d67/faa 0 2024-03-22T10:42:47.970 INFO:tasks.workunit.client.0.smithi115.stdout:5/616: creat d2/df/fd1 x:0 0 0 2024-03-22T10:42:47.971 INFO:tasks.workunit.client.0.smithi115.stdout:2/574: chown da/d1c/c46 51 1 2024-03-22T10:42:47.972 INFO:tasks.workunit.client.0.smithi115.stdout:5/617: stat d2/df/d13/d8b/d93/da6 0 2024-03-22T10:42:47.972 INFO:tasks.workunit.client.0.smithi115.stdout:3/592: mkdir d3/d82/dc2 0 2024-03-22T10:42:47.973 INFO:tasks.workunit.client.0.smithi115.stdout:6/553: dread d4/d5e/f97 [0,4194304] 0 2024-03-22T10:42:47.974 INFO:tasks.workunit.client.1.smithi150.stdout:5/132: sync 2024-03-22T10:42:47.976 INFO:tasks.workunit.client.0.smithi115.stdout:7/653: creat d2/d7/d12/d10/d23/d3d/dd2/fe3 x:0 0 0 2024-03-22T10:42:47.977 INFO:tasks.workunit.client.1.smithi150.stdout:7/218: getdents d2/d5/d7/d8/d2c 0 2024-03-22T10:42:47.977 INFO:tasks.workunit.client.1.smithi150.stdout:5/133: write d0/d8/f23 [1010459,110788] 0 2024-03-22T10:42:47.978 INFO:tasks.workunit.client.0.smithi115.stdout:6/554: truncate d4/d11/d12/d1c/fa5 566985 0 2024-03-22T10:42:47.978 INFO:tasks.workunit.client.0.smithi115.stdout:4/590: fsync d1/d16/d35/d40/d54/d5e/f7e 0 2024-03-22T10:42:47.980 INFO:tasks.workunit.client.0.smithi115.stdout:4/591: readlink d1/d16/d35/d40/d54/l7f 0 2024-03-22T10:42:47.980 INFO:tasks.workunit.client.1.smithi150.stdout:9/250: dread d1/d8/de/df/d20/d32/f2a [0,4194304] 0 2024-03-22T10:42:47.982 INFO:tasks.workunit.client.0.smithi115.stdout:4/592: write d1/d16/d35/f9c [268242,94408] 0 2024-03-22T10:42:47.984 INFO:tasks.workunit.client.0.smithi115.stdout:4/593: stat d1/d16/d35/d40/d5c/d55/f5a 0 2024-03-22T10:42:47.985 INFO:tasks.workunit.client.0.smithi115.stdout:2/575: creat da/d25/d30/d4f/d54/fab x:0 0 0 2024-03-22T10:42:47.987 INFO:tasks.workunit.client.0.smithi115.stdout:5/618: read - d2/df/d13/f76 zero size 2024-03-22T10:42:47.989 INFO:tasks.workunit.client.0.smithi115.stdout:3/593: creat d3/d19/d2b/d49/d91/dad/fc3 x:0 0 0 2024-03-22T10:42:47.990 INFO:tasks.workunit.client.0.smithi115.stdout:5/619: write d2/df/f67 [1885072,76862] 0 2024-03-22T10:42:47.992 INFO:tasks.workunit.client.1.smithi150.stdout:7/219: creat d2/d5/d7/f4d x:0 0 0 2024-03-22T10:42:47.992 INFO:tasks.workunit.client.1.smithi150.stdout:0/236: dwrite d0/d12/d4f/f31 [0,4194304] 0 2024-03-22T10:42:47.994 INFO:tasks.workunit.client.0.smithi115.stdout:4/594: rmdir d1/d30/d4c/d86/d9f 39 2024-03-22T10:42:47.995 INFO:tasks.workunit.client.0.smithi115.stdout:9/623: write d6/d7/d20/d22/f95 [250881,19445] 0 2024-03-22T10:42:47.997 INFO:tasks.workunit.client.0.smithi115.stdout:7/654: symlink d2/d7/d12/d10/d19/le4 0 2024-03-22T10:42:47.997 INFO:tasks.workunit.client.0.smithi115.stdout:9/624: fdatasync d6/f62 0 2024-03-22T10:42:47.998 INFO:tasks.workunit.client.0.smithi115.stdout:2/576: creat da/d25/d3d/d57/d7d/fac x:0 0 0 2024-03-22T10:42:47.998 INFO:tasks.workunit.client.1.smithi150.stdout:5/134: getdents d0 0 2024-03-22T10:42:47.999 INFO:tasks.workunit.client.0.smithi115.stdout:7/655: stat d2/d7/d12/d10/d19/d42/f6d 0 2024-03-22T10:42:48.001 INFO:tasks.workunit.client.1.smithi150.stdout:0/237: rmdir d0/d12/d4f/d42/d10/d38 39 2024-03-22T10:42:48.001 INFO:tasks.workunit.client.1.smithi150.stdout:5/135: readlink d0/l9 0 2024-03-22T10:42:48.003 INFO:tasks.workunit.client.0.smithi115.stdout:6/555: creat d4/d36/d48/d72/daa/fb2 x:0 0 0 2024-03-22T10:42:48.006 INFO:tasks.workunit.client.1.smithi150.stdout:7/220: truncate d2/d5/d7/d9/db/f1a 290427 0 2024-03-22T10:42:48.006 INFO:tasks.workunit.client.0.smithi115.stdout:5/620: rmdir d2/df/d13/d8b/d93/da6 39 2024-03-22T10:42:48.008 INFO:tasks.workunit.client.0.smithi115.stdout:3/594: rename d3/d82/f8c to d3/d82/d52/fc4 0 2024-03-22T10:42:48.010 INFO:tasks.workunit.client.1.smithi150.stdout:5/136: write d0/d8/dc/de/d14/f25 [2229318,58081] 0 2024-03-22T10:42:48.010 INFO:tasks.workunit.client.1.smithi150.stdout:4/227: dwrite d0/d1b/f24 [0,4194304] 0 2024-03-22T10:42:48.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:47 smithi115 ceph-mon[29111]: pgmap v12: 65 pgs: 65 active+clean; 1.5 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 44 MiB/s rd, 218 MiB/s wr, 432 op/s 2024-03-22T10:42:48.011 INFO:tasks.workunit.client.0.smithi115.stdout:9/625: mknod d6/d8/d3c/d59/d68/d97/da4/cdb 0 2024-03-22T10:42:48.011 INFO:tasks.workunit.client.1.smithi150.stdout:9/251: dwrite d1/d8/de/df/d20/d32/f5e [0,4194304] 0 2024-03-22T10:42:48.014 INFO:tasks.workunit.client.1.smithi150.stdout:4/228: chown d0/d1b/f24 45256 1 2024-03-22T10:42:48.014 INFO:tasks.workunit.client.1.smithi150.stdout:7/221: creat d2/d5/d7/d8/d46/f4e x:0 0 0 2024-03-22T10:42:48.014 INFO:tasks.workunit.client.0.smithi115.stdout:2/577: write da/d25/d51/f55 [3155365,84691] 0 2024-03-22T10:42:48.015 INFO:tasks.workunit.client.1.smithi150.stdout:5/137: fdatasync d0/d8/dc/d1d/f1f 0 2024-03-22T10:42:48.015 INFO:tasks.workunit.client.1.smithi150.stdout:0/238: link d0/d1/c1d d0/d12/d4f/d11/d26/c50 0 2024-03-22T10:42:48.016 INFO:tasks.workunit.client.1.smithi150.stdout:9/252: symlink d1/d8/de/l66 0 2024-03-22T10:42:48.017 INFO:tasks.workunit.client.1.smithi150.stdout:6/161: sync 2024-03-22T10:42:48.017 INFO:tasks.workunit.client.1.smithi150.stdout:1/202: sync 2024-03-22T10:42:48.017 INFO:tasks.workunit.client.1.smithi150.stdout:5/138: write d0/d8/fd [3746691,54721] 0 2024-03-22T10:42:48.017 INFO:tasks.workunit.client.1.smithi150.stdout:2/245: sync 2024-03-22T10:42:48.018 INFO:tasks.workunit.client.0.smithi115.stdout:9/626: chown d6/d7/d30/d37 95342 1 2024-03-22T10:42:48.018 INFO:tasks.workunit.client.0.smithi115.stdout:7/656: write d2/d7/d12/d10/d23/f38 [332471,9762] 0 2024-03-22T10:42:48.019 INFO:tasks.workunit.client.0.smithi115.stdout:3/595: write d3/f30 [5219000,129166] 0 2024-03-22T10:42:48.021 INFO:tasks.workunit.client.0.smithi115.stdout:6/556: symlink d4/lb3 0 2024-03-22T10:42:48.022 INFO:tasks.workunit.client.0.smithi115.stdout:4/595: dwrite d1/d16/d35/fbd [0,4194304] 0 2024-03-22T10:42:48.027 INFO:tasks.workunit.client.1.smithi150.stdout:6/162: fdatasync db/dd/f2c 0 2024-03-22T10:42:48.029 INFO:tasks.workunit.client.0.smithi115.stdout:2/578: mkdir da/d25/d3d/dad 0 2024-03-22T10:42:48.030 INFO:tasks.workunit.client.0.smithi115.stdout:7/657: chown d2/d3a/d6e/d78/dbf/dae/dbb/c9f 3 1 2024-03-22T10:42:48.031 INFO:tasks.workunit.client.0.smithi115.stdout:5/621: dwrite d2/df/f55 [0,4194304] 0 2024-03-22T10:42:48.032 INFO:tasks.workunit.client.1.smithi150.stdout:2/246: rename f10 to d12/d48/f4c 0 2024-03-22T10:42:48.033 INFO:tasks.workunit.client.0.smithi115.stdout:3/596: mkdir d3/d82/d57/dc5 0 2024-03-22T10:42:48.040 INFO:tasks.workunit.client.1.smithi150.stdout:5/139: unlink d0/d8/dc/d1d/f1f 0 2024-03-22T10:42:48.043 INFO:tasks.workunit.client.1.smithi150.stdout:1/203: dwrite d1/f7 [0,4194304] 0 2024-03-22T10:42:48.049 INFO:tasks.workunit.client.0.smithi115.stdout:5/622: creat d2/d15/d1b/d7c/d38/fd2 x:0 0 0 2024-03-22T10:42:48.050 INFO:tasks.workunit.client.0.smithi115.stdout:3/597: write d3/fb2 [52192,112234] 0 2024-03-22T10:42:48.052 INFO:tasks.workunit.client.1.smithi150.stdout:2/247: creat d12/d48/f4d x:0 0 0 2024-03-22T10:42:48.052 INFO:tasks.workunit.client.0.smithi115.stdout:5/623: stat d2/fe 0 2024-03-22T10:42:48.053 INFO:tasks.workunit.client.1.smithi150.stdout:5/140: creat d0/d8/dc/de/d18/f27 x:0 0 0 2024-03-22T10:42:48.058 INFO:tasks.workunit.client.1.smithi150.stdout:1/204: mknod d1/d19/d1a/c39 0 2024-03-22T10:42:48.058 INFO:tasks.workunit.client.0.smithi115.stdout:6/557: dwrite d4/d11/f54 [4194304,4194304] 0 2024-03-22T10:42:48.060 INFO:tasks.workunit.client.1.smithi150.stdout:0/239: link d0/d12/d4f/d42/d10/d1e/d35/l3c d0/l51 0 2024-03-22T10:42:48.061 INFO:tasks.workunit.client.1.smithi150.stdout:2/248: creat d12/d19/d2d/d3b/f4e x:0 0 0 2024-03-22T10:42:48.062 INFO:tasks.workunit.client.1.smithi150.stdout:1/205: write d1/da/fd [7732350,83384] 0 2024-03-22T10:42:48.063 INFO:tasks.workunit.client.1.smithi150.stdout:7/222: rename d2/d5/d7/d9/db/c41 to d2/d5/d7/c4f 0 2024-03-22T10:42:48.063 INFO:tasks.workunit.client.0.smithi115.stdout:5/624: mkdir d2/df/d3b/d8e/dd3 0 2024-03-22T10:42:48.064 INFO:tasks.workunit.client.0.smithi115.stdout:9/627: dwrite d6/d7/f16 [0,4194304] 0 2024-03-22T10:42:48.065 INFO:tasks.workunit.client.1.smithi150.stdout:8/247: sync 2024-03-22T10:42:48.066 INFO:tasks.workunit.client.1.smithi150.stdout:3/224: sync 2024-03-22T10:42:48.067 INFO:tasks.workunit.client.1.smithi150.stdout:1/206: chown d1/d19/d1a/c22 26001304 1 2024-03-22T10:42:48.067 INFO:tasks.workunit.client.0.smithi115.stdout:3/598: creat d3/d19/d2b/d49/fc6 x:0 0 0 2024-03-22T10:42:48.071 INFO:tasks.workunit.client.0.smithi115.stdout:5/625: symlink d2/d15/d1b/d7c/daf/ld4 0 2024-03-22T10:42:48.071 INFO:tasks.workunit.client.0.smithi115.stdout:4/596: dwrite d1/d30/d4c/d86/d9f/f34 [0,4194304] 0 2024-03-22T10:42:48.075 INFO:tasks.workunit.client.0.smithi115.stdout:5/626: write d2/df/f10 [8807680,103913] 0 2024-03-22T10:42:48.077 INFO:tasks.workunit.client.0.smithi115.stdout:3/599: creat d3/d82/d57/fc7 x:0 0 0 2024-03-22T10:42:48.079 INFO:tasks.workunit.client.0.smithi115.stdout:4/597: creat d1/d30/d4c/d86/d9f/d73/fcc x:0 0 0 2024-03-22T10:42:48.079 INFO:tasks.workunit.client.0.smithi115.stdout:3/600: symlink d3/d16/d29/d35/lc8 0 2024-03-22T10:42:48.079 INFO:tasks.workunit.client.1.smithi150.stdout:6/163: rename db/dd/d21/c23 to db/dd/d25/c3c 0 2024-03-22T10:42:48.082 INFO:tasks.workunit.client.0.smithi115.stdout:9/628: creat d6/d8/d3c/fdc x:0 0 0 2024-03-22T10:42:48.082 INFO:tasks.workunit.client.0.smithi115.stdout:5/627: write d2/d15/d1b/d7c/d38/f61 [3264890,65812] 0 2024-03-22T10:42:48.083 INFO:tasks.workunit.client.0.smithi115.stdout:4/598: fdatasync d1/d30/d4c/d86/fac 0 2024-03-22T10:42:48.084 INFO:tasks.workunit.client.1.smithi150.stdout:8/248: creat d0/d2/d19/d29/d2e/f58 x:0 0 0 2024-03-22T10:42:48.086 INFO:tasks.workunit.client.1.smithi150.stdout:0/240: mkdir d0/d12/d4f/d42/d10/d38/d52 0 2024-03-22T10:42:48.089 INFO:tasks.workunit.client.1.smithi150.stdout:3/225: creat d5/da/f45 x:0 0 0 2024-03-22T10:42:48.091 INFO:tasks.workunit.client.0.smithi115.stdout:5/628: chown d2/d15/d1b/d7c/f8a 1011768 1 2024-03-22T10:42:48.095 INFO:tasks.workunit.client.0.smithi115.stdout:5/629: mknod d2/d15/d1b/d7c/d38/d4a/cd5 0 2024-03-22T10:42:48.097 INFO:tasks.workunit.client.1.smithi150.stdout:1/207: creat d1/f3a x:0 0 0 2024-03-22T10:42:48.098 INFO:tasks.workunit.client.1.smithi150.stdout:4/229: dread d0/d1b/f23 [0,4194304] 0 2024-03-22T10:42:48.099 INFO:tasks.workunit.client.1.smithi150.stdout:6/164: fsync db/dd/d25/f34 0 2024-03-22T10:42:48.099 INFO:tasks.workunit.client.1.smithi150.stdout:0/241: chown d0/d12/d4f/d42/d10/d1e/d35 63174 1 2024-03-22T10:42:48.101 INFO:tasks.workunit.client.0.smithi115.stdout:3/601: dwrite d3/d16/d29/f3c [0,4194304] 0 2024-03-22T10:42:48.102 INFO:tasks.workunit.client.1.smithi150.stdout:9/253: truncate d1/d8/de/df/f1f 1548661 0 2024-03-22T10:42:48.102 INFO:tasks.workunit.client.1.smithi150.stdout:5/141: dwrite d0/f22 [0,4194304] 0 2024-03-22T10:42:48.103 INFO:tasks.workunit.client.0.smithi115.stdout:5/630: creat d2/df/d3b/d8e/fd6 x:0 0 0 2024-03-22T10:42:48.104 INFO:tasks.workunit.client.1.smithi150.stdout:2/249: dwrite d12/d19/d2d/d3b/f3d [0,4194304] 0 2024-03-22T10:42:48.105 INFO:tasks.workunit.client.1.smithi150.stdout:9/254: chown d1/d8/de/df/l51 169552 1 2024-03-22T10:42:48.106 INFO:tasks.workunit.client.0.smithi115.stdout:4/599: dwrite d1/d16/d35/d40/d5c/d14/f3e [0,4194304] 0 2024-03-22T10:42:48.106 INFO:tasks.workunit.client.1.smithi150.stdout:0/242: write d0/d12/d4f/d11/d26/f2e [4470747,80448] 0 2024-03-22T10:42:48.111 INFO:tasks.workunit.client.0.smithi115.stdout:3/602: write d3/d16/f1c [9038530,113076] 0 2024-03-22T10:42:48.111 INFO:tasks.workunit.client.1.smithi150.stdout:2/250: chown d12/d48/c45 8464302 1 2024-03-22T10:42:48.111 INFO:tasks.workunit.client.1.smithi150.stdout:7/223: link d2/d5/d7/d9/f28 d2/d5/d7/f50 0 2024-03-22T10:42:48.112 INFO:tasks.workunit.client.0.smithi115.stdout:6/558: dread d4/d11/d12/d1c/f69 [0,4194304] 0 2024-03-22T10:42:48.113 INFO:tasks.workunit.client.1.smithi150.stdout:1/208: unlink d1/cf 0 2024-03-22T10:42:48.113 INFO:tasks.workunit.client.1.smithi150.stdout:8/249: fdatasync d0/d2/d19/d29/d2f/df/d1b/f24 0 2024-03-22T10:42:48.114 INFO:tasks.workunit.client.1.smithi150.stdout:6/165: creat db/dd/d25/f3d x:0 0 0 2024-03-22T10:42:48.115 INFO:tasks.workunit.client.1.smithi150.stdout:3/226: chown d5/d22/l2e 115867 1 2024-03-22T10:42:48.116 INFO:tasks.workunit.client.0.smithi115.stdout:4/600: symlink d1/d16/d28/lcd 0 2024-03-22T10:42:48.117 INFO:tasks.workunit.client.0.smithi115.stdout:9/629: dwrite d6/d7/d19/d4e/f6c [0,4194304] 0 2024-03-22T10:42:48.118 INFO:tasks.workunit.client.0.smithi115.stdout:5/631: readlink d2/df/d13/d8b/d93/da6/dbc/l42 0 2024-03-22T10:42:48.119 INFO:tasks.workunit.client.1.smithi150.stdout:4/230: write d0/de/f4b [711020,7853] 0 2024-03-22T10:42:48.120 INFO:tasks.workunit.client.1.smithi150.stdout:8/250: chown d0/d2/d19/d29/d2f/f3b 1778815 1 2024-03-22T10:42:48.121 INFO:tasks.workunit.client.1.smithi150.stdout:5/142: rename d0/d8/dc/de/d18/f27 to d0/f28 0 2024-03-22T10:42:48.122 INFO:tasks.workunit.client.1.smithi150.stdout:9/255: mknod d1/d8/de/df/d36/d40/d41/c67 0 2024-03-22T10:42:48.124 INFO:tasks.workunit.client.1.smithi150.stdout:5/143: fsync d0/d8/dc/f12 0 2024-03-22T10:42:48.126 INFO:tasks.workunit.client.1.smithi150.stdout:9/256: write d1/d8/de/df/d36/f39 [974555,85867] 0 2024-03-22T10:42:48.129 INFO:tasks.workunit.client.1.smithi150.stdout:5/144: truncate d0/d8/f23 2032585 0 2024-03-22T10:42:48.129 INFO:tasks.workunit.client.1.smithi150.stdout:6/166: symlink db/dd/d15/l3e 0 2024-03-22T10:42:48.129 INFO:tasks.workunit.client.0.smithi115.stdout:4/601: rename d1/d16/f81 to d1/d30/d4c/d86/d9f/d75/d99/d98/d9b/fce 0 2024-03-22T10:42:48.130 INFO:tasks.workunit.client.0.smithi115.stdout:5/632: symlink d2/d15/dc8/ld7 0 2024-03-22T10:42:48.132 INFO:tasks.workunit.client.0.smithi115.stdout:5/633: dread - d2/df/d13/fbb zero size 2024-03-22T10:42:48.135 INFO:tasks.workunit.client.0.smithi115.stdout:5/634: fsync d2/df/f10 0 2024-03-22T10:42:48.138 INFO:tasks.workunit.client.0.smithi115.stdout:3/603: dwrite d3/f9 [0,4194304] 0 2024-03-22T10:42:48.138 INFO:tasks.workunit.client.1.smithi150.stdout:7/224: dread d2/d5/d7/f2b [0,4194304] 0 2024-03-22T10:42:48.138 INFO:tasks.workunit.client.1.smithi150.stdout:0/243: mkdir d0/d12/d4f/d42/d10/d53 0 2024-03-22T10:42:48.142 INFO:tasks.workunit.client.0.smithi115.stdout:5/635: getdents d2/df/d3b 0 2024-03-22T10:42:48.144 INFO:tasks.workunit.client.1.smithi150.stdout:1/209: creat d1/da/d15/d18/f3b x:0 0 0 2024-03-22T10:42:48.146 INFO:tasks.workunit.client.0.smithi115.stdout:5/636: unlink d2/d15/d1b/l54 0 2024-03-22T10:42:48.148 INFO:tasks.workunit.client.1.smithi150.stdout:9/257: unlink d1/d8/de/f16 0 2024-03-22T10:42:48.149 INFO:tasks.workunit.client.0.smithi115.stdout:6/559: write d4/d13/f25 [666537,52207] 0 2024-03-22T10:42:48.149 INFO:tasks.workunit.client.0.smithi115.stdout:9/630: write d6/d7/d20/d9c/f9f [870566,78490] 0 2024-03-22T10:42:48.150 INFO:tasks.workunit.client.0.smithi115.stdout:1/680: sync 2024-03-22T10:42:48.150 INFO:tasks.workunit.client.0.smithi115.stdout:5/637: readlink d2/d15/d1b/d7c/daf/lbe 0 2024-03-22T10:42:48.153 INFO:tasks.workunit.client.0.smithi115.stdout:9/631: stat d6/d7/d20/d9c/fb0 0 2024-03-22T10:42:48.155 INFO:tasks.workunit.client.0.smithi115.stdout:5/638: unlink d2/d15/d1b/fc9 0 2024-03-22T10:42:48.155 INFO:tasks.workunit.client.1.smithi150.stdout:6/167: symlink db/l3f 0 2024-03-22T10:42:48.157 INFO:tasks.workunit.client.0.smithi115.stdout:5/639: write d2/f71 [1795283,6588] 0 2024-03-22T10:42:48.159 INFO:tasks.workunit.client.0.smithi115.stdout:9/632: creat d6/fdd x:0 0 0 2024-03-22T10:42:48.160 INFO:tasks.workunit.client.0.smithi115.stdout:1/681: rename d0/d68/c9d to d0/d8/db/dd7/cea 0 2024-03-22T10:42:48.161 INFO:tasks.workunit.client.0.smithi115.stdout:5/640: dread - d2/df/d13/d8b/d93/da6/fa8 zero size 2024-03-22T10:42:48.165 INFO:tasks.workunit.client.1.smithi150.stdout:3/227: link d5/d22/f2b d5/f46 0 2024-03-22T10:42:48.167 INFO:tasks.workunit.client.0.smithi115.stdout:9/633: creat d6/d7/d20/d9c/fde x:0 0 0 2024-03-22T10:42:48.167 INFO:tasks.workunit.client.1.smithi150.stdout:0/244: write d0/d12/d4f/f28 [3082980,4381] 0 2024-03-22T10:42:48.168 INFO:tasks.workunit.client.0.smithi115.stdout:4/602: truncate d1/d66/d82/f8e 4142513 0 2024-03-22T10:42:48.169 INFO:tasks.workunit.client.1.smithi150.stdout:3/228: chown d5/d22/d2f/f41 261211537 1 2024-03-22T10:42:48.169 INFO:tasks.workunit.client.0.smithi115.stdout:1/682: creat d0/d8/de/d1a/d91/feb x:0 0 0 2024-03-22T10:42:48.171 INFO:tasks.workunit.client.0.smithi115.stdout:3/604: truncate d3/d82/d52/fc4 120919 0 2024-03-22T10:42:48.173 INFO:tasks.workunit.client.0.smithi115.stdout:9/634: creat d6/d7/d6a/fdf x:0 0 0 2024-03-22T10:42:48.173 INFO:tasks.workunit.client.1.smithi150.stdout:5/145: creat d0/d26/f29 x:0 0 0 2024-03-22T10:42:48.174 INFO:tasks.workunit.client.0.smithi115.stdout:5/641: symlink d2/df/d3b/d8e/dd3/ld8 0 2024-03-22T10:42:48.174 INFO:tasks.workunit.client.0.smithi115.stdout:4/603: creat d1/d30/d4c/d86/d9f/d32/fcf x:0 0 0 2024-03-22T10:42:48.175 INFO:tasks.workunit.client.1.smithi150.stdout:0/245: write d0/d12/d4f/f31 [3168559,115173] 0 2024-03-22T10:42:48.176 INFO:tasks.workunit.client.0.smithi115.stdout:1/683: read d0/d8/d4d/d56/db6/ddc/d82/f9a [145272,100793] 0 2024-03-22T10:42:48.178 INFO:tasks.workunit.client.0.smithi115.stdout:1/684: fdatasync d0/d12/d2b/f35 0 2024-03-22T10:42:48.179 INFO:tasks.workunit.client.0.smithi115.stdout:9/635: symlink d6/d7/d6a/d6f/le0 0 2024-03-22T10:42:48.180 INFO:tasks.workunit.client.1.smithi150.stdout:1/210: dwrite d1/da/d15/d18/f3b [0,4194304] 0 2024-03-22T10:42:48.186 INFO:tasks.workunit.client.0.smithi115.stdout:1/685: rmdir d0/d8/db/dd7 39 2024-03-22T10:42:48.186 INFO:tasks.workunit.client.1.smithi150.stdout:3/229: symlink d5/d22/l47 0 2024-03-22T10:42:48.186 INFO:tasks.workunit.client.1.smithi150.stdout:8/251: getdents d0/d2/d19/d29/d2f/df/d1b 0 2024-03-22T10:42:48.187 INFO:tasks.workunit.client.1.smithi150.stdout:6/168: dwrite db/dd/d21/f32 [0,4194304] 0 2024-03-22T10:42:48.190 INFO:tasks.workunit.client.1.smithi150.stdout:9/258: link d1/d19/c1d d1/d8/c68 0 2024-03-22T10:42:48.195 INFO:tasks.workunit.client.1.smithi150.stdout:9/259: dread - d1/d8/de/df/d20/d29/f54 zero size 2024-03-22T10:42:48.196 INFO:tasks.workunit.client.1.smithi150.stdout:0/246: read d0/d12/d4f/d42/fc [1958689,89137] 0 2024-03-22T10:42:48.199 INFO:tasks.workunit.client.0.smithi115.stdout:3/605: dwrite d3/d82/d54/f59 [0,4194304] 0 2024-03-22T10:42:48.201 INFO:tasks.workunit.client.1.smithi150.stdout:4/231: dread d0/d2f/f3e [0,4194304] 0 2024-03-22T10:42:48.201 INFO:tasks.workunit.client.1.smithi150.stdout:2/251: dread d12/d16/f43 [0,4194304] 0 2024-03-22T10:42:48.202 INFO:tasks.workunit.client.0.smithi115.stdout:3/606: write d3/f30 [2677729,52790] 0 2024-03-22T10:42:48.205 INFO:tasks.workunit.client.1.smithi150.stdout:1/211: chown d1/d19/d1a/c2d 41 1 2024-03-22T10:42:48.206 INFO:tasks.workunit.client.0.smithi115.stdout:5/642: dwrite d2/df/f10 [8388608,4194304] 0 2024-03-22T10:42:48.206 INFO:tasks.workunit.client.0.smithi115.stdout:3/607: read d3/f30 [4011295,43226] 0 2024-03-22T10:42:48.208 INFO:tasks.workunit.client.0.smithi115.stdout:6/560: dread d4/d11/f23 [0,4194304] 0 2024-03-22T10:42:48.208 INFO:tasks.workunit.client.1.smithi150.stdout:2/252: write d12/d19/f1d [2040058,61595] 0 2024-03-22T10:42:48.213 INFO:tasks.workunit.client.0.smithi115.stdout:3/608: creat d3/d19/d61/da6/fc9 x:0 0 0 2024-03-22T10:42:48.215 INFO:tasks.workunit.client.0.smithi115.stdout:6/561: symlink d4/d36/d48/lb4 0 2024-03-22T10:42:48.217 INFO:tasks.workunit.client.1.smithi150.stdout:0/247: dwrite d0/d12/d4f/ff [0,4194304] 0 2024-03-22T10:42:48.217 INFO:tasks.workunit.client.1.smithi150.stdout:3/230: mknod d5/d22/d2f/d38/c48 0 2024-03-22T10:42:48.219 INFO:tasks.workunit.client.1.smithi150.stdout:7/225: truncate d2/d5/d7/d8/f2d 2509637 0 2024-03-22T10:42:48.219 INFO:tasks.workunit.client.1.smithi150.stdout:6/169: symlink db/dd/d21/l40 0 2024-03-22T10:42:48.221 INFO:tasks.workunit.client.1.smithi150.stdout:3/231: dread - d5/d22/d2f/f41 zero size 2024-03-22T10:42:48.221 INFO:tasks.workunit.client.1.smithi150.stdout:6/170: readlink db/l16 0 2024-03-22T10:42:48.222 INFO:tasks.workunit.client.1.smithi150.stdout:5/146: link d0/f11 d0/d8/dc/de/d18/f2a 0 2024-03-22T10:42:48.223 INFO:tasks.workunit.client.0.smithi115.stdout:8/786: sync 2024-03-22T10:42:48.231 INFO:tasks.workunit.client.1.smithi150.stdout:7/226: truncate d2/d5/d7/d8/d2c/f40 458285 0 2024-03-22T10:42:48.235 INFO:tasks.workunit.client.1.smithi150.stdout:4/232: write d0/d1b/f23 [817779,7803] 0 2024-03-22T10:42:48.236 INFO:tasks.workunit.client.1.smithi150.stdout:9/260: rename d1/d8/de/df/d20/c63 to d1/d8/de/df/d24/c69 0 2024-03-22T10:42:48.236 INFO:tasks.workunit.client.0.smithi115.stdout:5/643: dwrite d2/df/d13/fbb [0,4194304] 0 2024-03-22T10:42:48.237 INFO:tasks.workunit.client.1.smithi150.stdout:1/212: dwrite d1/f1d [0,4194304] 0 2024-03-22T10:42:48.239 INFO:tasks.workunit.client.0.smithi115.stdout:9/636: write d6/d7/d20/d9c/fa1 [1645915,91106] 0 2024-03-22T10:42:48.241 INFO:tasks.workunit.client.1.smithi150.stdout:7/227: dread - d2/d5/d7/d38/f4b zero size 2024-03-22T10:42:48.244 INFO:tasks.workunit.client.0.smithi115.stdout:3/609: dwrite d3/d16/f1c [0,4194304] 0 2024-03-22T10:42:48.245 INFO:tasks.workunit.client.1.smithi150.stdout:9/261: fdatasync d1/d8/de/df/d20/d32/f5e 0 2024-03-22T10:42:48.245 INFO:tasks.workunit.client.1.smithi150.stdout:2/253: readlink d12/l1e 0 2024-03-22T10:42:48.248 INFO:tasks.workunit.client.1.smithi150.stdout:6/171: rmdir db/dd 39 2024-03-22T10:42:48.249 INFO:tasks.workunit.client.1.smithi150.stdout:3/232: mknod d5/da/c49 0 2024-03-22T10:42:48.249 INFO:tasks.workunit.client.1.smithi150.stdout:5/147: creat d0/f2b x:0 0 0 2024-03-22T10:42:48.249 INFO:tasks.workunit.client.0.smithi115.stdout:9/637: truncate d6/d7/d20/d9c/fa1 2251928 0 2024-03-22T10:42:48.250 INFO:tasks.workunit.client.0.smithi115.stdout:6/562: dwrite d4/d11/d12/d1c/fa5 [0,4194304] 0 2024-03-22T10:42:48.251 INFO:tasks.workunit.client.0.smithi115.stdout:5/644: read - d2/df/d13/d8b/d93/da6/fcf zero size 2024-03-22T10:42:48.252 INFO:tasks.workunit.client.1.smithi150.stdout:3/233: write d5/da/f33 [2795018,59073] 0 2024-03-22T10:42:48.253 INFO:tasks.workunit.client.1.smithi150.stdout:7/228: mkdir d2/d5/d7/d8/d51 0 2024-03-22T10:42:48.256 INFO:tasks.workunit.client.1.smithi150.stdout:3/234: chown d5/d22/d2f/f41 1805328266 1 2024-03-22T10:42:48.259 INFO:tasks.workunit.client.1.smithi150.stdout:0/248: creat d0/d12/d4f/d42/d10/d38/d52/f54 x:0 0 0 2024-03-22T10:42:48.261 INFO:tasks.workunit.client.0.smithi115.stdout:1/686: dread d0/d8/f53 [0,4194304] 0 2024-03-22T10:42:48.261 INFO:tasks.workunit.client.0.smithi115.stdout:9/638: symlink d6/d8/d3c/d59/d68/d97/le1 0 2024-03-22T10:42:48.264 INFO:tasks.workunit.client.0.smithi115.stdout:5/645: mknod d2/d15/d1b/cd9 0 2024-03-22T10:42:48.265 INFO:tasks.workunit.client.0.smithi115.stdout:4/604: dread d1/d66/f6d [0,4194304] 0 2024-03-22T10:42:48.265 INFO:tasks.workunit.client.1.smithi150.stdout:3/235: fdatasync d5/d22/d2f/d38/f42 0 2024-03-22T10:42:48.267 INFO:tasks.workunit.client.0.smithi115.stdout:9/639: write d6/d8/d3c/f96 [182920,34248] 0 2024-03-22T10:42:48.270 INFO:tasks.workunit.client.0.smithi115.stdout:5/646: chown d2/d15/d1b/d7c/d38/d43/d49/d81/d83 21007 1 2024-03-22T10:42:48.270 INFO:tasks.workunit.client.1.smithi150.stdout:3/236: chown d5/da 317366 1 2024-03-22T10:42:48.274 INFO:tasks.workunit.client.1.smithi150.stdout:1/213: dwrite d1/f7 [4194304,4194304] 0 2024-03-22T10:42:48.275 INFO:tasks.workunit.client.0.smithi115.stdout:4/605: creat d1/d30/d4c/d86/d9f/d75/d99/da8/fd0 x:0 0 0 2024-03-22T10:42:48.280 INFO:tasks.workunit.client.0.smithi115.stdout:3/610: dwrite d3/d36/f3e [0,4194304] 0 2024-03-22T10:42:48.280 INFO:tasks.workunit.client.0.smithi115.stdout:9/640: rename d6/d7/d30/d3b/c42 to d6/d7/d6a/d6f/ce2 0 2024-03-22T10:42:48.282 INFO:tasks.workunit.client.1.smithi150.stdout:6/172: creat db/dd/d15/d28/f41 x:0 0 0 2024-03-22T10:42:48.286 INFO:tasks.workunit.client.0.smithi115.stdout:6/563: dread d4/d11/f78 [0,4194304] 0 2024-03-22T10:42:48.288 INFO:tasks.workunit.client.0.smithi115.stdout:2/579: sync 2024-03-22T10:42:48.288 INFO:tasks.workunit.client.0.smithi115.stdout:0/577: sync 2024-03-22T10:42:48.288 INFO:tasks.workunit.client.0.smithi115.stdout:7/658: sync 2024-03-22T10:42:48.291 INFO:tasks.workunit.client.1.smithi150.stdout:6/173: write db/dd/d25/f3d [32052,118217] 0 2024-03-22T10:42:48.292 INFO:tasks.workunit.client.1.smithi150.stdout:0/249: symlink d0/d12/d4f/d42/d10/d38/d52/l55 0 2024-03-22T10:42:48.293 INFO:tasks.workunit.client.0.smithi115.stdout:2/580: fdatasync da/d25/d3d/f9f 0 2024-03-22T10:42:48.294 INFO:tasks.workunit.client.1.smithi150.stdout:7/229: dwrite d2/d5/d7/f4d [0,4194304] 0 2024-03-22T10:42:48.297 INFO:tasks.workunit.client.0.smithi115.stdout:9/641: creat d6/d7/d20/d22/fe3 x:0 0 0 2024-03-22T10:42:48.298 INFO:tasks.workunit.client.1.smithi150.stdout:4/233: dread d0/de/f18 [0,4194304] 0 2024-03-22T10:42:48.299 INFO:tasks.workunit.client.0.smithi115.stdout:6/564: mknod d4/d11/d12/d1c/d8a/d98/d9f/cb5 0 2024-03-22T10:42:48.305 INFO:tasks.workunit.client.1.smithi150.stdout:3/237: dwrite d5/f35 [0,4194304] 0 2024-03-22T10:42:48.311 INFO:tasks.workunit.client.1.smithi150.stdout:6/174: unlink db/dd/d2e/f31 0 2024-03-22T10:42:48.312 INFO:tasks.workunit.client.0.smithi115.stdout:4/606: dwrite d1/d30/d4c/d86/d9f/d32/fb3 [0,4194304] 0 2024-03-22T10:42:48.313 INFO:tasks.workunit.client.0.smithi115.stdout:9/642: rmdir d6/d7/d20/d38/db2 0 2024-03-22T10:42:48.314 INFO:tasks.workunit.client.0.smithi115.stdout:6/565: creat d4/fb6 x:0 0 0 2024-03-22T10:42:48.316 INFO:tasks.workunit.client.1.smithi150.stdout:7/230: creat d2/d5/f52 x:0 0 0 2024-03-22T10:42:48.316 INFO:tasks.workunit.client.1.smithi150.stdout:8/252: write d0/d3c/f4a [661407,69596] 0 2024-03-22T10:42:48.318 INFO:tasks.workunit.client.1.smithi150.stdout:5/148: dread d0/d8/dc/de/f15 [0,4194304] 0 2024-03-22T10:42:48.321 INFO:tasks.workunit.client.0.smithi115.stdout:7/659: dwrite d2/d7/d12/d10/d19/d42/f85 [4194304,4194304] 0 2024-03-22T10:42:48.323 INFO:tasks.workunit.client.0.smithi115.stdout:6/566: read d4/d11/d12/d1c/f69 [3893098,114779] 0 2024-03-22T10:42:48.327 INFO:tasks.workunit.client.0.smithi115.stdout:9/643: symlink d6/d8/d3c/le4 0 2024-03-22T10:42:48.328 INFO:tasks.workunit.client.1.smithi150.stdout:3/238: unlink d5/d22/f2b 0 2024-03-22T10:42:48.328 INFO:tasks.workunit.client.0.smithi115.stdout:2/581: dwrite da/d25/d3d/f28 [0,4194304] 0 2024-03-22T10:42:48.330 INFO:tasks.workunit.client.1.smithi150.stdout:1/214: creat d1/f3c x:0 0 0 2024-03-22T10:42:48.330 INFO:tasks.workunit.client.1.smithi150.stdout:9/262: write d1/d8/de/df/f1f [684606,106432] 0 2024-03-22T10:42:48.332 INFO:tasks.workunit.client.1.smithi150.stdout:3/239: dread - d5/d22/d2f/d38/d3e/f3f zero size 2024-03-22T10:42:48.335 INFO:tasks.workunit.client.1.smithi150.stdout:9/263: write d1/d8/de/df/d20/f5d [600733,95316] 0 2024-03-22T10:42:48.335 INFO:tasks.workunit.client.0.smithi115.stdout:7/660: mkdir d2/d7/d12/d10/d23/d3d/da4/de5 0 2024-03-22T10:42:48.342 INFO:tasks.workunit.client.0.smithi115.stdout:6/567: rename d4/d11/d12/f21 to d4/d13/d33/d58/fb7 0 2024-03-22T10:42:48.343 INFO:tasks.workunit.client.1.smithi150.stdout:0/250: creat d0/f56 x:0 0 0 2024-03-22T10:42:48.345 INFO:tasks.workunit.client.1.smithi150.stdout:1/215: creat d1/da/d15/d18/f3d x:0 0 0 2024-03-22T10:42:48.346 INFO:tasks.workunit.client.1.smithi150.stdout:8/253: creat d0/d2/d4d/f59 x:0 0 0 2024-03-22T10:42:48.347 INFO:tasks.workunit.client.0.smithi115.stdout:9/644: mkdir d6/d7/d20/d22/de5 0 2024-03-22T10:42:48.348 INFO:tasks.workunit.client.1.smithi150.stdout:4/234: link d0/f14 d0/d32/f4e 0 2024-03-22T10:42:48.350 INFO:tasks.workunit.client.0.smithi115.stdout:1/687: write d0/d8/db/d25/fa7 [860447,27139] 0 2024-03-22T10:42:48.350 INFO:tasks.workunit.client.1.smithi150.stdout:0/251: chown d0/d12/d4f/d42/d10/d38/d52 118229 1 2024-03-22T10:42:48.351 INFO:tasks.workunit.client.0.smithi115.stdout:6/568: symlink d4/d11/d12/d1c/d8a/lb8 0 2024-03-22T10:42:48.352 INFO:tasks.workunit.client.1.smithi150.stdout:6/175: link db/l1d db/dd/d15/l42 0 2024-03-22T10:42:48.353 INFO:tasks.workunit.client.1.smithi150.stdout:0/252: chown d0/d12/d4f/fe 3 1 2024-03-22T10:42:48.354 INFO:tasks.workunit.client.1.smithi150.stdout:8/254: write d0/d2/d19/d29/d2f/df/d1b/f24 [953470,92590] 0 2024-03-22T10:42:48.355 INFO:tasks.workunit.client.1.smithi150.stdout:9/264: creat d1/d8/de/df/d36/d40/d44/f6a x:0 0 0 2024-03-22T10:42:48.356 INFO:tasks.workunit.client.0.smithi115.stdout:8/787: dwrite d6/dd/f33 [0,4194304] 0 2024-03-22T10:42:48.356 INFO:tasks.workunit.client.0.smithi115.stdout:9/645: symlink d6/le6 0 2024-03-22T10:42:48.356 INFO:tasks.workunit.client.0.smithi115.stdout:5/647: truncate d2/df/f55 2910565 0 2024-03-22T10:42:48.360 INFO:tasks.workunit.client.1.smithi150.stdout:9/265: write d1/d8/de/df/f1f [1722442,79902] 0 2024-03-22T10:42:48.361 INFO:tasks.workunit.client.0.smithi115.stdout:3/611: dread d3/f12 [0,4194304] 0 2024-03-22T10:42:48.361 INFO:tasks.workunit.client.0.smithi115.stdout:1/688: mkdir d0/d12/d17/d37/d9e/de9/dec 0 2024-03-22T10:42:48.362 INFO:tasks.workunit.client.1.smithi150.stdout:4/235: creat d0/d2f/f4f x:0 0 0 2024-03-22T10:42:48.363 INFO:tasks.workunit.client.0.smithi115.stdout:7/661: link d2/d7/d12/d10/d19/f8b d2/fe6 0 2024-03-22T10:42:48.363 INFO:tasks.workunit.client.0.smithi115.stdout:1/689: readlink d0/d8/ld 0 2024-03-22T10:42:48.364 INFO:tasks.workunit.client.1.smithi150.stdout:5/149: dread d0/f10 [0,4194304] 0 2024-03-22T10:42:48.364 INFO:tasks.workunit.client.1.smithi150.stdout:4/236: chown d0/c22 6960 1 2024-03-22T10:42:48.366 INFO:tasks.workunit.client.1.smithi150.stdout:5/150: stat d0/d8/dc/lf 0 2024-03-22T10:42:48.367 INFO:tasks.workunit.client.0.smithi115.stdout:9/646: creat d6/d7/d20/d22/d3f/fe7 x:0 0 0 2024-03-22T10:42:48.368 INFO:tasks.workunit.client.0.smithi115.stdout:9/647: chown d6/d7/l71 498 1 2024-03-22T10:42:48.369 INFO:tasks.workunit.client.0.smithi115.stdout:7/662: symlink d2/le7 0 2024-03-22T10:42:48.369 INFO:tasks.workunit.client.1.smithi150.stdout:5/151: readlink d0/lb 0 2024-03-22T10:42:48.370 INFO:tasks.workunit.client.1.smithi150.stdout:9/266: truncate d1/d8/de/df/d36/d40/d44/f6a 617461 0 2024-03-22T10:42:48.373 INFO:tasks.workunit.client.1.smithi150.stdout:5/152: read d0/f5 [254958,70686] 0 2024-03-22T10:42:48.373 INFO:tasks.workunit.client.0.smithi115.stdout:7/663: write d2/d7/d12/d10/d23/f97 [777962,116112] 0 2024-03-22T10:42:48.374 INFO:tasks.workunit.client.1.smithi150.stdout:5/153: chown d0/d8 1376 1 2024-03-22T10:42:48.374 INFO:tasks.workunit.client.0.smithi115.stdout:3/612: rename d3/d19/d2b/d55 to d3/d89/dca 0 2024-03-22T10:42:48.376 INFO:tasks.workunit.client.0.smithi115.stdout:9/648: mknod d6/d7/d6a/d6f/ce8 0 2024-03-22T10:42:48.379 INFO:tasks.workunit.client.1.smithi150.stdout:6/176: creat db/dd/d15/d30/f43 x:0 0 0 2024-03-22T10:42:48.381 INFO:tasks.workunit.client.0.smithi115.stdout:5/648: dwrite d2/df/d3b/fb0 [4194304,4194304] 0 2024-03-22T10:42:48.382 INFO:tasks.workunit.client.1.smithi150.stdout:1/216: creat d1/da/d15/f3e x:0 0 0 2024-03-22T10:42:48.382 INFO:tasks.workunit.client.1.smithi150.stdout:5/154: dread d0/f3 [0,4194304] 0 2024-03-22T10:42:48.384 INFO:tasks.workunit.client.0.smithi115.stdout:8/788: rmdir d6/dbc/d100 0 2024-03-22T10:42:48.386 INFO:tasks.workunit.client.0.smithi115.stdout:3/613: creat d3/d89/dca/fcb x:0 0 0 2024-03-22T10:42:48.387 INFO:tasks.workunit.client.1.smithi150.stdout:1/217: write d1/f3c [950389,125402] 0 2024-03-22T10:42:48.388 INFO:tasks.workunit.client.1.smithi150.stdout:6/177: chown db/dd/c1e 1092 1 2024-03-22T10:42:48.389 INFO:tasks.workunit.client.1.smithi150.stdout:4/237: getdents d0/d1b/d4d 0 2024-03-22T10:42:48.389 INFO:tasks.workunit.client.0.smithi115.stdout:5/649: readlink d2/d15/d1b/d7c/d38/d56/lcb 0 2024-03-22T10:42:48.391 INFO:tasks.workunit.client.0.smithi115.stdout:3/614: truncate d3/d82/d52/fc0 1037583 0 2024-03-22T10:42:48.391 INFO:tasks.workunit.client.1.smithi150.stdout:4/238: dread - d0/d32/f4a zero size 2024-03-22T10:42:48.392 INFO:tasks.workunit.client.1.smithi150.stdout:8/255: creat d0/f5a x:0 0 0 2024-03-22T10:42:48.393 INFO:tasks.workunit.client.0.smithi115.stdout:8/789: chown d6/dd/d24/d4a/d7b/d8f/fd8 153 1 2024-03-22T10:42:48.394 INFO:tasks.workunit.client.0.smithi115.stdout:1/690: getdents d0/d8/d4d/d56/dbd 0 2024-03-22T10:42:48.395 INFO:tasks.workunit.client.0.smithi115.stdout:4/607: truncate d1/d16/d35/d40/d5c/f23 4260763 0 2024-03-22T10:42:48.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:47 smithi150 ceph-mon[34518]: pgmap v12: 65 pgs: 65 active+clean; 1.5 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 44 MiB/s rd, 218 MiB/s wr, 432 op/s 2024-03-22T10:42:48.398 INFO:tasks.workunit.client.0.smithi115.stdout:2/582: dread da/d25/d3d/d57/f85 [0,4194304] 0 2024-03-22T10:42:48.398 INFO:tasks.workunit.client.0.smithi115.stdout:9/649: mkdir d6/d7/d48/d70/da9/dbd/de9 0 2024-03-22T10:42:48.399 INFO:tasks.workunit.client.1.smithi150.stdout:9/267: link d1/d8/de/df/d36/d40/d41/f47 d1/d8/de/df/d24/f6b 0 2024-03-22T10:42:48.400 INFO:tasks.workunit.client.0.smithi115.stdout:1/691: chown d0/d8/d4d/d56/db6/ddc/d82 242 1 2024-03-22T10:42:48.400 INFO:tasks.workunit.client.0.smithi115.stdout:4/608: symlink d1/d66/ld1 0 2024-03-22T10:42:48.400 INFO:tasks.workunit.client.0.smithi115.stdout:8/790: write d6/dd/d24/d4a/d7b/fa2 [1992388,31007] 0 2024-03-22T10:42:48.401 INFO:tasks.workunit.client.1.smithi150.stdout:5/155: mkdir d0/d8/dc/d2c 0 2024-03-22T10:42:48.404 INFO:tasks.workunit.client.1.smithi150.stdout:5/156: chown d0/f22 0 1 2024-03-22T10:42:48.404 INFO:tasks.workunit.client.0.smithi115.stdout:5/650: symlink d2/df/d13/d8b/d93/lda 0 2024-03-22T10:42:48.404 INFO:tasks.workunit.client.0.smithi115.stdout:1/692: dread - d0/d8/db/d25/d9f/fd3 zero size 2024-03-22T10:42:48.404 INFO:tasks.workunit.client.1.smithi150.stdout:3/240: truncate d5/d22/d2f/f30 3920245 0 2024-03-22T10:42:48.405 INFO:tasks.workunit.client.0.smithi115.stdout:9/650: mknod d6/d7/d48/cea 0 2024-03-22T10:42:48.405 INFO:tasks.workunit.client.1.smithi150.stdout:7/231: write d2/d5/d7/d8/f2d [2004042,73690] 0 2024-03-22T10:42:48.406 INFO:tasks.workunit.client.0.smithi115.stdout:8/791: symlink d6/d49/d68/dcf/l103 0 2024-03-22T10:42:48.408 INFO:tasks.workunit.client.1.smithi150.stdout:0/253: write d0/f3a [143133,12636] 0 2024-03-22T10:42:48.409 INFO:tasks.workunit.client.0.smithi115.stdout:1/693: dread - d0/d8/db/d25/d9f/fc2 zero size 2024-03-22T10:42:48.411 INFO:tasks.workunit.client.0.smithi115.stdout:9/651: rmdir d6/d7/d48/d70/da9 39 2024-03-22T10:42:48.412 INFO:tasks.workunit.client.0.smithi115.stdout:5/651: rmdir d2/d15/d1b/d7c/d84 39 2024-03-22T10:42:48.412 INFO:tasks.workunit.client.0.smithi115.stdout:3/615: dread d3/d36/d41/f3f [0,4194304] 0 2024-03-22T10:42:48.415 INFO:tasks.workunit.client.0.smithi115.stdout:8/792: symlink d6/dd/d24/d4a/d41/l104 0 2024-03-22T10:42:48.415 INFO:tasks.workunit.client.1.smithi150.stdout:0/254: dread - d0/d12/d4f/d42/d10/d38/d52/f54 zero size 2024-03-22T10:42:48.416 INFO:tasks.workunit.client.1.smithi150.stdout:1/218: symlink d1/l3f 0 2024-03-22T10:42:48.421 INFO:tasks.workunit.client.0.smithi115.stdout:3/616: creat d3/d36/d41/d85/fcc x:0 0 0 2024-03-22T10:42:48.422 INFO:tasks.workunit.client.0.smithi115.stdout:4/609: dread d1/d66/d82/f9a [0,4194304] 0 2024-03-22T10:42:48.424 INFO:tasks.workunit.client.1.smithi150.stdout:5/157: dread d0/f10 [0,4194304] 0 2024-03-22T10:42:48.426 INFO:tasks.workunit.client.0.smithi115.stdout:8/793: mkdir d6/dd/d24/d4a/d7b/d8f/da0/d105 0 2024-03-22T10:42:48.426 INFO:tasks.workunit.client.1.smithi150.stdout:7/232: creat d2/d5/d7/d9/f53 x:0 0 0 2024-03-22T10:42:48.429 INFO:tasks.workunit.client.1.smithi150.stdout:3/241: write d5/da/f45 [766726,70236] 0 2024-03-22T10:42:48.430 INFO:tasks.workunit.client.1.smithi150.stdout:9/268: dwrite d1/d8/de/df/d36/f43 [0,4194304] 0 2024-03-22T10:42:48.433 INFO:tasks.workunit.client.1.smithi150.stdout:0/255: link d0/d12/d4f/d42/d10/d38/f4c d0/d12/d4f/d42/d10/d53/f57 0 2024-03-22T10:42:48.434 INFO:tasks.workunit.client.1.smithi150.stdout:5/158: fsync d0/d8/fd 0 2024-03-22T10:42:48.436 INFO:tasks.workunit.client.0.smithi115.stdout:5/652: dwrite d2/df/d13/fbb [4194304,4194304] 0 2024-03-22T10:42:48.436 INFO:tasks.workunit.client.0.smithi115.stdout:8/794: read d6/dd/d24/d3a/f54 [4044977,57736] 0 2024-03-22T10:42:48.440 INFO:tasks.workunit.client.1.smithi150.stdout:8/256: dread d0/f17 [0,4194304] 0 2024-03-22T10:42:48.440 INFO:tasks.workunit.client.1.smithi150.stdout:9/269: truncate d1/d8/de/df/d24/f5f 747397 0 2024-03-22T10:42:48.441 INFO:tasks.workunit.client.1.smithi150.stdout:7/233: truncate d2/d5/d7/d2e/f3c 526256 0 2024-03-22T10:42:48.441 INFO:tasks.workunit.client.1.smithi150.stdout:6/178: truncate db/dd/f37 3340695 0 2024-03-22T10:42:48.441 INFO:tasks.workunit.client.0.smithi115.stdout:4/610: truncate d1/d16/d35/d40/d5c/d14/f58 4496477 0 2024-03-22T10:42:48.441 INFO:tasks.workunit.client.0.smithi115.stdout:9/652: truncate d6/d7/d20/d22/fad 1832716 0 2024-03-22T10:42:48.442 INFO:tasks.workunit.client.0.smithi115.stdout:5/653: fsync d2/f71 0 2024-03-22T10:42:48.442 INFO:tasks.workunit.client.1.smithi150.stdout:0/256: creat d0/d12/f58 x:0 0 0 2024-03-22T10:42:48.443 INFO:tasks.workunit.client.1.smithi150.stdout:5/159: mknod d0/d8/dc/c2d 0 2024-03-22T10:42:48.444 INFO:tasks.workunit.client.1.smithi150.stdout:3/242: chown d5/d22/l2e 63772 1 2024-03-22T10:42:48.444 INFO:tasks.workunit.client.1.smithi150.stdout:1/219: dwrite d1/f3a [0,4194304] 0 2024-03-22T10:42:48.446 INFO:tasks.workunit.client.1.smithi150.stdout:7/234: dread - d2/d5/d7/d38/f4b zero size 2024-03-22T10:42:48.453 INFO:tasks.workunit.client.0.smithi115.stdout:1/694: dread d0/d22/f3d [0,4194304] 0 2024-03-22T10:42:48.453 INFO:tasks.workunit.client.0.smithi115.stdout:9/653: chown d6/d7/d48/d70/fc0 157 1 2024-03-22T10:42:48.454 INFO:tasks.workunit.client.0.smithi115.stdout:3/617: dwrite d3/d82/fa4 [0,4194304] 0 2024-03-22T10:42:48.455 INFO:tasks.workunit.client.1.smithi150.stdout:5/160: mkdir d0/d8/dc/de/d14/d2e 0 2024-03-22T10:42:48.455 INFO:tasks.workunit.client.1.smithi150.stdout:0/257: dread d0/f3a [0,4194304] 0 2024-03-22T10:42:48.455 INFO:tasks.workunit.client.1.smithi150.stdout:6/179: creat db/f44 x:0 0 0 2024-03-22T10:42:48.457 INFO:tasks.workunit.client.0.smithi115.stdout:4/611: rename d1/d16/d35/d40/d5c/d14/f76 to d1/d30/fd2 0 2024-03-22T10:42:48.457 INFO:tasks.workunit.client.1.smithi150.stdout:3/243: truncate d5/da/f34 617094 0 2024-03-22T10:42:48.459 INFO:tasks.workunit.client.0.smithi115.stdout:3/618: write d3/d16/d29/f43 [2778861,96062] 0 2024-03-22T10:42:48.459 INFO:tasks.workunit.client.1.smithi150.stdout:1/220: mknod d1/d19/d1a/c40 0 2024-03-22T10:42:48.459 INFO:tasks.workunit.client.1.smithi150.stdout:7/235: dread - d2/d5/d7/d38/f4b zero size 2024-03-22T10:42:48.461 INFO:tasks.workunit.client.0.smithi115.stdout:4/612: readlink d1/d16/d35/d40/d5c/d14/l4e 0 2024-03-22T10:42:48.462 INFO:tasks.workunit.client.0.smithi115.stdout:1/695: write d0/d8/d4d/d56/f6d [4093122,48086] 0 2024-03-22T10:42:48.462 INFO:tasks.workunit.client.1.smithi150.stdout:7/236: chown d2/d5/d7/d8 22 1 2024-03-22T10:42:48.462 INFO:tasks.workunit.client.1.smithi150.stdout:8/257: getdents d0/d2/d19/d29 0 2024-03-22T10:42:48.463 INFO:tasks.workunit.client.0.smithi115.stdout:3/619: readlink d3/d19/d2b/d49/d91/da5/l98 0 2024-03-22T10:42:48.466 INFO:tasks.workunit.client.0.smithi115.stdout:1/696: chown d0/d8/db/d25/f73 16 1 2024-03-22T10:42:48.466 INFO:tasks.workunit.client.0.smithi115.stdout:9/654: rename d6/d8/l13 to d6/d7/d20/d22/de5/leb 0 2024-03-22T10:42:48.469 INFO:tasks.workunit.client.0.smithi115.stdout:4/613: creat d1/d66/d82/da1/fd3 x:0 0 0 2024-03-22T10:42:48.470 INFO:tasks.workunit.client.1.smithi150.stdout:6/180: creat db/dd/d25/f45 x:0 0 0 2024-03-22T10:42:48.471 INFO:tasks.workunit.client.0.smithi115.stdout:8/795: write d6/f16 [1967472,110174] 0 2024-03-22T10:42:48.473 INFO:tasks.workunit.client.1.smithi150.stdout:3/244: mknod d5/d22/d37/c4a 0 2024-03-22T10:42:48.474 INFO:tasks.workunit.client.0.smithi115.stdout:1/697: write d0/d8/d4d/d56/dbd/fd0 [77226,98708] 0 2024-03-22T10:42:48.474 INFO:tasks.workunit.client.0.smithi115.stdout:4/614: fdatasync d1/d30/d4c/d86/d9f/d73/fcc 0 2024-03-22T10:42:48.475 INFO:tasks.workunit.client.1.smithi150.stdout:7/237: truncate d2/d5/d7/f50 889301 0 2024-03-22T10:42:48.476 INFO:tasks.workunit.client.1.smithi150.stdout:3/245: chown d5/da/c49 2346404 1 2024-03-22T10:42:48.480 INFO:tasks.workunit.client.1.smithi150.stdout:6/181: dread db/dd/d25/f3d [0,4194304] 0 2024-03-22T10:42:48.480 INFO:tasks.workunit.client.1.smithi150.stdout:3/246: write d5/da/f40 [3906700,84358] 0 2024-03-22T10:42:48.481 INFO:tasks.workunit.client.0.smithi115.stdout:1/698: stat d0/d12/d17/d37/d9e/de9/dde 0 2024-03-22T10:42:48.481 INFO:tasks.workunit.client.0.smithi115.stdout:3/620: rename d3/d16/c7b to d3/d16/d29/d9b/ccd 0 2024-03-22T10:42:48.481 INFO:tasks.workunit.client.0.smithi115.stdout:4/615: chown d1/d66/d82/fa6 16632 1 2024-03-22T10:42:48.481 INFO:tasks.workunit.client.0.smithi115.stdout:3/621: chown d3/d82/la8 27098 1 2024-03-22T10:42:48.484 INFO:tasks.workunit.client.0.smithi115.stdout:8/796: truncate d6/dd/d12/d31/f90 638236 0 2024-03-22T10:42:48.486 INFO:tasks.workunit.client.1.smithi150.stdout:3/247: stat d5/da/c11 0 2024-03-22T10:42:48.486 INFO:tasks.workunit.client.0.smithi115.stdout:1/699: fsync d0/f52 0 2024-03-22T10:42:48.487 INFO:tasks.workunit.client.0.smithi115.stdout:4/616: write d1/d30/d4c/d86/d9f/d32/fcf [86634,26481] 0 2024-03-22T10:42:48.490 INFO:tasks.workunit.client.1.smithi150.stdout:8/258: getdents d0 0 2024-03-22T10:42:48.490 INFO:tasks.workunit.client.1.smithi150.stdout:1/221: link d1/d19/d1a/c2d d1/d19/c41 0 2024-03-22T10:42:48.493 INFO:tasks.workunit.client.1.smithi150.stdout:3/248: symlink d5/d22/d2f/l4b 0 2024-03-22T10:42:48.495 INFO:tasks.workunit.client.0.smithi115.stdout:4/617: write d1/d66/d82/da1/fd3 [488903,130379] 0 2024-03-22T10:42:48.496 INFO:tasks.workunit.client.1.smithi150.stdout:8/259: write d0/f1 [1490468,5460] 0 2024-03-22T10:42:48.497 INFO:tasks.workunit.client.0.smithi115.stdout:9/655: dwrite d6/d7/d30/d3b/f58 [4194304,4194304] 0 2024-03-22T10:42:48.498 INFO:tasks.workunit.client.1.smithi150.stdout:3/249: chown d5/da/l18 7824667 1 2024-03-22T10:42:48.499 INFO:tasks.workunit.client.0.smithi115.stdout:1/700: mkdir d0/d12/d17/d37/d9e/de9/dec/ded 0 2024-03-22T10:42:48.499 INFO:tasks.workunit.client.0.smithi115.stdout:8/797: symlink d6/dd/d24/d4a/d7b/l106 0 2024-03-22T10:42:48.500 INFO:tasks.workunit.client.1.smithi150.stdout:8/260: dread - d0/d2/d19/d29/d31/f49 zero size 2024-03-22T10:42:48.502 INFO:tasks.workunit.client.0.smithi115.stdout:4/618: mknod d1/d30/d4c/d86/d9f/d75/cd4 0 2024-03-22T10:42:48.504 INFO:tasks.workunit.client.1.smithi150.stdout:5/161: write d0/f11 [4658135,111915] 0 2024-03-22T10:42:48.507 INFO:tasks.workunit.client.1.smithi150.stdout:3/250: rename d5/d22/l43 to d5/da/l4c 0 2024-03-22T10:42:48.507 INFO:tasks.workunit.client.0.smithi115.stdout:8/798: creat d6/dd/d24/d3a/f107 x:0 0 0 2024-03-22T10:42:48.507 INFO:tasks.workunit.client.1.smithi150.stdout:8/261: creat d0/f5b x:0 0 0 2024-03-22T10:42:48.508 INFO:tasks.workunit.client.1.smithi150.stdout:9/270: truncate d1/d8/de/df/d36/f5c 541631 0 2024-03-22T10:42:48.509 INFO:tasks.workunit.client.0.smithi115.stdout:3/622: dwrite d3/d82/d54/fae [0,4194304] 0 2024-03-22T10:42:48.511 INFO:tasks.workunit.client.0.smithi115.stdout:1/701: rename d0/d12/d17/d37/f83 to d0/d8/d4d/d56/db6/fee 0 2024-03-22T10:42:48.513 INFO:tasks.workunit.client.1.smithi150.stdout:3/251: fdatasync d5/d22/d37/f3d 0 2024-03-22T10:42:48.514 INFO:tasks.workunit.client.0.smithi115.stdout:8/799: fdatasync d6/dd/f76 0 2024-03-22T10:42:48.515 INFO:tasks.workunit.client.1.smithi150.stdout:2/254: sync 2024-03-22T10:42:48.515 INFO:tasks.workunit.client.1.smithi150.stdout:1/222: rename d1/da/f13 to d1/da/f42 0 2024-03-22T10:42:48.515 INFO:tasks.workunit.client.1.smithi150.stdout:5/162: mkdir d0/d8/dc/d2f 0 2024-03-22T10:42:48.517 INFO:tasks.workunit.client.1.smithi150.stdout:0/258: dread d0/d12/d4f/f28 [0,4194304] 0 2024-03-22T10:42:48.517 INFO:tasks.workunit.client.0.smithi115.stdout:3/623: chown d3/d19/d2b/d49/d91/da5/fa7 11731679 1 2024-03-22T10:42:48.518 INFO:tasks.workunit.client.1.smithi150.stdout:7/238: dwrite d2/d5/d7/f2b [0,4194304] 0 2024-03-22T10:42:48.518 INFO:tasks.workunit.client.1.smithi150.stdout:3/252: chown d5/d22/c2d 596182 1 2024-03-22T10:42:48.519 INFO:tasks.workunit.client.0.smithi115.stdout:8/800: mknod d6/dd/d12/dc6/c108 0 2024-03-22T10:42:48.519 INFO:tasks.workunit.client.0.smithi115.stdout:1/702: symlink d0/d8/d4d/d56/dcf/lef 0 2024-03-22T10:42:48.520 INFO:tasks.workunit.client.0.smithi115.stdout:3/624: write d3/f23 [2057442,102523] 0 2024-03-22T10:42:48.523 INFO:tasks.workunit.client.1.smithi150.stdout:7/239: fdatasync d2/d5/d7/d38/f4b 0 2024-03-22T10:42:48.525 INFO:tasks.workunit.client.1.smithi150.stdout:3/253: read - d5/f31 zero size 2024-03-22T10:42:48.527 INFO:tasks.workunit.client.1.smithi150.stdout:0/259: write d0/d12/d4f/d42/d10/f13 [5290221,113622] 0 2024-03-22T10:42:48.527 INFO:tasks.workunit.client.0.smithi115.stdout:8/801: creat d6/dd/d12/d31/f109 x:0 0 0 2024-03-22T10:42:48.529 INFO:tasks.workunit.client.0.smithi115.stdout:3/625: rename d3/d82/d54/fae to d3/d19/d61/db3/fce 0 2024-03-22T10:42:48.530 INFO:tasks.workunit.client.0.smithi115.stdout:4/619: dwrite d1/d66/f77 [4194304,4194304] 0 2024-03-22T10:42:48.531 INFO:tasks.workunit.client.0.smithi115.stdout:1/703: rmdir d0/d12/d17/d37/d5c/dc9 0 2024-03-22T10:42:48.531 INFO:tasks.workunit.client.1.smithi150.stdout:5/163: dread d0/d8/dc/f12 [0,4194304] 0 2024-03-22T10:42:48.532 INFO:tasks.workunit.client.1.smithi150.stdout:1/223: creat d1/da/f43 x:0 0 0 2024-03-22T10:42:48.533 INFO:tasks.workunit.client.1.smithi150.stdout:7/240: mkdir d2/d5/d7/d2e/d54 0 2024-03-22T10:42:48.536 INFO:tasks.workunit.client.0.smithi115.stdout:3/626: mknod d3/d19/d2b/ccf 0 2024-03-22T10:42:48.537 INFO:tasks.workunit.client.0.smithi115.stdout:1/704: fdatasync d0/d12/d17/d37/d9e/de9/f90 0 2024-03-22T10:42:48.538 INFO:tasks.workunit.client.1.smithi150.stdout:3/254: creat d5/d22/d2f/d38/d3e/f4d x:0 0 0 2024-03-22T10:42:48.540 INFO:tasks.workunit.client.1.smithi150.stdout:0/260: truncate d0/d12/f3b 1097817 0 2024-03-22T10:42:48.542 INFO:tasks.workunit.client.1.smithi150.stdout:8/262: unlink d0/d2/d19/d29/d2f/l41 0 2024-03-22T10:42:48.543 INFO:tasks.workunit.client.1.smithi150.stdout:3/255: write d5/d22/d2f/d38/f42 [907850,79493] 0 2024-03-22T10:42:48.546 INFO:tasks.workunit.client.0.smithi115.stdout:1/705: creat d0/ff0 x:0 0 0 2024-03-22T10:42:48.547 INFO:tasks.workunit.client.0.smithi115.stdout:8/802: link d6/d49/d88/fe9 d6/dd/d24/d4a/dfc/f10a 0 2024-03-22T10:42:48.547 INFO:tasks.workunit.client.1.smithi150.stdout:0/261: readlink d0/d12/d4f/d42/d10/l34 0 2024-03-22T10:42:48.551 INFO:tasks.workunit.client.0.smithi115.stdout:4/620: dwrite d1/d69/f65 [0,4194304] 0 2024-03-22T10:42:48.551 INFO:tasks.workunit.client.0.smithi115.stdout:8/803: mknod d6/c10b 0 2024-03-22T10:42:48.551 INFO:tasks.workunit.client.1.smithi150.stdout:7/241: truncate d2/d5/d7/d9/db/f1a 496107 0 2024-03-22T10:42:48.552 INFO:tasks.workunit.client.1.smithi150.stdout:2/255: dread d12/d48/f4c [0,4194304] 0 2024-03-22T10:42:48.556 INFO:tasks.workunit.client.1.smithi150.stdout:7/242: fdatasync d2/d5/d7/f4d 0 2024-03-22T10:42:48.556 INFO:tasks.workunit.client.0.smithi115.stdout:4/621: fsync d1/d16/d68/f6c 0 2024-03-22T10:42:48.561 INFO:tasks.workunit.client.1.smithi150.stdout:2/256: creat d12/d21/f4f x:0 0 0 2024-03-22T10:42:48.567 INFO:tasks.workunit.client.1.smithi150.stdout:2/257: chown d12/d19/d1a/l49 5 1 2024-03-22T10:42:48.569 INFO:tasks.workunit.client.0.smithi115.stdout:1/706: dwrite d0/d8/d4d/d56/db6/ddc/d82/f9a [0,4194304] 0 2024-03-22T10:42:48.572 INFO:tasks.workunit.client.0.smithi115.stdout:1/707: fsync d0/d12/d17/f7d 0 2024-03-22T10:42:48.573 INFO:tasks.workunit.client.1.smithi150.stdout:5/164: truncate d0/f6 3806018 0 2024-03-22T10:42:48.573 INFO:tasks.workunit.client.1.smithi150.stdout:1/224: rmdir d1/da 39 2024-03-22T10:42:48.575 INFO:tasks.workunit.client.0.smithi115.stdout:3/627: truncate d3/f53 2362826 0 2024-03-22T10:42:48.576 INFO:tasks.workunit.client.1.smithi150.stdout:5/165: readlink d0/d8/l1c 0 2024-03-22T10:42:48.576 INFO:tasks.workunit.client.1.smithi150.stdout:2/258: mknod d12/d48/c50 0 2024-03-22T10:42:48.576 INFO:tasks.workunit.client.0.smithi115.stdout:4/622: dwrite d1/d66/fc1 [0,4194304] 0 2024-03-22T10:42:48.577 INFO:tasks.workunit.client.1.smithi150.stdout:8/263: dwrite d0/d2/d19/d29/d2e/f3a [0,4194304] 0 2024-03-22T10:42:48.581 INFO:tasks.workunit.client.1.smithi150.stdout:5/166: fdatasync d0/d8/dc/de/d14/f16 0 2024-03-22T10:42:48.582 INFO:tasks.workunit.client.0.smithi115.stdout:3/628: rename d3/d89/f92 to d3/d82/d57/fd0 0 2024-03-22T10:42:48.584 INFO:tasks.workunit.client.1.smithi150.stdout:0/262: dwrite d0/d12/d4f/d42/f4a [0,4194304] 0 2024-03-22T10:42:48.591 INFO:tasks.workunit.client.0.smithi115.stdout:3/629: unlink d3/d19/f31 0 2024-03-22T10:42:48.593 INFO:tasks.workunit.client.1.smithi150.stdout:7/243: dwrite d2/d5/d7/d8/f2d [0,4194304] 0 2024-03-22T10:42:48.594 INFO:tasks.workunit.client.1.smithi150.stdout:2/259: creat d12/d19/d2d/d3b/f51 x:0 0 0 2024-03-22T10:42:48.595 INFO:tasks.workunit.client.0.smithi115.stdout:3/630: creat d3/d82/d57/fd1 x:0 0 0 2024-03-22T10:42:48.597 INFO:tasks.workunit.client.0.smithi115.stdout:8/804: dwrite d6/dd/f70 [0,4194304] 0 2024-03-22T10:42:48.599 INFO:tasks.workunit.client.0.smithi115.stdout:8/805: dread - d6/dd/d12/fea zero size 2024-03-22T10:42:48.599 INFO:tasks.workunit.client.1.smithi150.stdout:0/263: write d0/d1/f39 [571339,85109] 0 2024-03-22T10:42:48.603 INFO:tasks.workunit.client.0.smithi115.stdout:8/806: mknod d6/d49/d68/c10c 0 2024-03-22T10:42:48.606 INFO:tasks.workunit.client.0.smithi115.stdout:3/631: dread d3/d19/f1d [0,4194304] 0 2024-03-22T10:42:48.609 INFO:tasks.workunit.client.1.smithi150.stdout:8/264: rename d0/d2/d19/d29/d31/c35 to d0/d2/d19/d29/d2f/df/d1b/d3d/c5c 0 2024-03-22T10:42:48.609 INFO:tasks.workunit.client.1.smithi150.stdout:3/256: dread d5/da/f28 [0,4194304] 0 2024-03-22T10:42:48.610 INFO:tasks.workunit.client.0.smithi115.stdout:8/807: rename d6/c30 to d6/dd/d12/de3/c10d 0 2024-03-22T10:42:48.610 INFO:tasks.workunit.client.0.smithi115.stdout:3/632: write d3/d16/d29/d9b/fb9 [1030884,63698] 0 2024-03-22T10:42:48.612 INFO:tasks.workunit.client.1.smithi150.stdout:1/225: getdents d1/da/d2a 0 2024-03-22T10:42:48.613 INFO:tasks.workunit.client.1.smithi150.stdout:9/271: dread d1/d8/de/df/f18 [4194304,4194304] 0 2024-03-22T10:42:48.614 INFO:tasks.workunit.client.1.smithi150.stdout:2/260: mknod d12/d19/d2d/d3b/d3c/c52 0 2024-03-22T10:42:48.620 INFO:tasks.workunit.client.1.smithi150.stdout:7/244: rename d2/d5/d7/d9/f18 to d2/d5/d7/d2e/d54/f55 0 2024-03-22T10:42:48.621 INFO:tasks.workunit.client.1.smithi150.stdout:5/167: rmdir d0/d8/dc/d2f 0 2024-03-22T10:42:48.624 INFO:tasks.workunit.client.1.smithi150.stdout:8/265: write d0/d2/d19/d29/d2f/df/d1b/f1d [749323,48441] 0 2024-03-22T10:42:48.626 INFO:tasks.workunit.client.1.smithi150.stdout:9/272: mkdir d1/d8/de/df/d36/d40/d6c 0 2024-03-22T10:42:48.628 INFO:tasks.workunit.client.1.smithi150.stdout:8/266: readlink d0/d2/d19/d29/l43 0 2024-03-22T10:42:48.633 INFO:tasks.workunit.client.1.smithi150.stdout:3/257: creat d5/f4e x:0 0 0 2024-03-22T10:42:48.634 INFO:tasks.workunit.client.1.smithi150.stdout:1/226: dwrite d1/da/d15/f3e [0,4194304] 0 2024-03-22T10:42:48.635 INFO:tasks.workunit.client.1.smithi150.stdout:0/264: rename d0/d12/l14 to d0/l59 0 2024-03-22T10:42:48.637 INFO:tasks.workunit.client.1.smithi150.stdout:7/245: rmdir d2/d5/d7/d8/d51 0 2024-03-22T10:42:48.640 INFO:tasks.workunit.client.1.smithi150.stdout:3/258: chown d5/da/c1a 31607 1 2024-03-22T10:42:48.642 INFO:tasks.workunit.client.1.smithi150.stdout:4/239: sync 2024-03-22T10:42:48.642 INFO:tasks.workunit.client.1.smithi150.stdout:0/265: chown d0/d12/d4f/d42/d10/d38/f46 124 1 2024-03-22T10:42:48.644 INFO:tasks.workunit.client.1.smithi150.stdout:9/273: symlink d1/d8/de/df/d24/d58/l6d 0 2024-03-22T10:42:48.646 INFO:tasks.workunit.client.1.smithi150.stdout:2/261: dwrite d12/d16/f43 [0,4194304] 0 2024-03-22T10:42:48.646 INFO:tasks.workunit.client.1.smithi150.stdout:9/274: write d1/d8/de/df/f4d [168728,21944] 0 2024-03-22T10:42:48.654 INFO:tasks.workunit.client.1.smithi150.stdout:2/262: fdatasync d12/d48/f4d 0 2024-03-22T10:42:48.654 INFO:tasks.workunit.client.1.smithi150.stdout:3/259: chown d5/f35 4227852 1 2024-03-22T10:42:48.654 INFO:tasks.workunit.client.1.smithi150.stdout:7/246: mknod d2/d5/d7/d2e/c56 0 2024-03-22T10:42:48.672 INFO:tasks.workunit.client.0.smithi115.stdout:4/623: write d1/d66/f70 [525036,112645] 0 2024-03-22T10:42:48.675 INFO:tasks.workunit.client.0.smithi115.stdout:3/633: rename d3/d16/f69 to d3/fd2 0 2024-03-22T10:42:48.676 INFO:tasks.workunit.client.0.smithi115.stdout:4/624: write d1/d66/f6d [4369241,46940] 0 2024-03-22T10:42:48.680 INFO:tasks.workunit.client.0.smithi115.stdout:8/808: rename d6/dd/l82 to d6/d49/d68/deb/l10e 0 2024-03-22T10:42:48.683 INFO:tasks.workunit.client.0.smithi115.stdout:8/809: fsync d6/d49/d68/deb/ff6 0 2024-03-22T10:42:48.683 INFO:tasks.workunit.client.0.smithi115.stdout:3/634: rename d3/d16/d29/d35/lc8 to d3/d82/ld3 0 2024-03-22T10:42:48.685 INFO:tasks.workunit.client.0.smithi115.stdout:1/708: dread d0/d12/d17/f33 [0,4194304] 0 2024-03-22T10:42:48.692 INFO:tasks.workunit.client.0.smithi115.stdout:3/635: rename d3/d82/d57/c86 to d3/d16/d29/d35/cd4 0 2024-03-22T10:42:48.692 INFO:tasks.workunit.client.0.smithi115.stdout:8/810: link d6/dd/d24/d3a/lf7 d6/d49/d68/l10f 0 2024-03-22T10:42:48.694 INFO:tasks.workunit.client.0.smithi115.stdout:1/709: symlink d0/d12/d17/d67/dd8/de0/lf1 0 2024-03-22T10:42:48.695 INFO:tasks.workunit.client.0.smithi115.stdout:4/625: dwrite d1/d16/d68/f6c [0,4194304] 0 2024-03-22T10:42:48.700 INFO:tasks.workunit.client.0.smithi115.stdout:3/636: rename d3/d16/d29/f43 to d3/d36/d41/fd5 0 2024-03-22T10:42:48.700 INFO:tasks.workunit.client.0.smithi115.stdout:8/811: mknod d6/dd/d24/d4a/d7b/d8f/dd6/ddb/c110 0 2024-03-22T10:42:48.708 INFO:tasks.workunit.client.0.smithi115.stdout:4/626: link d1/d66/d82/fa6 d1/d30/d4c/d86/d9f/d75/d99/fd5 0 2024-03-22T10:42:48.709 INFO:tasks.workunit.client.0.smithi115.stdout:8/812: rmdir d6/dd/d24/d4a/d7b/d8f 39 2024-03-22T10:42:48.710 INFO:tasks.workunit.client.0.smithi115.stdout:1/710: write d0/d68/f7f [290284,54306] 0 2024-03-22T10:42:48.712 INFO:tasks.workunit.client.0.smithi115.stdout:4/627: chown d1/d30/d4c/d86/d9f/d75/d99/d98/lb5 4 1 2024-03-22T10:42:48.712 INFO:tasks.workunit.client.0.smithi115.stdout:3/637: creat d3/d36/fd6 x:0 0 0 2024-03-22T10:42:48.714 INFO:tasks.workunit.client.0.smithi115.stdout:8/813: chown d6/dd/d12/ld7 172149676 1 2024-03-22T10:42:48.714 INFO:tasks.workunit.client.0.smithi115.stdout:1/711: creat d0/d12/d7b/ff2 x:0 0 0 2024-03-22T10:42:48.715 INFO:tasks.workunit.client.0.smithi115.stdout:3/638: symlink d3/d16/d29/ld7 0 2024-03-22T10:42:48.716 INFO:tasks.workunit.client.0.smithi115.stdout:1/712: chown d0/d12/d2b/fca 0 1 2024-03-22T10:42:48.717 INFO:tasks.workunit.client.0.smithi115.stdout:4/628: creat d1/d30/d4c/d86/d9f/fd6 x:0 0 0 2024-03-22T10:42:48.718 INFO:tasks.workunit.client.0.smithi115.stdout:1/713: readlink d0/d8/d4d/d56/db6/ddc/l9b 0 2024-03-22T10:42:48.720 INFO:tasks.workunit.client.0.smithi115.stdout:3/639: rename d3/d16/d29/d9b to d3/d19/dd8 0 2024-03-22T10:42:48.722 INFO:tasks.workunit.client.0.smithi115.stdout:4/629: rmdir d1/d16/d35 39 2024-03-22T10:42:48.725 INFO:tasks.workunit.client.0.smithi115.stdout:1/714: mkdir d0/d68/d81/df3 0 2024-03-22T10:42:48.725 INFO:tasks.workunit.client.0.smithi115.stdout:4/630: read d1/d30/d4c/d86/d9f/d75/d99/fd5 [2418166,74283] 0 2024-03-22T10:42:48.726 INFO:tasks.workunit.client.1.smithi150.stdout:4/240: write d0/d1b/f37 [1292190,119793] 0 2024-03-22T10:42:48.726 INFO:tasks.workunit.client.0.smithi115.stdout:3/640: truncate d3/d19/d61/f79 1024128 0 2024-03-22T10:42:48.727 INFO:tasks.workunit.client.1.smithi150.stdout:3/260: dread - d5/d22/d37/f3d zero size 2024-03-22T10:42:48.727 INFO:tasks.workunit.client.1.smithi150.stdout:8/267: write d0/fb [180232,7228] 0 2024-03-22T10:42:48.727 INFO:tasks.workunit.client.1.smithi150.stdout:5/168: write d0/f6 [2462943,59338] 0 2024-03-22T10:42:48.730 INFO:tasks.workunit.client.1.smithi150.stdout:1/227: dwrite d1/da/d15/d18/f38 [0,4194304] 0 2024-03-22T10:42:48.731 INFO:tasks.workunit.client.1.smithi150.stdout:7/247: read d2/d5/d7/d2e/d4a/f39 [1015363,107298] 0 2024-03-22T10:42:48.731 INFO:tasks.workunit.client.1.smithi150.stdout:2/263: creat d12/d19/d2d/f53 x:0 0 0 2024-03-22T10:42:48.731 INFO:tasks.workunit.client.0.smithi115.stdout:4/631: creat d1/d16/d28/fd7 x:0 0 0 2024-03-22T10:42:48.731 INFO:tasks.workunit.client.1.smithi150.stdout:9/275: creat d1/d8/de/df/d36/d40/d44/f6e x:0 0 0 2024-03-22T10:42:48.732 INFO:tasks.workunit.client.0.smithi115.stdout:3/641: read - d3/d36/d41/d85/fcc zero size 2024-03-22T10:42:48.734 INFO:tasks.workunit.client.1.smithi150.stdout:3/261: chown d5/d22/d2f/d38/f39 1 1 2024-03-22T10:42:48.734 INFO:tasks.workunit.client.1.smithi150.stdout:2/264: chown d12/d16/l18 3 1 2024-03-22T10:42:48.735 INFO:tasks.workunit.client.0.smithi115.stdout:8/814: write d6/dd/d24/f72 [440924,130223] 0 2024-03-22T10:42:48.735 INFO:tasks.workunit.client.0.smithi115.stdout:1/715: link d0/d12/c40 d0/d8/d4d/d56/dcf/cf4 0 2024-03-22T10:42:48.736 INFO:tasks.workunit.client.1.smithi150.stdout:4/241: creat d0/d2f/f50 x:0 0 0 2024-03-22T10:42:48.737 INFO:tasks.workunit.client.1.smithi150.stdout:5/169: unlink d0/f28 0 2024-03-22T10:42:48.738 INFO:tasks.workunit.client.0.smithi115.stdout:3/642: rename d3/d82/d52/c96 to d3/d36/d41/d33/cd9 0 2024-03-22T10:42:48.740 INFO:tasks.workunit.client.1.smithi150.stdout:8/268: mkdir d0/d5d 0 2024-03-22T10:42:48.741 INFO:tasks.workunit.client.1.smithi150.stdout:2/265: truncate d12/d21/f4f 479837 0 2024-03-22T10:42:48.741 INFO:tasks.workunit.client.0.smithi115.stdout:8/815: creat d6/d49/f111 x:0 0 0 2024-03-22T10:42:48.742 INFO:tasks.workunit.client.0.smithi115.stdout:3/643: dread d3/d19/d2b/d49/d91/fa3 [0,4194304] 0 2024-03-22T10:42:48.743 INFO:tasks.workunit.client.0.smithi115.stdout:8/816: fsync d6/dd/d12/de3/fe7 0 2024-03-22T10:42:48.744 INFO:tasks.workunit.client.1.smithi150.stdout:5/170: fdatasync d0/d8/dc/d1d/f24 0 2024-03-22T10:42:48.746 INFO:tasks.workunit.client.1.smithi150.stdout:7/248: rename d2/d5/d7/d2e to d2/d5/d7/d8/d57 0 2024-03-22T10:42:48.747 INFO:tasks.workunit.client.0.smithi115.stdout:8/817: mkdir d6/dd/d24/d112 0 2024-03-22T10:42:48.747 INFO:tasks.workunit.client.1.smithi150.stdout:4/242: creat d0/d32/f51 x:0 0 0 2024-03-22T10:42:48.748 INFO:tasks.workunit.client.0.smithi115.stdout:8/818: mknod d6/d49/d88/c113 0 2024-03-22T10:42:48.750 INFO:tasks.workunit.client.1.smithi150.stdout:2/266: mkdir d12/d48/d54 0 2024-03-22T10:42:48.752 INFO:tasks.workunit.client.0.smithi115.stdout:8/819: mknod d6/dd/d24/d4a/d7b/d8f/da0/c114 0 2024-03-22T10:42:48.754 INFO:tasks.workunit.client.1.smithi150.stdout:7/249: rmdir d2/d5/d7/d8/d57/d4a 39 2024-03-22T10:42:48.755 INFO:tasks.workunit.client.1.smithi150.stdout:3/262: creat d5/f4f x:0 0 0 2024-03-22T10:42:48.755 INFO:tasks.workunit.client.0.smithi115.stdout:8/820: mknod d6/dd/d24/d4a/d7b/d8f/dd6/ddb/c115 0 2024-03-22T10:42:48.756 INFO:tasks.workunit.client.1.smithi150.stdout:4/243: write d0/d1b/f21 [3935985,722] 0 2024-03-22T10:42:48.758 INFO:tasks.workunit.client.1.smithi150.stdout:1/228: dwrite d1/da/d15/d18/f3d [0,4194304] 0 2024-03-22T10:42:48.759 INFO:tasks.workunit.client.0.smithi115.stdout:3/644: dwrite d3/d19/d2b/d49/d91/fa3 [0,4194304] 0 2024-03-22T10:42:48.759 INFO:tasks.workunit.client.1.smithi150.stdout:2/267: getdents d12/d48/d54 0 2024-03-22T10:42:48.762 INFO:tasks.workunit.client.0.smithi115.stdout:3/645: mkdir d3/d19/dda 0 2024-03-22T10:42:48.762 INFO:tasks.workunit.client.0.smithi115.stdout:8/821: read d6/dd/d24/d3a/f3c [2081799,38874] 0 2024-03-22T10:42:48.763 INFO:tasks.workunit.client.1.smithi150.stdout:4/244: creat d0/d2f/f52 x:0 0 0 2024-03-22T10:42:48.769 INFO:tasks.workunit.client.1.smithi150.stdout:9/276: write d1/d8/de/df/d24/f6b [2677679,91577] 0 2024-03-22T10:42:48.770 INFO:tasks.workunit.client.1.smithi150.stdout:7/250: rename d2/d5/d7/d8/f42 to d2/d5/d7/d8/d46/d49/f58 0 2024-03-22T10:42:48.775 INFO:tasks.workunit.client.1.smithi150.stdout:7/251: dread - d2/d5/f52 zero size 2024-03-22T10:42:48.778 INFO:tasks.workunit.client.1.smithi150.stdout:9/277: mknod d1/d8/de/df/d20/d29/c6f 0 2024-03-22T10:42:48.778 INFO:tasks.workunit.client.1.smithi150.stdout:6/182: sync 2024-03-22T10:42:48.783 INFO:tasks.workunit.client.1.smithi150.stdout:2/268: creat d12/f55 x:0 0 0 2024-03-22T10:42:48.785 INFO:tasks.workunit.client.1.smithi150.stdout:8/269: dwrite d0/d2/f3f [4194304,4194304] 0 2024-03-22T10:42:48.789 INFO:tasks.workunit.client.1.smithi150.stdout:9/278: chown d1/d8/de/df/d36/d40/c46 1 1 2024-03-22T10:42:48.789 INFO:tasks.workunit.client.1.smithi150.stdout:6/183: rename db/f2a to db/dd/d15/f46 0 2024-03-22T10:42:48.794 INFO:tasks.workunit.client.1.smithi150.stdout:1/229: dwrite d1/da/d15/d18/f3d [0,4194304] 0 2024-03-22T10:42:48.794 INFO:tasks.workunit.client.1.smithi150.stdout:8/270: stat d0/d2/d19/d29/d2e/c55 0 2024-03-22T10:42:48.795 INFO:tasks.workunit.client.1.smithi150.stdout:7/252: write d2/d5/d7/d9/f28 [894045,21970] 0 2024-03-22T10:42:48.795 INFO:tasks.workunit.client.1.smithi150.stdout:2/269: mknod d12/d19/d2d/d3b/c56 0 2024-03-22T10:42:48.801 INFO:tasks.workunit.client.1.smithi150.stdout:5/171: dwrite d0/d8/dc/de/f15 [0,4194304] 0 2024-03-22T10:42:48.810 INFO:tasks.workunit.client.1.smithi150.stdout:6/184: mknod db/dd/d15/c47 0 2024-03-22T10:42:48.812 INFO:tasks.workunit.client.1.smithi150.stdout:1/230: creat d1/d19/d1a/f44 x:0 0 0 2024-03-22T10:42:48.819 INFO:tasks.workunit.client.1.smithi150.stdout:5/172: symlink d0/l30 0 2024-03-22T10:42:48.825 INFO:tasks.workunit.client.1.smithi150.stdout:7/253: dwrite d2/d5/d7/f50 [0,4194304] 0 2024-03-22T10:42:48.831 INFO:tasks.workunit.client.1.smithi150.stdout:4/245: dread d0/f14 [0,4194304] 0 2024-03-22T10:42:48.831 INFO:tasks.workunit.client.1.smithi150.stdout:1/231: symlink d1/d19/l45 0 2024-03-22T10:42:48.837 INFO:tasks.workunit.client.1.smithi150.stdout:8/271: link d0/d2/d19/d29/d2f/df/d1b/l32 d0/d2/d4d/l5e 0 2024-03-22T10:42:48.842 INFO:tasks.workunit.client.1.smithi150.stdout:2/270: truncate d12/d16/f43 4260668 0 2024-03-22T10:42:48.843 INFO:tasks.workunit.client.0.smithi115.stdout:8/822: creat d6/dd/d12/dc6/f116 x:0 0 0 2024-03-22T10:42:48.846 INFO:tasks.workunit.client.0.smithi115.stdout:8/823: write d6/fc [1746249,110725] 0 2024-03-22T10:42:48.848 INFO:tasks.workunit.client.0.smithi115.stdout:1/716: dwrite d0/d12/d17/d37/db5/fd9 [0,4194304] 0 2024-03-22T10:42:48.849 INFO:tasks.workunit.client.0.smithi115.stdout:4/632: dread d1/d30/d4c/d86/d9f/d32/f60 [0,4194304] 0 2024-03-22T10:42:48.854 INFO:tasks.workunit.client.1.smithi150.stdout:6/185: write db/dd/f37 [424189,121154] 0 2024-03-22T10:42:48.854 INFO:tasks.workunit.client.1.smithi150.stdout:2/271: creat d12/d16/d2c/f57 x:0 0 0 2024-03-22T10:42:48.856 INFO:tasks.workunit.client.1.smithi150.stdout:6/186: chown db/dd/d15 1777749 1 2024-03-22T10:42:48.862 INFO:tasks.workunit.client.1.smithi150.stdout:1/232: rename d1/d19/d1a/f44 to d1/d19/d1a/f46 0 2024-03-22T10:42:48.865 INFO:tasks.workunit.client.1.smithi150.stdout:2/272: creat d12/d19/f58 x:0 0 0 2024-03-22T10:42:48.868 INFO:tasks.workunit.client.1.smithi150.stdout:6/187: unlink db/dd/d15/c1f 0 2024-03-22T10:42:48.870 INFO:tasks.workunit.client.1.smithi150.stdout:9/279: dread d1/d8/de/df/d20/d32/f4f [0,4194304] 0 2024-03-22T10:42:48.871 INFO:tasks.workunit.client.1.smithi150.stdout:2/273: creat d12/d19/f59 x:0 0 0 2024-03-22T10:42:48.875 INFO:tasks.workunit.client.1.smithi150.stdout:6/188: dread - db/dd/d15/d30/f43 zero size 2024-03-22T10:42:48.877 INFO:tasks.workunit.client.1.smithi150.stdout:1/233: truncate d1/da/d2a/f35 972437 0 2024-03-22T10:42:48.880 INFO:tasks.workunit.client.1.smithi150.stdout:2/274: read d12/d21/f4f [42176,98189] 0 2024-03-22T10:42:48.882 INFO:tasks.workunit.client.1.smithi150.stdout:0/266: sync 2024-03-22T10:42:48.883 INFO:tasks.workunit.client.0.smithi115.stdout:3/646: mkdir d3/d19/dd8/ddb 0 2024-03-22T10:42:48.887 INFO:tasks.workunit.client.1.smithi150.stdout:5/173: dread d0/d8/dc/de/f13 [0,4194304] 0 2024-03-22T10:42:48.888 INFO:tasks.workunit.client.1.smithi150.stdout:1/234: mknod d1/d19/d1a/c47 0 2024-03-22T10:42:48.888 INFO:tasks.workunit.client.0.smithi115.stdout:8/824: fsync d6/dd/fe4 0 2024-03-22T10:42:48.889 INFO:tasks.workunit.client.0.smithi115.stdout:1/717: symlink d0/d12/d17/d67/dd8/lf5 0 2024-03-22T10:42:48.890 INFO:tasks.workunit.client.1.smithi150.stdout:0/267: readlink d0/d12/d4f/d42/d10/d38/d52/l55 0 2024-03-22T10:42:48.890 INFO:tasks.workunit.client.0.smithi115.stdout:8/825: write d6/f16 [3020935,106602] 0 2024-03-22T10:42:48.891 INFO:tasks.workunit.client.1.smithi150.stdout:8/272: dwrite d0/d3c/f44 [0,4194304] 0 2024-03-22T10:42:48.892 INFO:tasks.workunit.client.1.smithi150.stdout:9/280: dread d1/d8/de/df/d36/d40/d44/f6a [0,4194304] 0 2024-03-22T10:42:48.893 INFO:tasks.workunit.client.0.smithi115.stdout:1/718: creat d0/d68/ff6 x:0 0 0 2024-03-22T10:42:48.894 INFO:tasks.workunit.client.1.smithi150.stdout:7/254: dwrite d2/d5/d7/d8/d57/d4a/f22 [0,4194304] 0 2024-03-22T10:42:48.895 INFO:tasks.workunit.client.1.smithi150.stdout:2/275: creat d12/d19/d2d/d3b/f5a x:0 0 0 2024-03-22T10:42:48.897 INFO:tasks.workunit.client.1.smithi150.stdout:4/246: dread d0/f7 [0,4194304] 0 2024-03-22T10:42:48.898 INFO:tasks.workunit.client.0.smithi115.stdout:1/719: symlink d0/d8/db/d8e/lf7 0 2024-03-22T10:42:48.899 INFO:tasks.workunit.client.0.smithi115.stdout:8/826: truncate d6/dd/d24/fa7 774758 0 2024-03-22T10:42:48.901 INFO:tasks.workunit.client.0.smithi115.stdout:1/720: fdatasync d0/d22/f86 0 2024-03-22T10:42:48.904 INFO:tasks.workunit.client.0.smithi115.stdout:8/827: mknod d6/dd/d24/d112/c117 0 2024-03-22T10:42:48.905 INFO:tasks.workunit.client.1.smithi150.stdout:0/268: mkdir d0/d12/d4f/d5a 0 2024-03-22T10:42:48.906 INFO:tasks.workunit.client.1.smithi150.stdout:3/263: sync 2024-03-22T10:42:48.909 INFO:tasks.workunit.client.1.smithi150.stdout:6/189: rename db/dd/d15/l22 to db/dd/d15/l48 0 2024-03-22T10:42:48.911 INFO:tasks.workunit.client.1.smithi150.stdout:8/273: truncate d0/f23 119541 0 2024-03-22T10:42:48.912 INFO:tasks.workunit.client.0.smithi115.stdout:8/828: getdents d6/dd/d12/dc6 0 2024-03-22T10:42:48.918 INFO:tasks.workunit.client.1.smithi150.stdout:9/281: creat d1/d8/de/df/d24/d58/f70 x:0 0 0 2024-03-22T10:42:48.919 INFO:tasks.workunit.client.1.smithi150.stdout:0/269: fsync d0/d12/d4f/f28 0 2024-03-22T10:42:48.920 INFO:tasks.workunit.client.0.smithi115.stdout:8/829: symlink d6/dd/d12/dc6/dfd/l118 0 2024-03-22T10:42:48.921 INFO:tasks.workunit.client.1.smithi150.stdout:6/190: readlink db/l1d 0 2024-03-22T10:42:48.923 INFO:tasks.workunit.client.1.smithi150.stdout:2/276: fsync d12/d16/f43 0 2024-03-22T10:42:48.923 INFO:tasks.workunit.client.0.smithi115.stdout:3/647: dread d3/d36/f44 [0,4194304] 0 2024-03-22T10:42:48.924 INFO:tasks.workunit.client.1.smithi150.stdout:6/191: dread - db/dd/d15/d28/f41 zero size 2024-03-22T10:42:48.926 INFO:tasks.workunit.client.1.smithi150.stdout:6/192: write db/ff [1877418,20465] 0 2024-03-22T10:42:48.927 INFO:tasks.workunit.client.1.smithi150.stdout:5/174: mkdir d0/d8/dc/d2c/d31 0 2024-03-22T10:42:48.929 INFO:tasks.workunit.client.1.smithi150.stdout:4/247: creat d0/d1b/d4d/f53 x:0 0 0 2024-03-22T10:42:48.934 INFO:tasks.workunit.client.1.smithi150.stdout:4/248: chown d0/d1b/c43 12365292 1 2024-03-22T10:42:48.938 INFO:tasks.workunit.client.1.smithi150.stdout:1/235: rename d1/d19/c41 to d1/da/c48 0 2024-03-22T10:42:48.940 INFO:tasks.workunit.client.0.smithi115.stdout:8/830: dwrite d6/dd/d24/d4a/d7b/d8f/dd6/fda [0,4194304] 0 2024-03-22T10:42:48.941 INFO:tasks.workunit.client.1.smithi150.stdout:1/236: stat d1/d19/d1a/f46 0 2024-03-22T10:42:48.942 INFO:tasks.workunit.client.0.smithi115.stdout:8/831: write d6/dd/d24/d3a/fcd [4238186,76960] 0 2024-03-22T10:42:48.943 INFO:tasks.workunit.client.1.smithi150.stdout:6/193: symlink db/dd/d15/l49 0 2024-03-22T10:42:48.945 INFO:tasks.workunit.client.1.smithi150.stdout:4/249: rmdir d0/de 39 2024-03-22T10:42:48.946 INFO:tasks.workunit.client.1.smithi150.stdout:0/270: dread d0/d12/d4f/ff [0,4194304] 0 2024-03-22T10:42:48.947 INFO:tasks.workunit.client.0.smithi115.stdout:8/832: link d6/dd/d12/d31/f5c d6/d49/d88/f119 0 2024-03-22T10:42:48.948 INFO:tasks.workunit.client.0.smithi115.stdout:8/833: readlink d6/dd/d12/d31/lb9 0 2024-03-22T10:42:48.949 INFO:tasks.workunit.client.1.smithi150.stdout:2/277: dread d12/d48/f4c [0,4194304] 0 2024-03-22T10:42:48.950 INFO:tasks.workunit.client.0.smithi115.stdout:8/834: write d6/dd/f76 [2702762,16235] 0 2024-03-22T10:42:48.951 INFO:tasks.workunit.client.0.smithi115.stdout:8/835: chown d6/dd/d24/d4a/f87 84884 1 2024-03-22T10:42:48.953 INFO:tasks.workunit.client.0.smithi115.stdout:8/836: fdatasync d6/dd/d24/d4a/d7b/fe0 0 2024-03-22T10:42:48.954 INFO:tasks.workunit.client.1.smithi150.stdout:7/255: dwrite d2/d5/d7/d8/d57/d4a/f39 [4194304,4194304] 0 2024-03-22T10:42:48.960 INFO:tasks.workunit.client.1.smithi150.stdout:4/250: write d0/de/f3c [834566,2532] 0 2024-03-22T10:42:48.961 INFO:tasks.workunit.client.1.smithi150.stdout:3/264: rename d5/l9 to d5/d22/d37/l50 0 2024-03-22T10:42:48.967 INFO:tasks.workunit.client.1.smithi150.stdout:5/175: dwrite d0/d8/fd [0,4194304] 0 2024-03-22T10:42:48.968 INFO:tasks.workunit.client.0.smithi115.stdout:8/837: dwrite d6/dd/d24/d3a/f107 [0,4194304] 0 2024-03-22T10:42:48.972 INFO:tasks.workunit.client.1.smithi150.stdout:0/271: mknod d0/d12/d4f/d42/d10/d38/d52/c5b 0 2024-03-22T10:42:48.986 INFO:tasks.workunit.client.1.smithi150.stdout:8/274: truncate d0/d2/d19/d29/d2f/f3b 2296124 0 2024-03-22T10:42:48.988 INFO:tasks.workunit.client.0.smithi115.stdout:8/838: dwrite d6/d49/f83 [4194304,4194304] 0 2024-03-22T10:42:48.989 INFO:tasks.workunit.client.1.smithi150.stdout:6/194: creat db/f4a x:0 0 0 2024-03-22T10:42:48.995 INFO:tasks.workunit.client.1.smithi150.stdout:6/195: write db/dd/d15/d28/f41 [133848,63403] 0 2024-03-22T10:42:48.995 INFO:tasks.workunit.client.1.smithi150.stdout:5/176: symlink d0/l32 0 2024-03-22T10:42:48.996 INFO:tasks.workunit.client.1.smithi150.stdout:7/256: dwrite d2/d5/d7/d8/d57/f3c [0,4194304] 0 2024-03-22T10:42:48.999 INFO:tasks.workunit.client.1.smithi150.stdout:8/275: read d0/d2/fe [4191824,9063] 0 2024-03-22T10:42:48.999 INFO:tasks.workunit.client.1.smithi150.stdout:1/237: dread d1/f3c [0,4194304] 0 2024-03-22T10:42:49.006 INFO:tasks.workunit.client.1.smithi150.stdout:0/272: rename d0/d12/d4f/d42/d10/l34 to d0/d12/d4f/d42/d10/d1e/d35/l5c 0 2024-03-22T10:42:49.007 INFO:tasks.workunit.client.1.smithi150.stdout:7/257: chown d2/d5/d7/f2b 7 1 2024-03-22T10:42:49.009 INFO:tasks.workunit.client.1.smithi150.stdout:9/282: dread d1/d8/de/df/d24/f53 [4194304,4194304] 0 2024-03-22T10:42:49.010 INFO:tasks.workunit.client.1.smithi150.stdout:6/196: write db/dd/f2c [607141,120232] 0 2024-03-22T10:42:49.011 INFO:tasks.workunit.client.1.smithi150.stdout:5/177: mkdir d0/d33 0 2024-03-22T10:42:49.013 INFO:tasks.workunit.client.1.smithi150.stdout:0/273: chown d0/d12/d4f/d42/fc 969888 1 2024-03-22T10:42:49.014 INFO:tasks.workunit.client.1.smithi150.stdout:3/265: dwrite d5/da/f45 [0,4194304] 0 2024-03-22T10:42:49.017 INFO:tasks.workunit.client.1.smithi150.stdout:1/238: creat d1/d19/d1a/f49 x:0 0 0 2024-03-22T10:42:49.018 INFO:tasks.workunit.client.0.smithi115.stdout:0/578: sync 2024-03-22T10:42:49.022 INFO:tasks.workunit.client.0.smithi115.stdout:0/579: link d3/f46 d3/da/d4c/d69/fbd 0 2024-03-22T10:42:49.026 INFO:tasks.workunit.client.0.smithi115.stdout:0/580: getdents d3/da/d4c/da3/da0 0 2024-03-22T10:42:49.027 INFO:tasks.workunit.client.1.smithi150.stdout:8/276: rename d0/fd to d0/d2/f5f 0 2024-03-22T10:42:49.027 INFO:tasks.workunit.client.0.smithi115.stdout:0/581: write d3/f1a [295610,74347] 0 2024-03-22T10:42:49.034 INFO:tasks.workunit.client.0.smithi115.stdout:0/582: creat d3/d7/d1c/d6c/d9a/fbe x:0 0 0 2024-03-22T10:42:49.036 INFO:tasks.workunit.client.1.smithi150.stdout:9/283: write d1/d8/de/df/f18 [6627545,60201] 0 2024-03-22T10:42:49.036 INFO:tasks.workunit.client.1.smithi150.stdout:6/197: creat db/dd/d15/d30/f4b x:0 0 0 2024-03-22T10:42:49.038 INFO:tasks.workunit.client.1.smithi150.stdout:0/274: creat d0/d12/d4f/d11/f5d x:0 0 0 2024-03-22T10:42:49.040 INFO:tasks.workunit.client.0.smithi115.stdout:8/839: creat d6/dd/d12/f11a x:0 0 0 2024-03-22T10:42:49.042 INFO:tasks.workunit.client.0.smithi115.stdout:0/583: creat d3/d7/d1c/d6c/da6/fbf x:0 0 0 2024-03-22T10:42:49.042 INFO:tasks.workunit.client.0.smithi115.stdout:8/840: rmdir d6/dd/d24/dc1 39 2024-03-22T10:42:49.043 INFO:tasks.workunit.client.0.smithi115.stdout:0/584: stat d3/d7/d1b/d1f/lb5 0 2024-03-22T10:42:49.044 INFO:tasks.workunit.client.1.smithi150.stdout:6/198: write db/dd/d21/f32 [4800509,76873] 0 2024-03-22T10:42:49.045 INFO:tasks.workunit.client.1.smithi150.stdout:1/239: unlink d1/da/d15/d18/f3b 0 2024-03-22T10:42:49.047 INFO:tasks.workunit.client.0.smithi115.stdout:8/841: creat d6/d49/f11b x:0 0 0 2024-03-22T10:42:49.049 INFO:tasks.workunit.client.1.smithi150.stdout:4/251: write d0/f1a [1739338,36238] 0 2024-03-22T10:42:49.050 INFO:tasks.workunit.client.0.smithi115.stdout:8/842: truncate d6/dd/d24/d3a/f54 3876203 0 2024-03-22T10:42:49.053 INFO:tasks.workunit.client.0.smithi115.stdout:8/843: unlink d6/dd/d24/ff2 0 2024-03-22T10:42:49.054 INFO:tasks.workunit.client.1.smithi150.stdout:8/277: rename d0/d2/d19/d29/d2e/l48 to d0/d2/d19/d29/d2f/df/d1b/l60 0 2024-03-22T10:42:49.054 INFO:tasks.workunit.client.0.smithi115.stdout:8/844: dread - d6/dd/fe4 zero size 2024-03-22T10:42:49.057 INFO:tasks.workunit.client.0.smithi115.stdout:0/585: dwrite d3/d7/d1b/f92 [4194304,4194304] 0 2024-03-22T10:42:49.059 INFO:tasks.workunit.client.0.smithi115.stdout:0/586: symlink d3/lc0 0 2024-03-22T10:42:49.061 INFO:tasks.workunit.client.0.smithi115.stdout:0/587: mkdir d3/d7/d1b/d1f/d78/dc1 0 2024-03-22T10:42:49.067 INFO:tasks.workunit.client.1.smithi150.stdout:5/178: creat d0/d8/f34 x:0 0 0 2024-03-22T10:42:49.068 INFO:tasks.workunit.client.1.smithi150.stdout:6/199: read db/dd/d15/d28/f2b [734323,103391] 0 2024-03-22T10:42:49.069 INFO:tasks.workunit.client.1.smithi150.stdout:7/258: truncate d2/d5/d7/d8/d57/f3c 1042499 0 2024-03-22T10:42:49.070 INFO:tasks.workunit.client.1.smithi150.stdout:4/252: dwrite d0/de/f4b [0,4194304] 0 2024-03-22T10:42:49.072 INFO:tasks.workunit.client.1.smithi150.stdout:0/275: creat d0/d12/d4f/d42/d10/d53/f5e x:0 0 0 2024-03-22T10:42:49.072 INFO:tasks.workunit.client.1.smithi150.stdout:4/253: chown d0/c22 175140 1 2024-03-22T10:42:49.075 INFO:tasks.workunit.client.0.smithi115.stdout:0/588: dwrite d3/da/d4c/d69/d6a/fa4 [0,4194304] 0 2024-03-22T10:42:49.076 INFO:tasks.workunit.client.1.smithi150.stdout:0/276: chown d0/d12/c33 836 1 2024-03-22T10:42:49.077 INFO:tasks.workunit.client.1.smithi150.stdout:8/278: write d0/d3c/f56 [9325,122845] 0 2024-03-22T10:42:49.077 INFO:tasks.workunit.client.1.smithi150.stdout:1/240: link d1/d19/d1a/f49 d1/da/d15/d18/f4a 0 2024-03-22T10:42:49.077 INFO:tasks.workunit.client.1.smithi150.stdout:6/200: truncate db/dd/f33 598276 0 2024-03-22T10:42:49.082 INFO:tasks.workunit.client.1.smithi150.stdout:6/201: chown db/dd/d15 14 1 2024-03-22T10:42:49.083 INFO:tasks.workunit.client.1.smithi150.stdout:0/277: truncate d0/d12/d4f/f31 6174117 0 2024-03-22T10:42:49.083 INFO:tasks.workunit.client.1.smithi150.stdout:6/202: chown db 560746 1 2024-03-22T10:42:49.086 INFO:tasks.workunit.client.1.smithi150.stdout:9/284: dread d1/d8/de/df/f11 [0,4194304] 0 2024-03-22T10:42:49.087 INFO:tasks.workunit.client.1.smithi150.stdout:7/259: creat d2/d5/d7/d8/d57/f59 x:0 0 0 2024-03-22T10:42:49.087 INFO:tasks.workunit.client.1.smithi150.stdout:6/203: chown db/dd/d25/f3d 6 1 2024-03-22T10:42:49.087 INFO:tasks.workunit.client.1.smithi150.stdout:8/279: readlink d0/d2/d19/d29/l54 0 2024-03-22T10:42:49.091 INFO:tasks.workunit.client.0.smithi115.stdout:0/589: dwrite d3/fa9 [0,4194304] 0 2024-03-22T10:42:49.092 INFO:tasks.workunit.client.1.smithi150.stdout:9/285: stat d1/d49/l33 0 2024-03-22T10:42:49.092 INFO:tasks.workunit.client.0.smithi115.stdout:0/590: fdatasync d3/d7/d1b/f24 0 2024-03-22T10:42:49.093 INFO:tasks.workunit.client.1.smithi150.stdout:8/280: chown d0/d2/d19/d29/d2e/f3a 1 1 2024-03-22T10:42:49.094 INFO:tasks.workunit.client.1.smithi150.stdout:5/179: dread d0/f3 [0,4194304] 0 2024-03-22T10:42:49.095 INFO:tasks.workunit.client.0.smithi115.stdout:0/591: readlink d3/da/d4c/d69/l96 0 2024-03-22T10:42:49.097 INFO:tasks.workunit.client.1.smithi150.stdout:0/278: mkdir d0/d12/d4f/d42/d10/d1e/d35/d5f 0 2024-03-22T10:42:49.099 INFO:tasks.workunit.client.1.smithi150.stdout:4/254: dread d0/de/f18 [0,4194304] 0 2024-03-22T10:42:49.103 INFO:tasks.workunit.client.1.smithi150.stdout:8/281: mknod d0/d2/c61 0 2024-03-22T10:42:49.104 INFO:tasks.workunit.client.1.smithi150.stdout:5/180: unlink d0/d26/f29 0 2024-03-22T10:42:49.108 INFO:tasks.workunit.client.1.smithi150.stdout:0/279: read d0/d12/d4f/d42/d10/f13 [3225396,48210] 0 2024-03-22T10:42:49.114 INFO:tasks.workunit.client.1.smithi150.stdout:6/204: rename db/dd/d15/l39 to db/dd/d25/l4c 0 2024-03-22T10:42:49.117 INFO:tasks.workunit.client.1.smithi150.stdout:7/260: dwrite d2/d5/d7/d8/d57/d4a/f33 [0,4194304] 0 2024-03-22T10:42:49.118 INFO:tasks.workunit.client.1.smithi150.stdout:1/241: dwrite d1/f7 [0,4194304] 0 2024-03-22T10:42:49.121 INFO:tasks.workunit.client.1.smithi150.stdout:5/181: unlink d0/d8/dc/de/c1b 0 2024-03-22T10:42:49.121 INFO:tasks.workunit.client.0.smithi115.stdout:0/592: dwrite d3/d4/f88 [0,4194304] 0 2024-03-22T10:42:49.122 INFO:tasks.workunit.client.1.smithi150.stdout:5/182: stat d0/f11 0 2024-03-22T10:42:49.122 INFO:tasks.workunit.client.1.smithi150.stdout:5/183: chown d0/d26 7614 1 2024-03-22T10:42:49.123 INFO:tasks.workunit.client.1.smithi150.stdout:1/242: chown d1/da/d15/f3e 130568015 1 2024-03-22T10:42:49.128 INFO:tasks.workunit.client.1.smithi150.stdout:0/280: truncate d0/d12/d4f/d11/f5d 470253 0 2024-03-22T10:42:49.128 INFO:tasks.workunit.client.1.smithi150.stdout:6/205: symlink db/dd/d15/l4d 0 2024-03-22T10:42:49.129 INFO:tasks.workunit.client.1.smithi150.stdout:8/282: mknod d0/d2/d19/c62 0 2024-03-22T10:42:49.131 INFO:tasks.workunit.client.1.smithi150.stdout:9/286: dwrite d1/d8/de/df/d36/d40/d41/f47 [0,4194304] 0 2024-03-22T10:42:49.133 INFO:tasks.workunit.client.1.smithi150.stdout:7/261: creat d2/d5/d7/d8/d46/f5a x:0 0 0 2024-03-22T10:42:49.134 INFO:tasks.workunit.client.1.smithi150.stdout:8/283: chown d0/d2/d4d 0 1 2024-03-22T10:42:49.135 INFO:tasks.workunit.client.1.smithi150.stdout:2/278: sync 2024-03-22T10:42:49.135 INFO:tasks.workunit.client.1.smithi150.stdout:3/266: sync 2024-03-22T10:42:49.136 INFO:tasks.workunit.client.0.smithi115.stdout:0/593: dwrite d3/d4/d6d/fb0 [0,4194304] 0 2024-03-22T10:42:49.137 INFO:tasks.workunit.client.1.smithi150.stdout:1/243: mkdir d1/da/d15/d18/d4b 0 2024-03-22T10:42:49.137 INFO:tasks.workunit.client.1.smithi150.stdout:8/284: readlink d0/d2/d19/l33 0 2024-03-22T10:42:49.141 INFO:tasks.workunit.client.0.smithi115.stdout:0/594: symlink d3/da/d4c/da3/da0/lc2 0 2024-03-22T10:42:49.144 INFO:tasks.workunit.client.1.smithi150.stdout:7/262: dread - d2/d5/d7/d8/d2c/f47 zero size 2024-03-22T10:42:49.147 INFO:tasks.workunit.client.1.smithi150.stdout:5/184: rename d0/d8/dc/c2d to d0/d8/dc/de/c35 0 2024-03-22T10:42:49.154 INFO:tasks.workunit.client.1.smithi150.stdout:8/285: truncate d0/d2/d19/d29/d2e/f58 200505 0 2024-03-22T10:42:49.155 INFO:tasks.workunit.client.1.smithi150.stdout:6/206: rename db/ce to db/c4e 0 2024-03-22T10:42:49.157 INFO:tasks.workunit.client.1.smithi150.stdout:3/267: write d5/fd [1011609,63072] 0 2024-03-22T10:42:49.159 INFO:tasks.workunit.client.1.smithi150.stdout:2/279: link d12/d19/f34 d12/d48/d54/f5b 0 2024-03-22T10:42:49.162 INFO:tasks.workunit.client.1.smithi150.stdout:2/280: chown d12/d16/l18 113 1 2024-03-22T10:42:49.162 INFO:tasks.workunit.client.1.smithi150.stdout:1/244: link d1/f7 d1/d19/f4c 0 2024-03-22T10:42:49.168 INFO:tasks.workunit.client.1.smithi150.stdout:0/281: rename d0/d12/d4f/d42/d10/d1e/d35/l49 to d0/d12/d4f/d42/l60 0 2024-03-22T10:42:49.169 INFO:tasks.workunit.client.1.smithi150.stdout:9/287: dwrite d1/d8/de/df/d24/f53 [4194304,4194304] 0 2024-03-22T10:42:49.170 INFO:tasks.workunit.client.1.smithi150.stdout:3/268: rename d5/d22/d2f/d38/c48 to d5/d22/d2f/d38/c51 0 2024-03-22T10:42:49.178 INFO:tasks.workunit.client.0.smithi115.stdout:6/569: sync 2024-03-22T10:42:49.179 INFO:tasks.workunit.client.1.smithi150.stdout:1/245: creat d1/d19/d1a/f4d x:0 0 0 2024-03-22T10:42:49.179 INFO:tasks.workunit.client.1.smithi150.stdout:5/185: dread d0/f5 [0,4194304] 0 2024-03-22T10:42:49.181 INFO:tasks.workunit.client.0.smithi115.stdout:6/570: write d4/d11/d12/d1c/fa5 [1154257,91186] 0 2024-03-22T10:42:49.185 INFO:tasks.workunit.client.0.smithi115.stdout:6/571: write d4/d11/d12/d1c/d8a/fa6 [565912,123025] 0 2024-03-22T10:42:49.185 INFO:tasks.workunit.client.1.smithi150.stdout:7/263: creat d2/d5/d7/d8/d57/f5b x:0 0 0 2024-03-22T10:42:49.187 INFO:tasks.workunit.client.1.smithi150.stdout:3/269: rmdir d5/d22/d37 39 2024-03-22T10:42:49.188 INFO:tasks.workunit.client.1.smithi150.stdout:8/286: rename d0/d2/f18 to d0/d2/d19/d29/f63 0 2024-03-22T10:42:49.191 INFO:tasks.workunit.client.1.smithi150.stdout:4/255: truncate d0/f7 1316449 0 2024-03-22T10:42:49.193 INFO:tasks.workunit.client.1.smithi150.stdout:1/246: mkdir d1/da/d2a/d4e 0 2024-03-22T10:42:49.194 INFO:tasks.workunit.client.1.smithi150.stdout:7/264: fsync d2/d5/d7/d8/d57/d4a/f22 0 2024-03-22T10:42:49.195 INFO:tasks.workunit.client.0.smithi115.stdout:6/572: write d4/d11/d12/f9c [1095296,37955] 0 2024-03-22T10:42:49.195 INFO:tasks.workunit.client.1.smithi150.stdout:6/207: dwrite db/dd/d15/d30/f4b [0,4194304] 0 2024-03-22T10:42:49.196 INFO:tasks.workunit.client.1.smithi150.stdout:2/281: rename d12/d48/c50 to d12/d19/d2d/d3b/c5c 0 2024-03-22T10:42:49.196 INFO:tasks.workunit.client.0.smithi115.stdout:6/573: stat d4/d11/d12/d1c/d8a/d98 0 2024-03-22T10:42:49.201 INFO:tasks.workunit.client.1.smithi150.stdout:1/247: chown d1/da/d15/f3e 7 1 2024-03-22T10:42:49.203 INFO:tasks.workunit.client.1.smithi150.stdout:5/186: dwrite d0/f6 [0,4194304] 0 2024-03-22T10:42:49.205 INFO:tasks.workunit.client.1.smithi150.stdout:9/288: creat d1/f71 x:0 0 0 2024-03-22T10:42:49.212 INFO:tasks.workunit.client.0.smithi115.stdout:6/574: dread d4/d11/d12/d1c/fa5 [0,4194304] 0 2024-03-22T10:42:49.219 INFO:tasks.workunit.client.0.smithi115.stdout:6/575: mkdir d4/d36/d48/d72/daa/db9 0 2024-03-22T10:42:49.220 INFO:tasks.workunit.client.1.smithi150.stdout:7/265: readlink d2/d5/d7/l48 0 2024-03-22T10:42:49.222 INFO:tasks.workunit.client.1.smithi150.stdout:5/187: dread d0/d8/f23 [0,4194304] 0 2024-03-22T10:42:49.222 INFO:tasks.workunit.client.0.smithi115.stdout:6/576: chown d4/d11/c15 8304652 1 2024-03-22T10:42:49.230 INFO:tasks.workunit.client.1.smithi150.stdout:9/289: dread d1/d8/de/df/f18 [4194304,4194304] 0 2024-03-22T10:42:49.230 INFO:tasks.workunit.client.1.smithi150.stdout:0/282: dread d0/d12/d4f/d42/d10/f43 [0,4194304] 0 2024-03-22T10:42:49.235 INFO:tasks.workunit.client.1.smithi150.stdout:3/270: creat d5/f52 x:0 0 0 2024-03-22T10:42:49.243 INFO:tasks.workunit.client.1.smithi150.stdout:3/271: fdatasync d5/da/f19 0 2024-03-22T10:42:49.248 INFO:tasks.workunit.client.1.smithi150.stdout:2/282: mknod d12/d48/d54/c5d 0 2024-03-22T10:42:49.248 INFO:tasks.workunit.client.1.smithi150.stdout:1/248: mkdir d1/da/d15/d4f 0 2024-03-22T10:42:49.248 INFO:tasks.workunit.client.1.smithi150.stdout:0/283: rename d0/d12/d4f/l29 to d0/d12/d4f/d42/d10/d38/d52/l61 0 2024-03-22T10:42:49.251 INFO:tasks.workunit.client.1.smithi150.stdout:7/266: dread d2/d5/d7/f4d [0,4194304] 0 2024-03-22T10:42:49.252 INFO:tasks.workunit.client.1.smithi150.stdout:6/208: creat db/f4f x:0 0 0 2024-03-22T10:42:49.253 INFO:tasks.workunit.client.1.smithi150.stdout:3/272: unlink d5/da/f28 0 2024-03-22T10:42:49.253 INFO:tasks.workunit.client.1.smithi150.stdout:0/284: readlink d0/l16 0 2024-03-22T10:42:49.255 INFO:tasks.workunit.client.0.smithi115.stdout:6/577: dread d4/d13/d33/f59 [0,4194304] 0 2024-03-22T10:42:49.258 INFO:tasks.workunit.client.1.smithi150.stdout:4/256: dwrite d0/de/f28 [0,4194304] 0 2024-03-22T10:42:49.259 INFO:tasks.workunit.client.1.smithi150.stdout:6/209: write db/f4f [447522,35504] 0 2024-03-22T10:42:49.261 INFO:tasks.workunit.client.1.smithi150.stdout:0/285: fdatasync d0/d12/d4f/f28 0 2024-03-22T10:42:49.262 INFO:tasks.workunit.client.1.smithi150.stdout:3/273: mkdir d5/d53 0 2024-03-22T10:42:49.262 INFO:tasks.workunit.client.1.smithi150.stdout:7/267: chown d2/d5/d7/d8/c44 13099422 1 2024-03-22T10:42:49.263 INFO:tasks.workunit.client.1.smithi150.stdout:5/188: dwrite d0/d8/dc/de/d18/f2a [0,4194304] 0 2024-03-22T10:42:49.263 INFO:tasks.workunit.client.1.smithi150.stdout:2/283: write d12/d19/d2d/f36 [7853910,52342] 0 2024-03-22T10:42:49.264 INFO:tasks.workunit.client.1.smithi150.stdout:4/257: chown d0/f1a 123772099 1 2024-03-22T10:42:49.269 INFO:tasks.workunit.client.1.smithi150.stdout:9/290: dwrite d1/d8/de/df/f18 [4194304,4194304] 0 2024-03-22T10:42:49.275 INFO:tasks.workunit.client.1.smithi150.stdout:7/268: symlink d2/d5/d7/d8/d2c/l5c 0 2024-03-22T10:42:49.276 INFO:tasks.workunit.client.1.smithi150.stdout:4/258: mkdir d0/d54 0 2024-03-22T10:42:49.277 INFO:tasks.workunit.client.1.smithi150.stdout:8/287: dwrite d0/d2/d19/d29/f63 [0,4194304] 0 2024-03-22T10:42:49.281 INFO:tasks.workunit.client.1.smithi150.stdout:2/284: write d12/d16/d2c/f57 [702834,118197] 0 2024-03-22T10:42:49.282 INFO:tasks.workunit.client.1.smithi150.stdout:2/285: fdatasync d12/d16/f2a 0 2024-03-22T10:42:49.282 INFO:tasks.workunit.client.1.smithi150.stdout:5/189: symlink d0/d8/dc/de/d18/l36 0 2024-03-22T10:42:49.283 INFO:tasks.workunit.client.1.smithi150.stdout:0/286: truncate d0/d12/d4f/d42/fc 29463 0 2024-03-22T10:42:49.285 INFO:tasks.workunit.client.1.smithi150.stdout:8/288: chown d0/d2/d19/d29/d2f/df/f26 43774 1 2024-03-22T10:42:49.286 INFO:tasks.workunit.client.1.smithi150.stdout:5/190: chown d0/d33 70 1 2024-03-22T10:42:49.286 INFO:tasks.workunit.client.1.smithi150.stdout:9/291: symlink d1/d8/de/df/d36/d40/d41/l72 0 2024-03-22T10:42:49.294 INFO:tasks.workunit.client.1.smithi150.stdout:4/259: readlink d0/d1b/l40 0 2024-03-22T10:42:49.294 INFO:tasks.workunit.client.1.smithi150.stdout:6/210: getdents db/dd/d15/d30 0 2024-03-22T10:42:49.295 INFO:tasks.workunit.client.1.smithi150.stdout:9/292: chown d1/d8/cd 24 1 2024-03-22T10:42:49.298 INFO:tasks.workunit.client.1.smithi150.stdout:0/287: creat d0/d12/d4f/d42/d10/d1e/d35/f62 x:0 0 0 2024-03-22T10:42:49.300 INFO:tasks.workunit.client.0.smithi115.stdout:6/578: dwrite d4/d36/d48/f9e [0,4194304] 0 2024-03-22T10:42:49.306 INFO:tasks.workunit.client.0.smithi115.stdout:6/579: creat d4/d13/d33/fba x:0 0 0 2024-03-22T10:42:49.306 INFO:tasks.workunit.client.1.smithi150.stdout:0/288: readlink d0/d1/l1b 0 2024-03-22T10:42:49.306 INFO:tasks.workunit.client.1.smithi150.stdout:5/191: getdents d0/d8/dc/d2c/d31 0 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:1/721: sync 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:4/633: sync 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:2/583: sync 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:0/595: sync 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:7/664: sync 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:5/654: sync 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:6/580: chown d4/d13/caf 941 1 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:3/648: sync 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:8/845: sync 2024-03-22T10:42:49.308 INFO:tasks.workunit.client.0.smithi115.stdout:9/656: sync 2024-03-22T10:42:49.309 INFO:tasks.workunit.client.1.smithi150.stdout:8/289: creat d0/d2/d19/d29/d2f/f64 x:0 0 0 2024-03-22T10:42:49.309 INFO:tasks.workunit.client.1.smithi150.stdout:5/192: fdatasync d0/f2b 0 2024-03-22T10:42:49.310 INFO:tasks.workunit.client.0.smithi115.stdout:0/596: stat d3/d4/d14/lae 0 2024-03-22T10:42:49.310 INFO:tasks.workunit.client.1.smithi150.stdout:7/269: dwrite d2/d5/d7/d9/f28 [0,4194304] 0 2024-03-22T10:42:49.312 INFO:tasks.workunit.client.1.smithi150.stdout:4/260: dread d0/f26 [0,4194304] 0 2024-03-22T10:42:49.312 INFO:tasks.workunit.client.0.smithi115.stdout:1/722: fdatasync d0/d8/db/d25/f44 0 2024-03-22T10:42:49.312 INFO:tasks.workunit.client.1.smithi150.stdout:0/289: write d0/d12/d4f/d42/f2b [25511,6739] 0 2024-03-22T10:42:49.314 INFO:tasks.workunit.client.0.smithi115.stdout:8/846: creat d6/dd/d24/d4a/dfc/f11c x:0 0 0 2024-03-22T10:42:49.315 INFO:tasks.workunit.client.0.smithi115.stdout:6/581: write d4/d11/d12/d1c/fa7 [253054,1066] 0 2024-03-22T10:42:49.315 INFO:tasks.workunit.client.0.smithi115.stdout:4/634: mkdir d1/dbb/dc6/dd8 0 2024-03-22T10:42:49.315 INFO:tasks.workunit.client.0.smithi115.stdout:2/584: write da/d1c/f88 [416421,30158] 0 2024-03-22T10:42:49.316 INFO:tasks.workunit.client.0.smithi115.stdout:1/723: fsync d0/d8/de/f26 0 2024-03-22T10:42:49.317 INFO:tasks.workunit.client.1.smithi150.stdout:7/270: fdatasync d2/d5/d7/d38/f4b 0 2024-03-22T10:42:49.319 INFO:tasks.workunit.client.0.smithi115.stdout:7/665: fdatasync d2/d3a/d6e/f59 0 2024-03-22T10:42:49.319 INFO:tasks.workunit.client.0.smithi115.stdout:6/582: fsync d4/d11/f9a 0 2024-03-22T10:42:49.319 INFO:tasks.workunit.client.0.smithi115.stdout:2/585: chown da/f15 908815707 1 2024-03-22T10:42:49.322 INFO:tasks.workunit.client.0.smithi115.stdout:9/657: unlink d6/d8/d3c/d59/d75/cc5 0 2024-03-22T10:42:49.325 INFO:tasks.workunit.client.0.smithi115.stdout:0/597: dread - d3/d7/d1b/d1f/f62 zero size 2024-03-22T10:42:49.326 INFO:tasks.workunit.client.1.smithi150.stdout:4/261: symlink d0/d42/l55 0 2024-03-22T10:42:49.327 INFO:tasks.workunit.client.0.smithi115.stdout:9/658: chown d6/d7/f78 936 1 2024-03-22T10:42:49.328 INFO:tasks.workunit.client.0.smithi115.stdout:5/655: write d2/d15/d1b/d7c/d84/d89/fb8 [3921867,94133] 0 2024-03-22T10:42:49.331 INFO:tasks.workunit.client.1.smithi150.stdout:2/286: dread d12/d19/f1d [0,4194304] 0 2024-03-22T10:42:49.332 INFO:tasks.workunit.client.0.smithi115.stdout:4/635: mkdir d1/d66/dd9 0 2024-03-22T10:42:49.332 INFO:tasks.workunit.client.0.smithi115.stdout:5/656: write d2/d15/d1b/d7c/d38/f61 [1873274,117011] 0 2024-03-22T10:42:49.334 INFO:tasks.workunit.client.0.smithi115.stdout:2/586: unlink da/d25/d3d/d57/d7d/f8b 0 2024-03-22T10:42:49.335 INFO:tasks.workunit.client.1.smithi150.stdout:3/274: rmdir d5/da 39 2024-03-22T10:42:49.335 INFO:tasks.workunit.client.1.smithi150.stdout:6/211: dwrite db/dd/d15/f46 [0,4194304] 0 2024-03-22T10:42:49.335 INFO:tasks.workunit.client.0.smithi115.stdout:5/657: dread - d2/d15/d1b/d7c/d38/d43/fce zero size 2024-03-22T10:42:49.336 INFO:tasks.workunit.client.0.smithi115.stdout:3/649: rmdir d3/d82/dc2 0 2024-03-22T10:42:49.340 INFO:tasks.workunit.client.0.smithi115.stdout:6/583: rename d4/fb6 to d4/d11/d12/d1c/d8a/d98/d9f/fbb 0 2024-03-22T10:42:49.345 INFO:tasks.workunit.client.1.smithi150.stdout:4/262: rmdir d0/d2f 39 2024-03-22T10:42:49.345 INFO:tasks.workunit.client.0.smithi115.stdout:8/847: unlink d6/dd/d24/d4a/l65 0 2024-03-22T10:42:49.346 INFO:tasks.workunit.client.1.smithi150.stdout:0/290: read d0/d12/d4f/f31 [5788907,38986] 0 2024-03-22T10:42:49.347 INFO:tasks.workunit.client.0.smithi115.stdout:9/659: unlink d6/d8/d3c/fd1 0 2024-03-22T10:42:49.349 INFO:tasks.workunit.client.1.smithi150.stdout:7/271: dwrite d2/d5/d7/d9/f28 [0,4194304] 0 2024-03-22T10:42:49.349 INFO:tasks.workunit.client.0.smithi115.stdout:8/848: read - d6/dd/d12/dc6/f116 zero size 2024-03-22T10:42:49.351 INFO:tasks.workunit.client.1.smithi150.stdout:0/291: chown d0/d12/d4f/d42/d10/d38/f4b 0 1 2024-03-22T10:42:49.351 INFO:tasks.workunit.client.1.smithi150.stdout:6/212: chown l8 8071 1 2024-03-22T10:42:49.356 INFO:tasks.workunit.client.0.smithi115.stdout:4/636: creat d1/d30/d4c/d86/d9f/d32/fda x:0 0 0 2024-03-22T10:42:49.356 INFO:tasks.workunit.client.1.smithi150.stdout:4/263: creat d0/d32/f56 x:0 0 0 2024-03-22T10:42:49.357 INFO:tasks.workunit.client.1.smithi150.stdout:6/213: mkdir db/dd/d21/d50 0 2024-03-22T10:42:49.357 INFO:tasks.workunit.client.1.smithi150.stdout:9/293: getdents d1/d8/de 0 2024-03-22T10:42:49.359 INFO:tasks.workunit.client.1.smithi150.stdout:7/272: unlink d2/d5/d7/d9/db/l11 0 2024-03-22T10:42:49.362 INFO:tasks.workunit.client.0.smithi115.stdout:8/849: creat d6/d49/d68/f11d x:0 0 0 2024-03-22T10:42:49.363 INFO:tasks.workunit.client.1.smithi150.stdout:2/287: write d12/d16/f43 [3859895,41765] 0 2024-03-22T10:42:49.364 INFO:tasks.workunit.client.1.smithi150.stdout:4/264: rename d0/d54 to d0/de/d57 0 2024-03-22T10:42:49.366 INFO:tasks.workunit.client.0.smithi115.stdout:2/587: getdents da/d7b/da4 0 2024-03-22T10:42:49.366 INFO:tasks.workunit.client.0.smithi115.stdout:5/658: dwrite d2/d15/d1b/d7c/d38/d43/d49/fb9 [0,4194304] 0 2024-03-22T10:42:49.366 INFO:tasks.workunit.client.0.smithi115.stdout:3/650: creat d3/d19/dd8/ddb/fdc x:0 0 0 2024-03-22T10:42:49.366 INFO:tasks.workunit.client.1.smithi150.stdout:9/294: mkdir d1/d8/de/d73 0 2024-03-22T10:42:49.366 INFO:tasks.workunit.client.1.smithi150.stdout:4/265: chown d0/d32/f4e 1 1 2024-03-22T10:42:49.369 INFO:tasks.workunit.client.1.smithi150.stdout:4/266: fsync d0/de/f4c 0 2024-03-22T10:42:49.369 INFO:tasks.workunit.client.1.smithi150.stdout:4/267: fsync d0/f1a 0 2024-03-22T10:42:49.370 INFO:tasks.workunit.client.0.smithi115.stdout:4/637: write d1/f57 [2961316,23588] 0 2024-03-22T10:42:49.370 INFO:tasks.workunit.client.0.smithi115.stdout:1/724: getdents d0/d8/d4d/d56/db6/ddc/d82 0 2024-03-22T10:42:49.372 INFO:tasks.workunit.client.0.smithi115.stdout:0/598: link d3/d7/d1c/l30 d3/da/d4c/d69/lc3 0 2024-03-22T10:42:49.374 INFO:tasks.workunit.client.0.smithi115.stdout:8/850: creat d6/dd/d12/de3/f11e x:0 0 0 2024-03-22T10:42:49.375 INFO:tasks.workunit.client.1.smithi150.stdout:7/273: mknod d2/d5/d7/d8/d2c/c5d 0 2024-03-22T10:42:49.375 INFO:tasks.workunit.client.0.smithi115.stdout:4/638: stat d1/dbb/dc6/dd8 0 2024-03-22T10:42:49.376 INFO:tasks.workunit.client.1.smithi150.stdout:2/288: write d12/d48/f4c [4764332,98062] 0 2024-03-22T10:42:49.376 INFO:tasks.workunit.client.0.smithi115.stdout:5/659: symlink d2/df/d3b/ldb 0 2024-03-22T10:42:49.376 INFO:tasks.workunit.client.0.smithi115.stdout:2/588: mknod da/da5/cae 0 2024-03-22T10:42:49.378 INFO:tasks.workunit.client.0.smithi115.stdout:3/651: chown d3/d19/d2b/d49/fc6 457619009 1 2024-03-22T10:42:49.379 INFO:tasks.workunit.client.0.smithi115.stdout:6/584: dread d4/d11/d12/f4e [0,4194304] 0 2024-03-22T10:42:49.380 INFO:tasks.workunit.client.1.smithi150.stdout:0/292: creat d0/f63 x:0 0 0 2024-03-22T10:42:49.380 INFO:tasks.workunit.client.1.smithi150.stdout:6/214: link db/dd/d25/f3d db/dd/d15/d30/f51 0 2024-03-22T10:42:49.384 INFO:tasks.workunit.client.1.smithi150.stdout:2/289: symlink d12/l5e 0 2024-03-22T10:42:49.385 INFO:tasks.workunit.client.1.smithi150.stdout:5/193: dread d0/f22 [0,4194304] 0 2024-03-22T10:42:49.389 INFO:tasks.workunit.client.0.smithi115.stdout:0/599: symlink d3/d7/d1c/lc4 0 2024-03-22T10:42:49.390 INFO:tasks.workunit.client.0.smithi115.stdout:2/589: symlink da/d25/d3d/d57/laf 0 2024-03-22T10:42:49.390 INFO:tasks.workunit.client.1.smithi150.stdout:5/194: mkdir d0/d37 0 2024-03-22T10:42:49.392 INFO:tasks.workunit.client.1.smithi150.stdout:2/290: mknod d12/d19/d2d/c5f 0 2024-03-22T10:42:49.393 INFO:tasks.workunit.client.1.smithi150.stdout:8/290: dread d0/d3c/f4a [0,4194304] 0 2024-03-22T10:42:49.395 INFO:tasks.workunit.client.1.smithi150.stdout:5/195: creat d0/f38 x:0 0 0 2024-03-22T10:42:49.396 INFO:tasks.workunit.client.1.smithi150.stdout:2/291: write d12/d48/f41 [1475892,3073] 0 2024-03-22T10:42:49.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:49 smithi150 ceph-mon[34518]: pgmap v13: 65 pgs: 65 active+clean; 1.5 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 161 MiB/s wr, 305 op/s 2024-03-22T10:42:49.400 INFO:tasks.workunit.client.1.smithi150.stdout:8/291: unlink d0/d2/f2b 0 2024-03-22T10:42:49.402 INFO:tasks.workunit.client.1.smithi150.stdout:0/293: rename d0/d12/d4f/d11/c36 to d0/d12/d4f/d42/d10/d38/d52/c64 0 2024-03-22T10:42:49.403 INFO:tasks.workunit.client.0.smithi115.stdout:6/585: truncate d4/d11/f78 1447657 0 2024-03-22T10:42:49.409 INFO:tasks.workunit.client.1.smithi150.stdout:8/292: stat d0/f1 0 2024-03-22T10:42:49.410 INFO:tasks.workunit.client.0.smithi115.stdout:3/652: link d3/d16/d29/f3c d3/d19/dda/fdd 0 2024-03-22T10:42:49.411 INFO:tasks.workunit.client.0.smithi115.stdout:6/586: dread - d4/d13/d33/fba zero size 2024-03-22T10:42:49.412 INFO:tasks.workunit.client.0.smithi115.stdout:4/639: dwrite d1/d30/d4c/d86/d9f/fd6 [0,4194304] 0 2024-03-22T10:42:49.413 INFO:tasks.workunit.client.1.smithi150.stdout:5/196: dwrite d0/f6 [0,4194304] 0 2024-03-22T10:42:49.413 INFO:tasks.workunit.client.1.smithi150.stdout:9/295: write d1/d8/de/df/d20/d32/f2a [3187232,43859] 0 2024-03-22T10:42:49.415 INFO:tasks.workunit.client.1.smithi150.stdout:2/292: mknod d12/d16/d2c/c60 0 2024-03-22T10:42:49.415 INFO:tasks.workunit.client.1.smithi150.stdout:7/274: truncate d2/d5/d7/d9/db/f1a 50000 0 2024-03-22T10:42:49.416 INFO:tasks.workunit.client.0.smithi115.stdout:8/851: write d6/fb [895361,62086] 0 2024-03-22T10:42:49.417 INFO:tasks.workunit.client.0.smithi115.stdout:5/660: link d2/d15/d1b/d7c/d38/d43/d49/f52 d2/df/d13/d8b/d93/da6/dbc/d50/db1/fdc 0 2024-03-22T10:42:49.422 INFO:tasks.workunit.client.1.smithi150.stdout:6/215: truncate db/dd/d15/f46 3640335 0 2024-03-22T10:42:49.423 INFO:tasks.workunit.client.1.smithi150.stdout:6/216: chown c6 176283 1 2024-03-22T10:42:49.424 INFO:tasks.workunit.client.1.smithi150.stdout:2/293: mkdir d12/d16/d61 0 2024-03-22T10:42:49.426 INFO:tasks.workunit.client.0.smithi115.stdout:4/640: write d1/d66/d82/f9a [1395911,397] 0 2024-03-22T10:42:49.426 INFO:tasks.workunit.client.0.smithi115.stdout:9/660: dwrite d6/d7/d20/d22/d23/f83 [0,4194304] 0 2024-03-22T10:42:49.426 INFO:tasks.workunit.client.1.smithi150.stdout:2/294: truncate d12/d16/f22 1927189 0 2024-03-22T10:42:49.428 INFO:tasks.workunit.client.0.smithi115.stdout:6/587: chown d4/d13/d33/d58/l7b 2080453 1 2024-03-22T10:42:49.428 INFO:tasks.workunit.client.0.smithi115.stdout:4/641: chown d1/d66/d82 2334790 1 2024-03-22T10:42:49.430 INFO:tasks.workunit.client.0.smithi115.stdout:2/590: dread da/d25/d3d/f97 [0,4194304] 0 2024-03-22T10:42:49.430 INFO:tasks.workunit.client.1.smithi150.stdout:7/275: creat d2/d5/d7/d8/f5e x:0 0 0 2024-03-22T10:42:49.432 INFO:tasks.workunit.client.1.smithi150.stdout:2/295: chown d12/d19/f1b 2002920 1 2024-03-22T10:42:49.437 INFO:tasks.workunit.client.1.smithi150.stdout:6/217: mkdir db/d52 0 2024-03-22T10:42:49.438 INFO:tasks.workunit.client.0.smithi115.stdout:1/725: dread d0/d22/f86 [0,4194304] 0 2024-03-22T10:42:49.439 INFO:tasks.workunit.client.1.smithi150.stdout:2/296: write d12/d19/f58 [450871,20723] 0 2024-03-22T10:42:49.440 INFO:tasks.workunit.client.0.smithi115.stdout:6/588: chown d4/d36/l4a 0 1 2024-03-22T10:42:49.442 INFO:tasks.workunit.client.1.smithi150.stdout:9/296: write d1/d8/de/df/d36/d40/d44/f6a [1402198,88671] 0 2024-03-22T10:42:49.443 INFO:tasks.workunit.client.0.smithi115.stdout:6/589: write d4/d13/f25 [2031307,48330] 0 2024-03-22T10:42:49.444 INFO:tasks.workunit.client.1.smithi150.stdout:8/293: truncate d0/d2/fa 3461029 0 2024-03-22T10:42:49.445 INFO:tasks.workunit.client.1.smithi150.stdout:0/294: write d0/d12/d4f/d11/f2f [1053910,90223] 0 2024-03-22T10:42:49.446 INFO:tasks.workunit.client.0.smithi115.stdout:4/642: chown d1/d16/d35/d40/d54/f67 62 1 2024-03-22T10:42:49.448 INFO:tasks.workunit.client.0.smithi115.stdout:3/653: rmdir d3/d36/d41/d33/dbf 0 2024-03-22T10:42:49.449 INFO:tasks.workunit.client.1.smithi150.stdout:5/197: rename d0/d8/dc/de/d18/c21 to d0/d8/c39 0 2024-03-22T10:42:49.452 INFO:tasks.workunit.client.0.smithi115.stdout:3/654: read d3/d19/dd8/fb9 [1036399,57204] 0 2024-03-22T10:42:49.452 INFO:tasks.workunit.client.0.smithi115.stdout:7/666: sync 2024-03-22T10:42:49.453 INFO:tasks.workunit.client.1.smithi150.stdout:0/295: dread d0/f3a [0,4194304] 0 2024-03-22T10:42:49.456 INFO:tasks.workunit.client.1.smithi150.stdout:0/296: fsync d0/d1/f39 0 2024-03-22T10:42:49.456 INFO:tasks.workunit.client.0.smithi115.stdout:2/591: creat da/d25/d3d/d57/d7d/da1/fb0 x:0 0 0 2024-03-22T10:42:49.457 INFO:tasks.workunit.client.0.smithi115.stdout:6/590: mknod d4/d13/d33/d58/cbc 0 2024-03-22T10:42:49.457 INFO:tasks.workunit.client.1.smithi150.stdout:6/218: rename db/dd/d21/f2f to db/dd/d21/f53 0 2024-03-22T10:42:49.464 INFO:tasks.workunit.client.1.smithi150.stdout:6/219: mknod db/dd/d21/c54 0 2024-03-22T10:42:49.467 INFO:tasks.workunit.client.1.smithi150.stdout:9/297: creat d1/d8/de/d73/f74 x:0 0 0 2024-03-22T10:42:49.467 INFO:tasks.workunit.client.1.smithi150.stdout:5/198: mknod d0/d8/c3a 0 2024-03-22T10:42:49.467 INFO:tasks.workunit.client.1.smithi150.stdout:4/268: dread d0/d1b/f24 [0,4194304] 0 2024-03-22T10:42:49.467 INFO:tasks.workunit.client.1.smithi150.stdout:2/297: dwrite d12/d16/f43 [0,4194304] 0 2024-03-22T10:42:49.467 INFO:tasks.workunit.client.0.smithi115.stdout:7/667: rename d2/d3a/d6e to d2/d7/d12/d10/d23/d3d/dd2/da3/de8 0 2024-03-22T10:42:49.472 INFO:tasks.workunit.client.1.smithi150.stdout:4/269: fdatasync d0/d1b/f21 0 2024-03-22T10:42:49.472 INFO:tasks.workunit.client.1.smithi150.stdout:0/297: truncate d0/d12/d4f/d42/d10/d38/d52/f54 443347 0 2024-03-22T10:42:49.472 INFO:tasks.workunit.client.1.smithi150.stdout:7/276: dread d2/d5/d7/d8/d57/d4a/f22 [4194304,4194304] 0 2024-03-22T10:42:49.472 INFO:tasks.workunit.client.1.smithi150.stdout:8/294: mknod d0/d2/d19/d29/d2f/c65 0 2024-03-22T10:42:49.472 INFO:tasks.workunit.client.0.smithi115.stdout:4/643: readlink d1/d30/d4c/d86/d9f/d4f/d93/l84 0 2024-03-22T10:42:49.476 INFO:tasks.workunit.client.1.smithi150.stdout:6/220: write db/dd/d15/d30/f43 [966912,106555] 0 2024-03-22T10:42:49.476 INFO:tasks.workunit.client.1.smithi150.stdout:2/298: mkdir d12/d19/d62 0 2024-03-22T10:42:49.476 INFO:tasks.workunit.client.0.smithi115.stdout:8/852: dwrite d6/dd/f26 [0,4194304] 0 2024-03-22T10:42:49.476 INFO:tasks.workunit.client.0.smithi115.stdout:0/600: dread d3/d7/d1b/d35/f42 [0,4194304] 0 2024-03-22T10:42:49.485 INFO:tasks.workunit.client.0.smithi115.stdout:6/591: rename d4/d11/d12/c2f to d4/d11/d12/d1c/cbd 0 2024-03-22T10:42:49.486 INFO:tasks.workunit.client.0.smithi115.stdout:4/644: fsync d1/d30/d4c/d86/d9f/f34 0 2024-03-22T10:42:49.489 INFO:tasks.workunit.client.0.smithi115.stdout:9/661: dwrite d6/d8/d3c/d59/f90 [0,4194304] 0 2024-03-22T10:42:49.497 INFO:tasks.workunit.client.0.smithi115.stdout:2/592: dwrite da/d25/d3d/f9f [0,4194304] 0 2024-03-22T10:42:49.497 INFO:tasks.workunit.client.0.smithi115.stdout:5/661: dread d2/d15/d1b/d7c/d84/d89/fb8 [0,4194304] 0 2024-03-22T10:42:49.499 INFO:tasks.workunit.client.0.smithi115.stdout:6/592: unlink d4/d11/f8f 0 2024-03-22T10:42:49.506 INFO:tasks.workunit.client.0.smithi115.stdout:5/662: mknod d2/d15/cdd 0 2024-03-22T10:42:49.506 INFO:tasks.workunit.client.0.smithi115.stdout:6/593: truncate d4/d11/d12/d1c/f7f 4823390 0 2024-03-22T10:42:49.510 INFO:tasks.workunit.client.0.smithi115.stdout:8/853: link d6/dd/d12/fb0 d6/f11f 0 2024-03-22T10:42:49.510 INFO:tasks.workunit.client.0.smithi115.stdout:2/593: mkdir da/db1 0 2024-03-22T10:42:49.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:49 smithi115 ceph-mon[29111]: pgmap v13: 65 pgs: 65 active+clean; 1.5 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 161 MiB/s wr, 305 op/s 2024-03-22T10:42:49.512 INFO:tasks.workunit.client.0.smithi115.stdout:4/645: write d1/d66/d82/fa6 [4715586,85426] 0 2024-03-22T10:42:49.512 INFO:tasks.workunit.client.0.smithi115.stdout:6/594: rmdir d4 39 2024-03-22T10:42:49.512 INFO:tasks.workunit.client.0.smithi115.stdout:5/663: write d2/d15/d1b/d7c/d38/d43/fce [773595,27814] 0 2024-03-22T10:42:49.513 INFO:tasks.workunit.client.1.smithi150.stdout:1/249: sync 2024-03-22T10:42:49.516 INFO:tasks.workunit.client.0.smithi115.stdout:9/662: getdents d6/d5e 0 2024-03-22T10:42:49.520 INFO:tasks.workunit.client.0.smithi115.stdout:8/854: write d6/dd/d12/dc6/fde [1293841,17956] 0 2024-03-22T10:42:49.520 INFO:tasks.workunit.client.0.smithi115.stdout:0/601: dwrite d3/d7/d1b/f47 [0,4194304] 0 2024-03-22T10:42:49.526 INFO:tasks.workunit.client.0.smithi115.stdout:9/663: mknod d6/d7/d30/d37/dbe/cec 0 2024-03-22T10:42:49.529 INFO:tasks.workunit.client.0.smithi115.stdout:9/664: write d6/d7/d19/f25 [230124,46579] 0 2024-03-22T10:42:49.530 INFO:tasks.workunit.client.1.smithi150.stdout:4/270: symlink d0/d32/l58 0 2024-03-22T10:42:49.530 INFO:tasks.workunit.client.1.smithi150.stdout:2/299: mkdir d12/d48/d63 0 2024-03-22T10:42:49.534 INFO:tasks.workunit.client.1.smithi150.stdout:7/277: write d2/d5/d7/d8/d46/d49/f58 [77015,39374] 0 2024-03-22T10:42:49.535 INFO:tasks.workunit.client.1.smithi150.stdout:2/300: chown d12/d19/d1a/l49 0 1 2024-03-22T10:42:49.536 INFO:tasks.workunit.client.0.smithi115.stdout:4/646: symlink d1/d30/d4c/d86/d9f/d75/d99/da8/ldb 0 2024-03-22T10:42:49.537 INFO:tasks.workunit.client.1.smithi150.stdout:2/301: dread - d12/d19/f2e zero size 2024-03-22T10:42:49.540 INFO:tasks.workunit.client.0.smithi115.stdout:5/664: rename d2/d15/d1b/d7c/d38/d43/d49/l4b to d2/df/d13/d8b/d93/da6/dbc/lde 0 2024-03-22T10:42:49.541 INFO:tasks.workunit.client.0.smithi115.stdout:0/602: fdatasync d3/da/d4c/d69/fbd 0 2024-03-22T10:42:49.543 INFO:tasks.workunit.client.0.smithi115.stdout:9/665: truncate d6/d8/f47 2864698 0 2024-03-22T10:42:49.544 INFO:tasks.workunit.client.1.smithi150.stdout:0/298: link d0/d12/d4f/d42/f44 d0/d12/d4f/d42/d10/d38/d52/f65 0 2024-03-22T10:42:49.544 INFO:tasks.workunit.client.1.smithi150.stdout:9/298: getdents d1/d8/de/df/d24/d58 0 2024-03-22T10:42:49.546 INFO:tasks.workunit.client.1.smithi150.stdout:7/278: creat d2/d5/d7/d8/d57/d54/f5f x:0 0 0 2024-03-22T10:42:49.548 INFO:tasks.workunit.client.1.smithi150.stdout:1/250: dread d1/da/d15/f3e [0,4194304] 0 2024-03-22T10:42:49.549 INFO:tasks.workunit.client.1.smithi150.stdout:2/302: rename d12/d19/l1f to d12/d21/l64 0 2024-03-22T10:42:49.549 INFO:tasks.workunit.client.1.smithi150.stdout:7/279: chown d2/d5/l34 362 1 2024-03-22T10:42:49.549 INFO:tasks.workunit.client.0.smithi115.stdout:8/855: symlink d6/dd/d24/d3a/l120 0 2024-03-22T10:42:49.549 INFO:tasks.workunit.client.0.smithi115.stdout:5/665: read d2/d15/d1b/d7c/d38/d43/d49/fb9 [3149399,76627] 0 2024-03-22T10:42:49.551 INFO:tasks.workunit.client.1.smithi150.stdout:8/295: getdents d0/d2 0 2024-03-22T10:42:49.552 INFO:tasks.workunit.client.0.smithi115.stdout:9/666: write d6/d7/d20/f57 [3369262,39283] 0 2024-03-22T10:42:49.552 INFO:tasks.workunit.client.1.smithi150.stdout:9/299: mkdir d1/d8/de/df/d20/d32/d75 0 2024-03-22T10:42:49.553 INFO:tasks.workunit.client.1.smithi150.stdout:8/296: chown d0/d2/d46 18656038 1 2024-03-22T10:42:49.553 INFO:tasks.workunit.client.1.smithi150.stdout:1/251: chown d1/da/d15/d18/f4a 0 1 2024-03-22T10:42:49.554 INFO:tasks.workunit.client.0.smithi115.stdout:4/647: symlink d1/d30/ldc 0 2024-03-22T10:42:49.555 INFO:tasks.workunit.client.1.smithi150.stdout:7/280: creat d2/f60 x:0 0 0 2024-03-22T10:42:49.555 INFO:tasks.workunit.client.1.smithi150.stdout:4/271: link d0/c4 d0/de/c59 0 2024-03-22T10:42:49.556 INFO:tasks.workunit.client.0.smithi115.stdout:8/856: readlink d6/dd/ld3 0 2024-03-22T10:42:49.557 INFO:tasks.workunit.client.1.smithi150.stdout:2/303: mkdir d12/d16/d2c/d65 0 2024-03-22T10:42:49.560 INFO:tasks.workunit.client.0.smithi115.stdout:8/857: chown d6/ff4 4848 1 2024-03-22T10:42:49.560 INFO:tasks.workunit.client.0.smithi115.stdout:9/667: creat d6/d5e/fed x:0 0 0 2024-03-22T10:42:49.561 INFO:tasks.workunit.client.1.smithi150.stdout:9/300: mknod d1/d8/de/df/d24/d58/c76 0 2024-03-22T10:42:49.564 INFO:tasks.workunit.client.1.smithi150.stdout:9/301: write d1/d8/de/df/d24/f5f [1788866,20703] 0 2024-03-22T10:42:49.568 INFO:tasks.workunit.client.1.smithi150.stdout:7/281: mknod d2/d5/d7/d8/d2c/c61 0 2024-03-22T10:42:49.568 INFO:tasks.workunit.client.1.smithi150.stdout:9/302: write d1/d8/de/f2c [1953452,2427] 0 2024-03-22T10:42:49.572 INFO:tasks.workunit.client.1.smithi150.stdout:0/299: rename d0/d12/d4f/d42/d10/d1e to d0/d12/d4f/d11/d66 0 2024-03-22T10:42:49.573 INFO:tasks.workunit.client.0.smithi115.stdout:4/648: getdents d1/d30/d4c/d86/d9f 0 2024-03-22T10:42:49.573 INFO:tasks.workunit.client.0.smithi115.stdout:9/668: symlink d6/d7/d20/d22/lee 0 2024-03-22T10:42:49.575 INFO:tasks.workunit.client.0.smithi115.stdout:8/858: rename d6/l9 to d6/dd/d12/l121 0 2024-03-22T10:42:49.575 INFO:tasks.workunit.client.1.smithi150.stdout:1/252: dwrite d1/da/d2a/f35 [0,4194304] 0 2024-03-22T10:42:49.580 INFO:tasks.workunit.client.0.smithi115.stdout:4/649: creat d1/dbb/dc6/dd8/fdd x:0 0 0 2024-03-22T10:42:49.581 INFO:tasks.workunit.client.1.smithi150.stdout:7/282: creat d2/d5/d7/d9/f62 x:0 0 0 2024-03-22T10:42:49.583 INFO:tasks.workunit.client.1.smithi150.stdout:9/303: creat d1/d8/de/df/d24/d4e/f77 x:0 0 0 2024-03-22T10:42:49.583 INFO:tasks.workunit.client.0.smithi115.stdout:6/595: dread d4/d36/f53 [0,4194304] 0 2024-03-22T10:42:49.584 INFO:tasks.workunit.client.1.smithi150.stdout:8/297: creat d0/d3c/f66 x:0 0 0 2024-03-22T10:42:49.585 INFO:tasks.workunit.client.1.smithi150.stdout:7/283: write d2/d5/d7/d9/f62 [425972,96600] 0 2024-03-22T10:42:49.586 INFO:tasks.workunit.client.0.smithi115.stdout:0/603: dwrite d3/d7/d1b/f92 [4194304,4194304] 0 2024-03-22T10:42:49.586 INFO:tasks.workunit.client.1.smithi150.stdout:9/304: read d1/d8/de/df/f1f [1188896,9288] 0 2024-03-22T10:42:49.590 INFO:tasks.workunit.client.1.smithi150.stdout:9/305: write d1/d8/de/df/f5b [483701,45109] 0 2024-03-22T10:42:49.590 INFO:tasks.workunit.client.1.smithi150.stdout:6/221: truncate db/dd/d21/f32 1524499 0 2024-03-22T10:42:49.592 INFO:tasks.workunit.client.0.smithi115.stdout:5/666: dwrite d2/d15/d1b/d7c/d38/d56/f87 [0,4194304] 0 2024-03-22T10:42:49.592 INFO:tasks.workunit.client.0.smithi115.stdout:9/669: write d6/d8/d3c/d59/d68/da8/fb4 [5068015,77579] 0 2024-03-22T10:42:49.594 INFO:tasks.workunit.client.0.smithi115.stdout:5/667: readlink d2/df/l47 0 2024-03-22T10:42:49.596 INFO:tasks.workunit.client.0.smithi115.stdout:4/650: mkdir d1/d30/d4c/d86/dde 0 2024-03-22T10:42:49.598 INFO:tasks.workunit.client.1.smithi150.stdout:1/253: mkdir d1/d19/d50 0 2024-03-22T10:42:49.601 INFO:tasks.workunit.client.0.smithi115.stdout:6/596: mkdir d4/d13/d33/dbe 0 2024-03-22T10:42:49.602 INFO:tasks.workunit.client.0.smithi115.stdout:9/670: symlink d6/d8/d3c/d59/d68/d97/lef 0 2024-03-22T10:42:49.603 INFO:tasks.workunit.client.0.smithi115.stdout:5/668: chown d2/df/c40 1555 1 2024-03-22T10:42:49.605 INFO:tasks.workunit.client.0.smithi115.stdout:0/604: rename d3/d7/d1b/f2a to d3/da/d4c/da3/fc5 0 2024-03-22T10:42:49.605 INFO:tasks.workunit.client.0.smithi115.stdout:4/651: dread - d1/d30/d4c/d86/d9f/d32/fda zero size 2024-03-22T10:42:49.605 INFO:tasks.workunit.client.1.smithi150.stdout:4/272: write d0/f14 [2114806,83982] 0 2024-03-22T10:42:49.607 INFO:tasks.workunit.client.0.smithi115.stdout:6/597: mknod d4/d36/cbf 0 2024-03-22T10:42:49.608 INFO:tasks.workunit.client.1.smithi150.stdout:8/298: fsync d0/f17 0 2024-03-22T10:42:49.608 INFO:tasks.workunit.client.1.smithi150.stdout:6/222: unlink db/l2d 0 2024-03-22T10:42:49.608 INFO:tasks.workunit.client.1.smithi150.stdout:0/300: link d0/c4d d0/d12/d4f/d11/d66/c67 0 2024-03-22T10:42:49.608 INFO:tasks.workunit.client.1.smithi150.stdout:4/273: write d0/de/f3c [376280,50082] 0 2024-03-22T10:42:49.609 INFO:tasks.workunit.client.0.smithi115.stdout:9/671: write d6/d7/d30/d37/dbe/fd4 [935678,20509] 0 2024-03-22T10:42:49.610 INFO:tasks.workunit.client.1.smithi150.stdout:9/306: rename d1/d8/de/df/d24/d58/f70 to d1/d8/de/df/d36/d40/f78 0 2024-03-22T10:42:49.614 INFO:tasks.workunit.client.1.smithi150.stdout:2/304: write d12/d21/f4b [1365338,68875] 0 2024-03-22T10:42:49.614 INFO:tasks.workunit.client.0.smithi115.stdout:4/652: mkdir d1/dbb/ddf 0 2024-03-22T10:42:49.615 INFO:tasks.workunit.client.1.smithi150.stdout:0/301: stat d0/d1/c1d 0 2024-03-22T10:42:49.615 INFO:tasks.workunit.client.1.smithi150.stdout:1/254: mkdir d1/d19/d50/d51 0 2024-03-22T10:42:49.615 INFO:tasks.workunit.client.1.smithi150.stdout:4/274: read - d0/d1b/d4d/f53 zero size 2024-03-22T10:42:49.615 INFO:tasks.workunit.client.0.smithi115.stdout:8/859: dwrite d6/f59 [0,4194304] 0 2024-03-22T10:42:49.616 INFO:tasks.workunit.client.0.smithi115.stdout:4/653: fdatasync d1/d66/fc1 0 2024-03-22T10:42:49.617 INFO:tasks.workunit.client.1.smithi150.stdout:8/299: rmdir d0/d2/d19/d29/d2f/df 39 2024-03-22T10:42:49.617 INFO:tasks.workunit.client.1.smithi150.stdout:7/284: dwrite d2/d5/d7/d8/d46/d49/f58 [4194304,4194304] 0 2024-03-22T10:42:49.619 INFO:tasks.workunit.client.1.smithi150.stdout:0/302: read - d0/f63 zero size 2024-03-22T10:42:49.619 INFO:tasks.workunit.client.0.smithi115.stdout:5/669: write d2/df/d3b/fa0 [398059,4969] 0 2024-03-22T10:42:49.620 INFO:tasks.workunit.client.0.smithi115.stdout:6/598: rename d4/d36/l4a to d4/d5e/lc0 0 2024-03-22T10:42:49.621 INFO:tasks.workunit.client.1.smithi150.stdout:9/307: creat d1/d19/f79 x:0 0 0 2024-03-22T10:42:49.622 INFO:tasks.workunit.client.1.smithi150.stdout:2/305: mkdir d12/d19/d2d/d66 0 2024-03-22T10:42:49.623 INFO:tasks.workunit.client.0.smithi115.stdout:9/672: rmdir d6/d7/d48 39 2024-03-22T10:42:49.624 INFO:tasks.workunit.client.1.smithi150.stdout:4/275: write d0/de/f28 [1532461,19119] 0 2024-03-22T10:42:49.626 INFO:tasks.workunit.client.0.smithi115.stdout:4/654: creat d1/d30/d4c/d86/d9f/d4f/fe0 x:0 0 0 2024-03-22T10:42:49.626 INFO:tasks.workunit.client.0.smithi115.stdout:0/605: mknod d3/d7/d1b/d1f/d5b/dba/cc6 0 2024-03-22T10:42:49.628 INFO:tasks.workunit.client.1.smithi150.stdout:1/255: rename d1/da/d15/d18/c27 to d1/da/d15/d18/d4b/c52 0 2024-03-22T10:42:49.628 INFO:tasks.workunit.client.0.smithi115.stdout:9/673: mkdir d6/d7/d6a/d6f/df0 0 2024-03-22T10:42:49.629 INFO:tasks.workunit.client.1.smithi150.stdout:7/285: dread - d2/d5/d7/d8/d57/d54/f5f zero size 2024-03-22T10:42:49.630 INFO:tasks.workunit.client.1.smithi150.stdout:8/300: unlink d0/d2/fa 0 2024-03-22T10:42:49.631 INFO:tasks.workunit.client.1.smithi150.stdout:9/308: fsync d1/d8/de/df/d24/f53 0 2024-03-22T10:42:49.632 INFO:tasks.workunit.client.0.smithi115.stdout:4/655: chown d1/d30/d4c/d86/d9f/d75/d99/f97 117764899 1 2024-03-22T10:42:49.633 INFO:tasks.workunit.client.1.smithi150.stdout:2/306: write d12/d19/f59 [28682,67219] 0 2024-03-22T10:42:49.633 INFO:tasks.workunit.client.0.smithi115.stdout:8/860: mknod d6/dd/d24/d3a/c122 0 2024-03-22T10:42:49.633 INFO:tasks.workunit.client.1.smithi150.stdout:0/303: mkdir d0/d12/d4f/d42/d10/d38/d52/d68 0 2024-03-22T10:42:49.635 INFO:tasks.workunit.client.0.smithi115.stdout:7/668: sync 2024-03-22T10:42:49.635 INFO:tasks.workunit.client.0.smithi115.stdout:3/655: sync 2024-03-22T10:42:49.635 INFO:tasks.workunit.client.0.smithi115.stdout:1/726: sync 2024-03-22T10:42:49.635 INFO:tasks.workunit.client.0.smithi115.stdout:2/594: sync 2024-03-22T10:42:49.637 INFO:tasks.workunit.client.0.smithi115.stdout:5/670: link d2/df/d13/f18 d2/df/fdf 0 2024-03-22T10:42:49.637 INFO:tasks.workunit.client.0.smithi115.stdout:3/656: stat d3/d36/f44 0 2024-03-22T10:42:49.638 INFO:tasks.workunit.client.0.smithi115.stdout:8/861: fsync d6/dd/d24/d4a/d7b/fba 0 2024-03-22T10:42:49.639 INFO:tasks.workunit.client.0.smithi115.stdout:2/595: stat da/d25/d3d/d7e 0 2024-03-22T10:42:49.640 INFO:tasks.workunit.client.0.smithi115.stdout:1/727: chown d0/d8/d4d/d56/c76 2604 1 2024-03-22T10:42:49.642 INFO:tasks.workunit.client.0.smithi115.stdout:6/599: creat d4/fc1 x:0 0 0 2024-03-22T10:42:49.642 INFO:tasks.workunit.client.0.smithi115.stdout:2/596: chown da/d25/d3d/d57/d7d 26949 1 2024-03-22T10:42:49.642 INFO:tasks.workunit.client.0.smithi115.stdout:4/656: creat d1/dbb/dc6/fe1 x:0 0 0 2024-03-22T10:42:49.643 INFO:tasks.workunit.client.1.smithi150.stdout:6/223: dwrite db/f4a [0,4194304] 0 2024-03-22T10:42:49.643 INFO:tasks.workunit.client.1.smithi150.stdout:9/309: write d1/d49/f31 [978385,83839] 0 2024-03-22T10:42:49.643 INFO:tasks.workunit.client.0.smithi115.stdout:8/862: dread - d6/dd/d24/d4a/dfc/f11c zero size 2024-03-22T10:42:49.644 INFO:tasks.workunit.client.0.smithi115.stdout:0/606: read d3/da/d4c/d69/f76 [771500,80177] 0 2024-03-22T10:42:49.645 INFO:tasks.workunit.client.0.smithi115.stdout:6/600: fdatasync d4/d11/d12/d1c/f7f 0 2024-03-22T10:42:49.646 INFO:tasks.workunit.client.0.smithi115.stdout:1/728: readlink d0/d12/d17/d67/dd8/de0/lf1 0 2024-03-22T10:42:49.647 INFO:tasks.workunit.client.1.smithi150.stdout:6/224: read db/dd/d15/f27 [708894,69592] 0 2024-03-22T10:42:49.648 INFO:tasks.workunit.client.0.smithi115.stdout:9/674: getdents d6/d8/d3c/d59/d75 0 2024-03-22T10:42:49.648 INFO:tasks.workunit.client.0.smithi115.stdout:7/669: fdatasync d2/d7/d12/d10/d19/dbd/fdb 0 2024-03-22T10:42:49.650 INFO:tasks.workunit.client.0.smithi115.stdout:1/729: chown d0/d8/db/d25 449 1 2024-03-22T10:42:49.651 INFO:tasks.workunit.client.1.smithi150.stdout:1/256: creat d1/da/d15/d18/d4b/f53 x:0 0 0 2024-03-22T10:42:49.651 INFO:tasks.workunit.client.1.smithi150.stdout:9/310: dread d1/d8/de/df/f4d [0,4194304] 0 2024-03-22T10:42:49.655 INFO:tasks.workunit.client.1.smithi150.stdout:3/275: sync 2024-03-22T10:42:49.655 INFO:tasks.workunit.client.1.smithi150.stdout:9/311: write d1/d8/de/df/d20/d32/f2a [991962,124333] 0 2024-03-22T10:42:49.655 INFO:tasks.workunit.client.0.smithi115.stdout:5/671: rename d2/fe to d2/df/d13/d8b/d93/da6/fe0 0 2024-03-22T10:42:49.655 INFO:tasks.workunit.client.1.smithi150.stdout:4/276: symlink d0/de/d57/l5a 0 2024-03-22T10:42:49.659 INFO:tasks.workunit.client.1.smithi150.stdout:9/312: write d1/d8/de/df/d24/f4a [398101,34984] 0 2024-03-22T10:42:49.660 INFO:tasks.workunit.client.1.smithi150.stdout:2/307: rename d12/d19/d2d/d3b to d12/d16/d67 0 2024-03-22T10:42:49.662 INFO:tasks.workunit.client.1.smithi150.stdout:2/308: readlink d12/l5e 0 2024-03-22T10:42:49.662 INFO:tasks.workunit.client.1.smithi150.stdout:8/301: dwrite d0/fb [0,4194304] 0 2024-03-22T10:42:49.665 INFO:tasks.workunit.client.1.smithi150.stdout:9/313: chown d1/d8/de/df/d36/d40/f78 78768668 1 2024-03-22T10:42:49.665 INFO:tasks.workunit.client.0.smithi115.stdout:2/597: dread da/d25/d3d/d57/f65 [0,4194304] 0 2024-03-22T10:42:49.668 INFO:tasks.workunit.client.1.smithi150.stdout:9/314: write d1/d8/de/df/d20/f5d [1498235,81027] 0 2024-03-22T10:42:49.671 INFO:tasks.workunit.client.1.smithi150.stdout:9/315: write d1/d8/de/df/d36/d40/d41/f47 [3502931,117590] 0 2024-03-22T10:42:49.672 INFO:tasks.workunit.client.0.smithi115.stdout:4/657: creat d1/d30/d4c/d86/d9f/d75/d99/d98/d9b/fe2 x:0 0 0 2024-03-22T10:42:49.674 INFO:tasks.workunit.client.0.smithi115.stdout:7/670: mknod d2/d7/d12/d10/d19/d42/d46/d94/ce9 0 2024-03-22T10:42:49.674 INFO:tasks.workunit.client.1.smithi150.stdout:9/316: stat d1/d8/de/df/d20/d32/d57 0 2024-03-22T10:42:49.675 INFO:tasks.workunit.client.0.smithi115.stdout:1/730: mkdir d0/d12/d7b/de2/df8 0 2024-03-22T10:42:49.676 INFO:tasks.workunit.client.0.smithi115.stdout:3/657: dwrite d3/d82/f83 [0,4194304] 0 2024-03-22T10:42:49.676 INFO:tasks.workunit.client.1.smithi150.stdout:2/309: stat f11 0 2024-03-22T10:42:49.678 INFO:tasks.workunit.client.0.smithi115.stdout:0/607: dwrite d3/d7/d1c/d6c/d9a/fbe [0,4194304] 0 2024-03-22T10:42:49.679 INFO:tasks.workunit.client.1.smithi150.stdout:7/286: link d2/d5/d7/d8/d46/f4e d2/d5/d7/d8/d57/d4a/f63 0 2024-03-22T10:42:49.680 INFO:tasks.workunit.client.0.smithi115.stdout:7/671: readlink d2/d7/d12/d10/d19/d42/d46/l47 0 2024-03-22T10:42:49.683 INFO:tasks.workunit.client.0.smithi115.stdout:1/731: chown d0/d68/cb8 189958 1 2024-03-22T10:42:49.685 INFO:tasks.workunit.client.1.smithi150.stdout:0/304: creat d0/f69 x:0 0 0 2024-03-22T10:42:49.686 INFO:tasks.workunit.client.0.smithi115.stdout:7/672: chown d2/d7/d12/d10/d23/d3d/dd2/da3/fab 5674 1 2024-03-22T10:42:49.688 INFO:tasks.workunit.client.0.smithi115.stdout:9/675: rename d6/d7/l71 to d6/d7/d48/d53/dc9/lf1 0 2024-03-22T10:42:49.688 INFO:tasks.workunit.client.1.smithi150.stdout:1/257: symlink d1/da/l54 0 2024-03-22T10:42:49.703 INFO:tasks.workunit.client.0.smithi115.stdout:0/608: dwrite d3/d52/f59 [0,4194304] 0 2024-03-22T10:42:49.705 INFO:tasks.workunit.client.1.smithi150.stdout:8/302: creat d0/d2/f67 x:0 0 0 2024-03-22T10:42:49.706 INFO:tasks.workunit.client.0.smithi115.stdout:1/732: creat d0/d8/d4d/d56/db6/ddc/ff9 x:0 0 0 2024-03-22T10:42:49.709 INFO:tasks.workunit.client.1.smithi150.stdout:9/317: mkdir d1/d8/de/df/d36/d40/d7a 0 2024-03-22T10:42:49.710 INFO:tasks.workunit.client.1.smithi150.stdout:7/287: dwrite d2/d5/d7/d8/d2c/f47 [0,4194304] 0 2024-03-22T10:42:49.710 INFO:tasks.workunit.client.1.smithi150.stdout:2/310: mkdir d12/d19/d2d/d68 0 2024-03-22T10:42:49.711 INFO:tasks.workunit.client.0.smithi115.stdout:6/601: write d4/d11/d12/d1c/f46 [2894959,41053] 0 2024-03-22T10:42:49.711 INFO:tasks.workunit.client.0.smithi115.stdout:9/676: creat d6/d7/d20/d22/de5/ff2 x:0 0 0 2024-03-22T10:42:49.712 INFO:tasks.workunit.client.0.smithi115.stdout:9/677: stat d6/f12 0 2024-03-22T10:42:49.713 INFO:tasks.workunit.client.0.smithi115.stdout:0/609: chown d3/d7/d1b/d1f/d5b/l7c 1032238 1 2024-03-22T10:42:49.715 INFO:tasks.workunit.client.0.smithi115.stdout:6/602: chown d4/le 54 1 2024-03-22T10:42:49.715 INFO:tasks.workunit.client.1.smithi150.stdout:6/225: creat db/f55 x:0 0 0 2024-03-22T10:42:49.716 INFO:tasks.workunit.client.1.smithi150.stdout:1/258: mkdir d1/da/d2a/d55 0 2024-03-22T10:42:49.718 INFO:tasks.workunit.client.0.smithi115.stdout:5/672: write d2/d15/d1b/d7c/d38/d43/d49/fb9 [4614774,57485] 0 2024-03-22T10:42:49.719 INFO:tasks.workunit.client.0.smithi115.stdout:7/673: getdents d2/d7/d12/d10/d19/dbd/dce 0 2024-03-22T10:42:49.719 INFO:tasks.workunit.client.0.smithi115.stdout:6/603: stat d4/d36/d48/d7a 0 2024-03-22T10:42:49.719 INFO:tasks.workunit.client.0.smithi115.stdout:1/733: dread - d0/d12/d2b/f97 zero size 2024-03-22T10:42:49.720 INFO:tasks.workunit.client.0.smithi115.stdout:4/658: link d1/d16/d35/d40/d5c/c13 d1/dbb/dc6/ce3 0 2024-03-22T10:42:49.720 INFO:tasks.workunit.client.0.smithi115.stdout:0/610: read d3/d7/d1b/d1f/d5b/f66 [1089805,94221] 0 2024-03-22T10:42:49.722 INFO:tasks.workunit.client.1.smithi150.stdout:3/276: getdents d5/d22 0 2024-03-22T10:42:49.723 INFO:tasks.workunit.client.0.smithi115.stdout:1/734: fsync d0/d12/f1c 0 2024-03-22T10:42:49.723 INFO:tasks.workunit.client.0.smithi115.stdout:5/673: write d2/df/fd1 [103210,44126] 0 2024-03-22T10:42:49.724 INFO:tasks.workunit.client.1.smithi150.stdout:4/277: dwrite d0/f9 [0,4194304] 0 2024-03-22T10:42:49.725 INFO:tasks.workunit.client.0.smithi115.stdout:7/674: chown d2/d7/d12/d10/d19 66047 1 2024-03-22T10:42:49.725 INFO:tasks.workunit.client.1.smithi150.stdout:6/226: mknod db/dd/d25/c56 0 2024-03-22T10:42:49.726 INFO:tasks.workunit.client.1.smithi150.stdout:7/288: unlink d2/d5/d7/c4f 0 2024-03-22T10:42:49.727 INFO:tasks.workunit.client.1.smithi150.stdout:1/259: rmdir d1/da 39 2024-03-22T10:42:49.728 INFO:tasks.workunit.client.0.smithi115.stdout:8/863: dwrite d6/d49/d88/f119 [0,4194304] 0 2024-03-22T10:42:49.729 INFO:tasks.workunit.client.0.smithi115.stdout:6/604: rename d4/d36/d48/lb4 to d4/d36/d48/d72/lc2 0 2024-03-22T10:42:49.732 INFO:tasks.workunit.client.1.smithi150.stdout:8/303: creat d0/d3c/f68 x:0 0 0 2024-03-22T10:42:49.734 INFO:tasks.workunit.client.0.smithi115.stdout:4/659: write d1/d30/d4c/d86/d9f/d32/f60 [4762025,67068] 0 2024-03-22T10:42:49.734 INFO:tasks.workunit.client.0.smithi115.stdout:0/611: write d3/d7/d1b/d35/f42 [4659884,28350] 0 2024-03-22T10:42:49.736 INFO:tasks.workunit.client.1.smithi150.stdout:7/289: fdatasync d2/d5/d7/d8/d57/d54/f5f 0 2024-03-22T10:42:49.736 INFO:tasks.workunit.client.1.smithi150.stdout:6/227: chown db/dd/d15/l3e 519020 1 2024-03-22T10:42:49.739 INFO:tasks.workunit.client.0.smithi115.stdout:5/674: write d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f69 [1473218,80922] 0 2024-03-22T10:42:49.740 INFO:tasks.workunit.client.0.smithi115.stdout:2/598: write da/d25/f49 [223512,5495] 0 2024-03-22T10:42:49.742 INFO:tasks.workunit.client.0.smithi115.stdout:4/660: chown d1/d16/d35/d40/d5c/d14/f1b 384612 1 2024-03-22T10:42:49.742 INFO:tasks.workunit.client.1.smithi150.stdout:6/228: mkdir db/dd/d15/d57 0 2024-03-22T10:42:49.742 INFO:tasks.workunit.client.1.smithi150.stdout:7/290: chown d2/d5/d7/d8/d2c 3 1 2024-03-22T10:42:49.742 INFO:tasks.workunit.client.0.smithi115.stdout:6/605: rmdir d4/d11 39 2024-03-22T10:42:49.744 INFO:tasks.workunit.client.0.smithi115.stdout:7/675: fsync d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/fe2 0 2024-03-22T10:42:49.747 INFO:tasks.workunit.client.0.smithi115.stdout:8/864: mknod d6/d49/c123 0 2024-03-22T10:42:49.749 INFO:tasks.workunit.client.0.smithi115.stdout:4/661: dread - d1/d30/d4c/d86/d9f/d75/d99/da8/fd0 zero size 2024-03-22T10:42:49.749 INFO:tasks.workunit.client.0.smithi115.stdout:5/675: stat d2/d15/d1b/d7c/d38/d43/ca4 0 2024-03-22T10:42:49.750 INFO:tasks.workunit.client.0.smithi115.stdout:9/678: dread d6/d7/d30/d37/dbe/fc6 [0,4194304] 0 2024-03-22T10:42:49.750 INFO:tasks.workunit.client.1.smithi150.stdout:7/291: creat d2/d5/d7/d8/f64 x:0 0 0 2024-03-22T10:42:49.752 INFO:tasks.workunit.client.1.smithi150.stdout:2/311: dread d12/d48/f41 [0,4194304] 0 2024-03-22T10:42:49.758 INFO:tasks.workunit.client.0.smithi115.stdout:6/606: mkdir d4/d11/d12/d1c/d8a/d98/d9f/dc3 0 2024-03-22T10:42:49.761 INFO:tasks.workunit.client.0.smithi115.stdout:5/676: chown d2/df/d3b/c4e 22744 1 2024-03-22T10:42:49.766 INFO:tasks.workunit.client.1.smithi150.stdout:3/277: dwrite d5/d22/d2f/d38/d3e/f3f [0,4194304] 0 2024-03-22T10:42:49.766 INFO:tasks.workunit.client.1.smithi150.stdout:4/278: dread d0/de/f4b [0,4194304] 0 2024-03-22T10:42:49.770 INFO:tasks.workunit.client.0.smithi115.stdout:6/607: fdatasync d4/d11/d12/f4e 0 2024-03-22T10:42:49.771 INFO:tasks.workunit.client.0.smithi115.stdout:0/612: dwrite d3/d52/f59 [0,4194304] 0 2024-03-22T10:42:49.771 INFO:tasks.workunit.client.0.smithi115.stdout:9/679: chown d6/d7/d48/d70/da9/cbc 55856229 1 2024-03-22T10:42:49.773 INFO:tasks.workunit.client.0.smithi115.stdout:7/676: dread d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f6a [0,4194304] 0 2024-03-22T10:42:49.774 INFO:tasks.workunit.client.1.smithi150.stdout:7/292: rename d2/d5/d7/d8/f64 to d2/d5/d7/d9/f65 0 2024-03-22T10:42:49.776 INFO:tasks.workunit.client.1.smithi150.stdout:1/260: dwrite d1/da/d2a/f35 [0,4194304] 0 2024-03-22T10:42:49.779 INFO:tasks.workunit.client.1.smithi150.stdout:3/278: fsync d5/f10 0 2024-03-22T10:42:49.779 INFO:tasks.workunit.client.1.smithi150.stdout:1/261: fsync d1/f3a 0 2024-03-22T10:42:49.784 INFO:tasks.workunit.client.1.smithi150.stdout:0/305: dwrite d0/d12/d4f/f31 [4194304,4194304] 0 2024-03-22T10:42:49.784 INFO:tasks.workunit.client.1.smithi150.stdout:6/229: dread db/dd/d15/d28/f2b [0,4194304] 0 2024-03-22T10:42:49.789 INFO:tasks.workunit.client.0.smithi115.stdout:5/677: rename d2/f1f to d2/df/d13/d8b/d93/da6/dbc/d50/db1/fe1 0 2024-03-22T10:42:49.789 INFO:tasks.workunit.client.0.smithi115.stdout:4/662: dwrite d1/d66/f77 [4194304,4194304] 0 2024-03-22T10:42:49.790 INFO:tasks.workunit.client.0.smithi115.stdout:3/658: write d3/d82/d52/fc4 [841027,112584] 0 2024-03-22T10:42:49.791 INFO:tasks.workunit.client.0.smithi115.stdout:9/680: fsync d6/d7/d30/d37/dbe/fc6 0 2024-03-22T10:42:49.792 INFO:tasks.workunit.client.1.smithi150.stdout:3/279: readlink d5/l2c 0 2024-03-22T10:42:49.793 INFO:tasks.workunit.client.1.smithi150.stdout:7/293: creat d2/d5/d7/d8/d46/f66 x:0 0 0 2024-03-22T10:42:49.796 INFO:tasks.workunit.client.0.smithi115.stdout:7/677: mknod d2/d7/d12/d10/d23/d3d/da4/cea 0 2024-03-22T10:42:49.800 INFO:tasks.workunit.client.0.smithi115.stdout:5/678: write d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f69 [2633381,117323] 0 2024-03-22T10:42:49.807 INFO:tasks.workunit.client.0.smithi115.stdout:9/681: fsync d6/d7/d48/fc2 0 2024-03-22T10:42:49.807 INFO:tasks.workunit.client.0.smithi115.stdout:0/613: dread d3/d7/d1b/d35/f80 [0,4194304] 0 2024-03-22T10:42:49.807 INFO:tasks.workunit.client.0.smithi115.stdout:5/679: chown d2/d15/d1b/d7c/d38/d43/fca 110 1 2024-03-22T10:42:49.812 INFO:tasks.workunit.client.0.smithi115.stdout:1/735: dread d0/d8/db/d25/f73 [0,4194304] 0 2024-03-22T10:42:49.821 INFO:tasks.workunit.client.0.smithi115.stdout:3/659: fdatasync d3/d19/d2b/d49/f60 0 2024-03-22T10:42:49.821 INFO:tasks.workunit.client.0.smithi115.stdout:4/663: truncate d1/d30/d4c/d86/d9f/d32/f60 5608006 0 2024-03-22T10:42:49.821 INFO:tasks.workunit.client.0.smithi115.stdout:7/678: mknod d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/ceb 0 2024-03-22T10:42:49.821 INFO:tasks.workunit.client.0.smithi115.stdout:2/599: dwrite da/d25/d3d/d57/d7d/f7f [0,4194304] 0 2024-03-22T10:42:49.821 INFO:tasks.workunit.client.0.smithi115.stdout:6/608: dwrite d4/d11/d12/f80 [0,4194304] 0 2024-03-22T10:42:49.834 INFO:tasks.workunit.client.0.smithi115.stdout:5/680: mknod d2/df/d13/d8b/d93/da6/ce2 0 2024-03-22T10:42:49.834 INFO:tasks.workunit.client.0.smithi115.stdout:9/682: creat d6/d7/d48/d70/da9/dbd/ff3 x:0 0 0 2024-03-22T10:42:49.837 INFO:tasks.workunit.client.0.smithi115.stdout:5/681: truncate d2/d15/d1b/d7c/d38/d4a/f75 1283701 0 2024-03-22T10:42:49.838 INFO:tasks.workunit.client.0.smithi115.stdout:6/609: dread d4/d11/d12/f80 [0,4194304] 0 2024-03-22T10:42:49.839 INFO:tasks.workunit.client.0.smithi115.stdout:0/614: creat d3/d7/d1b/d1f/d78/dc1/fc7 x:0 0 0 2024-03-22T10:42:49.840 INFO:tasks.workunit.client.0.smithi115.stdout:4/664: rename d1/dbb/dc6/fe1 to d1/d30/d4c/d86/d9f/fe4 0 2024-03-22T10:42:49.841 INFO:tasks.workunit.client.1.smithi150.stdout:5/199: sync 2024-03-22T10:42:49.847 INFO:tasks.workunit.client.0.smithi115.stdout:7/679: symlink d2/d7/dc6/lec 0 2024-03-22T10:42:49.847 INFO:tasks.workunit.client.0.smithi115.stdout:1/736: creat d0/d8/de/d1a/d91/dc7/ffa x:0 0 0 2024-03-22T10:42:49.850 INFO:tasks.workunit.client.0.smithi115.stdout:9/683: rename d6/d8/d3c/d59/d75 to d6/d7/d48/df4 0 2024-03-22T10:42:49.855 INFO:tasks.workunit.client.0.smithi115.stdout:4/665: unlink d1/f9 0 2024-03-22T10:42:49.857 INFO:tasks.workunit.client.0.smithi115.stdout:8/865: dread d6/d49/d88/fe9 [0,4194304] 0 2024-03-22T10:42:49.857 INFO:tasks.workunit.client.0.smithi115.stdout:0/615: mkdir d3/d7/d1b/d1f/d5b/dba/dc8 0 2024-03-22T10:42:49.858 INFO:tasks.workunit.client.0.smithi115.stdout:1/737: chown d0/d8/db/f2d 16518 1 2024-03-22T10:42:49.859 INFO:tasks.workunit.client.1.smithi150.stdout:8/304: getdents d0/d2 0 2024-03-22T10:42:49.859 INFO:tasks.workunit.client.1.smithi150.stdout:6/230: truncate db/dd/d21/f53 3588937 0 2024-03-22T10:42:49.860 INFO:tasks.workunit.client.1.smithi150.stdout:4/279: truncate d0/de/f4b 2121656 0 2024-03-22T10:42:49.860 INFO:tasks.workunit.client.0.smithi115.stdout:3/660: write d3/d16/d29/f32 [204644,69318] 0 2024-03-22T10:42:49.862 INFO:tasks.workunit.client.0.smithi115.stdout:7/680: rmdir d2/d7/d12/d10/d19/dbd/dce 39 2024-03-22T10:42:49.865 INFO:tasks.workunit.client.0.smithi115.stdout:4/666: creat d1/d16/d28/fe5 x:0 0 0 2024-03-22T10:42:49.865 INFO:tasks.workunit.client.0.smithi115.stdout:6/610: write d4/d36/d48/f62 [944019,26854] 0 2024-03-22T10:42:49.867 INFO:tasks.workunit.client.1.smithi150.stdout:5/200: symlink d0/d8/dc/de/l3b 0 2024-03-22T10:42:49.870 INFO:tasks.workunit.client.0.smithi115.stdout:2/600: dwrite da/d25/d3d/f3c [0,4194304] 0 2024-03-22T10:42:49.870 INFO:tasks.workunit.client.0.smithi115.stdout:0/616: truncate d3/da/d4c/d74/fb6 1022158 0 2024-03-22T10:42:49.871 INFO:tasks.workunit.client.0.smithi115.stdout:5/682: dwrite d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:49.871 INFO:tasks.workunit.client.0.smithi115.stdout:4/667: creat d1/d69/fe6 x:0 0 0 2024-03-22T10:42:49.872 INFO:tasks.workunit.client.1.smithi150.stdout:4/280: getdents d0/d47 0 2024-03-22T10:42:49.875 INFO:tasks.workunit.client.1.smithi150.stdout:2/312: dwrite d12/d19/f1b [0,4194304] 0 2024-03-22T10:42:49.875 INFO:tasks.workunit.client.0.smithi115.stdout:7/681: readlink d2/d7/d12/d10/d23/d3d/dd2/da3/ld8 0 2024-03-22T10:42:49.876 INFO:tasks.workunit.client.1.smithi150.stdout:6/231: getdents db/dd/d15/d28 0 2024-03-22T10:42:49.877 INFO:tasks.workunit.client.1.smithi150.stdout:0/306: dwrite d0/d12/d4f/d11/d66/d35/f62 [0,4194304] 0 2024-03-22T10:42:49.881 INFO:tasks.workunit.client.0.smithi115.stdout:7/682: truncate d2/d7/d12/d10/d23/f64 1948949 0 2024-03-22T10:42:49.881 INFO:tasks.workunit.client.0.smithi115.stdout:1/738: creat d0/d12/ffb x:0 0 0 2024-03-22T10:42:49.882 INFO:tasks.workunit.client.0.smithi115.stdout:8/866: symlink d6/dd/d24/d4a/d7b/d8f/dd6/ddb/l124 0 2024-03-22T10:42:49.883 INFO:tasks.workunit.client.1.smithi150.stdout:8/305: getdents d0/d2/d19/d29/d2f/df/d1b/d3d 0 2024-03-22T10:42:49.885 INFO:tasks.workunit.client.0.smithi115.stdout:2/601: mkdir da/d1c/d67/d78/db2 0 2024-03-22T10:42:49.886 INFO:tasks.workunit.client.1.smithi150.stdout:3/280: dread d5/fb [0,4194304] 0 2024-03-22T10:42:49.888 INFO:tasks.workunit.client.1.smithi150.stdout:6/232: write db/dd/f37 [3056968,117230] 0 2024-03-22T10:42:49.891 INFO:tasks.workunit.client.0.smithi115.stdout:4/668: mkdir d1/d30/d4c/d86/d9f/db8/de7 0 2024-03-22T10:42:49.896 INFO:tasks.workunit.client.0.smithi115.stdout:5/683: getdents d2/df 0 2024-03-22T10:42:49.897 INFO:tasks.workunit.client.0.smithi115.stdout:7/683: mknod d2/d7/d12/d10/d23/d3d/ced 0 2024-03-22T10:42:49.902 INFO:tasks.workunit.client.0.smithi115.stdout:3/661: dwrite d3/d19/d2b/d49/fc6 [0,4194304] 0 2024-03-22T10:42:49.907 INFO:tasks.workunit.client.0.smithi115.stdout:1/739: mkdir d0/d12/d2b/dfc 0 2024-03-22T10:42:49.909 INFO:tasks.workunit.client.0.smithi115.stdout:3/662: stat d3/d82/l81 0 2024-03-22T10:42:49.910 INFO:tasks.workunit.client.0.smithi115.stdout:9/684: dwrite d6/d7/d48/d70/da9/fac [0,4194304] 0 2024-03-22T10:42:49.912 INFO:tasks.workunit.client.0.smithi115.stdout:2/602: mkdir da/d25/d3d/dad/db3 0 2024-03-22T10:42:49.918 INFO:tasks.workunit.client.0.smithi115.stdout:6/611: dwrite d4/fc1 [0,4194304] 0 2024-03-22T10:42:49.919 INFO:tasks.workunit.client.1.smithi150.stdout:7/294: dwrite d2/d5/d7/d8/f5e [0,4194304] 0 2024-03-22T10:42:49.920 INFO:tasks.workunit.client.0.smithi115.stdout:5/684: creat d2/df/d13/d8b/d93/da6/dc3/fe3 x:0 0 0 2024-03-22T10:42:49.921 INFO:tasks.workunit.client.0.smithi115.stdout:2/603: chown da/d1c/d67/d78/db2 56 1 2024-03-22T10:42:49.921 INFO:tasks.workunit.client.0.smithi115.stdout:7/684: truncate d2/d7/d12/d10/d23/d3d/dd2/da3/fda 4566976 0 2024-03-22T10:42:49.922 INFO:tasks.workunit.client.0.smithi115.stdout:8/867: link d6/dd/d12/d31/fdf d6/dd/d12/f125 0 2024-03-22T10:42:49.925 INFO:tasks.workunit.client.0.smithi115.stdout:0/617: dwrite d3/f7f [0,4194304] 0 2024-03-22T10:42:49.925 INFO:tasks.workunit.client.1.smithi150.stdout:5/201: dread d0/f22 [0,4194304] 0 2024-03-22T10:42:49.927 INFO:tasks.workunit.client.1.smithi150.stdout:3/281: creat d5/d53/f54 x:0 0 0 2024-03-22T10:42:49.930 INFO:tasks.workunit.client.0.smithi115.stdout:1/740: dread - d0/d8/d4d/d56/db6/ddc/fbe zero size 2024-03-22T10:42:49.930 INFO:tasks.workunit.client.0.smithi115.stdout:2/604: dread - da/d25/d30/d4f/d54/fab zero size 2024-03-22T10:42:49.930 INFO:tasks.workunit.client.1.smithi150.stdout:5/202: fdatasync d0/f22 0 2024-03-22T10:42:49.932 INFO:tasks.workunit.client.1.smithi150.stdout:7/295: getdents d2/d5/d7/d8/d2c/d45 0 2024-03-22T10:42:49.935 INFO:tasks.workunit.client.1.smithi150.stdout:0/307: dwrite d0/d12/d4f/d11/f5d [0,4194304] 0 2024-03-22T10:42:49.935 INFO:tasks.workunit.client.1.smithi150.stdout:4/281: dwrite d0/d32/f56 [0,4194304] 0 2024-03-22T10:42:49.935 INFO:tasks.workunit.client.0.smithi115.stdout:7/685: write d2/d7/d12/d10/d19/d42/fca [321217,92803] 0 2024-03-22T10:42:49.936 INFO:tasks.workunit.client.1.smithi150.stdout:5/203: read d0/d8/dc/de/f15 [2425312,74746] 0 2024-03-22T10:42:49.941 INFO:tasks.workunit.client.0.smithi115.stdout:3/663: creat d3/d36/d41/fde x:0 0 0 2024-03-22T10:42:49.941 INFO:tasks.workunit.client.0.smithi115.stdout:0/618: symlink d3/d7/d1b/d1f/d5b/lc9 0 2024-03-22T10:42:49.943 INFO:tasks.workunit.client.1.smithi150.stdout:7/296: unlink d2/d5/d7/d8/d2c/c5d 0 2024-03-22T10:42:49.944 INFO:tasks.workunit.client.1.smithi150.stdout:3/282: rename d5/d22/d37/f3d to d5/d22/d2f/d38/d3e/f55 0 2024-03-22T10:42:49.945 INFO:tasks.workunit.client.0.smithi115.stdout:2/605: mkdir da/d1c/d67/d78/d86/db4 0 2024-03-22T10:42:49.946 INFO:tasks.workunit.client.0.smithi115.stdout:1/741: symlink d0/d68/d81/lfd 0 2024-03-22T10:42:49.947 INFO:tasks.workunit.client.1.smithi150.stdout:4/282: creat d0/de/f5b x:0 0 0 2024-03-22T10:42:49.948 INFO:tasks.workunit.client.0.smithi115.stdout:8/868: symlink d6/dd/d24/d4a/dfc/d102/l126 0 2024-03-22T10:42:49.948 INFO:tasks.workunit.client.0.smithi115.stdout:2/606: chown da/d25/d83/c8e 16945 1 2024-03-22T10:42:49.948 INFO:tasks.workunit.client.0.smithi115.stdout:5/685: rename d2/df/d13/d8b/d93/da6/dbc/d50/l91 to d2/df/d13/d8b/d93/da6/dbc/d50/d96/le4 0 2024-03-22T10:42:49.950 INFO:tasks.workunit.client.0.smithi115.stdout:0/619: rmdir d3/da/d4c/d69/d6a 39 2024-03-22T10:42:49.951 INFO:tasks.workunit.client.1.smithi150.stdout:3/283: symlink d5/d22/d2f/d38/d3e/l56 0 2024-03-22T10:42:49.952 INFO:tasks.workunit.client.1.smithi150.stdout:0/308: symlink d0/d12/d4f/d5a/l6a 0 2024-03-22T10:42:49.953 INFO:tasks.workunit.client.0.smithi115.stdout:6/612: getdents d4/d11/d12/d1c/d8a/d98 0 2024-03-22T10:42:49.954 INFO:tasks.workunit.client.1.smithi150.stdout:4/283: stat d0/d2f/l34 0 2024-03-22T10:42:49.956 INFO:tasks.workunit.client.0.smithi115.stdout:5/686: fdatasync d2/df/d13/f76 0 2024-03-22T10:42:49.957 INFO:tasks.workunit.client.1.smithi150.stdout:5/204: link d0/d8/l1c d0/d37/l3c 0 2024-03-22T10:42:49.958 INFO:tasks.workunit.client.1.smithi150.stdout:0/309: creat d0/d12/d4f/f6b x:0 0 0 2024-03-22T10:42:49.962 INFO:tasks.workunit.client.0.smithi115.stdout:0/620: unlink d3/d7/d1b/d1f/f58 0 2024-03-22T10:42:49.963 INFO:tasks.workunit.client.0.smithi115.stdout:8/869: rmdir d6/dd/d24/d4a/d7b/d8f/dd6/ddb 39 2024-03-22T10:42:49.965 INFO:tasks.workunit.client.1.smithi150.stdout:3/284: creat d5/da/f57 x:0 0 0 2024-03-22T10:42:49.967 INFO:tasks.workunit.client.0.smithi115.stdout:1/742: creat d0/d12/d7b/de2/df8/ffe x:0 0 0 2024-03-22T10:42:49.968 INFO:tasks.workunit.client.1.smithi150.stdout:5/205: creat d0/d26/f3d x:0 0 0 2024-03-22T10:42:49.970 INFO:tasks.workunit.client.1.smithi150.stdout:7/297: getdents d2/d5/d7 0 2024-03-22T10:42:49.975 INFO:tasks.workunit.client.0.smithi115.stdout:5/687: creat d2/df/d13/d8b/d93/da6/fe5 x:0 0 0 2024-03-22T10:42:49.976 INFO:tasks.workunit.client.1.smithi150.stdout:7/298: write d2/d5/d7/d8/d57/f5b [647541,121835] 0 2024-03-22T10:42:49.977 INFO:tasks.workunit.client.0.smithi115.stdout:5/688: write d2/f71 [1522305,88928] 0 2024-03-22T10:42:49.979 INFO:tasks.workunit.client.1.smithi150.stdout:9/318: sync 2024-03-22T10:42:49.980 INFO:tasks.workunit.client.0.smithi115.stdout:0/621: rename d3/d7/d1b/d1f/d5b/dba/dc8 to d3/d7/d1b/d1f/d5b/dba/dca 0 2024-03-22T10:42:49.980 INFO:tasks.workunit.client.0.smithi115.stdout:4/669: dread d1/d66/f6d [0,4194304] 0 2024-03-22T10:42:49.980 INFO:tasks.workunit.client.1.smithi150.stdout:0/310: dread d0/d12/d4f/f31 [4194304,4194304] 0 2024-03-22T10:42:49.984 INFO:tasks.workunit.client.1.smithi150.stdout:5/206: creat d0/d8/dc/de/d14/d2e/f3e x:0 0 0 2024-03-22T10:42:49.984 INFO:tasks.workunit.client.0.smithi115.stdout:2/607: dwrite da/d7b/f87 [0,4194304] 0 2024-03-22T10:42:49.985 INFO:tasks.workunit.client.0.smithi115.stdout:1/743: mkdir d0/d12/d17/d37/d9e/da0/dff 0 2024-03-22T10:42:49.989 INFO:tasks.workunit.client.1.smithi150.stdout:4/284: dread d0/d1b/f2b [0,4194304] 0 2024-03-22T10:42:49.989 INFO:tasks.workunit.client.0.smithi115.stdout:1/744: chown d0/d12/d17/d37/l5b 107 1 2024-03-22T10:42:49.989 INFO:tasks.workunit.client.0.smithi115.stdout:6/613: rename d4/d11/f78 to d4/d36/d48/d72/daa/fc4 0 2024-03-22T10:42:49.991 INFO:tasks.workunit.client.0.smithi115.stdout:7/686: getdents d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9 0 2024-03-22T10:42:49.991 INFO:tasks.workunit.client.1.smithi150.stdout:5/207: write d0/d8/dc/de/d14/d2e/f3e [521155,206] 0 2024-03-22T10:42:49.993 INFO:tasks.workunit.client.0.smithi115.stdout:9/685: dwrite d6/d5e/f60 [0,4194304] 0 2024-03-22T10:42:49.995 INFO:tasks.workunit.client.1.smithi150.stdout:3/285: dwrite d5/f35 [0,4194304] 0 2024-03-22T10:42:50.000 INFO:tasks.workunit.client.0.smithi115.stdout:2/608: mknod da/d25/d30/d4f/d54/cb5 0 2024-03-22T10:42:50.001 INFO:tasks.workunit.client.1.smithi150.stdout:9/319: creat d1/f7b x:0 0 0 2024-03-22T10:42:50.003 INFO:tasks.workunit.client.0.smithi115.stdout:6/614: chown d4/d11/f2a 720 1 2024-03-22T10:42:50.004 INFO:tasks.workunit.client.1.smithi150.stdout:4/285: mknod d0/c5c 0 2024-03-22T10:42:50.005 INFO:tasks.workunit.client.0.smithi115.stdout:1/745: fsync d0/d22/d89/fba 0 2024-03-22T10:42:50.006 INFO:tasks.workunit.client.0.smithi115.stdout:6/615: read d4/d13/f25 [167283,99894] 0 2024-03-22T10:42:50.008 INFO:tasks.workunit.client.0.smithi115.stdout:1/746: fdatasync d0/d12/d17/d37/db5/fd9 0 2024-03-22T10:42:50.010 INFO:tasks.workunit.client.1.smithi150.stdout:5/208: mkdir d0/d8/dc/de/d18/d3f 0 2024-03-22T10:42:50.010 INFO:tasks.workunit.client.1.smithi150.stdout:9/320: fsync d1/d8/de/df/d20/d32/f5e 0 2024-03-22T10:42:50.013 INFO:tasks.workunit.client.0.smithi115.stdout:8/870: write d6/dd/d12/d31/fdf [364271,94171] 0 2024-03-22T10:42:50.014 INFO:tasks.workunit.client.1.smithi150.stdout:7/299: getdents d2/d5/d7/d9/db 0 2024-03-22T10:42:50.015 INFO:tasks.workunit.client.1.smithi150.stdout:9/321: write d1/d8/de/df/f5b [69558,52557] 0 2024-03-22T10:42:50.016 INFO:tasks.workunit.client.0.smithi115.stdout:5/689: mknod d2/d15/d1b/ce6 0 2024-03-22T10:42:50.016 INFO:tasks.workunit.client.0.smithi115.stdout:2/609: fdatasync da/d1c/f52 0 2024-03-22T10:42:50.019 INFO:tasks.workunit.client.1.smithi150.stdout:9/322: dread - d1/d8/de/d73/f74 zero size 2024-03-22T10:42:50.019 INFO:tasks.workunit.client.1.smithi150.stdout:4/286: mknod d0/de/d57/c5d 0 2024-03-22T10:42:50.020 INFO:tasks.workunit.client.1.smithi150.stdout:0/311: dwrite d0/d12/d4f/d42/f4a [0,4194304] 0 2024-03-22T10:42:50.023 INFO:tasks.workunit.client.1.smithi150.stdout:8/306: dread d0/d2/d19/d29/d2f/f3b [0,4194304] 0 2024-03-22T10:42:50.025 INFO:tasks.workunit.client.0.smithi115.stdout:7/687: creat d2/d7/d12/d10/d19/dbd/dce/fee x:0 0 0 2024-03-22T10:42:50.026 INFO:tasks.workunit.client.0.smithi115.stdout:4/670: rmdir d1/d30/d4c/d86/d9f/d4f/d93/dc7 0 2024-03-22T10:42:50.026 INFO:tasks.workunit.client.0.smithi115.stdout:1/747: unlink d0/d58/l6a 0 2024-03-22T10:42:50.027 INFO:tasks.workunit.client.1.smithi150.stdout:7/300: unlink d2/d5/d7/d8/d57/f59 0 2024-03-22T10:42:50.028 INFO:tasks.workunit.client.1.smithi150.stdout:1/262: sync 2024-03-22T10:42:50.028 INFO:tasks.workunit.client.1.smithi150.stdout:0/312: truncate d0/d12/d4f/f6b 219984 0 2024-03-22T10:42:50.029 INFO:tasks.workunit.client.0.smithi115.stdout:8/871: creat d6/dd/d12/dc6/dfd/f127 x:0 0 0 2024-03-22T10:42:50.029 INFO:tasks.workunit.client.0.smithi115.stdout:7/688: fdatasync d2/d7/d12/d10/d19/dbd/fdb 0 2024-03-22T10:42:50.030 INFO:tasks.workunit.client.1.smithi150.stdout:8/307: fsync d0/d2/d19/d29/d2f/df/f45 0 2024-03-22T10:42:50.033 INFO:tasks.workunit.client.1.smithi150.stdout:9/323: creat d1/d8/de/df/d36/f7c x:0 0 0 2024-03-22T10:42:50.033 INFO:tasks.workunit.client.0.smithi115.stdout:0/622: truncate d3/da/d4c/f91 903418 0 2024-03-22T10:42:50.034 INFO:tasks.workunit.client.0.smithi115.stdout:9/686: rename d6/d8/d3c/d59/d68/d97/da4/fb7 to d6/d8/d3c/ff5 0 2024-03-22T10:42:50.035 INFO:tasks.workunit.client.1.smithi150.stdout:9/324: dread - d1/d8/de/df/d36/f7c zero size 2024-03-22T10:42:50.035 INFO:tasks.workunit.client.1.smithi150.stdout:1/263: unlink d1/da/fd 0 2024-03-22T10:42:50.036 INFO:tasks.workunit.client.1.smithi150.stdout:3/286: truncate d5/d22/d2f/d38/d3e/f3f 3183484 0 2024-03-22T10:42:50.036 INFO:tasks.workunit.client.1.smithi150.stdout:1/264: chown d1/d19 0 1 2024-03-22T10:42:50.037 INFO:tasks.workunit.client.1.smithi150.stdout:8/308: mkdir d0/d2/d19/d29/d69 0 2024-03-22T10:42:50.038 INFO:tasks.workunit.client.1.smithi150.stdout:3/287: chown c4 203 1 2024-03-22T10:42:50.039 INFO:tasks.workunit.client.0.smithi115.stdout:8/872: mknod d6/dd/d12/d31/c128 0 2024-03-22T10:42:50.040 INFO:tasks.workunit.client.1.smithi150.stdout:5/209: truncate d0/d8/dc/de/f13 2508752 0 2024-03-22T10:42:50.041 INFO:tasks.workunit.client.1.smithi150.stdout:3/288: read d5/f35 [1940208,81411] 0 2024-03-22T10:42:50.043 INFO:tasks.workunit.client.1.smithi150.stdout:0/313: symlink d0/d1/l6c 0 2024-03-22T10:42:50.044 INFO:tasks.workunit.client.0.smithi115.stdout:9/687: truncate d6/d8/d3c/d59/fd8 520881 0 2024-03-22T10:42:50.044 INFO:tasks.workunit.client.1.smithi150.stdout:9/325: mkdir d1/d19/d7d 0 2024-03-22T10:42:50.045 INFO:tasks.workunit.client.1.smithi150.stdout:1/265: fdatasync d1/da/f42 0 2024-03-22T10:42:50.046 INFO:tasks.workunit.client.0.smithi115.stdout:7/689: mkdir d2/d7/d12/d10/d19/d42/d76/def 0 2024-03-22T10:42:50.047 INFO:tasks.workunit.client.1.smithi150.stdout:3/289: write d5/d22/d2f/d38/f39 [317800,3810] 0 2024-03-22T10:42:50.047 INFO:tasks.workunit.client.1.smithi150.stdout:8/309: symlink d0/d2/d4d/l6a 0 2024-03-22T10:42:50.048 INFO:tasks.workunit.client.1.smithi150.stdout:7/301: dwrite d2/d5/f52 [0,4194304] 0 2024-03-22T10:42:50.049 INFO:tasks.workunit.client.0.smithi115.stdout:1/748: dwrite d0/d8/db/d25/fa7 [0,4194304] 0 2024-03-22T10:42:50.049 INFO:tasks.workunit.client.0.smithi115.stdout:0/623: rename d3/d4/l65 to d3/d4/d14/d64/lcb 0 2024-03-22T10:42:50.051 INFO:tasks.workunit.client.0.smithi115.stdout:2/610: creat da/d25/fb6 x:0 0 0 2024-03-22T10:42:50.051 INFO:tasks.workunit.client.1.smithi150.stdout:8/310: write d0/d3c/f68 [950087,70336] 0 2024-03-22T10:42:50.055 INFO:tasks.workunit.client.0.smithi115.stdout:9/688: chown d6/fcf 230228265 1 2024-03-22T10:42:50.055 INFO:tasks.workunit.client.1.smithi150.stdout:5/210: creat d0/d8/dc/f40 x:0 0 0 2024-03-22T10:42:50.056 INFO:tasks.workunit.client.0.smithi115.stdout:9/689: chown d6/f12 316 1 2024-03-22T10:42:50.056 INFO:tasks.workunit.client.0.smithi115.stdout:1/749: dread - d0/d8/de/d1a/d91/dc7/ffa zero size 2024-03-22T10:42:50.060 INFO:tasks.workunit.client.0.smithi115.stdout:6/616: dwrite d4/f30 [0,4194304] 0 2024-03-22T10:42:50.061 INFO:tasks.workunit.client.1.smithi150.stdout:4/287: truncate d0/d42/f48 3425231 0 2024-03-22T10:42:50.062 INFO:tasks.workunit.client.1.smithi150.stdout:9/326: mknod d1/d8/de/df/c7e 0 2024-03-22T10:42:50.065 INFO:tasks.workunit.client.1.smithi150.stdout:9/327: chown d1/d8/de/df/d36/d40/d7a 901992950 1 2024-03-22T10:42:50.066 INFO:tasks.workunit.client.1.smithi150.stdout:7/302: creat d2/f67 x:0 0 0 2024-03-22T10:42:50.068 INFO:tasks.workunit.client.0.smithi115.stdout:9/690: rmdir d6/d7/d20/d38 39 2024-03-22T10:42:50.071 INFO:tasks.workunit.client.0.smithi115.stdout:6/617: creat d4/d11/d12/d1c/d8a/d98/fc5 x:0 0 0 2024-03-22T10:42:50.073 INFO:tasks.workunit.client.0.smithi115.stdout:1/750: rename d0/d8/d4d/f71 to d0/d12/d17/d37/d9e/de9/dec/ded/f100 0 2024-03-22T10:42:50.073 INFO:tasks.workunit.client.1.smithi150.stdout:1/266: dwrite d1/da/d15/d18/f3d [0,4194304] 0 2024-03-22T10:42:50.074 INFO:tasks.workunit.client.0.smithi115.stdout:6/618: fsync d4/d36/d48/f67 0 2024-03-22T10:42:50.078 INFO:tasks.workunit.client.0.smithi115.stdout:7/690: dwrite d2/d7/d12/d10/d19/dbd/dce/fee [0,4194304] 0 2024-03-22T10:42:50.080 INFO:tasks.workunit.client.1.smithi150.stdout:5/211: rename d0/d8/fd to d0/d33/f41 0 2024-03-22T10:42:50.081 INFO:tasks.workunit.client.0.smithi115.stdout:7/691: chown d2/d62/f4f 17495846 1 2024-03-22T10:42:50.081 INFO:tasks.workunit.client.1.smithi150.stdout:4/288: creat d0/de/f5e x:0 0 0 2024-03-22T10:42:50.081 INFO:tasks.workunit.client.0.smithi115.stdout:9/691: creat d6/d8/d3c/d59/d68/ff6 x:0 0 0 2024-03-22T10:42:50.082 INFO:tasks.workunit.client.1.smithi150.stdout:9/328: rmdir d1/d8/de/df/d36/d40 39 2024-03-22T10:42:50.082 INFO:tasks.workunit.client.1.smithi150.stdout:6/233: sync 2024-03-22T10:42:50.082 INFO:tasks.workunit.client.1.smithi150.stdout:2/313: sync 2024-03-22T10:42:50.083 INFO:tasks.workunit.client.1.smithi150.stdout:0/314: creat d0/f6d x:0 0 0 2024-03-22T10:42:50.084 INFO:tasks.workunit.client.1.smithi150.stdout:7/303: creat d2/d5/d7/d8/d46/d49/f68 x:0 0 0 2024-03-22T10:42:50.087 INFO:tasks.workunit.client.0.smithi115.stdout:9/692: chown d6/d8/d3c/c76 0 1 2024-03-22T10:42:50.089 INFO:tasks.workunit.client.0.smithi115.stdout:2/611: link da/d1c/f4c da/d25/d30/d4f/d54/fb7 0 2024-03-22T10:42:50.090 INFO:tasks.workunit.client.0.smithi115.stdout:8/873: truncate d6/dd/d12/f27 4162377 0 2024-03-22T10:42:50.091 INFO:tasks.workunit.client.1.smithi150.stdout:5/212: chown d0/d8/dc/de/l3b 0 1 2024-03-22T10:42:50.091 INFO:tasks.workunit.client.1.smithi150.stdout:4/289: write d0/d32/f41 [695900,130641] 0 2024-03-22T10:42:50.092 INFO:tasks.workunit.client.1.smithi150.stdout:1/267: symlink d1/da/d2a/d4e/l56 0 2024-03-22T10:42:50.092 INFO:tasks.workunit.client.0.smithi115.stdout:1/751: creat d0/d8/d4d/d56/db6/f101 x:0 0 0 2024-03-22T10:42:50.092 INFO:tasks.workunit.client.0.smithi115.stdout:5/690: dwrite d2/d15/d1b/d7c/d38/d43/d49/fa9 [0,4194304] 0 2024-03-22T10:42:50.094 INFO:tasks.workunit.client.1.smithi150.stdout:3/290: getdents d5/d22/d2f/d38 0 2024-03-22T10:42:50.094 INFO:tasks.workunit.client.0.smithi115.stdout:0/624: truncate d3/d4/d6d/fb0 3280698 0 2024-03-22T10:42:50.097 INFO:tasks.workunit.client.0.smithi115.stdout:8/874: write d6/d49/f83 [6229880,89723] 0 2024-03-22T10:42:50.097 INFO:tasks.workunit.client.1.smithi150.stdout:2/314: unlink d12/l1e 0 2024-03-22T10:42:50.099 INFO:tasks.workunit.client.1.smithi150.stdout:0/315: write d0/d12/d4f/f28 [3677164,82127] 0 2024-03-22T10:42:50.099 INFO:tasks.workunit.client.0.smithi115.stdout:0/625: read - d3/d4/d6d/f9e zero size 2024-03-22T10:42:50.100 INFO:tasks.workunit.client.1.smithi150.stdout:2/315: chown d12/d19/l20 94 1 2024-03-22T10:42:50.103 INFO:tasks.workunit.client.0.smithi115.stdout:1/752: symlink d0/d12/d17/d37/d9e/da0/l102 0 2024-03-22T10:42:50.103 INFO:tasks.workunit.client.1.smithi150.stdout:3/291: write d5/da/f57 [151740,112237] 0 2024-03-22T10:42:50.104 INFO:tasks.workunit.client.1.smithi150.stdout:0/316: readlink d0/d12/d4f/d42/d10/l1a 0 2024-03-22T10:42:50.105 INFO:tasks.workunit.client.0.smithi115.stdout:1/753: readlink d0/d8/l69 0 2024-03-22T10:42:50.105 INFO:tasks.workunit.client.0.smithi115.stdout:6/619: link d4/c51 d4/d11/d12/d1c/d8a/cc6 0 2024-03-22T10:42:50.109 INFO:tasks.workunit.client.0.smithi115.stdout:7/692: creat d2/d7/d12/d10/d23/d3d/ff0 x:0 0 0 2024-03-22T10:42:50.110 INFO:tasks.workunit.client.0.smithi115.stdout:8/875: symlink d6/dd/d12/dc6/l129 0 2024-03-22T10:42:50.111 INFO:tasks.workunit.client.1.smithi150.stdout:6/234: dread db/dd/f2c [0,4194304] 0 2024-03-22T10:42:50.112 INFO:tasks.workunit.client.1.smithi150.stdout:8/311: write d0/d2/d19/d29/f63 [3311809,57307] 0 2024-03-22T10:42:50.113 INFO:tasks.workunit.client.0.smithi115.stdout:0/626: unlink d3/d7/d1b/d1f/d78/dc1/fc7 0 2024-03-22T10:42:50.113 INFO:tasks.workunit.client.1.smithi150.stdout:6/235: stat db 0 2024-03-22T10:42:50.116 INFO:tasks.workunit.client.1.smithi150.stdout:9/329: creat d1/d8/de/df/d36/d40/f7f x:0 0 0 2024-03-22T10:42:50.117 INFO:tasks.workunit.client.1.smithi150.stdout:2/316: mknod d12/d19/d2d/c69 0 2024-03-22T10:42:50.117 INFO:tasks.workunit.client.1.smithi150.stdout:8/312: fsync d0/d2/d19/f47 0 2024-03-22T10:42:50.118 INFO:tasks.workunit.client.0.smithi115.stdout:5/691: creat d2/d15/d1b/d7c/fe7 x:0 0 0 2024-03-22T10:42:50.118 INFO:tasks.workunit.client.1.smithi150.stdout:3/292: rmdir d5/d22/d37 39 2024-03-22T10:42:50.119 INFO:tasks.workunit.client.1.smithi150.stdout:0/317: unlink d0/d12/d4f/d42/d10/f47 0 2024-03-22T10:42:50.119 INFO:tasks.workunit.client.1.smithi150.stdout:6/236: fsync db/dd/d15/f35 0 2024-03-22T10:42:50.120 INFO:tasks.workunit.client.0.smithi115.stdout:7/693: mknod d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/cf1 0 2024-03-22T10:42:50.120 INFO:tasks.workunit.client.0.smithi115.stdout:5/692: readlink d2/df/d3b/d8e/l73 0 2024-03-22T10:42:50.121 INFO:tasks.workunit.client.0.smithi115.stdout:1/754: mkdir d0/d12/d2b/d103 0 2024-03-22T10:42:50.122 INFO:tasks.workunit.client.1.smithi150.stdout:5/213: symlink d0/d8/l42 0 2024-03-22T10:42:50.124 INFO:tasks.workunit.client.0.smithi115.stdout:7/694: creat d2/d62/ff2 x:0 0 0 2024-03-22T10:42:50.126 INFO:tasks.workunit.client.0.smithi115.stdout:0/627: dread d3/d4/f88 [0,4194304] 0 2024-03-22T10:42:50.127 INFO:tasks.workunit.client.0.smithi115.stdout:1/755: symlink d0/d8/de/d1a/d91/l104 0 2024-03-22T10:42:50.128 INFO:tasks.workunit.client.1.smithi150.stdout:5/214: write d0/f6 [2480891,12073] 0 2024-03-22T10:42:50.128 INFO:tasks.workunit.client.1.smithi150.stdout:7/304: dwrite d2/d5/d7/d8/d46/f66 [0,4194304] 0 2024-03-22T10:42:50.129 INFO:tasks.workunit.client.0.smithi115.stdout:2/612: write da/d25/d3d/d57/f65 [1082463,84402] 0 2024-03-22T10:42:50.129 INFO:tasks.workunit.client.1.smithi150.stdout:2/317: mknod d12/d16/d2c/c6a 0 2024-03-22T10:42:50.130 INFO:tasks.workunit.client.0.smithi115.stdout:5/693: symlink d2/df/d13/le8 0 2024-03-22T10:42:50.131 INFO:tasks.workunit.client.1.smithi150.stdout:2/318: chown d12/d48/d54 75385 1 2024-03-22T10:42:50.133 INFO:tasks.workunit.client.1.smithi150.stdout:6/237: rmdir db 39 2024-03-22T10:42:50.133 INFO:tasks.workunit.client.1.smithi150.stdout:1/268: dwrite d1/f3a [0,4194304] 0 2024-03-22T10:42:50.134 INFO:tasks.workunit.client.1.smithi150.stdout:7/305: chown d2/d5/d7/d8/d57/c56 75850096 1 2024-03-22T10:42:50.134 INFO:tasks.workunit.client.1.smithi150.stdout:2/319: chown d12/d48/d54/c5d 1 1 2024-03-22T10:42:50.135 INFO:tasks.workunit.client.0.smithi115.stdout:1/756: creat d0/d58/f105 x:0 0 0 2024-03-22T10:42:50.137 INFO:tasks.workunit.client.0.smithi115.stdout:9/693: write d6/d7/d20/d22/d23/fce [1322366,123727] 0 2024-03-22T10:42:50.139 INFO:tasks.workunit.client.1.smithi150.stdout:7/306: write d2/d5/d7/d8/d2c/f47 [2274136,103294] 0 2024-03-22T10:42:50.139 INFO:tasks.workunit.client.1.smithi150.stdout:4/290: getdents d0/d42 0 2024-03-22T10:42:50.142 INFO:tasks.workunit.client.1.smithi150.stdout:4/291: read d0/d1b/f24 [3851600,97212] 0 2024-03-22T10:42:50.146 INFO:tasks.workunit.client.0.smithi115.stdout:7/695: dwrite d2/fb7 [0,4194304] 0 2024-03-22T10:42:50.147 INFO:tasks.workunit.client.1.smithi150.stdout:8/313: rename d0/d2/c4c to d0/d2/c6b 0 2024-03-22T10:42:50.148 INFO:tasks.workunit.client.1.smithi150.stdout:5/215: mknod d0/d33/c43 0 2024-03-22T10:42:50.149 INFO:tasks.workunit.client.0.smithi115.stdout:8/876: write f5 [4546103,54547] 0 2024-03-22T10:42:50.149 INFO:tasks.workunit.client.0.smithi115.stdout:9/694: creat d6/d7/d48/d70/da9/dbd/ff7 x:0 0 0 2024-03-22T10:42:50.152 INFO:tasks.workunit.client.1.smithi150.stdout:3/293: dwrite d5/d22/d2f/d38/f42 [0,4194304] 0 2024-03-22T10:42:50.154 INFO:tasks.workunit.client.0.smithi115.stdout:8/877: stat d6/dd/d12/d31/lb9 0 2024-03-22T10:42:50.155 INFO:tasks.workunit.client.0.smithi115.stdout:0/628: unlink d3/da/d4c/da3/f5e 0 2024-03-22T10:42:50.156 INFO:tasks.workunit.client.1.smithi150.stdout:9/330: unlink d1/d8/de/df/d24/l5a 0 2024-03-22T10:42:50.159 INFO:tasks.workunit.client.1.smithi150.stdout:2/320: creat d12/d16/d67/d3c/f6b x:0 0 0 2024-03-22T10:42:50.160 INFO:tasks.workunit.client.0.smithi115.stdout:0/629: read d3/da/d4c/d74/fb6 [64662,80083] 0 2024-03-22T10:42:50.166 INFO:tasks.workunit.client.0.smithi115.stdout:2/613: dwrite da/d25/d3d/f97 [0,4194304] 0 2024-03-22T10:42:50.167 INFO:tasks.workunit.client.1.smithi150.stdout:4/292: dwrite d0/d1b/f39 [4194304,4194304] 0 2024-03-22T10:42:50.169 INFO:tasks.workunit.client.0.smithi115.stdout:2/614: chown da/d25/d30/d4f/d54/la6 29 1 2024-03-22T10:42:50.171 INFO:tasks.workunit.client.1.smithi150.stdout:4/293: read d0/de/f18 [1209675,102910] 0 2024-03-22T10:42:50.172 INFO:tasks.workunit.client.1.smithi150.stdout:3/294: symlink d5/da/l58 0 2024-03-22T10:42:50.174 INFO:tasks.workunit.client.1.smithi150.stdout:6/238: creat db/dd/d15/d28/f58 x:0 0 0 2024-03-22T10:42:50.177 INFO:tasks.workunit.client.1.smithi150.stdout:8/314: creat d0/d5d/f6c x:0 0 0 2024-03-22T10:42:50.177 INFO:tasks.workunit.client.0.smithi115.stdout:0/630: creat d3/d7/d1b/d93/fcc x:0 0 0 2024-03-22T10:42:50.179 INFO:tasks.workunit.client.1.smithi150.stdout:2/321: creat d12/d48/d63/f6c x:0 0 0 2024-03-22T10:42:50.179 INFO:tasks.workunit.client.1.smithi150.stdout:3/295: mkdir d5/d22/d2f/d38/d59 0 2024-03-22T10:42:50.183 INFO:tasks.workunit.client.0.smithi115.stdout:8/878: dwrite d6/d49/f84 [0,4194304] 0 2024-03-22T10:42:50.183 INFO:tasks.workunit.client.0.smithi115.stdout:2/615: symlink da/db1/lb8 0 2024-03-22T10:42:50.183 INFO:tasks.workunit.client.1.smithi150.stdout:9/331: mknod d1/d8/de/df/d36/d40/d6c/c80 0 2024-03-22T10:42:50.185 INFO:tasks.workunit.client.1.smithi150.stdout:7/307: dwrite d2/d5/d7/d9/f65 [0,4194304] 0 2024-03-22T10:42:50.186 INFO:tasks.workunit.client.0.smithi115.stdout:0/631: write d3/d4/f88 [4923284,54636] 0 2024-03-22T10:42:50.188 INFO:tasks.workunit.client.1.smithi150.stdout:4/294: unlink d0/d32/l58 0 2024-03-22T10:42:50.188 INFO:tasks.workunit.client.1.smithi150.stdout:7/308: dread - d2/f60 zero size 2024-03-22T10:42:50.189 INFO:tasks.workunit.client.0.smithi115.stdout:2/616: truncate da/d25/d30/f6e 889223 0 2024-03-22T10:42:50.194 INFO:tasks.workunit.client.0.smithi115.stdout:8/879: unlink d6/f59 0 2024-03-22T10:42:50.195 INFO:tasks.workunit.client.0.smithi115.stdout:5/694: truncate d2/df/d3b/f7b 3313186 0 2024-03-22T10:42:50.195 INFO:tasks.workunit.client.1.smithi150.stdout:1/269: getdents d1/d19/d1a 0 2024-03-22T10:42:50.199 INFO:tasks.workunit.client.1.smithi150.stdout:8/315: dread - d0/d2/d19/d29/d31/f49 zero size 2024-03-22T10:42:50.199 INFO:tasks.workunit.client.1.smithi150.stdout:2/322: mknod d12/d16/d61/c6d 0 2024-03-22T10:42:50.201 INFO:tasks.workunit.client.0.smithi115.stdout:8/880: symlink d6/dd/d12/d31/l12a 0 2024-03-22T10:42:50.201 INFO:tasks.workunit.client.0.smithi115.stdout:1/757: dwrite d0/f52 [0,4194304] 0 2024-03-22T10:42:50.203 INFO:tasks.workunit.client.0.smithi115.stdout:5/695: creat d2/d15/d1b/d7c/d84/fe9 x:0 0 0 2024-03-22T10:42:50.206 INFO:tasks.workunit.client.0.smithi115.stdout:8/881: chown d6/dd/d24/d4a/dfc/f11c 15277 1 2024-03-22T10:42:50.208 INFO:tasks.workunit.client.1.smithi150.stdout:0/318: truncate d0/d12/d4f/d11/f2f 4292406 0 2024-03-22T10:42:50.209 INFO:tasks.workunit.client.1.smithi150.stdout:6/239: truncate db/dd/d15/f20 4358541 0 2024-03-22T10:42:50.210 INFO:tasks.workunit.client.0.smithi115.stdout:1/758: creat d0/d8/f106 x:0 0 0 2024-03-22T10:42:50.214 INFO:tasks.workunit.client.1.smithi150.stdout:8/316: creat d0/d2/d19/d29/d31/f6d x:0 0 0 2024-03-22T10:42:50.215 INFO:tasks.workunit.client.1.smithi150.stdout:0/319: dread - d0/d12/d4f/d42/d10/d53/f5e zero size 2024-03-22T10:42:50.216 INFO:tasks.workunit.client.0.smithi115.stdout:8/882: write d6/dd/d24/d4a/dfc/f11c [245748,37] 0 2024-03-22T10:42:50.217 INFO:tasks.workunit.client.1.smithi150.stdout:4/295: creat d0/f5f x:0 0 0 2024-03-22T10:42:50.217 INFO:tasks.workunit.client.0.smithi115.stdout:0/632: dwrite d3/d4/f88 [4194304,4194304] 0 2024-03-22T10:42:50.218 INFO:tasks.workunit.client.0.smithi115.stdout:1/759: write d0/d8/d4d/d56/f6d [2767679,74528] 0 2024-03-22T10:42:50.220 INFO:tasks.workunit.client.1.smithi150.stdout:9/332: getdents d1/d8 0 2024-03-22T10:42:50.221 INFO:tasks.workunit.client.1.smithi150.stdout:3/296: getdents d5/d53 0 2024-03-22T10:42:50.221 INFO:tasks.workunit.client.1.smithi150.stdout:4/296: readlink d0/d32/l3f 0 2024-03-22T10:42:50.222 INFO:tasks.workunit.client.0.smithi115.stdout:0/633: mknod d3/ccd 0 2024-03-22T10:42:50.225 INFO:tasks.workunit.client.0.smithi115.stdout:9/695: dread d6/d7/d30/d3b/f46 [0,4194304] 0 2024-03-22T10:42:50.229 INFO:tasks.workunit.client.0.smithi115.stdout:7/696: dread d2/d7/d12/d10/d23/d3d/dd2/da3/de8/fd5 [0,4194304] 0 2024-03-22T10:42:50.231 INFO:tasks.workunit.client.1.smithi150.stdout:3/297: fsync d5/da/f19 0 2024-03-22T10:42:50.233 INFO:tasks.workunit.client.1.smithi150.stdout:5/216: truncate d0/f11 4145054 0 2024-03-22T10:42:50.234 INFO:tasks.workunit.client.0.smithi115.stdout:5/696: dread d2/d15/d1b/d7c/d38/d4a/f75 [0,4194304] 0 2024-03-22T10:42:50.234 INFO:tasks.workunit.client.0.smithi115.stdout:1/760: creat d0/d8/de/f107 x:0 0 0 2024-03-22T10:42:50.235 INFO:tasks.workunit.client.0.smithi115.stdout:9/696: mknod d6/cf8 0 2024-03-22T10:42:50.236 INFO:tasks.workunit.client.0.smithi115.stdout:7/697: mkdir d2/d7/d12/d10/d23/d3d/dd2/da3/df3 0 2024-03-22T10:42:50.237 INFO:tasks.workunit.client.1.smithi150.stdout:9/333: fsync d1/d8/de/df/d36/d40/d41/f47 0 2024-03-22T10:42:50.237 INFO:tasks.workunit.client.0.smithi115.stdout:1/761: chown d0/d12/d2b/fca 4 1 2024-03-22T10:42:50.238 INFO:tasks.workunit.client.1.smithi150.stdout:1/270: dwrite d1/da/f43 [0,4194304] 0 2024-03-22T10:42:50.239 INFO:tasks.workunit.client.0.smithi115.stdout:9/697: write d6/d7/d20/d22/d3f/fab [4359530,22085] 0 2024-03-22T10:42:50.240 INFO:tasks.workunit.client.0.smithi115.stdout:0/634: truncate d3/d7/d1c/f5d 776893 0 2024-03-22T10:42:50.242 INFO:tasks.workunit.client.0.smithi115.stdout:5/697: creat d2/d15/d1b/d7c/fea x:0 0 0 2024-03-22T10:42:50.246 INFO:tasks.workunit.client.1.smithi150.stdout:0/320: readlink d0/d12/d4f/d11/d66/d35/l3c 0 2024-03-22T10:42:50.250 INFO:tasks.workunit.client.0.smithi115.stdout:9/698: symlink d6/d7/d30/d3b/lf9 0 2024-03-22T10:42:50.251 INFO:tasks.workunit.client.1.smithi150.stdout:9/334: write d1/d8/de/df/d36/d40/d41/f47 [2134654,83156] 0 2024-03-22T10:42:50.251 INFO:tasks.workunit.client.1.smithi150.stdout:3/298: getdents d5/d22/d2f/d38/d59 0 2024-03-22T10:42:50.251 INFO:tasks.workunit.client.0.smithi115.stdout:5/698: stat d2/df/d13/c3f 0 2024-03-22T10:42:50.252 INFO:tasks.workunit.client.0.smithi115.stdout:7/698: mknod d2/d7/d12/d10/d19/d42/d46/d94/cf4 0 2024-03-22T10:42:50.254 INFO:tasks.workunit.client.0.smithi115.stdout:9/699: write d6/d7/d19/f25 [699125,4339] 0 2024-03-22T10:42:50.255 INFO:tasks.workunit.client.0.smithi115.stdout:5/699: chown d2/df/d3b/f92 18 1 2024-03-22T10:42:50.256 INFO:tasks.workunit.client.1.smithi150.stdout:6/240: dwrite db/dd/d25/f34 [0,4194304] 0 2024-03-22T10:42:50.256 INFO:tasks.workunit.client.1.smithi150.stdout:8/317: dwrite d0/f5a [0,4194304] 0 2024-03-22T10:42:50.256 INFO:tasks.workunit.client.1.smithi150.stdout:4/297: rename d0/d1b/f37 to d0/d1b/f60 0 2024-03-22T10:42:50.259 INFO:tasks.workunit.client.0.smithi115.stdout:5/700: readlink d2/d15/db5/lba 0 2024-03-22T10:42:50.263 INFO:tasks.workunit.client.0.smithi115.stdout:5/701: stat d2/df/d13/d8b/d93/da6/dbc/d50/f86 0 2024-03-22T10:42:50.265 INFO:tasks.workunit.client.0.smithi115.stdout:2/617: dread da/d1c/d67/f6f [0,4194304] 0 2024-03-22T10:42:50.270 INFO:tasks.workunit.client.0.smithi115.stdout:8/883: write d6/dd/d24/d3a/f3c [2956717,65327] 0 2024-03-22T10:42:50.271 INFO:tasks.workunit.client.1.smithi150.stdout:4/298: rename d0/d42 to d0/de/d57/d61 0 2024-03-22T10:42:50.273 INFO:tasks.workunit.client.1.smithi150.stdout:2/323: truncate d12/f15 181515 0 2024-03-22T10:42:50.275 INFO:tasks.workunit.client.1.smithi150.stdout:1/271: dwrite d1/d19/f4c [4194304,4194304] 0 2024-03-22T10:42:50.277 INFO:tasks.workunit.client.1.smithi150.stdout:2/324: dread - d12/d48/f4d zero size 2024-03-22T10:42:50.278 INFO:tasks.workunit.client.0.smithi115.stdout:8/884: write d6/d49/d88/fef [511716,21134] 0 2024-03-22T10:42:50.279 INFO:tasks.workunit.client.1.smithi150.stdout:5/217: dwrite d0/d8/f23 [0,4194304] 0 2024-03-22T10:42:50.283 INFO:tasks.workunit.client.1.smithi150.stdout:0/321: mknod d0/d12/d4f/d42/d10/c6e 0 2024-03-22T10:42:50.285 INFO:tasks.workunit.client.1.smithi150.stdout:3/299: symlink d5/d22/d37/l5a 0 2024-03-22T10:42:50.289 INFO:tasks.workunit.client.0.smithi115.stdout:0/635: dread d3/da/d4c/d69/d6a/fa4 [0,4194304] 0 2024-03-22T10:42:50.294 INFO:tasks.workunit.client.1.smithi150.stdout:8/318: mknod d0/d2/d4d/c6e 0 2024-03-22T10:42:50.294 INFO:tasks.workunit.client.1.smithi150.stdout:6/241: dread db/dd/d15/d30/f51 [0,4194304] 0 2024-03-22T10:42:50.297 INFO:tasks.workunit.client.1.smithi150.stdout:8/319: dread - d0/d2/d19/d29/d31/f49 zero size 2024-03-22T10:42:50.297 INFO:tasks.workunit.client.1.smithi150.stdout:4/299: chown d0/de/c59 5 1 2024-03-22T10:42:50.298 INFO:tasks.workunit.client.1.smithi150.stdout:6/242: write db/dd/f37 [442583,27283] 0 2024-03-22T10:42:50.298 INFO:tasks.workunit.client.1.smithi150.stdout:9/335: write d1/d8/de/df/f4d [612315,96879] 0 2024-03-22T10:42:50.299 INFO:tasks.workunit.client.0.smithi115.stdout:1/762: mknod d0/d12/d17/d37/d9e/da0/db9/c108 0 2024-03-22T10:42:50.300 INFO:tasks.workunit.client.0.smithi115.stdout:2/618: dread da/d25/f34 [0,4194304] 0 2024-03-22T10:42:50.302 INFO:tasks.workunit.client.1.smithi150.stdout:6/243: write db/f4f [324882,43985] 0 2024-03-22T10:42:50.303 INFO:tasks.workunit.client.1.smithi150.stdout:2/325: rename d12/c47 to d12/d21/c6e 0 2024-03-22T10:42:50.303 INFO:tasks.workunit.client.0.smithi115.stdout:2/619: dread - da/d25/d83/f8a zero size 2024-03-22T10:42:50.304 INFO:tasks.workunit.client.0.smithi115.stdout:5/702: unlink d2/df/d13/d8b/d93/da6/ce2 0 2024-03-22T10:42:50.304 INFO:tasks.workunit.client.0.smithi115.stdout:9/700: mkdir d6/d7/d48/df4/dfa 0 2024-03-22T10:42:50.308 INFO:tasks.workunit.client.0.smithi115.stdout:8/885: rename d6/d49/d88/c113 to d6/dd/d24/d4a/c12b 0 2024-03-22T10:42:50.309 INFO:tasks.workunit.client.0.smithi115.stdout:0/636: mkdir d3/d7/d1b/d1f/d5b/dba/dca/dce 0 2024-03-22T10:42:50.312 INFO:tasks.workunit.client.0.smithi115.stdout:7/699: link d2/d7/d12/d10/d23/f38 d2/d7/d12/d10/d23/d3d/dd2/ff5 0 2024-03-22T10:42:50.314 INFO:tasks.workunit.client.1.smithi150.stdout:8/320: dread - d0/d2/d46/f4e zero size 2024-03-22T10:42:50.316 INFO:tasks.workunit.client.1.smithi150.stdout:1/272: mkdir d1/d19/d57 0 2024-03-22T10:42:50.317 INFO:tasks.workunit.client.0.smithi115.stdout:1/763: mkdir d0/d12/d17/d37/d9e/da0/d109 0 2024-03-22T10:42:50.318 INFO:tasks.workunit.client.1.smithi150.stdout:8/321: write d0/d2/d19/f47 [1549757,120963] 0 2024-03-22T10:42:50.321 INFO:tasks.workunit.client.0.smithi115.stdout:5/703: creat d2/d15/d1b/feb x:0 0 0 2024-03-22T10:42:50.322 INFO:tasks.workunit.client.1.smithi150.stdout:9/336: dwrite d1/d8/de/df/d36/f43 [0,4194304] 0 2024-03-22T10:42:50.324 INFO:tasks.workunit.client.1.smithi150.stdout:2/326: rmdir d12/d35 39 2024-03-22T10:42:50.325 INFO:tasks.workunit.client.1.smithi150.stdout:9/337: dread - d1/d8/f65 zero size 2024-03-22T10:42:50.326 INFO:tasks.workunit.client.0.smithi115.stdout:7/700: link d2/d7/d12/fdc d2/d7/d12/d10/d23/d3d/ff6 0 2024-03-22T10:42:50.327 INFO:tasks.workunit.client.1.smithi150.stdout:6/244: symlink db/dd/d15/d28/l59 0 2024-03-22T10:42:50.331 INFO:tasks.workunit.client.0.smithi115.stdout:0/637: link d3/l4e d3/da/d4c/d69/d6a/lcf 0 2024-03-22T10:42:50.332 INFO:tasks.workunit.client.1.smithi150.stdout:4/300: dread d0/d2f/f3e [0,4194304] 0 2024-03-22T10:42:50.334 INFO:tasks.workunit.client.1.smithi150.stdout:0/322: dread d0/d12/d4f/d11/d66/d37/f3d [0,4194304] 0 2024-03-22T10:42:50.334 INFO:tasks.workunit.client.0.smithi115.stdout:7/701: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/ff7 x:0 0 0 2024-03-22T10:42:50.335 INFO:tasks.workunit.client.0.smithi115.stdout:5/704: getdents d2/df/d13/d8b/d93/da6/dbc/d7a 0 2024-03-22T10:42:50.336 INFO:tasks.workunit.client.0.smithi115.stdout:5/705: readlink d2/df/d3b/l31 0 2024-03-22T10:42:50.337 INFO:tasks.workunit.client.1.smithi150.stdout:0/323: write d0/f6d [81866,44187] 0 2024-03-22T10:42:50.337 INFO:tasks.workunit.client.1.smithi150.stdout:6/245: creat db/dd/d2e/f5a x:0 0 0 2024-03-22T10:42:50.340 INFO:tasks.workunit.client.0.smithi115.stdout:5/706: fsync d2/d15/d1b/d7c/d38/d43/d49/fb9 0 2024-03-22T10:42:50.341 INFO:tasks.workunit.client.1.smithi150.stdout:1/273: symlink d1/da/d2a/d55/l58 0 2024-03-22T10:42:50.342 INFO:tasks.workunit.client.0.smithi115.stdout:5/707: chown d2/df/d13/d8b/d93/da6/dbc/d50 2 1 2024-03-22T10:42:50.343 INFO:tasks.workunit.client.0.smithi115.stdout:5/708: chown d2/c62 644019 1 2024-03-22T10:42:50.344 INFO:tasks.workunit.client.1.smithi150.stdout:6/246: creat db/dd/f5b x:0 0 0 2024-03-22T10:42:50.345 INFO:tasks.workunit.client.0.smithi115.stdout:7/702: unlink d2/d7/d12/d10/d23/f84 0 2024-03-22T10:42:50.345 INFO:tasks.workunit.client.1.smithi150.stdout:0/324: rmdir d0/d12/d4f/d42/d10 39 2024-03-22T10:42:50.349 INFO:tasks.workunit.client.1.smithi150.stdout:7/309: sync 2024-03-22T10:42:50.349 INFO:tasks.workunit.client.1.smithi150.stdout:9/338: dwrite d1/d8/de/df/d36/f43 [0,4194304] 0 2024-03-22T10:42:50.350 INFO:tasks.workunit.client.0.smithi115.stdout:7/703: symlink d2/d7/d12/d10/d23/d3d/dd2/da3/de8/dc7/lf8 0 2024-03-22T10:42:50.350 INFO:tasks.workunit.client.1.smithi150.stdout:3/300: truncate d5/d22/d2f/d38/f39 132015 0 2024-03-22T10:42:50.352 INFO:tasks.workunit.client.1.smithi150.stdout:2/327: write f11 [5493141,60799] 0 2024-03-22T10:42:50.352 INFO:tasks.workunit.client.1.smithi150.stdout:1/274: chown d1/da/c48 25031088 1 2024-03-22T10:42:50.354 INFO:tasks.workunit.client.0.smithi115.stdout:7/704: chown d2/d7/d12/d10/d23/d3d/dd2/da3/fda 1676313 1 2024-03-22T10:42:50.357 INFO:tasks.workunit.client.1.smithi150.stdout:7/310: mknod d2/c69 0 2024-03-22T10:42:50.361 INFO:tasks.workunit.client.1.smithi150.stdout:6/247: rename db/dd/d15/d30 to db/dd/d15/d30/d5c 22 2024-03-22T10:42:50.365 INFO:tasks.workunit.client.1.smithi150.stdout:3/301: truncate d5/da/f57 1250133 0 2024-03-22T10:42:50.365 INFO:tasks.workunit.client.1.smithi150.stdout:4/301: dwrite d0/f11 [0,4194304] 0 2024-03-22T10:42:50.369 INFO:tasks.workunit.client.0.smithi115.stdout:5/709: dwrite d2/df/d13/d8b/d93/da6/dbc/fb2 [0,4194304] 0 2024-03-22T10:42:50.374 INFO:tasks.workunit.client.0.smithi115.stdout:2/620: dwrite da/f1d [4194304,4194304] 0 2024-03-22T10:42:50.375 INFO:tasks.workunit.client.0.smithi115.stdout:1/764: dwrite d0/d8/db/f2c [0,4194304] 0 2024-03-22T10:42:50.375 INFO:tasks.workunit.client.1.smithi150.stdout:1/275: rename d1/d19/d1a/c40 to d1/d19/d50/d51/c59 0 2024-03-22T10:42:50.376 INFO:tasks.workunit.client.1.smithi150.stdout:7/311: chown d2/d5/d7/d8/d57 8091 1 2024-03-22T10:42:50.376 INFO:tasks.workunit.client.0.smithi115.stdout:0/638: dread d3/d7/d1b/d1f/f41 [0,4194304] 0 2024-03-22T10:42:50.378 INFO:tasks.workunit.client.1.smithi150.stdout:5/218: dwrite d0/d8/dc/de/d14/f25 [0,4194304] 0 2024-03-22T10:42:50.378 INFO:tasks.workunit.client.1.smithi150.stdout:0/325: write d0/d12/d4f/d42/d10/d53/f57 [537528,101571] 0 2024-03-22T10:42:50.380 INFO:tasks.workunit.client.1.smithi150.stdout:9/339: write d1/d8/de/df/d36/f5c [1251120,86538] 0 2024-03-22T10:42:50.380 INFO:tasks.workunit.client.1.smithi150.stdout:6/248: read - db/dd/f5b zero size 2024-03-22T10:42:50.381 INFO:tasks.workunit.client.0.smithi115.stdout:2/621: creat da/d1c/d67/d78/fb9 x:0 0 0 2024-03-22T10:42:50.383 INFO:tasks.workunit.client.0.smithi115.stdout:1/765: mkdir d0/d12/d17/d37/d5c/d10a 0 2024-03-22T10:42:50.384 INFO:tasks.workunit.client.1.smithi150.stdout:4/302: chown d0/d2f/f52 38 1 2024-03-22T10:42:50.385 INFO:tasks.workunit.client.1.smithi150.stdout:1/276: readlink d1/da/l25 0 2024-03-22T10:42:50.385 INFO:tasks.workunit.client.0.smithi115.stdout:0/639: mknod d3/da/d4c/da3/da0/cd0 0 2024-03-22T10:42:50.387 INFO:tasks.workunit.client.0.smithi115.stdout:0/640: readlink d3/d7/d1b/d1f/l43 0 2024-03-22T10:42:50.388 INFO:tasks.workunit.client.1.smithi150.stdout:5/219: chown d0/d8/dc/d1d/f24 87284 1 2024-03-22T10:42:50.388 INFO:tasks.workunit.client.1.smithi150.stdout:1/277: fdatasync d1/da/f16 0 2024-03-22T10:42:50.391 INFO:tasks.workunit.client.1.smithi150.stdout:3/302: mkdir d5/d22/d2f/d38/d3e/d5b 0 2024-03-22T10:42:50.391 INFO:tasks.workunit.client.1.smithi150.stdout:9/340: mknod d1/d8/de/d73/c81 0 2024-03-22T10:42:50.392 INFO:tasks.workunit.client.1.smithi150.stdout:6/249: write db/dd/d2e/f5a [968737,5996] 0 2024-03-22T10:42:50.392 INFO:tasks.workunit.client.1.smithi150.stdout:7/312: write d2/d5/d7/d8/d46/f5a [952432,2456] 0 2024-03-22T10:42:50.392 INFO:tasks.workunit.client.1.smithi150.stdout:9/341: stat d1/f71 0 2024-03-22T10:42:50.395 INFO:tasks.workunit.client.1.smithi150.stdout:8/322: dread d0/d2/d19/d29/d2f/df/f42 [0,4194304] 0 2024-03-22T10:42:50.398 INFO:tasks.workunit.client.0.smithi115.stdout:0/641: chown d3/d7/d6b 20 1 2024-03-22T10:42:50.402 INFO:tasks.workunit.client.0.smithi115.stdout:0/642: unlink d3/da/d4c/da3/c95 0 2024-03-22T10:42:50.403 INFO:tasks.workunit.client.1.smithi150.stdout:4/303: creat d0/de/d57/f62 x:0 0 0 2024-03-22T10:42:50.406 INFO:tasks.workunit.client.1.smithi150.stdout:1/278: write d1/d19/d1a/f2b [1165020,123980] 0 2024-03-22T10:42:50.409 INFO:tasks.workunit.client.0.smithi115.stdout:7/705: dwrite d2/d7/d12/d10/d19/d42/d46/d94/fde [0,4194304] 0 2024-03-22T10:42:50.409 INFO:tasks.workunit.client.0.smithi115.stdout:0/643: truncate d3/da/f90 246058 0 2024-03-22T10:42:50.413 INFO:tasks.workunit.client.1.smithi150.stdout:8/323: dread - d0/d2/d4d/f59 zero size 2024-03-22T10:42:50.415 INFO:tasks.workunit.client.1.smithi150.stdout:3/303: unlink d5/d22/c3a 0 2024-03-22T10:42:50.418 INFO:tasks.workunit.client.0.smithi115.stdout:7/706: chown d2/d7/d12/d10/d19/d42/d76/def 4 1 2024-03-22T10:42:50.419 INFO:tasks.workunit.client.1.smithi150.stdout:6/250: rename db/dd/d15/l4d to db/dd/d15/d28/l5d 0 2024-03-22T10:42:50.420 INFO:tasks.workunit.client.0.smithi115.stdout:7/707: chown d2/d7/dc6/lec 438669 1 2024-03-22T10:42:50.422 INFO:tasks.workunit.client.1.smithi150.stdout:3/304: write d5/f52 [720535,125472] 0 2024-03-22T10:42:50.425 INFO:tasks.workunit.client.1.smithi150.stdout:4/304: mkdir d0/d2f/d63 0 2024-03-22T10:42:50.425 INFO:tasks.workunit.client.0.smithi115.stdout:0/644: truncate d3/da/d4c/d74/fb6 1209281 0 2024-03-22T10:42:50.425 INFO:tasks.workunit.client.1.smithi150.stdout:0/326: dwrite d0/d12/d4f/d42/d10/d38/f4b [0,4194304] 0 2024-03-22T10:42:50.426 INFO:tasks.workunit.client.1.smithi150.stdout:1/279: write d1/da/d15/d18/f38 [4245899,13679] 0 2024-03-22T10:42:50.427 INFO:tasks.workunit.client.1.smithi150.stdout:4/305: fdatasync d0/d2f/f3b 0 2024-03-22T10:42:50.428 INFO:tasks.workunit.client.0.smithi115.stdout:7/708: symlink d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/lf9 0 2024-03-22T10:42:50.429 INFO:tasks.workunit.client.1.smithi150.stdout:5/220: mknod d0/d8/dc/de/d18/c44 0 2024-03-22T10:42:50.430 INFO:tasks.workunit.client.0.smithi115.stdout:0/645: mkdir d3/da/d4c/d69/dd1 0 2024-03-22T10:42:50.432 INFO:tasks.workunit.client.1.smithi150.stdout:7/313: mknod d2/d5/d7/d8/d57/c6a 0 2024-03-22T10:42:50.433 INFO:tasks.workunit.client.1.smithi150.stdout:6/251: symlink db/dd/l5e 0 2024-03-22T10:42:50.436 INFO:tasks.workunit.client.0.smithi115.stdout:2/622: dwrite da/d25/d30/d4f/f8d [0,4194304] 0 2024-03-22T10:42:50.437 INFO:tasks.workunit.client.0.smithi115.stdout:0/646: symlink d3/d7/d1b/d9b/ld2 0 2024-03-22T10:42:50.438 INFO:tasks.workunit.client.1.smithi150.stdout:0/327: unlink d0/f3a 0 2024-03-22T10:42:50.443 INFO:tasks.workunit.client.0.smithi115.stdout:1/766: dread d0/f8f [4194304,4194304] 0 2024-03-22T10:42:50.443 INFO:tasks.workunit.client.1.smithi150.stdout:5/221: fdatasync d0/d8/dc/de/d14/f25 0 2024-03-22T10:42:50.444 INFO:tasks.workunit.client.0.smithi115.stdout:2/623: mkdir da/d25/d3d/d57/d7d/da1/dba 0 2024-03-22T10:42:50.446 INFO:tasks.workunit.client.0.smithi115.stdout:2/624: readlink da/d25/d51/l68 0 2024-03-22T10:42:50.446 INFO:tasks.workunit.client.0.smithi115.stdout:7/709: link d2/d7/d12/d10/d23/d3d/f5a d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/ffa 0 2024-03-22T10:42:50.447 INFO:tasks.workunit.client.1.smithi150.stdout:5/222: write d0/d8/dc/f40 [666487,92244] 0 2024-03-22T10:42:50.448 INFO:tasks.workunit.client.1.smithi150.stdout:5/223: readlink d0/l32 0 2024-03-22T10:42:50.449 INFO:tasks.workunit.client.1.smithi150.stdout:7/314: symlink d2/d5/l6b 0 2024-03-22T10:42:50.452 INFO:tasks.workunit.client.1.smithi150.stdout:3/305: mknod d5/d22/c5c 0 2024-03-22T10:42:50.452 INFO:tasks.workunit.client.0.smithi115.stdout:5/710: dread d2/df/d13/d8b/d93/da6/dbc/d50/db1/fdc [0,4194304] 0 2024-03-22T10:42:50.453 INFO:tasks.workunit.client.0.smithi115.stdout:7/710: write d2/d7/d12/d10/d23/d3d/ff0 [304190,90769] 0 2024-03-22T10:42:50.455 INFO:tasks.workunit.client.1.smithi150.stdout:4/306: creat d0/d47/f64 x:0 0 0 2024-03-22T10:42:50.455 INFO:tasks.workunit.client.1.smithi150.stdout:0/328: symlink d0/d12/d4f/d5a/l6f 0 2024-03-22T10:42:50.455 INFO:tasks.workunit.client.0.smithi115.stdout:1/767: rename d0/d12/d7b to d0/d8/d4d/d56/dcf/d10b 0 2024-03-22T10:42:50.457 INFO:tasks.workunit.client.0.smithi115.stdout:5/711: dread - d2/d15/d1b/d7c/d38/d43/d49/fb7 zero size 2024-03-22T10:42:50.458 INFO:tasks.workunit.client.1.smithi150.stdout:1/280: dread d1/da/fb [0,4194304] 0 2024-03-22T10:42:50.458 INFO:tasks.workunit.client.1.smithi150.stdout:8/324: creat d0/f6f x:0 0 0 2024-03-22T10:42:50.459 INFO:tasks.workunit.client.0.smithi115.stdout:0/647: dread d3/d7/d1c/d6c/d9a/fbe [0,4194304] 0 2024-03-22T10:42:50.461 INFO:tasks.workunit.client.0.smithi115.stdout:1/768: creat d0/d8/d4d/d56/dbd/f10c x:0 0 0 2024-03-22T10:42:50.462 INFO:tasks.workunit.client.1.smithi150.stdout:5/224: truncate d0/f10 3501860 0 2024-03-22T10:42:50.462 INFO:tasks.workunit.client.1.smithi150.stdout:6/252: dwrite db/dd/d15/d28/f58 [0,4194304] 0 2024-03-22T10:42:50.463 INFO:tasks.workunit.client.0.smithi115.stdout:7/711: creat d2/d7/d12/d10/d23/d3d/dd2/da3/ffb x:0 0 0 2024-03-22T10:42:50.465 INFO:tasks.workunit.client.0.smithi115.stdout:5/712: read - d2/df/d13/d8b/d93/da6/dc3/fe3 zero size 2024-03-22T10:42:50.468 INFO:tasks.workunit.client.1.smithi150.stdout:3/306: mkdir d5/d22/d2f/d38/d59/d5d 0 2024-03-22T10:42:50.468 INFO:tasks.workunit.client.0.smithi115.stdout:2/625: dwrite da/d25/d30/f6e [0,4194304] 0 2024-03-22T10:42:50.469 INFO:tasks.workunit.client.0.smithi115.stdout:5/713: chown d2/df/d3b/l31 61186702 1 2024-03-22T10:42:50.470 INFO:tasks.workunit.client.0.smithi115.stdout:7/712: creat d2/d7/d12/d10/d19/d42/d46/ffc x:0 0 0 2024-03-22T10:42:50.472 INFO:tasks.workunit.client.0.smithi115.stdout:1/769: creat d0/d12/d17/d37/d9e/da0/dff/f10d x:0 0 0 2024-03-22T10:42:50.474 INFO:tasks.workunit.client.1.smithi150.stdout:8/325: rename d0/d2/d19/d29/d2f/df/d1b/d3d to d0/d2/d19/d29/d2e/d70 0 2024-03-22T10:42:50.475 INFO:tasks.workunit.client.0.smithi115.stdout:2/626: mknod da/d25/d51/cbb 0 2024-03-22T10:42:50.479 INFO:tasks.workunit.client.0.smithi115.stdout:2/627: fsync da/d25/d3d/f9f 0 2024-03-22T10:42:50.479 INFO:tasks.workunit.client.1.smithi150.stdout:1/281: symlink d1/d19/l5a 0 2024-03-22T10:42:50.479 INFO:tasks.workunit.client.1.smithi150.stdout:3/307: mknod d5/d53/c5e 0 2024-03-22T10:42:50.480 INFO:tasks.workunit.client.0.smithi115.stdout:2/628: chown da/d7b 947 1 2024-03-22T10:42:50.480 INFO:tasks.workunit.client.1.smithi150.stdout:3/308: dread - d5/d53/f54 zero size 2024-03-22T10:42:50.484 INFO:tasks.workunit.client.1.smithi150.stdout:0/329: creat d0/d1/f70 x:0 0 0 2024-03-22T10:42:50.485 INFO:tasks.workunit.client.1.smithi150.stdout:6/253: mknod db/dd/d21/d50/c5f 0 2024-03-22T10:42:50.486 INFO:tasks.workunit.client.0.smithi115.stdout:2/629: rename da/d25/d3d/d7e/f99 to da/d25/d3d/d57/fbc 0 2024-03-22T10:42:50.486 INFO:tasks.workunit.client.1.smithi150.stdout:0/330: fdatasync d0/d1/f48 0 2024-03-22T10:42:50.488 INFO:tasks.workunit.client.1.smithi150.stdout:5/225: rename d0/d8/dc/de/d18 to d0/d8/dc/de/d14/d2e/d45 0 2024-03-22T10:42:50.489 INFO:tasks.workunit.client.0.smithi115.stdout:1/770: dwrite d0/d8/db/d25/f44 [0,4194304] 0 2024-03-22T10:42:50.489 INFO:tasks.workunit.client.1.smithi150.stdout:3/309: symlink d5/d22/d2f/d38/d59/l5f 0 2024-03-22T10:42:50.491 INFO:tasks.workunit.client.0.smithi115.stdout:2/630: creat da/d25/d3d/dad/db3/fbd x:0 0 0 2024-03-22T10:42:50.491 INFO:tasks.workunit.client.1.smithi150.stdout:5/226: write d0/d26/f3d [287645,39283] 0 2024-03-22T10:42:50.492 INFO:tasks.workunit.client.0.smithi115.stdout:1/771: read - d0/d12/d17/d37/d9e/da0/dff/f10d zero size 2024-03-22T10:42:50.495 INFO:tasks.workunit.client.1.smithi150.stdout:6/254: fdatasync db/dd/d15/d28/f2b 0 2024-03-22T10:42:50.496 INFO:tasks.workunit.client.1.smithi150.stdout:9/342: truncate d1/d8/de/df/d36/d40/d41/f47 3665230 0 2024-03-22T10:42:50.498 INFO:tasks.workunit.client.0.smithi115.stdout:2/631: rename da/d25/d3d/d57/laf to da/d1c/d67/d78/d86/d9d/lbe 0 2024-03-22T10:42:50.499 INFO:tasks.workunit.client.1.smithi150.stdout:3/310: mkdir d5/d53/d60 0 2024-03-22T10:42:50.499 INFO:tasks.workunit.client.0.smithi115.stdout:2/632: stat da/d1c/f1f 0 2024-03-22T10:42:50.505 INFO:tasks.workunit.client.0.smithi115.stdout:2/633: rename da/d25/d30/d4f to da/d25/d3d/d7e/d90/dbf 0 2024-03-22T10:42:50.507 INFO:tasks.workunit.client.1.smithi150.stdout:9/343: creat d1/d8/f82 x:0 0 0 2024-03-22T10:42:50.508 INFO:tasks.workunit.client.1.smithi150.stdout:9/344: dread - d1/f71 zero size 2024-03-22T10:42:50.509 INFO:tasks.workunit.client.1.smithi150.stdout:9/345: fdatasync d1/f7b 0 2024-03-22T10:42:50.510 INFO:tasks.workunit.client.0.smithi115.stdout:2/634: getdents da/d7b/da4 0 2024-03-22T10:42:50.512 INFO:tasks.workunit.client.1.smithi150.stdout:8/326: rename d0/d2/d4d to d0/d3c/d71 0 2024-03-22T10:42:50.514 INFO:tasks.workunit.client.1.smithi150.stdout:8/327: stat d0/d2/d19/d29/d2f 0 2024-03-22T10:42:50.515 INFO:tasks.workunit.client.0.smithi115.stdout:3/664: sync 2024-03-22T10:42:50.516 INFO:tasks.workunit.client.1.smithi150.stdout:3/311: mknod d5/d22/c61 0 2024-03-22T10:42:50.530 INFO:tasks.workunit.client.0.smithi115.stdout:5/714: dread d2/df/fdf [0,4194304] 0 2024-03-22T10:42:50.531 INFO:tasks.workunit.client.0.smithi115.stdout:3/665: dwrite d3/d36/d41/fa2 [0,4194304] 0 2024-03-22T10:42:50.534 INFO:tasks.workunit.client.0.smithi115.stdout:5/715: creat d2/df/d13/d8b/d93/da6/dc3/fec x:0 0 0 2024-03-22T10:42:50.538 INFO:tasks.workunit.client.0.smithi115.stdout:3/666: symlink d3/d19/dd8/ldf 0 2024-03-22T10:42:50.547 INFO:tasks.workunit.client.0.smithi115.stdout:5/716: symlink d2/d15/d1b/d7c/d38/led 0 2024-03-22T10:42:50.547 INFO:tasks.workunit.client.0.smithi115.stdout:3/667: link d3/f8e d3/d36/d41/d33/fe0 0 2024-03-22T10:42:50.549 INFO:tasks.workunit.client.0.smithi115.stdout:3/668: write d3/fb2 [942167,76829] 0 2024-03-22T10:42:50.549 INFO:tasks.workunit.client.0.smithi115.stdout:5/717: rmdir d2/df/d13/d68 39 2024-03-22T10:42:50.551 INFO:tasks.workunit.client.0.smithi115.stdout:5/718: fdatasync d2/df/d3b/fb0 0 2024-03-22T10:42:50.553 INFO:tasks.workunit.client.0.smithi115.stdout:6/620: sync 2024-03-22T10:42:50.553 INFO:tasks.workunit.client.0.smithi115.stdout:4/671: sync 2024-03-22T10:42:50.553 INFO:tasks.workunit.client.0.smithi115.stdout:3/669: mkdir d3/d6d/db4/de1 0 2024-03-22T10:42:50.555 INFO:tasks.workunit.client.0.smithi115.stdout:6/621: truncate d4/d5e/f97 1200994 0 2024-03-22T10:42:50.557 INFO:tasks.workunit.client.0.smithi115.stdout:4/672: creat d1/d16/d28/fe8 x:0 0 0 2024-03-22T10:42:50.558 INFO:tasks.workunit.client.0.smithi115.stdout:3/670: creat d3/d36/d41/d33/fe2 x:0 0 0 2024-03-22T10:42:50.561 INFO:tasks.workunit.client.0.smithi115.stdout:4/673: write d1/d69/fe6 [286854,13091] 0 2024-03-22T10:42:50.568 INFO:tasks.workunit.client.0.smithi115.stdout:3/671: link d3/d36/d41/fd5 d3/d82/d52/fe3 0 2024-03-22T10:42:50.569 INFO:tasks.workunit.client.0.smithi115.stdout:6/622: dread d4/d36/d48/d72/f76 [0,4194304] 0 2024-03-22T10:42:50.571 INFO:tasks.workunit.client.0.smithi115.stdout:6/623: fdatasync d4/d11/d12/d1c/f7f 0 2024-03-22T10:42:50.572 INFO:tasks.workunit.client.0.smithi115.stdout:5/719: dwrite d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:50.573 INFO:tasks.workunit.client.0.smithi115.stdout:3/672: creat d3/d19/d2b/d49/fe4 x:0 0 0 2024-03-22T10:42:50.574 INFO:tasks.workunit.client.0.smithi115.stdout:6/624: mkdir d4/d11/dc7 0 2024-03-22T10:42:50.577 INFO:tasks.workunit.client.0.smithi115.stdout:5/720: mkdir d2/d15/d1b/d7c/dee 0 2024-03-22T10:42:50.577 INFO:tasks.workunit.client.0.smithi115.stdout:3/673: read d3/f12 [3988420,21566] 0 2024-03-22T10:42:50.578 INFO:tasks.workunit.client.1.smithi150.stdout:2/328: sync 2024-03-22T10:42:50.581 INFO:tasks.workunit.client.0.smithi115.stdout:6/625: creat d4/d11/d12/d1c/d8a/d98/fc8 x:0 0 0 2024-03-22T10:42:50.582 INFO:tasks.workunit.client.0.smithi115.stdout:6/626: chown d4/d13/f25 329193 1 2024-03-22T10:42:50.584 INFO:tasks.workunit.client.0.smithi115.stdout:5/721: unlink d2/d15/d1b/d7c/d84/d89/c9e 0 2024-03-22T10:42:50.585 INFO:tasks.workunit.client.1.smithi150.stdout:9/346: mknod d1/d8/de/df/d36/d40/c83 0 2024-03-22T10:42:50.585 INFO:tasks.workunit.client.0.smithi115.stdout:6/627: creat d4/d5e/fc9 x:0 0 0 2024-03-22T10:42:50.586 INFO:tasks.workunit.client.1.smithi150.stdout:6/255: rename db/dd/d21 to db/dd/d15/d60 0 2024-03-22T10:42:50.586 INFO:tasks.workunit.client.0.smithi115.stdout:5/722: readlink d2/d15/d32/l9f 0 2024-03-22T10:42:50.587 INFO:tasks.workunit.client.0.smithi115.stdout:5/723: fsync d2/d15/d1b/d7c/d38/f61 0 2024-03-22T10:42:50.588 INFO:tasks.workunit.client.0.smithi115.stdout:6/628: symlink d4/d5e/lca 0 2024-03-22T10:42:50.588 INFO:tasks.workunit.client.0.smithi115.stdout:3/674: read d3/f53 [1145620,127871] 0 2024-03-22T10:42:50.590 INFO:tasks.workunit.client.1.smithi150.stdout:3/312: mkdir d5/da/d62 0 2024-03-22T10:42:50.592 INFO:tasks.workunit.client.1.smithi150.stdout:3/313: chown d5/f52 0 1 2024-03-22T10:42:50.592 INFO:tasks.workunit.client.0.smithi115.stdout:3/675: creat d3/d89/fe5 x:0 0 0 2024-03-22T10:42:50.593 INFO:tasks.workunit.client.0.smithi115.stdout:3/676: stat d3/l10 0 2024-03-22T10:42:50.594 INFO:tasks.workunit.client.1.smithi150.stdout:5/227: dwrite d0/d33/f41 [0,4194304] 0 2024-03-22T10:42:50.595 INFO:tasks.workunit.client.1.smithi150.stdout:8/328: mknod d0/d2/d19/d29/d2f/df/d1b/c72 0 2024-03-22T10:42:50.596 INFO:tasks.workunit.client.0.smithi115.stdout:6/629: unlink d4/d13/l1e 0 2024-03-22T10:42:50.596 INFO:tasks.workunit.client.1.smithi150.stdout:5/228: dread - d0/f38 zero size 2024-03-22T10:42:50.597 INFO:tasks.workunit.client.1.smithi150.stdout:0/331: dread d0/f45 [0,4194304] 0 2024-03-22T10:42:50.597 INFO:tasks.workunit.client.0.smithi115.stdout:3/677: creat d3/d82/d52/fe6 x:0 0 0 2024-03-22T10:42:50.598 INFO:tasks.workunit.client.1.smithi150.stdout:8/329: write d0/d3c/f68 [1403102,91975] 0 2024-03-22T10:42:50.599 INFO:tasks.workunit.client.1.smithi150.stdout:3/314: write d5/f8 [5158329,8442] 0 2024-03-22T10:42:50.601 INFO:tasks.workunit.client.1.smithi150.stdout:0/332: stat d0/d12/d4f/d42/d10/d53/f57 0 2024-03-22T10:42:50.602 INFO:tasks.workunit.client.1.smithi150.stdout:9/347: rename d1/d8/de/df/d36/d40/d44/f6a to d1/d8/de/df/d20/d32/d75/f84 0 2024-03-22T10:42:50.602 INFO:tasks.workunit.client.1.smithi150.stdout:3/315: write d5/da/f57 [1573304,79562] 0 2024-03-22T10:42:50.603 INFO:tasks.workunit.client.0.smithi115.stdout:3/678: read d3/d36/d41/d33/fe0 [1412037,106107] 0 2024-03-22T10:42:50.604 INFO:tasks.workunit.client.1.smithi150.stdout:6/256: getdents db/dd/d15/d57 0 2024-03-22T10:42:50.608 INFO:tasks.workunit.client.0.smithi115.stdout:6/630: fsync d4/f92 0 2024-03-22T10:42:50.610 INFO:tasks.workunit.client.1.smithi150.stdout:3/316: write d5/f25 [9386089,9418] 0 2024-03-22T10:42:50.611 INFO:tasks.workunit.client.1.smithi150.stdout:9/348: mknod d1/d19/c85 0 2024-03-22T10:42:50.612 INFO:tasks.workunit.client.0.smithi115.stdout:3/679: write d3/d19/dd8/ddb/fdc [735348,51462] 0 2024-03-22T10:42:50.612 INFO:tasks.workunit.client.1.smithi150.stdout:2/329: link d12/f15 d12/d19/d62/f6f 0 2024-03-22T10:42:50.613 INFO:tasks.workunit.client.1.smithi150.stdout:8/330: getdents d0/d2/d19/d29/d69 0 2024-03-22T10:42:50.616 INFO:tasks.workunit.client.1.smithi150.stdout:6/257: mknod db/c61 0 2024-03-22T10:42:50.616 INFO:tasks.workunit.client.0.smithi115.stdout:1/772: write d0/d22/f98 [661907,102184] 0 2024-03-22T10:42:50.616 INFO:tasks.workunit.client.0.smithi115.stdout:8/886: sync 2024-03-22T10:42:50.616 INFO:tasks.workunit.client.0.smithi115.stdout:9/701: sync 2024-03-22T10:42:50.617 INFO:tasks.workunit.client.1.smithi150.stdout:2/330: mkdir d12/d16/d70 0 2024-03-22T10:42:50.617 INFO:tasks.workunit.client.1.smithi150.stdout:3/317: chown d5/d22/l2a 365227721 1 2024-03-22T10:42:50.620 INFO:tasks.workunit.client.1.smithi150.stdout:2/331: chown d12/d16/d2c 224 1 2024-03-22T10:42:50.620 INFO:tasks.workunit.client.1.smithi150.stdout:6/258: rename db/dd to db/dd/d15/d28/d62 22 2024-03-22T10:42:50.621 INFO:tasks.workunit.client.0.smithi115.stdout:5/724: dwrite d2/df/d13/d8b/d93/da6/dbc/d50/db1/fdc [0,4194304] 0 2024-03-22T10:42:50.622 INFO:tasks.workunit.client.0.smithi115.stdout:3/680: mkdir d3/d82/d54/de7 0 2024-03-22T10:42:50.623 INFO:tasks.workunit.client.1.smithi150.stdout:3/318: creat d5/da/f63 x:0 0 0 2024-03-22T10:42:50.623 INFO:tasks.workunit.client.1.smithi150.stdout:2/332: write d12/d16/d2c/f33 [834887,28170] 0 2024-03-22T10:42:50.627 INFO:tasks.workunit.client.1.smithi150.stdout:6/259: chown db/dd/c10 15 1 2024-03-22T10:42:50.628 INFO:tasks.workunit.client.0.smithi115.stdout:4/674: write d1/d30/d4c/d86/f8a [1991576,43558] 0 2024-03-22T10:42:50.628 INFO:tasks.workunit.client.0.smithi115.stdout:8/887: chown d6/dd/d12/f11a 831635 1 2024-03-22T10:42:50.628 INFO:tasks.workunit.client.1.smithi150.stdout:2/333: write f11 [1161754,38705] 0 2024-03-22T10:42:50.629 INFO:tasks.workunit.client.1.smithi150.stdout:8/331: rename d0/d2/d19/d29/d2f/df/d1b/f24 to d0/d2/d19/d29/d2f/f73 0 2024-03-22T10:42:50.630 INFO:tasks.workunit.client.1.smithi150.stdout:6/260: chown db/l16 8344540 1 2024-03-22T10:42:50.635 INFO:tasks.workunit.client.0.smithi115.stdout:3/681: fdatasync d3/d36/f5c 0 2024-03-22T10:42:50.636 INFO:tasks.workunit.client.0.smithi115.stdout:2/635: dwrite da/d25/d30/f2e [4194304,4194304] 0 2024-03-22T10:42:50.639 INFO:tasks.workunit.client.0.smithi115.stdout:3/682: fsync d3/d82/d52/fc0 0 2024-03-22T10:42:50.640 INFO:tasks.workunit.client.1.smithi150.stdout:2/334: dread - d12/d48/d63/f6c zero size 2024-03-22T10:42:50.640 INFO:tasks.workunit.client.1.smithi150.stdout:0/333: dwrite d0/d12/d4f/d42/d10/f13 [8388608,4194304] 0 2024-03-22T10:42:50.641 INFO:tasks.workunit.client.1.smithi150.stdout:3/319: dread d5/d22/d2f/d38/f39 [0,4194304] 0 2024-03-22T10:42:50.643 INFO:tasks.workunit.client.0.smithi115.stdout:7/713: dread d2/d7/d12/d10/d19/d42/d76/f83 [4194304,4194304] 0 2024-03-22T10:42:50.645 INFO:tasks.workunit.client.1.smithi150.stdout:2/335: write d12/d16/d2c/f33 [302309,83223] 0 2024-03-22T10:42:50.645 INFO:tasks.workunit.client.1.smithi150.stdout:3/320: dread - d5/f4e zero size 2024-03-22T10:42:50.645 INFO:tasks.workunit.client.1.smithi150.stdout:9/349: unlink d1/d19/f26 0 2024-03-22T10:42:50.648 INFO:tasks.workunit.client.0.smithi115.stdout:0/648: dwrite d3/da/f90 [0,4194304] 0 2024-03-22T10:42:50.648 INFO:tasks.workunit.client.0.smithi115.stdout:6/631: dwrite d4/d13/d33/f42 [4194304,4194304] 0 2024-03-22T10:42:50.649 INFO:tasks.workunit.client.0.smithi115.stdout:2/636: dread - da/d1c/d67/d78/d86/d9d/fa0 zero size 2024-03-22T10:42:50.649 INFO:tasks.workunit.client.0.smithi115.stdout:4/675: mkdir d1/d30/d4c/d86/d9f/de9 0 2024-03-22T10:42:50.650 INFO:tasks.workunit.client.1.smithi150.stdout:2/336: read d12/d21/f4b [2035817,33292] 0 2024-03-22T10:42:50.653 INFO:tasks.workunit.client.0.smithi115.stdout:4/676: fdatasync d1/d66/f70 0 2024-03-22T10:42:50.658 INFO:tasks.workunit.client.1.smithi150.stdout:6/261: dwrite db/f4a [0,4194304] 0 2024-03-22T10:42:50.659 INFO:tasks.workunit.client.0.smithi115.stdout:4/677: dread - d1/d16/d3b/fb7 zero size 2024-03-22T10:42:50.663 INFO:tasks.workunit.client.1.smithi150.stdout:3/321: readlink d5/da/l4c 0 2024-03-22T10:42:50.664 INFO:tasks.workunit.client.0.smithi115.stdout:4/678: write d1/d30/d4c/d86/d9f/d32/fb3 [2314731,59541] 0 2024-03-22T10:42:50.668 INFO:tasks.workunit.client.1.smithi150.stdout:8/332: link d0/f17 d0/d2/d19/d29/d2f/f74 0 2024-03-22T10:42:50.668 INFO:tasks.workunit.client.0.smithi115.stdout:1/773: dread d0/d8/db/f2c [0,4194304] 0 2024-03-22T10:42:50.670 INFO:tasks.workunit.client.0.smithi115.stdout:5/725: creat d2/df/d13/fef x:0 0 0 2024-03-22T10:42:50.673 INFO:tasks.workunit.client.1.smithi150.stdout:0/334: fdatasync d0/d12/d4f/d42/d10/f13 0 2024-03-22T10:42:50.677 INFO:tasks.workunit.client.1.smithi150.stdout:8/333: creat d0/d2/d19/d29/d2f/df/d1b/f75 x:0 0 0 2024-03-22T10:42:50.679 INFO:tasks.workunit.client.1.smithi150.stdout:0/335: chown d0/d12/d4f/d42/d10/d38/d52/c64 1365 1 2024-03-22T10:42:50.679 INFO:tasks.workunit.client.0.smithi115.stdout:5/726: chown d2/d15/d1b/d7c/d38/d43/d49/d81/d83/caa 4 1 2024-03-22T10:42:50.680 INFO:tasks.workunit.client.1.smithi150.stdout:0/336: stat d0/d12/d4f/d5a 0 2024-03-22T10:42:50.680 INFO:tasks.workunit.client.0.smithi115.stdout:9/702: write d6/d7/d20/d22/d23/fc7 [402385,2789] 0 2024-03-22T10:42:50.682 INFO:tasks.workunit.client.1.smithi150.stdout:8/334: fsync d0/d3c/f4a 0 2024-03-22T10:42:50.684 INFO:tasks.workunit.client.1.smithi150.stdout:3/322: link d5/ce d5/d53/c64 0 2024-03-22T10:42:50.686 INFO:tasks.workunit.client.0.smithi115.stdout:7/714: unlink d2/fe6 0 2024-03-22T10:42:50.686 INFO:tasks.workunit.client.1.smithi150.stdout:8/335: read d0/d2/f14 [6918,26303] 0 2024-03-22T10:42:50.689 INFO:tasks.workunit.client.0.smithi115.stdout:4/679: mkdir d1/d30/d4c/d86/d9f/d75/d99/da8/dea 0 2024-03-22T10:42:50.692 INFO:tasks.workunit.client.0.smithi115.stdout:4/680: dread - d1/d16/d35/fb0 zero size 2024-03-22T10:42:50.693 INFO:tasks.workunit.client.1.smithi150.stdout:3/323: symlink d5/d53/l65 0 2024-03-22T10:42:50.695 INFO:tasks.workunit.client.0.smithi115.stdout:5/727: truncate d2/df/d13/d8b/d93/da6/fa8 31576 0 2024-03-22T10:42:50.696 INFO:tasks.workunit.client.1.smithi150.stdout:9/350: write d1/d8/de/df/f1f [174997,19647] 0 2024-03-22T10:42:50.696 INFO:tasks.workunit.client.0.smithi115.stdout:7/715: chown d2/d7/d12/d10/d19/dbd/fdb 60181812 1 2024-03-22T10:42:50.699 INFO:tasks.workunit.client.0.smithi115.stdout:1/774: mkdir d0/d12/d2b/d10e 0 2024-03-22T10:42:50.702 INFO:tasks.workunit.client.0.smithi115.stdout:4/681: creat d1/d66/d82/feb x:0 0 0 2024-03-22T10:42:50.702 INFO:tasks.workunit.client.1.smithi150.stdout:0/337: creat d0/d12/d4f/d42/d10/f71 x:0 0 0 2024-03-22T10:42:50.703 INFO:tasks.workunit.client.0.smithi115.stdout:1/775: chown d0/d12/d17/d37/d9e/da0/db9 2411518 1 2024-03-22T10:42:50.703 INFO:tasks.workunit.client.1.smithi150.stdout:2/337: write d12/d19/d62/f6f [1118146,100597] 0 2024-03-22T10:42:50.706 INFO:tasks.workunit.client.0.smithi115.stdout:5/728: chown d2/d15/d1b/d7c/d38/d43/d49/c4d 28 1 2024-03-22T10:42:50.707 INFO:tasks.workunit.client.1.smithi150.stdout:3/324: rename d5/da/l18 to d5/d53/l66 0 2024-03-22T10:42:50.707 INFO:tasks.workunit.client.1.smithi150.stdout:6/262: write db/dd/d15/d60/f53 [3597013,46127] 0 2024-03-22T10:42:50.709 INFO:tasks.workunit.client.1.smithi150.stdout:9/351: symlink d1/d8/de/df/d20/d29/l86 0 2024-03-22T10:42:50.711 INFO:tasks.workunit.client.1.smithi150.stdout:2/338: symlink d12/d16/d2c/l71 0 2024-03-22T10:42:50.711 INFO:tasks.workunit.client.0.smithi115.stdout:6/632: dwrite d4/d5e/f8b [0,4194304] 0 2024-03-22T10:42:50.711 INFO:tasks.workunit.client.0.smithi115.stdout:0/649: link d3/d7/d1b/c7a d3/d7/d1b/d93/cd3 0 2024-03-22T10:42:50.711 INFO:tasks.workunit.client.0.smithi115.stdout:7/716: creat d2/d7/d12/d10/d23/d3d/dd2/ffd x:0 0 0 2024-03-22T10:42:50.719 INFO:tasks.workunit.client.0.smithi115.stdout:7/717: dread - d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/fd9 zero size 2024-03-22T10:42:50.719 INFO:tasks.workunit.client.0.smithi115.stdout:5/729: rename d2/c62 to d2/df/d3b/d8e/dd3/cf0 0 2024-03-22T10:42:50.721 INFO:tasks.workunit.client.0.smithi115.stdout:9/703: write d6/d7/d30/d3b/f8f [5058884,89816] 0 2024-03-22T10:42:50.721 INFO:tasks.workunit.client.0.smithi115.stdout:3/683: dread d3/fd2 [0,4194304] 0 2024-03-22T10:42:50.724 INFO:tasks.workunit.client.0.smithi115.stdout:0/650: creat d3/d7/d1c/d6c/d9a/fd4 x:0 0 0 2024-03-22T10:42:50.724 INFO:tasks.workunit.client.0.smithi115.stdout:6/633: dread d4/d11/d12/d1c/fa7 [0,4194304] 0 2024-03-22T10:42:50.724 INFO:tasks.workunit.client.0.smithi115.stdout:8/888: dread d6/d49/d88/fe2 [0,4194304] 0 2024-03-22T10:42:50.725 INFO:tasks.workunit.client.0.smithi115.stdout:4/682: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:50.726 INFO:tasks.workunit.client.0.smithi115.stdout:3/684: symlink d3/d82/d54/le8 0 2024-03-22T10:42:50.726 INFO:tasks.workunit.client.0.smithi115.stdout:0/651: chown d3/da/d4c/d74 1774695915 1 2024-03-22T10:42:50.726 INFO:tasks.workunit.client.0.smithi115.stdout:7/718: read d2/d62/f4f [8205956,13296] 0 2024-03-22T10:42:50.727 INFO:tasks.workunit.client.0.smithi115.stdout:5/730: dread - d2/df/d13/d8b/d93/da6/dbc/d50/f86 zero size 2024-03-22T10:42:50.727 INFO:tasks.workunit.client.1.smithi150.stdout:0/338: dwrite d0/d1/f70 [0,4194304] 0 2024-03-22T10:42:50.729 INFO:tasks.workunit.client.0.smithi115.stdout:7/719: stat d2/d7/d12/fb8 0 2024-03-22T10:42:50.729 INFO:tasks.workunit.client.0.smithi115.stdout:9/704: write d6/d7/d20/d22/de5/ff2 [821591,18198] 0 2024-03-22T10:42:50.730 INFO:tasks.workunit.client.1.smithi150.stdout:6/263: dwrite db/dd/f1b [4194304,4194304] 0 2024-03-22T10:42:50.731 INFO:tasks.workunit.client.0.smithi115.stdout:6/634: fsync d4/f92 0 2024-03-22T10:42:50.734 INFO:tasks.workunit.client.0.smithi115.stdout:0/652: write d3/d7/d1c/f77 [2676144,112745] 0 2024-03-22T10:42:50.735 INFO:tasks.workunit.client.0.smithi115.stdout:7/720: creat d2/ffe x:0 0 0 2024-03-22T10:42:50.738 INFO:tasks.workunit.client.1.smithi150.stdout:6/264: mknod db/dd/d15/d28/c63 0 2024-03-22T10:42:50.739 INFO:tasks.workunit.client.0.smithi115.stdout:3/685: creat d3/d19/d61/da6/fe9 x:0 0 0 2024-03-22T10:42:50.742 INFO:tasks.workunit.client.0.smithi115.stdout:4/683: read d1/d16/d35/d40/d5c/d14/f58 [1591013,52039] 0 2024-03-22T10:42:50.743 INFO:tasks.workunit.client.1.smithi150.stdout:6/265: write db/dd/f2c [1240848,61929] 0 2024-03-22T10:42:50.744 INFO:tasks.workunit.client.1.smithi150.stdout:9/352: dwrite d1/d8/de/df/f11 [0,4194304] 0 2024-03-22T10:42:50.746 INFO:tasks.workunit.client.1.smithi150.stdout:6/266: chown db/dd/d15 390499 1 2024-03-22T10:42:50.748 INFO:tasks.workunit.client.0.smithi115.stdout:5/731: rename d2/d15/d1b/d7c/fe7 to d2/df/d13/d8b/d93/da6/dbc/ff1 0 2024-03-22T10:42:50.749 INFO:tasks.workunit.client.0.smithi115.stdout:9/705: creat d6/d7/ffb x:0 0 0 2024-03-22T10:42:50.750 INFO:tasks.workunit.client.0.smithi115.stdout:6/635: symlink d4/d36/d48/lcb 0 2024-03-22T10:42:50.750 INFO:tasks.workunit.client.1.smithi150.stdout:7/315: sync 2024-03-22T10:42:50.755 INFO:tasks.workunit.client.1.smithi150.stdout:0/339: dwrite d0/d12/d4f/d11/f5d [4194304,4194304] 0 2024-03-22T10:42:50.756 INFO:tasks.workunit.client.0.smithi115.stdout:6/636: write d4/d36/d48/f62 [1713781,29661] 0 2024-03-22T10:42:50.756 INFO:tasks.workunit.client.1.smithi150.stdout:6/267: creat db/dd/d15/d60/d50/f64 x:0 0 0 2024-03-22T10:42:50.757 INFO:tasks.workunit.client.0.smithi115.stdout:9/706: rename d6/d7/d20/d22/d23/l4a to d6/d7/d6a/lfc 0 2024-03-22T10:42:50.759 INFO:tasks.workunit.client.0.smithi115.stdout:5/732: write d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f69 [425211,117873] 0 2024-03-22T10:42:50.759 INFO:tasks.workunit.client.0.smithi115.stdout:3/686: write d3/fd2 [4598453,4774] 0 2024-03-22T10:42:50.760 INFO:tasks.workunit.client.1.smithi150.stdout:2/339: dread d12/d21/f4b [0,4194304] 0 2024-03-22T10:42:50.762 INFO:tasks.workunit.client.1.smithi150.stdout:6/268: creat db/dd/d15/d57/f65 x:0 0 0 2024-03-22T10:42:50.765 INFO:tasks.workunit.client.1.smithi150.stdout:0/340: creat d0/d12/d4f/d42/f72 x:0 0 0 2024-03-22T10:42:50.766 INFO:tasks.workunit.client.0.smithi115.stdout:0/653: dwrite d3/d4/f88 [0,4194304] 0 2024-03-22T10:42:50.768 INFO:tasks.workunit.client.1.smithi150.stdout:6/269: symlink db/dd/d15/l66 0 2024-03-22T10:42:50.769 INFO:tasks.workunit.client.1.smithi150.stdout:2/340: write d12/d19/f58 [1424997,27140] 0 2024-03-22T10:42:50.773 INFO:tasks.workunit.client.1.smithi150.stdout:6/270: truncate db/dd/d25/f3d 237035 0 2024-03-22T10:42:50.774 INFO:tasks.workunit.client.1.smithi150.stdout:0/341: truncate d0/d12/d4f/d42/fc 883723 0 2024-03-22T10:42:50.775 INFO:tasks.workunit.client.0.smithi115.stdout:3/687: symlink d3/d16/d29/d35/lea 0 2024-03-22T10:42:50.776 INFO:tasks.workunit.client.0.smithi115.stdout:6/637: fdatasync d4/d36/d48/d72/f76 0 2024-03-22T10:42:50.777 INFO:tasks.workunit.client.1.smithi150.stdout:2/341: rename d12/d16/f22 to d12/d48/f72 0 2024-03-22T10:42:50.780 INFO:tasks.workunit.client.0.smithi115.stdout:4/684: truncate d1/d16/d35/d40/d5c/f23 3642459 0 2024-03-22T10:42:50.781 INFO:tasks.workunit.client.1.smithi150.stdout:7/316: write d2/d5/d7/d8/d46/f4e [192909,48653] 0 2024-03-22T10:42:50.782 INFO:tasks.workunit.client.0.smithi115.stdout:8/889: write d6/fad [1200243,70282] 0 2024-03-22T10:42:50.782 INFO:tasks.workunit.client.0.smithi115.stdout:9/707: creat d6/d7/d6a/d6f/df0/ffd x:0 0 0 2024-03-22T10:42:50.782 INFO:tasks.workunit.client.1.smithi150.stdout:6/271: rename db/ff to db/dd/d15/d60/f67 0 2024-03-22T10:42:50.784 INFO:tasks.workunit.client.1.smithi150.stdout:6/272: chown db/dd/d2e 209023 1 2024-03-22T10:42:50.787 INFO:tasks.workunit.client.1.smithi150.stdout:0/342: truncate d0/d12/d4f/d42/d10/f13 6172265 0 2024-03-22T10:42:50.787 INFO:tasks.workunit.client.1.smithi150.stdout:6/273: chown db/l1d 56656532 1 2024-03-22T10:42:50.793 INFO:tasks.workunit.client.1.smithi150.stdout:9/353: dwrite d1/d8/de/df/d24/f6b [0,4194304] 0 2024-03-22T10:42:50.798 INFO:tasks.workunit.client.1.smithi150.stdout:6/274: symlink db/dd/d15/d28/l68 0 2024-03-22T10:42:50.798 INFO:tasks.workunit.client.1.smithi150.stdout:9/354: write d1/d8/de/df/d20/f50 [1266907,43176] 0 2024-03-22T10:42:50.802 INFO:tasks.workunit.client.1.smithi150.stdout:6/275: chown db/dd/d15/d28/l68 16364877 1 2024-03-22T10:42:50.804 INFO:tasks.workunit.client.1.smithi150.stdout:7/317: dwrite d2/d5/d7/d8/f5e [4194304,4194304] 0 2024-03-22T10:42:50.806 INFO:tasks.workunit.client.1.smithi150.stdout:9/355: mknod d1/d8/de/df/d20/d32/c87 0 2024-03-22T10:42:50.811 INFO:tasks.workunit.client.1.smithi150.stdout:9/356: creat d1/d8/f88 x:0 0 0 2024-03-22T10:42:50.814 INFO:tasks.workunit.client.1.smithi150.stdout:9/357: chown d1/d8/de/df/d20/d29/f54 0 1 2024-03-22T10:42:50.822 INFO:tasks.workunit.client.1.smithi150.stdout:2/342: dwrite d12/d21/f4b [0,4194304] 0 2024-03-22T10:42:50.822 INFO:tasks.workunit.client.1.smithi150.stdout:7/318: getdents d2/d5/d7/d8/d57/d4a 0 2024-03-22T10:42:50.823 INFO:tasks.workunit.client.1.smithi150.stdout:2/343: dread - d12/f55 zero size 2024-03-22T10:42:50.825 INFO:tasks.workunit.client.1.smithi150.stdout:7/319: dread - d2/d5/d7/d9/f53 zero size 2024-03-22T10:42:50.826 INFO:tasks.workunit.client.1.smithi150.stdout:9/358: truncate d1/d8/de/df/d36/d40/d41/f61 990934 0 2024-03-22T10:42:50.828 INFO:tasks.workunit.client.1.smithi150.stdout:2/344: mkdir d12/d16/d2c/d73 0 2024-03-22T10:42:50.830 INFO:tasks.workunit.client.1.smithi150.stdout:0/343: write d0/d12/d4f/d11/d26/f2e [2772847,5564] 0 2024-03-22T10:42:50.831 INFO:tasks.workunit.client.1.smithi150.stdout:9/359: write d1/d8/de/df/d20/d32/d75/f84 [1524571,117687] 0 2024-03-22T10:42:50.834 INFO:tasks.workunit.client.1.smithi150.stdout:2/345: creat d12/d16/d67/f74 x:0 0 0 2024-03-22T10:42:50.834 INFO:tasks.workunit.client.1.smithi150.stdout:9/360: stat d1/d8/de/df/d20/d32/c2b 0 2024-03-22T10:42:50.835 INFO:tasks.workunit.client.1.smithi150.stdout:7/320: dread d2/d5/d7/d8/d57/f5b [0,4194304] 0 2024-03-22T10:42:50.840 INFO:tasks.workunit.client.1.smithi150.stdout:7/321: mkdir d2/d5/d7/d9/db/d6c 0 2024-03-22T10:42:50.840 INFO:tasks.workunit.client.1.smithi150.stdout:9/361: truncate d1/d8/de/df/d20/d32/d75/f84 1742572 0 2024-03-22T10:42:50.844 INFO:tasks.workunit.client.1.smithi150.stdout:9/362: creat d1/d8/de/df/d24/f89 x:0 0 0 2024-03-22T10:42:50.851 INFO:tasks.workunit.client.1.smithi150.stdout:9/363: write d1/d8/de/df/d36/d40/d41/f61 [184028,8663] 0 2024-03-22T10:42:50.853 INFO:tasks.workunit.client.1.smithi150.stdout:2/346: dwrite d12/d19/f58 [0,4194304] 0 2024-03-22T10:42:50.855 INFO:tasks.workunit.client.1.smithi150.stdout:0/344: dwrite d0/d12/d4f/d42/d10/d38/f46 [0,4194304] 0 2024-03-22T10:42:50.858 INFO:tasks.workunit.client.1.smithi150.stdout:9/364: fdatasync d1/d8/de/df/d36/f39 0 2024-03-22T10:42:50.859 INFO:tasks.workunit.client.0.smithi115.stdout:7/721: getdents d2/d7/d12/d10/d23/d3d/dd2/da3/de8/dc7 0 2024-03-22T10:42:50.860 INFO:tasks.workunit.client.0.smithi115.stdout:5/733: symlink d2/d15/d1b/d7c/dee/lf2 0 2024-03-22T10:42:50.861 INFO:tasks.workunit.client.0.smithi115.stdout:6/638: symlink d4/d11/d99/lcc 0 2024-03-22T10:42:50.863 INFO:tasks.workunit.client.0.smithi115.stdout:3/688: rmdir d3/d19/d2b/d49/d91/da5 39 2024-03-22T10:42:50.863 INFO:tasks.workunit.client.1.smithi150.stdout:7/322: dwrite d2/d5/d7/d8/d2c/f47 [0,4194304] 0 2024-03-22T10:42:50.864 INFO:tasks.workunit.client.0.smithi115.stdout:3/689: chown d3/d19/d2b/l68 676966442 1 2024-03-22T10:42:50.866 INFO:tasks.workunit.client.1.smithi150.stdout:7/323: readlink d2/d5/d7/l48 0 2024-03-22T10:42:50.867 INFO:tasks.workunit.client.1.smithi150.stdout:0/345: rename d0/d12/d4f/ff to d0/d12/d4f/d5a/f73 0 2024-03-22T10:42:50.867 INFO:tasks.workunit.client.0.smithi115.stdout:3/690: fdatasync d3/d89/dca/fcb 0 2024-03-22T10:42:50.870 INFO:tasks.workunit.client.1.smithi150.stdout:7/324: fdatasync d2/d5/d7/f4d 0 2024-03-22T10:42:50.871 INFO:tasks.workunit.client.0.smithi115.stdout:6/639: write d4/d11/d12/d1c/f69 [4339010,53574] 0 2024-03-22T10:42:50.872 INFO:tasks.workunit.client.0.smithi115.stdout:4/685: symlink d1/d30/d4c/d86/lec 0 2024-03-22T10:42:50.875 INFO:tasks.workunit.client.1.smithi150.stdout:9/365: rename d1/d8/de/df/d36/d40/d6c/c80 to d1/d8/de/df/d20/c8a 0 2024-03-22T10:42:50.876 INFO:tasks.workunit.client.1.smithi150.stdout:0/346: unlink d0/d12/d4f/d42/d10/d38/f46 0 2024-03-22T10:42:50.877 INFO:tasks.workunit.client.0.smithi115.stdout:7/722: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/dc7/fff x:0 0 0 2024-03-22T10:42:50.878 INFO:tasks.workunit.client.1.smithi150.stdout:7/325: fsync d2/d5/d7/d8/d57/f3c 0 2024-03-22T10:42:50.879 INFO:tasks.workunit.client.0.smithi115.stdout:4/686: readlink d1/d69/l37 0 2024-03-22T10:42:50.879 INFO:tasks.workunit.client.1.smithi150.stdout:0/347: stat d0/d12/d4f/d42/l4e 0 2024-03-22T10:42:50.881 INFO:tasks.workunit.client.1.smithi150.stdout:2/347: dwrite d12/f37 [0,4194304] 0 2024-03-22T10:42:50.882 INFO:tasks.workunit.client.0.smithi115.stdout:3/691: fsync d3/d19/dda/fdd 0 2024-03-22T10:42:50.882 INFO:tasks.workunit.client.0.smithi115.stdout:5/734: creat d2/d15/ff3 x:0 0 0 2024-03-22T10:42:50.884 INFO:tasks.workunit.client.1.smithi150.stdout:0/348: chown d0/c4d 34 1 2024-03-22T10:42:50.885 INFO:tasks.workunit.client.1.smithi150.stdout:2/348: write d12/d19/d1a/f2b [2059366,109586] 0 2024-03-22T10:42:50.887 INFO:tasks.workunit.client.0.smithi115.stdout:7/723: unlink d2/d7/d12/d10/d19/dbd/dce/fee 0 2024-03-22T10:42:50.888 INFO:tasks.workunit.client.0.smithi115.stdout:0/654: dwrite d3/da/d4c/d74/fb6 [0,4194304] 0 2024-03-22T10:42:50.888 INFO:tasks.workunit.client.0.smithi115.stdout:4/687: write d1/d30/d4c/d86/d9f/d32/fcf [1040759,1715] 0 2024-03-22T10:42:50.888 INFO:tasks.workunit.client.1.smithi150.stdout:9/366: getdents d1/d8/de/df/d20/d32 0 2024-03-22T10:42:50.889 INFO:tasks.workunit.client.1.smithi150.stdout:7/326: truncate d2/d5/d7/d8/d57/d4a/f22 5698053 0 2024-03-22T10:42:50.889 INFO:tasks.workunit.client.1.smithi150.stdout:2/349: read - d12/d16/d67/f5a zero size 2024-03-22T10:42:50.892 INFO:tasks.workunit.client.0.smithi115.stdout:6/640: symlink d4/d13/d33/dbe/lcd 0 2024-03-22T10:42:50.892 INFO:tasks.workunit.client.1.smithi150.stdout:0/349: rename d0/l51 to d0/d12/d4f/d5a/l74 0 2024-03-22T10:42:50.893 INFO:tasks.workunit.client.0.smithi115.stdout:9/708: getdents d6/d8/d3c/d59/d68/da8 0 2024-03-22T10:42:50.893 INFO:tasks.workunit.client.1.smithi150.stdout:9/367: write d1/d8/de/df/f4d [863283,65350] 0 2024-03-22T10:42:50.894 INFO:tasks.workunit.client.0.smithi115.stdout:3/692: truncate d3/d6d/f9e 56287 0 2024-03-22T10:42:50.895 INFO:tasks.workunit.client.0.smithi115.stdout:9/709: chown d6/d7/d6a 5 1 2024-03-22T10:42:50.896 INFO:tasks.workunit.client.0.smithi115.stdout:4/688: write d1/d66/d82/feb [905598,38757] 0 2024-03-22T10:42:50.897 INFO:tasks.workunit.client.1.smithi150.stdout:2/350: mkdir d12/d16/d67/d3c/d75 0 2024-03-22T10:42:50.897 INFO:tasks.workunit.client.1.smithi150.stdout:7/327: mknod d2/d5/d7/d9/db/c6d 0 2024-03-22T10:42:50.899 INFO:tasks.workunit.client.1.smithi150.stdout:0/350: write d0/d12/d4f/d42/d10/d38/f4b [4550810,5306] 0 2024-03-22T10:42:50.902 INFO:tasks.workunit.client.1.smithi150.stdout:2/351: symlink d12/d48/d54/l76 0 2024-03-22T10:42:50.903 INFO:tasks.workunit.client.1.smithi150.stdout:9/368: creat d1/d8/de/df/d20/d32/d57/f8b x:0 0 0 2024-03-22T10:42:50.904 INFO:tasks.workunit.client.1.smithi150.stdout:0/351: rmdir d0/d12/d4f/d11/d66 39 2024-03-22T10:42:50.907 INFO:tasks.workunit.client.1.smithi150.stdout:2/352: creat d12/d16/d67/d3c/f77 x:0 0 0 2024-03-22T10:42:50.907 INFO:tasks.workunit.client.0.smithi115.stdout:0/655: creat d3/d4/fd5 x:0 0 0 2024-03-22T10:42:50.909 INFO:tasks.workunit.client.0.smithi115.stdout:4/689: rmdir d1/d30/d78 39 2024-03-22T10:42:50.910 INFO:tasks.workunit.client.1.smithi150.stdout:9/369: fdatasync d1/d8/de/df/d20/d29/f54 0 2024-03-22T10:42:50.912 INFO:tasks.workunit.client.0.smithi115.stdout:6/641: rename d4/c51 to d4/d11/d12/d1c/d8a/d98/d9f/cce 0 2024-03-22T10:42:50.914 INFO:tasks.workunit.client.0.smithi115.stdout:5/735: link d2/d15/dc8/ld7 d2/d15/d1b/d7c/lf4 0 2024-03-22T10:42:50.915 INFO:tasks.workunit.client.0.smithi115.stdout:8/890: dwrite d6/dd/d24/fa7 [0,4194304] 0 2024-03-22T10:42:50.917 INFO:tasks.workunit.client.1.smithi150.stdout:9/370: truncate d1/d8/de/df/f5b 1133890 0 2024-03-22T10:42:50.918 INFO:tasks.workunit.client.0.smithi115.stdout:4/690: chown d1/d16/d35/d40/d5c/d55/fcb 1566 1 2024-03-22T10:42:50.918 INFO:tasks.workunit.client.1.smithi150.stdout:2/353: mkdir d12/d19/d2d/d68/d78 0 2024-03-22T10:42:50.919 INFO:tasks.workunit.client.1.smithi150.stdout:2/354: fsync d12/d16/d67/f5a 0 2024-03-22T10:42:50.920 INFO:tasks.workunit.client.0.smithi115.stdout:4/691: readlink d1/d16/d35/d40/l72 0 2024-03-22T10:42:50.924 INFO:tasks.workunit.client.0.smithi115.stdout:0/656: mkdir d3/d7/d1c/d6c/d87/dd6 0 2024-03-22T10:42:50.926 INFO:tasks.workunit.client.1.smithi150.stdout:2/355: truncate d12/d19/f34 227250 0 2024-03-22T10:42:50.927 INFO:tasks.workunit.client.0.smithi115.stdout:8/891: symlink d6/dd/d24/d4a/d41/dc4/l12c 0 2024-03-22T10:42:50.930 INFO:tasks.workunit.client.1.smithi150.stdout:2/356: fdatasync d12/d48/d54/f5b 0 2024-03-22T10:42:50.931 INFO:tasks.workunit.client.0.smithi115.stdout:6/642: mknod d4/d36/ccf 0 2024-03-22T10:42:50.932 INFO:tasks.workunit.client.0.smithi115.stdout:4/692: rmdir d1/d16/d35 39 2024-03-22T10:42:50.934 INFO:tasks.workunit.client.1.smithi150.stdout:0/352: dread d0/d12/d4f/f28 [0,4194304] 0 2024-03-22T10:42:50.935 INFO:tasks.workunit.client.1.smithi150.stdout:2/357: write d12/d48/f4c [4372390,120315] 0 2024-03-22T10:42:50.936 INFO:tasks.workunit.client.0.smithi115.stdout:6/643: chown d4/d11/c47 13628871 1 2024-03-22T10:42:50.938 INFO:tasks.workunit.client.0.smithi115.stdout:8/892: creat d6/dd/d12/dc6/f12d x:0 0 0 2024-03-22T10:42:50.939 INFO:tasks.workunit.client.0.smithi115.stdout:7/724: dread d2/d7/d12/d10/f17 [4194304,4194304] 0 2024-03-22T10:42:50.940 INFO:tasks.workunit.client.0.smithi115.stdout:9/710: dread d6/d7/d20/d9c/fa1 [0,4194304] 0 2024-03-22T10:42:50.942 INFO:tasks.workunit.client.1.smithi150.stdout:9/371: dwrite d1/d8/de/df/f4d [0,4194304] 0 2024-03-22T10:42:50.944 INFO:tasks.workunit.client.0.smithi115.stdout:3/693: dwrite d3/d19/d2b/d49/d91/f9f [0,4194304] 0 2024-03-22T10:42:50.945 INFO:tasks.workunit.client.0.smithi115.stdout:5/736: write d2/df/d13/d8b/d93/da6/fe0 [3015043,115760] 0 2024-03-22T10:42:50.952 INFO:tasks.workunit.client.1.smithi150.stdout:2/358: dwrite d12/d16/f42 [0,4194304] 0 2024-03-22T10:42:50.958 INFO:tasks.workunit.client.1.smithi150.stdout:2/359: rename d12/d48/f41 to d12/d48/f79 0 2024-03-22T10:42:50.959 INFO:tasks.workunit.client.0.smithi115.stdout:3/694: link d3/d89/f8b d3/d19/d2b/d49/feb 0 2024-03-22T10:42:50.960 INFO:tasks.workunit.client.0.smithi115.stdout:9/711: link d6/l10 d6/d8/d3c/d59/d68/d97/da4/lfe 0 2024-03-22T10:42:50.960 INFO:tasks.workunit.client.1.smithi150.stdout:9/372: dread d1/d49/f31 [0,4194304] 0 2024-03-22T10:42:50.961 INFO:tasks.workunit.client.1.smithi150.stdout:2/360: write d12/d48/f4d [310272,114879] 0 2024-03-22T10:42:50.963 INFO:tasks.workunit.client.0.smithi115.stdout:3/695: mknod d3/d19/dda/cec 0 2024-03-22T10:42:50.963 INFO:tasks.workunit.client.1.smithi150.stdout:2/361: chown d12/d19 0 1 2024-03-22T10:42:50.964 INFO:tasks.workunit.client.1.smithi150.stdout:9/373: chown d1/d8/de/df/f1f 1 1 2024-03-22T10:42:50.964 INFO:tasks.workunit.client.1.smithi150.stdout:0/353: write d0/d12/d4f/d42/d10/f43 [744739,48656] 0 2024-03-22T10:42:50.964 INFO:tasks.workunit.client.0.smithi115.stdout:9/712: fsync d6/d7/f2d 0 2024-03-22T10:42:50.965 INFO:tasks.workunit.client.1.smithi150.stdout:1/282: sync 2024-03-22T10:42:50.965 INFO:tasks.workunit.client.1.smithi150.stdout:4/307: sync 2024-03-22T10:42:50.965 INFO:tasks.workunit.client.0.smithi115.stdout:3/696: creat d3/d6d/db4/fed x:0 0 0 2024-03-22T10:42:50.969 INFO:tasks.workunit.client.1.smithi150.stdout:2/362: write d12/d48/f4c [283238,120702] 0 2024-03-22T10:42:50.970 INFO:tasks.workunit.client.0.smithi115.stdout:5/737: read d2/f71 [718567,25093] 0 2024-03-22T10:42:50.971 INFO:tasks.workunit.client.1.smithi150.stdout:7/328: dread d2/d5/d7/f50 [0,4194304] 0 2024-03-22T10:42:50.971 INFO:tasks.workunit.client.1.smithi150.stdout:4/308: stat d0/d2f 0 2024-03-22T10:42:50.971 INFO:tasks.workunit.client.1.smithi150.stdout:7/329: stat d2/d5/d7/d9 0 2024-03-22T10:42:50.972 INFO:tasks.workunit.client.1.smithi150.stdout:1/283: rmdir d1/d19/d1a 39 2024-03-22T10:42:50.975 INFO:tasks.workunit.client.0.smithi115.stdout:5/738: mkdir d2/df/d13/d8b/d93/da6/dbc/d50/df5 0 2024-03-22T10:42:50.977 INFO:tasks.workunit.client.0.smithi115.stdout:6/644: write d4/d13/f25 [1147586,55912] 0 2024-03-22T10:42:50.982 INFO:tasks.workunit.client.0.smithi115.stdout:8/893: write d6/dd/d24/d4a/d7b/d8f/da0/fb8 [4272959,123357] 0 2024-03-22T10:42:50.982 INFO:tasks.workunit.client.0.smithi115.stdout:0/657: dread d3/d4/d14/f4f [0,4194304] 0 2024-03-22T10:42:50.983 INFO:tasks.workunit.client.1.smithi150.stdout:2/363: rename c9 to d12/d19/d2d/d68/d78/c7a 0 2024-03-22T10:42:50.985 INFO:tasks.workunit.client.1.smithi150.stdout:7/330: rmdir d2/d5/d7/d8 39 2024-03-22T10:42:50.985 INFO:tasks.workunit.client.0.smithi115.stdout:8/894: write d6/d49/f111 [984806,29303] 0 2024-03-22T10:42:50.989 INFO:tasks.workunit.client.1.smithi150.stdout:2/364: fdatasync d12/f13 0 2024-03-22T10:42:50.990 INFO:tasks.workunit.client.1.smithi150.stdout:9/374: dwrite d1/d8/de/df/d24/f89 [0,4194304] 0 2024-03-22T10:42:50.991 INFO:tasks.workunit.client.1.smithi150.stdout:1/284: chown d1/d19/d1a/c39 1 1 2024-03-22T10:42:50.993 INFO:tasks.workunit.client.1.smithi150.stdout:4/309: rename d0/d32/f4a to d0/d1b/d4d/f65 0 2024-03-22T10:42:50.995 INFO:tasks.workunit.client.0.smithi115.stdout:0/658: rename d3/d7/d1c/d6c/da6 to d3/da/d4c/d74/d84/dd7 0 2024-03-22T10:42:50.996 INFO:tasks.workunit.client.1.smithi150.stdout:2/365: chown d12/d48/d54/l76 1942580 1 2024-03-22T10:42:50.999 INFO:tasks.workunit.client.1.smithi150.stdout:7/331: write d2/d5/d7/d8/f2d [1945321,103209] 0 2024-03-22T10:42:50.999 INFO:tasks.workunit.client.0.smithi115.stdout:7/725: dwrite d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/ffa [0,4194304] 0 2024-03-22T10:42:50.999 INFO:tasks.workunit.client.0.smithi115.stdout:6/645: dread d4/d5e/f8b [0,4194304] 0 2024-03-22T10:42:51.000 INFO:tasks.workunit.client.0.smithi115.stdout:0/659: stat d3/d7/c2e 0 2024-03-22T10:42:51.002 INFO:tasks.workunit.client.1.smithi150.stdout:7/332: write d2/f60 [588735,45302] 0 2024-03-22T10:42:51.003 INFO:tasks.workunit.client.0.smithi115.stdout:9/713: dwrite d6/d8/d3c/d59/d68/d8e/f9a [0,4194304] 0 2024-03-22T10:42:51.004 INFO:tasks.workunit.client.0.smithi115.stdout:5/739: dwrite d2/d15/d1b/d7c/f63 [0,4194304] 0 2024-03-22T10:42:51.006 INFO:tasks.workunit.client.0.smithi115.stdout:6/646: truncate d4/d11/d12/d1c/f7f 5343665 0 2024-03-22T10:42:51.008 INFO:tasks.workunit.client.1.smithi150.stdout:0/354: dwrite d0/d12/d4f/d42/d10/d38/d52/f54 [0,4194304] 0 2024-03-22T10:42:51.010 INFO:tasks.workunit.client.0.smithi115.stdout:5/740: creat d2/d15/d1b/d7c/daf/ff6 x:0 0 0 2024-03-22T10:42:51.010 INFO:tasks.workunit.client.1.smithi150.stdout:1/285: creat d1/d19/d57/f5b x:0 0 0 2024-03-22T10:42:51.012 INFO:tasks.workunit.client.0.smithi115.stdout:7/726: rename d2/d7/d12/d10/d19/d42/f6d to d2/d7/d12/d10/d23/d3d/dd2/da3/df3/f100 0 2024-03-22T10:42:51.013 INFO:tasks.workunit.client.1.smithi150.stdout:2/366: symlink d12/d16/d67/l7b 0 2024-03-22T10:42:51.014 INFO:tasks.workunit.client.1.smithi150.stdout:0/355: write d0/d12/f3b [2119693,89200] 0 2024-03-22T10:42:51.014 INFO:tasks.workunit.client.1.smithi150.stdout:4/310: mknod d0/de/d57/d61/c66 0 2024-03-22T10:42:51.015 INFO:tasks.workunit.client.0.smithi115.stdout:9/714: unlink d6/d7/d20/d22/cc3 0 2024-03-22T10:42:51.017 INFO:tasks.workunit.client.0.smithi115.stdout:5/741: truncate d2/df/d13/d8b/d93/da6/dc3/fe3 252954 0 2024-03-22T10:42:51.017 INFO:tasks.workunit.client.1.smithi150.stdout:9/375: dwrite d1/f71 [0,4194304] 0 2024-03-22T10:42:51.018 INFO:tasks.workunit.client.1.smithi150.stdout:7/333: mkdir d2/d6e 0 2024-03-22T10:42:51.020 INFO:tasks.workunit.client.0.smithi115.stdout:7/727: write d2/d7/d12/d10/d19/d42/d76/f98 [3285589,43212] 0 2024-03-22T10:42:51.021 INFO:tasks.workunit.client.0.smithi115.stdout:9/715: creat d6/d7/d48/d70/da9/fff x:0 0 0 2024-03-22T10:42:51.023 INFO:tasks.workunit.client.1.smithi150.stdout:9/376: write d1/d8/de/df/f11 [3613094,11632] 0 2024-03-22T10:42:51.023 INFO:tasks.workunit.client.1.smithi150.stdout:1/286: dread d1/da/fb [0,4194304] 0 2024-03-22T10:42:51.029 INFO:tasks.workunit.client.1.smithi150.stdout:9/377: read d1/d8/de/df/d20/f50 [376238,6482] 0 2024-03-22T10:42:51.031 INFO:tasks.workunit.client.0.smithi115.stdout:9/716: creat d6/d7/d48/d53/dc9/f100 x:0 0 0 2024-03-22T10:42:51.031 INFO:tasks.workunit.client.1.smithi150.stdout:5/229: sync 2024-03-22T10:42:51.038 INFO:tasks.workunit.client.0.smithi115.stdout:3/697: dread d3/d19/d61/db3/fce [0,4194304] 0 2024-03-22T10:42:51.038 INFO:tasks.workunit.client.0.smithi115.stdout:7/728: dread d2/d7/d12/d10/f17 [4194304,4194304] 0 2024-03-22T10:42:51.039 INFO:tasks.workunit.client.1.smithi150.stdout:4/311: rename d0/de/f4c to d0/de/d57/f67 0 2024-03-22T10:42:51.039 INFO:tasks.workunit.client.0.smithi115.stdout:9/717: read - d6/d7/d20/d22/fe3 zero size 2024-03-22T10:42:51.039 INFO:tasks.workunit.client.0.smithi115.stdout:5/742: dwrite d2/fc [0,4194304] 0 2024-03-22T10:42:51.043 INFO:tasks.workunit.client.0.smithi115.stdout:5/743: chown d2/df/d13/d68 1658 1 2024-03-22T10:42:51.048 INFO:tasks.workunit.client.0.smithi115.stdout:5/744: write d2/df/d13/d8b/d93/da6/dbc/fb2 [1870997,90203] 0 2024-03-22T10:42:51.048 INFO:tasks.workunit.client.0.smithi115.stdout:9/718: rename d6/d7/d30/d37/dbe/fc6 to d6/d7/f101 0 2024-03-22T10:42:51.051 INFO:tasks.workunit.client.0.smithi115.stdout:7/729: rename d2/d7/fbe to d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/f101 0 2024-03-22T10:42:51.054 INFO:tasks.workunit.client.1.smithi150.stdout:9/378: rename d1/d8/de/df/d20/d32/d57 to d1/d8/de/df/d24/d58/d8c 0 2024-03-22T10:42:51.054 INFO:tasks.workunit.client.0.smithi115.stdout:5/745: creat d2/d15/d1b/d7c/d38/d43/d49/d81/dbd/ff7 x:0 0 0 2024-03-22T10:42:51.056 INFO:tasks.workunit.client.0.smithi115.stdout:5/746: write d2/d15/ff3 [553853,23424] 0 2024-03-22T10:42:51.062 INFO:tasks.workunit.client.0.smithi115.stdout:5/747: mknod d2/d15/d1b/d7c/d38/d43/d49/cf8 0 2024-03-22T10:42:51.062 INFO:tasks.workunit.client.1.smithi150.stdout:1/287: dwrite d1/da/d15/d18/f3d [0,4194304] 0 2024-03-22T10:42:51.063 INFO:tasks.workunit.client.1.smithi150.stdout:5/230: mkdir d0/d8/dc/de/d14/d2e/d46 0 2024-03-22T10:42:51.065 INFO:tasks.workunit.client.0.smithi115.stdout:5/748: chown d2/d15/d1b/d7c/d38/d43/fca 7250314 1 2024-03-22T10:42:51.066 INFO:tasks.workunit.client.0.smithi115.stdout:6/647: dread d4/d36/d48/f67 [0,4194304] 0 2024-03-22T10:42:51.068 INFO:tasks.workunit.client.1.smithi150.stdout:9/379: dread - d1/d8/de/df/d36/d40/d44/f6e zero size 2024-03-22T10:42:51.068 INFO:tasks.workunit.client.0.smithi115.stdout:5/749: chown d2/df/d13/d8b/d93/da6/dbc/fb2 0 1 2024-03-22T10:42:51.069 INFO:tasks.workunit.client.1.smithi150.stdout:8/336: sync 2024-03-22T10:42:51.069 INFO:tasks.workunit.client.1.smithi150.stdout:6/276: sync 2024-03-22T10:42:51.069 INFO:tasks.workunit.client.1.smithi150.stdout:3/325: sync 2024-03-22T10:42:51.070 INFO:tasks.workunit.client.1.smithi150.stdout:7/334: dwrite d2/d5/d7/d8/d57/d4a/f22 [0,4194304] 0 2024-03-22T10:42:51.071 INFO:tasks.workunit.client.0.smithi115.stdout:7/730: dwrite d2/d7/d12/d10/d19/fc1 [0,4194304] 0 2024-03-22T10:42:51.071 INFO:tasks.workunit.client.0.smithi115.stdout:6/648: chown d4/d11/d12/l55 29896666 1 2024-03-22T10:42:51.072 INFO:tasks.workunit.client.1.smithi150.stdout:2/367: dwrite d12/d19/d2d/f53 [0,4194304] 0 2024-03-22T10:42:51.073 INFO:tasks.workunit.client.0.smithi115.stdout:5/750: dread - d2/d15/d1b/d7c/f8a zero size 2024-03-22T10:42:51.074 INFO:tasks.workunit.client.0.smithi115.stdout:7/731: truncate d2/d62/fa0 1743414 0 2024-03-22T10:42:51.074 INFO:tasks.workunit.client.1.smithi150.stdout:9/380: chown d1/d8/de/df/d36/d40/f7f 68073 1 2024-03-22T10:42:51.075 INFO:tasks.workunit.client.0.smithi115.stdout:6/649: mkdir d4/d5e/dd0 0 2024-03-22T10:42:51.076 INFO:tasks.workunit.client.1.smithi150.stdout:0/356: dwrite d0/d12/d4f/d11/d66/d35/f62 [0,4194304] 0 2024-03-22T10:42:51.081 INFO:tasks.workunit.client.1.smithi150.stdout:5/231: mkdir d0/d26/d47 0 2024-03-22T10:42:51.081 INFO:tasks.workunit.client.0.smithi115.stdout:6/650: chown d4/d11/d12/d1c/d8a/lae 5165823 1 2024-03-22T10:42:51.082 INFO:tasks.workunit.client.0.smithi115.stdout:3/698: truncate d3/d19/f4e 2722717 0 2024-03-22T10:42:51.085 INFO:tasks.workunit.client.1.smithi150.stdout:9/381: chown d1/d8/de/df 2354 1 2024-03-22T10:42:51.086 INFO:tasks.workunit.client.1.smithi150.stdout:4/312: getdents d0/d47 0 2024-03-22T10:42:51.087 INFO:tasks.workunit.client.0.smithi115.stdout:5/751: rename d2/df/d13/l25 to d2/df/d13/d68/lf9 0 2024-03-22T10:42:51.090 INFO:tasks.workunit.client.1.smithi150.stdout:7/335: read - d2/d5/d7/d9/f53 zero size 2024-03-22T10:42:51.094 INFO:tasks.workunit.client.0.smithi115.stdout:5/752: fsync d2/df/d13/d8b/d93/da6/fa8 0 2024-03-22T10:42:51.096 INFO:tasks.workunit.client.0.smithi115.stdout:6/651: truncate d4/d13/d33/f6c 518614 0 2024-03-22T10:42:51.100 INFO:tasks.workunit.client.0.smithi115.stdout:5/753: chown d2/d15/d1b/d7c/d38/d4a 1 1 2024-03-22T10:42:51.102 INFO:tasks.workunit.client.1.smithi150.stdout:9/382: creat d1/d8/de/df/d20/d32/d75/f8d x:0 0 0 2024-03-22T10:42:51.103 INFO:tasks.workunit.client.0.smithi115.stdout:9/719: dwrite d6/d7/ff [0,4194304] 0 2024-03-22T10:42:51.104 INFO:tasks.workunit.client.1.smithi150.stdout:1/288: getdents d1/da 0 2024-03-22T10:42:51.105 INFO:tasks.workunit.client.1.smithi150.stdout:3/326: mknod d5/d22/d2f/d38/d59/d5d/c67 0 2024-03-22T10:42:51.108 INFO:tasks.workunit.client.0.smithi115.stdout:9/720: fsync f0 0 2024-03-22T10:42:51.108 INFO:tasks.workunit.client.1.smithi150.stdout:4/313: mkdir d0/d1b/d68 0 2024-03-22T10:42:51.110 INFO:tasks.workunit.client.0.smithi115.stdout:6/652: fsync d4/d13/f3d 0 2024-03-22T10:42:51.112 INFO:tasks.workunit.client.1.smithi150.stdout:2/368: dread d12/f37 [0,4194304] 0 2024-03-22T10:42:51.114 INFO:tasks.workunit.client.1.smithi150.stdout:3/327: readlink d5/d22/d2f/d38/d59/l5f 0 2024-03-22T10:42:51.115 INFO:tasks.workunit.client.0.smithi115.stdout:3/699: dwrite d3/d19/d2b/d49/d91/f9f [0,4194304] 0 2024-03-22T10:42:51.115 INFO:tasks.workunit.client.0.smithi115.stdout:6/653: fsync d4/d13/f25 0 2024-03-22T10:42:51.117 INFO:tasks.workunit.client.1.smithi150.stdout:6/277: link db/dd/d15/d28/l59 db/dd/d15/d60/l69 0 2024-03-22T10:42:51.117 INFO:tasks.workunit.client.1.smithi150.stdout:2/369: chown d12/f15 10 1 2024-03-22T10:42:51.117 INFO:tasks.workunit.client.1.smithi150.stdout:5/232: dread d0/d8/dc/f40 [0,4194304] 0 2024-03-22T10:42:51.117 INFO:tasks.workunit.client.1.smithi150.stdout:7/336: getdents d2/d5/d7/d8/d57/d4a 0 2024-03-22T10:42:51.118 INFO:tasks.workunit.client.0.smithi115.stdout:3/700: read - d3/d19/d2b/d49/fbd zero size 2024-03-22T10:42:51.122 INFO:tasks.workunit.client.1.smithi150.stdout:0/357: getdents d0/d1 0 2024-03-22T10:42:51.122 INFO:tasks.workunit.client.0.smithi115.stdout:5/754: dread d2/d15/d1b/d7c/d38/d56/fcd [0,4194304] 0 2024-03-22T10:42:51.126 INFO:tasks.workunit.client.1.smithi150.stdout:5/233: stat d0/c4 0 2024-03-22T10:42:51.126 INFO:tasks.workunit.client.1.smithi150.stdout:2/370: creat d12/d19/d1a/f7c x:0 0 0 2024-03-22T10:42:51.126 INFO:tasks.workunit.client.0.smithi115.stdout:9/721: link d6/d8/f47 d6/d7/d20/d38/f102 0 2024-03-22T10:42:51.128 INFO:tasks.workunit.client.1.smithi150.stdout:6/278: mkdir db/dd/d15/d6a 0 2024-03-22T10:42:51.129 INFO:tasks.workunit.client.0.smithi115.stdout:3/701: rename d3/d19/d61/da6/fb7 to d3/d36/fee 0 2024-03-22T10:42:51.132 INFO:tasks.workunit.client.1.smithi150.stdout:4/314: rename d0/d1b/f60 to d0/de/d57/d61/f69 0 2024-03-22T10:42:51.136 INFO:tasks.workunit.client.0.smithi115.stdout:6/654: dwrite d4/d5e/fc9 [0,4194304] 0 2024-03-22T10:42:51.137 INFO:tasks.workunit.client.0.smithi115.stdout:3/702: rename d3/d19/d2b/l5d to d3/d6d/db4/lef 0 2024-03-22T10:42:51.141 INFO:tasks.workunit.client.1.smithi150.stdout:2/371: chown d12/d48/f72 12 1 2024-03-22T10:42:51.142 INFO:tasks.workunit.client.0.smithi115.stdout:3/703: creat d3/d82/d57/ff0 x:0 0 0 2024-03-22T10:42:51.142 INFO:tasks.workunit.client.0.smithi115.stdout:5/755: dwrite d2/d15/d1b/d7c/d38/f61 [4194304,4194304] 0 2024-03-22T10:42:51.143 INFO:tasks.workunit.client.1.smithi150.stdout:7/337: creat d2/d5/d7/d8/d2c/f6f x:0 0 0 2024-03-22T10:42:51.145 INFO:tasks.workunit.client.0.smithi115.stdout:3/704: chown d3/d19/d2b/d49/fc6 1 1 2024-03-22T10:42:51.146 INFO:tasks.workunit.client.0.smithi115.stdout:3/705: readlink d3/l10 0 2024-03-22T10:42:51.147 INFO:tasks.workunit.client.0.smithi115.stdout:7/732: dread d2/d7/d12/d10/d23/d3d/dd2/da3/de8/dc7/fe0 [0,4194304] 0 2024-03-22T10:42:51.148 INFO:tasks.workunit.client.0.smithi115.stdout:3/706: chown d3/d36/d41/d33 401597899 1 2024-03-22T10:42:51.149 INFO:tasks.workunit.client.1.smithi150.stdout:1/289: dwrite d1/f7 [4194304,4194304] 0 2024-03-22T10:42:51.149 INFO:tasks.workunit.client.0.smithi115.stdout:3/707: dread - d3/d19/d2b/d49/fbd zero size 2024-03-22T10:42:51.150 INFO:tasks.workunit.client.0.smithi115.stdout:7/733: write d2/d7/d12/d10/d23/fd4 [260581,19720] 0 2024-03-22T10:42:51.151 INFO:tasks.workunit.client.1.smithi150.stdout:9/383: dwrite d1/d8/f65 [0,4194304] 0 2024-03-22T10:42:51.155 INFO:tasks.workunit.client.0.smithi115.stdout:7/734: mkdir d2/d7/d102 0 2024-03-22T10:42:51.155 INFO:tasks.workunit.client.1.smithi150.stdout:4/315: link d0/d1b/f39 d0/d1b/d68/f6a 0 2024-03-22T10:42:51.157 INFO:tasks.workunit.client.1.smithi150.stdout:2/372: stat d12/d16 0 2024-03-22T10:42:51.163 INFO:tasks.workunit.client.1.smithi150.stdout:7/338: write d2/d5/d7/d8/d46/f66 [4578494,118642] 0 2024-03-22T10:42:51.166 INFO:tasks.workunit.client.0.smithi115.stdout:9/722: write d6/d7/d20/f4c [120750,84749] 0 2024-03-22T10:42:51.166 INFO:tasks.workunit.client.1.smithi150.stdout:5/234: truncate d0/d8/dc/d1d/f24 758837 0 2024-03-22T10:42:51.166 INFO:tasks.workunit.client.0.smithi115.stdout:6/655: write d4/d11/f2a [377445,87035] 0 2024-03-22T10:42:51.170 INFO:tasks.workunit.client.0.smithi115.stdout:1/776: sync 2024-03-22T10:42:51.170 INFO:tasks.workunit.client.0.smithi115.stdout:2/637: sync 2024-03-22T10:42:51.174 INFO:tasks.workunit.client.1.smithi150.stdout:2/373: chown d12/d16/d2c/d73 0 1 2024-03-22T10:42:51.174 INFO:tasks.workunit.client.0.smithi115.stdout:9/723: rename d6/d8/f1a to d6/d7/d87/f103 0 2024-03-22T10:42:51.174 INFO:tasks.workunit.client.0.smithi115.stdout:2/638: read - da/d25/d3d/d57/d7d/fac zero size 2024-03-22T10:42:51.175 INFO:tasks.workunit.client.1.smithi150.stdout:8/337: dwrite d0/d2/d19/d29/d2f/df/d1b/f30 [0,4194304] 0 2024-03-22T10:42:51.175 INFO:tasks.workunit.client.1.smithi150.stdout:0/358: dwrite d0/d12/d4f/d11/d26/f2e [0,4194304] 0 2024-03-22T10:42:51.178 INFO:tasks.workunit.client.0.smithi115.stdout:3/708: write d3/f50 [1292213,4401] 0 2024-03-22T10:42:51.180 INFO:tasks.workunit.client.1.smithi150.stdout:3/328: dwrite d5/fb [0,4194304] 0 2024-03-22T10:42:51.180 INFO:tasks.workunit.client.1.smithi150.stdout:9/384: truncate d1/d8/de/df/d36/f7c 22093 0 2024-03-22T10:42:51.181 INFO:tasks.workunit.client.1.smithi150.stdout:5/235: readlink d0/d8/dc/de/d14/d2e/d45/l36 0 2024-03-22T10:42:51.182 INFO:tasks.workunit.client.0.smithi115.stdout:9/724: rename d6/d7/d6a/fd9 to d6/d7/d6a/d6f/f104 0 2024-03-22T10:42:51.183 INFO:tasks.workunit.client.0.smithi115.stdout:3/709: mknod d3/d82/d54/cf1 0 2024-03-22T10:42:51.184 INFO:tasks.workunit.client.0.smithi115.stdout:9/725: rmdir d6 39 2024-03-22T10:42:51.185 INFO:tasks.workunit.client.1.smithi150.stdout:8/338: fsync d0/d2/d19/d29/d2f/df/d1b/f75 0 2024-03-22T10:42:51.186 INFO:tasks.workunit.client.0.smithi115.stdout:6/656: dwrite d4/f30 [0,4194304] 0 2024-03-22T10:42:51.187 INFO:tasks.workunit.client.0.smithi115.stdout:3/710: mknod d3/d82/d57/cf2 0 2024-03-22T10:42:51.188 INFO:tasks.workunit.client.1.smithi150.stdout:9/385: fdatasync d1/d8/de/df/d36/d40/d41/f61 0 2024-03-22T10:42:51.189 INFO:tasks.workunit.client.0.smithi115.stdout:1/777: dwrite d0/d8/f53 [0,4194304] 0 2024-03-22T10:42:51.191 INFO:tasks.workunit.client.1.smithi150.stdout:8/339: dread - d0/f5b zero size 2024-03-22T10:42:51.191 INFO:tasks.workunit.client.1.smithi150.stdout:0/359: chown d0/d12/d4f/d42/d10/f13 1183190 1 2024-03-22T10:42:51.193 INFO:tasks.workunit.client.1.smithi150.stdout:4/316: link d0/d1b/l40 d0/d1b/d4d/l6b 0 2024-03-22T10:42:51.193 INFO:tasks.workunit.client.1.smithi150.stdout:7/339: symlink d2/d5/d7/d9/db/l70 0 2024-03-22T10:42:51.194 INFO:tasks.workunit.client.0.smithi115.stdout:3/711: creat d3/d16/d29/dc1/ff3 x:0 0 0 2024-03-22T10:42:51.197 INFO:tasks.workunit.client.1.smithi150.stdout:7/340: stat d2/d5/d7/d9 0 2024-03-22T10:42:51.198 INFO:tasks.workunit.client.0.smithi115.stdout:3/712: chown d3/d16/d29/d35/l7d 0 1 2024-03-22T10:42:51.199 INFO:tasks.workunit.client.0.smithi115.stdout:7/735: write d2/d7/d12/d10/d19/d42/d46/f68 [440665,110405] 0 2024-03-22T10:42:51.199 INFO:tasks.workunit.client.0.smithi115.stdout:6/657: creat d4/d11/d12/d1c/d8a/d98/d9f/dc3/fd1 x:0 0 0 2024-03-22T10:42:51.200 INFO:tasks.workunit.client.0.smithi115.stdout:1/778: mknod d0/d12/d17/c10f 0 2024-03-22T10:42:51.201 INFO:tasks.workunit.client.1.smithi150.stdout:9/386: symlink d1/d8/de/l8e 0 2024-03-22T10:42:51.201 INFO:tasks.workunit.client.1.smithi150.stdout:0/360: fdatasync d0/d12/d4f/d42/f72 0 2024-03-22T10:42:51.203 INFO:tasks.workunit.client.1.smithi150.stdout:9/387: fsync d1/d8/de/f2c 0 2024-03-22T10:42:51.203 INFO:tasks.workunit.client.0.smithi115.stdout:3/713: mkdir d3/d82/d54/df4 0 2024-03-22T10:42:51.205 INFO:tasks.workunit.client.0.smithi115.stdout:5/756: read d2/d15/d1b/f7f [48657,72029] 0 2024-03-22T10:42:51.207 INFO:tasks.workunit.client.0.smithi115.stdout:7/736: symlink d2/l103 0 2024-03-22T10:42:51.207 INFO:tasks.workunit.client.0.smithi115.stdout:6/658: readlink d4/d11/d99/la8 0 2024-03-22T10:42:51.209 INFO:tasks.workunit.client.1.smithi150.stdout:3/329: rename d5/da/f19 to d5/d22/d2f/f68 0 2024-03-22T10:42:51.209 INFO:tasks.workunit.client.1.smithi150.stdout:9/388: chown d1/d8/de/d73 31 1 2024-03-22T10:42:51.210 INFO:tasks.workunit.client.0.smithi115.stdout:7/737: chown d2/d7/d12/d10/d19/d42/d76/f98 371957152 1 2024-03-22T10:42:51.211 INFO:tasks.workunit.client.0.smithi115.stdout:3/714: mkdir d3/d19/df5 0 2024-03-22T10:42:51.212 INFO:tasks.workunit.client.0.smithi115.stdout:1/779: creat d0/d12/d17/d67/de6/f110 x:0 0 0 2024-03-22T10:42:51.212 INFO:tasks.workunit.client.1.smithi150.stdout:3/330: write d5/f10 [5321765,37593] 0 2024-03-22T10:42:51.214 INFO:tasks.workunit.client.0.smithi115.stdout:5/757: mknod d2/d15/d1b/d7c/d38/d4a/cfa 0 2024-03-22T10:42:51.216 INFO:tasks.workunit.client.1.smithi150.stdout:5/236: dread d0/d8/dc/de/d14/d2e/f3e [0,4194304] 0 2024-03-22T10:42:51.216 INFO:tasks.workunit.client.0.smithi115.stdout:6/659: symlink d4/d11/ld2 0 2024-03-22T10:42:51.216 INFO:tasks.workunit.client.1.smithi150.stdout:3/331: write d5/d53/f54 [508973,127299] 0 2024-03-22T10:42:51.218 INFO:tasks.workunit.client.1.smithi150.stdout:6/279: dread db/dd/f37 [0,4194304] 0 2024-03-22T10:42:51.218 INFO:tasks.workunit.client.0.smithi115.stdout:7/738: dread d2/d7/d12/d10/f96 [0,4194304] 0 2024-03-22T10:42:51.218 INFO:tasks.workunit.client.0.smithi115.stdout:3/715: fsync d3/d36/d41/f48 0 2024-03-22T10:42:51.222 INFO:tasks.workunit.client.0.smithi115.stdout:1/780: rename d0/d22/f86 to d0/d12/d17/d67/dd8/f111 0 2024-03-22T10:42:51.226 INFO:tasks.workunit.client.1.smithi150.stdout:0/361: mknod d0/d12/d4f/d42/d10/d53/c75 0 2024-03-22T10:42:51.227 INFO:tasks.workunit.client.1.smithi150.stdout:8/340: write d0/d2/d19/d29/d2f/df/f42 [2243705,9624] 0 2024-03-22T10:42:51.227 INFO:tasks.workunit.client.1.smithi150.stdout:2/374: dread d12/d16/d2c/f57 [0,4194304] 0 2024-03-22T10:42:51.227 INFO:tasks.workunit.client.1.smithi150.stdout:5/237: unlink d0/lb 0 2024-03-22T10:42:51.229 INFO:tasks.workunit.client.1.smithi150.stdout:5/238: stat d0/d26 0 2024-03-22T10:42:51.230 INFO:tasks.workunit.client.1.smithi150.stdout:7/341: mkdir d2/d6e/d71 0 2024-03-22T10:42:51.231 INFO:tasks.workunit.client.1.smithi150.stdout:9/389: write d1/d8/de/df/d36/d40/d41/f47 [2980094,88335] 0 2024-03-22T10:42:51.234 INFO:tasks.workunit.client.0.smithi115.stdout:7/739: creat d2/d7/d12/d10/d19/dbd/f104 x:0 0 0 2024-03-22T10:42:51.234 INFO:tasks.workunit.client.1.smithi150.stdout:2/375: chown d12/d16/d67/d3c/d75 111298 1 2024-03-22T10:42:51.236 INFO:tasks.workunit.client.1.smithi150.stdout:0/362: unlink d0/d12/d4f/d5a/l6f 0 2024-03-22T10:42:51.237 INFO:tasks.workunit.client.0.smithi115.stdout:2/639: dwrite da/d25/f34 [0,4194304] 0 2024-03-22T10:42:51.238 INFO:tasks.workunit.client.0.smithi115.stdout:3/716: getdents d3/d82/d57 0 2024-03-22T10:42:51.239 INFO:tasks.workunit.client.0.smithi115.stdout:1/781: truncate d0/d8/de/d1a/f24 1109478 0 2024-03-22T10:42:51.240 INFO:tasks.workunit.client.1.smithi150.stdout:5/239: write d0/f3 [4868880,70289] 0 2024-03-22T10:42:51.240 INFO:tasks.workunit.client.1.smithi150.stdout:9/390: mknod d1/d49/c8f 0 2024-03-22T10:42:51.240 INFO:tasks.workunit.client.0.smithi115.stdout:5/758: dread d2/df/fd1 [0,4194304] 0 2024-03-22T10:42:51.241 INFO:tasks.workunit.client.1.smithi150.stdout:3/332: creat d5/d22/d2f/d38/d3e/d5b/f69 x:0 0 0 2024-03-22T10:42:51.242 INFO:tasks.workunit.client.0.smithi115.stdout:2/640: chown da/d1c/d67/d78/c9a 1 1 2024-03-22T10:42:51.242 INFO:tasks.workunit.client.1.smithi150.stdout:6/280: creat db/dd/d15/d6a/f6b x:0 0 0 2024-03-22T10:42:51.243 INFO:tasks.workunit.client.0.smithi115.stdout:7/740: read - d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f95 zero size 2024-03-22T10:42:51.245 INFO:tasks.workunit.client.0.smithi115.stdout:1/782: chown d0/d8/db/f2d 12 1 2024-03-22T10:42:51.250 INFO:tasks.workunit.client.0.smithi115.stdout:7/741: dread - d2/d7/d12/d10/d19/d42/d46/ffc zero size 2024-03-22T10:42:51.250 INFO:tasks.workunit.client.1.smithi150.stdout:5/240: mknod d0/d26/c48 0 2024-03-22T10:42:51.250 INFO:tasks.workunit.client.1.smithi150.stdout:2/376: rmdir d12/d16/d70 0 2024-03-22T10:42:51.254 INFO:tasks.workunit.client.1.smithi150.stdout:6/281: write db/dd/d25/f3d [1028260,6482] 0 2024-03-22T10:42:51.254 INFO:tasks.workunit.client.1.smithi150.stdout:2/377: write d12/d16/d2c/f33 [1160188,7517] 0 2024-03-22T10:42:51.255 INFO:tasks.workunit.client.0.smithi115.stdout:2/641: rename da/d25/d3d/f28 to da/d25/d3d/d57/d7d/fc0 0 2024-03-22T10:42:51.255 INFO:tasks.workunit.client.0.smithi115.stdout:9/726: dread d6/d7/d30/d3b/f58 [4194304,4194304] 0 2024-03-22T10:42:51.255 INFO:tasks.workunit.client.0.smithi115.stdout:3/717: creat d3/d19/d61/daa/ff6 x:0 0 0 2024-03-22T10:42:51.256 INFO:tasks.workunit.client.1.smithi150.stdout:4/317: rename d0/de/d57 to d0/d1b/d4d/d6c 0 2024-03-22T10:42:51.257 INFO:tasks.workunit.client.1.smithi150.stdout:2/378: truncate d12/d48/f44 715528 0 2024-03-22T10:42:51.258 INFO:tasks.workunit.client.1.smithi150.stdout:3/333: unlink d5/f21 0 2024-03-22T10:42:51.259 INFO:tasks.workunit.client.1.smithi150.stdout:2/379: write d12/d16/d67/f3d [1572423,37571] 0 2024-03-22T10:42:51.260 INFO:tasks.workunit.client.0.smithi115.stdout:7/742: chown d2/d7/d12/d10/d19/d42/fc4 107 1 2024-03-22T10:42:51.263 INFO:tasks.workunit.client.0.smithi115.stdout:7/743: chown d2/d7/d12/fb8 61573669 1 2024-03-22T10:42:51.263 INFO:tasks.workunit.client.0.smithi115.stdout:6/660: dread d4/d11/f23 [0,4194304] 0 2024-03-22T10:42:51.265 INFO:tasks.workunit.client.1.smithi150.stdout:0/363: truncate d0/d12/d4f/d42/fc 1357059 0 2024-03-22T10:42:51.272 INFO:tasks.workunit.client.1.smithi150.stdout:4/318: symlink d0/l6d 0 2024-03-22T10:42:51.273 INFO:tasks.workunit.client.1.smithi150.stdout:9/391: link d1/d8/de/df/d24/d4e/l64 d1/d19/l90 0 2024-03-22T10:42:51.275 INFO:tasks.workunit.client.1.smithi150.stdout:2/380: mkdir d12/d35/d7d 0 2024-03-22T10:42:51.279 INFO:tasks.workunit.client.1.smithi150.stdout:2/381: readlink d12/d19/l24 0 2024-03-22T10:42:51.280 INFO:tasks.workunit.client.1.smithi150.stdout:9/392: dread - d1/d8/de/df/d20/d29/f54 zero size 2024-03-22T10:42:51.280 INFO:tasks.workunit.client.1.smithi150.stdout:1/290: write d1/d19/d1a/f49 [726354,104298] 0 2024-03-22T10:42:51.281 INFO:tasks.workunit.client.1.smithi150.stdout:2/382: stat d12 0 2024-03-22T10:42:51.283 INFO:tasks.workunit.client.1.smithi150.stdout:4/319: mknod d0/d1b/d4d/d6c/c6e 0 2024-03-22T10:42:51.284 INFO:tasks.workunit.client.0.smithi115.stdout:1/783: symlink d0/d22/l112 0 2024-03-22T10:42:51.284 INFO:tasks.workunit.client.1.smithi150.stdout:7/342: dread d2/d5/d7/d8/d57/d54/f55 [0,4194304] 0 2024-03-22T10:42:51.285 INFO:tasks.workunit.client.0.smithi115.stdout:2/642: fdatasync da/d25/f37 0 2024-03-22T10:42:51.285 INFO:tasks.workunit.client.0.smithi115.stdout:9/727: dread d6/d8/d3c/f96 [0,4194304] 0 2024-03-22T10:42:51.288 INFO:tasks.workunit.client.0.smithi115.stdout:1/784: dread - d0/d8/de/f107 zero size 2024-03-22T10:42:51.288 INFO:tasks.workunit.client.0.smithi115.stdout:5/759: dwrite d2/f4 [4194304,4194304] 0 2024-03-22T10:42:51.288 INFO:tasks.workunit.client.1.smithi150.stdout:4/320: readlink d0/l15 0 2024-03-22T10:42:51.290 INFO:tasks.workunit.client.0.smithi115.stdout:7/744: mkdir d2/d7/d12/d10/d19/dbd/dce/d105 0 2024-03-22T10:42:51.291 INFO:tasks.workunit.client.0.smithi115.stdout:1/785: fdatasync d0/d12/d17/f7d 0 2024-03-22T10:42:51.291 INFO:tasks.workunit.client.0.smithi115.stdout:2/643: chown da/d25/d3d/d57/l9c 0 1 2024-03-22T10:42:51.294 INFO:tasks.workunit.client.1.smithi150.stdout:6/282: dwrite db/dd/d15/f35 [0,4194304] 0 2024-03-22T10:42:51.295 INFO:tasks.workunit.client.0.smithi115.stdout:6/661: dread d4/d11/d12/d1c/f2d [0,4194304] 0 2024-03-22T10:42:51.295 INFO:tasks.workunit.client.1.smithi150.stdout:8/341: write d0/f17 [4590997,117564] 0 2024-03-22T10:42:51.297 INFO:tasks.workunit.client.0.smithi115.stdout:9/728: creat d6/d7/d6a/f105 x:0 0 0 2024-03-22T10:42:51.298 INFO:tasks.workunit.client.0.smithi115.stdout:6/662: read - d4/d11/d12/d1c/d8a/d98/d9f/fbb zero size 2024-03-22T10:42:51.298 INFO:tasks.workunit.client.0.smithi115.stdout:1/786: fdatasync d0/d12/f30 0 2024-03-22T10:42:51.301 INFO:tasks.workunit.client.0.smithi115.stdout:7/745: read d2/d7/d12/d10/d19/d42/d46/f5e [457775,77376] 0 2024-03-22T10:42:51.301 INFO:tasks.workunit.client.1.smithi150.stdout:6/283: stat db/f4a 0 2024-03-22T10:42:51.301 INFO:tasks.workunit.client.1.smithi150.stdout:3/334: dwrite d5/f8 [0,4194304] 0 2024-03-22T10:42:51.302 INFO:tasks.workunit.client.0.smithi115.stdout:3/718: link d3/d36/d41/d85/fcc d3/d19/d2b/d49/ff7 0 2024-03-22T10:42:51.303 INFO:tasks.workunit.client.1.smithi150.stdout:1/291: fdatasync d1/da/d15/d18/f4a 0 2024-03-22T10:42:51.305 INFO:tasks.workunit.client.0.smithi115.stdout:5/760: unlink d2/l9 0 2024-03-22T10:42:51.306 INFO:tasks.workunit.client.0.smithi115.stdout:4/693: sync 2024-03-22T10:42:51.308 INFO:tasks.workunit.client.0.smithi115.stdout:9/729: unlink d6/d8/d3c/f96 0 2024-03-22T10:42:51.311 INFO:tasks.workunit.client.1.smithi150.stdout:4/321: mknod d0/d2f/c6f 0 2024-03-22T10:42:51.315 INFO:tasks.workunit.client.1.smithi150.stdout:5/241: dwrite d0/d8/dc/de/d14/d2e/f3e [0,4194304] 0 2024-03-22T10:42:51.315 INFO:tasks.workunit.client.0.smithi115.stdout:5/761: mkdir d2/df/d3b/dfb 0 2024-03-22T10:42:51.321 INFO:tasks.workunit.client.1.smithi150.stdout:6/284: mkdir db/dd/d15/d28/d6c 0 2024-03-22T10:42:51.325 INFO:tasks.workunit.client.1.smithi150.stdout:1/292: write d1/d19/d57/f5b [684206,78989] 0 2024-03-22T10:42:51.325 INFO:tasks.workunit.client.1.smithi150.stdout:9/393: link d1/d8/de/df/d36/d40/f7f d1/d8/de/df/d24/d58/f91 0 2024-03-22T10:42:51.325 INFO:tasks.workunit.client.1.smithi150.stdout:0/364: dwrite d0/f6d [0,4194304] 0 2024-03-22T10:42:51.328 INFO:tasks.workunit.client.1.smithi150.stdout:4/322: creat d0/de/f70 x:0 0 0 2024-03-22T10:42:51.331 INFO:tasks.workunit.client.0.smithi115.stdout:9/730: rename d6/d8/d3c/d59/d68/d97 to d6/d7/d20/d22/d23/d106 0 2024-03-22T10:42:51.332 INFO:tasks.workunit.client.1.smithi150.stdout:7/343: creat d2/d5/d7/d8/d2c/d45/f72 x:0 0 0 2024-03-22T10:42:51.334 INFO:tasks.workunit.client.0.smithi115.stdout:1/787: link d0/d68/fae d0/d12/d17/d37/d5c/f113 0 2024-03-22T10:42:51.335 INFO:tasks.workunit.client.0.smithi115.stdout:2/644: link da/d25/d3d/d57/c5a da/d1c/d67/d78/d86/cc1 0 2024-03-22T10:42:51.338 INFO:tasks.workunit.client.1.smithi150.stdout:9/394: mknod d1/d8/de/df/d20/d32/c92 0 2024-03-22T10:42:51.339 INFO:tasks.workunit.client.1.smithi150.stdout:0/365: mkdir d0/d12/d4f/d11/d26/d76 0 2024-03-22T10:42:51.343 INFO:tasks.workunit.client.1.smithi150.stdout:7/344: write d2/d5/d7/d8/d2c/f6f [424594,58636] 0 2024-03-22T10:42:51.344 INFO:tasks.workunit.client.0.smithi115.stdout:9/731: unlink d6/f62 0 2024-03-22T10:42:51.345 INFO:tasks.workunit.client.1.smithi150.stdout:0/366: dread - d0/f63 zero size 2024-03-22T10:42:51.346 INFO:tasks.workunit.client.0.smithi115.stdout:2/645: mkdir da/d1c/d67/d78/d86/d9d/dc2 0 2024-03-22T10:42:51.347 INFO:tasks.workunit.client.1.smithi150.stdout:8/342: dwrite d0/f1 [4194304,4194304] 0 2024-03-22T10:42:51.348 INFO:tasks.workunit.client.1.smithi150.stdout:5/242: rmdir d0/d8/dc/de/d14/d2e/d45/d3f 0 2024-03-22T10:42:51.350 INFO:tasks.workunit.client.0.smithi115.stdout:7/746: getdents d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae 0 2024-03-22T10:42:51.351 INFO:tasks.workunit.client.1.smithi150.stdout:3/335: dwrite d5/d22/d2f/d38/f42 [0,4194304] 0 2024-03-22T10:42:51.352 INFO:tasks.workunit.client.1.smithi150.stdout:9/395: write d1/d8/de/df/d24/d4e/f77 [769971,15299] 0 2024-03-22T10:42:51.357 INFO:tasks.workunit.client.1.smithi150.stdout:8/343: read d0/d3c/f56 [122167,104653] 0 2024-03-22T10:42:51.359 INFO:tasks.workunit.client.1.smithi150.stdout:3/336: chown d5/d22/d2f/f68 6 1 2024-03-22T10:42:51.360 INFO:tasks.workunit.client.0.smithi115.stdout:7/747: symlink d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/l106 0 2024-03-22T10:42:51.365 INFO:tasks.workunit.client.1.smithi150.stdout:3/337: read d5/d22/d2f/f68 [4064031,124598] 0 2024-03-22T10:42:51.366 INFO:tasks.workunit.client.1.smithi150.stdout:2/383: dread d12/d16/d67/f3d [0,4194304] 0 2024-03-22T10:42:51.369 INFO:tasks.workunit.client.1.smithi150.stdout:3/338: chown d5/f25 397 1 2024-03-22T10:42:51.372 INFO:tasks.workunit.client.1.smithi150.stdout:9/396: creat d1/d8/de/df/d24/d4e/f93 x:0 0 0 2024-03-22T10:42:51.379 INFO:tasks.workunit.client.1.smithi150.stdout:2/384: fdatasync d12/d19/d1a/f30 0 2024-03-22T10:42:51.380 INFO:tasks.workunit.client.1.smithi150.stdout:0/367: creat d0/d12/d4f/d42/f77 x:0 0 0 2024-03-22T10:42:51.381 INFO:tasks.workunit.client.1.smithi150.stdout:2/385: fdatasync d12/d16/d67/d3c/f77 0 2024-03-22T10:42:51.382 INFO:tasks.workunit.client.1.smithi150.stdout:8/344: mknod d0/c76 0 2024-03-22T10:42:51.382 INFO:tasks.workunit.client.1.smithi150.stdout:6/285: dwrite db/dd/d15/f20 [0,4194304] 0 2024-03-22T10:42:51.389 INFO:tasks.workunit.client.1.smithi150.stdout:6/286: dread - db/dd/f5b zero size 2024-03-22T10:42:51.390 INFO:tasks.workunit.client.1.smithi150.stdout:3/339: symlink d5/d22/d2f/d38/d59/d5d/l6a 0 2024-03-22T10:42:51.393 INFO:tasks.workunit.client.1.smithi150.stdout:4/323: dread d0/f14 [0,4194304] 0 2024-03-22T10:42:51.395 INFO:tasks.workunit.client.1.smithi150.stdout:8/345: creat d0/d2/d46/f77 x:0 0 0 2024-03-22T10:42:51.398 INFO:tasks.workunit.client.1.smithi150.stdout:0/368: mknod d0/d12/d4f/d11/d66/d35/d5f/c78 0 2024-03-22T10:42:51.399 INFO:tasks.workunit.client.1.smithi150.stdout:9/397: dread d1/d8/f65 [0,4194304] 0 2024-03-22T10:42:51.400 INFO:tasks.workunit.client.1.smithi150.stdout:9/398: fdatasync d1/f7b 0 2024-03-22T10:42:51.401 INFO:tasks.workunit.client.1.smithi150.stdout:5/243: dwrite d0/f10 [0,4194304] 0 2024-03-22T10:42:51.403 INFO:tasks.workunit.client.1.smithi150.stdout:6/287: creat db/dd/d25/f6d x:0 0 0 2024-03-22T10:42:51.409 INFO:tasks.workunit.client.1.smithi150.stdout:6/288: write db/dd/d25/f34 [6676638,116834] 0 2024-03-22T10:42:51.410 INFO:tasks.workunit.client.1.smithi150.stdout:4/324: mkdir d0/d1b/d4d/d6c/d61/d71 0 2024-03-22T10:42:51.415 INFO:tasks.workunit.client.1.smithi150.stdout:6/289: write db/dd/d15/f20 [1626616,119415] 0 2024-03-22T10:42:51.417 INFO:tasks.workunit.client.1.smithi150.stdout:5/244: rmdir d0/d8/dc/de 39 2024-03-22T10:42:51.423 INFO:tasks.workunit.client.1.smithi150.stdout:7/345: dread d2/d5/d7/d8/d57/d4a/f39 [4194304,4194304] 0 2024-03-22T10:42:51.424 INFO:tasks.workunit.client.1.smithi150.stdout:2/386: symlink d12/d19/d2d/d66/l7e 0 2024-03-22T10:42:51.426 INFO:tasks.workunit.client.1.smithi150.stdout:0/369: dwrite d0/d12/d4f/d42/d10/d53/f57 [0,4194304] 0 2024-03-22T10:42:51.428 INFO:tasks.workunit.client.0.smithi115.stdout:5/762: write d2/d15/d1b/d7c/d38/d56/fcd [10583,81663] 0 2024-03-22T10:42:51.428 INFO:tasks.workunit.client.0.smithi115.stdout:2/646: dread da/d25/d3d/d7e/d90/dbf/f8d [0,4194304] 0 2024-03-22T10:42:51.429 INFO:tasks.workunit.client.0.smithi115.stdout:4/694: truncate d1/d16/d35/d40/f48 754563 0 2024-03-22T10:42:51.433 INFO:tasks.workunit.client.0.smithi115.stdout:7/748: write d2/d7/d12/d10/d23/f97 [1942103,6007] 0 2024-03-22T10:42:51.433 INFO:tasks.workunit.client.0.smithi115.stdout:6/663: dwrite d4/d13/f3d [0,4194304] 0 2024-03-22T10:42:51.434 INFO:tasks.workunit.client.1.smithi150.stdout:3/340: link d5/da/c1a d5/da/c6b 0 2024-03-22T10:42:51.434 INFO:tasks.workunit.client.1.smithi150.stdout:1/293: dread d1/da/d2a/f35 [0,4194304] 0 2024-03-22T10:42:51.437 INFO:tasks.workunit.client.1.smithi150.stdout:3/341: chown d5/d22/d2f/d38/d3e 3371 1 2024-03-22T10:42:51.438 INFO:tasks.workunit.client.1.smithi150.stdout:8/346: link d0/d3c/f68 d0/d2/d19/f78 0 2024-03-22T10:42:51.438 INFO:tasks.workunit.client.0.smithi115.stdout:1/788: dread d0/d12/d17/f33 [0,4194304] 0 2024-03-22T10:42:51.442 INFO:tasks.workunit.client.0.smithi115.stdout:6/664: stat d4/d11/d12/d1c/f46 0 2024-03-22T10:42:51.443 INFO:tasks.workunit.client.1.smithi150.stdout:8/347: read - d0/d2/d19/d29/d2f/f64 zero size 2024-03-22T10:42:51.443 INFO:tasks.workunit.client.1.smithi150.stdout:7/346: creat d2/d6e/f73 x:0 0 0 2024-03-22T10:42:51.443 INFO:tasks.workunit.client.0.smithi115.stdout:4/695: mknod d1/d30/d4c/d86/d9f/d75/d99/ced 0 2024-03-22T10:42:51.443 INFO:tasks.workunit.client.0.smithi115.stdout:3/719: dwrite d3/d82/d52/fe3 [0,4194304] 0 2024-03-22T10:42:51.444 INFO:tasks.workunit.client.1.smithi150.stdout:5/245: dwrite d0/f10 [0,4194304] 0 2024-03-22T10:42:51.446 INFO:tasks.workunit.client.0.smithi115.stdout:5/763: fdatasync d2/d15/d1b/d7c/d38/d4a/f75 0 2024-03-22T10:42:51.446 INFO:tasks.workunit.client.0.smithi115.stdout:7/749: chown d2/d7/d12/d10/d19/d42/d76/d74/f77 1 1 2024-03-22T10:42:51.448 INFO:tasks.workunit.client.0.smithi115.stdout:3/720: chown d3/fd2 0 1 2024-03-22T10:42:51.448 INFO:tasks.workunit.client.1.smithi150.stdout:5/246: write d0/d8/dc/f12 [1348716,56019] 0 2024-03-22T10:42:51.450 INFO:tasks.workunit.client.1.smithi150.stdout:2/387: creat d12/d16/d67/d3c/f7f x:0 0 0 2024-03-22T10:42:51.450 INFO:tasks.workunit.client.0.smithi115.stdout:1/789: write d0/d8/d4d/d56/f6d [704601,64832] 0 2024-03-22T10:42:51.452 INFO:tasks.workunit.client.0.smithi115.stdout:3/721: chown d3/d36/d41/d85/db8 5211099 1 2024-03-22T10:42:51.454 INFO:tasks.workunit.client.0.smithi115.stdout:5/764: creat d2/d15/d1b/d7c/daf/ffc x:0 0 0 2024-03-22T10:42:51.455 INFO:tasks.workunit.client.1.smithi150.stdout:6/290: mknod db/c6e 0 2024-03-22T10:42:51.456 INFO:tasks.workunit.client.0.smithi115.stdout:3/722: creat d3/d19/dda/ff8 x:0 0 0 2024-03-22T10:42:51.459 INFO:tasks.workunit.client.0.smithi115.stdout:3/723: mknod d3/d82/cf9 0 2024-03-22T10:42:51.461 INFO:tasks.workunit.client.0.smithi115.stdout:7/750: readlink d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/dbb/lb0 0 2024-03-22T10:42:51.462 INFO:tasks.workunit.client.1.smithi150.stdout:1/294: write d1/da/d15/f3e [4791376,26869] 0 2024-03-22T10:42:51.464 INFO:tasks.workunit.client.1.smithi150.stdout:3/342: creat d5/d22/d37/f6c x:0 0 0 2024-03-22T10:42:51.464 INFO:tasks.workunit.client.0.smithi115.stdout:3/724: write d3/d82/d52/fe6 [232506,122851] 0 2024-03-22T10:42:51.464 INFO:tasks.workunit.client.0.smithi115.stdout:5/765: truncate d2/df/d13/d8b/d93/da6/dbc/ff1 364339 0 2024-03-22T10:42:51.465 INFO:tasks.workunit.client.0.smithi115.stdout:3/725: fsync d3/f23 0 2024-03-22T10:42:51.468 INFO:tasks.workunit.client.0.smithi115.stdout:3/726: chown d3/d82/d57/ff0 84 1 2024-03-22T10:42:51.469 INFO:tasks.workunit.client.0.smithi115.stdout:1/790: link d0/d8/d4d/d56/db6/ddc/l9b d0/d12/d17/d37/d9e/de9/dde/l114 0 2024-03-22T10:42:51.470 INFO:tasks.workunit.client.0.smithi115.stdout:3/727: chown d3/d82/d54/cbc 195 1 2024-03-22T10:42:51.471 INFO:tasks.workunit.client.0.smithi115.stdout:7/751: readlink d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/dbb/lb0 0 2024-03-22T10:42:51.474 INFO:tasks.workunit.client.0.smithi115.stdout:3/728: creat d3/d82/d54/ffa x:0 0 0 2024-03-22T10:42:51.476 INFO:tasks.workunit.client.1.smithi150.stdout:2/388: unlink l8 0 2024-03-22T10:42:51.477 INFO:tasks.workunit.client.0.smithi115.stdout:1/791: creat d0/d12/d17/d37/d9e/de9/dec/ded/f115 x:0 0 0 2024-03-22T10:42:51.477 INFO:tasks.workunit.client.1.smithi150.stdout:7/347: dread d2/d5/d7/d8/d57/d4a/f22 [0,4194304] 0 2024-03-22T10:42:51.478 INFO:tasks.workunit.client.1.smithi150.stdout:9/399: creat d1/d8/de/df/d36/d40/d41/f94 x:0 0 0 2024-03-22T10:42:51.478 INFO:tasks.workunit.client.0.smithi115.stdout:3/729: creat d3/d82/d54/ffb x:0 0 0 2024-03-22T10:42:51.479 INFO:tasks.workunit.client.0.smithi115.stdout:2/647: write da/d25/d3d/f2c [972369,31527] 0 2024-03-22T10:42:51.482 INFO:tasks.workunit.client.1.smithi150.stdout:5/247: unlink d0/d26/c48 0 2024-03-22T10:42:51.482 INFO:tasks.workunit.client.1.smithi150.stdout:4/325: dread d0/d1b/f23 [0,4194304] 0 2024-03-22T10:42:51.482 INFO:tasks.workunit.client.1.smithi150.stdout:2/389: write d12/d16/f2a [373340,55494] 0 2024-03-22T10:42:51.483 INFO:tasks.workunit.client.0.smithi115.stdout:2/648: truncate da/d25/f49 1440093 0 2024-03-22T10:42:51.483 INFO:tasks.workunit.client.0.smithi115.stdout:1/792: symlink d0/d12/d17/d37/d9e/de9/dec/ded/l116 0 2024-03-22T10:42:51.484 INFO:tasks.workunit.client.1.smithi150.stdout:8/348: dwrite d0/d2/d19/d29/f63 [0,4194304] 0 2024-03-22T10:42:51.487 INFO:tasks.workunit.client.0.smithi115.stdout:3/730: creat d3/d6d/db4/de1/ffc x:0 0 0 2024-03-22T10:42:51.487 INFO:tasks.workunit.client.0.smithi115.stdout:0/660: sync 2024-03-22T10:42:51.487 INFO:tasks.workunit.client.0.smithi115.stdout:8/895: sync 2024-03-22T10:42:51.489 INFO:tasks.workunit.client.1.smithi150.stdout:6/291: creat db/d52/f6f x:0 0 0 2024-03-22T10:42:51.490 INFO:tasks.workunit.client.1.smithi150.stdout:7/348: write d2/d5/f52 [2926673,124234] 0 2024-03-22T10:42:51.491 INFO:tasks.workunit.client.0.smithi115.stdout:3/731: chown d3/d36/d41/fba 222891234 1 2024-03-22T10:42:51.492 INFO:tasks.workunit.client.1.smithi150.stdout:7/349: chown d2 906987049 1 2024-03-22T10:42:51.492 INFO:tasks.workunit.client.1.smithi150.stdout:4/326: chown d0/d1b/f39 432472960 1 2024-03-22T10:42:51.493 INFO:tasks.workunit.client.0.smithi115.stdout:2/649: creat da/d1c/d67/d78/d86/db4/fc3 x:0 0 0 2024-03-22T10:42:51.494 INFO:tasks.workunit.client.1.smithi150.stdout:7/350: fsync d2/d5/d7/d8/d2c/d45/f72 0 2024-03-22T10:42:51.494 INFO:tasks.workunit.client.1.smithi150.stdout:4/327: write d0/d1b/d4d/f53 [357184,40297] 0 2024-03-22T10:42:51.494 INFO:tasks.workunit.client.1.smithi150.stdout:0/370: dwrite d0/d12/d4f/d42/f44 [0,4194304] 0 2024-03-22T10:42:51.495 INFO:tasks.workunit.client.0.smithi115.stdout:6/665: dread d4/d11/d12/f31 [0,4194304] 0 2024-03-22T10:42:51.495 INFO:tasks.workunit.client.0.smithi115.stdout:3/732: write d3/d19/dd8/ddb/fdc [349250,42427] 0 2024-03-22T10:42:51.498 INFO:tasks.workunit.client.1.smithi150.stdout:9/400: mkdir d1/d8/d95 0 2024-03-22T10:42:51.498 INFO:tasks.workunit.client.0.smithi115.stdout:0/661: write d3/d7/d1c/d6c/d9a/fd4 [249087,15403] 0 2024-03-22T10:42:51.501 INFO:tasks.workunit.client.1.smithi150.stdout:8/349: readlink d0/d2/d19/d29/d2f/df/d1b/l32 0 2024-03-22T10:42:51.502 INFO:tasks.workunit.client.1.smithi150.stdout:6/292: mkdir db/dd/d15/d70 0 2024-03-22T10:42:51.502 INFO:tasks.workunit.client.1.smithi150.stdout:9/401: readlink d1/d8/de/df/l51 0 2024-03-22T10:42:51.503 INFO:tasks.workunit.client.0.smithi115.stdout:7/752: dread d2/d7/d12/d10/d19/d42/d46/f52 [0,4194304] 0 2024-03-22T10:42:51.504 INFO:tasks.workunit.client.1.smithi150.stdout:6/293: write db/f44 [794361,118465] 0 2024-03-22T10:42:51.506 INFO:tasks.workunit.client.0.smithi115.stdout:2/650: rmdir da/d25/d3d/d7e 39 2024-03-22T10:42:51.511 INFO:tasks.workunit.client.1.smithi150.stdout:7/351: creat d2/d5/d7/d8/d46/f74 x:0 0 0 2024-03-22T10:42:51.511 INFO:tasks.workunit.client.1.smithi150.stdout:0/371: mkdir d0/d12/d4f/d42/d10/d38/d79 0 2024-03-22T10:42:51.511 INFO:tasks.workunit.client.1.smithi150.stdout:0/372: read d0/d12/d4f/f6b [52913,20621] 0 2024-03-22T10:42:51.512 INFO:tasks.workunit.client.0.smithi115.stdout:4/696: dwrite d1/d16/d35/d40/d54/d5e/f7e [0,4194304] 0 2024-03-22T10:42:51.512 INFO:tasks.workunit.client.1.smithi150.stdout:5/248: creat d0/d37/f49 x:0 0 0 2024-03-22T10:42:51.513 INFO:tasks.workunit.client.0.smithi115.stdout:7/753: write d2/d7/d12/d10/d19/fc1 [4344554,123315] 0 2024-03-22T10:42:51.514 INFO:tasks.workunit.client.0.smithi115.stdout:6/666: creat d4/d13/d33/dbe/fd3 x:0 0 0 2024-03-22T10:42:51.516 INFO:tasks.workunit.client.1.smithi150.stdout:8/350: symlink d0/d2/d19/d29/d2f/df/d1b/l79 0 2024-03-22T10:42:51.516 INFO:tasks.workunit.client.0.smithi115.stdout:2/651: mknod da/d1c/d67/d78/cc4 0 2024-03-22T10:42:51.518 INFO:tasks.workunit.client.0.smithi115.stdout:7/754: chown d2/d62/fa0 211095538 1 2024-03-22T10:42:51.522 INFO:tasks.workunit.client.0.smithi115.stdout:0/662: creat d3/d7/fd8 x:0 0 0 2024-03-22T10:42:51.522 INFO:tasks.workunit.client.1.smithi150.stdout:4/328: dread d0/f26 [0,4194304] 0 2024-03-22T10:42:51.523 INFO:tasks.workunit.client.0.smithi115.stdout:1/793: dwrite d0/d8/d4d/d56/dbd/fd0 [0,4194304] 0 2024-03-22T10:42:51.525 INFO:tasks.workunit.client.0.smithi115.stdout:2/652: creat da/d1c/d67/fc5 x:0 0 0 2024-03-22T10:42:51.525 INFO:tasks.workunit.client.1.smithi150.stdout:6/294: read db/dd/d15/d60/f32 [601171,78806] 0 2024-03-22T10:42:51.526 INFO:tasks.workunit.client.0.smithi115.stdout:4/697: creat d1/d30/d78/fee x:0 0 0 2024-03-22T10:42:51.527 INFO:tasks.workunit.client.1.smithi150.stdout:3/343: dread d5/da/f45 [0,4194304] 0 2024-03-22T10:42:51.528 INFO:tasks.workunit.client.0.smithi115.stdout:6/667: unlink d4/d13/c35 0 2024-03-22T10:42:51.530 INFO:tasks.workunit.client.1.smithi150.stdout:7/352: write d2/d5/d7/f50 [209955,65487] 0 2024-03-22T10:42:51.531 INFO:tasks.workunit.client.1.smithi150.stdout:4/329: fdatasync d0/d1b/f39 0 2024-03-22T10:42:51.534 INFO:tasks.workunit.client.0.smithi115.stdout:3/733: dwrite d3/d36/d41/f48 [0,4194304] 0 2024-03-22T10:42:51.534 INFO:tasks.workunit.client.0.smithi115.stdout:5/766: dread d2/d15/d1b/d7c/d38/d43/d49/fa9 [0,4194304] 0 2024-03-22T10:42:51.535 INFO:tasks.workunit.client.0.smithi115.stdout:4/698: chown d1/d30/d4c/d86/d9f/fe4 46734 1 2024-03-22T10:42:51.535 INFO:tasks.workunit.client.0.smithi115.stdout:7/755: fsync d2/d7/d12/d10/d23/d3d/dd2/f88 0 2024-03-22T10:42:51.536 INFO:tasks.workunit.client.1.smithi150.stdout:6/295: mknod db/dd/d15/d57/c71 0 2024-03-22T10:42:51.540 INFO:tasks.workunit.client.0.smithi115.stdout:6/668: mknod d4/d36/d48/d72/cd4 0 2024-03-22T10:42:51.546 INFO:tasks.workunit.client.1.smithi150.stdout:3/344: rename d5/d22/d2f/l4b to d5/d22/d2f/d38/d3e/l6d 0 2024-03-22T10:42:51.548 INFO:tasks.workunit.client.0.smithi115.stdout:4/699: unlink d1/f7 0 2024-03-22T10:42:51.548 INFO:tasks.workunit.client.1.smithi150.stdout:4/330: fdatasync d0/f1a 0 2024-03-22T10:42:51.550 INFO:tasks.workunit.client.0.smithi115.stdout:1/794: mkdir d0/d12/d2b/dfc/d117 0 2024-03-22T10:42:51.551 INFO:tasks.workunit.client.0.smithi115.stdout:3/734: mkdir d3/d16/d29/dc1/dfd 0 2024-03-22T10:42:51.552 INFO:tasks.workunit.client.1.smithi150.stdout:9/402: dread d1/d8/de/df/d20/d32/f2a [0,4194304] 0 2024-03-22T10:42:51.552 INFO:tasks.workunit.client.1.smithi150.stdout:7/353: mknod d2/d5/d7/d8/d46/d49/c75 0 2024-03-22T10:42:51.552 INFO:tasks.workunit.client.0.smithi115.stdout:0/663: dread d3/d7/d1b/d35/f79 [0,4194304] 0 2024-03-22T10:42:51.554 INFO:tasks.workunit.client.1.smithi150.stdout:7/354: fsync d2/d5/d7/d8/f5e 0 2024-03-22T10:42:51.554 INFO:tasks.workunit.client.1.smithi150.stdout:5/249: symlink d0/d8/dc/de/l4a 0 2024-03-22T10:42:51.555 INFO:tasks.workunit.client.0.smithi115.stdout:2/653: rmdir da/d1c/d67/d78/d86/d9d/dc2 0 2024-03-22T10:42:51.557 INFO:tasks.workunit.client.1.smithi150.stdout:7/355: fdatasync d2/d5/d7/d9/f65 0 2024-03-22T10:42:51.557 INFO:tasks.workunit.client.1.smithi150.stdout:6/296: mknod db/dd/d15/d30/c72 0 2024-03-22T10:42:51.559 INFO:tasks.workunit.client.0.smithi115.stdout:2/654: read - da/d25/d3d/d57/f93 zero size 2024-03-22T10:42:51.561 INFO:tasks.workunit.client.0.smithi115.stdout:2/655: fdatasync da/f14 0 2024-03-22T10:42:51.568 INFO:tasks.workunit.client.0.smithi115.stdout:9/732: sync 2024-03-22T10:42:51.571 INFO:tasks.workunit.client.1.smithi150.stdout:3/345: unlink d5/f44 0 2024-03-22T10:42:51.574 INFO:tasks.workunit.client.0.smithi115.stdout:5/767: link d2/d15/ff3 d2/d15/d1b/ffd 0 2024-03-22T10:42:51.576 INFO:tasks.workunit.client.1.smithi150.stdout:7/356: rename d2/d5/f52 to d2/d5/d7/d9/db/f76 0 2024-03-22T10:42:51.578 INFO:tasks.workunit.client.0.smithi115.stdout:7/756: getdents d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae 0 2024-03-22T10:42:51.579 INFO:tasks.workunit.client.0.smithi115.stdout:3/735: truncate d3/d19/d2b/d49/d91/da5/fa7 973815 0 2024-03-22T10:42:51.579 INFO:tasks.workunit.client.0.smithi115.stdout:5/768: write d2/d15/d1b/d7c/d84/fe9 [515814,61689] 0 2024-03-22T10:42:51.581 INFO:tasks.workunit.client.0.smithi115.stdout:8/896: dwrite d6/ff4 [0,4194304] 0 2024-03-22T10:42:51.585 INFO:tasks.workunit.client.0.smithi115.stdout:4/700: dread d1/d16/d35/d40/f48 [0,4194304] 0 2024-03-22T10:42:51.585 INFO:tasks.workunit.client.1.smithi150.stdout:5/250: symlink d0/d8/dc/l4b 0 2024-03-22T10:42:51.587 INFO:tasks.workunit.client.1.smithi150.stdout:6/297: getdents db/dd/d15/d70 0 2024-03-22T10:42:51.588 INFO:tasks.workunit.client.1.smithi150.stdout:1/295: truncate d1/da/d15/d18/f38 3419263 0 2024-03-22T10:42:51.589 INFO:tasks.workunit.client.1.smithi150.stdout:2/390: truncate f11 729390 0 2024-03-22T10:42:51.591 INFO:tasks.workunit.client.1.smithi150.stdout:5/251: write d0/f2b [842738,93881] 0 2024-03-22T10:42:51.593 INFO:tasks.workunit.client.1.smithi150.stdout:4/331: write d0/d1b/f39 [1602107,9173] 0 2024-03-22T10:42:51.593 INFO:tasks.workunit.client.1.smithi150.stdout:9/403: creat d1/d8/de/df/d36/f96 x:0 0 0 2024-03-22T10:42:51.594 INFO:tasks.workunit.client.1.smithi150.stdout:5/252: stat d0/l32 0 2024-03-22T10:42:51.596 INFO:tasks.workunit.client.0.smithi115.stdout:7/757: truncate d2/d7/d12/d10/d19/dbd/fdb 587151 0 2024-03-22T10:42:51.596 INFO:tasks.workunit.client.0.smithi115.stdout:4/701: write d1/d16/d28/fe5 [455083,73434] 0 2024-03-22T10:42:51.596 INFO:tasks.workunit.client.0.smithi115.stdout:8/897: mknod d6/dd/d24/d4a/d7b/d8f/dd6/ddb/c12e 0 2024-03-22T10:42:51.599 INFO:tasks.workunit.client.1.smithi150.stdout:2/391: write d12/f55 [459765,74678] 0 2024-03-22T10:42:51.599 INFO:tasks.workunit.client.1.smithi150.stdout:8/351: truncate d0/d2/d19/d29/d2f/df/d1b/f1c 4003221 0 2024-03-22T10:42:51.600 INFO:tasks.workunit.client.0.smithi115.stdout:7/758: creat d2/d7/dc6/f107 x:0 0 0 2024-03-22T10:42:51.600 INFO:tasks.workunit.client.1.smithi150.stdout:9/404: fsync d1/d8/de/df/f18 0 2024-03-22T10:42:51.600 INFO:tasks.workunit.client.1.smithi150.stdout:4/332: write d0/f1a [1813659,43160] 0 2024-03-22T10:42:51.604 INFO:tasks.workunit.client.1.smithi150.stdout:5/253: chown d0/f3 6488 1 2024-03-22T10:42:51.606 INFO:tasks.workunit.client.1.smithi150.stdout:4/333: dread - d0/d32/f51 zero size 2024-03-22T10:42:51.610 INFO:tasks.workunit.client.0.smithi115.stdout:5/769: dwrite d2/d15/d1b/d7c/d38/d56/fcd [0,4194304] 0 2024-03-22T10:42:51.611 INFO:tasks.workunit.client.1.smithi150.stdout:2/392: write d12/d16/d67/f4e [312914,61607] 0 2024-03-22T10:42:51.611 INFO:tasks.workunit.client.1.smithi150.stdout:7/357: dread d2/d5/d7/d8/f5e [4194304,4194304] 0 2024-03-22T10:42:51.612 INFO:tasks.workunit.client.1.smithi150.stdout:9/405: chown d1/d8/de/df/d20/l13 175 1 2024-03-22T10:42:51.613 INFO:tasks.workunit.client.1.smithi150.stdout:0/373: dread d0/d12/d4f/d11/f2f [0,4194304] 0 2024-03-22T10:42:51.615 INFO:tasks.workunit.client.0.smithi115.stdout:6/669: truncate d4/d11/d12/d1c/fa5 4012428 0 2024-03-22T10:42:51.615 INFO:tasks.workunit.client.1.smithi150.stdout:4/334: mkdir d0/d47/d72 0 2024-03-22T10:42:51.617 INFO:tasks.workunit.client.0.smithi115.stdout:6/670: chown d4/d36 103 1 2024-03-22T10:42:51.618 INFO:tasks.workunit.client.1.smithi150.stdout:2/393: readlink d12/d19/d1a/l49 0 2024-03-22T10:42:51.619 INFO:tasks.workunit.client.1.smithi150.stdout:5/254: chown d0/d8/dc/de/c35 42 1 2024-03-22T10:42:51.621 INFO:tasks.workunit.client.1.smithi150.stdout:6/298: dwrite db/d52/f6f [0,4194304] 0 2024-03-22T10:42:51.624 INFO:tasks.workunit.client.1.smithi150.stdout:6/299: readlink db/l16 0 2024-03-22T10:42:51.625 INFO:tasks.workunit.client.1.smithi150.stdout:0/374: write d0/d12/d4f/d11/d26/f2e [7118528,84305] 0 2024-03-22T10:42:51.627 INFO:tasks.workunit.client.0.smithi115.stdout:1/795: dwrite d0/d12/d2b/fca [0,4194304] 0 2024-03-22T10:42:51.628 INFO:tasks.workunit.client.1.smithi150.stdout:5/255: fdatasync d0/d8/dc/de/d14/d2e/f3e 0 2024-03-22T10:42:51.628 INFO:tasks.workunit.client.1.smithi150.stdout:2/394: write d12/d16/d67/d3c/f6b [139840,33338] 0 2024-03-22T10:42:51.629 INFO:tasks.workunit.client.1.smithi150.stdout:1/296: dwrite d1/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:42:51.631 INFO:tasks.workunit.client.1.smithi150.stdout:9/406: rename d1/d8 to d1/d8/de/df/d36/d40/d6c/d97 22 2024-03-22T10:42:51.635 INFO:tasks.workunit.client.0.smithi115.stdout:6/671: unlink d4/d13/d33/d58/cbc 0 2024-03-22T10:42:51.640 INFO:tasks.workunit.client.1.smithi150.stdout:0/375: rmdir d0 39 2024-03-22T10:42:51.640 INFO:tasks.workunit.client.0.smithi115.stdout:1/796: mkdir d0/d12/d17/d37/d9e/da0/d118 0 2024-03-22T10:42:51.641 INFO:tasks.workunit.client.0.smithi115.stdout:0/664: dread d3/d7/d1b/f92 [4194304,4194304] 0 2024-03-22T10:42:51.643 INFO:tasks.workunit.client.0.smithi115.stdout:7/759: write d2/d7/d12/d10/d23/d3d/dd2/da3/fac [759619,79701] 0 2024-03-22T10:42:51.644 INFO:tasks.workunit.client.1.smithi150.stdout:8/352: dwrite d0/d2/d19/f78 [0,4194304] 0 2024-03-22T10:42:51.644 INFO:tasks.workunit.client.0.smithi115.stdout:6/672: chown d4/d11/d12/d1c/d8a 59 1 2024-03-22T10:42:51.646 INFO:tasks.workunit.client.1.smithi150.stdout:2/395: rmdir d12/d19/d2d 39 2024-03-22T10:42:51.648 INFO:tasks.workunit.client.0.smithi115.stdout:5/770: dread d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f9b [0,4194304] 0 2024-03-22T10:42:51.649 INFO:tasks.workunit.client.0.smithi115.stdout:1/797: write d0/d12/d2b/fca [3263699,1333] 0 2024-03-22T10:42:51.649 INFO:tasks.workunit.client.0.smithi115.stdout:6/673: fdatasync d4/d11/f2a 0 2024-03-22T10:42:51.650 INFO:tasks.workunit.client.0.smithi115.stdout:3/736: dread d3/d19/d2b/d49/d91/fa3 [0,4194304] 0 2024-03-22T10:42:51.650 INFO:tasks.workunit.client.1.smithi150.stdout:5/256: creat d0/f4c x:0 0 0 2024-03-22T10:42:51.651 INFO:tasks.workunit.client.1.smithi150.stdout:9/407: mkdir d1/d49/d98 0 2024-03-22T10:42:51.652 INFO:tasks.workunit.client.0.smithi115.stdout:2/656: dread da/d25/d3d/f3c [0,4194304] 0 2024-03-22T10:42:51.655 INFO:tasks.workunit.client.0.smithi115.stdout:4/702: dwrite d1/d16/d35/d40/d5c/d14/f96 [0,4194304] 0 2024-03-22T10:42:51.656 INFO:tasks.workunit.client.1.smithi150.stdout:9/408: dread - d1/d19/f79 zero size 2024-03-22T10:42:51.661 INFO:tasks.workunit.client.1.smithi150.stdout:8/353: mkdir d0/d3c/d71/d7a 0 2024-03-22T10:42:51.661 INFO:tasks.workunit.client.1.smithi150.stdout:4/335: dwrite d0/d2f/f44 [0,4194304] 0 2024-03-22T10:42:51.662 INFO:tasks.workunit.client.0.smithi115.stdout:6/674: chown d4/d13/d33/d58/l7b 55 1 2024-03-22T10:42:51.664 INFO:tasks.workunit.client.0.smithi115.stdout:8/898: dwrite d6/f11f [0,4194304] 0 2024-03-22T10:42:51.665 INFO:tasks.workunit.client.1.smithi150.stdout:4/336: read d0/de/f18 [4177351,48405] 0 2024-03-22T10:42:51.665 INFO:tasks.workunit.client.1.smithi150.stdout:5/257: write d0/d8/dc/f40 [156470,30468] 0 2024-03-22T10:42:51.666 INFO:tasks.workunit.client.0.smithi115.stdout:5/771: mkdir d2/d15/d1b/d7c/daf/dfe 0 2024-03-22T10:42:51.667 INFO:tasks.workunit.client.1.smithi150.stdout:9/409: rmdir d1/d8/de/df/d36/d40/d41 39 2024-03-22T10:42:51.671 INFO:tasks.workunit.client.0.smithi115.stdout:4/703: creat d1/d16/d35/d40/d5c/d55/fef x:0 0 0 2024-03-22T10:42:51.672 INFO:tasks.workunit.client.1.smithi150.stdout:0/376: truncate d0/d12/d4f/f6b 1210558 0 2024-03-22T10:42:51.672 INFO:tasks.workunit.client.1.smithi150.stdout:2/396: mknod d12/c80 0 2024-03-22T10:42:51.675 INFO:tasks.workunit.client.0.smithi115.stdout:4/704: readlink d1/d69/l94 0 2024-03-22T10:42:51.676 INFO:tasks.workunit.client.1.smithi150.stdout:9/410: chown d1/d8/de/df/l10 4453 1 2024-03-22T10:42:51.676 INFO:tasks.workunit.client.1.smithi150.stdout:1/297: getdents d1/da/d2a/d55 0 2024-03-22T10:42:51.681 INFO:tasks.workunit.client.0.smithi115.stdout:3/737: truncate d3/d36/d41/f9a 636007 0 2024-03-22T10:42:51.682 INFO:tasks.workunit.client.0.smithi115.stdout:4/705: write d1/d66/d82/fa6 [827370,20964] 0 2024-03-22T10:42:51.682 INFO:tasks.workunit.client.1.smithi150.stdout:5/258: mkdir d0/d8/dc/de/d14/d2e/d46/d4d 0 2024-03-22T10:42:51.682 INFO:tasks.workunit.client.1.smithi150.stdout:1/298: creat d1/d19/d50/f5c x:0 0 0 2024-03-22T10:42:51.683 INFO:tasks.workunit.client.1.smithi150.stdout:2/397: mkdir d12/d21/d81 0 2024-03-22T10:42:51.683 INFO:tasks.workunit.client.1.smithi150.stdout:8/354: dwrite d0/f5a [0,4194304] 0 2024-03-22T10:42:51.684 INFO:tasks.workunit.client.1.smithi150.stdout:9/411: truncate d1/d8/de/df/d36/f39 2027458 0 2024-03-22T10:42:51.684 INFO:tasks.workunit.client.0.smithi115.stdout:2/657: rename da/d1c/d67/d78/db2 to da/d25/d3d/d57/d7d/dc6 0 2024-03-22T10:42:51.685 INFO:tasks.workunit.client.0.smithi115.stdout:3/738: truncate d3/d36/f44 5166790 0 2024-03-22T10:42:51.687 INFO:tasks.workunit.client.0.smithi115.stdout:8/899: mkdir d6/dd/d12/d12f 0 2024-03-22T10:42:51.688 INFO:tasks.workunit.client.1.smithi150.stdout:5/259: unlink d0/d37/f49 0 2024-03-22T10:42:51.690 INFO:tasks.workunit.client.0.smithi115.stdout:2/658: mknod da/d25/d83/cc7 0 2024-03-22T10:42:51.691 INFO:tasks.workunit.client.0.smithi115.stdout:4/706: mkdir d1/d16/df0 0 2024-03-22T10:42:51.693 INFO:tasks.workunit.client.1.smithi150.stdout:5/260: write d0/d8/dc/f12 [194914,33449] 0 2024-03-22T10:42:51.694 INFO:tasks.workunit.client.1.smithi150.stdout:1/299: symlink d1/da/d2a/l5d 0 2024-03-22T10:42:51.695 INFO:tasks.workunit.client.0.smithi115.stdout:2/659: unlink da/d1c/f73 0 2024-03-22T10:42:51.697 INFO:tasks.workunit.client.1.smithi150.stdout:1/300: fsync d1/f3a 0 2024-03-22T10:42:51.697 INFO:tasks.workunit.client.0.smithi115.stdout:8/900: symlink d6/dd/d12/dc6/dee/l130 0 2024-03-22T10:42:51.698 INFO:tasks.workunit.client.1.smithi150.stdout:6/300: dread db/dd/d15/d60/f53 [0,4194304] 0 2024-03-22T10:42:51.700 INFO:tasks.workunit.client.1.smithi150.stdout:1/301: dread - d1/d19/d1a/f46 zero size 2024-03-22T10:42:51.702 INFO:tasks.workunit.client.1.smithi150.stdout:6/301: write db/f4f [112395,57251] 0 2024-03-22T10:42:51.706 INFO:tasks.workunit.client.0.smithi115.stdout:3/739: dwrite d3/d16/d29/dc1/ff3 [0,4194304] 0 2024-03-22T10:42:51.709 INFO:tasks.workunit.client.1.smithi150.stdout:0/377: getdents d0/d12/d4f/d42/d10/d38 0 2024-03-22T10:42:51.710 INFO:tasks.workunit.client.0.smithi115.stdout:3/740: chown d3/d36/d41/fa2 123 1 2024-03-22T10:42:51.712 INFO:tasks.workunit.client.0.smithi115.stdout:8/901: rename d6/dd/d12/d31/cfa to d6/dd/d12/d31/c131 0 2024-03-22T10:42:51.718 INFO:tasks.workunit.client.1.smithi150.stdout:1/302: dread d1/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:42:51.723 INFO:tasks.workunit.client.1.smithi150.stdout:8/355: creat d0/d2/d19/d29/d2f/f7b x:0 0 0 2024-03-22T10:42:51.725 INFO:tasks.workunit.client.1.smithi150.stdout:0/378: dread d0/f6d [0,4194304] 0 2024-03-22T10:42:51.726 INFO:tasks.workunit.client.0.smithi115.stdout:6/675: dwrite d4/d13/d33/d58/fb7 [0,4194304] 0 2024-03-22T10:42:51.729 INFO:tasks.workunit.client.0.smithi115.stdout:8/902: mknod d6/d49/d68/dcf/c132 0 2024-03-22T10:42:51.730 INFO:tasks.workunit.client.1.smithi150.stdout:4/337: truncate d0/d1b/f2c 912890 0 2024-03-22T10:42:51.731 INFO:tasks.workunit.client.1.smithi150.stdout:5/261: mknod d0/d8/dc/d2c/d31/c4e 0 2024-03-22T10:42:51.732 INFO:tasks.workunit.client.0.smithi115.stdout:0/665: dread d3/d4/d6d/f9c [0,4194304] 0 2024-03-22T10:42:51.733 INFO:tasks.workunit.client.0.smithi115.stdout:3/741: creat d3/d19/d2b/d49/ffe x:0 0 0 2024-03-22T10:42:51.734 INFO:tasks.workunit.client.1.smithi150.stdout:2/398: rename d12/d19/f2e to d12/d19/d2d/f82 0 2024-03-22T10:42:51.735 INFO:tasks.workunit.client.1.smithi150.stdout:9/412: truncate d1/d8/de/df/f18 4369251 0 2024-03-22T10:42:51.736 INFO:tasks.workunit.client.0.smithi115.stdout:6/676: creat d4/d11/d91/fd5 x:0 0 0 2024-03-22T10:42:51.737 INFO:tasks.workunit.client.0.smithi115.stdout:0/666: chown d3/ccd 2 1 2024-03-22T10:42:51.737 INFO:tasks.workunit.client.1.smithi150.stdout:9/413: fsync d1/f71 0 2024-03-22T10:42:51.737 INFO:tasks.workunit.client.0.smithi115.stdout:4/707: truncate d1/d16/d35/d40/d54/d5e/f7e 2447808 0 2024-03-22T10:42:51.738 INFO:tasks.workunit.client.1.smithi150.stdout:4/338: mknod d0/d2f/c73 0 2024-03-22T10:42:51.740 INFO:tasks.workunit.client.1.smithi150.stdout:8/356: mknod d0/c7c 0 2024-03-22T10:42:51.742 INFO:tasks.workunit.client.0.smithi115.stdout:0/667: fsync d3/da/d4c/f67 0 2024-03-22T10:42:51.745 INFO:tasks.workunit.client.1.smithi150.stdout:6/302: rename db/c17 to db/dd/d15/d28/c73 0 2024-03-22T10:42:51.746 INFO:tasks.workunit.client.1.smithi150.stdout:8/357: truncate d0/d2/d19/d29/d31/f6d 188354 0 2024-03-22T10:42:51.749 INFO:tasks.workunit.client.0.smithi115.stdout:0/668: write d3/d4/d6d/f98 [2831725,67548] 0 2024-03-22T10:42:51.749 INFO:tasks.workunit.client.0.smithi115.stdout:6/677: chown d4/d11/c22 77 1 2024-03-22T10:42:51.749 INFO:tasks.workunit.client.0.smithi115.stdout:8/903: link d6/dd/d24/d4a/f4d d6/d49/d88/f133 0 2024-03-22T10:42:51.750 INFO:tasks.workunit.client.0.smithi115.stdout:8/904: fsync d6/ff4 0 2024-03-22T10:42:51.751 INFO:tasks.workunit.client.1.smithi150.stdout:9/414: write d1/d8/de/df/d36/d40/d41/f94 [115572,118398] 0 2024-03-22T10:42:51.753 INFO:tasks.workunit.client.0.smithi115.stdout:4/708: write d1/d30/d78/fb4 [1126718,43772] 0 2024-03-22T10:42:51.754 INFO:tasks.workunit.client.0.smithi115.stdout:5/772: dread d2/df/d3b/fb0 [4194304,4194304] 0 2024-03-22T10:42:51.755 INFO:tasks.workunit.client.1.smithi150.stdout:9/415: truncate d1/d8/de/df/d36/f39 2620816 0 2024-03-22T10:42:51.756 INFO:tasks.workunit.client.0.smithi115.stdout:0/669: mkdir d3/d7/d1b/d1f/d78/dd9 0 2024-03-22T10:42:51.756 INFO:tasks.workunit.client.0.smithi115.stdout:8/905: fdatasync d6/dd/d24/d3a/f3c 0 2024-03-22T10:42:51.757 INFO:tasks.workunit.client.1.smithi150.stdout:8/358: rmdir d0/d5d 39 2024-03-22T10:42:51.757 INFO:tasks.workunit.client.0.smithi115.stdout:1/798: dread d0/d12/d17/d37/db5/fd9 [0,4194304] 0 2024-03-22T10:42:51.758 INFO:tasks.workunit.client.0.smithi115.stdout:2/660: dwrite da/d25/d30/f59 [0,4194304] 0 2024-03-22T10:42:51.758 INFO:tasks.workunit.client.0.smithi115.stdout:9/733: sync 2024-03-22T10:42:51.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:51 smithi115 ceph-mon[29111]: pgmap v14: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 73 MiB/s rd, 271 MiB/s wr, 510 op/s 2024-03-22T10:42:51.762 INFO:tasks.workunit.client.1.smithi150.stdout:8/359: write d0/d2/d19/d29/d31/f4b [630867,41344] 0 2024-03-22T10:42:51.763 INFO:tasks.workunit.client.0.smithi115.stdout:7/760: dread d2/d7/d12/d10/f40 [0,4194304] 0 2024-03-22T10:42:51.763 INFO:tasks.workunit.client.0.smithi115.stdout:6/678: creat d4/d5e/dd0/fd6 x:0 0 0 2024-03-22T10:42:51.764 INFO:tasks.workunit.client.0.smithi115.stdout:5/773: write d2/d15/d1b/d7c/f63 [3791034,60762] 0 2024-03-22T10:42:51.765 INFO:tasks.workunit.client.0.smithi115.stdout:2/661: fdatasync da/d25/d3d/f9f 0 2024-03-22T10:42:51.766 INFO:tasks.workunit.client.1.smithi150.stdout:9/416: rename d1/d8/de/df/d36/d40/d7a to d1/d8/de/df/d20/d32/d75/d99 0 2024-03-22T10:42:51.768 INFO:tasks.workunit.client.0.smithi115.stdout:0/670: write d3/da/d4c/d74/fb6 [927128,63798] 0 2024-03-22T10:42:51.768 INFO:tasks.workunit.client.0.smithi115.stdout:8/906: creat d6/d49/d88/f134 x:0 0 0 2024-03-22T10:42:51.769 INFO:tasks.workunit.client.1.smithi150.stdout:1/303: dread d1/da/f42 [0,4194304] 0 2024-03-22T10:42:51.771 INFO:tasks.workunit.client.1.smithi150.stdout:2/399: link d12/d16/d2c/l3f d12/d21/d81/l83 0 2024-03-22T10:42:51.773 INFO:tasks.workunit.client.1.smithi150.stdout:1/304: dread - d1/d19/d1a/f46 zero size 2024-03-22T10:42:51.775 INFO:tasks.workunit.client.0.smithi115.stdout:5/774: chown d2/d15/d1b/d7c/d84/f97 7 1 2024-03-22T10:42:51.776 INFO:tasks.workunit.client.1.smithi150.stdout:4/339: link d0/d2f/f4f d0/d1b/f74 0 2024-03-22T10:42:51.780 INFO:tasks.workunit.client.1.smithi150.stdout:0/379: truncate d0/d12/d4f/d11/d66/d35/f62 3293668 0 2024-03-22T10:42:51.780 INFO:tasks.workunit.client.0.smithi115.stdout:9/734: dread d6/d7/d30/d37/fa0 [0,4194304] 0 2024-03-22T10:42:51.781 INFO:tasks.workunit.client.0.smithi115.stdout:7/761: dread d2/d7/d12/d10/d23/d3d/ff0 [0,4194304] 0 2024-03-22T10:42:51.781 INFO:tasks.workunit.client.0.smithi115.stdout:8/907: stat d6/dd/f2e 0 2024-03-22T10:42:51.782 INFO:tasks.workunit.client.1.smithi150.stdout:2/400: mknod d12/d21/d81/c84 0 2024-03-22T10:42:51.786 INFO:tasks.workunit.client.0.smithi115.stdout:6/679: dread d4/d13/f3d [0,4194304] 0 2024-03-22T10:42:51.786 INFO:tasks.workunit.client.1.smithi150.stdout:1/305: mknod d1/da/d15/c5e 0 2024-03-22T10:42:51.787 INFO:tasks.workunit.client.0.smithi115.stdout:3/742: dwrite d3/d19/dda/fdd [0,4194304] 0 2024-03-22T10:42:51.787 INFO:tasks.workunit.client.0.smithi115.stdout:1/799: rmdir d0/d12/d17/d37/d9e/da0/d109 0 2024-03-22T10:42:51.787 INFO:tasks.workunit.client.0.smithi115.stdout:8/908: chown d6/dd/d24/d4a/d41/fe1 26 1 2024-03-22T10:42:51.788 INFO:tasks.workunit.client.1.smithi150.stdout:0/380: symlink d0/d1/l7a 0 2024-03-22T10:42:51.789 INFO:tasks.workunit.client.1.smithi150.stdout:1/306: stat d1/da/f16 0 2024-03-22T10:42:51.790 INFO:tasks.workunit.client.0.smithi115.stdout:3/743: fsync d3/d82/d52/fe6 0 2024-03-22T10:42:51.792 INFO:tasks.workunit.client.0.smithi115.stdout:4/709: rename d1/d30/d4c/d86/d9f/d75/d99/da8/fa9 to d1/d16/d35/d40/d5c/d14/ff1 0 2024-03-22T10:42:51.793 INFO:tasks.workunit.client.1.smithi150.stdout:4/340: link d0/d1b/d4d/d6c/c6e d0/de/c75 0 2024-03-22T10:42:51.794 INFO:tasks.workunit.client.0.smithi115.stdout:3/744: chown d3/d19/dda/cec 47713 1 2024-03-22T10:42:51.794 INFO:tasks.workunit.client.1.smithi150.stdout:8/360: dwrite d0/d2/d19/d29/d2f/f64 [0,4194304] 0 2024-03-22T10:42:51.795 INFO:tasks.workunit.client.0.smithi115.stdout:8/909: chown d6/dd/d24/d4a/d7b/d8f/dd6/ddb/lf9 367687 1 2024-03-22T10:42:51.797 INFO:tasks.workunit.client.1.smithi150.stdout:2/401: truncate d12/d16/d67/d3c/f77 381207 0 2024-03-22T10:42:51.798 INFO:tasks.workunit.client.0.smithi115.stdout:1/800: mkdir d0/d12/d17/d37/d119 0 2024-03-22T10:42:51.799 INFO:tasks.workunit.client.1.smithi150.stdout:9/417: dread d1/d8/de/df/d24/f89 [0,4194304] 0 2024-03-22T10:42:51.800 INFO:tasks.workunit.client.0.smithi115.stdout:5/775: creat d2/df/d3b/dfb/fff x:0 0 0 2024-03-22T10:42:51.801 INFO:tasks.workunit.client.1.smithi150.stdout:2/402: write d12/d16/d67/d3c/f7f [689595,81599] 0 2024-03-22T10:42:51.802 INFO:tasks.workunit.client.0.smithi115.stdout:5/776: chown d2/f71 0 1 2024-03-22T10:42:51.803 INFO:tasks.workunit.client.0.smithi115.stdout:6/680: creat d4/d36/d48/d72/daa/db9/fd7 x:0 0 0 2024-03-22T10:42:51.803 INFO:tasks.workunit.client.1.smithi150.stdout:4/341: creat d0/d2f/f76 x:0 0 0 2024-03-22T10:42:51.806 INFO:tasks.workunit.client.1.smithi150.stdout:2/403: chown d12/d16/d2c/c60 233928083 1 2024-03-22T10:42:51.806 INFO:tasks.workunit.client.0.smithi115.stdout:1/801: mkdir d0/d8/db/d25/d11a 0 2024-03-22T10:42:51.807 INFO:tasks.workunit.client.1.smithi150.stdout:6/303: dread db/dd/d15/f46 [0,4194304] 0 2024-03-22T10:42:51.809 INFO:tasks.workunit.client.0.smithi115.stdout:7/762: link d2/d7/d12/d10/d19/d42/d46/fb5 d2/d62/f108 0 2024-03-22T10:42:51.810 INFO:tasks.workunit.client.1.smithi150.stdout:4/342: mknod d0/d32/c77 0 2024-03-22T10:42:51.811 INFO:tasks.workunit.client.1.smithi150.stdout:6/304: stat db/dd/d15/c47 0 2024-03-22T10:42:51.812 INFO:tasks.workunit.client.0.smithi115.stdout:6/681: truncate d4/d11/d12/d1c/fa7 354531 0 2024-03-22T10:42:51.812 INFO:tasks.workunit.client.0.smithi115.stdout:3/745: rename d3/d82/l6e to d3/d36/d41/lff 0 2024-03-22T10:42:51.813 INFO:tasks.workunit.client.1.smithi150.stdout:4/343: dread - d0/d2f/f76 zero size 2024-03-22T10:42:51.814 INFO:tasks.workunit.client.0.smithi115.stdout:3/746: readlink d3/d16/d29/d35/lea 0 2024-03-22T10:42:51.815 INFO:tasks.workunit.client.0.smithi115.stdout:8/910: creat d6/dd/d24/d3a/f135 x:0 0 0 2024-03-22T10:42:51.819 INFO:tasks.workunit.client.0.smithi115.stdout:7/763: symlink d2/d7/d12/d10/d23/d3d/da4/l109 0 2024-03-22T10:42:51.821 INFO:tasks.workunit.client.0.smithi115.stdout:2/662: write da/f16 [802303,4768] 0 2024-03-22T10:42:51.825 INFO:tasks.workunit.client.0.smithi115.stdout:1/802: rename d0/d12/d2b/dfc to d0/d8/d4d/d56/db6/d11b 0 2024-03-22T10:42:51.826 INFO:tasks.workunit.client.1.smithi150.stdout:0/381: link d0/d12/d4f/d42/d10/d38/d52/f54 d0/f7b 0 2024-03-22T10:42:51.827 INFO:tasks.workunit.client.0.smithi115.stdout:6/682: dread d4/d13/f4b [0,4194304] 0 2024-03-22T10:42:51.828 INFO:tasks.workunit.client.1.smithi150.stdout:1/307: dwrite d1/d19/f26 [0,4194304] 0 2024-03-22T10:42:51.828 INFO:tasks.workunit.client.0.smithi115.stdout:8/911: read d6/dd/d12/d31/f90 [285121,60403] 0 2024-03-22T10:42:51.830 INFO:tasks.workunit.client.1.smithi150.stdout:8/361: dwrite d0/d2/d19/f78 [4194304,4194304] 0 2024-03-22T10:42:51.830 INFO:tasks.workunit.client.1.smithi150.stdout:6/305: unlink db/dd/d15/d28/l59 0 2024-03-22T10:42:51.832 INFO:tasks.workunit.client.1.smithi150.stdout:6/306: chown db/dd/d15 3663 1 2024-03-22T10:42:51.839 INFO:tasks.workunit.client.0.smithi115.stdout:1/803: truncate d0/d8/d4d/f5a 540336 0 2024-03-22T10:42:51.840 INFO:tasks.workunit.client.1.smithi150.stdout:6/307: chown db/dd/d25/f34 601 1 2024-03-22T10:42:51.841 INFO:tasks.workunit.client.1.smithi150.stdout:1/308: creat d1/da/d15/d18/f5f x:0 0 0 2024-03-22T10:42:51.845 INFO:tasks.workunit.client.0.smithi115.stdout:9/735: dread d6/d7/d19/f27 [4194304,4194304] 0 2024-03-22T10:42:51.845 INFO:tasks.workunit.client.0.smithi115.stdout:8/912: rename d6/c18 to d6/dd/d12/d31/c136 0 2024-03-22T10:42:51.846 INFO:tasks.workunit.client.1.smithi150.stdout:8/362: mkdir d0/d2/d19/d29/d37/d7d 0 2024-03-22T10:42:51.846 INFO:tasks.workunit.client.1.smithi150.stdout:2/404: dwrite d12/d19/f59 [0,4194304] 0 2024-03-22T10:42:51.848 INFO:tasks.workunit.client.1.smithi150.stdout:6/308: rmdir db/dd/d15/d6a 39 2024-03-22T10:42:51.850 INFO:tasks.workunit.client.0.smithi115.stdout:6/683: dread d4/d36/d48/d72/fa9 [0,4194304] 0 2024-03-22T10:42:51.851 INFO:tasks.workunit.client.1.smithi150.stdout:9/418: dwrite d1/d8/de/df/d20/f50 [0,4194304] 0 2024-03-22T10:42:51.852 INFO:tasks.workunit.client.0.smithi115.stdout:2/663: rmdir da/d7b/da4 0 2024-03-22T10:42:51.852 INFO:tasks.workunit.client.0.smithi115.stdout:7/764: write d2/d7/d12/d10/f96 [88834,62574] 0 2024-03-22T10:42:51.855 INFO:tasks.workunit.client.0.smithi115.stdout:2/664: write da/f1d [9374103,102138] 0 2024-03-22T10:42:51.856 INFO:tasks.workunit.client.0.smithi115.stdout:4/710: dread d1/d30/d4c/d86/d9f/fd6 [0,4194304] 0 2024-03-22T10:42:51.859 INFO:tasks.workunit.client.0.smithi115.stdout:6/684: mknod d4/d36/cd8 0 2024-03-22T10:42:51.860 INFO:tasks.workunit.client.0.smithi115.stdout:1/804: creat d0/f11c x:0 0 0 2024-03-22T10:42:51.861 INFO:tasks.workunit.client.1.smithi150.stdout:2/405: mknod d12/d19/d2d/d68/d78/c85 0 2024-03-22T10:42:51.865 INFO:tasks.workunit.client.0.smithi115.stdout:7/765: creat d2/d7/d12/d10/d19/d42/d76/def/f10a x:0 0 0 2024-03-22T10:42:51.866 INFO:tasks.workunit.client.1.smithi150.stdout:8/363: dread d0/d2/d19/d29/d31/f4b [0,4194304] 0 2024-03-22T10:42:51.866 INFO:tasks.workunit.client.0.smithi115.stdout:4/711: truncate d1/d66/f70 1592294 0 2024-03-22T10:42:51.866 INFO:tasks.workunit.client.0.smithi115.stdout:2/665: read - da/d25/d3d/d57/d7d/fac zero size 2024-03-22T10:42:51.866 INFO:tasks.workunit.client.0.smithi115.stdout:1/805: rmdir d0/d8/db/d25 39 2024-03-22T10:42:51.869 INFO:tasks.workunit.client.1.smithi150.stdout:8/364: chown d0/d2/d19/d29/l54 0 1 2024-03-22T10:42:51.872 INFO:tasks.workunit.client.0.smithi115.stdout:4/712: symlink d1/dbb/dc6/lf2 0 2024-03-22T10:42:51.873 INFO:tasks.workunit.client.0.smithi115.stdout:5/777: dwrite d2/d15/d1b/d7c/d38/d43/d49/d81/dbd/fc1 [0,4194304] 0 2024-03-22T10:42:51.875 INFO:tasks.workunit.client.1.smithi150.stdout:6/309: dread db/f44 [0,4194304] 0 2024-03-22T10:42:51.877 INFO:tasks.workunit.client.0.smithi115.stdout:1/806: read d0/d68/f7f [41276,85759] 0 2024-03-22T10:42:51.881 INFO:tasks.workunit.client.1.smithi150.stdout:0/382: dread d0/d12/d4f/f31 [4194304,4194304] 0 2024-03-22T10:42:51.882 INFO:tasks.workunit.client.1.smithi150.stdout:6/310: mknod db/dd/d2e/c74 0 2024-03-22T10:42:51.883 INFO:tasks.workunit.client.0.smithi115.stdout:3/747: dread d3/f23 [0,4194304] 0 2024-03-22T10:42:51.884 INFO:tasks.workunit.client.1.smithi150.stdout:8/365: mkdir d0/d2/d7e 0 2024-03-22T10:42:51.884 INFO:tasks.workunit.client.0.smithi115.stdout:9/736: dwrite d6/d7/d20/f57 [0,4194304] 0 2024-03-22T10:42:51.885 INFO:tasks.workunit.client.0.smithi115.stdout:4/713: getdents d1/d30/d4c/d86/d9f/d75/d99/da8/dea 0 2024-03-22T10:42:51.885 INFO:tasks.workunit.client.1.smithi150.stdout:0/383: rmdir d0/d12/d4f/d42/d10/d38 39 2024-03-22T10:42:51.886 INFO:tasks.workunit.client.1.smithi150.stdout:8/366: dread - d0/d3c/f66 zero size 2024-03-22T10:42:51.888 INFO:tasks.workunit.client.0.smithi115.stdout:3/748: write d3/f23 [1728473,11228] 0 2024-03-22T10:42:51.897 INFO:tasks.workunit.client.0.smithi115.stdout:2/666: dread da/d25/d30/f59 [0,4194304] 0 2024-03-22T10:42:51.897 INFO:tasks.workunit.client.0.smithi115.stdout:3/749: rename d3/d16/d29/d35/lea to d3/d19/d2b/d49/d91/dad/l100 0 2024-03-22T10:42:51.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:51 smithi150 ceph-mon[34518]: pgmap v14: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 73 MiB/s rd, 271 MiB/s wr, 510 op/s 2024-03-22T10:42:51.903 INFO:tasks.workunit.client.1.smithi150.stdout:4/344: dread d0/d2f/f3b [0,4194304] 0 2024-03-22T10:42:51.903 INFO:tasks.workunit.client.0.smithi115.stdout:5/778: dwrite d2/df/d13/fbb [4194304,4194304] 0 2024-03-22T10:42:51.903 INFO:tasks.workunit.client.1.smithi150.stdout:1/309: dread d1/f7 [0,4194304] 0 2024-03-22T10:42:51.905 INFO:tasks.workunit.client.0.smithi115.stdout:8/913: write d6/dd/d24/d4a/f61 [4932647,103703] 0 2024-03-22T10:42:51.906 INFO:tasks.workunit.client.1.smithi150.stdout:0/384: dread d0/d12/d4f/d42/d10/d38/d52/f54 [0,4194304] 0 2024-03-22T10:42:51.907 INFO:tasks.workunit.client.0.smithi115.stdout:5/779: dread - d2/df/d3b/d8e/fd6 zero size 2024-03-22T10:42:51.909 INFO:tasks.workunit.client.0.smithi115.stdout:3/750: symlink d3/d19/d2b/d49/d91/dad/l101 0 2024-03-22T10:42:51.910 INFO:tasks.workunit.client.1.smithi150.stdout:9/419: truncate d1/d8/de/df/d20/f5d 963596 0 2024-03-22T10:42:51.911 INFO:tasks.workunit.client.0.smithi115.stdout:9/737: getdents d6/d8/d3c/d59/d68 0 2024-03-22T10:42:51.913 INFO:tasks.workunit.client.1.smithi150.stdout:8/367: dwrite d0/d2/d19/d29/d2e/f3a [0,4194304] 0 2024-03-22T10:42:51.913 INFO:tasks.workunit.client.1.smithi150.stdout:4/345: fdatasync d0/f9 0 2024-03-22T10:42:51.914 INFO:tasks.workunit.client.0.smithi115.stdout:3/751: chown d3/d19/d61/da6 817911728 1 2024-03-22T10:42:51.916 INFO:tasks.workunit.client.0.smithi115.stdout:7/766: write d2/d7/d12/f99 [769349,4058] 0 2024-03-22T10:42:51.916 INFO:tasks.workunit.client.1.smithi150.stdout:1/310: creat d1/da/d15/d4f/f60 x:0 0 0 2024-03-22T10:42:51.917 INFO:tasks.workunit.client.0.smithi115.stdout:4/714: dwrite d1/d30/d4c/d86/d9f/f34 [0,4194304] 0 2024-03-22T10:42:51.917 INFO:tasks.workunit.client.0.smithi115.stdout:8/914: creat d6/f137 x:0 0 0 2024-03-22T10:42:51.919 INFO:tasks.workunit.client.1.smithi150.stdout:1/311: readlink d1/d19/d1a/l37 0 2024-03-22T10:42:51.920 INFO:tasks.workunit.client.0.smithi115.stdout:9/738: creat d6/d7/d20/d38/f107 x:0 0 0 2024-03-22T10:42:51.925 INFO:tasks.workunit.client.0.smithi115.stdout:4/715: mkdir d1/d16/d35/d40/d54/d5e/df3 0 2024-03-22T10:42:51.926 INFO:tasks.workunit.client.0.smithi115.stdout:2/667: dwrite da/d1c/f88 [0,4194304] 0 2024-03-22T10:42:51.928 INFO:tasks.workunit.client.0.smithi115.stdout:4/716: chown d1/fd 47930363 1 2024-03-22T10:42:51.930 INFO:tasks.workunit.client.1.smithi150.stdout:1/312: mknod d1/d19/d57/c61 0 2024-03-22T10:42:51.932 INFO:tasks.workunit.client.0.smithi115.stdout:8/915: creat d6/dd/d24/d4a/dfc/d102/f138 x:0 0 0 2024-03-22T10:42:51.933 INFO:tasks.workunit.client.0.smithi115.stdout:2/668: creat da/d25/d51/fc8 x:0 0 0 2024-03-22T10:42:51.934 INFO:tasks.workunit.client.0.smithi115.stdout:4/717: chown d1/d16/d3b 82375669 1 2024-03-22T10:42:51.939 INFO:tasks.workunit.client.0.smithi115.stdout:8/916: readlink d6/dd/d24/d3a/lf7 0 2024-03-22T10:42:51.943 INFO:tasks.workunit.client.0.smithi115.stdout:4/718: creat d1/dbb/dc6/ff4 x:0 0 0 2024-03-22T10:42:51.945 INFO:tasks.workunit.client.0.smithi115.stdout:2/669: getdents da/d25/d30 0 2024-03-22T10:42:51.946 INFO:tasks.workunit.client.1.smithi150.stdout:6/311: dwrite db/dd/d15/f27 [0,4194304] 0 2024-03-22T10:42:51.947 INFO:tasks.workunit.client.0.smithi115.stdout:4/719: write d1/d16/f64 [619061,90056] 0 2024-03-22T10:42:51.949 INFO:tasks.workunit.client.0.smithi115.stdout:7/767: dwrite d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/dbb/fdd [0,4194304] 0 2024-03-22T10:42:51.950 INFO:tasks.workunit.client.1.smithi150.stdout:8/368: dwrite d0/f5b [0,4194304] 0 2024-03-22T10:42:51.951 INFO:tasks.workunit.client.0.smithi115.stdout:2/670: write da/d1c/f4c [1829,111466] 0 2024-03-22T10:42:51.951 INFO:tasks.workunit.client.0.smithi115.stdout:1/807: dwrite d0/f87 [0,4194304] 0 2024-03-22T10:42:51.953 INFO:tasks.workunit.client.1.smithi150.stdout:8/369: fsync d0/d2/d19/f47 0 2024-03-22T10:42:51.957 INFO:tasks.workunit.client.0.smithi115.stdout:7/768: unlink d2/c15 0 2024-03-22T10:42:51.958 INFO:tasks.workunit.client.0.smithi115.stdout:4/720: chown d1/d30/d4c/d86/d9f/d32/f5b 368 1 2024-03-22T10:42:51.964 INFO:tasks.workunit.client.0.smithi115.stdout:1/808: symlink d0/d8/d4d/d56/dcf/d10b/de2/df8/l11d 0 2024-03-22T10:42:51.965 INFO:tasks.workunit.client.1.smithi150.stdout:0/385: truncate d0/d12/d4f/d42/d10/f43 484208 0 2024-03-22T10:42:51.965 INFO:tasks.workunit.client.0.smithi115.stdout:7/769: chown d2/d7/d12/d10/d23/d3d/dd2/da3/de8/ldf 31 1 2024-03-22T10:42:51.966 INFO:tasks.workunit.client.1.smithi150.stdout:4/346: dwrite d0/d1b/d4d/d6c/d61/f48 [0,4194304] 0 2024-03-22T10:42:51.966 INFO:tasks.workunit.client.0.smithi115.stdout:2/671: truncate da/d25/d3d/f76 4298907 0 2024-03-22T10:42:51.967 INFO:tasks.workunit.client.1.smithi150.stdout:8/370: link d0/d2/d19/d29/d2e/f58 d0/d3c/d71/f7f 0 2024-03-22T10:42:51.971 INFO:tasks.workunit.client.0.smithi115.stdout:4/721: symlink d1/dbb/ddf/lf5 0 2024-03-22T10:42:51.972 INFO:tasks.workunit.client.0.smithi115.stdout:7/770: chown d2/d7/d12/d10/d19/d42/d76/def/f10a 604406 1 2024-03-22T10:42:51.975 INFO:tasks.workunit.client.1.smithi150.stdout:4/347: mknod d0/d2f/c78 0 2024-03-22T10:42:51.975 INFO:tasks.workunit.client.1.smithi150.stdout:8/371: read - d0/d2/d19/d29/d2f/df/f45 zero size 2024-03-22T10:42:51.976 INFO:tasks.workunit.client.0.smithi115.stdout:5/780: truncate d2/d15/d1b/d7c/d38/d56/f87 2707956 0 2024-03-22T10:42:51.977 INFO:tasks.workunit.client.1.smithi150.stdout:8/372: write d0/f6f [621877,23708] 0 2024-03-22T10:42:51.978 INFO:tasks.workunit.client.1.smithi150.stdout:4/348: chown d0/f9 913 1 2024-03-22T10:42:51.981 INFO:tasks.workunit.client.0.smithi115.stdout:9/739: write d6/d7/d20/d22/fad [1094396,48660] 0 2024-03-22T10:42:51.983 INFO:tasks.workunit.client.0.smithi115.stdout:5/781: creat d2/df/d3b/dfb/f100 x:0 0 0 2024-03-22T10:42:51.984 INFO:tasks.workunit.client.1.smithi150.stdout:8/373: chown d0/f1 96685161 1 2024-03-22T10:42:51.984 INFO:tasks.workunit.client.1.smithi150.stdout:4/349: chown d0/d2f/l35 7 1 2024-03-22T10:42:51.986 INFO:tasks.workunit.client.1.smithi150.stdout:6/312: dwrite db/dd/d25/f3b [0,4194304] 0 2024-03-22T10:42:51.989 INFO:tasks.workunit.client.1.smithi150.stdout:6/313: write db/d52/f6f [1878655,25675] 0 2024-03-22T10:42:51.990 INFO:tasks.workunit.client.0.smithi115.stdout:1/809: creat d0/d8/f11e x:0 0 0 2024-03-22T10:42:51.991 INFO:tasks.workunit.client.1.smithi150.stdout:6/314: write db/dd/d15/d28/f58 [3107681,17704] 0 2024-03-22T10:42:51.993 INFO:tasks.workunit.client.1.smithi150.stdout:8/374: mknod d0/c80 0 2024-03-22T10:42:51.994 INFO:tasks.workunit.client.0.smithi115.stdout:8/917: dread d6/d49/d88/fe2 [0,4194304] 0 2024-03-22T10:42:51.996 INFO:tasks.workunit.client.1.smithi150.stdout:9/420: dread d1/d8/de/df/d24/f6b [0,4194304] 0 2024-03-22T10:42:51.997 INFO:tasks.workunit.client.1.smithi150.stdout:4/350: rename d0/d2f/l35 to d0/d47/l79 0 2024-03-22T10:42:51.997 INFO:tasks.workunit.client.0.smithi115.stdout:2/672: dwrite da/d25/d3d/d57/f65 [0,4194304] 0 2024-03-22T10:42:52.000 INFO:tasks.workunit.client.1.smithi150.stdout:9/421: dread - d1/d8/de/df/d20/d29/f54 zero size 2024-03-22T10:42:52.000 INFO:tasks.workunit.client.1.smithi150.stdout:6/315: rmdir db/dd/d15 39 2024-03-22T10:42:52.000 INFO:tasks.workunit.client.0.smithi115.stdout:5/782: mknod d2/d15/d1b/d7c/d38/d43/d49/d81/dbd/c101 0 2024-03-22T10:42:52.002 INFO:tasks.workunit.client.1.smithi150.stdout:9/422: stat d1/d8/de/df/d24/d58/d8c/f8b 0 2024-03-22T10:42:52.004 INFO:tasks.workunit.client.1.smithi150.stdout:8/375: readlink d0/d2/d19/d29/d2f/df/d1b/l60 0 2024-03-22T10:42:52.007 INFO:tasks.workunit.client.1.smithi150.stdout:4/351: mknod d0/d2f/d63/c7a 0 2024-03-22T10:42:52.008 INFO:tasks.workunit.client.1.smithi150.stdout:8/376: read d0/d2/f5f [228240,19401] 0 2024-03-22T10:42:52.011 INFO:tasks.workunit.client.0.smithi115.stdout:9/740: dwrite d6/d8/d3c/d59/d68/d8e/f9a [0,4194304] 0 2024-03-22T10:42:52.012 INFO:tasks.workunit.client.1.smithi150.stdout:8/377: write d0/d3c/f44 [5057867,56419] 0 2024-03-22T10:42:52.014 INFO:tasks.workunit.client.0.smithi115.stdout:4/722: write d1/d30/d4c/d86/d9f/d32/f5b [1010317,14713] 0 2024-03-22T10:42:52.015 INFO:tasks.workunit.client.0.smithi115.stdout:5/783: read d2/df/d13/d8b/d93/da6/dbc/fb2 [700450,63570] 0 2024-03-22T10:42:52.021 INFO:tasks.workunit.client.0.smithi115.stdout:9/741: chown d6/d7/d19/l29 29 1 2024-03-22T10:42:52.022 INFO:tasks.workunit.client.1.smithi150.stdout:4/352: fdatasync d0/d1b/f2c 0 2024-03-22T10:42:52.026 INFO:tasks.workunit.client.0.smithi115.stdout:4/723: write d1/d30/d4c/d86/d9f/d75/d99/da8/fd0 [301509,86827] 0 2024-03-22T10:42:52.027 INFO:tasks.workunit.client.0.smithi115.stdout:9/742: chown d6/d8/d3c/f54 405630 1 2024-03-22T10:42:52.027 INFO:tasks.workunit.client.0.smithi115.stdout:5/784: dread - d2/df/d13/d8b/d93/da6/dbc/d50/d96/fcc zero size 2024-03-22T10:42:52.033 INFO:tasks.workunit.client.0.smithi115.stdout:7/771: dwrite d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f59 [0,4194304] 0 2024-03-22T10:42:52.033 INFO:tasks.workunit.client.0.smithi115.stdout:5/785: chown d2/df/d13/le8 160 1 2024-03-22T10:42:52.034 INFO:tasks.workunit.client.1.smithi150.stdout:4/353: mkdir d0/d2f/d63/d7b 0 2024-03-22T10:42:52.034 INFO:tasks.workunit.client.0.smithi115.stdout:2/673: dread da/d25/d3d/d7e/d90/dbf/d54/f7a [0,4194304] 0 2024-03-22T10:42:52.037 INFO:tasks.workunit.client.1.smithi150.stdout:4/354: mkdir d0/d32/d7c 0 2024-03-22T10:42:52.038 INFO:tasks.workunit.client.0.smithi115.stdout:2/674: chown da/d25/d30/f59 3961 1 2024-03-22T10:42:52.041 INFO:tasks.workunit.client.0.smithi115.stdout:2/675: stat da/d1c/d67/d78/d86 0 2024-03-22T10:42:52.041 INFO:tasks.workunit.client.0.smithi115.stdout:5/786: write d2/d15/d1b/d7c/daf/ffc [595849,56156] 0 2024-03-22T10:42:52.041 INFO:tasks.workunit.client.0.smithi115.stdout:7/772: write d2/d7/d12/d10/d19/dbd/f104 [469151,107011] 0 2024-03-22T10:42:52.042 INFO:tasks.workunit.client.1.smithi150.stdout:4/355: fsync d0/de/f18 0 2024-03-22T10:42:52.044 INFO:tasks.workunit.client.0.smithi115.stdout:2/676: chown da/d25/d3d/f97 352 1 2024-03-22T10:42:52.044 INFO:tasks.workunit.client.1.smithi150.stdout:8/378: getdents d0/d2/d19 0 2024-03-22T10:42:52.047 INFO:tasks.workunit.client.0.smithi115.stdout:9/743: unlink d6/d7/d20/d22/de5/ff2 0 2024-03-22T10:42:52.048 INFO:tasks.workunit.client.1.smithi150.stdout:8/379: unlink d0/d2/d19/c62 0 2024-03-22T10:42:52.050 INFO:tasks.workunit.client.1.smithi150.stdout:0/386: dread d0/d12/d4f/d42/d10/d38/f4b [0,4194304] 0 2024-03-22T10:42:52.050 INFO:tasks.workunit.client.1.smithi150.stdout:4/356: mknod d0/d1b/d4d/d6c/d61/c7d 0 2024-03-22T10:42:52.052 INFO:tasks.workunit.client.0.smithi115.stdout:8/918: dread d6/dd/f2e [0,4194304] 0 2024-03-22T10:42:52.054 INFO:tasks.workunit.client.0.smithi115.stdout:9/744: mkdir d6/d7/d20/d9c/d108 0 2024-03-22T10:42:52.055 INFO:tasks.workunit.client.1.smithi150.stdout:6/316: truncate db/dd/d15/f35 3843393 0 2024-03-22T10:42:52.056 INFO:tasks.workunit.client.0.smithi115.stdout:4/724: link d1/d30/d4c/d86/d9f/d4f/d93/c91 d1/d30/d4c/d86/d9f/de9/cf6 0 2024-03-22T10:42:52.057 INFO:tasks.workunit.client.0.smithi115.stdout:9/745: chown d6/d8/d3c 893957 1 2024-03-22T10:42:52.059 INFO:tasks.workunit.client.1.smithi150.stdout:4/357: dread d0/d32/f41 [0,4194304] 0 2024-03-22T10:42:52.060 INFO:tasks.workunit.client.1.smithi150.stdout:6/317: dread - db/dd/d15/d6a/f6b zero size 2024-03-22T10:42:52.060 INFO:tasks.workunit.client.0.smithi115.stdout:4/725: mkdir d1/dbb/dc6/dd8/df7 0 2024-03-22T10:42:52.061 INFO:tasks.workunit.client.0.smithi115.stdout:5/787: write d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f9b [1727042,52093] 0 2024-03-22T10:42:52.062 INFO:tasks.workunit.client.0.smithi115.stdout:9/746: creat d6/d7/d19/d4e/f109 x:0 0 0 2024-03-22T10:42:52.068 INFO:tasks.workunit.client.1.smithi150.stdout:4/358: truncate d0/d32/f4e 1766305 0 2024-03-22T10:42:52.068 INFO:tasks.workunit.client.0.smithi115.stdout:7/773: dwrite d2/d7/d12/f67 [4194304,4194304] 0 2024-03-22T10:42:52.069 INFO:tasks.workunit.client.0.smithi115.stdout:5/788: chown d2/d15/d1b/d7c/d38/d43/d49/d81/d83/l6a 55522 1 2024-03-22T10:42:52.072 INFO:tasks.workunit.client.1.smithi150.stdout:0/387: truncate d0/d12/d4f/d42/d10/d38/f4c 3350299 0 2024-03-22T10:42:52.075 INFO:tasks.workunit.client.0.smithi115.stdout:2/677: dwrite da/d25/d3d/d57/f5d [0,4194304] 0 2024-03-22T10:42:52.077 INFO:tasks.workunit.client.1.smithi150.stdout:8/380: write d0/d2/d19/f20 [6256157,77532] 0 2024-03-22T10:42:52.077 INFO:tasks.workunit.client.1.smithi150.stdout:4/359: write d0/d1b/d68/f6a [2482297,97096] 0 2024-03-22T10:42:52.080 INFO:tasks.workunit.client.0.smithi115.stdout:5/789: creat d2/df/d3b/dfb/f102 x:0 0 0 2024-03-22T10:42:52.083 INFO:tasks.workunit.client.1.smithi150.stdout:4/360: creat d0/d47/f7e x:0 0 0 2024-03-22T10:42:52.084 INFO:tasks.workunit.client.1.smithi150.stdout:8/381: symlink d0/d2/d19/d29/d69/l81 0 2024-03-22T10:42:52.085 INFO:tasks.workunit.client.1.smithi150.stdout:6/318: dwrite db/dd/d15/d30/f4b [4194304,4194304] 0 2024-03-22T10:42:52.087 INFO:tasks.workunit.client.1.smithi150.stdout:4/361: write d0/d1b/d4d/d6c/d61/f48 [98866,46085] 0 2024-03-22T10:42:52.088 INFO:tasks.workunit.client.0.smithi115.stdout:2/678: chown da/d25/d3d/d57/c5c 71817 1 2024-03-22T10:42:52.088 INFO:tasks.workunit.client.0.smithi115.stdout:9/747: rename d6/d7/d20/d22/d3f/fe7 to d6/d7/d20/f10a 0 2024-03-22T10:42:52.088 INFO:tasks.workunit.client.1.smithi150.stdout:6/319: write db/f4a [1798114,122779] 0 2024-03-22T10:42:52.089 INFO:tasks.workunit.client.0.smithi115.stdout:1/810: dread d0/d8/de/d1a/f61 [0,4194304] 0 2024-03-22T10:42:52.089 INFO:tasks.workunit.client.0.smithi115.stdout:8/919: dwrite d6/dd/d24/d3a/f54 [0,4194304] 0 2024-03-22T10:42:52.090 INFO:tasks.workunit.client.1.smithi150.stdout:8/382: write d0/f1 [2744135,27854] 0 2024-03-22T10:42:52.097 INFO:tasks.workunit.client.0.smithi115.stdout:1/811: fdatasync d0/d8/de/d1a/d91/feb 0 2024-03-22T10:42:52.097 INFO:tasks.workunit.client.0.smithi115.stdout:0/671: sync 2024-03-22T10:42:52.098 INFO:tasks.workunit.client.1.smithi150.stdout:6/320: mknod db/dd/d15/d70/c75 0 2024-03-22T10:42:52.099 INFO:tasks.workunit.client.0.smithi115.stdout:2/679: unlink da/d25/d2a/l45 0 2024-03-22T10:42:52.100 INFO:tasks.workunit.client.1.smithi150.stdout:8/383: creat d0/d2/d7e/f82 x:0 0 0 2024-03-22T10:42:52.103 INFO:tasks.workunit.client.0.smithi115.stdout:5/790: creat d2/df/d13/f103 x:0 0 0 2024-03-22T10:42:52.103 INFO:tasks.workunit.client.0.smithi115.stdout:7/774: link d2/d7/d12/l8e d2/d7/d12/d10/d23/d3d/dd2/da3/l10b 0 2024-03-22T10:42:52.105 INFO:tasks.workunit.client.0.smithi115.stdout:5/791: dread - d2/fc7 zero size 2024-03-22T10:42:52.106 INFO:tasks.workunit.client.1.smithi150.stdout:8/384: rmdir d0/d2/d19/d29/d2f/df/d1b 39 2024-03-22T10:42:52.107 INFO:tasks.workunit.client.0.smithi115.stdout:0/672: stat d3/d7/d6b/ca2 0 2024-03-22T10:42:52.109 INFO:tasks.workunit.client.0.smithi115.stdout:0/673: chown d3/da/d4c/d69 14164792 1 2024-03-22T10:42:52.112 INFO:tasks.workunit.client.1.smithi150.stdout:4/362: dwrite d0/d1b/d4d/d6c/d61/f69 [0,4194304] 0 2024-03-22T10:42:52.112 INFO:tasks.workunit.client.0.smithi115.stdout:7/775: write d2/d7/d12/fb8 [704492,92849] 0 2024-03-22T10:42:52.114 INFO:tasks.workunit.client.0.smithi115.stdout:1/812: write d0/d12/d17/d37/d9e/de9/dec/ded/f115 [946127,50623] 0 2024-03-22T10:42:52.116 INFO:tasks.workunit.client.1.smithi150.stdout:4/363: chown d0/d1b/c43 10585 1 2024-03-22T10:42:52.116 INFO:tasks.workunit.client.1.smithi150.stdout:8/385: chown d0/d2/d19/d29/d2f/df/d1b/c72 232463284 1 2024-03-22T10:42:52.119 INFO:tasks.workunit.client.1.smithi150.stdout:4/364: creat d0/de/f7f x:0 0 0 2024-03-22T10:42:52.120 INFO:tasks.workunit.client.0.smithi115.stdout:4/726: dwrite d1/d30/d4c/d86/d9f/d75/d99/f97 [0,4194304] 0 2024-03-22T10:42:52.121 INFO:tasks.workunit.client.1.smithi150.stdout:6/321: dwrite db/dd/d15/d30/f51 [0,4194304] 0 2024-03-22T10:42:52.123 INFO:tasks.workunit.client.1.smithi150.stdout:4/365: write d0/d2f/f30 [4318566,94592] 0 2024-03-22T10:42:52.124 INFO:tasks.workunit.client.0.smithi115.stdout:6/685: sync 2024-03-22T10:42:52.124 INFO:tasks.workunit.client.0.smithi115.stdout:3/752: sync 2024-03-22T10:42:52.125 INFO:tasks.workunit.client.0.smithi115.stdout:1/813: stat d0/d12/d17/d37/d9e/de9/dde 0 2024-03-22T10:42:52.125 INFO:tasks.workunit.client.0.smithi115.stdout:4/727: chown d1/d16 1335 1 2024-03-22T10:42:52.126 INFO:tasks.workunit.client.1.smithi150.stdout:6/322: creat db/dd/d25/f76 x:0 0 0 2024-03-22T10:42:52.126 INFO:tasks.workunit.client.0.smithi115.stdout:7/776: write d2/d7/d12/d10/d19/d42/fc4 [522512,19453] 0 2024-03-22T10:42:52.126 INFO:tasks.workunit.client.0.smithi115.stdout:0/674: rename d3/l4e to d3/d4/d14/d64/lda 0 2024-03-22T10:42:52.132 INFO:tasks.workunit.client.1.smithi150.stdout:8/386: rename d0/d2/d19/d29/d2f/df/f51 to d0/d2/f83 0 2024-03-22T10:42:52.134 INFO:tasks.workunit.client.1.smithi150.stdout:6/323: getdents db/dd/d15/d70 0 2024-03-22T10:42:52.136 INFO:tasks.workunit.client.1.smithi150.stdout:6/324: chown db/dd/f2c 878191 1 2024-03-22T10:42:52.139 INFO:tasks.workunit.client.0.smithi115.stdout:1/814: fsync d0/d8/f9 0 2024-03-22T10:42:52.140 INFO:tasks.workunit.client.0.smithi115.stdout:9/748: dwrite f3 [0,4194304] 0 2024-03-22T10:42:52.141 INFO:tasks.workunit.client.1.smithi150.stdout:6/325: creat db/dd/d15/f77 x:0 0 0 2024-03-22T10:42:52.141 INFO:tasks.workunit.client.0.smithi115.stdout:8/920: dwrite d6/f11f [4194304,4194304] 0 2024-03-22T10:42:52.143 INFO:tasks.workunit.client.1.smithi150.stdout:4/366: dwrite d0/de/f28 [0,4194304] 0 2024-03-22T10:42:52.151 INFO:tasks.workunit.client.1.smithi150.stdout:4/367: symlink d0/d1b/d4d/l80 0 2024-03-22T10:42:52.153 INFO:tasks.workunit.client.1.smithi150.stdout:4/368: readlink d0/d1b/d4d/l6b 0 2024-03-22T10:42:52.154 INFO:tasks.workunit.client.0.smithi115.stdout:7/777: read d2/d7/d12/d10/d23/d3d/dd2/da3/fab [217121,33760] 0 2024-03-22T10:42:52.154 INFO:tasks.workunit.client.0.smithi115.stdout:0/675: symlink d3/d7/d1b/d1f/d5b/dba/ldb 0 2024-03-22T10:42:52.157 INFO:tasks.workunit.client.0.smithi115.stdout:5/792: rename d2/df/d13/d8b/d93/da6/fe5 to d2/d15/f104 0 2024-03-22T10:42:52.159 INFO:tasks.workunit.client.0.smithi115.stdout:8/921: write d6/dd/d24/d4a/d7b/d8f/da0/fb8 [3486682,29886] 0 2024-03-22T10:42:52.160 INFO:tasks.workunit.client.1.smithi150.stdout:4/369: creat d0/de/f81 x:0 0 0 2024-03-22T10:42:52.161 INFO:tasks.workunit.client.0.smithi115.stdout:4/728: mkdir d1/dbb/dc6/dd8/df7/df8 0 2024-03-22T10:42:52.164 INFO:tasks.workunit.client.1.smithi150.stdout:6/326: dwrite db/dd/d15/d30/f51 [0,4194304] 0 2024-03-22T10:42:52.166 INFO:tasks.workunit.client.0.smithi115.stdout:4/729: dread - d1/d16/d28/fd7 zero size 2024-03-22T10:42:52.173 INFO:tasks.workunit.client.0.smithi115.stdout:9/749: creat d6/d7/d48/df4/dfa/f10b x:0 0 0 2024-03-22T10:42:52.175 INFO:tasks.workunit.client.0.smithi115.stdout:2/680: dwrite da/d25/d3d/f4e [0,4194304] 0 2024-03-22T10:42:52.176 INFO:tasks.workunit.client.1.smithi150.stdout:6/327: getdents db/dd/d15/d60/d50 0 2024-03-22T10:42:52.178 INFO:tasks.workunit.client.0.smithi115.stdout:3/753: rename d3/d89/lb6 to d3/d36/d41/d85/l102 0 2024-03-22T10:42:52.182 INFO:tasks.workunit.client.0.smithi115.stdout:8/922: mkdir d6/dd/d24/d4a/d139 0 2024-03-22T10:42:52.189 INFO:tasks.workunit.client.0.smithi115.stdout:0/676: mknod d3/da/cdc 0 2024-03-22T10:42:52.189 INFO:tasks.workunit.client.1.smithi150.stdout:6/328: dread db/dd/d15/d30/f4b [4194304,4194304] 0 2024-03-22T10:42:52.189 INFO:tasks.workunit.client.0.smithi115.stdout:6/686: dwrite d4/d36/f5c [0,4194304] 0 2024-03-22T10:42:52.191 INFO:tasks.workunit.client.1.smithi150.stdout:8/387: dread d0/d3c/f44 [0,4194304] 0 2024-03-22T10:42:52.195 INFO:tasks.workunit.client.0.smithi115.stdout:4/730: mkdir d1/dbb/ddf/df9 0 2024-03-22T10:42:52.196 INFO:tasks.workunit.client.1.smithi150.stdout:8/388: symlink d0/d2/d19/d29/d37/l84 0 2024-03-22T10:42:52.196 INFO:tasks.workunit.client.0.smithi115.stdout:6/687: stat d4/d11/c47 0 2024-03-22T10:42:52.197 INFO:tasks.workunit.client.0.smithi115.stdout:2/681: rename da/d25/d3d/c9b to da/d25/d3d/d7e/cc9 0 2024-03-22T10:42:52.197 INFO:tasks.workunit.client.0.smithi115.stdout:8/923: fsync d6/dd/d24/d4a/f87 0 2024-03-22T10:42:52.198 INFO:tasks.workunit.client.0.smithi115.stdout:9/750: creat d6/d7/d20/d22/d23/d106/f10c x:0 0 0 2024-03-22T10:42:52.204 INFO:tasks.workunit.client.0.smithi115.stdout:7/778: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f10c x:0 0 0 2024-03-22T10:42:52.207 INFO:tasks.workunit.client.1.smithi150.stdout:4/370: dread d0/d2f/f30 [0,4194304] 0 2024-03-22T10:42:52.207 INFO:tasks.workunit.client.1.smithi150.stdout:8/389: rename d0/d3c/f68 to d0/d2/d19/d29/d37/f85 0 2024-03-22T10:42:52.209 INFO:tasks.workunit.client.1.smithi150.stdout:6/329: dwrite db/dd/d15/d30/f51 [0,4194304] 0 2024-03-22T10:42:52.210 INFO:tasks.workunit.client.0.smithi115.stdout:5/793: creat d2/d15/d1b/d7c/f105 x:0 0 0 2024-03-22T10:42:52.210 INFO:tasks.workunit.client.0.smithi115.stdout:6/688: creat d4/d13/d33/dbe/fd9 x:0 0 0 2024-03-22T10:42:52.211 INFO:tasks.workunit.client.0.smithi115.stdout:7/779: stat d2/d7/d12/d10/d23/d3d/ff0 0 2024-03-22T10:42:52.212 INFO:tasks.workunit.client.0.smithi115.stdout:3/754: dwrite d3/fb2 [0,4194304] 0 2024-03-22T10:42:52.212 INFO:tasks.workunit.client.1.smithi150.stdout:8/390: write d0/d2/d19/d29/d2f/df/f26 [569797,1323] 0 2024-03-22T10:42:52.213 INFO:tasks.workunit.client.0.smithi115.stdout:2/682: unlink da/d1c/l8f 0 2024-03-22T10:42:52.217 INFO:tasks.workunit.client.1.smithi150.stdout:8/391: creat d0/d2/f86 x:0 0 0 2024-03-22T10:42:52.217 INFO:tasks.workunit.client.0.smithi115.stdout:4/731: symlink d1/d30/d4c/d86/d9f/d32/lfa 0 2024-03-22T10:42:52.222 INFO:tasks.workunit.client.1.smithi150.stdout:8/392: getdents d0/d2/d19/d29/d2f/df/d1b 0 2024-03-22T10:42:52.225 INFO:tasks.workunit.client.0.smithi115.stdout:9/751: write d6/d7/d20/d38/f56 [240680,100648] 0 2024-03-22T10:42:52.226 INFO:tasks.workunit.client.0.smithi115.stdout:8/924: mkdir d6/dd/d12/dc6/d13a 0 2024-03-22T10:42:52.228 INFO:tasks.workunit.client.0.smithi115.stdout:2/683: mknod da/d25/d3d/d7e/d90/dbf/cca 0 2024-03-22T10:42:52.228 INFO:tasks.workunit.client.0.smithi115.stdout:5/794: chown d2/df/d13/d8b/d93/da6/dbc/d50/c57 2343 1 2024-03-22T10:42:52.228 INFO:tasks.workunit.client.1.smithi150.stdout:8/393: truncate d0/d2/d46/f4e 990401 0 2024-03-22T10:42:52.229 INFO:tasks.workunit.client.1.smithi150.stdout:4/371: dwrite d0/f14 [0,4194304] 0 2024-03-22T10:42:52.230 INFO:tasks.workunit.client.0.smithi115.stdout:1/815: dread d0/d8/de/d1a/f32 [0,4194304] 0 2024-03-22T10:42:52.234 INFO:tasks.workunit.client.0.smithi115.stdout:6/689: mknod d4/d11/d12/d1c/d8a/d98/d9f/dc3/cda 0 2024-03-22T10:42:52.235 INFO:tasks.workunit.client.1.smithi150.stdout:6/330: dwrite db/dd/d25/f6d [0,4194304] 0 2024-03-22T10:42:52.235 INFO:tasks.workunit.client.1.smithi150.stdout:8/394: mknod d0/d5d/c87 0 2024-03-22T10:42:52.237 INFO:tasks.workunit.client.0.smithi115.stdout:7/780: dread d2/d7/d12/d10/d19/d42/fca [0,4194304] 0 2024-03-22T10:42:52.243 INFO:tasks.workunit.client.0.smithi115.stdout:5/795: stat d2/df/cb4 0 2024-03-22T10:42:52.244 INFO:tasks.workunit.client.1.smithi150.stdout:6/331: getdents db/dd/d15/d57 0 2024-03-22T10:42:52.244 INFO:tasks.workunit.client.1.smithi150.stdout:6/332: fsync db/dd/d15/f77 0 2024-03-22T10:42:52.250 INFO:tasks.workunit.client.0.smithi115.stdout:1/816: symlink d0/d8/db/d8e/l11f 0 2024-03-22T10:42:52.253 INFO:tasks.workunit.client.0.smithi115.stdout:7/781: dread - d2/d7/d12/d10/d19/d42/d46/fb5 zero size 2024-03-22T10:42:52.254 INFO:tasks.workunit.client.0.smithi115.stdout:5/796: fdatasync d2/d15/ff3 0 2024-03-22T10:42:52.267 INFO:tasks.workunit.client.0.smithi115.stdout:6/690: rename d4/d11/d12/d1c/d8a to d4/d11/d12/d1c/d8a/ddb 22 2024-03-22T10:42:52.270 INFO:tasks.workunit.client.0.smithi115.stdout:4/732: write d1/d16/d35/d40/d5c/d14/f1b [6237391,120403] 0 2024-03-22T10:42:52.277 INFO:tasks.workunit.client.0.smithi115.stdout:8/925: dwrite d6/dd/d12/f125 [0,4194304] 0 2024-03-22T10:42:52.282 INFO:tasks.workunit.client.0.smithi115.stdout:0/677: dread d3/d4/d14/f15 [0,4194304] 0 2024-03-22T10:42:52.283 INFO:tasks.workunit.client.0.smithi115.stdout:3/755: dwrite d3/d16/d29/f32 [0,4194304] 0 2024-03-22T10:42:52.290 INFO:tasks.workunit.client.1.smithi150.stdout:6/333: rename db/l16 to db/dd/d15/d60/l78 0 2024-03-22T10:42:52.293 INFO:tasks.workunit.client.1.smithi150.stdout:6/334: mkdir db/d52/d79 0 2024-03-22T10:42:52.294 INFO:tasks.workunit.client.0.smithi115.stdout:7/782: creat d2/d7/d12/d10/d23/d3d/da4/f10d x:0 0 0 2024-03-22T10:42:52.294 INFO:tasks.workunit.client.0.smithi115.stdout:9/752: dread d6/d7/d20/d22/d23/f83 [0,4194304] 0 2024-03-22T10:42:52.294 INFO:tasks.workunit.client.0.smithi115.stdout:5/797: dwrite d2/f4 [8388608,4194304] 0 2024-03-22T10:42:52.296 INFO:tasks.workunit.client.0.smithi115.stdout:2/684: dread da/d25/d30/f6e [0,4194304] 0 2024-03-22T10:42:52.297 INFO:tasks.workunit.client.0.smithi115.stdout:6/691: truncate d4/d11/d12/d1c/fa5 4503116 0 2024-03-22T10:42:52.300 INFO:tasks.workunit.client.1.smithi150.stdout:6/335: link db/dd/d15/d28/c73 db/dd/d2e/c7a 0 2024-03-22T10:42:52.301 INFO:tasks.workunit.client.0.smithi115.stdout:2/685: stat da/d1c/d67/d78/f95 0 2024-03-22T10:42:52.302 INFO:tasks.workunit.client.0.smithi115.stdout:6/692: chown d4/d11/d12/d1c/f7f 112187435 1 2024-03-22T10:42:52.303 INFO:tasks.workunit.client.0.smithi115.stdout:2/686: fdatasync da/d25/d3d/d57/f65 0 2024-03-22T10:42:52.304 INFO:tasks.workunit.client.0.smithi115.stdout:7/783: rmdir d2/d7/d12/d10/d19/dbd/dce 39 2024-03-22T10:42:52.307 INFO:tasks.workunit.client.1.smithi150.stdout:6/336: creat db/dd/d15/f7b x:0 0 0 2024-03-22T10:42:52.312 INFO:tasks.workunit.client.0.smithi115.stdout:5/798: mkdir d2/df/d13/d68/d106 0 2024-03-22T10:42:52.313 INFO:tasks.workunit.client.0.smithi115.stdout:6/693: write d4/d11/d12/d1c/d8a/fa6 [1175691,12063] 0 2024-03-22T10:42:52.318 INFO:tasks.workunit.client.1.smithi150.stdout:6/337: truncate db/dd/d15/d30/f51 2532738 0 2024-03-22T10:42:52.318 INFO:tasks.workunit.client.0.smithi115.stdout:4/733: getdents d1/d30/d4c/d86/d9f/d32 0 2024-03-22T10:42:52.319 INFO:tasks.workunit.client.1.smithi150.stdout:6/338: write db/dd/f33 [192790,3076] 0 2024-03-22T10:42:52.321 INFO:tasks.workunit.client.0.smithi115.stdout:7/784: fdatasync d2/d7/d12/d10/d23/d3d/dd2/f92 0 2024-03-22T10:42:52.325 INFO:tasks.workunit.client.1.smithi150.stdout:6/339: rename db/f55 to db/dd/d15/d30/f7c 0 2024-03-22T10:42:52.326 INFO:tasks.workunit.client.1.smithi150.stdout:6/340: stat db/dd/d25/f76 0 2024-03-22T10:42:52.326 INFO:tasks.workunit.client.0.smithi115.stdout:5/799: read d2/d15/d1b/f7f [178938,71774] 0 2024-03-22T10:42:52.326 INFO:tasks.workunit.client.0.smithi115.stdout:4/734: mkdir d1/d30/dfb 0 2024-03-22T10:42:52.327 INFO:tasks.workunit.client.0.smithi115.stdout:7/785: stat d2/d7/d12/d10/d23/d3d/dd2/da3/df3 0 2024-03-22T10:42:52.328 INFO:tasks.workunit.client.0.smithi115.stdout:4/735: chown d1/d16/df0 519736 1 2024-03-22T10:42:52.330 INFO:tasks.workunit.client.0.smithi115.stdout:1/817: write d0/d12/d17/f80 [13097738,1107] 0 2024-03-22T10:42:52.332 INFO:tasks.workunit.client.0.smithi115.stdout:2/687: dread da/d1c/f88 [0,4194304] 0 2024-03-22T10:42:52.332 INFO:tasks.workunit.client.0.smithi115.stdout:5/800: dread - d2/df/d13/d8b/d93/da6/dbc/d50/d96/fcc zero size 2024-03-22T10:42:52.333 INFO:tasks.workunit.client.0.smithi115.stdout:1/818: stat d0/d12/f30 0 2024-03-22T10:42:52.334 INFO:tasks.workunit.client.0.smithi115.stdout:7/786: readlink d2/d7/d12/d10/d19/l6c 0 2024-03-22T10:42:52.335 INFO:tasks.workunit.client.0.smithi115.stdout:0/678: dread d3/d7/d1b/d35/f7e [0,4194304] 0 2024-03-22T10:42:52.335 INFO:tasks.workunit.client.0.smithi115.stdout:4/736: write d1/d30/d4c/d86/d9f/d75/d99/da8/fd0 [347125,55582] 0 2024-03-22T10:42:52.335 INFO:tasks.workunit.client.0.smithi115.stdout:8/926: write d6/d49/d88/f8a [108058,13437] 0 2024-03-22T10:42:52.337 INFO:tasks.workunit.client.0.smithi115.stdout:3/756: write d3/d19/f4e [2970228,356] 0 2024-03-22T10:42:52.339 INFO:tasks.workunit.client.0.smithi115.stdout:2/688: chown da/f14 39405409 1 2024-03-22T10:42:52.340 INFO:tasks.workunit.client.1.smithi150.stdout:6/341: dwrite db/dd/d15/d57/f65 [0,4194304] 0 2024-03-22T10:42:52.343 INFO:tasks.workunit.client.1.smithi150.stdout:6/342: fsync db/dd/d15/f20 0 2024-03-22T10:42:52.343 INFO:tasks.workunit.client.0.smithi115.stdout:5/801: read d2/d15/d1b/d7c/d38/d4a/f75 [404804,72532] 0 2024-03-22T10:42:52.345 INFO:tasks.workunit.client.0.smithi115.stdout:7/787: chown d2/d7/d12/d10/d23/d3d/ff0 113528913 1 2024-03-22T10:42:52.345 INFO:tasks.workunit.client.1.smithi150.stdout:6/343: stat db/dd/d15/d60/c24 0 2024-03-22T10:42:52.346 INFO:tasks.workunit.client.0.smithi115.stdout:4/737: creat d1/d16/df0/ffc x:0 0 0 2024-03-22T10:42:52.347 INFO:tasks.workunit.client.0.smithi115.stdout:8/927: dread - d6/dd/d24/d4a/d7b/d8f/da0/fc5 zero size 2024-03-22T10:42:52.348 INFO:tasks.workunit.client.1.smithi150.stdout:6/344: creat db/dd/d15/d57/f7d x:0 0 0 2024-03-22T10:42:52.349 INFO:tasks.workunit.client.1.smithi150.stdout:6/345: stat db/dd/d25 0 2024-03-22T10:42:52.349 INFO:tasks.workunit.client.0.smithi115.stdout:1/819: unlink d0/d12/d17/d37/d9e/de9/dec/ded/f115 0 2024-03-22T10:42:52.351 INFO:tasks.workunit.client.1.smithi150.stdout:6/346: truncate db/dd/f5b 467567 0 2024-03-22T10:42:52.351 INFO:tasks.workunit.client.0.smithi115.stdout:3/757: truncate d3/d36/d41/d33/fe2 883834 0 2024-03-22T10:42:52.353 INFO:tasks.workunit.client.1.smithi150.stdout:6/347: write db/dd/d15/f77 [810819,82848] 0 2024-03-22T10:42:52.354 INFO:tasks.workunit.client.0.smithi115.stdout:2/689: creat da/d25/d3d/d7e/d90/d96/fcb x:0 0 0 2024-03-22T10:42:52.354 INFO:tasks.workunit.client.1.smithi150.stdout:2/406: sync 2024-03-22T10:42:52.354 INFO:tasks.workunit.client.1.smithi150.stdout:3/346: sync 2024-03-22T10:42:52.354 INFO:tasks.workunit.client.1.smithi150.stdout:5/262: sync 2024-03-22T10:42:52.354 INFO:tasks.workunit.client.1.smithi150.stdout:7/358: sync 2024-03-22T10:42:52.355 INFO:tasks.workunit.client.1.smithi150.stdout:1/313: sync 2024-03-22T10:42:52.356 INFO:tasks.workunit.client.0.smithi115.stdout:5/802: mknod d2/d15/d1b/c107 0 2024-03-22T10:42:52.357 INFO:tasks.workunit.client.0.smithi115.stdout:6/694: dwrite d4/d36/d48/f9e [0,4194304] 0 2024-03-22T10:42:52.358 INFO:tasks.workunit.client.0.smithi115.stdout:1/820: mkdir d0/d8/de/d1a/d120 0 2024-03-22T10:42:52.358 INFO:tasks.workunit.client.1.smithi150.stdout:1/314: write d1/da/f42 [800565,32054] 0 2024-03-22T10:42:52.359 INFO:tasks.workunit.client.1.smithi150.stdout:2/407: chown d12/d19/d2d/f53 10742 1 2024-03-22T10:42:52.361 INFO:tasks.workunit.client.1.smithi150.stdout:3/347: unlink d5/d22/d2f/d38/d3e/d5b/f69 0 2024-03-22T10:42:52.362 INFO:tasks.workunit.client.0.smithi115.stdout:8/928: read d6/dd/d24/d4a/f5f [2496717,53574] 0 2024-03-22T10:42:52.362 INFO:tasks.workunit.client.0.smithi115.stdout:0/679: rename d3/d7/d1b/d9b/cb8 to d3/d7/d1b/d35/cdd 0 2024-03-22T10:42:52.363 INFO:tasks.workunit.client.1.smithi150.stdout:2/408: write d12/d48/f4c [1084379,118628] 0 2024-03-22T10:42:52.363 INFO:tasks.workunit.client.0.smithi115.stdout:7/788: mkdir d2/d7/d12/d10/d19/d10e 0 2024-03-22T10:42:52.363 INFO:tasks.workunit.client.0.smithi115.stdout:4/738: fdatasync d1/d16/d35/d40/d5c/f25 0 2024-03-22T10:42:52.364 INFO:tasks.workunit.client.0.smithi115.stdout:1/821: mknod d0/d8/de/d1a/c121 0 2024-03-22T10:42:52.367 INFO:tasks.workunit.client.0.smithi115.stdout:5/803: symlink d2/df/d13/d8b/d93/da6/dbc/d7a/l108 0 2024-03-22T10:42:52.370 INFO:tasks.workunit.client.0.smithi115.stdout:0/680: read d3/d7/d1b/d35/f42 [1721656,50574] 0 2024-03-22T10:42:52.370 INFO:tasks.workunit.client.0.smithi115.stdout:4/739: dread - d1/d30/fd2 zero size 2024-03-22T10:42:52.370 INFO:tasks.workunit.client.1.smithi150.stdout:5/263: rename d0/d26 to d0/d8/dc/de/d14/d4f 0 2024-03-22T10:42:52.373 INFO:tasks.workunit.client.1.smithi150.stdout:5/264: truncate d0/f4c 669714 0 2024-03-22T10:42:52.374 INFO:tasks.workunit.client.0.smithi115.stdout:6/695: creat d4/d36/d48/d72/fdc x:0 0 0 2024-03-22T10:42:52.377 INFO:tasks.workunit.client.0.smithi115.stdout:7/789: write d2/d7/d12/d10/f25 [4231554,60809] 0 2024-03-22T10:42:52.382 INFO:tasks.workunit.client.0.smithi115.stdout:2/690: link da/d25/d3d/dad/db3/fbd da/d1c/d67/d78/d86/fcc 0 2024-03-22T10:42:52.386 INFO:tasks.workunit.client.0.smithi115.stdout:4/740: mkdir d1/d30/d4c/d86/d9f/d73/dfd 0 2024-03-22T10:42:52.386 INFO:tasks.workunit.client.0.smithi115.stdout:1/822: mknod d0/d12/d17/d37/d9e/da0/dff/c122 0 2024-03-22T10:42:52.388 INFO:tasks.workunit.client.0.smithi115.stdout:4/741: readlink d1/dbb/dc6/lf2 0 2024-03-22T10:42:52.392 INFO:tasks.workunit.client.1.smithi150.stdout:1/315: dwrite d1/d19/d1a/f49 [0,4194304] 0 2024-03-22T10:42:52.392 INFO:tasks.workunit.client.1.smithi150.stdout:3/348: dwrite d5/f52 [0,4194304] 0 2024-03-22T10:42:52.393 INFO:tasks.workunit.client.0.smithi115.stdout:3/758: dwrite d3/d16/f58 [0,4194304] 0 2024-03-22T10:42:52.394 INFO:tasks.workunit.client.0.smithi115.stdout:0/681: symlink d3/d7/d1b/d1f/lde 0 2024-03-22T10:42:52.394 INFO:tasks.workunit.client.0.smithi115.stdout:2/691: rename da/d7b to da/d25/d30/dcd 0 2024-03-22T10:42:52.397 INFO:tasks.workunit.client.1.smithi150.stdout:2/409: dwrite d12/d19/d1a/f7c [0,4194304] 0 2024-03-22T10:42:52.400 INFO:tasks.workunit.client.0.smithi115.stdout:4/742: truncate d1/d16/d35/d40/d5c/d55/fef 150964 0 2024-03-22T10:42:52.400 INFO:tasks.workunit.client.1.smithi150.stdout:9/423: sync 2024-03-22T10:42:52.400 INFO:tasks.workunit.client.1.smithi150.stdout:0/388: sync 2024-03-22T10:42:52.401 INFO:tasks.workunit.client.0.smithi115.stdout:6/696: mknod d4/d11/d12/d1c/d84/cdd 0 2024-03-22T10:42:52.405 INFO:tasks.workunit.client.0.smithi115.stdout:6/697: chown d4/d13/d33/f42 1091304 1 2024-03-22T10:42:52.406 INFO:tasks.workunit.client.1.smithi150.stdout:1/316: read d1/d19/d1a/f1e [679693,109056] 0 2024-03-22T10:42:52.406 INFO:tasks.workunit.client.0.smithi115.stdout:1/823: mknod d0/d8/de/d1a/d91/dc7/c123 0 2024-03-22T10:42:52.408 INFO:tasks.workunit.client.1.smithi150.stdout:2/410: chown d12/d19/l20 386593777 1 2024-03-22T10:42:52.409 INFO:tasks.workunit.client.0.smithi115.stdout:2/692: creat da/d1c/d67/fce x:0 0 0 2024-03-22T10:42:52.409 INFO:tasks.workunit.client.0.smithi115.stdout:0/682: rmdir d3/d7/d1b/d93 39 2024-03-22T10:42:52.411 INFO:tasks.workunit.client.0.smithi115.stdout:6/698: readlink d4/d13/d33/dbe/lcd 0 2024-03-22T10:42:52.411 INFO:tasks.workunit.client.1.smithi150.stdout:9/424: symlink d1/l9a 0 2024-03-22T10:42:52.416 INFO:tasks.workunit.client.0.smithi115.stdout:4/743: creat d1/d66/d82/ffe x:0 0 0 2024-03-22T10:42:52.416 INFO:tasks.workunit.client.0.smithi115.stdout:5/804: dread d2/df/fdf [0,4194304] 0 2024-03-22T10:42:52.417 INFO:tasks.workunit.client.0.smithi115.stdout:1/824: fdatasync d0/d12/d17/d37/d9e/de9/f8d 0 2024-03-22T10:42:52.417 INFO:tasks.workunit.client.1.smithi150.stdout:6/348: dread db/dd/d15/d28/f58 [0,4194304] 0 2024-03-22T10:42:52.418 INFO:tasks.workunit.client.1.smithi150.stdout:0/389: creat d0/d12/d4f/d5a/f7c x:0 0 0 2024-03-22T10:42:52.419 INFO:tasks.workunit.client.1.smithi150.stdout:7/359: dread d2/d5/d7/d8/d46/f66 [0,4194304] 0 2024-03-22T10:42:52.420 INFO:tasks.workunit.client.0.smithi115.stdout:8/929: dwrite d6/dd/fe4 [0,4194304] 0 2024-03-22T10:42:52.422 INFO:tasks.workunit.client.1.smithi150.stdout:1/317: rmdir d1/da/d15/d18/d4b 39 2024-03-22T10:42:52.423 INFO:tasks.workunit.client.0.smithi115.stdout:0/683: mknod d3/da/d4c/d74/d84/dd7/cdf 0 2024-03-22T10:42:52.427 INFO:tasks.workunit.client.0.smithi115.stdout:3/759: link d3/d82/d57/caf d3/d19/d61/daa/c103 0 2024-03-22T10:42:52.428 INFO:tasks.workunit.client.0.smithi115.stdout:5/805: mknod d2/d15/d1b/d7c/d38/d43/d49/d81/d83/c109 0 2024-03-22T10:42:52.428 INFO:tasks.workunit.client.1.smithi150.stdout:9/425: creat d1/d8/de/df/d20/d29/f9b x:0 0 0 2024-03-22T10:42:52.429 INFO:tasks.workunit.client.0.smithi115.stdout:1/825: mknod d0/d8/de/d1a/d91/dc7/c124 0 2024-03-22T10:42:52.430 INFO:tasks.workunit.client.0.smithi115.stdout:1/826: fsync d0/f87 0 2024-03-22T10:42:52.431 INFO:tasks.workunit.client.1.smithi150.stdout:6/349: chown db/dd/l38 0 1 2024-03-22T10:42:52.431 INFO:tasks.workunit.client.1.smithi150.stdout:1/318: rmdir d1/da/d15/d4f 39 2024-03-22T10:42:52.432 INFO:tasks.workunit.client.0.smithi115.stdout:0/684: unlink d3/d52/f59 0 2024-03-22T10:42:52.432 INFO:tasks.workunit.client.0.smithi115.stdout:3/760: write d3/d82/d52/fe3 [23488,74839] 0 2024-03-22T10:42:52.432 INFO:tasks.workunit.client.0.smithi115.stdout:5/806: write d2/df/d3b/f2a [213771,126323] 0 2024-03-22T10:42:52.437 INFO:tasks.workunit.client.0.smithi115.stdout:4/744: creat d1/dbb/ddf/df9/fff x:0 0 0 2024-03-22T10:42:52.441 INFO:tasks.workunit.client.1.smithi150.stdout:0/390: rename d0/d1/l6c to d0/d12/d4f/d11/d66/l7d 0 2024-03-22T10:42:52.441 INFO:tasks.workunit.client.0.smithi115.stdout:8/930: rename d6/dd/d12/d31/l12a to d6/d49/l13b 0 2024-03-22T10:42:52.443 INFO:tasks.workunit.client.0.smithi115.stdout:5/807: unlink d2/d15/d1b/d7c/d38/d43/fce 0 2024-03-22T10:42:52.445 INFO:tasks.workunit.client.0.smithi115.stdout:0/685: mkdir d3/d7/d1b/d1f/d5b/dba/dca/de0 0 2024-03-22T10:42:52.446 INFO:tasks.workunit.client.1.smithi150.stdout:0/391: write d0/f63 [667899,19909] 0 2024-03-22T10:42:52.448 INFO:tasks.workunit.client.0.smithi115.stdout:4/745: creat d1/dbb/dc6/dd8/df7/f100 x:0 0 0 2024-03-22T10:42:52.448 INFO:tasks.workunit.client.1.smithi150.stdout:1/319: link d1/f3a d1/d19/d57/f62 0 2024-03-22T10:42:52.450 INFO:tasks.workunit.client.1.smithi150.stdout:2/411: dwrite d12/d48/f79 [0,4194304] 0 2024-03-22T10:42:52.451 INFO:tasks.workunit.client.0.smithi115.stdout:8/931: unlink d6/d49/fa8 0 2024-03-22T10:42:52.452 INFO:tasks.workunit.client.0.smithi115.stdout:3/761: mkdir d3/d82/d54/de7/d104 0 2024-03-22T10:42:52.454 INFO:tasks.workunit.client.1.smithi150.stdout:9/426: link d1/d8/de/df/d24/c2d d1/d8/de/df/d20/d32/c9c 0 2024-03-22T10:42:52.460 INFO:tasks.workunit.client.1.smithi150.stdout:7/360: dwrite d2/d5/d7/f4d [0,4194304] 0 2024-03-22T10:42:52.460 INFO:tasks.workunit.client.1.smithi150.stdout:2/412: creat d12/d16/d61/f86 x:0 0 0 2024-03-22T10:42:52.460 INFO:tasks.workunit.client.0.smithi115.stdout:4/746: mknod d1/d30/d4c/d86/d9f/d75/d99/d98/c101 0 2024-03-22T10:42:52.460 INFO:tasks.workunit.client.1.smithi150.stdout:5/265: dread d0/d8/dc/de/d14/f25 [0,4194304] 0 2024-03-22T10:42:52.465 INFO:tasks.workunit.client.0.smithi115.stdout:5/808: dwrite d2/df/d13/d8b/d93/da6/dbc/d7a/f98 [0,4194304] 0 2024-03-22T10:42:52.466 INFO:tasks.workunit.client.1.smithi150.stdout:2/413: symlink d12/d48/l87 0 2024-03-22T10:42:52.466 INFO:tasks.workunit.client.1.smithi150.stdout:1/320: getdents d1/d19/d50 0 2024-03-22T10:42:52.468 INFO:tasks.workunit.client.0.smithi115.stdout:4/747: mkdir d1/dbb/d102 0 2024-03-22T10:42:52.473 INFO:tasks.workunit.client.1.smithi150.stdout:7/361: unlink d2/d5/d7/d8/d2c/l35 0 2024-03-22T10:42:52.473 INFO:tasks.workunit.client.1.smithi150.stdout:9/427: stat d1/d8/de/df/d24/c2d 0 2024-03-22T10:42:52.473 INFO:tasks.workunit.client.1.smithi150.stdout:4/372: sync 2024-03-22T10:42:52.473 INFO:tasks.workunit.client.1.smithi150.stdout:8/395: sync 2024-03-22T10:42:52.473 INFO:tasks.workunit.client.0.smithi115.stdout:8/932: dwrite d6/dd/d24/d3a/f54 [0,4194304] 0 2024-03-22T10:42:52.477 INFO:tasks.workunit.client.1.smithi150.stdout:2/414: mknod d12/d19/d2d/d66/c88 0 2024-03-22T10:42:52.478 INFO:tasks.workunit.client.1.smithi150.stdout:4/373: chown d0/d1b/f2c 3 1 2024-03-22T10:42:52.484 INFO:tasks.workunit.client.0.smithi115.stdout:0/686: dwrite d3/d7/d1b/d35/f80 [0,4194304] 0 2024-03-22T10:42:52.485 INFO:tasks.workunit.client.1.smithi150.stdout:5/266: fdatasync d0/d8/dc/de/d14/d2e/d45/f2a 0 2024-03-22T10:42:52.489 INFO:tasks.workunit.client.0.smithi115.stdout:4/748: read d1/d66/f77 [1296665,76891] 0 2024-03-22T10:42:52.489 INFO:tasks.workunit.client.1.smithi150.stdout:6/350: write db/dd/d15/d60/f32 [2557780,59957] 0 2024-03-22T10:42:52.490 INFO:tasks.workunit.client.1.smithi150.stdout:7/362: mknod d2/d5/d7/d9/db/c77 0 2024-03-22T10:42:52.496 INFO:tasks.workunit.client.1.smithi150.stdout:5/267: write d0/f3 [5368985,109522] 0 2024-03-22T10:42:52.498 INFO:tasks.workunit.client.0.smithi115.stdout:5/809: getdents d2/d15/d1b/d7c/d38/d43 0 2024-03-22T10:42:52.499 INFO:tasks.workunit.client.0.smithi115.stdout:0/687: truncate d3/da/d4c/f91 1237252 0 2024-03-22T10:42:52.501 INFO:tasks.workunit.client.1.smithi150.stdout:1/321: symlink d1/l63 0 2024-03-22T10:42:52.501 INFO:tasks.workunit.client.1.smithi150.stdout:2/415: dread d12/d16/d67/d3c/f6b [0,4194304] 0 2024-03-22T10:42:52.501 INFO:tasks.workunit.client.0.smithi115.stdout:4/749: read d1/d16/d35/d40/d5c/f6f [289736,14475] 0 2024-03-22T10:42:52.502 INFO:tasks.workunit.client.1.smithi150.stdout:2/416: chown d12/d35 7 1 2024-03-22T10:42:52.503 INFO:tasks.workunit.client.0.smithi115.stdout:8/933: rename d6/dd/d12/dc6 to d6/dd/d12/d13c 0 2024-03-22T10:42:52.503 INFO:tasks.workunit.client.1.smithi150.stdout:1/322: chown d1/da/d15/d18/l33 945 1 2024-03-22T10:42:52.505 INFO:tasks.workunit.client.1.smithi150.stdout:7/363: stat d2/d5/d7/d8/d46/f5a 0 2024-03-22T10:42:52.505 INFO:tasks.workunit.client.1.smithi150.stdout:1/323: stat d1/da/d15/c17 0 2024-03-22T10:42:52.506 INFO:tasks.workunit.client.1.smithi150.stdout:7/364: dread - d2/f67 zero size 2024-03-22T10:42:52.507 INFO:tasks.workunit.client.0.smithi115.stdout:5/810: dread - d2/df/d13/d68/f6c zero size 2024-03-22T10:42:52.507 INFO:tasks.workunit.client.0.smithi115.stdout:6/699: dread d4/d13/d33/f42 [4194304,4194304] 0 2024-03-22T10:42:52.509 INFO:tasks.workunit.client.0.smithi115.stdout:1/827: dwrite d0/d8/d4d/f5a [0,4194304] 0 2024-03-22T10:42:52.509 INFO:tasks.workunit.client.0.smithi115.stdout:0/688: write d3/d4/d6d/f9c [4350320,97563] 0 2024-03-22T10:42:52.509 INFO:tasks.workunit.client.0.smithi115.stdout:8/934: fdatasync d6/dd/d24/f9d 0 2024-03-22T10:42:52.510 INFO:tasks.workunit.client.1.smithi150.stdout:3/349: dread d5/fd [0,4194304] 0 2024-03-22T10:42:52.512 INFO:tasks.workunit.client.0.smithi115.stdout:4/750: fsync d1/d30/d4c/d86/d9f/d75/d99/d98/d9b/fa0 0 2024-03-22T10:42:52.512 INFO:tasks.workunit.client.0.smithi115.stdout:3/762: write d3/d82/f22 [6215623,75312] 0 2024-03-22T10:42:52.512 INFO:tasks.workunit.client.1.smithi150.stdout:2/417: rename lc to d12/d48/d63/l89 0 2024-03-22T10:42:52.514 INFO:tasks.workunit.client.0.smithi115.stdout:6/700: mknod d4/d13/d33/d58/cde 0 2024-03-22T10:42:52.515 INFO:tasks.workunit.client.1.smithi150.stdout:2/418: write d12/d19/d62/f6f [976759,9872] 0 2024-03-22T10:42:52.516 INFO:tasks.workunit.client.1.smithi150.stdout:0/392: dread d0/d12/d4f/d11/d26/f2e [0,4194304] 0 2024-03-22T10:42:52.520 INFO:tasks.workunit.client.0.smithi115.stdout:6/701: chown d4/d5e/f97 43 1 2024-03-22T10:42:52.520 INFO:tasks.workunit.client.1.smithi150.stdout:5/268: link d0/d8/dc/de/d14/f16 d0/d8/dc/de/d14/f50 0 2024-03-22T10:42:52.521 INFO:tasks.workunit.client.0.smithi115.stdout:6/702: readlink d4/d5e/l85 0 2024-03-22T10:42:52.521 INFO:tasks.workunit.client.1.smithi150.stdout:0/393: dread - d0/d12/f58 zero size 2024-03-22T10:42:52.522 INFO:tasks.workunit.client.0.smithi115.stdout:0/689: creat d3/da/d4c/d69/fe1 x:0 0 0 2024-03-22T10:42:52.524 INFO:tasks.workunit.client.0.smithi115.stdout:0/690: fdatasync d3/d4/d6d/f9e 0 2024-03-22T10:42:52.525 INFO:tasks.workunit.client.0.smithi115.stdout:1/828: mkdir d0/d8/de/d1a/d91/dc7/d125 0 2024-03-22T10:42:52.525 INFO:tasks.workunit.client.1.smithi150.stdout:7/365: rmdir d2/d6e 39 2024-03-22T10:42:52.526 INFO:tasks.workunit.client.1.smithi150.stdout:0/394: dread - d0/d12/d4f/d42/d10/d53/f5e zero size 2024-03-22T10:42:52.527 INFO:tasks.workunit.client.0.smithi115.stdout:4/751: mkdir d1/d30/d4c/d86/dde/d103 0 2024-03-22T10:42:52.527 INFO:tasks.workunit.client.0.smithi115.stdout:6/703: symlink d4/d11/d12/d1c/d8a/d98/ldf 0 2024-03-22T10:42:52.531 INFO:tasks.workunit.client.1.smithi150.stdout:1/324: symlink d1/da/d15/d4f/l64 0 2024-03-22T10:42:52.532 INFO:tasks.workunit.client.0.smithi115.stdout:0/691: chown d3/da/d4c/d74/d84/dd7/fbf 11 1 2024-03-22T10:42:52.535 INFO:tasks.workunit.client.1.smithi150.stdout:0/395: read - d0/d12/d4f/d42/d10/d53/f5e zero size 2024-03-22T10:42:52.535 INFO:tasks.workunit.client.1.smithi150.stdout:6/351: dwrite db/dd/d25/f45 [0,4194304] 0 2024-03-22T10:42:52.535 INFO:tasks.workunit.client.1.smithi150.stdout:6/352: chown l8 769503 1 2024-03-22T10:42:52.536 INFO:tasks.workunit.client.1.smithi150.stdout:1/325: chown d1/f29 352040 1 2024-03-22T10:42:52.537 INFO:tasks.workunit.client.1.smithi150.stdout:3/350: dread - d5/f3c zero size 2024-03-22T10:42:52.537 INFO:tasks.workunit.client.1.smithi150.stdout:0/396: chown d0/d12/d4f/d42/d10/d53/f5e 711 1 2024-03-22T10:42:52.538 INFO:tasks.workunit.client.0.smithi115.stdout:0/692: symlink d3/da/d4c/d74/d84/le2 0 2024-03-22T10:42:52.540 INFO:tasks.workunit.client.0.smithi115.stdout:5/811: dread d2/df/d3b/f2a [0,4194304] 0 2024-03-22T10:42:52.541 INFO:tasks.workunit.client.1.smithi150.stdout:2/419: mknod d12/d16/d67/d3c/d75/c8a 0 2024-03-22T10:42:52.543 INFO:tasks.workunit.client.1.smithi150.stdout:2/420: write d12/d16/f42 [4325344,80050] 0 2024-03-22T10:42:52.551 INFO:tasks.workunit.client.0.smithi115.stdout:3/763: dread d3/d82/d54/f59 [0,4194304] 0 2024-03-22T10:42:52.552 INFO:tasks.workunit.client.1.smithi150.stdout:5/269: dwrite d0/d8/dc/f12 [0,4194304] 0 2024-03-22T10:42:52.554 INFO:tasks.workunit.client.0.smithi115.stdout:4/752: dwrite d1/d16/d35/d40/d54/f67 [0,4194304] 0 2024-03-22T10:42:52.556 INFO:tasks.workunit.client.0.smithi115.stdout:3/764: write d3/d36/d41/fde [20850,8103] 0 2024-03-22T10:42:52.558 INFO:tasks.workunit.client.0.smithi115.stdout:1/829: write d0/d8/de/d1a/d91/da3/fdb [543063,6245] 0 2024-03-22T10:42:52.558 INFO:tasks.workunit.client.0.smithi115.stdout:3/765: chown d3/f50 1778252 1 2024-03-22T10:42:52.559 INFO:tasks.workunit.client.1.smithi150.stdout:6/353: mknod db/dd/d15/d70/c7e 0 2024-03-22T10:42:52.561 INFO:tasks.workunit.client.0.smithi115.stdout:4/753: unlink d1/d16/l7b 0 2024-03-22T10:42:52.568 INFO:tasks.workunit.client.0.smithi115.stdout:3/766: rename d3/d82/d54/cf1 to d3/d19/df5/c105 0 2024-03-22T10:42:52.569 INFO:tasks.workunit.client.0.smithi115.stdout:4/754: symlink d1/d30/l104 0 2024-03-22T10:42:52.572 INFO:tasks.workunit.client.0.smithi115.stdout:0/693: dwrite d3/d7/d1c/f56 [0,4194304] 0 2024-03-22T10:42:52.574 INFO:tasks.workunit.client.1.smithi150.stdout:3/351: mknod d5/da/c6e 0 2024-03-22T10:42:52.580 INFO:tasks.workunit.client.0.smithi115.stdout:4/755: truncate d1/d16/d68/f9e 533447 0 2024-03-22T10:42:52.581 INFO:tasks.workunit.client.0.smithi115.stdout:1/830: link d0/d8/d4d/d56/db6/ddc/la6 d0/d12/d17/d37/d5c/d10a/l126 0 2024-03-22T10:42:52.587 INFO:tasks.workunit.client.0.smithi115.stdout:4/756: creat d1/d30/f105 x:0 0 0 2024-03-22T10:42:52.587 INFO:tasks.workunit.client.0.smithi115.stdout:5/812: dwrite d2/df/d3b/f7b [0,4194304] 0 2024-03-22T10:42:52.588 INFO:tasks.workunit.client.1.smithi150.stdout:5/270: write d0/d8/dc/de/d14/f50 [795300,72892] 0 2024-03-22T10:42:52.589 INFO:tasks.workunit.client.1.smithi150.stdout:6/354: write db/dd/d15/d30/f4b [6900491,2298] 0 2024-03-22T10:42:52.591 INFO:tasks.workunit.client.1.smithi150.stdout:3/352: readlink d5/l6 0 2024-03-22T10:42:52.592 INFO:tasks.workunit.client.1.smithi150.stdout:9/428: write d1/d8/de/df/d24/d58/f91 [106937,59002] 0 2024-03-22T10:42:52.592 INFO:tasks.workunit.client.0.smithi115.stdout:5/813: write d2/d15/d1b/d7c/d38/d43/d49/f52 [3906798,89492] 0 2024-03-22T10:42:52.595 INFO:tasks.workunit.client.0.smithi115.stdout:5/814: dread - d2/d15/d1b/d7c/d38/d43/fca zero size 2024-03-22T10:42:52.597 INFO:tasks.workunit.client.0.smithi115.stdout:5/815: dread - d2/d15/f104 zero size 2024-03-22T10:42:52.601 INFO:tasks.workunit.client.1.smithi150.stdout:8/396: dwrite d0/d2/f14 [0,4194304] 0 2024-03-22T10:42:52.605 INFO:tasks.workunit.client.1.smithi150.stdout:8/397: readlink d0/d2/d19/d29/d69/l81 0 2024-03-22T10:42:52.606 INFO:tasks.workunit.client.1.smithi150.stdout:9/429: stat d1/d19/c1d 0 2024-03-22T10:42:52.607 INFO:tasks.workunit.client.1.smithi150.stdout:3/353: creat d5/d22/d2f/d38/d3e/f6f x:0 0 0 2024-03-22T10:42:52.608 INFO:tasks.workunit.client.0.smithi115.stdout:4/757: dread d1/d16/d35/d40/d54/f7a [0,4194304] 0 2024-03-22T10:42:52.614 INFO:tasks.workunit.client.0.smithi115.stdout:3/767: dwrite d3/f53 [0,4194304] 0 2024-03-22T10:42:52.615 INFO:tasks.workunit.client.1.smithi150.stdout:7/366: truncate d2/d5/d7/f50 1255094 0 2024-03-22T10:42:52.615 INFO:tasks.workunit.client.0.smithi115.stdout:0/694: write d3/da/d4c/d69/d6a/f86 [1226157,28960] 0 2024-03-22T10:42:52.616 INFO:tasks.workunit.client.1.smithi150.stdout:9/430: mknod d1/d8/de/d73/c9d 0 2024-03-22T10:42:52.616 INFO:tasks.workunit.client.1.smithi150.stdout:6/355: rename db/dd/d15/d60/c54 to db/dd/d15/d60/d50/c7f 0 2024-03-22T10:42:52.618 INFO:tasks.workunit.client.0.smithi115.stdout:3/768: mknod d3/d36/d41/c106 0 2024-03-22T10:42:52.620 INFO:tasks.workunit.client.1.smithi150.stdout:2/421: dwrite d12/f26 [0,4194304] 0 2024-03-22T10:42:52.622 INFO:tasks.workunit.client.0.smithi115.stdout:0/695: creat d3/d7/d1b/d1f/d78/dd9/fe3 x:0 0 0 2024-03-22T10:42:52.623 INFO:tasks.workunit.client.1.smithi150.stdout:3/354: truncate d5/d22/d2f/d38/d3e/f4d 218940 0 2024-03-22T10:42:52.626 INFO:tasks.workunit.client.0.smithi115.stdout:1/831: dwrite d0/d12/d17/f39 [0,4194304] 0 2024-03-22T10:42:52.627 INFO:tasks.workunit.client.0.smithi115.stdout:3/769: rename d3/d19/d61 to d3/d6d/db4/d107 0 2024-03-22T10:42:52.627 INFO:tasks.workunit.client.1.smithi150.stdout:9/431: chown d1/d8/de/df/f4d 582082 1 2024-03-22T10:42:52.629 INFO:tasks.workunit.client.1.smithi150.stdout:3/355: chown d5/d22/c61 6991418 1 2024-03-22T10:42:52.629 INFO:tasks.workunit.client.1.smithi150.stdout:8/398: dwrite d0/d2/d19/d29/d2f/df/f42 [0,4194304] 0 2024-03-22T10:42:52.629 INFO:tasks.workunit.client.0.smithi115.stdout:0/696: getdents d3/d7/d1b/d1f/d78 0 2024-03-22T10:42:52.631 INFO:tasks.workunit.client.0.smithi115.stdout:4/758: dwrite d1/d16/df0/ffc [0,4194304] 0 2024-03-22T10:42:52.633 INFO:tasks.workunit.client.1.smithi150.stdout:9/432: write d1/d8/de/f17 [4208562,115548] 0 2024-03-22T10:42:52.634 INFO:tasks.workunit.client.0.smithi115.stdout:3/770: truncate d3/d89/dca/f88 18571 0 2024-03-22T10:42:52.636 INFO:tasks.workunit.client.0.smithi115.stdout:0/697: truncate d3/d7/d1c/d6c/d9a/fa5 1081404 0 2024-03-22T10:42:52.636 INFO:tasks.workunit.client.1.smithi150.stdout:8/399: write d0/f17 [5255577,105231] 0 2024-03-22T10:42:52.638 INFO:tasks.workunit.client.1.smithi150.stdout:9/433: write d1/f71 [4427446,115485] 0 2024-03-22T10:42:52.646 INFO:tasks.workunit.client.1.smithi150.stdout:7/367: dwrite d2/d5/d7/d8/d2c/f47 [0,4194304] 0 2024-03-22T10:42:52.651 INFO:tasks.workunit.client.0.smithi115.stdout:1/832: dwrite d0/d8/d4d/d56/db6/ddc/fa5 [0,4194304] 0 2024-03-22T10:42:52.654 INFO:tasks.workunit.client.0.smithi115.stdout:3/771: dwrite d3/d19/d2b/d49/fe4 [0,4194304] 0 2024-03-22T10:42:52.655 INFO:tasks.workunit.client.0.smithi115.stdout:1/833: write d0/d12/d17/d37/d5c/fb4 [1028951,123145] 0 2024-03-22T10:42:52.656 INFO:tasks.workunit.client.1.smithi150.stdout:4/374: sync 2024-03-22T10:42:52.657 INFO:tasks.workunit.client.0.smithi115.stdout:3/772: write d3/d36/d41/fa2 [3900914,28831] 0 2024-03-22T10:42:52.657 INFO:tasks.workunit.client.1.smithi150.stdout:6/356: dwrite db/dd/d15/f77 [0,4194304] 0 2024-03-22T10:42:52.665 INFO:tasks.workunit.client.1.smithi150.stdout:7/368: chown d2/d5/d7/d9/db/l3d 154794 1 2024-03-22T10:42:52.665 INFO:tasks.workunit.client.1.smithi150.stdout:2/422: link d12/d19/l24 d12/d35/l8b 0 2024-03-22T10:42:52.671 INFO:tasks.workunit.client.1.smithi150.stdout:2/423: chown d12/l5e 80 1 2024-03-22T10:42:52.671 INFO:tasks.workunit.client.0.smithi115.stdout:1/834: dwrite d0/d8/de/f26 [0,4194304] 0 2024-03-22T10:42:52.672 INFO:tasks.workunit.client.1.smithi150.stdout:7/369: chown d2/d5/d7/d8/d57/f5b 0 1 2024-03-22T10:42:52.673 INFO:tasks.workunit.client.1.smithi150.stdout:2/424: chown d12/d19/d62 371 1 2024-03-22T10:42:52.676 INFO:tasks.workunit.client.1.smithi150.stdout:9/434: dwrite d1/d19/f79 [0,4194304] 0 2024-03-22T10:42:52.676 INFO:tasks.workunit.client.1.smithi150.stdout:6/357: chown db/c4e 126085 1 2024-03-22T10:42:52.679 INFO:tasks.workunit.client.1.smithi150.stdout:1/326: write d1/d19/d57/f62 [2097601,14204] 0 2024-03-22T10:42:52.682 INFO:tasks.workunit.client.1.smithi150.stdout:9/435: stat d1/d8/de/df/d36/d40/f7f 0 2024-03-22T10:42:52.686 INFO:tasks.workunit.client.1.smithi150.stdout:9/436: write d1/d8/de/df/f1f [1729380,49951] 0 2024-03-22T10:42:52.687 INFO:tasks.workunit.client.1.smithi150.stdout:2/425: dread d12/d16/d67/f4e [0,4194304] 0 2024-03-22T10:42:52.692 INFO:tasks.workunit.client.1.smithi150.stdout:4/375: creat d0/d32/d7c/f82 x:0 0 0 2024-03-22T10:42:52.701 INFO:tasks.workunit.client.1.smithi150.stdout:2/426: write d12/d19/d2d/f82 [269191,112790] 0 2024-03-22T10:42:52.710 INFO:tasks.workunit.client.1.smithi150.stdout:6/358: dread db/dd/d25/f3d [0,4194304] 0 2024-03-22T10:42:52.713 INFO:tasks.workunit.client.1.smithi150.stdout:4/376: symlink d0/d2f/d63/d7b/l83 0 2024-03-22T10:42:52.715 INFO:tasks.workunit.client.1.smithi150.stdout:5/271: dwrite d0/d8/dc/de/d14/f50 [0,4194304] 0 2024-03-22T10:42:52.716 INFO:tasks.workunit.client.1.smithi150.stdout:1/327: dwrite d1/d19/d57/f5b [0,4194304] 0 2024-03-22T10:42:52.729 INFO:tasks.workunit.client.1.smithi150.stdout:1/328: mkdir d1/da/d2a/d65 0 2024-03-22T10:42:52.729 INFO:tasks.workunit.client.0.smithi115.stdout:0/698: rename d3/d7/d1b/d35/f42 to d3/d7/d6b/d94/fe4 0 2024-03-22T10:42:52.734 INFO:tasks.workunit.client.1.smithi150.stdout:6/359: link db/dd/f33 db/dd/d15/d60/f80 0 2024-03-22T10:42:52.738 INFO:tasks.workunit.client.1.smithi150.stdout:1/329: write d1/d19/d1a/f2b [2800654,30972] 0 2024-03-22T10:42:52.739 INFO:tasks.workunit.client.0.smithi115.stdout:4/759: fsync d1/d16/df0/ffc 0 2024-03-22T10:42:52.742 INFO:tasks.workunit.client.1.smithi150.stdout:9/437: rename d1/d8/de/df/d36/d40/f7f to d1/d8/de/f9e 0 2024-03-22T10:42:52.742 INFO:tasks.workunit.client.0.smithi115.stdout:1/835: mknod d0/d8/d4d/d56/dcf/c127 0 2024-03-22T10:42:52.743 INFO:tasks.workunit.client.0.smithi115.stdout:4/760: write d1/d66/d82/fa6 [1108503,43635] 0 2024-03-22T10:42:52.743 INFO:tasks.workunit.client.0.smithi115.stdout:0/699: mknod d3/d7/d1b/d1f/d78/dc1/ce5 0 2024-03-22T10:42:52.743 INFO:tasks.workunit.client.0.smithi115.stdout:3/773: fdatasync d3/d36/d41/f9a 0 2024-03-22T10:42:52.744 INFO:tasks.workunit.client.1.smithi150.stdout:5/272: dwrite d0/d8/dc/f12 [0,4194304] 0 2024-03-22T10:42:52.745 INFO:tasks.workunit.client.1.smithi150.stdout:1/330: creat d1/d19/d50/f66 x:0 0 0 2024-03-22T10:42:52.745 INFO:tasks.workunit.client.0.smithi115.stdout:4/761: stat d1/d16/d68 0 2024-03-22T10:42:52.750 INFO:tasks.workunit.client.0.smithi115.stdout:1/836: unlink d0/d8/de/d1a/d91/fcd 0 2024-03-22T10:42:52.750 INFO:tasks.workunit.client.0.smithi115.stdout:0/700: dread - d3/d7/d1b/fa8 zero size 2024-03-22T10:42:52.750 INFO:tasks.workunit.client.1.smithi150.stdout:9/438: dread - d1/d8/de/d73/f74 zero size 2024-03-22T10:42:52.751 INFO:tasks.workunit.client.1.smithi150.stdout:5/273: chown d0/d8/dc/de/d14/d4f/f3d 153190662 1 2024-03-22T10:42:52.753 INFO:tasks.workunit.client.1.smithi150.stdout:4/377: rename d0/c22 to d0/d2f/d63/d7b/c84 0 2024-03-22T10:42:52.754 INFO:tasks.workunit.client.0.smithi115.stdout:4/762: rename d1/d16 to d1/d66/d106 0 2024-03-22T10:42:52.757 INFO:tasks.workunit.client.1.smithi150.stdout:0/397: creat d0/d12/d4f/d11/f7e x:0 0 0 2024-03-22T10:42:52.757 INFO:tasks.workunit.client.1.smithi150.stdout:8/400: mkdir d0/d2/d19/d29/d88 0 2024-03-22T10:42:52.757 INFO:tasks.workunit.client.0.smithi115.stdout:1/837: creat d0/d12/d17/d67/dd8/de0/f128 x:0 0 0 2024-03-22T10:42:52.759 INFO:tasks.workunit.client.0.smithi115.stdout:0/701: rename d3/d7/d1b/d1f/c63 to d3/da/d4c/d74/d84/ce6 0 2024-03-22T10:42:52.760 INFO:tasks.workunit.client.0.smithi115.stdout:3/774: getdents d3/d82/d57 0 2024-03-22T10:42:52.760 INFO:tasks.workunit.client.0.smithi115.stdout:1/838: write d0/d8/de/f26 [2365278,103636] 0 2024-03-22T10:42:52.762 INFO:tasks.workunit.client.0.smithi115.stdout:4/763: creat d1/d30/d4c/d86/d9f/db8/f107 x:0 0 0 2024-03-22T10:42:52.763 INFO:tasks.workunit.client.0.smithi115.stdout:3/775: chown d3/d19/dda/ff8 26440 1 2024-03-22T10:42:52.763 INFO:tasks.workunit.client.1.smithi150.stdout:5/274: unlink d0/d33/f41 0 2024-03-22T10:42:52.764 INFO:tasks.workunit.client.0.smithi115.stdout:0/702: fdatasync d3/d7/d1b/d93/fcc 0 2024-03-22T10:42:52.765 INFO:tasks.workunit.client.0.smithi115.stdout:1/839: dread - d0/d8/d4d/d56/db6/ddc/fbc zero size 2024-03-22T10:42:52.769 INFO:tasks.workunit.client.0.smithi115.stdout:0/703: chown d3/d7/d1b/d93/c97 30 1 2024-03-22T10:42:52.770 INFO:tasks.workunit.client.0.smithi115.stdout:4/764: creat d1/d66/d106/df0/f108 x:0 0 0 2024-03-22T10:42:52.772 INFO:tasks.workunit.client.1.smithi150.stdout:4/378: unlink d0/l49 0 2024-03-22T10:42:52.774 INFO:tasks.workunit.client.1.smithi150.stdout:4/379: write d0/d1b/d4d/d6c/f62 [1040328,23505] 0 2024-03-22T10:42:52.774 INFO:tasks.workunit.client.0.smithi115.stdout:1/840: truncate d0/d58/f105 163235 0 2024-03-22T10:42:52.775 INFO:tasks.workunit.client.1.smithi150.stdout:4/380: chown d0/de/f3c 665 1 2024-03-22T10:42:52.777 INFO:tasks.workunit.client.0.smithi115.stdout:0/704: truncate d3/d7/d1c/d6c/d9a/fbe 5089798 0 2024-03-22T10:42:52.778 INFO:tasks.workunit.client.1.smithi150.stdout:4/381: write d0/de/f5e [130142,117705] 0 2024-03-22T10:42:52.779 INFO:tasks.workunit.client.0.smithi115.stdout:4/765: mkdir d1/d66/d106/d35/d40/d54/d5e/df3/d109 0 2024-03-22T10:42:52.786 INFO:tasks.workunit.client.0.smithi115.stdout:1/841: getdents d0/d12/d17/d37/d9e/da0/dff 0 2024-03-22T10:42:52.786 INFO:tasks.workunit.client.0.smithi115.stdout:4/766: chown d1/d66/d82/da1/fc9 79697044 1 2024-03-22T10:42:52.789 INFO:tasks.workunit.client.1.smithi150.stdout:4/382: write d0/d2f/f30 [4658553,76152] 0 2024-03-22T10:42:52.789 INFO:tasks.workunit.client.0.smithi115.stdout:3/776: dwrite d3/d19/d2b/d49/d91/f9f [0,4194304] 0 2024-03-22T10:42:52.790 INFO:tasks.workunit.client.0.smithi115.stdout:4/767: write d1/d30/d4c/d86/fac [2146665,24399] 0 2024-03-22T10:42:52.793 INFO:tasks.workunit.client.0.smithi115.stdout:3/777: chown d3/d36/d41 1371 1 2024-03-22T10:42:52.794 INFO:tasks.workunit.client.0.smithi115.stdout:4/768: write d1/dbb/ddf/df9/fff [280474,71476] 0 2024-03-22T10:42:52.796 INFO:tasks.workunit.client.1.smithi150.stdout:5/275: creat d0/d8/dc/de/d14/d2e/d46/d4d/f51 x:0 0 0 2024-03-22T10:42:52.799 INFO:tasks.workunit.client.1.smithi150.stdout:0/398: link d0/d12/d4f/d42/d10/d38/d52/l55 d0/d12/l7f 0 2024-03-22T10:42:52.802 INFO:tasks.workunit.client.0.smithi115.stdout:3/778: chown d3/d82/f27 65517543 1 2024-03-22T10:42:52.806 INFO:tasks.workunit.client.0.smithi115.stdout:4/769: rename d1/d30/fd2 to d1/d30/d4c/d86/d9f/d4f/f10a 0 2024-03-22T10:42:52.807 INFO:tasks.workunit.client.1.smithi150.stdout:5/276: creat d0/d8/dc/de/d14/d4f/f52 x:0 0 0 2024-03-22T10:42:52.808 INFO:tasks.workunit.client.1.smithi150.stdout:3/356: truncate d5/da/f33 3360552 0 2024-03-22T10:42:52.811 INFO:tasks.workunit.client.1.smithi150.stdout:6/360: dread db/dd/d15/d30/f4b [0,4194304] 0 2024-03-22T10:42:52.815 INFO:tasks.workunit.client.0.smithi115.stdout:3/779: rename d3/d19/dda/cec to d3/d6d/db4/d107/da6/c108 0 2024-03-22T10:42:52.816 INFO:tasks.workunit.client.0.smithi115.stdout:1/842: dwrite d0/d8/d4d/d56/dcf/d10b/de2/df8/ffe [0,4194304] 0 2024-03-22T10:42:52.816 INFO:tasks.workunit.client.1.smithi150.stdout:1/331: dread d1/da/f43 [0,4194304] 0 2024-03-22T10:42:52.818 INFO:tasks.workunit.client.0.smithi115.stdout:3/780: rmdir d3 39 2024-03-22T10:42:52.819 INFO:tasks.workunit.client.1.smithi150.stdout:4/383: dwrite d0/de/f5b [0,4194304] 0 2024-03-22T10:42:52.819 INFO:tasks.workunit.client.1.smithi150.stdout:2/427: write d12/d16/d67/d3c/f6b [874041,62920] 0 2024-03-22T10:42:52.823 INFO:tasks.workunit.client.1.smithi150.stdout:6/361: creat db/dd/d2e/f81 x:0 0 0 2024-03-22T10:42:52.823 INFO:tasks.workunit.client.1.smithi150.stdout:1/332: write d1/d19/d1a/f2b [4198268,83746] 0 2024-03-22T10:42:52.824 INFO:tasks.workunit.client.0.smithi115.stdout:0/705: dread d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:52.825 INFO:tasks.workunit.client.0.smithi115.stdout:1/843: fdatasync d0/d12/d17/d37/d9e/de9/dc1/fa2 0 2024-03-22T10:42:52.829 INFO:tasks.workunit.client.1.smithi150.stdout:5/277: rename d0/d8/dc/d2c to d0/d8/dc/d1d/d53 0 2024-03-22T10:42:52.832 INFO:tasks.workunit.client.0.smithi115.stdout:0/706: symlink d3/da/d4c/d74/d84/dd7/le7 0 2024-03-22T10:42:52.832 INFO:tasks.workunit.client.0.smithi115.stdout:9/753: sync 2024-03-22T10:42:52.832 INFO:tasks.workunit.client.0.smithi115.stdout:3/781: read - d3/d36/d41/d33/f8d zero size 2024-03-22T10:42:52.836 INFO:tasks.workunit.client.1.smithi150.stdout:4/384: mknod d0/d1b/d4d/d6c/d61/c85 0 2024-03-22T10:42:52.837 INFO:tasks.workunit.client.0.smithi115.stdout:3/782: fdatasync d3/d36/d41/fba 0 2024-03-22T10:42:52.840 INFO:tasks.workunit.client.1.smithi150.stdout:3/357: link d5/d22/d2f/f41 d5/da/f70 0 2024-03-22T10:42:52.841 INFO:tasks.workunit.client.0.smithi115.stdout:4/770: truncate d1/d66/d106/d35/fbd 1698629 0 2024-03-22T10:42:52.843 INFO:tasks.workunit.client.0.smithi115.stdout:3/783: rmdir d3/d6d/db4/de1 39 2024-03-22T10:42:52.844 INFO:tasks.workunit.client.1.smithi150.stdout:6/362: mkdir db/dd/d82 0 2024-03-22T10:42:52.844 INFO:tasks.workunit.client.0.smithi115.stdout:4/771: stat d1/d30/d4c/d86/d9f/d75/d99/da8 0 2024-03-22T10:42:52.845 INFO:tasks.workunit.client.0.smithi115.stdout:0/707: truncate d3/da/d4c/d69/d6a/fa4 3153612 0 2024-03-22T10:42:52.847 INFO:tasks.workunit.client.0.smithi115.stdout:9/754: link d6/d7/d20/d22/d23/f85 d6/d7/d6a/d6f/f10d 0 2024-03-22T10:42:52.848 INFO:tasks.workunit.client.1.smithi150.stdout:3/358: write d5/d22/d2f/d38/d3e/f55 [613044,17750] 0 2024-03-22T10:42:52.851 INFO:tasks.workunit.client.0.smithi115.stdout:1/844: link d0/d8/de/d1a/c4e d0/d68/d81/c129 0 2024-03-22T10:42:52.853 INFO:tasks.workunit.client.0.smithi115.stdout:0/708: creat d3/d7/d1b/d1f/d78/fe8 x:0 0 0 2024-03-22T10:42:52.853 INFO:tasks.workunit.client.0.smithi115.stdout:0/709: stat d3/d4/f71 0 2024-03-22T10:42:52.854 INFO:tasks.workunit.client.0.smithi115.stdout:4/772: unlink d1/d66/d106/d35/d40/d5c/d55/l3a 0 2024-03-22T10:42:52.854 INFO:tasks.workunit.client.1.smithi150.stdout:5/278: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T10:42:52.854 INFO:tasks.workunit.client.1.smithi150.stdout:8/401: truncate d0/f5a 1098961 0 2024-03-22T10:42:52.855 INFO:tasks.workunit.client.0.smithi115.stdout:3/784: creat d3/d6d/db4/f109 x:0 0 0 2024-03-22T10:42:52.856 INFO:tasks.workunit.client.0.smithi115.stdout:0/710: readlink d3/da/d4c/d74/d84/dac/lad 0 2024-03-22T10:42:52.859 INFO:tasks.workunit.client.0.smithi115.stdout:4/773: creat d1/d66/d82/f10b x:0 0 0 2024-03-22T10:42:52.860 INFO:tasks.workunit.client.1.smithi150.stdout:8/402: chown d0/d2/d19/f20 299389 1 2024-03-22T10:42:52.861 INFO:tasks.workunit.client.0.smithi115.stdout:1/845: mknod d0/d12/d17/d37/d119/c12a 0 2024-03-22T10:42:52.862 INFO:tasks.workunit.client.0.smithi115.stdout:0/711: unlink d3/d7/d1b/f92 0 2024-03-22T10:42:52.865 INFO:tasks.workunit.client.1.smithi150.stdout:5/279: write d0/d8/dc/de/f13 [865930,37657] 0 2024-03-22T10:42:52.865 INFO:tasks.workunit.client.1.smithi150.stdout:8/403: stat d0/d2/f5f 0 2024-03-22T10:42:52.865 INFO:tasks.workunit.client.1.smithi150.stdout:0/399: dread d0/d12/d4f/d11/f5d [0,4194304] 0 2024-03-22T10:42:52.865 INFO:tasks.workunit.client.0.smithi115.stdout:9/755: getdents d6/d7 0 2024-03-22T10:42:52.866 INFO:tasks.workunit.client.1.smithi150.stdout:3/359: rmdir d5/d53/d60 0 2024-03-22T10:42:52.868 INFO:tasks.workunit.client.0.smithi115.stdout:3/785: link d3/d16/d29/f3c d3/d19/dd8/f10a 0 2024-03-22T10:42:52.869 INFO:tasks.workunit.client.0.smithi115.stdout:9/756: fsync d6/d7/d20/d22/d23/d106/f98 0 2024-03-22T10:42:52.870 INFO:tasks.workunit.client.0.smithi115.stdout:4/774: fsync d1/d66/d82/f8e 0 2024-03-22T10:42:52.871 INFO:tasks.workunit.client.0.smithi115.stdout:1/846: symlink d0/d8/de/l12b 0 2024-03-22T10:42:52.872 INFO:tasks.workunit.client.1.smithi150.stdout:8/404: write d0/d2/d19/f20 [234310,50413] 0 2024-03-22T10:42:52.874 INFO:tasks.workunit.client.0.smithi115.stdout:9/757: creat d6/f10e x:0 0 0 2024-03-22T10:42:52.874 INFO:tasks.workunit.client.1.smithi150.stdout:1/333: dwrite d1/da/d15/d18/d4b/f53 [0,4194304] 0 2024-03-22T10:42:52.875 INFO:tasks.workunit.client.1.smithi150.stdout:4/385: dwrite d0/d1b/d4d/d6c/d61/f69 [0,4194304] 0 2024-03-22T10:42:52.876 INFO:tasks.workunit.client.1.smithi150.stdout:8/405: stat d0/d2/d19/d29/d2e 0 2024-03-22T10:42:52.877 INFO:tasks.workunit.client.0.smithi115.stdout:9/758: mknod d6/d7/c10f 0 2024-03-22T10:42:52.880 INFO:tasks.workunit.client.1.smithi150.stdout:4/386: write d0/d1b/d68/f6a [3343514,130697] 0 2024-03-22T10:42:52.880 INFO:tasks.workunit.client.0.smithi115.stdout:4/775: rename d1/d66/d106/d35/d40/cb2 to d1/d30/d4c/d86/d9f/d75/d99/c10c 0 2024-03-22T10:42:52.881 INFO:tasks.workunit.client.0.smithi115.stdout:4/776: write d1/d66/d106/df0/ffc [4786424,105386] 0 2024-03-22T10:42:52.888 INFO:tasks.workunit.client.0.smithi115.stdout:4/777: creat d1/dbb/dc6/dd8/df7/f10d x:0 0 0 2024-03-22T10:42:52.889 INFO:tasks.workunit.client.1.smithi150.stdout:0/400: rename d0/d12/f3b to d0/d12/d4f/d42/d10/d38/d52/d68/f80 0 2024-03-22T10:42:52.891 INFO:tasks.workunit.client.1.smithi150.stdout:4/387: mknod d0/d1b/d4d/d6c/c86 0 2024-03-22T10:42:52.893 INFO:tasks.workunit.client.1.smithi150.stdout:4/388: dread - d0/f5f zero size 2024-03-22T10:42:52.894 INFO:tasks.workunit.client.0.smithi115.stdout:4/778: mkdir d1/d66/dd9/d10e 0 2024-03-22T10:42:52.894 INFO:tasks.workunit.client.1.smithi150.stdout:8/406: mkdir d0/d3c/d71/d7a/d89 0 2024-03-22T10:42:52.898 INFO:tasks.workunit.client.1.smithi150.stdout:4/389: write d0/de/f3d [986379,9071] 0 2024-03-22T10:42:52.899 INFO:tasks.workunit.client.0.smithi115.stdout:4/779: mkdir d1/dbb/dc6/dd8/df7/d10f 0 2024-03-22T10:42:52.902 INFO:tasks.workunit.client.1.smithi150.stdout:1/334: getdents d1/da/d2a/d4e 0 2024-03-22T10:42:52.902 INFO:tasks.workunit.client.0.smithi115.stdout:4/780: mkdir d1/d66/d82/da1/d110 0 2024-03-22T10:42:52.903 INFO:tasks.workunit.client.0.smithi115.stdout:0/712: dread d3/d4/d14/f4f [0,4194304] 0 2024-03-22T10:42:52.904 INFO:tasks.workunit.client.1.smithi150.stdout:4/390: chown d0/d1b/d4d/d6c/d61/c45 449945 1 2024-03-22T10:42:52.905 INFO:tasks.workunit.client.1.smithi150.stdout:6/363: truncate db/dd/d15/d60/f53 770268 0 2024-03-22T10:42:52.905 INFO:tasks.workunit.client.0.smithi115.stdout:3/786: write d3/d82/d52/f56 [3428690,52522] 0 2024-03-22T10:42:52.906 INFO:tasks.workunit.client.0.smithi115.stdout:0/713: stat d3/l12 0 2024-03-22T10:42:52.907 INFO:tasks.workunit.client.1.smithi150.stdout:4/391: dread - d0/f5f zero size 2024-03-22T10:42:52.909 INFO:tasks.workunit.client.1.smithi150.stdout:4/392: stat d0/d47/f64 0 2024-03-22T10:42:52.911 INFO:tasks.workunit.client.0.smithi115.stdout:1/847: write d0/d12/d17/d37/d5c/f113 [2793936,12909] 0 2024-03-22T10:42:52.911 INFO:tasks.workunit.client.0.smithi115.stdout:7/790: sync 2024-03-22T10:42:52.912 INFO:tasks.workunit.client.1.smithi150.stdout:8/407: link d0/d2/d19/d29/d2f/df/f26 d0/d2/d19/d29/d2f/df/f8a 0 2024-03-22T10:42:52.913 INFO:tasks.workunit.client.1.smithi150.stdout:5/280: truncate d0/d8/f23 3163332 0 2024-03-22T10:42:52.914 INFO:tasks.workunit.client.1.smithi150.stdout:4/393: symlink d0/d2f/d63/d7b/l87 0 2024-03-22T10:42:52.915 INFO:tasks.workunit.client.1.smithi150.stdout:3/360: write d5/d22/d2f/f30 [2755994,94284] 0 2024-03-22T10:42:52.918 INFO:tasks.workunit.client.1.smithi150.stdout:8/408: symlink d0/d2/l8b 0 2024-03-22T10:42:52.919 INFO:tasks.workunit.client.0.smithi115.stdout:3/787: unlink d3/d6d/db4/d107/da6/c108 0 2024-03-22T10:42:52.920 INFO:tasks.workunit.client.0.smithi115.stdout:9/759: mkdir d6/d7/d30/d37/dbe/d110 0 2024-03-22T10:42:52.922 INFO:tasks.workunit.client.1.smithi150.stdout:3/361: stat d5/ce 0 2024-03-22T10:42:52.922 INFO:tasks.workunit.client.1.smithi150.stdout:8/409: write d0/f6f [42288,3455] 0 2024-03-22T10:42:52.922 INFO:tasks.workunit.client.1.smithi150.stdout:4/394: symlink d0/d1b/d4d/d6c/l88 0 2024-03-22T10:42:52.922 INFO:tasks.workunit.client.0.smithi115.stdout:3/788: readlink d3/d82/l97 0 2024-03-22T10:42:52.922 INFO:tasks.workunit.client.0.smithi115.stdout:4/781: write d1/d66/d82/f8e [2343625,93204] 0 2024-03-22T10:42:52.924 INFO:tasks.workunit.client.1.smithi150.stdout:4/395: stat d0/l29 0 2024-03-22T10:42:52.925 INFO:tasks.workunit.client.0.smithi115.stdout:3/789: fdatasync d3/d19/d2b/d49/fbd 0 2024-03-22T10:42:52.926 INFO:tasks.workunit.client.1.smithi150.stdout:0/401: truncate d0/d1/f70 1859855 0 2024-03-22T10:42:52.926 INFO:tasks.workunit.client.1.smithi150.stdout:3/362: truncate d5/da/f70 286891 0 2024-03-22T10:42:52.928 INFO:tasks.workunit.client.1.smithi150.stdout:1/335: dwrite d1/d19/d57/f62 [0,4194304] 0 2024-03-22T10:42:52.928 INFO:tasks.workunit.client.1.smithi150.stdout:5/281: getdents d0 0 2024-03-22T10:42:52.928 INFO:tasks.workunit.client.1.smithi150.stdout:4/396: chown d0/cc 5 1 2024-03-22T10:42:52.932 INFO:tasks.workunit.client.0.smithi115.stdout:3/790: chown d3/d89/dca/f88 6287 1 2024-03-22T10:42:52.932 INFO:tasks.workunit.client.1.smithi150.stdout:5/282: chown d0/c20 51673804 1 2024-03-22T10:42:52.932 INFO:tasks.workunit.client.1.smithi150.stdout:0/402: mkdir d0/d1/d81 0 2024-03-22T10:42:52.933 INFO:tasks.workunit.client.0.smithi115.stdout:9/760: truncate d6/d7/d48/d70/fc0 681373 0 2024-03-22T10:42:52.934 INFO:tasks.workunit.client.1.smithi150.stdout:1/336: fsync d1/d19/d1a/f1e 0 2024-03-22T10:42:52.935 INFO:tasks.workunit.client.0.smithi115.stdout:4/782: creat d1/d30/d4c/d86/dde/d103/f111 x:0 0 0 2024-03-22T10:42:52.935 INFO:tasks.workunit.client.1.smithi150.stdout:4/397: dread d0/f26 [0,4194304] 0 2024-03-22T10:42:52.936 INFO:tasks.workunit.client.1.smithi150.stdout:8/410: rename d0/d2/d19/d29/d2e/d70/c5c to d0/d2/d19/d29/d37/c8c 0 2024-03-22T10:42:52.936 INFO:tasks.workunit.client.0.smithi115.stdout:9/761: dread - d6/d8/d3c/d59/d68/ff6 zero size 2024-03-22T10:42:52.936 INFO:tasks.workunit.client.1.smithi150.stdout:3/363: link d5/d22/d37/f6c d5/d53/f71 0 2024-03-22T10:42:52.937 INFO:tasks.workunit.client.0.smithi115.stdout:1/848: link d0/d8/db/d25/d9f/fc2 d0/d12/d17/d37/d9e/de9/f12c 0 2024-03-22T10:42:52.938 INFO:tasks.workunit.client.1.smithi150.stdout:5/283: creat d0/d8/dc/de/d14/d2e/f54 x:0 0 0 2024-03-22T10:42:52.938 INFO:tasks.workunit.client.0.smithi115.stdout:7/791: dwrite d2/d7/d12/f67 [0,4194304] 0 2024-03-22T10:42:52.940 INFO:tasks.workunit.client.1.smithi150.stdout:5/284: write d0/d8/dc/f40 [1758161,6979] 0 2024-03-22T10:42:52.940 INFO:tasks.workunit.client.1.smithi150.stdout:4/398: symlink d0/d32/d7c/l89 0 2024-03-22T10:42:52.945 INFO:tasks.workunit.client.1.smithi150.stdout:4/399: write d0/f5f [336811,120527] 0 2024-03-22T10:42:52.946 INFO:tasks.workunit.client.0.smithi115.stdout:0/714: dwrite d3/d4/d6d/fb0 [0,4194304] 0 2024-03-22T10:42:52.949 INFO:tasks.workunit.client.0.smithi115.stdout:9/762: read d6/d8/d3c/d59/fb5 [216097,106241] 0 2024-03-22T10:42:52.949 INFO:tasks.workunit.client.0.smithi115.stdout:3/791: creat d3/d6d/db4/de1/f10b x:0 0 0 2024-03-22T10:42:52.949 INFO:tasks.workunit.client.1.smithi150.stdout:0/403: getdents d0/d12/d4f/d11 0 2024-03-22T10:42:52.950 INFO:tasks.workunit.client.1.smithi150.stdout:5/285: mknod d0/d8/dc/de/d14/d4f/d47/c55 0 2024-03-22T10:42:52.951 INFO:tasks.workunit.client.0.smithi115.stdout:3/792: write d3/d36/d41/d33/fe2 [1438039,42657] 0 2024-03-22T10:42:52.951 INFO:tasks.workunit.client.0.smithi115.stdout:7/792: creat d2/d7/d12/d10/d23/d3d/da4/f10f x:0 0 0 2024-03-22T10:42:52.952 INFO:tasks.workunit.client.1.smithi150.stdout:4/400: mknod d0/de/c8a 0 2024-03-22T10:42:52.954 INFO:tasks.workunit.client.1.smithi150.stdout:4/401: read d0/de/f28 [3717360,96890] 0 2024-03-22T10:42:52.955 INFO:tasks.workunit.client.0.smithi115.stdout:1/849: rename d0/d8/db/f2d to d0/d12/d17/d37/d9e/de9/dc1/f12d 0 2024-03-22T10:42:52.956 INFO:tasks.workunit.client.1.smithi150.stdout:5/286: readlink d0/d8/l1c 0 2024-03-22T10:42:52.958 INFO:tasks.workunit.client.1.smithi150.stdout:9/439: sync 2024-03-22T10:42:52.958 INFO:tasks.workunit.client.1.smithi150.stdout:7/370: sync 2024-03-22T10:42:52.958 INFO:tasks.workunit.client.1.smithi150.stdout:2/428: sync 2024-03-22T10:42:52.960 INFO:tasks.workunit.client.1.smithi150.stdout:5/287: chown d0/d8/dc 1962247 1 2024-03-22T10:42:52.960 INFO:tasks.workunit.client.1.smithi150.stdout:1/337: dwrite d1/f1d [0,4194304] 0 2024-03-22T10:42:52.961 INFO:tasks.workunit.client.0.smithi115.stdout:0/715: unlink d3/d7/d1b/d1f/d5b/l7c 0 2024-03-22T10:42:52.961 INFO:tasks.workunit.client.0.smithi115.stdout:9/763: dread d6/d7/d30/d37/dbe/fd4 [0,4194304] 0 2024-03-22T10:42:52.962 INFO:tasks.workunit.client.1.smithi150.stdout:6/364: dread db/dd/f14 [0,4194304] 0 2024-03-22T10:42:52.962 INFO:tasks.workunit.client.0.smithi115.stdout:3/793: creat d3/d19/dda/f10c x:0 0 0 2024-03-22T10:42:52.962 INFO:tasks.workunit.client.1.smithi150.stdout:7/371: dread d2/d5/d7/d9/db/f1a [0,4194304] 0 2024-03-22T10:42:52.963 INFO:tasks.workunit.client.0.smithi115.stdout:1/850: dread - d0/d8/de/d1a/d91/fd2 zero size 2024-03-22T10:42:52.965 INFO:tasks.workunit.client.0.smithi115.stdout:4/783: dwrite d1/d30/d4c/d86/d9f/fd6 [0,4194304] 0 2024-03-22T10:42:52.968 INFO:tasks.workunit.client.1.smithi150.stdout:4/402: symlink d0/d47/l8b 0 2024-03-22T10:42:52.969 INFO:tasks.workunit.client.0.smithi115.stdout:6/704: sync 2024-03-22T10:42:52.970 INFO:tasks.workunit.client.0.smithi115.stdout:2/693: sync 2024-03-22T10:42:52.970 INFO:tasks.workunit.client.0.smithi115.stdout:8/935: sync 2024-03-22T10:42:52.970 INFO:tasks.workunit.client.0.smithi115.stdout:5/816: sync 2024-03-22T10:42:52.970 INFO:tasks.workunit.client.0.smithi115.stdout:7/793: creat d2/d7/d12/d10/d19/f110 x:0 0 0 2024-03-22T10:42:52.971 INFO:tasks.workunit.client.0.smithi115.stdout:0/716: rmdir d3/d7/d1b/d35 39 2024-03-22T10:42:52.972 INFO:tasks.workunit.client.1.smithi150.stdout:1/338: creat d1/da/f67 x:0 0 0 2024-03-22T10:42:52.978 INFO:tasks.workunit.client.0.smithi115.stdout:9/764: creat d6/d7/d48/d53/dc9/f111 x:0 0 0 2024-03-22T10:42:52.982 INFO:tasks.workunit.client.1.smithi150.stdout:2/429: creat d12/d16/d61/f8c x:0 0 0 2024-03-22T10:42:52.983 INFO:tasks.workunit.client.0.smithi115.stdout:8/936: stat d6/dd/d24/d4a 0 2024-03-22T10:42:52.983 INFO:tasks.workunit.client.1.smithi150.stdout:9/440: creat d1/d19/f9f x:0 0 0 2024-03-22T10:42:52.984 INFO:tasks.workunit.client.0.smithi115.stdout:4/784: creat d1/d66/d82/da1/f112 x:0 0 0 2024-03-22T10:42:52.985 INFO:tasks.workunit.client.0.smithi115.stdout:0/717: mknod d3/d7/d1b/d1f/d78/dc1/ce9 0 2024-03-22T10:42:52.987 INFO:tasks.workunit.client.0.smithi115.stdout:6/705: symlink d4/d11/d12/d1c/d8a/d98/d9f/dc3/le0 0 2024-03-22T10:42:52.988 INFO:tasks.workunit.client.1.smithi150.stdout:9/441: fdatasync d1/d8/de/df/d36/f96 0 2024-03-22T10:42:52.988 INFO:tasks.workunit.client.1.smithi150.stdout:5/288: dread d0/d8/dc/de/d14/f25 [0,4194304] 0 2024-03-22T10:42:52.989 INFO:tasks.workunit.client.0.smithi115.stdout:0/718: readlink d3/d52/l82 0 2024-03-22T10:42:52.992 INFO:tasks.workunit.client.0.smithi115.stdout:7/794: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/f111 x:0 0 0 2024-03-22T10:42:52.992 INFO:tasks.workunit.client.1.smithi150.stdout:5/289: write d0/d8/dc/de/d14/d2e/f3e [1960285,89966] 0 2024-03-22T10:42:52.993 INFO:tasks.workunit.client.1.smithi150.stdout:9/442: write d1/f7b [622286,10250] 0 2024-03-22T10:42:52.997 INFO:tasks.workunit.client.1.smithi150.stdout:4/403: creat d0/d1b/d4d/f8c x:0 0 0 2024-03-22T10:42:52.998 INFO:tasks.workunit.client.1.smithi150.stdout:9/443: chown d1/d8/de/df/d20 60197 1 2024-03-22T10:42:52.999 INFO:tasks.workunit.client.0.smithi115.stdout:0/719: chown d3/d7/d6b 0 1 2024-03-22T10:42:53.000 INFO:tasks.workunit.client.0.smithi115.stdout:9/765: symlink d6/d7/d48/df4/dfa/l112 0 2024-03-22T10:42:53.001 INFO:tasks.workunit.client.1.smithi150.stdout:2/430: mkdir d12/d16/d67/d8d 0 2024-03-22T10:42:53.002 INFO:tasks.workunit.client.1.smithi150.stdout:8/411: write d0/d2/d19/d29/d2f/f3b [1126357,16452] 0 2024-03-22T10:42:53.004 INFO:tasks.workunit.client.0.smithi115.stdout:6/706: mknod d4/d5e/dd0/ce1 0 2024-03-22T10:42:53.004 INFO:tasks.workunit.client.0.smithi115.stdout:3/794: dwrite d3/d36/f3e [0,4194304] 0 2024-03-22T10:42:53.005 INFO:tasks.workunit.client.1.smithi150.stdout:0/404: rename d0/d12/d4f/d42/d10/f13 to d0/d12/d4f/d42/f82 0 2024-03-22T10:42:53.008 INFO:tasks.workunit.client.0.smithi115.stdout:5/817: link d2/df/d13/d8b/d93/da6/dbc/d50/c7d d2/d15/d1b/d7c/d84/d89/c10a 0 2024-03-22T10:42:53.009 INFO:tasks.workunit.client.0.smithi115.stdout:7/795: read d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f6a [567575,68652] 0 2024-03-22T10:42:53.010 INFO:tasks.workunit.client.1.smithi150.stdout:5/290: mkdir d0/d8/dc/de/d14/d2e/d45/d56 0 2024-03-22T10:42:53.013 INFO:tasks.workunit.client.0.smithi115.stdout:4/785: dread d1/d30/d4c/d86/d9f/d75/d99/da8/fd0 [0,4194304] 0 2024-03-22T10:42:53.013 INFO:tasks.workunit.client.0.smithi115.stdout:0/720: truncate d3/da/d4c/d69/fbd 1291347 0 2024-03-22T10:42:53.015 INFO:tasks.workunit.client.1.smithi150.stdout:8/412: mknod d0/d2/d46/c8d 0 2024-03-22T10:42:53.015 INFO:tasks.workunit.client.1.smithi150.stdout:2/431: symlink d12/d16/d67/d8d/l8e 0 2024-03-22T10:42:53.016 INFO:tasks.workunit.client.0.smithi115.stdout:6/707: creat d4/d13/d33/d58/fe2 x:0 0 0 2024-03-22T10:42:53.018 INFO:tasks.workunit.client.1.smithi150.stdout:3/364: dwrite d5/d22/d2f/d38/f39 [0,4194304] 0 2024-03-22T10:42:53.018 INFO:tasks.workunit.client.1.smithi150.stdout:4/404: symlink d0/d1b/l8d 0 2024-03-22T10:42:53.018 INFO:tasks.workunit.client.0.smithi115.stdout:5/818: rename d2/d15/dc8 to d2/df/d13/d68/d106/d10b 0 2024-03-22T10:42:53.020 INFO:tasks.workunit.client.1.smithi150.stdout:5/291: mknod d0/d33/c57 0 2024-03-22T10:42:53.023 INFO:tasks.workunit.client.0.smithi115.stdout:0/721: write d3/da/d4c/d69/d6a/f86 [751449,94984] 0 2024-03-22T10:42:53.023 INFO:tasks.workunit.client.0.smithi115.stdout:8/937: dwrite d6/dd/d12/d13c/dfd/f127 [0,4194304] 0 2024-03-22T10:42:53.028 INFO:tasks.workunit.client.1.smithi150.stdout:2/432: symlink d12/d16/d67/d8d/l8f 0 2024-03-22T10:42:53.030 INFO:tasks.workunit.client.0.smithi115.stdout:6/708: mknod d4/d13/d33/d58/ce3 0 2024-03-22T10:42:53.033 INFO:tasks.workunit.client.1.smithi150.stdout:9/444: dwrite d1/d8/f82 [0,4194304] 0 2024-03-22T10:42:53.033 INFO:tasks.workunit.client.0.smithi115.stdout:3/795: rename d3/d82/d57/c70 to d3/d82/d54/c10d 0 2024-03-22T10:42:53.034 INFO:tasks.workunit.client.0.smithi115.stdout:2/694: write da/d25/d3d/f76 [4022001,92929] 0 2024-03-22T10:42:53.034 INFO:tasks.workunit.client.0.smithi115.stdout:4/786: dread d1/d66/d106/d68/f9e [0,4194304] 0 2024-03-22T10:42:53.035 INFO:tasks.workunit.client.0.smithi115.stdout:8/938: write d6/f91 [3016201,84648] 0 2024-03-22T10:42:53.035 INFO:tasks.workunit.client.0.smithi115.stdout:2/695: stat da/d25/d3d/d57/d7d 0 2024-03-22T10:42:53.036 INFO:tasks.workunit.client.0.smithi115.stdout:0/722: mknod d3/da/d4c/da3/cea 0 2024-03-22T10:42:53.039 INFO:tasks.workunit.client.1.smithi150.stdout:3/365: mkdir d5/d22/d2f/d38/d3e/d72 0 2024-03-22T10:42:53.039 INFO:tasks.workunit.client.0.smithi115.stdout:6/709: symlink d4/d11/d99/le4 0 2024-03-22T10:42:53.043 INFO:tasks.workunit.client.1.smithi150.stdout:4/405: unlink d0/d1b/d4d/d6c/l88 0 2024-03-22T10:42:53.044 INFO:tasks.workunit.client.0.smithi115.stdout:3/796: rmdir d3/d82/d54/df4 0 2024-03-22T10:42:53.046 INFO:tasks.workunit.client.1.smithi150.stdout:7/372: dread d2/d5/d7/d9/f65 [0,4194304] 0 2024-03-22T10:42:53.046 INFO:tasks.workunit.client.1.smithi150.stdout:8/413: symlink d0/d2/d19/d29/d2f/l8e 0 2024-03-22T10:42:53.048 INFO:tasks.workunit.client.0.smithi115.stdout:8/939: dread - d6/dd/d24/d4a/d7b/d8f/da0/fc3 zero size 2024-03-22T10:42:53.049 INFO:tasks.workunit.client.0.smithi115.stdout:3/797: stat d3/d16/d29/d35/l66 0 2024-03-22T10:42:53.049 INFO:tasks.workunit.client.0.smithi115.stdout:6/710: symlink d4/d11/d91/le5 0 2024-03-22T10:42:53.051 INFO:tasks.workunit.client.0.smithi115.stdout:3/798: write d3/d36/f5c [455268,31470] 0 2024-03-22T10:42:53.051 INFO:tasks.workunit.client.1.smithi150.stdout:0/405: getdents d0/d12/d4f/d42/d10/d38 0 2024-03-22T10:42:53.051 INFO:tasks.workunit.client.1.smithi150.stdout:8/414: write d0/d2/d19/d29/d2f/f64 [1758621,113816] 0 2024-03-22T10:42:53.054 INFO:tasks.workunit.client.1.smithi150.stdout:2/433: symlink d12/d16/l90 0 2024-03-22T10:42:53.055 INFO:tasks.workunit.client.0.smithi115.stdout:3/799: fdatasync d3/d36/d41/f48 0 2024-03-22T10:42:53.055 INFO:tasks.workunit.client.1.smithi150.stdout:5/292: dwrite d0/d8/dc/de/d14/f25 [0,4194304] 0 2024-03-22T10:42:53.057 INFO:tasks.workunit.client.0.smithi115.stdout:8/940: creat d6/dd/f13d x:0 0 0 2024-03-22T10:42:53.059 INFO:tasks.workunit.client.1.smithi150.stdout:3/366: symlink d5/d22/d2f/d38/d59/d5d/l73 0 2024-03-22T10:42:53.059 INFO:tasks.workunit.client.1.smithi150.stdout:9/445: creat d1/d8/de/df/d24/d58/d8c/fa0 x:0 0 0 2024-03-22T10:42:53.061 INFO:tasks.workunit.client.1.smithi150.stdout:4/406: fsync d0/d1b/f74 0 2024-03-22T10:42:53.063 INFO:tasks.workunit.client.1.smithi150.stdout:3/367: fsync d5/f3c 0 2024-03-22T10:42:53.064 INFO:tasks.workunit.client.0.smithi115.stdout:3/800: symlink d3/d6d/db4/d107/db3/l10e 0 2024-03-22T10:42:53.066 INFO:tasks.workunit.client.0.smithi115.stdout:8/941: creat d6/dd/d12/d13c/f13e x:0 0 0 2024-03-22T10:42:53.068 INFO:tasks.workunit.client.1.smithi150.stdout:8/415: readlink d0/d2/d19/l33 0 2024-03-22T10:42:53.068 INFO:tasks.workunit.client.0.smithi115.stdout:6/711: getdents d4/d11/d12 0 2024-03-22T10:42:53.071 INFO:tasks.workunit.client.0.smithi115.stdout:3/801: truncate d3/d6d/db4/d107/f6f 106947 0 2024-03-22T10:42:53.072 INFO:tasks.workunit.client.0.smithi115.stdout:5/819: truncate d2/f4 7309353 0 2024-03-22T10:42:53.075 INFO:tasks.workunit.client.1.smithi150.stdout:2/434: mknod d12/d35/c91 0 2024-03-22T10:42:53.076 INFO:tasks.workunit.client.0.smithi115.stdout:2/696: truncate da/d1c/f88 3578527 0 2024-03-22T10:42:53.078 INFO:tasks.workunit.client.0.smithi115.stdout:7/796: dread d2/d7/d12/d10/d19/d42/f85 [4194304,4194304] 0 2024-03-22T10:42:53.080 INFO:tasks.workunit.client.1.smithi150.stdout:9/446: rename d1/d8/de/f9e to d1/d8/fa1 0 2024-03-22T10:42:53.081 INFO:tasks.workunit.client.0.smithi115.stdout:4/787: truncate d1/d30/d4c/d86/d9f/d75/d99/fd5 2942843 0 2024-03-22T10:42:53.081 INFO:tasks.workunit.client.0.smithi115.stdout:8/942: truncate d6/dd/d24/d4a/d7b/fba 2943818 0 2024-03-22T10:42:53.083 INFO:tasks.workunit.client.0.smithi115.stdout:3/802: mkdir d3/d36/d41/d33/d10f 0 2024-03-22T10:42:53.083 INFO:tasks.workunit.client.0.smithi115.stdout:8/943: dread - d6/f137 zero size 2024-03-22T10:42:53.084 INFO:tasks.workunit.client.1.smithi150.stdout:1/339: link d1/d19/d1a/f49 d1/da/d15/d18/d4b/f68 0 2024-03-22T10:42:53.085 INFO:tasks.workunit.client.0.smithi115.stdout:5/820: truncate d2/d15/d1b/d7c/d84/f97 8963 0 2024-03-22T10:42:53.085 INFO:tasks.workunit.client.1.smithi150.stdout:4/407: dwrite d0/d2f/f30 [4194304,4194304] 0 2024-03-22T10:42:53.085 INFO:tasks.workunit.client.0.smithi115.stdout:9/766: dread d6/d7/d30/f9d [0,4194304] 0 2024-03-22T10:42:53.086 INFO:tasks.workunit.client.1.smithi150.stdout:4/408: fdatasync d0/d47/f64 0 2024-03-22T10:42:53.087 INFO:tasks.workunit.client.1.smithi150.stdout:3/368: creat d5/d22/d2f/d38/d59/d5d/f74 x:0 0 0 2024-03-22T10:42:53.088 INFO:tasks.workunit.client.0.smithi115.stdout:8/944: chown d6/d49/d68/f6f 282366975 1 2024-03-22T10:42:53.089 INFO:tasks.workunit.client.0.smithi115.stdout:4/788: read d1/d30/f41 [2743188,86982] 0 2024-03-22T10:42:53.089 INFO:tasks.workunit.client.1.smithi150.stdout:4/409: write d0/d1b/f39 [1921096,83359] 0 2024-03-22T10:42:53.094 INFO:tasks.workunit.client.0.smithi115.stdout:9/767: write d6/d7/d48/fc2 [503479,3507] 0 2024-03-22T10:42:53.094 INFO:tasks.workunit.client.1.smithi150.stdout:0/406: write d0/d1/f39 [3089719,102869] 0 2024-03-22T10:42:53.095 INFO:tasks.workunit.client.0.smithi115.stdout:7/797: dread d2/d7/d12/d10/d19/d42/fc4 [0,4194304] 0 2024-03-22T10:42:53.095 INFO:tasks.workunit.client.0.smithi115.stdout:1/851: sync 2024-03-22T10:42:53.096 INFO:tasks.workunit.client.1.smithi150.stdout:2/435: creat d12/d16/d67/d3c/f92 x:0 0 0 2024-03-22T10:42:53.098 INFO:tasks.workunit.client.1.smithi150.stdout:2/436: fdatasync d12/d19/d2d/f82 0 2024-03-22T10:42:53.099 INFO:tasks.workunit.client.0.smithi115.stdout:4/789: creat d1/d30/d4c/d86/d9f/d75/d99/f113 x:0 0 0 2024-03-22T10:42:53.103 INFO:tasks.workunit.client.1.smithi150.stdout:1/340: creat d1/d19/d50/f69 x:0 0 0 2024-03-22T10:42:53.103 INFO:tasks.workunit.client.0.smithi115.stdout:0/723: truncate d3/da/d4c/d69/fbd 310956 0 2024-03-22T10:42:53.105 INFO:tasks.workunit.client.0.smithi115.stdout:1/852: stat d0/d12/d17/d37/d9e 0 2024-03-22T10:42:53.105 INFO:tasks.workunit.client.1.smithi150.stdout:3/369: creat d5/d53/f75 x:0 0 0 2024-03-22T10:42:53.107 INFO:tasks.workunit.client.1.smithi150.stdout:3/370: write d5/f3c [631372,51282] 0 2024-03-22T10:42:53.108 INFO:tasks.workunit.client.1.smithi150.stdout:4/410: rmdir d0/d2f/d63/d7b 39 2024-03-22T10:42:53.111 INFO:tasks.workunit.client.1.smithi150.stdout:3/371: write d5/da/f40 [1710296,1263] 0 2024-03-22T10:42:53.119 INFO:tasks.workunit.client.0.smithi115.stdout:2/697: dwrite da/d25/d2a/f72 [0,4194304] 0 2024-03-22T10:42:53.121 INFO:tasks.workunit.client.0.smithi115.stdout:1/853: mknod d0/d8/db/d25/d9f/c12e 0 2024-03-22T10:42:53.121 INFO:tasks.workunit.client.0.smithi115.stdout:3/803: write d3/d6d/db4/d107/f6f [868859,51279] 0 2024-03-22T10:42:53.122 INFO:tasks.workunit.client.0.smithi115.stdout:9/768: dread d6/d7/d48/d70/fc0 [0,4194304] 0 2024-03-22T10:42:53.123 INFO:tasks.workunit.client.1.smithi150.stdout:7/373: truncate d2/d5/d7/d8/d57/f5b 72725 0 2024-03-22T10:42:53.123 INFO:tasks.workunit.client.0.smithi115.stdout:4/790: truncate d1/dbb/dc6/dd8/df7/f10d 1000513 0 2024-03-22T10:42:53.123 INFO:tasks.workunit.client.0.smithi115.stdout:8/945: link d6/dd/d24/d4a/d7b/d8f/dd6/ddb/cdc d6/dd/d24/d4a/dfc/c13f 0 2024-03-22T10:42:53.124 INFO:tasks.workunit.client.0.smithi115.stdout:7/798: dread - d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f10c zero size 2024-03-22T10:42:53.125 INFO:tasks.workunit.client.1.smithi150.stdout:1/341: rename d1/d19/f4c to d1/da/d15/d18/d4b/f6a 0 2024-03-22T10:42:53.127 INFO:tasks.workunit.client.0.smithi115.stdout:6/712: dwrite d4/d36/d48/d72/daa/fc4 [0,4194304] 0 2024-03-22T10:42:53.130 INFO:tasks.workunit.client.0.smithi115.stdout:8/946: dread - d6/dd/d12/f11a zero size 2024-03-22T10:42:53.130 INFO:tasks.workunit.client.1.smithi150.stdout:1/342: fsync d1/d19/d57/f62 0 2024-03-22T10:42:53.132 INFO:tasks.workunit.client.0.smithi115.stdout:4/791: chown d1/d30/d4c/d86/d9f 0 1 2024-03-22T10:42:53.134 INFO:tasks.workunit.client.0.smithi115.stdout:5/821: dwrite d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f69 [4194304,4194304] 0 2024-03-22T10:42:53.134 INFO:tasks.workunit.client.1.smithi150.stdout:6/365: sync 2024-03-22T10:42:53.135 INFO:tasks.workunit.client.1.smithi150.stdout:3/372: unlink d5/f8 0 2024-03-22T10:42:53.135 INFO:tasks.workunit.client.1.smithi150.stdout:6/366: fsync db/dd/f5b 0 2024-03-22T10:42:53.137 INFO:tasks.workunit.client.0.smithi115.stdout:3/804: write d3/d82/d54/ffb [547324,80214] 0 2024-03-22T10:42:53.137 INFO:tasks.workunit.client.0.smithi115.stdout:5/822: chown d2/df/d3b 0 1 2024-03-22T10:42:53.138 INFO:tasks.workunit.client.1.smithi150.stdout:5/293: getdents d0/d8/dc/de/d14/d2e/d46/d4d 0 2024-03-22T10:42:53.139 INFO:tasks.workunit.client.1.smithi150.stdout:3/373: dread d5/d22/d2f/d38/d3e/f4d [0,4194304] 0 2024-03-22T10:42:53.139 INFO:tasks.workunit.client.0.smithi115.stdout:2/698: fdatasync da/d25/d3d/f3c 0 2024-03-22T10:42:53.139 INFO:tasks.workunit.client.0.smithi115.stdout:9/769: mknod d6/d8/d3c/d59/d68/d8e/c113 0 2024-03-22T10:42:53.140 INFO:tasks.workunit.client.0.smithi115.stdout:4/792: chown d1/d66/d106/d28/fe8 2 1 2024-03-22T10:42:53.140 INFO:tasks.workunit.client.1.smithi150.stdout:7/374: mknod d2/d5/d7/d8/d57/d54/c78 0 2024-03-22T10:42:53.140 INFO:tasks.workunit.client.1.smithi150.stdout:9/447: truncate d1/d8/f82 534814 0 2024-03-22T10:42:53.141 INFO:tasks.workunit.client.0.smithi115.stdout:1/854: mkdir d0/d12/d17/d37/d9e/de9/dc1/d12f 0 2024-03-22T10:42:53.146 INFO:tasks.workunit.client.1.smithi150.stdout:8/416: link d0/l11 d0/d2/d19/d29/d37/l8f 0 2024-03-22T10:42:53.150 INFO:tasks.workunit.client.1.smithi150.stdout:1/343: creat d1/d19/d50/d51/f6b x:0 0 0 2024-03-22T10:42:53.155 INFO:tasks.workunit.client.1.smithi150.stdout:2/437: dread d12/d16/f43 [0,4194304] 0 2024-03-22T10:42:53.156 INFO:tasks.workunit.client.1.smithi150.stdout:4/411: dread d0/d1b/f39 [0,4194304] 0 2024-03-22T10:42:53.156 INFO:tasks.workunit.client.1.smithi150.stdout:8/417: dread d0/d3c/f56 [0,4194304] 0 2024-03-22T10:42:53.158 INFO:tasks.workunit.client.1.smithi150.stdout:6/367: symlink db/dd/d15/d57/l83 0 2024-03-22T10:42:53.158 INFO:tasks.workunit.client.1.smithi150.stdout:4/412: fsync d0/de/f70 0 2024-03-22T10:42:53.159 INFO:tasks.workunit.client.1.smithi150.stdout:5/294: unlink d0/d8/ca 0 2024-03-22T10:42:53.161 INFO:tasks.workunit.client.1.smithi150.stdout:3/374: creat d5/d22/d2f/d38/d3e/d5b/f76 x:0 0 0 2024-03-22T10:42:53.166 INFO:tasks.workunit.client.1.smithi150.stdout:5/295: chown d0/d8/dc/de/d14/f16 3454 1 2024-03-22T10:42:53.166 INFO:tasks.workunit.client.0.smithi115.stdout:4/793: rmdir d1/d66/d106 39 2024-03-22T10:42:53.167 INFO:tasks.workunit.client.0.smithi115.stdout:6/713: rename d4/d11/l7e to d4/d13/d33/dbe/le6 0 2024-03-22T10:42:53.168 INFO:tasks.workunit.client.0.smithi115.stdout:0/724: write d3/f33 [202050,39286] 0 2024-03-22T10:42:53.169 INFO:tasks.workunit.client.0.smithi115.stdout:3/805: symlink d3/d36/l110 0 2024-03-22T10:42:53.172 INFO:tasks.workunit.client.1.smithi150.stdout:0/407: dwrite d0/d12/d4f/d11/d66/d35/f62 [0,4194304] 0 2024-03-22T10:42:53.174 INFO:tasks.workunit.client.0.smithi115.stdout:6/714: dread - d4/d5e/dd0/fd6 zero size 2024-03-22T10:42:53.174 INFO:tasks.workunit.client.0.smithi115.stdout:3/806: dread - d3/d19/d2b/d49/fbd zero size 2024-03-22T10:42:53.175 INFO:tasks.workunit.client.0.smithi115.stdout:1/855: link d0/d8/f106 d0/d12/d17/d37/d5c/f130 0 2024-03-22T10:42:53.176 INFO:tasks.workunit.client.1.smithi150.stdout:9/448: rename d1/d8/de/df/d24/d58 to d1/d8/de/df/da2 0 2024-03-22T10:42:53.178 INFO:tasks.workunit.client.1.smithi150.stdout:4/413: unlink d0/d1b/d4d/f53 0 2024-03-22T10:42:53.178 INFO:tasks.workunit.client.0.smithi115.stdout:9/770: link d6/d7/d48/d70/c79 d6/d7/d48/d70/da9/dbd/de9/c114 0 2024-03-22T10:42:53.179 INFO:tasks.workunit.client.0.smithi115.stdout:4/794: chown d1/d66/d106/d35/d40/d5c/l56 62952238 1 2024-03-22T10:42:53.180 INFO:tasks.workunit.client.1.smithi150.stdout:3/375: readlink d5/d22/d2f/d38/d3e/l6d 0 2024-03-22T10:42:53.183 INFO:tasks.workunit.client.1.smithi150.stdout:0/408: chown d0/d12/d4f/d42/l60 4 1 2024-03-22T10:42:53.187 INFO:tasks.workunit.client.0.smithi115.stdout:6/715: fdatasync d4/d13/d33/f6c 0 2024-03-22T10:42:53.187 INFO:tasks.workunit.client.0.smithi115.stdout:2/699: dwrite da/d25/d3d/d57/d7d/f7f [0,4194304] 0 2024-03-22T10:42:53.188 INFO:tasks.workunit.client.0.smithi115.stdout:0/725: write d3/d4/f60 [112979,43644] 0 2024-03-22T10:42:53.189 INFO:tasks.workunit.client.1.smithi150.stdout:8/418: mknod d0/d2/d19/c90 0 2024-03-22T10:42:53.191 INFO:tasks.workunit.client.0.smithi115.stdout:3/807: mkdir d3/d19/d2b/d49/d91/da5/d111 0 2024-03-22T10:42:53.192 INFO:tasks.workunit.client.0.smithi115.stdout:1/856: dread - d0/d12/d17/d37/d9e/de9/dc1/f8b zero size 2024-03-22T10:42:53.194 INFO:tasks.workunit.client.1.smithi150.stdout:5/296: rename d0/f4c to d0/d8/dc/de/d14/f58 0 2024-03-22T10:42:53.196 INFO:tasks.workunit.client.1.smithi150.stdout:6/368: truncate db/dd/d15/d60/f80 306018 0 2024-03-22T10:42:53.197 INFO:tasks.workunit.client.0.smithi115.stdout:1/857: fsync d0/d12/d17/d37/d9e/de9/dec/ded/f100 0 2024-03-22T10:42:53.197 INFO:tasks.workunit.client.0.smithi115.stdout:0/726: mkdir d3/da/d4c/d74/deb 0 2024-03-22T10:42:53.197 INFO:tasks.workunit.client.0.smithi115.stdout:3/808: creat d3/d89/f112 x:0 0 0 2024-03-22T10:42:53.198 INFO:tasks.workunit.client.1.smithi150.stdout:4/414: creat d0/d32/f8e x:0 0 0 2024-03-22T10:42:53.198 INFO:tasks.workunit.client.1.smithi150.stdout:3/376: mknod d5/d22/d37/c77 0 2024-03-22T10:42:53.201 INFO:tasks.workunit.client.0.smithi115.stdout:0/727: chown d3/d7/db7/cbb 99343 1 2024-03-22T10:42:53.201 INFO:tasks.workunit.client.1.smithi150.stdout:9/449: creat d1/d8/de/df/d36/d40/d44/fa3 x:0 0 0 2024-03-22T10:42:53.202 INFO:tasks.workunit.client.0.smithi115.stdout:3/809: dread - d3/d82/d57/fd1 zero size 2024-03-22T10:42:53.203 INFO:tasks.workunit.client.1.smithi150.stdout:3/377: read - d5/f4e zero size 2024-03-22T10:42:53.205 INFO:tasks.workunit.client.0.smithi115.stdout:1/858: rmdir d0/d8/d4d/d56/db6 39 2024-03-22T10:42:53.205 INFO:tasks.workunit.client.1.smithi150.stdout:1/344: creat d1/f6c x:0 0 0 2024-03-22T10:42:53.205 INFO:tasks.workunit.client.1.smithi150.stdout:4/415: read d0/d32/f41 [478714,12260] 0 2024-03-22T10:42:53.206 INFO:tasks.workunit.client.0.smithi115.stdout:2/700: mknod da/d25/d3d/d57/da7/ccf 0 2024-03-22T10:42:53.207 INFO:tasks.workunit.client.0.smithi115.stdout:7/799: dread d2/d7/d12/d10/f40 [0,4194304] 0 2024-03-22T10:42:53.207 INFO:tasks.workunit.client.0.smithi115.stdout:3/810: read d3/f23 [2485665,110205] 0 2024-03-22T10:42:53.207 INFO:tasks.workunit.client.1.smithi150.stdout:2/438: dread f11 [0,4194304] 0 2024-03-22T10:42:53.208 INFO:tasks.workunit.client.1.smithi150.stdout:5/297: symlink d0/d8/dc/de/d14/d4f/d47/l59 0 2024-03-22T10:42:53.208 INFO:tasks.workunit.client.1.smithi150.stdout:0/409: chown d0/d12/d4f/d42/d10/d38/d52/l55 5304 1 2024-03-22T10:42:53.209 INFO:tasks.workunit.client.0.smithi115.stdout:4/795: getdents d1/d30/d4c/d86/d9f/d4f 0 2024-03-22T10:42:53.209 INFO:tasks.workunit.client.0.smithi115.stdout:6/716: dwrite d4/d36/d48/f9e [0,4194304] 0 2024-03-22T10:42:53.209 INFO:tasks.workunit.client.1.smithi150.stdout:9/450: read d1/d49/f31 [124795,126171] 0 2024-03-22T10:42:53.210 INFO:tasks.workunit.client.1.smithi150.stdout:7/375: dread d2/d5/d7/d8/d57/d4a/f22 [0,4194304] 0 2024-03-22T10:42:53.212 INFO:tasks.workunit.client.1.smithi150.stdout:6/369: mkdir db/dd/d82/d84 0 2024-03-22T10:42:53.213 INFO:tasks.workunit.client.1.smithi150.stdout:3/378: mkdir d5/da/d78 0 2024-03-22T10:42:53.213 INFO:tasks.workunit.client.1.smithi150.stdout:4/416: chown d0/d47/l8b 3200 1 2024-03-22T10:42:53.214 INFO:tasks.workunit.client.0.smithi115.stdout:3/811: dread - d3/d6d/db4/f109 zero size 2024-03-22T10:42:53.215 INFO:tasks.workunit.client.0.smithi115.stdout:1/859: symlink d0/d8/db/d25/d9f/l131 0 2024-03-22T10:42:53.216 INFO:tasks.workunit.client.0.smithi115.stdout:4/796: stat d1/d66/d106/d35/d40/d54/cbf 0 2024-03-22T10:42:53.216 INFO:tasks.workunit.client.0.smithi115.stdout:6/717: fdatasync d4/d11/d12/d1c/f7f 0 2024-03-22T10:42:53.218 INFO:tasks.workunit.client.0.smithi115.stdout:0/728: link d3/d7/cb d3/d7/cec 0 2024-03-22T10:42:53.219 INFO:tasks.workunit.client.0.smithi115.stdout:7/800: dread - d2/d7/d12/d10/d23/d3d/da4/f10f zero size 2024-03-22T10:42:53.219 INFO:tasks.workunit.client.1.smithi150.stdout:9/451: readlink d1/d8/de/df/d20/l34 0 2024-03-22T10:42:53.219 INFO:tasks.workunit.client.1.smithi150.stdout:8/419: dread d0/d2/f14 [0,4194304] 0 2024-03-22T10:42:53.220 INFO:tasks.workunit.client.1.smithi150.stdout:7/376: write d2/d5/d7/f4d [1747377,83105] 0 2024-03-22T10:42:53.220 INFO:tasks.workunit.client.1.smithi150.stdout:4/417: write d0/d47/f7e [83623,120281] 0 2024-03-22T10:42:53.221 INFO:tasks.workunit.client.0.smithi115.stdout:3/812: unlink d3/f5b 0 2024-03-22T10:42:53.221 INFO:tasks.workunit.client.0.smithi115.stdout:2/701: getdents da/d25/d3d/d7e/d90 0 2024-03-22T10:42:53.223 INFO:tasks.workunit.client.0.smithi115.stdout:0/729: readlink d3/da/d4c/d74/d84/dac/lad 0 2024-03-22T10:42:53.224 INFO:tasks.workunit.client.0.smithi115.stdout:4/797: symlink d1/dbb/dc6/dd8/l114 0 2024-03-22T10:42:53.228 INFO:tasks.workunit.client.0.smithi115.stdout:6/718: unlink d4/d5e/lc0 0 2024-03-22T10:42:53.229 INFO:tasks.workunit.client.0.smithi115.stdout:3/813: stat d3/d82/d54/c10d 0 2024-03-22T10:42:53.230 INFO:tasks.workunit.client.1.smithi150.stdout:2/439: symlink d12/d19/d62/l93 0 2024-03-22T10:42:53.230 INFO:tasks.workunit.client.1.smithi150.stdout:1/345: symlink d1/da/l6d 0 2024-03-22T10:42:53.230 INFO:tasks.workunit.client.1.smithi150.stdout:3/379: creat d5/d22/d37/f79 x:0 0 0 2024-03-22T10:42:53.230 INFO:tasks.workunit.client.1.smithi150.stdout:5/298: mkdir d0/d8/dc/de/d14/d2e/d5a 0 2024-03-22T10:42:53.231 INFO:tasks.workunit.client.1.smithi150.stdout:2/440: write d12/d16/d67/d3c/f7f [1623777,62193] 0 2024-03-22T10:42:53.232 INFO:tasks.workunit.client.0.smithi115.stdout:2/702: rename da/d25/d3d/d57/d7d/da1/fb0 to da/d25/d83/fd0 0 2024-03-22T10:42:53.233 INFO:tasks.workunit.client.0.smithi115.stdout:3/814: chown d3/d19/d2b 10 1 2024-03-22T10:42:53.233 INFO:tasks.workunit.client.0.smithi115.stdout:9/771: dread d6/d7/d30/d3b/f46 [4194304,4194304] 0 2024-03-22T10:42:53.233 INFO:tasks.workunit.client.0.smithi115.stdout:0/730: write d3/d7/d1c/d6c/d9a/fbe [5538879,58717] 0 2024-03-22T10:42:53.234 INFO:tasks.workunit.client.1.smithi150.stdout:1/346: truncate d1/da/d15/d18/d4b/f68 5082122 0 2024-03-22T10:42:53.235 INFO:tasks.workunit.client.0.smithi115.stdout:1/860: fsync d0/d8/d4d/d56/db6/fee 0 2024-03-22T10:42:53.236 INFO:tasks.workunit.client.0.smithi115.stdout:2/703: chown da/d1c/d67/d78/fb9 88 1 2024-03-22T10:42:53.239 INFO:tasks.workunit.client.0.smithi115.stdout:3/815: write d3/f30 [1534339,110936] 0 2024-03-22T10:42:53.240 INFO:tasks.workunit.client.1.smithi150.stdout:2/441: read d12/d19/f59 [503008,102995] 0 2024-03-22T10:42:53.242 INFO:tasks.workunit.client.1.smithi150.stdout:7/377: mknod d2/d6e/c79 0 2024-03-22T10:42:53.244 INFO:tasks.workunit.client.1.smithi150.stdout:8/420: stat d0/d2/d19/d29/d2f/df/f42 0 2024-03-22T10:42:53.244 INFO:tasks.workunit.client.0.smithi115.stdout:0/731: symlink d3/d7/d1c/d6c/d87/led 0 2024-03-22T10:42:53.245 INFO:tasks.workunit.client.1.smithi150.stdout:3/380: creat d5/da/f7a x:0 0 0 2024-03-22T10:42:53.246 INFO:tasks.workunit.client.0.smithi115.stdout:7/801: dwrite d2/d7/d12/d10/f17 [4194304,4194304] 0 2024-03-22T10:42:53.253 INFO:tasks.workunit.client.0.smithi115.stdout:2/704: rename da/d25/d3d/d57/d7d/f7f to da/d25/d3d/d57/d7d/dc6/fd1 0 2024-03-22T10:42:53.253 INFO:tasks.workunit.client.0.smithi115.stdout:1/861: link d0/d8/d4d/d56/dcf/lef d0/d8/de/d1a/d91/l132 0 2024-03-22T10:42:53.253 INFO:tasks.workunit.client.0.smithi115.stdout:0/732: readlink d3/da/d4c/d74/l89 0 2024-03-22T10:42:53.255 INFO:tasks.workunit.client.0.smithi115.stdout:0/733: chown d3/d7/d6b/d94 2858 1 2024-03-22T10:42:53.256 INFO:tasks.workunit.client.0.smithi115.stdout:6/719: dwrite d4/d13/d33/f59 [0,4194304] 0 2024-03-22T10:42:53.258 INFO:tasks.workunit.client.0.smithi115.stdout:6/720: readlink d4/d13/d33/l57 0 2024-03-22T10:42:53.260 INFO:tasks.workunit.client.1.smithi150.stdout:0/410: dwrite d0/d12/d4f/d11/d26/f2e [0,4194304] 0 2024-03-22T10:42:53.262 INFO:tasks.workunit.client.1.smithi150.stdout:6/370: dwrite db/f44 [0,4194304] 0 2024-03-22T10:42:53.264 INFO:tasks.workunit.client.1.smithi150.stdout:3/381: symlink d5/d22/d37/l7b 0 2024-03-22T10:42:53.265 INFO:tasks.workunit.client.1.smithi150.stdout:5/299: creat d0/d8/dc/f5b x:0 0 0 2024-03-22T10:42:53.268 INFO:tasks.workunit.client.1.smithi150.stdout:4/418: dwrite d0/d47/f64 [0,4194304] 0 2024-03-22T10:42:53.270 INFO:tasks.workunit.client.0.smithi115.stdout:3/816: link d3/d19/d2b/d49/d91/da5/la9 d3/d36/d41/d85/db8/l113 0 2024-03-22T10:42:53.271 INFO:tasks.workunit.client.1.smithi150.stdout:7/378: mknod d2/d5/d7/d9/db/c7a 0 2024-03-22T10:42:53.272 INFO:tasks.workunit.client.1.smithi150.stdout:6/371: write db/f44 [2019031,92676] 0 2024-03-22T10:42:53.279 INFO:tasks.workunit.client.1.smithi150.stdout:3/382: mknod d5/d53/c7c 0 2024-03-22T10:42:53.284 INFO:tasks.workunit.client.1.smithi150.stdout:3/383: write d5/da/f40 [3982041,62455] 0 2024-03-22T10:42:53.285 INFO:tasks.workunit.client.1.smithi150.stdout:2/442: dwrite d12/d16/f2a [0,4194304] 0 2024-03-22T10:42:53.296 INFO:tasks.workunit.client.1.smithi150.stdout:4/419: symlink d0/d1b/d4d/l8f 0 2024-03-22T10:42:53.297 INFO:tasks.workunit.client.1.smithi150.stdout:3/384: chown d5/d22/d37/f6c 1758 1 2024-03-22T10:42:53.297 INFO:tasks.workunit.client.0.smithi115.stdout:4/798: dread d1/d69/f65 [0,4194304] 0 2024-03-22T10:42:53.300 INFO:tasks.workunit.client.1.smithi150.stdout:8/421: dwrite d0/d2/d46/f4e [0,4194304] 0 2024-03-22T10:42:53.302 INFO:tasks.workunit.client.0.smithi115.stdout:7/802: write d2/d7/d12/d10/f16 [6731210,115014] 0 2024-03-22T10:42:53.303 INFO:tasks.workunit.client.0.smithi115.stdout:4/799: truncate d1/dbb/dc6/dd8/df7/f100 210133 0 2024-03-22T10:42:53.308 INFO:tasks.workunit.client.0.smithi115.stdout:9/772: dwrite d6/d7/d87/f103 [0,4194304] 0 2024-03-22T10:42:53.312 INFO:tasks.workunit.client.1.smithi150.stdout:5/300: dread d0/d8/dc/de/d14/f50 [0,4194304] 0 2024-03-22T10:42:53.315 INFO:tasks.workunit.client.1.smithi150.stdout:6/372: mknod db/c85 0 2024-03-22T10:42:53.323 INFO:tasks.workunit.client.0.smithi115.stdout:2/705: creat da/d25/d30/fd2 x:0 0 0 2024-03-22T10:42:53.324 INFO:tasks.workunit.client.1.smithi150.stdout:7/379: mknod d2/d5/d7/c7b 0 2024-03-22T10:42:53.324 INFO:tasks.workunit.client.1.smithi150.stdout:3/385: creat d5/d22/d2f/d38/d3e/f7d x:0 0 0 2024-03-22T10:42:53.324 INFO:tasks.workunit.client.1.smithi150.stdout:8/422: chown d0/d2/d19/f78 54497740 1 2024-03-22T10:42:53.325 INFO:tasks.workunit.client.0.smithi115.stdout:6/721: creat d4/d5e/fe7 x:0 0 0 2024-03-22T10:42:53.325 INFO:tasks.workunit.client.0.smithi115.stdout:0/734: truncate d3/d7/d1c/f5d 1406114 0 2024-03-22T10:42:53.326 INFO:tasks.workunit.client.1.smithi150.stdout:5/301: mknod d0/d8/dc/de/d14/d4f/c5c 0 2024-03-22T10:42:53.329 INFO:tasks.workunit.client.1.smithi150.stdout:8/423: chown d0/d2/d19/d29/d31/f6d 393507121 1 2024-03-22T10:42:53.330 INFO:tasks.workunit.client.1.smithi150.stdout:7/380: creat d2/d5/f7c x:0 0 0 2024-03-22T10:42:53.330 INFO:tasks.workunit.client.0.smithi115.stdout:7/803: truncate d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f5c 592480 0 2024-03-22T10:42:53.330 INFO:tasks.workunit.client.0.smithi115.stdout:2/706: fsync da/d25/d3d/d57/f85 0 2024-03-22T10:42:53.331 INFO:tasks.workunit.client.1.smithi150.stdout:2/443: getdents d12/d35/d7d 0 2024-03-22T10:42:53.333 INFO:tasks.workunit.client.1.smithi150.stdout:8/424: fsync d0/f17 0 2024-03-22T10:42:53.334 INFO:tasks.workunit.client.1.smithi150.stdout:4/420: getdents d0/d1b/d68 0 2024-03-22T10:42:53.335 INFO:tasks.workunit.client.0.smithi115.stdout:4/800: dread d1/fd [0,4194304] 0 2024-03-22T10:42:53.335 INFO:tasks.workunit.client.0.smithi115.stdout:9/773: read d6/d7/d19/f31 [1249539,39980] 0 2024-03-22T10:42:53.337 INFO:tasks.workunit.client.1.smithi150.stdout:2/444: mknod d12/d16/d67/d8d/c94 0 2024-03-22T10:42:53.338 INFO:tasks.workunit.client.1.smithi150.stdout:7/381: stat d2/d5/d7/d8/d57/d54/f5f 0 2024-03-22T10:42:53.339 INFO:tasks.workunit.client.0.smithi115.stdout:4/801: symlink d1/d30/l115 0 2024-03-22T10:42:53.340 INFO:tasks.workunit.client.1.smithi150.stdout:1/347: truncate d1/d19/d1a/f2b 1587497 0 2024-03-22T10:42:53.342 INFO:tasks.workunit.client.1.smithi150.stdout:4/421: creat d0/de/f90 x:0 0 0 2024-03-22T10:42:53.348 INFO:tasks.workunit.client.0.smithi115.stdout:6/722: dwrite d4/d36/f53 [4194304,4194304] 0 2024-03-22T10:42:53.351 INFO:tasks.workunit.client.1.smithi150.stdout:4/422: chown d0/d47/l8b 18814 1 2024-03-22T10:42:53.351 INFO:tasks.workunit.client.1.smithi150.stdout:5/302: link d0/d8/c39 d0/d8/c5d 0 2024-03-22T10:42:53.351 INFO:tasks.workunit.client.1.smithi150.stdout:0/411: dwrite d0/f6d [4194304,4194304] 0 2024-03-22T10:42:53.351 INFO:tasks.workunit.client.0.smithi115.stdout:7/804: link d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/l106 d2/d7/d12/d10/d19/d42/d46/d94/l112 0 2024-03-22T10:42:53.354 INFO:tasks.workunit.client.0.smithi115.stdout:9/774: dread d6/d7/d20/d38/f56 [0,4194304] 0 2024-03-22T10:42:53.356 INFO:tasks.workunit.client.0.smithi115.stdout:4/802: dwrite d1/d30/d78/fb4 [0,4194304] 0 2024-03-22T10:42:53.357 INFO:tasks.workunit.client.1.smithi150.stdout:3/386: dwrite d5/d53/f75 [0,4194304] 0 2024-03-22T10:42:53.358 INFO:tasks.workunit.client.1.smithi150.stdout:6/373: rename db/dd/d15/d60/f80 to db/dd/d15/d60/f86 0 2024-03-22T10:42:53.359 INFO:tasks.workunit.client.0.smithi115.stdout:3/817: rmdir d3/d36 39 2024-03-22T10:42:53.362 INFO:tasks.workunit.client.0.smithi115.stdout:9/775: write d6/d7/d20/d22/d23/f83 [4096069,25747] 0 2024-03-22T10:42:53.363 INFO:tasks.workunit.client.0.smithi115.stdout:6/723: dread d4/d13/d33/f59 [0,4194304] 0 2024-03-22T10:42:53.364 INFO:tasks.workunit.client.0.smithi115.stdout:7/805: rename d2/d7/d12/d10/f26 to d2/d7/d12/d10/d19/d42/d76/d74/f113 0 2024-03-22T10:42:53.367 INFO:tasks.workunit.client.0.smithi115.stdout:3/818: write d3/d19/d2b/d49/d91/dad/fc3 [177599,16500] 0 2024-03-22T10:42:53.368 INFO:tasks.workunit.client.1.smithi150.stdout:4/423: creat d0/d32/d7c/f91 x:0 0 0 2024-03-22T10:42:53.368 INFO:tasks.workunit.client.0.smithi115.stdout:4/803: symlink d1/l116 0 2024-03-22T10:42:53.369 INFO:tasks.workunit.client.1.smithi150.stdout:9/452: sync 2024-03-22T10:42:53.369 INFO:tasks.workunit.client.1.smithi150.stdout:0/412: write d0/d12/d4f/d42/f82 [2576813,69878] 0 2024-03-22T10:42:53.370 INFO:tasks.workunit.client.0.smithi115.stdout:4/804: chown d1/d30/d4c/d86/d9f/db8/f107 12910 1 2024-03-22T10:42:53.382 INFO:tasks.workunit.client.0.smithi115.stdout:9/776: mknod d6/d8/d3c/d59/c115 0 2024-03-22T10:42:53.383 INFO:tasks.workunit.client.1.smithi150.stdout:4/424: write d0/de/f7f [607717,126295] 0 2024-03-22T10:42:53.385 INFO:tasks.workunit.client.0.smithi115.stdout:7/806: mkdir d2/d7/d114 0 2024-03-22T10:42:53.387 INFO:tasks.workunit.client.0.smithi115.stdout:3/819: dread - d3/d36/d41/f94 zero size 2024-03-22T10:42:53.388 INFO:tasks.workunit.client.0.smithi115.stdout:6/724: rename d4/d11/c96 to d4/d11/ce8 0 2024-03-22T10:42:53.393 INFO:tasks.workunit.client.0.smithi115.stdout:7/807: chown d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/fd9 41342 1 2024-03-22T10:42:53.394 INFO:tasks.workunit.client.1.smithi150.stdout:9/453: mknod d1/ca4 0 2024-03-22T10:42:53.399 INFO:tasks.workunit.client.1.smithi150.stdout:8/425: dwrite d0/d2/d19/d29/d2f/df/f42 [0,4194304] 0 2024-03-22T10:42:53.399 INFO:tasks.workunit.client.0.smithi115.stdout:9/777: rename d6/d7/d30/c44 to d6/d7/d20/d22/de5/c116 0 2024-03-22T10:42:53.401 INFO:tasks.workunit.client.0.smithi115.stdout:6/725: symlink d4/d11/le9 0 2024-03-22T10:42:53.405 INFO:tasks.workunit.client.0.smithi115.stdout:0/735: dread d3/da/d4c/d69/d6a/f86 [0,4194304] 0 2024-03-22T10:42:53.406 INFO:tasks.workunit.client.0.smithi115.stdout:6/726: chown d4/d11/d12/d1c/d8a 1 1 2024-03-22T10:42:53.407 INFO:tasks.workunit.client.1.smithi150.stdout:4/425: chown d0/d2f/d63/c7a 803 1 2024-03-22T10:42:53.409 INFO:tasks.workunit.client.1.smithi150.stdout:7/382: dwrite d2/d6e/f73 [0,4194304] 0 2024-03-22T10:42:53.411 INFO:tasks.workunit.client.0.smithi115.stdout:3/820: rmdir d3/d19/d2b/d49/d91/da5/d111 0 2024-03-22T10:42:53.411 INFO:tasks.workunit.client.1.smithi150.stdout:0/413: creat d0/f83 x:0 0 0 2024-03-22T10:42:53.415 INFO:tasks.workunit.client.0.smithi115.stdout:6/727: write d4/d13/f3d [4822587,60863] 0 2024-03-22T10:42:53.416 INFO:tasks.workunit.client.0.smithi115.stdout:0/736: unlink d3/d52/f6e 0 2024-03-22T10:42:53.418 INFO:tasks.workunit.client.1.smithi150.stdout:8/426: fdatasync d0/d2/d19/f20 0 2024-03-22T10:42:53.418 INFO:tasks.workunit.client.1.smithi150.stdout:4/426: rmdir d0/de 39 2024-03-22T10:42:53.421 INFO:tasks.workunit.client.0.smithi115.stdout:3/821: rename d3/d36/c4f to d3/d6d/db4/de1/c114 0 2024-03-22T10:42:53.424 INFO:tasks.workunit.client.0.smithi115.stdout:0/737: creat d3/d7/d1b/d93/fee x:0 0 0 2024-03-22T10:42:53.425 INFO:tasks.workunit.client.0.smithi115.stdout:3/822: read d3/d36/d41/f9a [79588,51673] 0 2024-03-22T10:42:53.426 INFO:tasks.workunit.client.1.smithi150.stdout:4/427: read d0/f5f [364997,96154] 0 2024-03-22T10:42:53.426 INFO:tasks.workunit.client.1.smithi150.stdout:7/383: stat d2/d5/d7/d9 0 2024-03-22T10:42:53.426 INFO:tasks.workunit.client.1.smithi150.stdout:1/348: dwrite d1/d19/d1a/f4d [0,4194304] 0 2024-03-22T10:42:53.427 INFO:tasks.workunit.client.0.smithi115.stdout:6/728: symlink d4/d11/d12/d1c/d8a/d98/d9f/lea 0 2024-03-22T10:42:53.430 INFO:tasks.workunit.client.0.smithi115.stdout:5/823: sync 2024-03-22T10:42:53.430 INFO:tasks.workunit.client.0.smithi115.stdout:8/947: sync 2024-03-22T10:42:53.430 INFO:tasks.workunit.client.0.smithi115.stdout:1/862: sync 2024-03-22T10:42:53.433 INFO:tasks.workunit.client.1.smithi150.stdout:5/303: dwrite d0/f2b [0,4194304] 0 2024-03-22T10:42:53.433 INFO:tasks.workunit.client.1.smithi150.stdout:6/374: dwrite db/dd/d15/f77 [0,4194304] 0 2024-03-22T10:42:53.434 INFO:tasks.workunit.client.1.smithi150.stdout:2/445: dread d12/d16/f42 [0,4194304] 0 2024-03-22T10:42:53.434 INFO:tasks.workunit.client.0.smithi115.stdout:0/738: creat d3/d7/d1b/d1f/d5b/dba/fef x:0 0 0 2024-03-22T10:42:53.437 INFO:tasks.workunit.client.1.smithi150.stdout:0/414: write d0/d12/d4f/d5a/f73 [809892,127362] 0 2024-03-22T10:42:53.437 INFO:tasks.workunit.client.1.smithi150.stdout:9/454: link d1/d19/c1d d1/d8/de/d73/ca5 0 2024-03-22T10:42:53.440 INFO:tasks.workunit.client.1.smithi150.stdout:2/446: stat d12/d16/d67/d3c 0 2024-03-22T10:42:53.442 INFO:tasks.workunit.client.0.smithi115.stdout:4/805: dread d1/d30/f4a [0,4194304] 0 2024-03-22T10:42:53.446 INFO:tasks.workunit.client.0.smithi115.stdout:2/707: truncate da/d25/d3d/d7e/d90/dbf/f8d 2944271 0 2024-03-22T10:42:53.447 INFO:tasks.workunit.client.0.smithi115.stdout:4/806: write d1/d66/d82/f8e [4285580,63045] 0 2024-03-22T10:42:53.454 INFO:tasks.workunit.client.1.smithi150.stdout:1/349: mkdir d1/d19/d50/d6e 0 2024-03-22T10:42:53.455 INFO:tasks.workunit.client.1.smithi150.stdout:4/428: rename d0/d2f/f4f to d0/d1b/d4d/f92 0 2024-03-22T10:42:53.455 INFO:tasks.workunit.client.1.smithi150.stdout:2/447: dread - d12/d16/d61/f8c zero size 2024-03-22T10:42:53.456 INFO:tasks.workunit.client.1.smithi150.stdout:0/415: rmdir d0/d12/d4f/d11/d66/d35/d5f 39 2024-03-22T10:42:53.458 INFO:tasks.workunit.client.0.smithi115.stdout:2/708: mknod da/d1c/cd3 0 2024-03-22T10:42:53.458 INFO:tasks.workunit.client.0.smithi115.stdout:3/823: read d3/d16/d29/f32 [1194370,17917] 0 2024-03-22T10:42:53.459 INFO:tasks.workunit.client.1.smithi150.stdout:9/455: chown d1/d19/l90 45719002 1 2024-03-22T10:42:53.460 INFO:tasks.workunit.client.0.smithi115.stdout:8/948: dwrite d6/dd/d24/d4a/d7b/fe0 [0,4194304] 0 2024-03-22T10:42:53.462 INFO:tasks.workunit.client.1.smithi150.stdout:1/350: mknod d1/d19/c6f 0 2024-03-22T10:42:53.465 INFO:tasks.workunit.client.1.smithi150.stdout:2/448: rmdir d12/d16/d67/d3c 39 2024-03-22T10:42:53.465 INFO:tasks.workunit.client.0.smithi115.stdout:7/808: truncate d2/d7/d12/d10/d19/d42/d76/f98 4140708 0 2024-03-22T10:42:53.466 INFO:tasks.workunit.client.0.smithi115.stdout:9/778: unlink d6/d7/d20/d22/de5/c116 0 2024-03-22T10:42:53.467 INFO:tasks.workunit.client.1.smithi150.stdout:3/387: dread d5/d22/d2f/d38/f42 [0,4194304] 0 2024-03-22T10:42:53.468 INFO:tasks.workunit.client.0.smithi115.stdout:2/709: creat da/d25/d51/fd4 x:0 0 0 2024-03-22T10:42:53.469 INFO:tasks.workunit.client.0.smithi115.stdout:6/729: truncate d4/d36/d48/f9e 73308 0 2024-03-22T10:42:53.470 INFO:tasks.workunit.client.0.smithi115.stdout:5/824: dwrite d2/d15/d1b/feb [0,4194304] 0 2024-03-22T10:42:53.471 INFO:tasks.workunit.client.1.smithi150.stdout:3/388: readlink d5/d53/l65 0 2024-03-22T10:42:53.471 INFO:tasks.workunit.client.1.smithi150.stdout:6/375: dwrite db/f4a [0,4194304] 0 2024-03-22T10:42:53.471 INFO:tasks.workunit.client.1.smithi150.stdout:1/351: read d1/d19/d1a/f49 [2542077,68729] 0 2024-03-22T10:42:53.473 INFO:tasks.workunit.client.0.smithi115.stdout:0/739: rename d3/d7/d1b/d1f/d5b/dba to d3/df0 0 2024-03-22T10:42:53.476 INFO:tasks.workunit.client.1.smithi150.stdout:9/456: readlink d1/d19/l21 0 2024-03-22T10:42:53.479 INFO:tasks.workunit.client.0.smithi115.stdout:9/779: symlink d6/d7/d19/l117 0 2024-03-22T10:42:53.481 INFO:tasks.workunit.client.1.smithi150.stdout:2/449: symlink d12/d21/l95 0 2024-03-22T10:42:53.483 INFO:tasks.workunit.client.1.smithi150.stdout:5/304: dread d0/f5 [4194304,4194304] 0 2024-03-22T10:42:53.486 INFO:tasks.workunit.client.0.smithi115.stdout:6/730: dread - d4/d11/d12/d1c/d8a/d98/fc5 zero size 2024-03-22T10:42:53.486 INFO:tasks.workunit.client.0.smithi115.stdout:0/740: creat d3/d4/d6d/ff1 x:0 0 0 2024-03-22T10:42:53.486 INFO:tasks.workunit.client.0.smithi115.stdout:4/807: rename d1/d66/d106/d35/d40/d5c/d55/f52 to d1/d30/d4c/d86/d9f/d75/d99/d98/d9b/f117 0 2024-03-22T10:42:53.486 INFO:tasks.workunit.client.1.smithi150.stdout:0/416: fsync d0/d12/d4f/d42/d10/d38/f4c 0 2024-03-22T10:42:53.487 INFO:tasks.workunit.client.1.smithi150.stdout:2/450: chown d12/d16/f43 9939283 1 2024-03-22T10:42:53.487 INFO:tasks.workunit.client.0.smithi115.stdout:5/825: truncate d2/d15/ff3 96073 0 2024-03-22T10:42:53.487 INFO:tasks.workunit.client.1.smithi150.stdout:6/376: chown db/dd/c12 91427 1 2024-03-22T10:42:53.489 INFO:tasks.workunit.client.1.smithi150.stdout:0/417: fsync d0/f6d 0 2024-03-22T10:42:53.490 INFO:tasks.workunit.client.0.smithi115.stdout:0/741: read - d3/df0/fef zero size 2024-03-22T10:42:53.492 INFO:tasks.workunit.client.1.smithi150.stdout:8/427: truncate d0/fb 2134604 0 2024-03-22T10:42:53.494 INFO:tasks.workunit.client.0.smithi115.stdout:4/808: chown d1/c5 3003359 1 2024-03-22T10:42:53.495 INFO:tasks.workunit.client.1.smithi150.stdout:4/429: rename d0/d2f/f30 to d0/f93 0 2024-03-22T10:42:53.496 INFO:tasks.workunit.client.0.smithi115.stdout:9/780: write d6/d7/d19/d4e/fd7 [773391,60936] 0 2024-03-22T10:42:53.496 INFO:tasks.workunit.client.1.smithi150.stdout:3/389: mkdir d5/d53/d7e 0 2024-03-22T10:42:53.499 INFO:tasks.workunit.client.0.smithi115.stdout:4/809: dread - d1/d30/d4c/d86/d9f/db8/f107 zero size 2024-03-22T10:42:53.500 INFO:tasks.workunit.client.0.smithi115.stdout:3/824: write d3/d19/dd8/fb9 [294222,96485] 0 2024-03-22T10:42:53.502 INFO:tasks.workunit.client.0.smithi115.stdout:0/742: symlink d3/d52/lf2 0 2024-03-22T10:42:53.505 INFO:tasks.workunit.client.1.smithi150.stdout:8/428: read d0/d2/d19/d29/d2f/df/d1b/f1d [523394,21956] 0 2024-03-22T10:42:53.505 INFO:tasks.workunit.client.1.smithi150.stdout:6/377: creat db/dd/d82/d84/f87 x:0 0 0 2024-03-22T10:42:53.510 INFO:tasks.workunit.client.0.smithi115.stdout:6/731: dread d4/d36/d48/f9e [0,4194304] 0 2024-03-22T10:42:53.512 INFO:tasks.workunit.client.1.smithi150.stdout:3/390: creat d5/d22/d2f/d38/d3e/d5b/f7f x:0 0 0 2024-03-22T10:42:53.512 INFO:tasks.workunit.client.1.smithi150.stdout:9/457: dread d1/d8/de/df/d20/f5d [0,4194304] 0 2024-03-22T10:42:53.515 INFO:tasks.workunit.client.0.smithi115.stdout:4/810: mknod d1/d66/d106/d35/d40/d5c/d55/c118 0 2024-03-22T10:42:53.516 INFO:tasks.workunit.client.0.smithi115.stdout:8/949: truncate d6/dd/d12/d13c/dfd/f127 994114 0 2024-03-22T10:42:53.518 INFO:tasks.workunit.client.1.smithi150.stdout:2/451: dread d12/f26 [0,4194304] 0 2024-03-22T10:42:53.519 INFO:tasks.workunit.client.1.smithi150.stdout:5/305: dread d0/d8/dc/f12 [0,4194304] 0 2024-03-22T10:42:53.520 INFO:tasks.workunit.client.0.smithi115.stdout:0/743: creat d3/d7/d1c/d6c/d87/dd6/ff3 x:0 0 0 2024-03-22T10:42:53.520 INFO:tasks.workunit.client.0.smithi115.stdout:8/950: fsync d6/d49/d68/f9c 0 2024-03-22T10:42:53.523 INFO:tasks.workunit.client.1.smithi150.stdout:9/458: fsync d1/d8/de/df/d20/f5d 0 2024-03-22T10:42:53.529 INFO:tasks.workunit.client.1.smithi150.stdout:6/378: truncate db/dd/d15/f35 1667212 0 2024-03-22T10:42:53.531 INFO:tasks.workunit.client.1.smithi150.stdout:2/452: write d12/d19/d1a/f2b [500005,59692] 0 2024-03-22T10:42:53.533 INFO:tasks.workunit.client.0.smithi115.stdout:8/951: chown d6/dd/d24/d3a/l6b 114 1 2024-03-22T10:42:53.538 INFO:tasks.workunit.client.0.smithi115.stdout:3/825: write d3/d82/f64 [572055,33566] 0 2024-03-22T10:42:53.539 INFO:tasks.workunit.client.0.smithi115.stdout:7/809: write d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/dbb/f7a [923511,116734] 0 2024-03-22T10:42:53.540 INFO:tasks.workunit.client.1.smithi150.stdout:6/379: symlink db/dd/d15/d60/d50/l88 0 2024-03-22T10:42:53.541 INFO:tasks.workunit.client.1.smithi150.stdout:4/430: dwrite d0/de/f81 [0,4194304] 0 2024-03-22T10:42:53.543 INFO:tasks.workunit.client.0.smithi115.stdout:0/744: write d3/d7/d1b/d35/f79 [681744,44196] 0 2024-03-22T10:42:53.543 INFO:tasks.workunit.client.0.smithi115.stdout:6/732: dwrite d4/d13/d33/dbe/fd9 [0,4194304] 0 2024-03-22T10:42:53.546 INFO:tasks.workunit.client.0.smithi115.stdout:7/810: dread - d2/d7/d12/fdc zero size 2024-03-22T10:42:53.547 INFO:tasks.workunit.client.1.smithi150.stdout:4/431: write d0/d1b/f2b [1649070,72157] 0 2024-03-22T10:42:53.549 INFO:tasks.workunit.client.1.smithi150.stdout:5/306: mknod d0/d8/dc/de/d14/d2e/d45/d56/c5e 0 2024-03-22T10:42:53.551 INFO:tasks.workunit.client.1.smithi150.stdout:9/459: creat d1/d8/d95/fa6 x:0 0 0 2024-03-22T10:42:53.551 INFO:tasks.workunit.client.0.smithi115.stdout:3/826: write d3/d82/f83 [1160634,47565] 0 2024-03-22T10:42:53.552 INFO:tasks.workunit.client.1.smithi150.stdout:2/453: symlink d12/d19/d62/l96 0 2024-03-22T10:42:53.552 INFO:tasks.workunit.client.1.smithi150.stdout:0/418: dwrite d0/d12/d4f/f6b [0,4194304] 0 2024-03-22T10:42:53.556 INFO:tasks.workunit.client.0.smithi115.stdout:3/827: chown d3/d82 461099 1 2024-03-22T10:42:53.558 INFO:tasks.workunit.client.1.smithi150.stdout:1/352: dread d1/d19/d1a/f49 [0,4194304] 0 2024-03-22T10:42:53.559 INFO:tasks.workunit.client.0.smithi115.stdout:6/733: mknod d4/d11/d91/ceb 0 2024-03-22T10:42:53.560 INFO:tasks.workunit.client.1.smithi150.stdout:9/460: truncate d1/d8/de/df/d36/d40/f78 422501 0 2024-03-22T10:42:53.560 INFO:tasks.workunit.client.0.smithi115.stdout:4/811: dwrite d1/d66/d106/d35/d40/d5c/d14/f1b [0,4194304] 0 2024-03-22T10:42:53.560 INFO:tasks.workunit.client.0.smithi115.stdout:5/826: dwrite d2/f5e [0,4194304] 0 2024-03-22T10:42:53.562 INFO:tasks.workunit.client.0.smithi115.stdout:0/745: truncate d3/d7/d1b/d93/fcc 880462 0 2024-03-22T10:42:53.563 INFO:tasks.workunit.client.1.smithi150.stdout:5/307: symlink d0/d8/dc/de/d14/d4f/l5f 0 2024-03-22T10:42:53.563 INFO:tasks.workunit.client.1.smithi150.stdout:1/353: chown d1/d19/d57/f62 5845 1 2024-03-22T10:42:53.563 INFO:tasks.workunit.client.1.smithi150.stdout:3/391: link d5/l16 d5/d53/l80 0 2024-03-22T10:42:53.564 INFO:tasks.workunit.client.0.smithi115.stdout:3/828: chown d3/d82/d52/l72 66719515 1 2024-03-22T10:42:53.564 INFO:tasks.workunit.client.0.smithi115.stdout:4/812: stat d1/d66/d82/f9a 0 2024-03-22T10:42:53.564 INFO:tasks.workunit.client.1.smithi150.stdout:2/454: rename d12/d19/d2d/d68 to d12/d16/d97 0 2024-03-22T10:42:53.567 INFO:tasks.workunit.client.0.smithi115.stdout:2/710: dread da/d25/d30/f2e [0,4194304] 0 2024-03-22T10:42:53.567 INFO:tasks.workunit.client.0.smithi115.stdout:7/811: creat d2/d7/d12/d10/d19/d42/d46/f115 x:0 0 0 2024-03-22T10:42:53.567 INFO:tasks.workunit.client.1.smithi150.stdout:6/380: symlink db/dd/d15/d28/d6c/l89 0 2024-03-22T10:42:53.575 INFO:tasks.workunit.client.1.smithi150.stdout:5/308: write d0/d8/dc/f12 [3518964,23608] 0 2024-03-22T10:42:53.575 INFO:tasks.workunit.client.0.smithi115.stdout:2/711: chown da/d1c/f48 1 1 2024-03-22T10:42:53.578 INFO:tasks.workunit.client.1.smithi150.stdout:7/384: sync 2024-03-22T10:42:53.579 INFO:tasks.workunit.client.1.smithi150.stdout:6/381: chown db/dd/d15/d28/l5d 72789 1 2024-03-22T10:42:53.580 INFO:tasks.workunit.client.0.smithi115.stdout:4/813: creat d1/dbb/dc6/f119 x:0 0 0 2024-03-22T10:42:53.580 INFO:tasks.workunit.client.1.smithi150.stdout:5/309: truncate d0/d8/f34 284481 0 2024-03-22T10:42:53.581 INFO:tasks.workunit.client.0.smithi115.stdout:2/712: mknod da/d25/d3d/d7e/cd5 0 2024-03-22T10:42:53.585 INFO:tasks.workunit.client.1.smithi150.stdout:8/429: dread d0/d2/d19/d29/d2f/f64 [0,4194304] 0 2024-03-22T10:42:53.586 INFO:tasks.workunit.client.0.smithi115.stdout:6/734: creat d4/fec x:0 0 0 2024-03-22T10:42:53.587 INFO:tasks.workunit.client.0.smithi115.stdout:8/952: link d6/dd/d24/la9 d6/dd/d24/d4a/d7b/d8f/dd6/l140 0 2024-03-22T10:42:53.588 INFO:tasks.workunit.client.1.smithi150.stdout:5/310: write d0/d8/dc/de/d14/d2e/f3e [3135936,125881] 0 2024-03-22T10:42:53.589 INFO:tasks.workunit.client.0.smithi115.stdout:2/713: write da/d1c/d67/d78/d86/db4/fc3 [783358,122315] 0 2024-03-22T10:42:53.590 INFO:tasks.workunit.client.0.smithi115.stdout:4/814: mkdir d1/d30/d4c/d86/d9f/d11a 0 2024-03-22T10:42:53.591 INFO:tasks.workunit.client.0.smithi115.stdout:8/953: stat d6/d49/d88/f8a 0 2024-03-22T10:42:53.592 INFO:tasks.workunit.client.1.smithi150.stdout:7/385: creat d2/d5/d7/d8/d2c/d45/f7d x:0 0 0 2024-03-22T10:42:53.593 INFO:tasks.workunit.client.0.smithi115.stdout:6/735: chown d4/d5e/dd0/fd6 948 1 2024-03-22T10:42:53.595 INFO:tasks.workunit.client.1.smithi150.stdout:4/432: link d0/de/l19 d0/d2f/d63/d7b/l94 0 2024-03-22T10:42:53.597 INFO:tasks.workunit.client.0.smithi115.stdout:9/781: dread d6/d7/f16 [0,4194304] 0 2024-03-22T10:42:53.598 INFO:tasks.workunit.client.1.smithi150.stdout:1/354: creat d1/f70 x:0 0 0 2024-03-22T10:42:53.598 INFO:tasks.workunit.client.0.smithi115.stdout:4/815: fdatasync d1/d30/f4a 0 2024-03-22T10:42:53.599 INFO:tasks.workunit.client.1.smithi150.stdout:4/433: chown d0/d2f/f3b 48837159 1 2024-03-22T10:42:53.600 INFO:tasks.workunit.client.1.smithi150.stdout:6/382: read db/dd/d15/d60/f86 [131116,96842] 0 2024-03-22T10:42:53.601 INFO:tasks.workunit.client.1.smithi150.stdout:8/430: creat d0/d2/d19/d29/d88/f91 x:0 0 0 2024-03-22T10:42:53.605 INFO:tasks.workunit.client.0.smithi115.stdout:8/954: chown d6/dd/d12/de3/ff0 36117 1 2024-03-22T10:42:53.608 INFO:tasks.workunit.client.0.smithi115.stdout:3/829: dwrite d3/d19/d2b/d49/fc6 [4194304,4194304] 0 2024-03-22T10:42:53.608 INFO:tasks.workunit.client.0.smithi115.stdout:0/746: dwrite d3/d7/d1c/f56 [4194304,4194304] 0 2024-03-22T10:42:53.611 INFO:tasks.workunit.client.1.smithi150.stdout:6/383: unlink db/l1d 0 2024-03-22T10:42:53.613 INFO:tasks.workunit.client.1.smithi150.stdout:2/455: rename d12/d19/d62 to d12/d16/d61/d98 0 2024-03-22T10:42:53.614 INFO:tasks.workunit.client.0.smithi115.stdout:5/827: dwrite d2/df/d13/d8b/d93/da6/dbc/d7a/f98 [0,4194304] 0 2024-03-22T10:42:53.615 INFO:tasks.workunit.client.0.smithi115.stdout:8/955: write d6/ff4 [4240229,87349] 0 2024-03-22T10:42:53.616 INFO:tasks.workunit.client.1.smithi150.stdout:1/355: mkdir d1/d19/d71 0 2024-03-22T10:42:53.616 INFO:tasks.workunit.client.0.smithi115.stdout:6/736: rename d4/d11/d12/d1c/d8a/d98/fc5 to d4/d11/d12/d1c/d8a/d98/fed 0 2024-03-22T10:42:53.616 INFO:tasks.workunit.client.0.smithi115.stdout:4/816: mknod d1/d66/d106/d35/d40/d5c/d55/c11b 0 2024-03-22T10:42:53.620 INFO:tasks.workunit.client.0.smithi115.stdout:7/812: dwrite d2/d7/d12/d10/d19/f1a [4194304,4194304] 0 2024-03-22T10:42:53.621 INFO:tasks.workunit.client.1.smithi150.stdout:2/456: write d12/d16/d61/f86 [286219,21871] 0 2024-03-22T10:42:53.621 INFO:tasks.workunit.client.0.smithi115.stdout:3/830: creat d3/d19/dd8/f115 x:0 0 0 2024-03-22T10:42:53.622 INFO:tasks.workunit.client.1.smithi150.stdout:5/311: dread d0/d8/dc/de/d14/f16 [0,4194304] 0 2024-03-22T10:42:53.623 INFO:tasks.workunit.client.0.smithi115.stdout:8/956: write f5 [3398716,126699] 0 2024-03-22T10:42:53.626 INFO:tasks.workunit.client.0.smithi115.stdout:4/817: write d1/d66/d82/f9a [314446,117723] 0 2024-03-22T10:42:53.626 INFO:tasks.workunit.client.1.smithi150.stdout:7/386: rename d2/d5/d7/d8/f2d to d2/d6e/f7e 0 2024-03-22T10:42:53.626 INFO:tasks.workunit.client.1.smithi150.stdout:6/384: mknod db/dd/d82/d84/c8a 0 2024-03-22T10:42:53.627 INFO:tasks.workunit.client.1.smithi150.stdout:4/434: dwrite d0/d2f/f76 [0,4194304] 0 2024-03-22T10:42:53.629 INFO:tasks.workunit.client.0.smithi115.stdout:5/828: read - d2/df/d13/f76 zero size 2024-03-22T10:42:53.630 INFO:tasks.workunit.client.0.smithi115.stdout:6/737: mkdir d4/d11/d91/dee 0 2024-03-22T10:42:53.631 INFO:tasks.workunit.client.1.smithi150.stdout:4/435: readlink d0/l15 0 2024-03-22T10:42:53.631 INFO:tasks.workunit.client.1.smithi150.stdout:8/431: symlink d0/d2/d19/d29/d2f/df/l92 0 2024-03-22T10:42:53.634 INFO:tasks.workunit.client.1.smithi150.stdout:7/387: truncate d2/d5/d7/d8/d2c/f6f 968630 0 2024-03-22T10:42:53.636 INFO:tasks.workunit.client.1.smithi150.stdout:2/457: unlink d12/d19/f58 0 2024-03-22T10:42:53.637 INFO:tasks.workunit.client.0.smithi115.stdout:2/714: write da/f15 [4003094,92540] 0 2024-03-22T10:42:53.637 INFO:tasks.workunit.client.0.smithi115.stdout:3/831: truncate d3/d19/d2b/d49/fc6 9093562 0 2024-03-22T10:42:53.638 INFO:tasks.workunit.client.1.smithi150.stdout:5/312: symlink d0/d37/l60 0 2024-03-22T10:42:53.638 INFO:tasks.workunit.client.0.smithi115.stdout:7/813: mkdir d2/d7/d12/d10/d23/d3d/da4/dc0/d116 0 2024-03-22T10:42:53.640 INFO:tasks.workunit.client.1.smithi150.stdout:8/432: fdatasync d0/d2/d19/d29/d2f/df/f8a 0 2024-03-22T10:42:53.640 INFO:tasks.workunit.client.0.smithi115.stdout:7/814: fsync d2/d7/d12/fb8 0 2024-03-22T10:42:53.645 INFO:tasks.workunit.client.0.smithi115.stdout:4/818: unlink d1/d30/d78/fee 0 2024-03-22T10:42:53.645 INFO:tasks.workunit.client.0.smithi115.stdout:2/715: stat da/d25/d3d/d7e/d90/dbf/d54/c94 0 2024-03-22T10:42:53.645 INFO:tasks.workunit.client.0.smithi115.stdout:2/716: readlink da/d25/d2a/l32 0 2024-03-22T10:42:53.645 INFO:tasks.workunit.client.0.smithi115.stdout:4/819: read - d1/d66/d82/ffe zero size 2024-03-22T10:42:53.645 INFO:tasks.workunit.client.0.smithi115.stdout:5/829: mkdir d2/d15/d1b/d7c/dee/d10c 0 2024-03-22T10:42:53.647 INFO:tasks.workunit.client.0.smithi115.stdout:8/957: truncate d6/dd/d24/d4a/f5f 411956 0 2024-03-22T10:42:53.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:53 smithi150 ceph-mon[34518]: pgmap v15: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 54 MiB/s rd, 202 MiB/s wr, 356 op/s 2024-03-22T10:42:53.650 INFO:tasks.workunit.client.1.smithi150.stdout:4/436: mknod d0/d1b/d68/c95 0 2024-03-22T10:42:53.650 INFO:tasks.workunit.client.1.smithi150.stdout:2/458: dread - d12/d16/d67/d3c/f92 zero size 2024-03-22T10:42:53.651 INFO:tasks.workunit.client.0.smithi115.stdout:5/830: write d2/df/d3b/fa0 [1055693,77197] 0 2024-03-22T10:42:53.651 INFO:tasks.workunit.client.0.smithi115.stdout:2/717: creat da/da5/fd6 x:0 0 0 2024-03-22T10:42:53.652 INFO:tasks.workunit.client.1.smithi150.stdout:6/385: mkdir db/dd/d15/d60/d8b 0 2024-03-22T10:42:53.655 INFO:tasks.workunit.client.0.smithi115.stdout:8/958: chown d6/dd/d24/dc1/ldd 1746375555 1 2024-03-22T10:42:53.656 INFO:tasks.workunit.client.1.smithi150.stdout:8/433: unlink d0/fb 0 2024-03-22T10:42:53.659 INFO:tasks.workunit.client.1.smithi150.stdout:2/459: chown d12/d16/d67/d3c/d75/c8a 102024056 1 2024-03-22T10:42:53.662 INFO:tasks.workunit.client.1.smithi150.stdout:4/437: write d0/d32/f51 [730657,89678] 0 2024-03-22T10:42:53.663 INFO:tasks.workunit.client.0.smithi115.stdout:3/832: link d3/d16/d29/d35/l65 d3/d16/d29/d35/l116 0 2024-03-22T10:42:53.663 INFO:tasks.workunit.client.0.smithi115.stdout:6/738: dwrite d4/d13/f3d [0,4194304] 0 2024-03-22T10:42:53.663 INFO:tasks.workunit.client.0.smithi115.stdout:8/959: fdatasync d6/dd/fce 0 2024-03-22T10:42:53.665 INFO:tasks.workunit.client.1.smithi150.stdout:9/461: dread d1/d8/de/f17 [0,4194304] 0 2024-03-22T10:42:53.668 INFO:tasks.workunit.client.1.smithi150.stdout:2/460: readlink d12/d19/d2d/l3a 0 2024-03-22T10:42:53.669 INFO:tasks.workunit.client.1.smithi150.stdout:6/386: truncate db/dd/d15/d30/f51 1243966 0 2024-03-22T10:42:53.669 INFO:tasks.workunit.client.0.smithi115.stdout:5/831: symlink d2/d15/d1b/d7c/l10d 0 2024-03-22T10:42:53.671 INFO:tasks.workunit.client.0.smithi115.stdout:8/960: chown d6/dd/d12/f3b 74 1 2024-03-22T10:42:53.673 INFO:tasks.workunit.client.0.smithi115.stdout:3/833: symlink d3/d82/d54/de7/d104/l117 0 2024-03-22T10:42:53.673 INFO:tasks.workunit.client.0.smithi115.stdout:5/832: chown d2/df/d13/d68/d106/d10b 6290 1 2024-03-22T10:42:53.674 INFO:tasks.workunit.client.0.smithi115.stdout:2/718: link da/d1c/d67/c81 da/d25/d3d/cd7 0 2024-03-22T10:42:53.674 INFO:tasks.workunit.client.0.smithi115.stdout:6/739: creat d4/d11/d12/d1c/d8a/d98/fef x:0 0 0 2024-03-22T10:42:53.675 INFO:tasks.workunit.client.1.smithi150.stdout:5/313: dread d0/d8/dc/de/f13 [0,4194304] 0 2024-03-22T10:42:53.677 INFO:tasks.workunit.client.0.smithi115.stdout:8/961: chown d6/fad 18675071 1 2024-03-22T10:42:53.680 INFO:tasks.workunit.client.1.smithi150.stdout:4/438: stat d0/de/f70 0 2024-03-22T10:42:53.680 INFO:tasks.workunit.client.0.smithi115.stdout:4/820: dread d1/d66/d106/d35/d40/d54/f67 [0,4194304] 0 2024-03-22T10:42:53.680 INFO:tasks.workunit.client.1.smithi150.stdout:6/387: chown db/dd/d25 5988329 1 2024-03-22T10:42:53.681 INFO:tasks.workunit.client.1.smithi150.stdout:2/461: readlink d12/d21/l95 0 2024-03-22T10:42:53.682 INFO:tasks.workunit.client.0.smithi115.stdout:3/834: write d3/fd2 [5444022,23797] 0 2024-03-22T10:42:53.684 INFO:tasks.workunit.client.1.smithi150.stdout:0/419: dread d0/d1/f39 [0,4194304] 0 2024-03-22T10:42:53.685 INFO:tasks.workunit.client.0.smithi115.stdout:3/835: chown d3/d16/d29/f3c 1197350 1 2024-03-22T10:42:53.685 INFO:tasks.workunit.client.0.smithi115.stdout:0/747: truncate d3/da/d4c/d74/fb6 2558162 0 2024-03-22T10:42:53.687 INFO:tasks.workunit.client.0.smithi115.stdout:6/740: mkdir d4/d11/d12/d1c/d8a/d98/d9f/df0 0 2024-03-22T10:42:53.690 INFO:tasks.workunit.client.0.smithi115.stdout:9/782: dread d6/d7/d87/f8c [0,4194304] 0 2024-03-22T10:42:53.691 INFO:tasks.workunit.client.0.smithi115.stdout:8/962: creat d6/dd/d24/d4a/d7b/d8f/dd6/ddb/f141 x:0 0 0 2024-03-22T10:42:53.692 INFO:tasks.workunit.client.0.smithi115.stdout:4/821: symlink d1/d30/d4c/d86/d9f/l11c 0 2024-03-22T10:42:53.692 INFO:tasks.workunit.client.1.smithi150.stdout:3/392: dread d5/f35 [0,4194304] 0 2024-03-22T10:42:53.692 INFO:tasks.workunit.client.0.smithi115.stdout:8/963: chown d6/fd2 18248246 1 2024-03-22T10:42:53.693 INFO:tasks.workunit.client.1.smithi150.stdout:8/434: link d0/d2/c6b d0/d2/d19/d29/d2e/c93 0 2024-03-22T10:42:53.694 INFO:tasks.workunit.client.0.smithi115.stdout:5/833: rename d2/df/d13/d68 to d2/df/d13/d8b/d93/da6/d10e 0 2024-03-22T10:42:53.694 INFO:tasks.workunit.client.1.smithi150.stdout:4/439: chown d0/d32/l3f 6909 1 2024-03-22T10:42:53.695 INFO:tasks.workunit.client.1.smithi150.stdout:1/356: dread d1/d19/d1a/f1e [0,4194304] 0 2024-03-22T10:42:53.695 INFO:tasks.workunit.client.0.smithi115.stdout:8/964: write d6/f137 [725652,73997] 0 2024-03-22T10:42:53.696 INFO:tasks.workunit.client.0.smithi115.stdout:3/836: symlink d3/d82/d54/de7/l118 0 2024-03-22T10:42:53.698 INFO:tasks.workunit.client.1.smithi150.stdout:7/388: dwrite d2/d5/d7/f4d [4194304,4194304] 0 2024-03-22T10:42:53.700 INFO:tasks.workunit.client.0.smithi115.stdout:6/741: creat d4/d11/d12/d1c/d8a/d98/d9f/dc3/ff1 x:0 0 0 2024-03-22T10:42:53.701 INFO:tasks.workunit.client.1.smithi150.stdout:5/314: dread d0/d8/dc/de/d14/f58 [0,4194304] 0 2024-03-22T10:42:53.701 INFO:tasks.workunit.client.1.smithi150.stdout:6/388: dread db/f4f [0,4194304] 0 2024-03-22T10:42:53.701 INFO:tasks.workunit.client.1.smithi150.stdout:9/462: rename d1/d8/de/df/d36/d40/d6c to d1/d8/de/df/d20/d32/d75/d99/da7 0 2024-03-22T10:42:53.702 INFO:tasks.workunit.client.0.smithi115.stdout:3/837: symlink d3/d89/dca/l119 0 2024-03-22T10:42:53.702 INFO:tasks.workunit.client.1.smithi150.stdout:4/440: truncate d0/f11 5179701 0 2024-03-22T10:42:53.704 INFO:tasks.workunit.client.0.smithi115.stdout:8/965: unlink d6/dd/d12/d31/l7d 0 2024-03-22T10:42:53.706 INFO:tasks.workunit.client.1.smithi150.stdout:0/420: read d0/d12/d4f/d11/d26/f2e [5826597,5687] 0 2024-03-22T10:42:53.707 INFO:tasks.workunit.client.1.smithi150.stdout:6/389: read db/f4a [3853643,60556] 0 2024-03-22T10:42:53.707 INFO:tasks.workunit.client.1.smithi150.stdout:3/393: mkdir d5/d22/d2f/d38/d59/d5d/d81 0 2024-03-22T10:42:53.709 INFO:tasks.workunit.client.1.smithi150.stdout:4/441: creat d0/d47/f96 x:0 0 0 2024-03-22T10:42:53.710 INFO:tasks.workunit.client.0.smithi115.stdout:8/966: rename d6/dd/d24/d4a/d139 to d6/dd/d24/d4a/d139/d142 22 2024-03-22T10:42:53.710 INFO:tasks.workunit.client.1.smithi150.stdout:8/435: stat d0/d2/d19/d29/d2f/df/d1b/f1d 0 2024-03-22T10:42:53.711 INFO:tasks.workunit.client.0.smithi115.stdout:6/742: truncate d4/d11/d91/fb0 507942 0 2024-03-22T10:42:53.712 INFO:tasks.workunit.client.1.smithi150.stdout:5/315: dread - d0/d8/dc/de/d14/d2e/f54 zero size 2024-03-22T10:42:53.713 INFO:tasks.workunit.client.0.smithi115.stdout:3/838: mknod d3/d16/c11a 0 2024-03-22T10:42:53.717 INFO:tasks.workunit.client.0.smithi115.stdout:6/743: read - d4/d5e/dd0/fd6 zero size 2024-03-22T10:42:53.717 INFO:tasks.workunit.client.1.smithi150.stdout:9/463: symlink d1/d8/de/df/la8 0 2024-03-22T10:42:53.718 INFO:tasks.workunit.client.1.smithi150.stdout:3/394: dread d5/da/f34 [0,4194304] 0 2024-03-22T10:42:53.720 INFO:tasks.workunit.client.0.smithi115.stdout:5/834: dwrite d2/d15/d1b/d7c/d38/d43/d49/fb9 [4194304,4194304] 0 2024-03-22T10:42:53.721 INFO:tasks.workunit.client.1.smithi150.stdout:6/390: truncate db/dd/d2e/f81 850506 0 2024-03-22T10:42:53.721 INFO:tasks.workunit.client.1.smithi150.stdout:2/462: dwrite d12/d16/d67/d3c/f77 [0,4194304] 0 2024-03-22T10:42:53.726 INFO:tasks.workunit.client.0.smithi115.stdout:2/719: dwrite da/d1c/f48 [0,4194304] 0 2024-03-22T10:42:53.726 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:53 smithi115 ceph-mon[29111]: pgmap v15: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 54 MiB/s rd, 202 MiB/s wr, 356 op/s 2024-03-22T10:42:53.727 INFO:tasks.workunit.client.0.smithi115.stdout:6/744: rename d4/d5e/l85 to d4/d11/d12/d1c/d8a/lf2 0 2024-03-22T10:42:53.728 INFO:tasks.workunit.client.0.smithi115.stdout:2/720: chown da/d25/d51 60029 1 2024-03-22T10:42:53.731 INFO:tasks.workunit.client.0.smithi115.stdout:2/721: chown da/d25/d51/fc8 15239 1 2024-03-22T10:42:53.735 INFO:tasks.workunit.client.1.smithi150.stdout:5/316: symlink d0/d8/dc/d1d/l61 0 2024-03-22T10:42:53.735 INFO:tasks.workunit.client.1.smithi150.stdout:8/436: creat d0/d3c/d71/d7a/f94 x:0 0 0 2024-03-22T10:42:53.739 INFO:tasks.workunit.client.0.smithi115.stdout:5/835: rename d2/d15/d1b/d7c/d38/d56/ca1 to d2/df/d13/d8b/d93/da6/d10e/d106/c10f 0 2024-03-22T10:42:53.742 INFO:tasks.workunit.client.1.smithi150.stdout:7/389: link d2/d5/d7/d8/d2c/f47 d2/d5/d7/d8/d57/d4a/f7f 0 2024-03-22T10:42:53.744 INFO:tasks.workunit.client.1.smithi150.stdout:3/395: fsync d5/f35 0 2024-03-22T10:42:53.746 INFO:tasks.workunit.client.0.smithi115.stdout:8/967: dwrite d6/dd/d24/d3a/f135 [0,4194304] 0 2024-03-22T10:42:53.752 INFO:tasks.workunit.client.1.smithi150.stdout:6/391: creat db/dd/d82/f8c x:0 0 0 2024-03-22T10:42:53.753 INFO:tasks.workunit.client.0.smithi115.stdout:9/783: dread d6/d7/d20/d22/d3f/fab [0,4194304] 0 2024-03-22T10:42:53.756 INFO:tasks.workunit.client.0.smithi115.stdout:2/722: symlink da/d25/d30/dcd/ld8 0 2024-03-22T10:42:53.757 INFO:tasks.workunit.client.0.smithi115.stdout:8/968: mkdir d6/dd/d24/d143 0 2024-03-22T10:42:53.758 INFO:tasks.workunit.client.1.smithi150.stdout:4/442: dread d0/f14 [0,4194304] 0 2024-03-22T10:42:53.760 INFO:tasks.workunit.client.0.smithi115.stdout:2/723: dread - da/da5/fd6 zero size 2024-03-22T10:42:53.760 INFO:tasks.workunit.client.1.smithi150.stdout:8/437: mknod d0/d2/d7e/c95 0 2024-03-22T10:42:53.764 INFO:tasks.workunit.client.1.smithi150.stdout:1/357: dread d1/da/d2a/f35 [0,4194304] 0 2024-03-22T10:42:53.765 INFO:tasks.workunit.client.1.smithi150.stdout:3/396: mknod d5/d22/d37/c82 0 2024-03-22T10:42:53.767 INFO:tasks.workunit.client.0.smithi115.stdout:3/839: rename d3/d16/d29/c3d to d3/d6d/db4/d107/daa/c11b 0 2024-03-22T10:42:53.767 INFO:tasks.workunit.client.1.smithi150.stdout:2/463: mkdir d12/d19/d2d/d99 0 2024-03-22T10:42:53.767 INFO:tasks.workunit.client.0.smithi115.stdout:4/822: dwrite d1/d66/d82/fa6 [0,4194304] 0 2024-03-22T10:42:53.768 INFO:tasks.workunit.client.1.smithi150.stdout:6/392: chown db/dd/d2e/c7a 2339886 1 2024-03-22T10:42:53.770 INFO:tasks.workunit.client.0.smithi115.stdout:5/836: dwrite d2/d15/d1b/d7c/d38/d43/d49/f52 [0,4194304] 0 2024-03-22T10:42:53.773 INFO:tasks.workunit.client.0.smithi115.stdout:2/724: unlink da/d25/d3d/d57/d7d/fc0 0 2024-03-22T10:42:53.776 INFO:tasks.workunit.client.0.smithi115.stdout:4/823: stat d1/dbb/dc6/ce3 0 2024-03-22T10:42:53.777 INFO:tasks.workunit.client.1.smithi150.stdout:9/464: creat d1/d8/de/df/d36/d40/d41/fa9 x:0 0 0 2024-03-22T10:42:53.780 INFO:tasks.workunit.client.1.smithi150.stdout:0/421: truncate d0/d12/d4f/d42/f82 3238079 0 2024-03-22T10:42:53.782 INFO:tasks.workunit.client.0.smithi115.stdout:4/824: chown d1/d30/d78/fb4 36919248 1 2024-03-22T10:42:53.782 INFO:tasks.workunit.client.1.smithi150.stdout:5/317: write d0/d8/dc/de/d14/d2e/d45/f2a [157958,69932] 0 2024-03-22T10:42:53.783 INFO:tasks.workunit.client.1.smithi150.stdout:2/464: mkdir d12/d16/d67/d9a 0 2024-03-22T10:42:53.783 INFO:tasks.workunit.client.1.smithi150.stdout:1/358: truncate d1/d19/d1a/f2b 2334451 0 2024-03-22T10:42:53.783 INFO:tasks.workunit.client.0.smithi115.stdout:4/825: readlink d1/d69/l37 0 2024-03-22T10:42:53.784 INFO:tasks.workunit.client.0.smithi115.stdout:2/725: creat da/d25/d3d/d7e/d90/fd9 x:0 0 0 2024-03-22T10:42:53.785 INFO:tasks.workunit.client.0.smithi115.stdout:9/784: dwrite d6/d7/d48/df4/dfa/f10b [0,4194304] 0 2024-03-22T10:42:53.788 INFO:tasks.workunit.client.0.smithi115.stdout:5/837: link d2/d15/d1b/d7c/d38/d43/d49/cf8 d2/df/d3b/d8e/c110 0 2024-03-22T10:42:53.793 INFO:tasks.workunit.client.0.smithi115.stdout:4/826: unlink d1/d66/d106/d28/c6a 0 2024-03-22T10:42:53.796 INFO:tasks.workunit.client.1.smithi150.stdout:4/443: dwrite d0/d1b/f2c [0,4194304] 0 2024-03-22T10:42:53.796 INFO:tasks.workunit.client.1.smithi150.stdout:9/465: symlink d1/d8/de/df/d20/d32/d75/d99/laa 0 2024-03-22T10:42:53.800 INFO:tasks.workunit.client.1.smithi150.stdout:6/393: mkdir db/dd/d15/d6a/d8d 0 2024-03-22T10:42:53.801 INFO:tasks.workunit.client.0.smithi115.stdout:8/969: write d6/dd/d24/d4a/f51 [3113352,93244] 0 2024-03-22T10:42:53.801 INFO:tasks.workunit.client.0.smithi115.stdout:5/838: creat d2/d15/d1b/d7c/d84/f111 x:0 0 0 2024-03-22T10:42:53.802 INFO:tasks.workunit.client.0.smithi115.stdout:4/827: symlink d1/d30/d4c/d86/d9f/d75/d99/l11d 0 2024-03-22T10:42:53.804 INFO:tasks.workunit.client.1.smithi150.stdout:5/318: creat d0/d8/dc/de/d14/d2e/f62 x:0 0 0 2024-03-22T10:42:53.804 INFO:tasks.workunit.client.1.smithi150.stdout:9/466: fdatasync d1/d8/de/df/d36/d40/d41/f94 0 2024-03-22T10:42:53.806 INFO:tasks.workunit.client.0.smithi115.stdout:2/726: rmdir da/d25/d3d/d57/d7d 39 2024-03-22T10:42:53.807 INFO:tasks.workunit.client.1.smithi150.stdout:2/465: truncate d12/d16/d67/f4e 720766 0 2024-03-22T10:42:53.808 INFO:tasks.workunit.client.0.smithi115.stdout:1/863: sync 2024-03-22T10:42:53.808 INFO:tasks.workunit.client.1.smithi150.stdout:0/422: mkdir d0/d1/d84 0 2024-03-22T10:42:53.812 INFO:tasks.workunit.client.1.smithi150.stdout:9/467: mknod d1/d49/cab 0 2024-03-22T10:42:53.813 INFO:tasks.workunit.client.0.smithi115.stdout:1/864: dread - d0/d12/d17/d67/de6/f110 zero size 2024-03-22T10:42:53.815 INFO:tasks.workunit.client.1.smithi150.stdout:6/394: truncate db/dd/d15/f27 4217117 0 2024-03-22T10:42:53.815 INFO:tasks.workunit.client.1.smithi150.stdout:3/397: dwrite d5/da/ff [0,4194304] 0 2024-03-22T10:42:53.815 INFO:tasks.workunit.client.1.smithi150.stdout:6/395: stat db/dd/d25/f76 0 2024-03-22T10:42:53.817 INFO:tasks.workunit.client.0.smithi115.stdout:8/970: mknod d6/dd/d24/d4a/dfc/c144 0 2024-03-22T10:42:53.821 INFO:tasks.workunit.client.0.smithi115.stdout:9/785: link d6/d7/d48/d70/da9/dbd/de9/c114 d6/d7/d30/d37/dbe/d110/c118 0 2024-03-22T10:42:53.821 INFO:tasks.workunit.client.1.smithi150.stdout:0/423: chown d0/d12/d4f/d42/d10/d53/f5e 1433 1 2024-03-22T10:42:53.821 INFO:tasks.workunit.client.0.smithi115.stdout:1/865: mkdir d0/d12/d17/d37/d133 0 2024-03-22T10:42:53.824 INFO:tasks.workunit.client.1.smithi150.stdout:0/424: dread - d0/f56 zero size 2024-03-22T10:42:53.825 INFO:tasks.workunit.client.0.smithi115.stdout:1/866: read - d0/ff0 zero size 2024-03-22T10:42:53.826 INFO:tasks.workunit.client.1.smithi150.stdout:5/319: unlink d0/d8/dc/de/d14/d4f/f52 0 2024-03-22T10:42:53.827 INFO:tasks.workunit.client.1.smithi150.stdout:0/425: fsync d0/f63 0 2024-03-22T10:42:53.833 INFO:tasks.workunit.client.1.smithi150.stdout:3/398: write d5/d22/d2f/d38/d3e/d5b/f76 [251187,54594] 0 2024-03-22T10:42:53.834 INFO:tasks.workunit.client.0.smithi115.stdout:4/828: dwrite d1/d66/d106/d28/fe5 [0,4194304] 0 2024-03-22T10:42:53.835 INFO:tasks.workunit.client.0.smithi115.stdout:5/839: dread d2/d15/d1b/d7c/d38/d43/f82 [0,4194304] 0 2024-03-22T10:42:53.836 INFO:tasks.workunit.client.1.smithi150.stdout:7/390: dread d2/d5/d7/d8/d46/f66 [0,4194304] 0 2024-03-22T10:42:53.844 INFO:tasks.workunit.client.0.smithi115.stdout:4/829: dread - d1/d30/d4c/d86/d9f/d75/d99/d98/d9b/fe2 zero size 2024-03-22T10:42:53.845 INFO:tasks.workunit.client.1.smithi150.stdout:4/444: dwrite d0/de/f7f [0,4194304] 0 2024-03-22T10:42:53.850 INFO:tasks.workunit.client.1.smithi150.stdout:5/320: mkdir d0/d8/dc/d1d/d53/d63 0 2024-03-22T10:42:53.850 INFO:tasks.workunit.client.0.smithi115.stdout:8/971: dwrite d6/dd/d12/d13c/fde [0,4194304] 0 2024-03-22T10:42:53.855 INFO:tasks.workunit.client.0.smithi115.stdout:3/840: dread d3/d36/d41/d33/fe0 [0,4194304] 0 2024-03-22T10:42:53.857 INFO:tasks.workunit.client.1.smithi150.stdout:0/426: chown d0/d12/d4f/d5a/l6a 300 1 2024-03-22T10:42:53.860 INFO:tasks.workunit.client.1.smithi150.stdout:2/466: dwrite d12/d16/d2c/f33 [0,4194304] 0 2024-03-22T10:42:53.862 INFO:tasks.workunit.client.1.smithi150.stdout:7/391: mknod d2/d5/d7/c80 0 2024-03-22T10:42:53.862 INFO:tasks.workunit.client.1.smithi150.stdout:8/438: dread d0/d2/d19/d29/d2f/df/f26 [0,4194304] 0 2024-03-22T10:42:53.864 INFO:tasks.workunit.client.1.smithi150.stdout:5/321: rmdir d0/d8/dc/de/d14/d2e/d45 39 2024-03-22T10:42:53.869 INFO:tasks.workunit.client.1.smithi150.stdout:8/439: readlink d0/d2/d19/d29/l43 0 2024-03-22T10:42:53.870 INFO:tasks.workunit.client.1.smithi150.stdout:2/467: symlink d12/d16/l9b 0 2024-03-22T10:42:53.870 INFO:tasks.workunit.client.1.smithi150.stdout:7/392: fdatasync d2/d5/d7/d9/f65 0 2024-03-22T10:42:53.872 INFO:tasks.workunit.client.1.smithi150.stdout:0/427: truncate d0/d12/d4f/d42/f2b 4313981 0 2024-03-22T10:42:53.875 INFO:tasks.workunit.client.1.smithi150.stdout:2/468: chown d12/d16/f43 0 1 2024-03-22T10:42:53.877 INFO:tasks.workunit.client.1.smithi150.stdout:5/322: mkdir d0/d8/dc/d64 0 2024-03-22T10:42:53.877 INFO:tasks.workunit.client.1.smithi150.stdout:1/359: dread d1/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:42:53.882 INFO:tasks.workunit.client.1.smithi150.stdout:5/323: chown d0/d8/dc/de/d14/d4f/f3d 227902387 1 2024-03-22T10:42:53.882 INFO:tasks.workunit.client.1.smithi150.stdout:6/396: write db/dd/f33 [781994,17912] 0 2024-03-22T10:42:53.885 INFO:tasks.workunit.client.1.smithi150.stdout:8/440: mkdir d0/d2/d7e/d96 0 2024-03-22T10:42:53.886 INFO:tasks.workunit.client.1.smithi150.stdout:2/469: dread f11 [0,4194304] 0 2024-03-22T10:42:53.889 INFO:tasks.workunit.client.1.smithi150.stdout:7/393: symlink d2/l81 0 2024-03-22T10:42:53.893 INFO:tasks.workunit.client.1.smithi150.stdout:2/470: readlink d12/d16/d61/d98/l93 0 2024-03-22T10:42:53.896 INFO:tasks.workunit.client.1.smithi150.stdout:6/397: creat db/d52/d79/f8e x:0 0 0 2024-03-22T10:42:53.898 INFO:tasks.workunit.client.1.smithi150.stdout:5/324: write d0/d8/dc/de/d14/d2e/d46/d4d/f51 [749089,60163] 0 2024-03-22T10:42:53.901 INFO:tasks.workunit.client.1.smithi150.stdout:1/360: creat d1/da/f72 x:0 0 0 2024-03-22T10:42:53.901 INFO:tasks.workunit.client.0.smithi115.stdout:4/830: mknod d1/dbb/ddf/df9/c11e 0 2024-03-22T10:42:53.902 INFO:tasks.workunit.client.1.smithi150.stdout:0/428: dread d0/d12/d4f/d42/fc [0,4194304] 0 2024-03-22T10:42:53.903 INFO:tasks.workunit.client.0.smithi115.stdout:3/841: rmdir d3/d36/d41/d85/db8 39 2024-03-22T10:42:53.904 INFO:tasks.workunit.client.0.smithi115.stdout:1/867: creat d0/d8/db/d25/d11a/f134 x:0 0 0 2024-03-22T10:42:53.905 INFO:tasks.workunit.client.1.smithi150.stdout:0/429: chown d0/f69 183930223 1 2024-03-22T10:42:53.905 INFO:tasks.workunit.client.1.smithi150.stdout:6/398: write db/dd/f37 [3261673,119820] 0 2024-03-22T10:42:53.905 INFO:tasks.workunit.client.1.smithi150.stdout:1/361: write d1/f3c [1624574,34898] 0 2024-03-22T10:42:53.906 INFO:tasks.workunit.client.0.smithi115.stdout:2/727: write da/d1c/d67/f6f [3487726,113143] 0 2024-03-22T10:42:53.911 INFO:tasks.workunit.client.1.smithi150.stdout:5/325: truncate d0/f22 2072239 0 2024-03-22T10:42:53.912 INFO:tasks.workunit.client.0.smithi115.stdout:3/842: symlink d3/d19/dda/l11c 0 2024-03-22T10:42:53.912 INFO:tasks.workunit.client.1.smithi150.stdout:0/430: write d0/d12/d4f/f6b [4128529,119055] 0 2024-03-22T10:42:53.913 INFO:tasks.workunit.client.0.smithi115.stdout:2/728: read da/f1d [2958065,38287] 0 2024-03-22T10:42:53.915 INFO:tasks.workunit.client.0.smithi115.stdout:3/843: readlink d3/d6d/db4/d107/l8a 0 2024-03-22T10:42:53.921 INFO:tasks.workunit.client.0.smithi115.stdout:8/972: dread d6/dd/d24/f3f [0,4194304] 0 2024-03-22T10:42:53.922 INFO:tasks.workunit.client.1.smithi150.stdout:4/445: dread d0/f9 [0,4194304] 0 2024-03-22T10:42:53.922 INFO:tasks.workunit.client.0.smithi115.stdout:2/729: dread da/d1c/f52 [0,4194304] 0 2024-03-22T10:42:53.924 INFO:tasks.workunit.client.0.smithi115.stdout:2/730: chown c1 16 1 2024-03-22T10:42:53.929 INFO:tasks.workunit.client.1.smithi150.stdout:6/399: mknod db/dd/d15/d60/c8f 0 2024-03-22T10:42:53.930 INFO:tasks.workunit.client.0.smithi115.stdout:3/844: creat d3/d36/d41/d85/f11d x:0 0 0 2024-03-22T10:42:53.930 INFO:tasks.workunit.client.0.smithi115.stdout:2/731: mknod da/d25/d3d/cda 0 2024-03-22T10:42:53.932 INFO:tasks.workunit.client.0.smithi115.stdout:3/845: fdatasync d3/d6d/db4/d107/da6/fe9 0 2024-03-22T10:42:53.933 INFO:tasks.workunit.client.1.smithi150.stdout:7/394: dwrite d2/d5/d7/d8/d46/f4e [0,4194304] 0 2024-03-22T10:42:53.933 INFO:tasks.workunit.client.1.smithi150.stdout:4/446: symlink d0/d2f/d63/d7b/l97 0 2024-03-22T10:42:53.936 INFO:tasks.workunit.client.1.smithi150.stdout:3/399: dwrite d5/da/f34 [0,4194304] 0 2024-03-22T10:42:53.936 INFO:tasks.workunit.client.1.smithi150.stdout:4/447: chown d0/d32/f8e 2081514322 1 2024-03-22T10:42:53.938 INFO:tasks.workunit.client.0.smithi115.stdout:2/732: creat da/d25/d3d/d7e/d90/d96/fdb x:0 0 0 2024-03-22T10:42:53.940 INFO:tasks.workunit.client.1.smithi150.stdout:1/362: dread d1/d19/d1a/f49 [0,4194304] 0 2024-03-22T10:42:53.940 INFO:tasks.workunit.client.0.smithi115.stdout:8/973: dwrite d6/dd/d12/d31/f5b [0,4194304] 0 2024-03-22T10:42:53.942 INFO:tasks.workunit.client.1.smithi150.stdout:1/363: readlink d1/d19/d1a/l37 0 2024-03-22T10:42:53.944 INFO:tasks.workunit.client.1.smithi150.stdout:6/400: symlink db/dd/d15/d30/l90 0 2024-03-22T10:42:53.950 INFO:tasks.workunit.client.1.smithi150.stdout:4/448: mkdir d0/d1b/d4d/d6c/d61/d98 0 2024-03-22T10:42:53.951 INFO:tasks.workunit.client.0.smithi115.stdout:3/846: creat d3/d19/d2b/f11e x:0 0 0 2024-03-22T10:42:53.953 INFO:tasks.workunit.client.1.smithi150.stdout:3/400: creat d5/d22/d2f/d38/d3e/d72/f83 x:0 0 0 2024-03-22T10:42:53.955 INFO:tasks.workunit.client.1.smithi150.stdout:4/449: mkdir d0/d32/d7c/d99 0 2024-03-22T10:42:53.957 INFO:tasks.workunit.client.1.smithi150.stdout:6/401: unlink db/dd/d15/f46 0 2024-03-22T10:42:53.957 INFO:tasks.workunit.client.0.smithi115.stdout:3/847: chown d3/d16/d29/d35/ca1 4620 1 2024-03-22T10:42:53.957 INFO:tasks.workunit.client.1.smithi150.stdout:8/441: dread d0/d2/d19/d29/d37/f85 [4194304,4194304] 0 2024-03-22T10:42:53.960 INFO:tasks.workunit.client.0.smithi115.stdout:2/733: mkdir da/d25/d3d/d57/d7d/da1/ddc 0 2024-03-22T10:42:53.961 INFO:tasks.workunit.client.1.smithi150.stdout:0/431: write d0/f7b [3368726,58086] 0 2024-03-22T10:42:53.962 INFO:tasks.workunit.client.1.smithi150.stdout:1/364: creat d1/f73 x:0 0 0 2024-03-22T10:42:53.965 INFO:tasks.workunit.client.0.smithi115.stdout:8/974: dwrite d6/f91 [0,4194304] 0 2024-03-22T10:42:53.965 INFO:tasks.workunit.client.0.smithi115.stdout:3/848: creat d3/d16/d29/dc1/f11f x:0 0 0 2024-03-22T10:42:53.965 INFO:tasks.workunit.client.1.smithi150.stdout:6/402: readlink db/dd/d15/l49 0 2024-03-22T10:42:53.967 INFO:tasks.workunit.client.0.smithi115.stdout:2/734: chown da/d25/d30/f59 4281390 1 2024-03-22T10:42:53.967 INFO:tasks.workunit.client.1.smithi150.stdout:0/432: truncate d0/d12/d4f/d11/f7e 856535 0 2024-03-22T10:42:53.968 INFO:tasks.workunit.client.0.smithi115.stdout:8/975: write f5 [15040,107298] 0 2024-03-22T10:42:53.971 INFO:tasks.workunit.client.0.smithi115.stdout:9/786: dread d6/d7/d20/d9c/fb0 [0,4194304] 0 2024-03-22T10:42:53.973 INFO:tasks.workunit.client.1.smithi150.stdout:8/442: write d0/d2/d19/d29/d2f/f73 [3674736,86087] 0 2024-03-22T10:42:53.973 INFO:tasks.workunit.client.1.smithi150.stdout:1/365: creat d1/da/d15/f74 x:0 0 0 2024-03-22T10:42:53.976 INFO:tasks.workunit.client.0.smithi115.stdout:2/735: unlink da/d25/d3d/d7e/d90/dbf/d54/fab 0 2024-03-22T10:42:53.978 INFO:tasks.workunit.client.0.smithi115.stdout:3/849: rename d3/d82/d54/c10d to d3/d36/d41/c120 0 2024-03-22T10:42:53.980 INFO:tasks.workunit.client.0.smithi115.stdout:8/976: symlink d6/dd/d24/d4a/dfc/d102/l145 0 2024-03-22T10:42:53.985 INFO:tasks.workunit.client.1.smithi150.stdout:6/403: dread db/f4a [0,4194304] 0 2024-03-22T10:42:53.989 INFO:tasks.workunit.client.1.smithi150.stdout:8/443: creat d0/d2/d19/d29/d2f/df/f97 x:0 0 0 2024-03-22T10:42:53.992 INFO:tasks.workunit.client.1.smithi150.stdout:6/404: mknod db/d52/d79/c91 0 2024-03-22T10:42:53.992 INFO:tasks.workunit.client.0.smithi115.stdout:3/850: getdents d3/d19/d2b/d49/d91/dad 0 2024-03-22T10:42:53.995 INFO:tasks.workunit.client.1.smithi150.stdout:0/433: dwrite d0/f56 [0,4194304] 0 2024-03-22T10:42:53.997 INFO:tasks.workunit.client.1.smithi150.stdout:8/444: chown d0/d2/d19/l33 22249 1 2024-03-22T10:42:53.999 INFO:tasks.workunit.client.1.smithi150.stdout:7/395: dread d2/d5/d7/f2b [0,4194304] 0 2024-03-22T10:42:54.003 INFO:tasks.workunit.client.1.smithi150.stdout:1/366: dwrite d1/f6c [0,4194304] 0 2024-03-22T10:42:54.003 INFO:tasks.workunit.client.1.smithi150.stdout:0/434: write d0/d12/d4f/d42/f77 [648046,128654] 0 2024-03-22T10:42:54.004 INFO:tasks.workunit.client.0.smithi115.stdout:5/840: dwrite d2/df/d13/d8b/d93/da6/dbc/fd0 [0,4194304] 0 2024-03-22T10:42:54.005 INFO:tasks.workunit.client.0.smithi115.stdout:2/736: dwrite da/d25/d30/f6e [0,4194304] 0 2024-03-22T10:42:54.006 INFO:tasks.workunit.client.0.smithi115.stdout:8/977: dwrite d6/d49/d88/fe2 [0,4194304] 0 2024-03-22T10:42:54.006 INFO:tasks.workunit.client.1.smithi150.stdout:7/396: chown d2/d5/d7/d8/d46/f5a 1517972325 1 2024-03-22T10:42:54.007 INFO:tasks.workunit.client.0.smithi115.stdout:3/851: symlink d3/d19/dd8/l121 0 2024-03-22T10:42:54.007 INFO:tasks.workunit.client.1.smithi150.stdout:1/367: write d1/d19/d57/f5b [330530,74151] 0 2024-03-22T10:42:54.008 INFO:tasks.workunit.client.1.smithi150.stdout:8/445: symlink d0/d2/d19/l98 0 2024-03-22T10:42:54.009 INFO:tasks.workunit.client.1.smithi150.stdout:6/405: link db/dd/d15/f7b db/dd/d15/d60/d8b/f92 0 2024-03-22T10:42:54.009 INFO:tasks.workunit.client.0.smithi115.stdout:8/978: truncate d6/d49/f111 1983657 0 2024-03-22T10:42:54.013 INFO:tasks.workunit.client.1.smithi150.stdout:6/406: creat db/d52/f93 x:0 0 0 2024-03-22T10:42:54.013 INFO:tasks.workunit.client.0.smithi115.stdout:2/737: chown da/d25/d30/l29 1292981197 1 2024-03-22T10:42:54.014 INFO:tasks.workunit.client.0.smithi115.stdout:4/831: write d1/d66/d106/d35/d40/d5c/d14/f90 [756957,78347] 0 2024-03-22T10:42:54.015 INFO:tasks.workunit.client.1.smithi150.stdout:6/407: write db/dd/f37 [3466794,3914] 0 2024-03-22T10:42:54.017 INFO:tasks.workunit.client.1.smithi150.stdout:0/435: creat d0/f85 x:0 0 0 2024-03-22T10:42:54.017 INFO:tasks.workunit.client.1.smithi150.stdout:8/446: read d0/d2/d46/f52 [844897,11289] 0 2024-03-22T10:42:54.021 INFO:tasks.workunit.client.0.smithi115.stdout:2/738: fdatasync da/d25/d51/f56 0 2024-03-22T10:42:54.024 INFO:tasks.workunit.client.1.smithi150.stdout:7/397: creat d2/f82 x:0 0 0 2024-03-22T10:42:54.029 INFO:tasks.workunit.client.1.smithi150.stdout:7/398: fsync d2/d5/d7/d8/d46/d49/f68 0 2024-03-22T10:42:54.032 INFO:tasks.workunit.client.1.smithi150.stdout:0/436: rename d0/d12/d4f to d0/d1/d84/d86 0 2024-03-22T10:42:54.037 INFO:tasks.workunit.client.1.smithi150.stdout:7/399: creat d2/d6e/f83 x:0 0 0 2024-03-22T10:42:54.039 INFO:tasks.workunit.client.0.smithi115.stdout:8/979: dwrite d6/dd/d12/f3b [0,4194304] 0 2024-03-22T10:42:54.045 INFO:tasks.workunit.client.1.smithi150.stdout:7/400: write d2/d5/f7c [279894,29156] 0 2024-03-22T10:42:54.048 INFO:tasks.workunit.client.0.smithi115.stdout:8/980: rename d6/f137 to d6/dd/d24/f146 0 2024-03-22T10:42:54.051 INFO:tasks.workunit.client.1.smithi150.stdout:7/401: creat d2/d5/d7/d8/d2c/d45/f84 x:0 0 0 2024-03-22T10:42:54.052 INFO:tasks.workunit.client.0.smithi115.stdout:8/981: symlink d6/dd/d24/d4a/d41/dc4/l147 0 2024-03-22T10:42:54.053 INFO:tasks.workunit.client.1.smithi150.stdout:0/437: dwrite d0/f7b [0,4194304] 0 2024-03-22T10:42:54.053 INFO:tasks.workunit.client.1.smithi150.stdout:7/402: stat d2/d5/d7/d9/f62 0 2024-03-22T10:42:54.053 INFO:tasks.workunit.client.0.smithi115.stdout:9/787: creat d6/d7/d48/f119 x:0 0 0 2024-03-22T10:42:54.054 INFO:tasks.workunit.client.1.smithi150.stdout:0/438: chown d0/d12/c33 3 1 2024-03-22T10:42:54.055 INFO:tasks.workunit.client.0.smithi115.stdout:8/982: dread - d6/dbc/fca zero size 2024-03-22T10:42:54.057 INFO:tasks.workunit.client.0.smithi115.stdout:9/788: fdatasync d6/d7/d30/d3b/f58 0 2024-03-22T10:42:54.059 INFO:tasks.workunit.client.0.smithi115.stdout:8/983: mkdir d6/dd/d24/d4a/d7b/d8f/da0/d148 0 2024-03-22T10:42:54.059 INFO:tasks.workunit.client.1.smithi150.stdout:7/403: rename d2/l3e to d2/d5/d7/d9/db/l85 0 2024-03-22T10:42:54.060 INFO:tasks.workunit.client.0.smithi115.stdout:9/789: getdents d6/d7/d20/d9c/d108 0 2024-03-22T10:42:54.061 INFO:tasks.workunit.client.0.smithi115.stdout:8/984: mknod d6/dd/d24/d4a/c149 0 2024-03-22T10:42:54.062 INFO:tasks.workunit.client.0.smithi115.stdout:9/790: dread - d6/d7/d48/d53/dc9/f111 zero size 2024-03-22T10:42:54.063 INFO:tasks.workunit.client.0.smithi115.stdout:8/985: write d6/dd/f13d [391490,34442] 0 2024-03-22T10:42:54.066 INFO:tasks.workunit.client.0.smithi115.stdout:8/986: unlink d6/d49/d68/fb7 0 2024-03-22T10:42:54.068 INFO:tasks.workunit.client.0.smithi115.stdout:8/987: chown d6/dd/ld3 101 1 2024-03-22T10:42:54.068 INFO:tasks.workunit.client.1.smithi150.stdout:0/439: link d0/d1/d84/d86/c17 d0/d1/d84/c87 0 2024-03-22T10:42:54.069 INFO:tasks.workunit.client.1.smithi150.stdout:7/404: creat d2/d5/d7/d38/f86 x:0 0 0 2024-03-22T10:42:54.070 INFO:tasks.workunit.client.0.smithi115.stdout:8/988: chown d6/d49/d88/f133 47271 1 2024-03-22T10:42:54.072 INFO:tasks.workunit.client.1.smithi150.stdout:0/440: chown d0/d1/c1d 2696 1 2024-03-22T10:42:54.073 INFO:tasks.workunit.client.0.smithi115.stdout:8/989: symlink d6/dd/d24/d112/l14a 0 2024-03-22T10:42:54.073 INFO:tasks.workunit.client.1.smithi150.stdout:7/405: symlink d2/d5/l87 0 2024-03-22T10:42:54.074 INFO:tasks.workunit.client.0.smithi115.stdout:3/852: truncate d3/d19/d2b/d49/fc6 2301467 0 2024-03-22T10:42:54.074 INFO:tasks.workunit.client.1.smithi150.stdout:7/406: dread - d2/f67 zero size 2024-03-22T10:42:54.076 INFO:tasks.workunit.client.1.smithi150.stdout:0/441: symlink d0/d1/d84/d86/d11/d66/l88 0 2024-03-22T10:42:54.077 INFO:tasks.workunit.client.0.smithi115.stdout:3/853: symlink d3/d36/d41/d85/db8/l122 0 2024-03-22T10:42:54.079 INFO:tasks.workunit.client.0.smithi115.stdout:8/990: getdents d6/dd/d12/d12f 0 2024-03-22T10:42:54.080 INFO:tasks.workunit.client.0.smithi115.stdout:8/991: read - d6/dd/fce zero size 2024-03-22T10:42:54.082 INFO:tasks.workunit.client.1.smithi150.stdout:6/408: fsync db/dd/d15/f7b 0 2024-03-22T10:42:54.082 INFO:tasks.workunit.client.0.smithi115.stdout:3/854: rename d3/d16/d29/ld7 to d3/d19/dda/l123 0 2024-03-22T10:42:54.084 INFO:tasks.workunit.client.0.smithi115.stdout:4/832: dwrite d1/d66/d106/d35/d40/d5c/d14/f58 [0,4194304] 0 2024-03-22T10:42:54.085 INFO:tasks.workunit.client.0.smithi115.stdout:8/992: rename d6/fa to d6/dd/d24/d4a/d41/f14b 0 2024-03-22T10:42:54.093 INFO:tasks.workunit.client.1.smithi150.stdout:6/409: truncate db/dd/d25/f3b 1644264 0 2024-03-22T10:42:54.095 INFO:tasks.workunit.client.0.smithi115.stdout:9/791: write d6/d7/d48/faf [1164364,71676] 0 2024-03-22T10:42:54.095 INFO:tasks.workunit.client.1.smithi150.stdout:1/368: dwrite d1/d19/d57/f5b [4194304,4194304] 0 2024-03-22T10:42:54.095 INFO:tasks.workunit.client.1.smithi150.stdout:7/407: dwrite d2/d6e/f73 [0,4194304] 0 2024-03-22T10:42:54.097 INFO:tasks.workunit.client.0.smithi115.stdout:2/739: dwrite da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:54.100 INFO:tasks.workunit.client.1.smithi150.stdout:1/369: dread - d1/da/f67 zero size 2024-03-22T10:42:54.104 INFO:tasks.workunit.client.0.smithi115.stdout:9/792: truncate d6/d7/d20/d22/d23/d106/f10c 887350 0 2024-03-22T10:42:54.107 INFO:tasks.workunit.client.0.smithi115.stdout:8/993: write d6/f95 [3073352,89928] 0 2024-03-22T10:42:54.107 INFO:tasks.workunit.client.0.smithi115.stdout:2/740: mknod da/d25/d3d/d57/cdd 0 2024-03-22T10:42:54.109 INFO:tasks.workunit.client.1.smithi150.stdout:8/447: dwrite d0/d2/d19/d29/d2f/df/d1b/f1c [0,4194304] 0 2024-03-22T10:42:54.110 INFO:tasks.workunit.client.0.smithi115.stdout:2/741: truncate da/f16 1272436 0 2024-03-22T10:42:54.112 INFO:tasks.workunit.client.0.smithi115.stdout:8/994: symlink d6/dd/d24/l14c 0 2024-03-22T10:42:54.116 INFO:tasks.workunit.client.1.smithi150.stdout:1/370: fdatasync d1/da/d15/d18/d4b/f68 0 2024-03-22T10:42:54.119 INFO:tasks.workunit.client.1.smithi150.stdout:6/410: truncate db/dd/d25/f3d 1648055 0 2024-03-22T10:42:54.121 INFO:tasks.workunit.client.0.smithi115.stdout:9/793: dread d6/d7/d30/f9d [0,4194304] 0 2024-03-22T10:42:54.123 INFO:tasks.workunit.client.1.smithi150.stdout:7/408: mkdir d2/d5/d7/d88 0 2024-03-22T10:42:54.124 INFO:tasks.workunit.client.0.smithi115.stdout:9/794: write d6/d7/d20/d22/d23/d106/f10c [1750910,35238] 0 2024-03-22T10:42:54.126 INFO:tasks.workunit.client.1.smithi150.stdout:7/409: write d2/d5/d7/d9/f62 [165409,115216] 0 2024-03-22T10:42:54.131 INFO:tasks.workunit.client.0.smithi115.stdout:9/795: rename d6/d7/d20/d9c/lb9 to d6/d7/d48/l11a 0 2024-03-22T10:42:54.132 INFO:tasks.workunit.client.1.smithi150.stdout:6/411: fdatasync db/dd/d15/d28/f2b 0 2024-03-22T10:42:54.134 INFO:tasks.workunit.client.1.smithi150.stdout:6/412: chown db/dd/d82 62415 1 2024-03-22T10:42:54.134 INFO:tasks.workunit.client.1.smithi150.stdout:8/448: dwrite d0/d2/d19/d29/d2f/df/f42 [4194304,4194304] 0 2024-03-22T10:42:54.136 INFO:tasks.workunit.client.1.smithi150.stdout:8/449: write d0/d2/d19/f20 [4480272,3444] 0 2024-03-22T10:42:54.137 INFO:tasks.workunit.client.1.smithi150.stdout:6/413: write db/dd/d2e/f81 [148863,9769] 0 2024-03-22T10:42:54.141 INFO:tasks.workunit.client.0.smithi115.stdout:2/742: dwrite da/d25/d51/f55 [0,4194304] 0 2024-03-22T10:42:54.141 INFO:tasks.workunit.client.0.smithi115.stdout:8/995: dwrite d6/dd/f92 [0,4194304] 0 2024-03-22T10:42:54.143 INFO:tasks.workunit.client.0.smithi115.stdout:2/743: stat da/d25/d3d/l36 0 2024-03-22T10:42:54.144 INFO:tasks.workunit.client.1.smithi150.stdout:8/450: rmdir d0/d2/d19/d29/d2e 39 2024-03-22T10:42:54.145 INFO:tasks.workunit.client.1.smithi150.stdout:1/371: write d1/d19/d1a/f2b [975502,67929] 0 2024-03-22T10:42:54.146 INFO:tasks.workunit.client.0.smithi115.stdout:2/744: rmdir da/db1 39 2024-03-22T10:42:54.146 INFO:tasks.workunit.client.0.smithi115.stdout:8/996: truncate d6/dd/d24/f72 5185905 0 2024-03-22T10:42:54.154 INFO:tasks.workunit.client.1.smithi150.stdout:8/451: creat d0/d2/d7e/d96/f99 x:0 0 0 2024-03-22T10:42:54.157 INFO:tasks.workunit.client.1.smithi150.stdout:1/372: unlink d1/da/f42 0 2024-03-22T10:42:54.165 INFO:tasks.workunit.client.1.smithi150.stdout:6/414: dwrite db/dd/f33 [0,4194304] 0 2024-03-22T10:42:54.167 INFO:tasks.workunit.client.1.smithi150.stdout:8/452: getdents d0/d3c/d71 0 2024-03-22T10:42:54.171 INFO:tasks.workunit.client.1.smithi150.stdout:6/415: mknod db/dd/d15/d60/d8b/c94 0 2024-03-22T10:42:54.179 INFO:tasks.workunit.client.0.smithi115.stdout:9/796: dread d6/d7/d20/d22/f2b [4194304,4194304] 0 2024-03-22T10:42:54.180 INFO:tasks.workunit.client.1.smithi150.stdout:6/416: write db/dd/d15/d57/f65 [781702,7048] 0 2024-03-22T10:42:54.182 INFO:tasks.workunit.client.1.smithi150.stdout:1/373: dwrite d1/f73 [0,4194304] 0 2024-03-22T10:42:54.182 INFO:tasks.workunit.client.0.smithi115.stdout:9/797: chown d6/d7/d48/d70/da9/dbd/ff7 344515792 1 2024-03-22T10:42:54.182 INFO:tasks.workunit.client.0.smithi115.stdout:2/745: truncate da/d25/d51/f55 614562 0 2024-03-22T10:42:54.184 INFO:tasks.workunit.client.1.smithi150.stdout:7/410: dread d2/d5/d7/d8/d2c/f47 [0,4194304] 0 2024-03-22T10:42:54.186 INFO:tasks.workunit.client.1.smithi150.stdout:7/411: write d2/d5/d7/d8/d57/f3c [747937,78654] 0 2024-03-22T10:42:54.187 INFO:tasks.workunit.client.0.smithi115.stdout:2/746: symlink da/d1c/d67/d78/d86/d9d/lde 0 2024-03-22T10:42:54.190 INFO:tasks.workunit.client.1.smithi150.stdout:7/412: chown d2/d5/d7/d9/f53 396 1 2024-03-22T10:42:54.190 INFO:tasks.workunit.client.1.smithi150.stdout:6/417: creat db/dd/d15/d6a/d8d/f95 x:0 0 0 2024-03-22T10:42:54.190 INFO:tasks.workunit.client.0.smithi115.stdout:8/997: dwrite d6/dd/fce [0,4194304] 0 2024-03-22T10:42:54.194 INFO:tasks.workunit.client.0.smithi115.stdout:8/998: write d6/dd/d12/fb0 [7486664,20625] 0 2024-03-22T10:42:54.194 INFO:tasks.workunit.client.1.smithi150.stdout:7/413: chown d2/d5/d7/d8/d57/c6a 303 1 2024-03-22T10:42:54.194 INFO:tasks.workunit.client.1.smithi150.stdout:8/453: write d0/d2/d19/d29/d2f/df/f26 [465075,82542] 0 2024-03-22T10:42:54.198 INFO:tasks.workunit.client.1.smithi150.stdout:8/454: truncate d0/d2/f67 499808 0 2024-03-22T10:42:54.199 INFO:tasks.workunit.client.0.smithi115.stdout:2/747: mkdir da/d25/d3d/d57/d7d/da1/dba/ddf 0 2024-03-22T10:42:54.200 INFO:tasks.workunit.client.0.smithi115.stdout:8/999: unlink d6/dd/l5d 0 2024-03-22T10:42:54.201 INFO:tasks.workunit.client.1.smithi150.stdout:6/418: creat db/dd/d82/d84/f96 x:0 0 0 2024-03-22T10:42:54.201 INFO:tasks.workunit.client.1.smithi150.stdout:7/414: mknod d2/d5/d7/d8/d2c/c89 0 2024-03-22T10:42:54.203 INFO:tasks.workunit.client.1.smithi150.stdout:6/419: read - db/dd/d82/f8c zero size 2024-03-22T10:42:54.203 INFO:tasks.workunit.client.0.smithi115.stdout:2/748: creat da/d25/d3d/d57/d7d/fe0 x:0 0 0 2024-03-22T10:42:54.205 INFO:tasks.workunit.client.1.smithi150.stdout:7/415: rmdir d2/d5/d7/d8 39 2024-03-22T10:42:54.207 INFO:tasks.workunit.client.0.smithi115.stdout:9/798: dwrite d6/d8/d3c/d59/fd8 [0,4194304] 0 2024-03-22T10:42:54.209 INFO:tasks.workunit.client.1.smithi150.stdout:1/374: write d1/da/d15/d18/f38 [3627952,109975] 0 2024-03-22T10:42:54.209 INFO:tasks.workunit.client.1.smithi150.stdout:6/420: dread db/dd/f5b [0,4194304] 0 2024-03-22T10:42:54.209 INFO:tasks.workunit.client.0.smithi115.stdout:9/799: chown d6/d7/d20/d22/d3f/fc8 419474327 1 2024-03-22T10:42:54.211 INFO:tasks.workunit.client.0.smithi115.stdout:9/800: write d6/d7/d48/d53/dc9/f100 [902266,130344] 0 2024-03-22T10:42:54.214 INFO:tasks.workunit.client.0.smithi115.stdout:9/801: dread d6/d7/d48/d70/fc0 [0,4194304] 0 2024-03-22T10:42:54.215 INFO:tasks.workunit.client.1.smithi150.stdout:7/416: mknod d2/d5/d7/d8/d2c/c8a 0 2024-03-22T10:42:54.217 INFO:tasks.workunit.client.0.smithi115.stdout:9/802: creat d6/d8/d3c/d59/d68/d8e/f11b x:0 0 0 2024-03-22T10:42:54.222 INFO:tasks.workunit.client.0.smithi115.stdout:9/803: truncate d6/d7/d20/d22/f2b 2011301 0 2024-03-22T10:42:54.234 INFO:tasks.workunit.client.1.smithi150.stdout:1/375: dwrite d1/f3c [0,4194304] 0 2024-03-22T10:42:54.234 INFO:tasks.workunit.client.1.smithi150.stdout:6/421: dwrite db/dd/d15/d30/f4b [0,4194304] 0 2024-03-22T10:42:54.236 INFO:tasks.workunit.client.1.smithi150.stdout:0/442: sync 2024-03-22T10:42:54.236 INFO:tasks.workunit.client.1.smithi150.stdout:2/471: sync 2024-03-22T10:42:54.236 INFO:tasks.workunit.client.1.smithi150.stdout:3/401: sync 2024-03-22T10:42:54.236 INFO:tasks.workunit.client.1.smithi150.stdout:5/326: sync 2024-03-22T10:42:54.237 INFO:tasks.workunit.client.1.smithi150.stdout:9/468: sync 2024-03-22T10:42:54.239 INFO:tasks.workunit.client.1.smithi150.stdout:4/450: sync 2024-03-22T10:42:54.239 INFO:tasks.workunit.client.1.smithi150.stdout:7/417: dwrite d2/d6e/f83 [0,4194304] 0 2024-03-22T10:42:54.241 INFO:tasks.workunit.client.1.smithi150.stdout:6/422: readlink db/dd/d15/d28/l68 0 2024-03-22T10:42:54.241 INFO:tasks.workunit.client.1.smithi150.stdout:2/472: chown d12/d16/c32 5033 1 2024-03-22T10:42:54.246 INFO:tasks.workunit.client.1.smithi150.stdout:7/418: write d2/d6e/f83 [764415,64620] 0 2024-03-22T10:42:54.251 INFO:tasks.workunit.client.1.smithi150.stdout:8/455: dread d0/d3c/f4a [0,4194304] 0 2024-03-22T10:42:54.251 INFO:tasks.workunit.client.1.smithi150.stdout:4/451: mknod d0/d1b/d4d/d6c/d61/c9a 0 2024-03-22T10:42:54.251 INFO:tasks.workunit.client.0.smithi115.stdout:9/804: link d6/f12 d6/d7/d48/df4/f11c 0 2024-03-22T10:42:54.252 INFO:tasks.workunit.client.1.smithi150.stdout:8/456: read d0/d2/d46/f52 [416678,33413] 0 2024-03-22T10:42:54.254 INFO:tasks.workunit.client.0.smithi115.stdout:9/805: truncate d6/d8/d3c/d59/f90 4597228 0 2024-03-22T10:42:54.255 INFO:tasks.workunit.client.1.smithi150.stdout:6/423: unlink db/dd/d15/d57/f7d 0 2024-03-22T10:42:54.256 INFO:tasks.workunit.client.0.smithi115.stdout:9/806: dread - d6/d7/d20/d22/d3f/fb1 zero size 2024-03-22T10:42:54.259 INFO:tasks.workunit.client.0.smithi115.stdout:9/807: mkdir d6/d7/d48/d70/da9/dbd/d11d 0 2024-03-22T10:42:54.259 INFO:tasks.workunit.client.0.smithi115.stdout:9/808: readlink d6/l15 0 2024-03-22T10:42:54.260 INFO:tasks.workunit.client.1.smithi150.stdout:7/419: creat d2/d5/d7/d8/d2c/f8b x:0 0 0 2024-03-22T10:42:54.263 INFO:tasks.workunit.client.1.smithi150.stdout:8/457: rename d0/d2/d19/d29/d2f/f74 to d0/d2/d19/d29/d88/f9a 0 2024-03-22T10:42:54.268 INFO:tasks.workunit.client.1.smithi150.stdout:8/458: truncate d0/d2/d19/d29/d31/f6d 401323 0 2024-03-22T10:42:54.271 INFO:tasks.workunit.client.1.smithi150.stdout:0/443: creat d0/f89 x:0 0 0 2024-03-22T10:42:54.273 INFO:tasks.workunit.client.1.smithi150.stdout:0/444: dread - d0/f83 zero size 2024-03-22T10:42:54.274 INFO:tasks.workunit.client.1.smithi150.stdout:4/452: mkdir d0/d9b 0 2024-03-22T10:42:54.276 INFO:tasks.workunit.client.0.smithi115.stdout:2/749: dread da/d1c/d67/f6f [0,4194304] 0 2024-03-22T10:42:54.277 INFO:tasks.workunit.client.1.smithi150.stdout:7/420: creat d2/d6e/f8c x:0 0 0 2024-03-22T10:42:54.279 INFO:tasks.workunit.client.1.smithi150.stdout:2/473: getdents d12/d19/d2d/d66 0 2024-03-22T10:42:54.280 INFO:tasks.workunit.client.0.smithi115.stdout:2/750: truncate da/d25/d3d/d57/d7d/fac 471164 0 2024-03-22T10:42:54.283 INFO:tasks.workunit.client.1.smithi150.stdout:0/445: stat d0/d1/d84/d86/d42/d10/d38/d52/c5b 0 2024-03-22T10:42:54.283 INFO:tasks.workunit.client.1.smithi150.stdout:2/474: chown d12/d48/d63/f6c 267447 1 2024-03-22T10:42:54.287 INFO:tasks.workunit.client.1.smithi150.stdout:1/376: creat d1/da/f75 x:0 0 0 2024-03-22T10:42:54.287 INFO:tasks.workunit.client.1.smithi150.stdout:9/469: creat d1/d8/de/df/d36/d40/d41/fac x:0 0 0 2024-03-22T10:42:54.288 INFO:tasks.workunit.client.1.smithi150.stdout:4/453: mkdir d0/d2f/d63/d7b/d9c 0 2024-03-22T10:42:54.290 INFO:tasks.workunit.client.1.smithi150.stdout:3/402: dread d5/fd [0,4194304] 0 2024-03-22T10:42:54.293 INFO:tasks.workunit.client.0.smithi115.stdout:2/751: dwrite da/f15 [0,4194304] 0 2024-03-22T10:42:54.296 INFO:tasks.workunit.client.1.smithi150.stdout:8/459: rename d0/l11 to d0/d2/d7e/l9b 0 2024-03-22T10:42:54.296 INFO:tasks.workunit.client.0.smithi115.stdout:9/809: dread f3 [0,4194304] 0 2024-03-22T10:42:54.298 INFO:tasks.workunit.client.1.smithi150.stdout:4/454: write d0/de/f18 [1621585,49320] 0 2024-03-22T10:42:54.299 INFO:tasks.workunit.client.1.smithi150.stdout:9/470: write d1/d8/de/df/d20/f50 [3823150,127746] 0 2024-03-22T10:42:54.302 INFO:tasks.workunit.client.0.smithi115.stdout:2/752: link da/d25/d3d/d57/c5a da/d25/d51/ce1 0 2024-03-22T10:42:54.306 INFO:tasks.workunit.client.0.smithi115.stdout:2/753: write da/d25/d3d/d7e/d90/d96/fdb [109412,101073] 0 2024-03-22T10:42:54.309 INFO:tasks.workunit.client.1.smithi150.stdout:6/424: dread db/dd/d15/f27 [0,4194304] 0 2024-03-22T10:42:54.309 INFO:tasks.workunit.client.1.smithi150.stdout:0/446: symlink d0/d1/d84/d86/d11/d66/d37/l8a 0 2024-03-22T10:42:54.309 INFO:tasks.workunit.client.0.smithi115.stdout:9/810: dread f3 [0,4194304] 0 2024-03-22T10:42:54.310 INFO:tasks.workunit.client.1.smithi150.stdout:7/421: dwrite d2/d5/d7/d8/d2c/f47 [0,4194304] 0 2024-03-22T10:42:54.313 INFO:tasks.workunit.client.1.smithi150.stdout:0/447: write d0/f7b [288594,125631] 0 2024-03-22T10:42:54.317 INFO:tasks.workunit.client.1.smithi150.stdout:8/460: creat d0/d2/d19/d29/d37/f9c x:0 0 0 2024-03-22T10:42:54.318 INFO:tasks.workunit.client.1.smithi150.stdout:9/471: symlink d1/d8/de/df/d20/d32/d75/lad 0 2024-03-22T10:42:54.321 INFO:tasks.workunit.client.1.smithi150.stdout:4/455: creat d0/d1b/d4d/d6c/f9d x:0 0 0 2024-03-22T10:42:54.323 INFO:tasks.workunit.client.1.smithi150.stdout:1/377: chown d1/da/d2a/f35 19319 1 2024-03-22T10:42:54.323 INFO:tasks.workunit.client.1.smithi150.stdout:4/456: write d0/de/f18 [196662,93131] 0 2024-03-22T10:42:54.329 INFO:tasks.workunit.client.1.smithi150.stdout:0/448: creat d0/d1/d84/d86/d11/d66/f8b x:0 0 0 2024-03-22T10:42:54.332 INFO:tasks.workunit.client.1.smithi150.stdout:0/449: readlink d0/d1/d84/d86/d11/d26/l30 0 2024-03-22T10:42:54.335 INFO:tasks.workunit.client.1.smithi150.stdout:2/475: truncate d12/d19/d1a/f2b 1573030 0 2024-03-22T10:42:54.335 INFO:tasks.workunit.client.1.smithi150.stdout:9/472: mknod d1/d8/de/cae 0 2024-03-22T10:42:54.338 INFO:tasks.workunit.client.1.smithi150.stdout:6/425: mknod db/dd/d15/c97 0 2024-03-22T10:42:54.340 INFO:tasks.workunit.client.0.smithi115.stdout:2/754: dread da/f1d [4194304,4194304] 0 2024-03-22T10:42:54.340 INFO:tasks.workunit.client.0.smithi115.stdout:9/811: dread d6/d8/d3c/d59/d68/d8e/f9a [0,4194304] 0 2024-03-22T10:42:54.342 INFO:tasks.workunit.client.1.smithi150.stdout:0/450: read d0/d1/d84/d86/f31 [98766,53368] 0 2024-03-22T10:42:54.343 INFO:tasks.workunit.client.1.smithi150.stdout:2/476: dread - d12/d16/d67/f51 zero size 2024-03-22T10:42:54.344 INFO:tasks.workunit.client.0.smithi115.stdout:9/812: truncate d6/d7/d19/d4e/f109 546242 0 2024-03-22T10:42:54.344 INFO:tasks.workunit.client.1.smithi150.stdout:8/461: fsync d0/f21 0 2024-03-22T10:42:54.344 INFO:tasks.workunit.client.1.smithi150.stdout:9/473: unlink d1/f7b 0 2024-03-22T10:42:54.344 INFO:tasks.workunit.client.0.smithi115.stdout:2/755: creat da/d25/d3d/d57/d7d/da1/ddc/fe2 x:0 0 0 2024-03-22T10:42:54.345 INFO:tasks.workunit.client.0.smithi115.stdout:2/756: dread - da/d25/fb6 zero size 2024-03-22T10:42:54.346 INFO:tasks.workunit.client.1.smithi150.stdout:1/378: creat d1/da/d2a/d65/f76 x:0 0 0 2024-03-22T10:42:54.346 INFO:tasks.workunit.client.0.smithi115.stdout:9/813: mknod d6/d7/d30/d3b/c11e 0 2024-03-22T10:42:54.348 INFO:tasks.workunit.client.1.smithi150.stdout:6/426: creat db/dd/d15/d30/f98 x:0 0 0 2024-03-22T10:42:54.348 INFO:tasks.workunit.client.1.smithi150.stdout:4/457: dwrite d0/d32/d7c/f91 [0,4194304] 0 2024-03-22T10:42:54.350 INFO:tasks.workunit.client.0.smithi115.stdout:9/814: creat d6/d7/d6a/d6f/f11f x:0 0 0 2024-03-22T10:42:54.351 INFO:tasks.workunit.client.1.smithi150.stdout:0/451: readlink d0/d1/d84/d86/d42/d10/d38/d52/l55 0 2024-03-22T10:42:54.352 INFO:tasks.workunit.client.0.smithi115.stdout:9/815: mknod d6/d7/d87/c120 0 2024-03-22T10:42:54.355 INFO:tasks.workunit.client.1.smithi150.stdout:8/462: fsync d0/d2/d19/d29/d2f/f64 0 2024-03-22T10:42:54.359 INFO:tasks.workunit.client.1.smithi150.stdout:6/427: readlink db/dd/d15/d60/l78 0 2024-03-22T10:42:54.361 INFO:tasks.workunit.client.1.smithi150.stdout:3/403: dwrite d5/da/f33 [0,4194304] 0 2024-03-22T10:42:54.363 INFO:tasks.workunit.client.1.smithi150.stdout:0/452: fdatasync d0/d1/d84/d86/d42/d10/d38/d52/f65 0 2024-03-22T10:42:54.367 INFO:tasks.workunit.client.1.smithi150.stdout:9/474: symlink d1/laf 0 2024-03-22T10:42:54.368 INFO:tasks.workunit.client.0.smithi115.stdout:2/757: write da/f1d [2355808,7099] 0 2024-03-22T10:42:54.372 INFO:tasks.workunit.client.0.smithi115.stdout:9/816: dwrite d6/d8/d3c/d59/fb5 [0,4194304] 0 2024-03-22T10:42:54.373 INFO:tasks.workunit.client.0.smithi115.stdout:4/833: sync 2024-03-22T10:42:54.373 INFO:tasks.workunit.client.0.smithi115.stdout:1/868: sync 2024-03-22T10:42:54.373 INFO:tasks.workunit.client.0.smithi115.stdout:7/815: sync 2024-03-22T10:42:54.374 INFO:tasks.workunit.client.0.smithi115.stdout:5/841: sync 2024-03-22T10:42:54.374 INFO:tasks.workunit.client.1.smithi150.stdout:7/422: dwrite d2/d5/d7/d8/d2c/f40 [0,4194304] 0 2024-03-22T10:42:54.374 INFO:tasks.workunit.client.1.smithi150.stdout:4/458: creat d0/d1b/d4d/d6c/d61/f9e x:0 0 0 2024-03-22T10:42:54.375 INFO:tasks.workunit.client.0.smithi115.stdout:6/745: sync 2024-03-22T10:42:54.375 INFO:tasks.workunit.client.0.smithi115.stdout:3/855: sync 2024-03-22T10:42:54.375 INFO:tasks.workunit.client.0.smithi115.stdout:0/748: sync 2024-03-22T10:42:54.375 INFO:tasks.workunit.client.0.smithi115.stdout:2/758: creat da/fe3 x:0 0 0 2024-03-22T10:42:54.378 INFO:tasks.workunit.client.0.smithi115.stdout:4/834: chown d1/c15 1204127315 1 2024-03-22T10:42:54.380 INFO:tasks.workunit.client.0.smithi115.stdout:1/869: unlink d0/d8/d4d/f6e 0 2024-03-22T10:42:54.380 INFO:tasks.workunit.client.1.smithi150.stdout:0/453: symlink d0/d1/d81/l8c 0 2024-03-22T10:42:54.383 INFO:tasks.workunit.client.1.smithi150.stdout:1/379: getdents d1/d19 0 2024-03-22T10:42:54.383 INFO:tasks.workunit.client.1.smithi150.stdout:8/463: creat d0/d2/d19/d29/f9d x:0 0 0 2024-03-22T10:42:54.383 INFO:tasks.workunit.client.0.smithi115.stdout:4/835: fsync d1/d30/f4a 0 2024-03-22T10:42:54.385 INFO:tasks.workunit.client.0.smithi115.stdout:2/759: symlink da/d25/d3d/dad/db3/le4 0 2024-03-22T10:42:54.386 INFO:tasks.workunit.client.1.smithi150.stdout:2/477: dwrite d12/d48/d63/f6c [0,4194304] 0 2024-03-22T10:42:54.390 INFO:tasks.workunit.client.0.smithi115.stdout:9/817: creat d6/d7/d20/f121 x:0 0 0 2024-03-22T10:42:54.391 INFO:tasks.workunit.client.0.smithi115.stdout:0/749: mkdir d3/da/d4c/d74/d84/dac/df4 0 2024-03-22T10:42:54.391 INFO:tasks.workunit.client.1.smithi150.stdout:4/459: mkdir d0/d1b/d4d/d9f 0 2024-03-22T10:42:54.391 INFO:tasks.workunit.client.1.smithi150.stdout:7/423: mkdir d2/d5/d7/d8/d57/d4a/d8d 0 2024-03-22T10:42:54.392 INFO:tasks.workunit.client.1.smithi150.stdout:9/475: mknod d1/d8/de/df/da2/d8c/cb0 0 2024-03-22T10:42:54.392 INFO:tasks.workunit.client.0.smithi115.stdout:4/836: readlink d1/d30/d4c/d86/d9f/l11c 0 2024-03-22T10:42:54.394 INFO:tasks.workunit.client.0.smithi115.stdout:5/842: rename d2/df/d3b/fa0 to d2/df/d13/d8b/d93/da6/dbc/d7a/f112 0 2024-03-22T10:42:54.394 INFO:tasks.workunit.client.1.smithi150.stdout:0/454: creat d0/d1/d84/d86/f8d x:0 0 0 2024-03-22T10:42:54.397 INFO:tasks.workunit.client.1.smithi150.stdout:1/380: truncate d1/da/f72 667363 0 2024-03-22T10:42:54.397 INFO:tasks.workunit.client.1.smithi150.stdout:9/476: dread - d1/d19/f9f zero size 2024-03-22T10:42:54.400 INFO:tasks.workunit.client.0.smithi115.stdout:5/843: dread - d2/d15/d1b/d7c/f105 zero size 2024-03-22T10:42:54.400 INFO:tasks.workunit.client.0.smithi115.stdout:7/816: mkdir d2/d7/d12/d10/d19/dbd/dce/d105/d117 0 2024-03-22T10:42:54.401 INFO:tasks.workunit.client.1.smithi150.stdout:2/478: mknod d12/c9c 0 2024-03-22T10:42:54.401 INFO:tasks.workunit.client.0.smithi115.stdout:4/837: chown d1/d66/d106/d35/d40/d5c/d55/f50 713559898 1 2024-03-22T10:42:54.401 INFO:tasks.workunit.client.1.smithi150.stdout:7/424: mknod d2/d5/d7/d8/d57/d54/c8e 0 2024-03-22T10:42:54.402 INFO:tasks.workunit.client.0.smithi115.stdout:1/870: getdents d0/d12/d17/d37/d9e/de9/dde 0 2024-03-22T10:42:54.402 INFO:tasks.workunit.client.1.smithi150.stdout:5/327: sync 2024-03-22T10:42:54.402 INFO:tasks.workunit.client.1.smithi150.stdout:1/381: chown d1/da/d2a/d4e/l56 804692417 1 2024-03-22T10:42:54.405 INFO:tasks.workunit.client.0.smithi115.stdout:5/844: dread - d2/df/d13/f76 zero size 2024-03-22T10:42:54.406 INFO:tasks.workunit.client.0.smithi115.stdout:7/817: unlink d2/d62/ff2 0 2024-03-22T10:42:54.410 INFO:tasks.workunit.client.1.smithi150.stdout:5/328: symlink d0/d8/dc/d1d/d53/d31/l65 0 2024-03-22T10:42:54.411 INFO:tasks.workunit.client.0.smithi115.stdout:2/760: rename da/d25/d2a/c4a to da/d25/d3d/ce5 0 2024-03-22T10:42:54.412 INFO:tasks.workunit.client.0.smithi115.stdout:1/871: stat d0/d12/d17/d37/d9e/de9/dc1/fa2 0 2024-03-22T10:42:54.414 INFO:tasks.workunit.client.0.smithi115.stdout:4/838: mknod d1/d30/d4c/d86/d9f/de9/c11f 0 2024-03-22T10:42:54.414 INFO:tasks.workunit.client.1.smithi150.stdout:1/382: write d1/d19/d50/f66 [626619,110767] 0 2024-03-22T10:42:54.415 INFO:tasks.workunit.client.1.smithi150.stdout:3/404: truncate d5/da/f40 2845992 0 2024-03-22T10:42:54.417 INFO:tasks.workunit.client.0.smithi115.stdout:2/761: stat da/d25/d30/f2e 0 2024-03-22T10:42:54.418 INFO:tasks.workunit.client.0.smithi115.stdout:7/818: symlink d2/d7/d12/d10/d19/d42/d76/d74/l118 0 2024-03-22T10:42:54.419 INFO:tasks.workunit.client.1.smithi150.stdout:5/329: stat d0/d8/dc/de/d14/d2e/f62 0 2024-03-22T10:42:54.419 INFO:tasks.workunit.client.1.smithi150.stdout:8/464: dwrite d0/d2/d7e/f82 [0,4194304] 0 2024-03-22T10:42:54.424 INFO:tasks.workunit.client.1.smithi150.stdout:7/425: creat d2/d5/d7/d88/f8f x:0 0 0 2024-03-22T10:42:54.425 INFO:tasks.workunit.client.0.smithi115.stdout:2/762: write da/f14 [4527967,112369] 0 2024-03-22T10:42:54.425 INFO:tasks.workunit.client.0.smithi115.stdout:4/839: truncate d1/d66/d82/feb 1499043 0 2024-03-22T10:42:54.425 INFO:tasks.workunit.client.1.smithi150.stdout:2/479: creat d12/d16/d2c/d65/f9d x:0 0 0 2024-03-22T10:42:54.426 INFO:tasks.workunit.client.1.smithi150.stdout:4/460: dwrite d0/f11 [4194304,4194304] 0 2024-03-22T10:42:54.427 INFO:tasks.workunit.client.0.smithi115.stdout:1/872: unlink d0/d12/la4 0 2024-03-22T10:42:54.427 INFO:tasks.workunit.client.1.smithi150.stdout:6/428: write db/dd/d15/d60/f53 [1197996,3786] 0 2024-03-22T10:42:54.428 INFO:tasks.workunit.client.0.smithi115.stdout:7/819: readlink d2/d7/d12/d10/d19/d42/l82 0 2024-03-22T10:42:54.431 INFO:tasks.workunit.client.1.smithi150.stdout:2/480: chown d12/l14 1 1 2024-03-22T10:42:54.431 INFO:tasks.workunit.client.1.smithi150.stdout:3/405: stat d5/da/f40 0 2024-03-22T10:42:54.435 INFO:tasks.workunit.client.0.smithi115.stdout:9/818: rename d6/d7/d20/d22/d23/d106/da4/cbf to d6/d7/d20/c122 0 2024-03-22T10:42:54.438 INFO:tasks.workunit.client.1.smithi150.stdout:0/455: dwrite d0/d1/d84/d86/d5a/f73 [0,4194304] 0 2024-03-22T10:42:54.438 INFO:tasks.workunit.client.1.smithi150.stdout:1/383: rename d1/d19/d1a/f4d to d1/d19/d50/d6e/f77 0 2024-03-22T10:42:54.439 INFO:tasks.workunit.client.1.smithi150.stdout:4/461: read d0/d1b/f2b [1761193,38059] 0 2024-03-22T10:42:54.441 INFO:tasks.workunit.client.1.smithi150.stdout:0/456: chown d0/d1/d84/d86/d11/d26/l30 3295 1 2024-03-22T10:42:54.442 INFO:tasks.workunit.client.1.smithi150.stdout:8/465: mkdir d0/d2/d19/d29/d37/d7d/d9e 0 2024-03-22T10:42:54.443 INFO:tasks.workunit.client.1.smithi150.stdout:6/429: unlink db/dd/d15/f27 0 2024-03-22T10:42:54.445 INFO:tasks.workunit.client.1.smithi150.stdout:6/430: write db/dd/f1b [2995451,117003] 0 2024-03-22T10:42:54.447 INFO:tasks.workunit.client.0.smithi115.stdout:2/763: link da/d25/d3d/f2c da/d25/d51/fe6 0 2024-03-22T10:42:54.447 INFO:tasks.workunit.client.0.smithi115.stdout:0/750: dwrite d3/d7/d6b/d94/fe4 [0,4194304] 0 2024-03-22T10:42:54.449 INFO:tasks.workunit.client.1.smithi150.stdout:8/466: truncate d0/d2/f67 815661 0 2024-03-22T10:42:54.453 INFO:tasks.workunit.client.0.smithi115.stdout:6/746: dwrite d4/d36/d48/d72/f81 [4194304,4194304] 0 2024-03-22T10:42:54.454 INFO:tasks.workunit.client.0.smithi115.stdout:7/820: unlink d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f95 0 2024-03-22T10:42:54.456 INFO:tasks.workunit.client.1.smithi150.stdout:5/330: mkdir d0/d8/dc/de/d14/d2e/d5a/d66 0 2024-03-22T10:42:54.459 INFO:tasks.workunit.client.1.smithi150.stdout:4/462: unlink d0/de/f90 0 2024-03-22T10:42:54.460 INFO:tasks.workunit.client.1.smithi150.stdout:4/463: read d0/f9 [59376,52456] 0 2024-03-22T10:42:54.461 INFO:tasks.workunit.client.0.smithi115.stdout:3/856: dread d3/d89/f8b [0,4194304] 0 2024-03-22T10:42:54.465 INFO:tasks.workunit.client.0.smithi115.stdout:3/857: fdatasync d3/d36/d41/fde 0 2024-03-22T10:42:54.467 INFO:tasks.workunit.client.0.smithi115.stdout:4/840: rename d1/d30/d4c/d86/d9f/de9/cf6 to d1/d69/c120 0 2024-03-22T10:42:54.467 INFO:tasks.workunit.client.1.smithi150.stdout:0/457: mknod d0/d1/d84/d86/d42/d10/d53/c8e 0 2024-03-22T10:42:54.468 INFO:tasks.workunit.client.0.smithi115.stdout:3/858: dread - d3/d6d/db4/d107/da6/fe9 zero size 2024-03-22T10:42:54.469 INFO:tasks.workunit.client.1.smithi150.stdout:3/406: mkdir d5/da/d62/d84 0 2024-03-22T10:42:54.473 INFO:tasks.workunit.client.0.smithi115.stdout:1/873: unlink d0/d8/d4d/d56/dcf/d10b/de2/df8/l11d 0 2024-03-22T10:42:54.473 INFO:tasks.workunit.client.1.smithi150.stdout:3/407: write d5/d53/f75 [102896,100358] 0 2024-03-22T10:42:54.476 INFO:tasks.workunit.client.1.smithi150.stdout:5/331: mknod d0/d8/dc/de/d14/d2e/d46/c67 0 2024-03-22T10:42:54.482 INFO:tasks.workunit.client.1.smithi150.stdout:6/431: symlink db/dd/l99 0 2024-03-22T10:42:54.482 INFO:tasks.workunit.client.0.smithi115.stdout:2/764: mknod da/da5/ce7 0 2024-03-22T10:42:54.484 INFO:tasks.workunit.client.1.smithi150.stdout:8/467: symlink d0/l9f 0 2024-03-22T10:42:54.488 INFO:tasks.workunit.client.1.smithi150.stdout:6/432: creat db/dd/d82/f9a x:0 0 0 2024-03-22T10:42:54.489 INFO:tasks.workunit.client.1.smithi150.stdout:0/458: truncate d0/d1/d84/d86/d11/f2f 1433899 0 2024-03-22T10:42:54.490 INFO:tasks.workunit.client.0.smithi115.stdout:7/821: truncate d2/d7/d12/d10/d23/d3d/dd2/fba 1351875 0 2024-03-22T10:42:54.492 INFO:tasks.workunit.client.0.smithi115.stdout:7/822: readlink d2/le7 0 2024-03-22T10:42:54.493 INFO:tasks.workunit.client.0.smithi115.stdout:9/819: symlink d6/d7/d30/d37/dbe/d110/l123 0 2024-03-22T10:42:54.495 INFO:tasks.workunit.client.1.smithi150.stdout:1/384: write d1/d19/d1a/f49 [2731368,121883] 0 2024-03-22T10:42:54.497 INFO:tasks.workunit.client.1.smithi150.stdout:6/433: write db/dd/f2c [2225887,131045] 0 2024-03-22T10:42:54.500 INFO:tasks.workunit.client.1.smithi150.stdout:0/459: write d0/d1/d84/d86/f6b [159752,74960] 0 2024-03-22T10:42:54.502 INFO:tasks.workunit.client.0.smithi115.stdout:1/874: mkdir d0/d8/d4d/d56/db6/ddc/d135 0 2024-03-22T10:42:54.506 INFO:tasks.workunit.client.1.smithi150.stdout:0/460: chown d0/l16 152 1 2024-03-22T10:42:54.508 INFO:tasks.workunit.client.0.smithi115.stdout:5/845: truncate d2/d15/d1b/d7c/d38/d43/d49/fb9 2211527 0 2024-03-22T10:42:54.509 INFO:tasks.workunit.client.1.smithi150.stdout:1/385: mkdir d1/da/d2a/d65/d78 0 2024-03-22T10:42:54.511 INFO:tasks.workunit.client.1.smithi150.stdout:7/426: dwrite d2/d5/d7/f50 [0,4194304] 0 2024-03-22T10:42:54.512 INFO:tasks.workunit.client.1.smithi150.stdout:2/481: dwrite f11 [0,4194304] 0 2024-03-22T10:42:54.514 INFO:tasks.workunit.client.1.smithi150.stdout:6/434: mkdir db/dd/d15/d28/d6c/d9b 0 2024-03-22T10:42:54.514 INFO:tasks.workunit.client.0.smithi115.stdout:5/846: write d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f9b [2505145,75180] 0 2024-03-22T10:42:54.515 INFO:tasks.workunit.client.1.smithi150.stdout:4/464: dwrite d0/d1b/f2b [0,4194304] 0 2024-03-22T10:42:54.517 INFO:tasks.workunit.client.0.smithi115.stdout:5/847: chown d2/d15/d1b/c107 9165 1 2024-03-22T10:42:54.518 INFO:tasks.workunit.client.1.smithi150.stdout:0/461: write d0/d1/d84/d86/d42/f2b [4297765,24760] 0 2024-03-22T10:42:54.520 INFO:tasks.workunit.client.0.smithi115.stdout:4/841: dread d1/d66/d106/d35/d40/d5c/d14/f96 [0,4194304] 0 2024-03-22T10:42:54.522 INFO:tasks.workunit.client.1.smithi150.stdout:2/482: write d12/f26 [917339,65815] 0 2024-03-22T10:42:54.525 INFO:tasks.workunit.client.0.smithi115.stdout:3/859: link d3/d36/fee d3/d89/dca/f124 0 2024-03-22T10:42:54.526 INFO:tasks.workunit.client.1.smithi150.stdout:6/435: chown db/dd/d15/d60/d50/l88 6979091 1 2024-03-22T10:42:54.528 INFO:tasks.workunit.client.0.smithi115.stdout:3/860: chown d3/d82/d57 418707 1 2024-03-22T10:42:54.531 INFO:tasks.workunit.client.1.smithi150.stdout:6/436: write db/f44 [4875166,128657] 0 2024-03-22T10:42:54.531 INFO:tasks.workunit.client.1.smithi150.stdout:3/408: dwrite d5/d22/d37/f6c [0,4194304] 0 2024-03-22T10:42:54.532 INFO:tasks.workunit.client.1.smithi150.stdout:7/427: symlink d2/d5/d7/d8/d2c/d45/l90 0 2024-03-22T10:42:54.533 INFO:tasks.workunit.client.0.smithi115.stdout:3/861: fsync d3/d82/d52/fc0 0 2024-03-22T10:42:54.533 INFO:tasks.workunit.client.1.smithi150.stdout:8/468: dwrite d0/d2/d19/f78 [0,4194304] 0 2024-03-22T10:42:54.536 INFO:tasks.workunit.client.0.smithi115.stdout:2/765: dread da/d25/d30/dcd/f87 [0,4194304] 0 2024-03-22T10:42:54.536 INFO:tasks.workunit.client.0.smithi115.stdout:3/862: fdatasync d3/d89/fe5 0 2024-03-22T10:42:54.537 INFO:tasks.workunit.client.1.smithi150.stdout:2/483: symlink d12/d35/d7d/l9e 0 2024-03-22T10:42:54.541 INFO:tasks.workunit.client.0.smithi115.stdout:2/766: write da/d25/d3d/f4e [4225347,101686] 0 2024-03-22T10:42:54.541 INFO:tasks.workunit.client.0.smithi115.stdout:5/848: fdatasync d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f65 0 2024-03-22T10:42:54.542 INFO:tasks.workunit.client.0.smithi115.stdout:4/842: rename d1/d66/d106/d35/d40/d5c/d14/f90 to d1/d30/d4c/d86/d9f/db8/f121 0 2024-03-22T10:42:54.544 INFO:tasks.workunit.client.1.smithi150.stdout:8/469: dread - d0/d2/d19/d29/d31/f49 zero size 2024-03-22T10:42:54.544 INFO:tasks.workunit.client.1.smithi150.stdout:1/386: dread d1/da/d2a/f35 [0,4194304] 0 2024-03-22T10:42:54.546 INFO:tasks.workunit.client.1.smithi150.stdout:7/428: dread - d2/d5/d7/d38/f86 zero size 2024-03-22T10:42:54.547 INFO:tasks.workunit.client.1.smithi150.stdout:0/462: creat d0/d1/d84/d86/d42/d10/d38/d79/f8f x:0 0 0 2024-03-22T10:42:54.549 INFO:tasks.workunit.client.0.smithi115.stdout:9/820: creat d6/d8/f124 x:0 0 0 2024-03-22T10:42:54.549 INFO:tasks.workunit.client.0.smithi115.stdout:0/751: getdents d3/d7/d1b/d93 0 2024-03-22T10:42:54.550 INFO:tasks.workunit.client.1.smithi150.stdout:8/470: creat d0/d2/d19/d29/d2f/df/d1b/fa0 x:0 0 0 2024-03-22T10:42:54.552 INFO:tasks.workunit.client.1.smithi150.stdout:2/484: dread d12/d16/d67/d3c/f6b [0,4194304] 0 2024-03-22T10:42:54.553 INFO:tasks.workunit.client.0.smithi115.stdout:2/767: dread - da/d25/d3d/d57/d7d/da1/ddc/fe2 zero size 2024-03-22T10:42:54.553 INFO:tasks.workunit.client.1.smithi150.stdout:6/437: creat db/dd/d15/d60/f9c x:0 0 0 2024-03-22T10:42:54.556 INFO:tasks.workunit.client.1.smithi150.stdout:6/438: write db/dd/f5b [698798,116173] 0 2024-03-22T10:42:54.557 INFO:tasks.workunit.client.0.smithi115.stdout:6/747: dwrite d4/d11/d12/f9c [0,4194304] 0 2024-03-22T10:42:54.557 INFO:tasks.workunit.client.0.smithi115.stdout:3/863: truncate d3/d6d/db4/d107/da6/fc9 1044627 0 2024-03-22T10:42:54.557 INFO:tasks.workunit.client.1.smithi150.stdout:1/387: mkdir d1/d79 0 2024-03-22T10:42:54.560 INFO:tasks.workunit.client.1.smithi150.stdout:0/463: rmdir d0/d1/d81 39 2024-03-22T10:42:54.561 INFO:tasks.workunit.client.1.smithi150.stdout:6/439: stat db/d52/f6f 0 2024-03-22T10:42:54.561 INFO:tasks.workunit.client.0.smithi115.stdout:0/752: symlink d3/d4/d14/d64/lf5 0 2024-03-22T10:42:54.561 INFO:tasks.workunit.client.0.smithi115.stdout:5/849: unlink d2/d15/ff3 0 2024-03-22T10:42:54.562 INFO:tasks.workunit.client.0.smithi115.stdout:2/768: mknod da/d1c/ce8 0 2024-03-22T10:42:54.562 INFO:tasks.workunit.client.1.smithi150.stdout:7/429: chown d2/d5/d7/c14 9494571 1 2024-03-22T10:42:54.564 INFO:tasks.workunit.client.1.smithi150.stdout:8/471: mkdir d0/d2/d19/d29/d37/da1 0 2024-03-22T10:42:54.564 INFO:tasks.workunit.client.0.smithi115.stdout:2/769: readlink da/d25/d3d/l36 0 2024-03-22T10:42:54.566 INFO:tasks.workunit.client.0.smithi115.stdout:6/748: unlink d4/d13/d33/d58/f75 0 2024-03-22T10:42:54.566 INFO:tasks.workunit.client.1.smithi150.stdout:4/465: write d0/de/f4b [3049458,120868] 0 2024-03-22T10:42:54.568 INFO:tasks.workunit.client.1.smithi150.stdout:0/464: fsync d0/d1/d84/d86/d11/d66/f8b 0 2024-03-22T10:42:54.569 INFO:tasks.workunit.client.0.smithi115.stdout:3/864: write d3/d89/dca/fcb [8515,51424] 0 2024-03-22T10:42:54.569 INFO:tasks.workunit.client.1.smithi150.stdout:7/430: truncate d2/f67 222253 0 2024-03-22T10:42:54.571 INFO:tasks.workunit.client.0.smithi115.stdout:0/753: creat d3/d7/d1b/d1f/d78/ff6 x:0 0 0 2024-03-22T10:42:54.571 INFO:tasks.workunit.client.1.smithi150.stdout:3/409: dwrite d5/d22/d2f/f30 [0,4194304] 0 2024-03-22T10:42:54.575 INFO:tasks.workunit.client.1.smithi150.stdout:1/388: link d1/d19/d1a/f46 d1/da/d2a/f7a 0 2024-03-22T10:42:54.576 INFO:tasks.workunit.client.0.smithi115.stdout:4/843: link d1/d66/d106/f64 d1/d30/d4c/d86/d9f/d75/d99/f122 0 2024-03-22T10:42:54.578 INFO:tasks.workunit.client.0.smithi115.stdout:6/749: write d4/d13/d33/fba [554813,31348] 0 2024-03-22T10:42:54.579 INFO:tasks.workunit.client.1.smithi150.stdout:4/466: dread - d0/d1b/d4d/f92 zero size 2024-03-22T10:42:54.579 INFO:tasks.workunit.client.0.smithi115.stdout:7/823: dread d2/d7/d12/fa7 [0,4194304] 0 2024-03-22T10:42:54.581 INFO:tasks.workunit.client.1.smithi150.stdout:0/465: mkdir d0/d1/d84/d86/d5a/d90 0 2024-03-22T10:42:54.581 INFO:tasks.workunit.client.1.smithi150.stdout:1/389: fdatasync d1/d19/d57/f62 0 2024-03-22T10:42:54.582 INFO:tasks.workunit.client.1.smithi150.stdout:7/431: fsync d2/d5/d7/d9/db/f1a 0 2024-03-22T10:42:54.584 INFO:tasks.workunit.client.0.smithi115.stdout:5/850: creat d2/f113 x:0 0 0 2024-03-22T10:42:54.584 INFO:tasks.workunit.client.0.smithi115.stdout:0/754: rename d3/d4/d14/f23 to d3/da/d4c/d74/d84/ff7 0 2024-03-22T10:42:54.586 INFO:tasks.workunit.client.0.smithi115.stdout:6/750: mkdir d4/df3 0 2024-03-22T10:42:54.589 INFO:tasks.workunit.client.0.smithi115.stdout:7/824: creat d2/d7/d12/d10/d23/d3d/dd2/da3/df3/f119 x:0 0 0 2024-03-22T10:42:54.590 INFO:tasks.workunit.client.1.smithi150.stdout:3/410: read d5/da/f40 [1646330,127149] 0 2024-03-22T10:42:54.593 INFO:tasks.workunit.client.1.smithi150.stdout:0/466: mknod d0/d1/d84/d86/d11/d66/c91 0 2024-03-22T10:42:54.593 INFO:tasks.workunit.client.0.smithi115.stdout:0/755: mkdir d3/d7/d1b/df8 0 2024-03-22T10:42:54.595 INFO:tasks.workunit.client.1.smithi150.stdout:7/432: fdatasync d2/d5/d7/d8/d57/d4a/f39 0 2024-03-22T10:42:54.596 INFO:tasks.workunit.client.1.smithi150.stdout:1/390: symlink d1/da/d2a/d4e/l7b 0 2024-03-22T10:42:54.599 INFO:tasks.workunit.client.0.smithi115.stdout:0/756: mkdir d3/d7/d1b/d1f/d78/dd9/df9 0 2024-03-22T10:42:54.603 INFO:tasks.workunit.client.1.smithi150.stdout:3/411: creat d5/d22/d2f/d38/f85 x:0 0 0 2024-03-22T10:42:54.606 INFO:tasks.workunit.client.1.smithi150.stdout:1/391: mkdir d1/d7c 0 2024-03-22T10:42:54.607 INFO:tasks.workunit.client.0.smithi115.stdout:7/825: link d2/d7/d12/d10/d23/d3d/da4/l109 d2/d7/d12/d10/d23/d3d/da4/l11a 0 2024-03-22T10:42:54.608 INFO:tasks.workunit.client.1.smithi150.stdout:1/392: read d1/f6c [2141937,78810] 0 2024-03-22T10:42:54.611 INFO:tasks.workunit.client.0.smithi115.stdout:7/826: unlink d2/d7/d12/d10/d23/f38 0 2024-03-22T10:42:54.614 INFO:tasks.workunit.client.1.smithi150.stdout:3/412: rename d5/d22/d2f/d38/d59/d5d to d5/da/d78/d86 0 2024-03-22T10:42:54.616 INFO:tasks.workunit.client.0.smithi115.stdout:6/751: dwrite d4/d13/d33/f42 [4194304,4194304] 0 2024-03-22T10:42:54.616 INFO:tasks.workunit.client.1.smithi150.stdout:9/477: sync 2024-03-22T10:42:54.619 INFO:tasks.workunit.client.0.smithi115.stdout:7/827: creat d2/d7/d12/d10/d23/f11b x:0 0 0 2024-03-22T10:42:54.619 INFO:tasks.workunit.client.0.smithi115.stdout:6/752: write d4/d13/d33/d58/fb7 [1084582,124204] 0 2024-03-22T10:42:54.624 INFO:tasks.workunit.client.1.smithi150.stdout:3/413: symlink d5/d22/d2f/d38/d3e/l87 0 2024-03-22T10:42:54.628 INFO:tasks.workunit.client.1.smithi150.stdout:9/478: creat d1/d8/de/df/d20/d29/fb1 x:0 0 0 2024-03-22T10:42:54.629 INFO:tasks.workunit.client.1.smithi150.stdout:7/433: dwrite d2/d5/d7/d38/f4b [0,4194304] 0 2024-03-22T10:42:54.630 INFO:tasks.workunit.client.0.smithi115.stdout:7/828: link d2/d7/d12/d10/d23/f97 d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f11c 0 2024-03-22T10:42:54.634 INFO:tasks.workunit.client.1.smithi150.stdout:1/393: dwrite d1/f3a [0,4194304] 0 2024-03-22T10:42:54.634 INFO:tasks.workunit.client.0.smithi115.stdout:7/829: rmdir d2/d7/d12/d10/d19/d42/d76/d74 39 2024-03-22T10:42:54.636 INFO:tasks.workunit.client.1.smithi150.stdout:0/467: truncate d0/d1/d84/d86/d42/f44 3833388 0 2024-03-22T10:42:54.639 INFO:tasks.workunit.client.1.smithi150.stdout:3/414: mkdir d5/d22/d2f/d38/d59/d88 0 2024-03-22T10:42:54.641 INFO:tasks.workunit.client.0.smithi115.stdout:9/821: write d6/d7/d20/d38/f67 [457932,2220] 0 2024-03-22T10:42:54.641 INFO:tasks.workunit.client.1.smithi150.stdout:0/468: dread - d0/d1/d84/d86/d42/d10/d53/f5e zero size 2024-03-22T10:42:54.645 INFO:tasks.workunit.client.1.smithi150.stdout:3/415: write d5/d22/d2f/d38/f42 [4565029,19353] 0 2024-03-22T10:42:54.646 INFO:tasks.workunit.client.1.smithi150.stdout:7/434: dread d2/d5/d7/d8/d57/f5b [0,4194304] 0 2024-03-22T10:42:54.649 INFO:tasks.workunit.client.1.smithi150.stdout:1/394: write d1/f3c [4531293,38443] 0 2024-03-22T10:42:54.650 INFO:tasks.workunit.client.1.smithi150.stdout:0/469: unlink d0/d12/c27 0 2024-03-22T10:42:54.654 INFO:tasks.workunit.client.0.smithi115.stdout:6/753: truncate d4/d5e/f97 1150468 0 2024-03-22T10:42:54.654 INFO:tasks.workunit.client.0.smithi115.stdout:0/757: write d3/d7/d1b/d1f/f62 [805966,11559] 0 2024-03-22T10:42:54.655 INFO:tasks.workunit.client.0.smithi115.stdout:5/851: write d2/df/d13/fb3 [697731,122577] 0 2024-03-22T10:42:54.659 INFO:tasks.workunit.client.1.smithi150.stdout:9/479: dwrite d1/d19/f79 [0,4194304] 0 2024-03-22T10:42:54.659 INFO:tasks.workunit.client.1.smithi150.stdout:4/467: dread d0/d1b/f2e [0,4194304] 0 2024-03-22T10:42:54.660 INFO:tasks.workunit.client.1.smithi150.stdout:7/435: getdents d2/d5/d7/d9 0 2024-03-22T10:42:54.663 INFO:tasks.workunit.client.1.smithi150.stdout:1/395: mkdir d1/d19/d1a/d7d 0 2024-03-22T10:42:54.663 INFO:tasks.workunit.client.1.smithi150.stdout:4/468: read - d0/d1b/d4d/f65 zero size 2024-03-22T10:42:54.664 INFO:tasks.workunit.client.0.smithi115.stdout:0/758: write d3/d7/fd8 [761821,109720] 0 2024-03-22T10:42:54.667 INFO:tasks.workunit.client.1.smithi150.stdout:3/416: rename d5/d22/d2f/c36 to d5/d22/d2f/d38/d59/d88/c89 0 2024-03-22T10:42:54.669 INFO:tasks.workunit.client.1.smithi150.stdout:0/470: mkdir d0/d1/d84/d86/d11/d66/d35/d92 0 2024-03-22T10:42:54.676 INFO:tasks.workunit.client.0.smithi115.stdout:5/852: mknod d2/df/d13/d8b/d93/da6/dbc/d7a/c114 0 2024-03-22T10:42:54.676 INFO:tasks.workunit.client.1.smithi150.stdout:7/436: write d2/d5/d7/d8/d57/d4a/f22 [3897965,121251] 0 2024-03-22T10:42:54.677 INFO:tasks.workunit.client.1.smithi150.stdout:3/417: write d5/f4e [317461,16522] 0 2024-03-22T10:42:54.677 INFO:tasks.workunit.client.1.smithi150.stdout:1/396: symlink d1/d19/d50/d6e/l7e 0 2024-03-22T10:42:54.677 INFO:tasks.workunit.client.0.smithi115.stdout:0/759: symlink d3/da/d4c/d74/d84/dac/lfa 0 2024-03-22T10:42:54.678 INFO:tasks.workunit.client.1.smithi150.stdout:0/471: readlink d0/d1/l41 0 2024-03-22T10:42:54.678 INFO:tasks.workunit.client.1.smithi150.stdout:4/469: read d0/f5f [399842,60604] 0 2024-03-22T10:42:54.680 INFO:tasks.workunit.client.1.smithi150.stdout:7/437: chown d2/d5/d7 1 1 2024-03-22T10:42:54.682 INFO:tasks.workunit.client.1.smithi150.stdout:0/472: write d0/d1/d84/d86/d42/d10/d53/f5e [375550,62979] 0 2024-03-22T10:42:54.683 INFO:tasks.workunit.client.1.smithi150.stdout:9/480: rename d1/d8/de/df/d20/d32/f4f to d1/d8/de/df/d36/d40/d41/fb2 0 2024-03-22T10:42:54.683 INFO:tasks.workunit.client.0.smithi115.stdout:2/770: dwrite da/d1c/d67/d78/d86/fcc [0,4194304] 0 2024-03-22T10:42:54.685 INFO:tasks.workunit.client.1.smithi150.stdout:1/397: creat d1/da/d2a/f7f x:0 0 0 2024-03-22T10:42:54.685 INFO:tasks.workunit.client.1.smithi150.stdout:3/418: write d5/da/f45 [4999955,105445] 0 2024-03-22T10:42:54.687 INFO:tasks.workunit.client.1.smithi150.stdout:7/438: mkdir d2/d5/d7/d8/d57/d54/d91 0 2024-03-22T10:42:54.689 INFO:tasks.workunit.client.1.smithi150.stdout:4/470: creat d0/d1b/d4d/d6c/d61/d71/fa0 x:0 0 0 2024-03-22T10:42:54.689 INFO:tasks.workunit.client.0.smithi115.stdout:0/760: chown d3/d4/d14/d64/fab 39 1 2024-03-22T10:42:54.690 INFO:tasks.workunit.client.0.smithi115.stdout:9/822: dwrite d6/d7/d20/f121 [0,4194304] 0 2024-03-22T10:42:54.691 INFO:tasks.workunit.client.1.smithi150.stdout:3/419: fdatasync d5/da/f34 0 2024-03-22T10:42:54.694 INFO:tasks.workunit.client.1.smithi150.stdout:4/471: fsync d0/d2f/f3b 0 2024-03-22T10:42:54.696 INFO:tasks.workunit.client.1.smithi150.stdout:7/439: rmdir d2/d5/d7 39 2024-03-22T10:42:54.697 INFO:tasks.workunit.client.1.smithi150.stdout:3/420: write d5/d22/d2f/d38/d3e/d5b/f76 [231789,23421] 0 2024-03-22T10:42:54.699 INFO:tasks.workunit.client.0.smithi115.stdout:6/754: dwrite d4/d11/d12/d1c/d8a/d98/fef [0,4194304] 0 2024-03-22T10:42:54.699 INFO:tasks.workunit.client.0.smithi115.stdout:7/830: dread d2/d7/d12/d10/d23/d33/f70 [0,4194304] 0 2024-03-22T10:42:54.699 INFO:tasks.workunit.client.1.smithi150.stdout:4/472: unlink d0/de/f28 0 2024-03-22T10:42:54.703 INFO:tasks.workunit.client.1.smithi150.stdout:3/421: creat d5/d22/d2f/d38/d59/d88/f8a x:0 0 0 2024-03-22T10:42:54.704 INFO:tasks.workunit.client.0.smithi115.stdout:0/761: rename d3/d7/d1b/d1f/d78/ff6 to d3/d7/d1c/d6c/d9a/ffb 0 2024-03-22T10:42:54.706 INFO:tasks.workunit.client.0.smithi115.stdout:0/762: chown d3/d7/d1b/d93/fee 12546 1 2024-03-22T10:42:54.706 INFO:tasks.workunit.client.0.smithi115.stdout:9/823: truncate d6/d7/d30/d37/dbe/fc1 347187 0 2024-03-22T10:42:54.709 INFO:tasks.workunit.client.0.smithi115.stdout:6/755: write d4/d11/d12/d1c/f2d [1482527,51849] 0 2024-03-22T10:42:54.711 INFO:tasks.workunit.client.1.smithi150.stdout:3/422: symlink d5/da/d62/l8b 0 2024-03-22T10:42:54.716 INFO:tasks.workunit.client.1.smithi150.stdout:9/481: dwrite d1/d8/de/f17 [0,4194304] 0 2024-03-22T10:42:54.717 INFO:tasks.workunit.client.0.smithi115.stdout:2/771: write da/d1c/f88 [1116483,110757] 0 2024-03-22T10:42:54.717 INFO:tasks.workunit.client.0.smithi115.stdout:7/831: fdatasync d2/d7/d12/d10/d23/d3d/dd2/ff5 0 2024-03-22T10:42:54.723 INFO:tasks.workunit.client.0.smithi115.stdout:5/853: getdents d2/d15/d1b/d7c/d38/d43/d49 0 2024-03-22T10:42:54.723 INFO:tasks.workunit.client.0.smithi115.stdout:6/756: mkdir d4/d11/d12/d1c/d8a/d98/d9f/df0/df4 0 2024-03-22T10:42:54.725 INFO:tasks.workunit.client.1.smithi150.stdout:3/423: read d5/da/f57 [148702,2947] 0 2024-03-22T10:42:54.725 INFO:tasks.workunit.client.1.smithi150.stdout:0/473: write d0/d1/d84/d86/d42/d10/f43 [870653,2774] 0 2024-03-22T10:42:54.725 INFO:tasks.workunit.client.0.smithi115.stdout:9/824: creat d6/d7/d30/d37/f125 x:0 0 0 2024-03-22T10:42:54.727 INFO:tasks.workunit.client.0.smithi115.stdout:2/772: mknod da/d1c/ce9 0 2024-03-22T10:42:54.728 INFO:tasks.workunit.client.0.smithi115.stdout:6/757: read - d4/d5e/fe7 zero size 2024-03-22T10:42:54.729 INFO:tasks.workunit.client.1.smithi150.stdout:1/398: dwrite d1/da/d2a/f7f [0,4194304] 0 2024-03-22T10:42:54.731 INFO:tasks.workunit.client.0.smithi115.stdout:5/854: fdatasync d2/f71 0 2024-03-22T10:42:54.731 INFO:tasks.workunit.client.0.smithi115.stdout:6/758: readlink d4/d36/d48/lcb 0 2024-03-22T10:42:54.733 INFO:tasks.workunit.client.0.smithi115.stdout:9/825: chown d6/d7/d20/d38/f107 1290793844 1 2024-03-22T10:42:54.735 INFO:tasks.workunit.client.1.smithi150.stdout:7/440: dwrite d2/d5/d7/f50 [0,4194304] 0 2024-03-22T10:42:54.735 INFO:tasks.workunit.client.1.smithi150.stdout:9/482: mknod d1/d8/de/df/d36/cb3 0 2024-03-22T10:42:54.737 INFO:tasks.workunit.client.1.smithi150.stdout:1/399: write d1/d19/d1a/f2b [1583522,100659] 0 2024-03-22T10:42:54.737 INFO:tasks.workunit.client.0.smithi115.stdout:6/759: symlink d4/d36/d48/d7a/lf5 0 2024-03-22T10:42:54.741 INFO:tasks.workunit.client.1.smithi150.stdout:0/474: mknod d0/d1/d84/d86/d42/d10/d38/d52/c93 0 2024-03-22T10:42:54.741 INFO:tasks.workunit.client.1.smithi150.stdout:5/332: sync 2024-03-22T10:42:54.742 INFO:tasks.workunit.client.0.smithi115.stdout:0/763: dwrite d3/d7/d1c/d6c/d9a/ffb [0,4194304] 0 2024-03-22T10:42:54.742 INFO:tasks.workunit.client.0.smithi115.stdout:9/826: read d6/d7/f78 [840977,41190] 0 2024-03-22T10:42:54.742 INFO:tasks.workunit.client.1.smithi150.stdout:3/424: truncate d5/d22/d2f/d38/d59/d88/f8a 409061 0 2024-03-22T10:42:54.743 INFO:tasks.workunit.client.0.smithi115.stdout:7/832: truncate d2/d7/d12/d10/d19/d42/d76/f83 3692803 0 2024-03-22T10:42:54.744 INFO:tasks.workunit.client.1.smithi150.stdout:7/441: rename d2/d6e/f7e to d2/d5/d7/d9/f92 0 2024-03-22T10:42:54.745 INFO:tasks.workunit.client.1.smithi150.stdout:1/400: fsync d1/da/d15/d18/d4b/f53 0 2024-03-22T10:42:54.745 INFO:tasks.workunit.client.1.smithi150.stdout:5/333: dread - d0/d8/dc/f5b zero size 2024-03-22T10:42:54.751 INFO:tasks.workunit.client.1.smithi150.stdout:0/475: rmdir d0/d1/d84/d86/d11/d26 39 2024-03-22T10:42:54.752 INFO:tasks.workunit.client.1.smithi150.stdout:0/476: write d0/f56 [534959,1491] 0 2024-03-22T10:42:54.757 INFO:tasks.workunit.client.0.smithi115.stdout:2/773: getdents da/d25/d3d/d7e/d90/dbf/d54 0 2024-03-22T10:42:54.757 INFO:tasks.workunit.client.1.smithi150.stdout:5/334: mkdir d0/d8/dc/de/d14/d68 0 2024-03-22T10:42:54.758 INFO:tasks.workunit.client.0.smithi115.stdout:9/827: write d6/d7/d20/d22/fe3 [676263,76498] 0 2024-03-22T10:42:54.760 INFO:tasks.workunit.client.0.smithi115.stdout:2/774: chown da/d25/d3d/d57/d7d/dc6 2 1 2024-03-22T10:42:54.761 INFO:tasks.workunit.client.1.smithi150.stdout:9/483: truncate d1/d8/de/f2c 1421286 0 2024-03-22T10:42:54.761 INFO:tasks.workunit.client.1.smithi150.stdout:3/425: getdents d5/da/d78/d86/d81 0 2024-03-22T10:42:54.762 INFO:tasks.workunit.client.0.smithi115.stdout:9/828: chown d6/d8/ce 1 1 2024-03-22T10:42:54.764 INFO:tasks.workunit.client.0.smithi115.stdout:2/775: creat da/d25/d3d/d7e/fea x:0 0 0 2024-03-22T10:42:54.768 INFO:tasks.workunit.client.0.smithi115.stdout:2/776: creat da/d25/d83/feb x:0 0 0 2024-03-22T10:42:54.771 INFO:tasks.workunit.client.1.smithi150.stdout:9/484: write d1/d8/de/df/d24/f4a [1325268,23268] 0 2024-03-22T10:42:54.771 INFO:tasks.workunit.client.1.smithi150.stdout:3/426: fsync d5/d22/d2f/d38/d3e/f55 0 2024-03-22T10:42:54.772 INFO:tasks.workunit.client.1.smithi150.stdout:0/477: rename d0/d1/d84/d86/d11/d66/d37/l8a to d0/l94 0 2024-03-22T10:42:54.773 INFO:tasks.workunit.client.0.smithi115.stdout:0/764: dread d3/d7/d1c/d6c/d9a/fa5 [0,4194304] 0 2024-03-22T10:42:54.776 INFO:tasks.workunit.client.0.smithi115.stdout:0/765: stat d3/da/d4c/da3/da0/lc2 0 2024-03-22T10:42:54.776 INFO:tasks.workunit.client.0.smithi115.stdout:2/777: getdents da/d25/d3d/d57/d7d/da1/dba 0 2024-03-22T10:42:54.777 INFO:tasks.workunit.client.1.smithi150.stdout:5/335: symlink d0/d8/dc/de/d14/d2e/l69 0 2024-03-22T10:42:54.779 INFO:tasks.workunit.client.0.smithi115.stdout:2/778: write da/d1c/d67/d78/d86/fcc [2862295,112087] 0 2024-03-22T10:42:54.780 INFO:tasks.workunit.client.0.smithi115.stdout:0/766: mkdir d3/da/d4c/d74/d84/dac/dfc 0 2024-03-22T10:42:54.781 INFO:tasks.workunit.client.1.smithi150.stdout:7/442: dwrite d2/d6e/f83 [0,4194304] 0 2024-03-22T10:42:54.783 INFO:tasks.workunit.client.1.smithi150.stdout:3/427: mknod d5/da/d78/d86/c8c 0 2024-03-22T10:42:54.784 INFO:tasks.workunit.client.1.smithi150.stdout:9/485: read - d1/d8/de/df/d36/f96 zero size 2024-03-22T10:42:54.784 INFO:tasks.workunit.client.1.smithi150.stdout:1/401: dwrite d1/da/f75 [0,4194304] 0 2024-03-22T10:42:54.785 INFO:tasks.workunit.client.1.smithi150.stdout:0/478: chown d0/d1/d84/d86/d42/d10/c6e 22349 1 2024-03-22T10:42:54.786 INFO:tasks.workunit.client.0.smithi115.stdout:2/779: symlink da/d25/d51/lec 0 2024-03-22T10:42:54.790 INFO:tasks.workunit.client.0.smithi115.stdout:0/767: creat d3/d4/ffd x:0 0 0 2024-03-22T10:42:54.792 INFO:tasks.workunit.client.0.smithi115.stdout:2/780: chown da/d1c/d67/d78/d86 12916298 1 2024-03-22T10:42:54.792 INFO:tasks.workunit.client.0.smithi115.stdout:5/855: dwrite d2/d15/d1b/d7c/d38/d43/f82 [0,4194304] 0 2024-03-22T10:42:54.796 INFO:tasks.workunit.client.1.smithi150.stdout:7/443: mkdir d2/d5/d7/d8/d46/d93 0 2024-03-22T10:42:54.796 INFO:tasks.workunit.client.1.smithi150.stdout:1/402: write d1/f1d [4647234,28355] 0 2024-03-22T10:42:54.796 INFO:tasks.workunit.client.1.smithi150.stdout:4/473: dread d0/d47/f64 [0,4194304] 0 2024-03-22T10:42:54.797 INFO:tasks.workunit.client.1.smithi150.stdout:9/486: creat d1/d19/fb4 x:0 0 0 2024-03-22T10:42:54.800 INFO:tasks.workunit.client.0.smithi115.stdout:0/768: rmdir d3/df0/dca 39 2024-03-22T10:42:54.801 INFO:tasks.workunit.client.0.smithi115.stdout:6/760: dwrite d4/d13/d33/f6c [0,4194304] 0 2024-03-22T10:42:54.802 INFO:tasks.workunit.client.1.smithi150.stdout:5/336: link d0/d8/dc/de/l3b d0/d8/dc/d1d/d53/d31/l6a 0 2024-03-22T10:42:54.803 INFO:tasks.workunit.client.1.smithi150.stdout:7/444: creat d2/d5/d7/d88/f94 x:0 0 0 2024-03-22T10:42:54.804 INFO:tasks.workunit.client.1.smithi150.stdout:4/474: truncate d0/d1b/f74 348712 0 2024-03-22T10:42:54.807 INFO:tasks.workunit.client.1.smithi150.stdout:9/487: mkdir d1/d8/de/df/d36/d40/d44/db5 0 2024-03-22T10:42:54.807 INFO:tasks.workunit.client.1.smithi150.stdout:1/403: write d1/d19/d50/d6e/f77 [357414,52762] 0 2024-03-22T10:42:54.811 INFO:tasks.workunit.client.0.smithi115.stdout:0/769: symlink d3/da/lfe 0 2024-03-22T10:42:54.812 INFO:tasks.workunit.client.1.smithi150.stdout:0/479: getdents d0/d1/d84 0 2024-03-22T10:42:54.812 INFO:tasks.workunit.client.1.smithi150.stdout:3/428: dwrite d5/d22/d2f/d38/f42 [4194304,4194304] 0 2024-03-22T10:42:54.812 INFO:tasks.workunit.client.0.smithi115.stdout:7/833: dwrite d2/d7/d12/d10/d19/d42/d46/f48 [0,4194304] 0 2024-03-22T10:42:54.815 INFO:tasks.workunit.client.1.smithi150.stdout:5/337: creat d0/d8/dc/d1d/d53/d31/f6b x:0 0 0 2024-03-22T10:42:54.815 INFO:tasks.workunit.client.1.smithi150.stdout:0/480: readlink d0/d1/l7a 0 2024-03-22T10:42:54.816 INFO:tasks.workunit.client.0.smithi115.stdout:9/829: dread d6/d7/d6a/f9e [0,4194304] 0 2024-03-22T10:42:54.818 INFO:tasks.workunit.client.1.smithi150.stdout:1/404: fdatasync d1/d19/d57/f5b 0 2024-03-22T10:42:54.823 INFO:tasks.workunit.client.0.smithi115.stdout:5/856: read d2/d15/d1b/d7c/d38/d43/d49/fa9 [2863566,81664] 0 2024-03-22T10:42:54.823 INFO:tasks.workunit.client.1.smithi150.stdout:0/481: unlink d0/d12/c19 0 2024-03-22T10:42:54.824 INFO:tasks.workunit.client.0.smithi115.stdout:2/781: link da/c10 da/d25/d3d/d57/d7d/da1/dba/ced 0 2024-03-22T10:42:54.824 INFO:tasks.workunit.client.0.smithi115.stdout:0/770: write d3/f1a [4434419,4077] 0 2024-03-22T10:42:54.826 INFO:tasks.workunit.client.0.smithi115.stdout:2/782: readlink da/d25/d3d/d57/l91 0 2024-03-22T10:42:54.828 INFO:tasks.workunit.client.1.smithi150.stdout:5/338: creat d0/d8/dc/de/d14/d2e/d46/d4d/f6c x:0 0 0 2024-03-22T10:42:54.828 INFO:tasks.workunit.client.1.smithi150.stdout:7/445: link d2/d5/l6b d2/d5/d7/d38/l95 0 2024-03-22T10:42:54.829 INFO:tasks.workunit.client.0.smithi115.stdout:5/857: symlink d2/df/d3b/d8e/l115 0 2024-03-22T10:42:54.831 INFO:tasks.workunit.client.1.smithi150.stdout:3/429: truncate d5/d22/d2f/d38/d3e/f3f 3709269 0 2024-03-22T10:42:54.831 INFO:tasks.workunit.client.0.smithi115.stdout:2/783: write da/d25/d3d/f4e [2534,89135] 0 2024-03-22T10:42:54.831 INFO:tasks.workunit.client.0.smithi115.stdout:2/784: readlink da/l17 0 2024-03-22T10:42:54.832 INFO:tasks.workunit.client.1.smithi150.stdout:0/482: write d0/d1/d84/d86/d42/d10/d38/d79/f8f [799160,45961] 0 2024-03-22T10:42:54.833 INFO:tasks.workunit.client.1.smithi150.stdout:3/430: mkdir d5/da/d8d 0 2024-03-22T10:42:54.833 INFO:tasks.workunit.client.0.smithi115.stdout:0/771: symlink d3/d7/d1b/lff 0 2024-03-22T10:42:54.835 INFO:tasks.workunit.client.1.smithi150.stdout:7/446: dread d2/d5/d7/d8/d57/f5b [0,4194304] 0 2024-03-22T10:42:54.836 INFO:tasks.workunit.client.0.smithi115.stdout:0/772: chown d3/d52/l8c 174363749 1 2024-03-22T10:42:54.838 INFO:tasks.workunit.client.1.smithi150.stdout:3/431: creat d5/da/d62/f8e x:0 0 0 2024-03-22T10:42:54.838 INFO:tasks.workunit.client.0.smithi115.stdout:5/858: rename d2/d15/d1b/d7c/d38/d4a/f75 to d2/d15/d1b/f116 0 2024-03-22T10:42:54.840 INFO:tasks.workunit.client.1.smithi150.stdout:5/339: getdents d0 0 2024-03-22T10:42:54.841 INFO:tasks.workunit.client.1.smithi150.stdout:7/447: readlink d2/d5/d7/d38/l95 0 2024-03-22T10:42:54.847 INFO:tasks.workunit.client.1.smithi150.stdout:9/488: dwrite d1/d8/de/df/f5b [0,4194304] 0 2024-03-22T10:42:54.848 INFO:tasks.workunit.client.1.smithi150.stdout:7/448: unlink d2/d5/d7/d9/f65 0 2024-03-22T10:42:54.848 INFO:tasks.workunit.client.0.smithi115.stdout:0/773: link d3/d4/c25 d3/da/d4c/d69/c100 0 2024-03-22T10:42:54.850 INFO:tasks.workunit.client.1.smithi150.stdout:5/340: write d0/d8/dc/de/d14/f50 [3246979,11224] 0 2024-03-22T10:42:54.850 INFO:tasks.workunit.client.1.smithi150.stdout:3/432: rename d5/d22/d2f/d38/d3e/l56 to d5/d22/d2f/d38/l8f 0 2024-03-22T10:42:54.852 INFO:tasks.workunit.client.1.smithi150.stdout:9/489: unlink d1/d8/l1e 0 2024-03-22T10:42:54.859 INFO:tasks.workunit.client.0.smithi115.stdout:0/774: getdents d3/d7/d1c/d6c/d87/dd6 0 2024-03-22T10:42:54.859 INFO:tasks.workunit.client.0.smithi115.stdout:0/775: symlink d3/d7/d6b/d94/l101 0 2024-03-22T10:42:54.860 INFO:tasks.workunit.client.0.smithi115.stdout:9/830: dread d6/d7/d20/d9c/fa1 [0,4194304] 0 2024-03-22T10:42:54.860 INFO:tasks.workunit.client.1.smithi150.stdout:2/485: sync 2024-03-22T10:42:54.863 INFO:tasks.workunit.client.0.smithi115.stdout:5/859: dread d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f9b [0,4194304] 0 2024-03-22T10:42:54.865 INFO:tasks.workunit.client.1.smithi150.stdout:5/341: write d0/d8/dc/f12 [5016052,129870] 0 2024-03-22T10:42:54.866 INFO:tasks.workunit.client.0.smithi115.stdout:9/831: read d6/d7/d19/d4e/f6d [1455507,41007] 0 2024-03-22T10:42:54.867 INFO:tasks.workunit.client.0.smithi115.stdout:9/832: chown f3 204033 1 2024-03-22T10:42:54.870 INFO:tasks.workunit.client.0.smithi115.stdout:5/860: link d2/d15/d1b/d7c/d38/d43/ca4 d2/d15/d1b/d7c/d84/d89/c117 0 2024-03-22T10:42:54.872 INFO:tasks.workunit.client.0.smithi115.stdout:9/833: getdents d6/d7/d48/d70/da9/dbd/d11d 0 2024-03-22T10:42:54.874 INFO:tasks.workunit.client.0.smithi115.stdout:9/834: readlink d6/d7/d19/l117 0 2024-03-22T10:42:54.874 INFO:tasks.workunit.client.0.smithi115.stdout:7/834: write d2/d7/d12/d10/d19/d42/d76/f98 [1326592,39036] 0 2024-03-22T10:42:54.874 INFO:tasks.workunit.client.0.smithi115.stdout:5/861: dread - d2/df/d13/d8b/d93/da6/dbc/d50/d96/fcc zero size 2024-03-22T10:42:54.875 INFO:tasks.workunit.client.0.smithi115.stdout:2/785: write da/d25/d51/f63 [474135,48667] 0 2024-03-22T10:42:54.878 INFO:tasks.workunit.client.1.smithi150.stdout:7/449: dwrite d2/d5/d7/d8/d46/d49/f58 [0,4194304] 0 2024-03-22T10:42:54.880 INFO:tasks.workunit.client.0.smithi115.stdout:7/835: chown d2/d7/d12/d10/d23/f64 365833 1 2024-03-22T10:42:54.880 INFO:tasks.workunit.client.1.smithi150.stdout:7/450: stat d2/d6e/f83 0 2024-03-22T10:42:54.881 INFO:tasks.workunit.client.0.smithi115.stdout:9/835: mkdir d6/d7/d30/d37/dbe/d110/d126 0 2024-03-22T10:42:54.883 INFO:tasks.workunit.client.0.smithi115.stdout:6/761: dwrite d4/d11/d91/fb0 [0,4194304] 0 2024-03-22T10:42:54.883 INFO:tasks.workunit.client.0.smithi115.stdout:5/862: creat d2/d15/d1b/d7c/d84/f118 x:0 0 0 2024-03-22T10:42:54.884 INFO:tasks.workunit.client.1.smithi150.stdout:9/490: link d1/d8/de/df/d20/d32/c9c d1/d8/de/df/d36/d40/d44/db5/cb6 0 2024-03-22T10:42:54.885 INFO:tasks.workunit.client.1.smithi150.stdout:7/451: chown d2/d5/d7/d8/d57/d54/f55 9818 1 2024-03-22T10:42:54.885 INFO:tasks.workunit.client.1.smithi150.stdout:3/433: dread d5/da/f57 [0,4194304] 0 2024-03-22T10:42:54.886 INFO:tasks.workunit.client.0.smithi115.stdout:9/836: creat d6/d7/d6a/f127 x:0 0 0 2024-03-22T10:42:54.887 INFO:tasks.workunit.client.0.smithi115.stdout:7/836: creat d2/d7/d12/d10/d19/d42/d46/d94/f11d x:0 0 0 2024-03-22T10:42:54.890 INFO:tasks.workunit.client.0.smithi115.stdout:2/786: fsync da/d25/d3d/d57/fbc 0 2024-03-22T10:42:54.891 INFO:tasks.workunit.client.0.smithi115.stdout:6/762: creat d4/d11/d12/d1c/d8a/d98/d9f/ff6 x:0 0 0 2024-03-22T10:42:54.893 INFO:tasks.workunit.client.1.smithi150.stdout:4/475: truncate d0/d1b/f74 257212 0 2024-03-22T10:42:54.893 INFO:tasks.workunit.client.1.smithi150.stdout:7/452: write d2/f60 [209764,89118] 0 2024-03-22T10:42:54.897 INFO:tasks.workunit.client.0.smithi115.stdout:7/837: fsync d2/d7/d12/d10/f7b 0 2024-03-22T10:42:54.898 INFO:tasks.workunit.client.0.smithi115.stdout:5/863: truncate d2/d15/d1b/d7c/d38/fd2 644144 0 2024-03-22T10:42:54.899 INFO:tasks.workunit.client.1.smithi150.stdout:4/476: chown d0/de 3012 1 2024-03-22T10:42:54.903 INFO:tasks.workunit.client.1.smithi150.stdout:3/434: mknod d5/d22/d2f/d38/d59/d88/c90 0 2024-03-22T10:42:54.903 INFO:tasks.workunit.client.0.smithi115.stdout:5/864: dread - d2/df/d13/d8b/d93/da6/fcf zero size 2024-03-22T10:42:54.904 INFO:tasks.workunit.client.1.smithi150.stdout:5/342: dwrite d0/d8/dc/f40 [0,4194304] 0 2024-03-22T10:42:54.908 INFO:tasks.workunit.client.1.smithi150.stdout:3/435: stat d5/da/d78/d86/d81 0 2024-03-22T10:42:54.909 INFO:tasks.workunit.client.0.smithi115.stdout:7/838: creat d2/d7/d12/d10/d23/d3d/dd2/da3/df3/f11e x:0 0 0 2024-03-22T10:42:54.909 INFO:tasks.workunit.client.1.smithi150.stdout:1/405: dread d1/da/d15/f3e [0,4194304] 0 2024-03-22T10:42:54.910 INFO:tasks.workunit.client.1.smithi150.stdout:7/453: rename d2/d6e to d2/d5/d7/d8/d2c/d96 0 2024-03-22T10:42:54.911 INFO:tasks.workunit.client.1.smithi150.stdout:4/477: fdatasync d0/d1b/d68/f6a 0 2024-03-22T10:42:54.912 INFO:tasks.workunit.client.1.smithi150.stdout:5/343: truncate d0/d8/dc/f5b 605189 0 2024-03-22T10:42:54.913 INFO:tasks.workunit.client.0.smithi115.stdout:9/837: dwrite d6/d7/d20/d38/f67 [0,4194304] 0 2024-03-22T10:42:54.914 INFO:tasks.workunit.client.0.smithi115.stdout:5/865: creat d2/df/d13/d8b/d93/da6/dbc/d50/db1/f119 x:0 0 0 2024-03-22T10:42:54.916 INFO:tasks.workunit.client.0.smithi115.stdout:2/787: dwrite da/d25/d3d/f76 [0,4194304] 0 2024-03-22T10:42:54.918 INFO:tasks.workunit.client.1.smithi150.stdout:3/436: fdatasync d5/d22/d2f/d38/d59/d88/f8a 0 2024-03-22T10:42:54.920 INFO:tasks.workunit.client.1.smithi150.stdout:9/491: link d1/d8/de/df/d20/d32/d75/d99/laa d1/d8/de/df/d36/d40/d41/lb7 0 2024-03-22T10:42:54.922 INFO:tasks.workunit.client.0.smithi115.stdout:7/839: write d2/d62/f108 [324847,67424] 0 2024-03-22T10:42:54.924 INFO:tasks.workunit.client.0.smithi115.stdout:5/866: mknod d2/d15/c11a 0 2024-03-22T10:42:54.925 INFO:tasks.workunit.client.0.smithi115.stdout:2/788: creat da/d25/d3d/d7e/d90/d96/fee x:0 0 0 2024-03-22T10:42:54.929 INFO:tasks.workunit.client.0.smithi115.stdout:9/838: link d6/d7/d20/d38/l84 d6/d7/d87/l128 0 2024-03-22T10:42:54.931 INFO:tasks.workunit.client.1.smithi150.stdout:5/344: write d0/d8/dc/de/d14/d2e/d45/f2a [3691090,53074] 0 2024-03-22T10:42:54.940 INFO:tasks.workunit.client.1.smithi150.stdout:7/454: truncate d2/d5/d7/d8/f5e 2017194 0 2024-03-22T10:42:54.940 INFO:tasks.workunit.client.0.smithi115.stdout:0/776: dread d3/d7/d1c/f1e [0,4194304] 0 2024-03-22T10:42:54.941 INFO:tasks.workunit.client.1.smithi150.stdout:1/406: getdents d1/d19/d50/d51 0 2024-03-22T10:42:54.942 INFO:tasks.workunit.client.0.smithi115.stdout:0/777: fdatasync d3/d7/d1b/f47 0 2024-03-22T10:42:54.943 INFO:tasks.workunit.client.0.smithi115.stdout:6/763: dread d4/d11/d12/d1c/f43 [0,4194304] 0 2024-03-22T10:42:54.945 INFO:tasks.workunit.client.0.smithi115.stdout:9/839: dwrite d6/d7/d19/f25 [0,4194304] 0 2024-03-22T10:42:54.948 INFO:tasks.workunit.client.0.smithi115.stdout:0/778: rename d3/d4/ffd to d3/d7/d1c/f102 0 2024-03-22T10:42:54.949 INFO:tasks.workunit.client.1.smithi150.stdout:4/478: dread d0/d1b/f2b [0,4194304] 0 2024-03-22T10:42:54.950 INFO:tasks.workunit.client.0.smithi115.stdout:0/779: chown d3/d4/c2b 61 1 2024-03-22T10:42:54.954 INFO:tasks.workunit.client.0.smithi115.stdout:0/780: truncate d3/d4/d14/f15 3691042 0 2024-03-22T10:42:54.956 INFO:tasks.workunit.client.0.smithi115.stdout:9/840: link d6/d8/d3c/d59/d68/da8/cda d6/d7/d48/d70/da9/dbd/d11d/c129 0 2024-03-22T10:42:54.958 INFO:tasks.workunit.client.0.smithi115.stdout:0/781: fdatasync d3/d52/f57 0 2024-03-22T10:42:54.962 INFO:tasks.workunit.client.0.smithi115.stdout:0/782: write d3/f7f [693791,113082] 0 2024-03-22T10:42:54.962 INFO:tasks.workunit.client.0.smithi115.stdout:9/841: fsync d6/d7/d30/d3b/f46 0 2024-03-22T10:42:54.964 INFO:tasks.workunit.client.1.smithi150.stdout:9/492: dwrite d1/d8/de/df/da2/d8c/fa0 [0,4194304] 0 2024-03-22T10:42:54.965 INFO:tasks.workunit.client.0.smithi115.stdout:9/842: read - d6/d7/d20/f55 zero size 2024-03-22T10:42:54.969 INFO:tasks.workunit.client.0.smithi115.stdout:9/843: chown d6/d7/d20/d22/d3f/l45 83167952 1 2024-03-22T10:42:54.970 INFO:tasks.workunit.client.1.smithi150.stdout:3/437: dwrite d5/d22/d2f/d38/f39 [0,4194304] 0 2024-03-22T10:42:54.972 INFO:tasks.workunit.client.0.smithi115.stdout:0/783: getdents d3/d4/d14/d64 0 2024-03-22T10:42:54.973 INFO:tasks.workunit.client.0.smithi115.stdout:0/784: readlink d3/lc0 0 2024-03-22T10:42:54.975 INFO:tasks.workunit.client.1.smithi150.stdout:2/486: dread d12/d19/f59 [0,4194304] 0 2024-03-22T10:42:54.978 INFO:tasks.workunit.client.1.smithi150.stdout:5/345: dwrite d0/d8/dc/d1d/d53/d31/f6b [0,4194304] 0 2024-03-22T10:42:54.978 INFO:tasks.workunit.client.1.smithi150.stdout:4/479: rename d0/de/f18 to d0/d32/d7c/fa1 0 2024-03-22T10:42:54.979 INFO:tasks.workunit.client.0.smithi115.stdout:7/840: dread d2/d7/d12/d10/d23/d3d/f2a [0,4194304] 0 2024-03-22T10:42:54.980 INFO:tasks.workunit.client.0.smithi115.stdout:0/785: rmdir d3/d7/d1b/d1f/d78 39 2024-03-22T10:42:54.984 INFO:tasks.workunit.client.1.smithi150.stdout:4/480: creat d0/d32/fa2 x:0 0 0 2024-03-22T10:42:54.986 INFO:tasks.workunit.client.0.smithi115.stdout:0/786: unlink d3/da/d4c/da3/f55 0 2024-03-22T10:42:54.987 INFO:tasks.workunit.client.1.smithi150.stdout:3/438: write d5/d22/d2f/d38/f42 [7735257,38976] 0 2024-03-22T10:42:54.988 INFO:tasks.workunit.client.0.smithi115.stdout:2/789: dread da/d25/d30/f43 [0,4194304] 0 2024-03-22T10:42:54.988 INFO:tasks.workunit.client.1.smithi150.stdout:2/487: dread - d12/d16/d67/f5a zero size 2024-03-22T10:42:54.988 INFO:tasks.workunit.client.0.smithi115.stdout:7/841: rename d2/d7/d12/d10/d23/d3d/da4/de5 to d2/d7/d12/d10/d23/d3d/dd2/da3/de8/dc7/d11f 0 2024-03-22T10:42:54.990 INFO:tasks.workunit.client.1.smithi150.stdout:7/455: dwrite d2/d5/d7/d8/d2c/f6f [0,4194304] 0 2024-03-22T10:42:54.992 INFO:tasks.workunit.client.0.smithi115.stdout:7/842: read - d2/d7/d12/d10/d19/d42/d46/ffc zero size 2024-03-22T10:42:54.994 INFO:tasks.workunit.client.1.smithi150.stdout:1/407: dwrite d1/da/d15/f3e [4194304,4194304] 0 2024-03-22T10:42:54.996 INFO:tasks.workunit.client.1.smithi150.stdout:8/472: sync 2024-03-22T10:42:54.996 INFO:tasks.workunit.client.1.smithi150.stdout:6/440: sync 2024-03-22T10:42:54.999 INFO:tasks.workunit.client.0.smithi115.stdout:7/843: creat d2/d7/d12/d10/d19/dbd/dce/d105/f120 x:0 0 0 2024-03-22T10:42:55.002 INFO:tasks.workunit.client.1.smithi150.stdout:2/488: write d12/d48/d63/f6c [456097,58780] 0 2024-03-22T10:42:55.003 INFO:tasks.workunit.client.0.smithi115.stdout:2/790: getdents da/d25/d3d/d7e/d90/dbf 0 2024-03-22T10:42:55.006 INFO:tasks.workunit.client.0.smithi115.stdout:6/764: dwrite d4/d11/d12/f31 [4194304,4194304] 0 2024-03-22T10:42:55.007 INFO:tasks.workunit.client.1.smithi150.stdout:1/408: fdatasync d1/da/d15/d18/f5f 0 2024-03-22T10:42:55.007 INFO:tasks.workunit.client.0.smithi115.stdout:9/844: truncate d6/d7/d30/d37/dbe/fd4 262821 0 2024-03-22T10:42:55.007 INFO:tasks.workunit.client.0.smithi115.stdout:0/787: write d3/fa9 [3456695,55731] 0 2024-03-22T10:42:55.008 INFO:tasks.workunit.client.0.smithi115.stdout:6/765: readlink d4/d13/d33/l57 0 2024-03-22T10:42:55.010 INFO:tasks.workunit.client.1.smithi150.stdout:9/493: dwrite d1/d8/de/df/d24/d4e/f93 [0,4194304] 0 2024-03-22T10:42:55.012 INFO:tasks.workunit.client.0.smithi115.stdout:9/845: fsync d6/d7/d20/d38/f56 0 2024-03-22T10:42:55.013 INFO:tasks.workunit.client.0.smithi115.stdout:0/788: mknod d3/da/d4c/d74/d84/dac/c103 0 2024-03-22T10:42:55.013 INFO:tasks.workunit.client.0.smithi115.stdout:2/791: read da/d25/d30/f2e [4954312,5285] 0 2024-03-22T10:42:55.015 INFO:tasks.workunit.client.0.smithi115.stdout:6/766: symlink d4/d11/d12/d1c/d8a/d98/d9f/df0/lf7 0 2024-03-22T10:42:55.015 INFO:tasks.workunit.client.1.smithi150.stdout:2/489: creat d12/d48/f9f x:0 0 0 2024-03-22T10:42:55.018 INFO:tasks.workunit.client.0.smithi115.stdout:2/792: creat da/d25/d51/fef x:0 0 0 2024-03-22T10:42:55.019 INFO:tasks.workunit.client.1.smithi150.stdout:2/490: readlink d12/d19/l20 0 2024-03-22T10:42:55.019 INFO:tasks.workunit.client.0.smithi115.stdout:6/767: read - d4/d36/d48/d72/daa/db9/fd7 zero size 2024-03-22T10:42:55.021 INFO:tasks.workunit.client.0.smithi115.stdout:9/846: creat d6/d7/d48/d70/f12a x:0 0 0 2024-03-22T10:42:55.021 INFO:tasks.workunit.client.0.smithi115.stdout:0/789: truncate d3/d4/f71 306897 0 2024-03-22T10:42:55.023 INFO:tasks.workunit.client.0.smithi115.stdout:6/768: write d4/d36/d48/d72/f81 [2932959,5076] 0 2024-03-22T10:42:55.024 INFO:tasks.workunit.client.0.smithi115.stdout:9/847: chown d6/d8/d3c/d59/d68/f94 20 1 2024-03-22T10:42:55.026 INFO:tasks.workunit.client.1.smithi150.stdout:4/481: symlink d0/d1b/d4d/d6c/d61/d98/la3 0 2024-03-22T10:42:55.027 INFO:tasks.workunit.client.0.smithi115.stdout:2/793: getdents da/d25/d30/dcd 0 2024-03-22T10:42:55.028 INFO:tasks.workunit.client.0.smithi115.stdout:6/769: mkdir d4/d11/d12/d1c/d8a/d98/d9f/df0/df8 0 2024-03-22T10:42:55.030 INFO:tasks.workunit.client.0.smithi115.stdout:0/790: rename d3/df0/dca/de0 to d3/d7/d1b/d93/d104 0 2024-03-22T10:42:55.032 INFO:tasks.workunit.client.0.smithi115.stdout:6/770: creat d4/d11/d91/ff9 x:0 0 0 2024-03-22T10:42:55.032 INFO:tasks.workunit.client.0.smithi115.stdout:9/848: mknod d6/d7/d30/d37/dbe/d110/d126/c12b 0 2024-03-22T10:42:55.032 INFO:tasks.workunit.client.1.smithi150.stdout:3/439: dwrite d5/d22/d2f/f68 [0,4194304] 0 2024-03-22T10:42:55.034 INFO:tasks.workunit.client.0.smithi115.stdout:2/794: rename da/d1c/d67/d78/f95 to da/d25/d30/dcd/ff0 0 2024-03-22T10:42:55.035 INFO:tasks.workunit.client.0.smithi115.stdout:6/771: mknod d4/d36/d48/d72/daa/cfa 0 2024-03-22T10:42:55.036 INFO:tasks.workunit.client.0.smithi115.stdout:2/795: readlink da/d25/d3d/d57/l91 0 2024-03-22T10:42:55.036 INFO:tasks.workunit.client.0.smithi115.stdout:9/849: symlink d6/d7/d20/d22/d23/l12c 0 2024-03-22T10:42:55.037 INFO:tasks.workunit.client.0.smithi115.stdout:0/791: truncate d3/d4/d14/d64/fab 222443 0 2024-03-22T10:42:55.037 INFO:tasks.workunit.client.1.smithi150.stdout:2/491: readlink d12/d19/l24 0 2024-03-22T10:42:55.039 INFO:tasks.workunit.client.1.smithi150.stdout:4/482: stat d0/d47/l79 0 2024-03-22T10:42:55.039 INFO:tasks.workunit.client.0.smithi115.stdout:0/792: chown d3/d7/d6b/d94/l101 120026144 1 2024-03-22T10:42:55.042 INFO:tasks.workunit.client.0.smithi115.stdout:9/850: fsync d6/d7/f66 0 2024-03-22T10:42:55.043 INFO:tasks.workunit.client.1.smithi150.stdout:3/440: symlink d5/d22/d2f/d38/d3e/d72/l91 0 2024-03-22T10:42:55.046 INFO:tasks.workunit.client.1.smithi150.stdout:4/483: write d0/d1b/d4d/d6c/d61/f48 [1378995,62264] 0 2024-03-22T10:42:55.047 INFO:tasks.workunit.client.1.smithi150.stdout:8/473: getdents d0/d2/d19/d29/d69 0 2024-03-22T10:42:55.050 INFO:tasks.workunit.client.1.smithi150.stdout:4/484: unlink d0/d2f/c31 0 2024-03-22T10:42:55.050 INFO:tasks.workunit.client.1.smithi150.stdout:3/441: symlink d5/d53/d7e/l92 0 2024-03-22T10:42:55.051 INFO:tasks.workunit.client.1.smithi150.stdout:8/474: mkdir d0/d2/d46/da2 0 2024-03-22T10:42:55.057 INFO:tasks.workunit.client.1.smithi150.stdout:3/442: creat d5/da/d78/d86/f93 x:0 0 0 2024-03-22T10:42:55.058 INFO:tasks.workunit.client.0.smithi115.stdout:6/772: dwrite d4/d13/d33/dbe/fd9 [0,4194304] 0 2024-03-22T10:42:55.058 INFO:tasks.workunit.client.1.smithi150.stdout:1/409: dread d1/d19/d50/d6e/f77 [0,4194304] 0 2024-03-22T10:42:55.059 INFO:tasks.workunit.client.1.smithi150.stdout:8/475: creat d0/d2/d19/d29/d2f/df/fa3 x:0 0 0 2024-03-22T10:42:55.062 INFO:tasks.workunit.client.1.smithi150.stdout:3/443: rename d5/c20 to d5/da/d8d/c94 0 2024-03-22T10:42:55.065 INFO:tasks.workunit.client.1.smithi150.stdout:8/476: getdents d0/d2/d19/d29/d37/d7d/d9e 0 2024-03-22T10:42:55.066 INFO:tasks.workunit.client.0.smithi115.stdout:0/793: write d3/d4/d14/f34 [3935292,50279] 0 2024-03-22T10:42:55.071 INFO:tasks.workunit.client.1.smithi150.stdout:8/477: write d0/d2/d19/d29/d2e/f3a [3196420,91446] 0 2024-03-22T10:42:55.071 INFO:tasks.workunit.client.0.smithi115.stdout:9/851: dwrite d6/d8/d3c/d59/d68/d8e/f9a [4194304,4194304] 0 2024-03-22T10:42:55.075 INFO:tasks.workunit.client.0.smithi115.stdout:0/794: symlink d3/d7/d1b/d93/l105 0 2024-03-22T10:42:55.077 INFO:tasks.workunit.client.1.smithi150.stdout:8/478: creat d0/d2/d19/d29/d37/da1/fa4 x:0 0 0 2024-03-22T10:42:55.077 INFO:tasks.workunit.client.0.smithi115.stdout:6/773: link d4/d13/caf d4/d11/cfb 0 2024-03-22T10:42:55.080 INFO:tasks.workunit.client.0.smithi115.stdout:2/796: dwrite da/d1c/f4c [0,4194304] 0 2024-03-22T10:42:55.083 INFO:tasks.workunit.client.1.smithi150.stdout:3/444: dwrite d5/d53/f54 [0,4194304] 0 2024-03-22T10:42:55.084 INFO:tasks.workunit.client.0.smithi115.stdout:2/797: chown da/d25/d3d/d7e/d90/dbf/d54/cb5 1000 0 2024-03-22T10:42:55.086 INFO:tasks.workunit.client.1.smithi150.stdout:3/445: mkdir d5/d53/d7e/d95 0 2024-03-22T10:42:55.086 INFO:tasks.workunit.client.0.smithi115.stdout:0/795: write d3/d7/d1c/f48 [4250678,101669] 0 2024-03-22T10:42:55.089 INFO:tasks.workunit.client.1.smithi150.stdout:4/485: dread d0/f93 [0,4194304] 0 2024-03-22T10:42:55.089 INFO:tasks.workunit.client.0.smithi115.stdout:2/798: rmdir da/d1c/d67/d78/d86/db4 39 2024-03-22T10:42:55.089 INFO:tasks.workunit.client.1.smithi150.stdout:3/446: symlink d5/d53/l96 0 2024-03-22T10:42:55.095 INFO:tasks.workunit.client.0.smithi115.stdout:9/852: dwrite f0 [0,4194304] 0 2024-03-22T10:42:55.098 INFO:tasks.workunit.client.0.smithi115.stdout:2/799: rename da/d25/d30/f43 to da/d25/d3d/d7e/d90/dbf/d54/ff1 0 2024-03-22T10:42:55.098 INFO:tasks.workunit.client.0.smithi115.stdout:9/853: stat d6/d8/d3c/d59/lcb 0 2024-03-22T10:42:55.101 INFO:tasks.workunit.client.0.smithi115.stdout:4/844: sync 2024-03-22T10:42:55.101 INFO:tasks.workunit.client.0.smithi115.stdout:1/875: sync 2024-03-22T10:42:55.101 INFO:tasks.workunit.client.0.smithi115.stdout:3/865: sync 2024-03-22T10:42:55.102 INFO:tasks.workunit.client.0.smithi115.stdout:4/845: fsync d1/d30/d4c/d86/fac 0 2024-03-22T10:42:55.105 INFO:tasks.workunit.client.0.smithi115.stdout:2/800: truncate da/d25/d51/fd4 835337 0 2024-03-22T10:42:55.111 INFO:tasks.workunit.client.0.smithi115.stdout:0/796: dwrite d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:55.113 INFO:tasks.workunit.client.1.smithi150.stdout:4/486: dwrite d0/d32/d7c/f91 [4194304,4194304] 0 2024-03-22T10:42:55.117 INFO:tasks.workunit.client.0.smithi115.stdout:2/801: creat da/d25/d3d/d57/d7d/dc6/ff2 x:0 0 0 2024-03-22T10:42:55.121 INFO:tasks.workunit.client.1.smithi150.stdout:3/447: dwrite d5/d22/d2f/d38/d3e/f7d [0,4194304] 0 2024-03-22T10:42:55.126 INFO:tasks.workunit.client.0.smithi115.stdout:2/802: stat da/d25/d3d/d57/c5a 0 2024-03-22T10:42:55.128 INFO:tasks.workunit.client.1.smithi150.stdout:4/487: fsync d0/d1b/d4d/d6c/f67 0 2024-03-22T10:42:55.128 INFO:tasks.workunit.client.1.smithi150.stdout:0/483: sync 2024-03-22T10:42:55.128 INFO:tasks.workunit.client.1.smithi150.stdout:5/346: truncate d0/d8/dc/f12 4977320 0 2024-03-22T10:42:55.130 INFO:tasks.workunit.client.0.smithi115.stdout:2/803: readlink da/d25/d3d/d57/l9c 0 2024-03-22T10:42:55.130 INFO:tasks.workunit.client.0.smithi115.stdout:3/866: dwrite d3/d36/f5c [0,4194304] 0 2024-03-22T10:42:55.131 INFO:tasks.workunit.client.1.smithi150.stdout:3/448: readlink d5/da/d78/d86/l73 0 2024-03-22T10:42:55.133 INFO:tasks.workunit.client.1.smithi150.stdout:2/492: write d12/d21/f4f [469932,48945] 0 2024-03-22T10:42:55.140 INFO:tasks.workunit.client.1.smithi150.stdout:6/441: dwrite db/dd/d15/d28/f2b [0,4194304] 0 2024-03-22T10:42:55.143 INFO:tasks.workunit.client.1.smithi150.stdout:6/442: fdatasync db/dd/d82/d84/f96 0 2024-03-22T10:42:55.145 INFO:tasks.workunit.client.0.smithi115.stdout:1/876: dwrite d0/d8/de/d1a/d91/feb [0,4194304] 0 2024-03-22T10:42:55.146 INFO:tasks.workunit.client.0.smithi115.stdout:2/804: write da/d25/d30/f59 [1419227,113432] 0 2024-03-22T10:42:55.148 INFO:tasks.workunit.client.0.smithi115.stdout:3/867: creat d3/d16/d29/f125 x:0 0 0 2024-03-22T10:42:55.149 INFO:tasks.workunit.client.1.smithi150.stdout:3/449: symlink d5/da/d62/l97 0 2024-03-22T10:42:55.152 INFO:tasks.workunit.client.0.smithi115.stdout:9/854: dwrite d6/d7/d30/d37/dbe/fd4 [0,4194304] 0 2024-03-22T10:42:55.154 INFO:tasks.workunit.client.1.smithi150.stdout:2/493: mknod d12/d48/ca0 0 2024-03-22T10:42:55.157 INFO:tasks.workunit.client.0.smithi115.stdout:1/877: creat d0/d8/db/d25/d11a/f136 x:0 0 0 2024-03-22T10:42:55.157 INFO:tasks.workunit.client.1.smithi150.stdout:1/410: truncate d1/d19/f26 1022185 0 2024-03-22T10:42:55.159 INFO:tasks.workunit.client.0.smithi115.stdout:4/846: dwrite d1/d66/d106/d28/fe5 [4194304,4194304] 0 2024-03-22T10:42:55.159 INFO:tasks.workunit.client.0.smithi115.stdout:1/878: readlink d0/d8/de/l12b 0 2024-03-22T10:42:55.164 INFO:tasks.workunit.client.1.smithi150.stdout:8/479: write d0/d2/f12 [1049643,67760] 0 2024-03-22T10:42:55.164 INFO:tasks.workunit.client.0.smithi115.stdout:2/805: mkdir da/d25/d83/df3 0 2024-03-22T10:42:55.164 INFO:tasks.workunit.client.1.smithi150.stdout:5/347: creat d0/d8/dc/de/d14/d2e/f6d x:0 0 0 2024-03-22T10:42:55.164 INFO:tasks.workunit.client.1.smithi150.stdout:3/450: mkdir d5/d98 0 2024-03-22T10:42:55.166 INFO:tasks.workunit.client.0.smithi115.stdout:9/855: chown d6/d7/l8a 25103 1 2024-03-22T10:42:55.169 INFO:tasks.workunit.client.1.smithi150.stdout:3/451: truncate d5/da/f7a 636246 0 2024-03-22T10:42:55.170 INFO:tasks.workunit.client.1.smithi150.stdout:2/494: mkdir d12/da1 0 2024-03-22T10:42:55.170 INFO:tasks.workunit.client.0.smithi115.stdout:4/847: creat d1/dbb/dc6/dd8/f123 x:0 0 0 2024-03-22T10:42:55.170 INFO:tasks.workunit.client.0.smithi115.stdout:2/806: creat da/d1c/d67/ff4 x:0 0 0 2024-03-22T10:42:55.170 INFO:tasks.workunit.client.1.smithi150.stdout:2/495: chown d12/d19 196790 1 2024-03-22T10:42:55.172 INFO:tasks.workunit.client.1.smithi150.stdout:5/348: dread d0/d8/dc/d1d/f24 [0,4194304] 0 2024-03-22T10:42:55.172 INFO:tasks.workunit.client.0.smithi115.stdout:3/868: symlink d3/d19/d2b/d49/l126 0 2024-03-22T10:42:55.175 INFO:tasks.workunit.client.1.smithi150.stdout:2/496: write d12/d19/d1a/f30 [1605410,127305] 0 2024-03-22T10:42:55.175 INFO:tasks.workunit.client.0.smithi115.stdout:4/848: read d1/d69/f65 [2360274,7758] 0 2024-03-22T10:42:55.176 INFO:tasks.workunit.client.1.smithi150.stdout:3/452: creat d5/d53/d7e/f99 x:0 0 0 2024-03-22T10:42:55.178 INFO:tasks.workunit.client.0.smithi115.stdout:9/856: mknod d6/d8/c12d 0 2024-03-22T10:42:55.178 INFO:tasks.workunit.client.0.smithi115.stdout:3/869: fsync d3/d19/dda/ff8 0 2024-03-22T10:42:55.179 INFO:tasks.workunit.client.1.smithi150.stdout:2/497: chown d12/d19/d2d/l4a 3912968 1 2024-03-22T10:42:55.180 INFO:tasks.workunit.client.0.smithi115.stdout:3/870: chown d3/d6d/db4/de1/f10b 1 1 2024-03-22T10:42:55.182 INFO:tasks.workunit.client.1.smithi150.stdout:2/498: write d12/d16/d2c/f33 [48281,99959] 0 2024-03-22T10:42:55.183 INFO:tasks.workunit.client.1.smithi150.stdout:3/453: rename d5/da/d78/d86/f74 to d5/d22/d2f/d38/d3e/d5b/f9a 0 2024-03-22T10:42:55.184 INFO:tasks.workunit.client.0.smithi115.stdout:4/849: creat d1/d66/d106/d35/d40/d54/d5e/f124 x:0 0 0 2024-03-22T10:42:55.187 INFO:tasks.workunit.client.1.smithi150.stdout:2/499: symlink d12/d16/d61/la2 0 2024-03-22T10:42:55.188 INFO:tasks.workunit.client.0.smithi115.stdout:3/871: getdents d3/d19/d2b/d49/d91/dad 0 2024-03-22T10:42:55.189 INFO:tasks.workunit.client.0.smithi115.stdout:9/857: dread d6/d7/d48/d70/fc0 [0,4194304] 0 2024-03-22T10:42:55.190 INFO:tasks.workunit.client.0.smithi115.stdout:9/858: readlink d6/d7/d48/df4/l8b 0 2024-03-22T10:42:55.198 INFO:tasks.workunit.client.1.smithi150.stdout:6/443: write db/dd/d15/f35 [266243,101300] 0 2024-03-22T10:42:55.198 INFO:tasks.workunit.client.0.smithi115.stdout:2/807: rmdir da/d25/d3d/d57/d7d/dc6 39 2024-03-22T10:42:55.200 INFO:tasks.workunit.client.1.smithi150.stdout:0/484: dwrite d0/d1/d84/d86/fe [0,4194304] 0 2024-03-22T10:42:55.201 INFO:tasks.workunit.client.0.smithi115.stdout:0/797: truncate d3/d7/d6b/d94/fe4 2561511 0 2024-03-22T10:42:55.202 INFO:tasks.workunit.client.1.smithi150.stdout:3/454: rmdir d5/d53 39 2024-03-22T10:42:55.203 INFO:tasks.workunit.client.1.smithi150.stdout:5/349: write d0/f22 [2591797,23213] 0 2024-03-22T10:42:55.203 INFO:tasks.workunit.client.0.smithi115.stdout:2/808: dread - da/d25/d3d/d7e/d90/d96/fcb zero size 2024-03-22T10:42:55.204 INFO:tasks.workunit.client.0.smithi115.stdout:3/872: creat d3/f127 x:0 0 0 2024-03-22T10:42:55.204 INFO:tasks.workunit.client.1.smithi150.stdout:0/485: write d0/f89 [272903,65372] 0 2024-03-22T10:42:55.205 INFO:tasks.workunit.client.0.smithi115.stdout:0/798: write d3/d4/d6d/ff1 [5492,102019] 0 2024-03-22T10:42:55.213 INFO:tasks.workunit.client.1.smithi150.stdout:5/350: dread d0/d8/dc/f5b [0,4194304] 0 2024-03-22T10:42:55.218 INFO:tasks.workunit.client.1.smithi150.stdout:0/486: rmdir d0/d1/d84/d86/d42/d10/d38/d79 39 2024-03-22T10:42:55.218 INFO:tasks.workunit.client.0.smithi115.stdout:2/809: mknod da/d1c/d67/d78/d86/cf5 0 2024-03-22T10:42:55.223 INFO:tasks.workunit.client.1.smithi150.stdout:5/351: creat d0/d8/dc/de/d14/d4f/d47/f6e x:0 0 0 2024-03-22T10:42:55.223 INFO:tasks.workunit.client.1.smithi150.stdout:0/487: read d0/f89 [124440,17902] 0 2024-03-22T10:42:55.226 INFO:tasks.workunit.client.1.smithi150.stdout:8/480: dread d0/d3c/f44 [0,4194304] 0 2024-03-22T10:42:55.227 INFO:tasks.workunit.client.1.smithi150.stdout:5/352: fsync d0/d8/dc/de/f13 0 2024-03-22T10:42:55.227 INFO:tasks.workunit.client.1.smithi150.stdout:7/456: sync 2024-03-22T10:42:55.229 INFO:tasks.workunit.client.1.smithi150.stdout:7/457: stat d2/d5/d7/d88 0 2024-03-22T10:42:55.230 INFO:tasks.workunit.client.1.smithi150.stdout:5/353: mkdir d0/d33/d6f 0 2024-03-22T10:42:55.231 INFO:tasks.workunit.client.1.smithi150.stdout:3/455: dwrite d5/d53/f75 [0,4194304] 0 2024-03-22T10:42:55.232 INFO:tasks.workunit.client.1.smithi150.stdout:7/458: fsync d2/d5/d7/d8/d2c/d96/f8c 0 2024-03-22T10:42:55.232 INFO:tasks.workunit.client.0.smithi115.stdout:9/859: truncate d6/d7/d20/d22/d23/d106/f10c 1658682 0 2024-03-22T10:42:55.235 INFO:tasks.workunit.client.1.smithi150.stdout:8/481: mknod d0/d2/d19/d29/d2f/ca5 0 2024-03-22T10:42:55.236 INFO:tasks.workunit.client.0.smithi115.stdout:1/879: dwrite d0/d12/d17/d37/d9e/de9/f8d [0,4194304] 0 2024-03-22T10:42:55.236 INFO:tasks.workunit.client.0.smithi115.stdout:3/873: dwrite d3/d36/d41/d33/fe2 [0,4194304] 0 2024-03-22T10:42:55.236 INFO:tasks.workunit.client.1.smithi150.stdout:7/459: write d2/d5/d7/d9/f62 [575369,121510] 0 2024-03-22T10:42:55.240 INFO:tasks.workunit.client.1.smithi150.stdout:8/482: readlink d0/d2/d19/l33 0 2024-03-22T10:42:55.241 INFO:tasks.workunit.client.1.smithi150.stdout:5/354: creat d0/d8/dc/de/f70 x:0 0 0 2024-03-22T10:42:55.242 INFO:tasks.workunit.client.1.smithi150.stdout:6/444: dwrite db/dd/d15/d30/f51 [0,4194304] 0 2024-03-22T10:42:55.242 INFO:tasks.workunit.client.1.smithi150.stdout:3/456: link d5/da/d62/f8e d5/da/d78/d86/d81/f9b 0 2024-03-22T10:42:55.244 INFO:tasks.workunit.client.1.smithi150.stdout:7/460: unlink d2/f60 0 2024-03-22T10:42:55.245 INFO:tasks.workunit.client.1.smithi150.stdout:8/483: read - d0/d2/d19/d29/d37/da1/fa4 zero size 2024-03-22T10:42:55.248 INFO:tasks.workunit.client.1.smithi150.stdout:8/484: chown d0/d3c/f56 388586337 1 2024-03-22T10:42:55.251 INFO:tasks.workunit.client.0.smithi115.stdout:4/850: dwrite d1/d30/d4c/d86/d9f/d75/d99/da8/fca [0,4194304] 0 2024-03-22T10:42:55.252 INFO:tasks.workunit.client.1.smithi150.stdout:3/457: truncate d5/da/d62/f8e 178202 0 2024-03-22T10:42:55.253 INFO:tasks.workunit.client.1.smithi150.stdout:2/500: dread d12/d16/f43 [0,4194304] 0 2024-03-22T10:42:55.253 INFO:tasks.workunit.client.1.smithi150.stdout:5/355: rmdir d0/d8/dc/de/d14/d2e/d5a 39 2024-03-22T10:42:55.257 INFO:tasks.workunit.client.1.smithi150.stdout:3/458: truncate d5/d53/f54 4879070 0 2024-03-22T10:42:55.257 INFO:tasks.workunit.client.1.smithi150.stdout:7/461: symlink d2/d5/d7/d8/d2c/d96/l97 0 2024-03-22T10:42:55.259 INFO:tasks.workunit.client.0.smithi115.stdout:9/860: link d6/d7/f16 d6/d7/d30/f12e 0 2024-03-22T10:42:55.262 INFO:tasks.workunit.client.1.smithi150.stdout:8/485: mknod d0/d2/d19/d29/d37/d7d/ca6 0 2024-03-22T10:42:55.262 INFO:tasks.workunit.client.1.smithi150.stdout:7/462: dread - d2/d5/d7/d8/d46/d49/f68 zero size 2024-03-22T10:42:55.265 INFO:tasks.workunit.client.0.smithi115.stdout:3/874: rename d3/d82/d57/fd0 to d3/d36/d41/f128 0 2024-03-22T10:42:55.266 INFO:tasks.workunit.client.1.smithi150.stdout:5/356: symlink d0/d8/dc/l71 0 2024-03-22T10:42:55.267 INFO:tasks.workunit.client.1.smithi150.stdout:3/459: mkdir d5/d22/d2f/d9c 0 2024-03-22T10:42:55.267 INFO:tasks.workunit.client.0.smithi115.stdout:3/875: stat d3/d36/d41/d85/db8 0 2024-03-22T10:42:55.270 INFO:tasks.workunit.client.1.smithi150.stdout:8/486: read d0/d2/d19/d29/d2f/f64 [418709,49581] 0 2024-03-22T10:42:55.270 INFO:tasks.workunit.client.0.smithi115.stdout:1/880: rename d0/d68/d81/df3 to d0/d12/d17/d37/d9e/de9/dc1/d137 0 2024-03-22T10:42:55.272 INFO:tasks.workunit.client.1.smithi150.stdout:7/463: mknod d2/d5/d7/d8/d57/d4a/d8d/c98 0 2024-03-22T10:42:55.273 INFO:tasks.workunit.client.0.smithi115.stdout:2/810: dwrite da/d1c/d67/d78/d86/d9d/fa0 [0,4194304] 0 2024-03-22T10:42:55.273 INFO:tasks.workunit.client.0.smithi115.stdout:3/876: stat d3/d6d/db4/d107/daa/c11b 0 2024-03-22T10:42:55.276 INFO:tasks.workunit.client.1.smithi150.stdout:7/464: symlink d2/d5/d7/d8/d57/d54/l99 0 2024-03-22T10:42:55.279 INFO:tasks.workunit.client.0.smithi115.stdout:9/861: mknod d6/d7/d20/d22/d23/c12f 0 2024-03-22T10:42:55.281 INFO:tasks.workunit.client.1.smithi150.stdout:8/487: symlink d0/d2/d19/d29/d37/d7d/la7 0 2024-03-22T10:42:55.281 INFO:tasks.workunit.client.1.smithi150.stdout:2/501: dwrite d12/d19/f1d [0,4194304] 0 2024-03-22T10:42:55.282 INFO:tasks.workunit.client.0.smithi115.stdout:0/799: dwrite d3/da/d4c/d69/d6a/fa4 [0,4194304] 0 2024-03-22T10:42:55.284 INFO:tasks.workunit.client.1.smithi150.stdout:6/445: dread db/dd/f33 [0,4194304] 0 2024-03-22T10:42:55.285 INFO:tasks.workunit.client.1.smithi150.stdout:7/465: mkdir d2/d5/d9a 0 2024-03-22T10:42:55.294 INFO:tasks.workunit.client.1.smithi150.stdout:7/466: readlink d2/d5/l34 0 2024-03-22T10:42:55.300 INFO:tasks.workunit.client.1.smithi150.stdout:6/446: symlink db/dd/d15/d60/d50/l9d 0 2024-03-22T10:42:55.302 INFO:tasks.workunit.client.1.smithi150.stdout:6/447: fsync db/dd/f5b 0 2024-03-22T10:42:55.303 INFO:tasks.workunit.client.1.smithi150.stdout:7/467: dread - d2/d5/d7/d8/d2c/f8b zero size 2024-03-22T10:42:55.311 INFO:tasks.workunit.client.1.smithi150.stdout:8/488: write d0/d2/d19/d29/d2f/df/f4f [960342,9991] 0 2024-03-22T10:42:55.311 INFO:tasks.workunit.client.1.smithi150.stdout:2/502: dwrite d12/d16/d61/d98/f6f [0,4194304] 0 2024-03-22T10:42:55.312 INFO:tasks.workunit.client.1.smithi150.stdout:5/357: dwrite d0/d8/f23 [0,4194304] 0 2024-03-22T10:42:55.312 INFO:tasks.workunit.client.1.smithi150.stdout:7/468: unlink d2/d5/d7/d8/d2c/d45/f84 0 2024-03-22T10:42:55.313 INFO:tasks.workunit.client.0.smithi115.stdout:0/800: getdents d3/d7/d1b/d1f/db9 0 2024-03-22T10:42:55.315 INFO:tasks.workunit.client.0.smithi115.stdout:4/851: rmdir d1/d66/d106/d35/d40/d54/d5e/df3/d109 0 2024-03-22T10:42:55.316 INFO:tasks.workunit.client.0.smithi115.stdout:0/801: dread - d3/d4/fd5 zero size 2024-03-22T10:42:55.317 INFO:tasks.workunit.client.1.smithi150.stdout:7/469: chown d2/d5/d7/d9/db/c77 31 1 2024-03-22T10:42:55.321 INFO:tasks.workunit.client.1.smithi150.stdout:4/488: sync 2024-03-22T10:42:55.322 INFO:tasks.workunit.client.1.smithi150.stdout:9/494: sync 2024-03-22T10:42:55.322 INFO:tasks.workunit.client.0.smithi115.stdout:4/852: unlink d1/d66/d106/d35/d40/d5c/d55/c11b 0 2024-03-22T10:42:55.323 INFO:tasks.workunit.client.1.smithi150.stdout:5/358: creat d0/d8/dc/de/d14/d2e/d46/f72 x:0 0 0 2024-03-22T10:42:55.325 INFO:tasks.workunit.client.0.smithi115.stdout:1/881: dread d0/d12/d17/f31 [0,4194304] 0 2024-03-22T10:42:55.329 INFO:tasks.workunit.client.1.smithi150.stdout:6/448: dwrite db/d52/d79/f8e [0,4194304] 0 2024-03-22T10:42:55.329 INFO:tasks.workunit.client.0.smithi115.stdout:1/882: creat d0/d8/d4d/f138 x:0 0 0 2024-03-22T10:42:55.330 INFO:tasks.workunit.client.1.smithi150.stdout:7/470: write d2/d5/d7/d9/f92 [2707550,85515] 0 2024-03-22T10:42:55.331 INFO:tasks.workunit.client.1.smithi150.stdout:5/359: write d0/f3 [2804832,75937] 0 2024-03-22T10:42:55.333 INFO:tasks.workunit.client.1.smithi150.stdout:6/449: chown db/dd/d15/d60 11933 1 2024-03-22T10:42:55.334 INFO:tasks.workunit.client.1.smithi150.stdout:9/495: unlink d1/d19/c85 0 2024-03-22T10:42:55.334 INFO:tasks.workunit.client.1.smithi150.stdout:4/489: creat d0/d1b/d4d/d6c/d61/d71/fa4 x:0 0 0 2024-03-22T10:42:55.334 INFO:tasks.workunit.client.1.smithi150.stdout:8/489: link d0/d2/d19/d29/d2f/df/d1b/f1d d0/d2/fa8 0 2024-03-22T10:42:55.349 INFO:tasks.workunit.client.1.smithi150.stdout:4/490: getdents d0/d1b/d4d/d6c 0 2024-03-22T10:42:55.352 INFO:tasks.workunit.client.0.smithi115.stdout:0/802: write d3/da/d4c/f91 [2199600,111291] 0 2024-03-22T10:42:55.353 INFO:tasks.workunit.client.0.smithi115.stdout:1/883: truncate d0/d8/de/f4c 7832010 0 2024-03-22T10:42:55.353 INFO:tasks.workunit.client.0.smithi115.stdout:4/853: truncate d1/d66/d106/d35/d40/d5c/d14/f1d 1010933 0 2024-03-22T10:42:55.358 INFO:tasks.workunit.client.1.smithi150.stdout:2/503: truncate d12/d16/d67/f3d 2247029 0 2024-03-22T10:42:55.359 INFO:tasks.workunit.client.0.smithi115.stdout:1/884: read - d0/d12/d17/d37/d5c/f130 zero size 2024-03-22T10:42:55.359 INFO:tasks.workunit.client.1.smithi150.stdout:9/496: dread d1/d49/f31 [0,4194304] 0 2024-03-22T10:42:55.359 INFO:tasks.workunit.client.0.smithi115.stdout:0/803: stat d3/d7/d1b/c7a 0 2024-03-22T10:42:55.361 INFO:tasks.workunit.client.0.smithi115.stdout:4/854: read d1/dbb/dc6/dd8/df7/f10d [828254,47793] 0 2024-03-22T10:42:55.361 INFO:tasks.workunit.client.0.smithi115.stdout:3/877: dwrite d3/d36/d41/f9a [0,4194304] 0 2024-03-22T10:42:55.365 INFO:tasks.workunit.client.1.smithi150.stdout:2/504: write d12/f37 [2933170,668] 0 2024-03-22T10:42:55.366 INFO:tasks.workunit.client.0.smithi115.stdout:9/862: dwrite d6/d7/d20/d22/d23/f85 [0,4194304] 0 2024-03-22T10:42:55.369 INFO:tasks.workunit.client.0.smithi115.stdout:4/855: chown d1/d30/d4c/d86/dde/d103/f111 96 1 2024-03-22T10:42:55.373 INFO:tasks.workunit.client.1.smithi150.stdout:6/450: dwrite db/dd/d15/d28/f58 [4194304,4194304] 0 2024-03-22T10:42:55.375 INFO:tasks.workunit.client.0.smithi115.stdout:0/804: truncate d3/f33 3343975 0 2024-03-22T10:42:55.376 INFO:tasks.workunit.client.0.smithi115.stdout:9/863: dread - d6/d7/d48/d70/da9/dbd/ff3 zero size 2024-03-22T10:42:55.378 INFO:tasks.workunit.client.0.smithi115.stdout:4/856: mkdir d1/d66/d125 0 2024-03-22T10:42:55.378 INFO:tasks.workunit.client.1.smithi150.stdout:5/360: dwrite d0/d8/dc/de/d14/f16 [0,4194304] 0 2024-03-22T10:42:55.380 INFO:tasks.workunit.client.1.smithi150.stdout:7/471: dwrite d2/d5/d7/d9/db/f76 [0,4194304] 0 2024-03-22T10:42:55.381 INFO:tasks.workunit.client.0.smithi115.stdout:3/878: getdents d3/d82/d54 0 2024-03-22T10:42:55.382 INFO:tasks.workunit.client.0.smithi115.stdout:7/844: sync 2024-03-22T10:42:55.383 INFO:tasks.workunit.client.0.smithi115.stdout:6/774: sync 2024-03-22T10:42:55.383 INFO:tasks.workunit.client.1.smithi150.stdout:5/361: chown d0/d8/dc/de/d14/d4f/d47/c55 968748796 1 2024-03-22T10:42:55.383 INFO:tasks.workunit.client.0.smithi115.stdout:5/867: sync 2024-03-22T10:42:55.385 INFO:tasks.workunit.client.0.smithi115.stdout:9/864: creat d6/d7/d19/d4e/f130 x:0 0 0 2024-03-22T10:42:55.386 INFO:tasks.workunit.client.0.smithi115.stdout:6/775: chown d4/d36/f53 1084926 1 2024-03-22T10:42:55.388 INFO:tasks.workunit.client.1.smithi150.stdout:8/490: dread d0/d2/d19/d29/d2f/df/f8a [0,4194304] 0 2024-03-22T10:42:55.388 INFO:tasks.workunit.client.1.smithi150.stdout:2/505: dwrite d12/d19/d2d/f53 [0,4194304] 0 2024-03-22T10:42:55.388 INFO:tasks.workunit.client.0.smithi115.stdout:4/857: rename d1/d66/d106/d35/d40/d54/d5e/c5f to d1/d66/d82/c126 0 2024-03-22T10:42:55.389 INFO:tasks.workunit.client.1.smithi150.stdout:7/472: symlink d2/d5/d7/d8/d57/l9b 0 2024-03-22T10:42:55.390 INFO:tasks.workunit.client.1.smithi150.stdout:5/362: creat d0/d33/f73 x:0 0 0 2024-03-22T10:42:55.390 INFO:tasks.workunit.client.1.smithi150.stdout:4/491: dread d0/d1b/d4d/d6c/f62 [0,4194304] 0 2024-03-22T10:42:55.390 INFO:tasks.workunit.client.0.smithi115.stdout:6/776: chown d4/d5e/l90 13 1 2024-03-22T10:42:55.391 INFO:tasks.workunit.client.0.smithi115.stdout:7/845: fdatasync d2/d7/d12/d10/d23/d3d/dd2/da3/df3/f119 0 2024-03-22T10:42:55.391 INFO:tasks.workunit.client.0.smithi115.stdout:5/868: fsync d2/d15/d1b/d7c/d38/d43/fca 0 2024-03-22T10:42:55.391 INFO:tasks.workunit.client.1.smithi150.stdout:2/506: fsync d12/d19/d1a/f30 0 2024-03-22T10:42:55.392 INFO:tasks.workunit.client.0.smithi115.stdout:3/879: symlink d3/d16/d29/dc1/l129 0 2024-03-22T10:42:55.399 INFO:tasks.workunit.client.0.smithi115.stdout:6/777: symlink d4/d11/d12/d1c/lfc 0 2024-03-22T10:42:55.401 INFO:tasks.workunit.client.1.smithi150.stdout:5/363: rmdir d0/d8/dc/d1d/d53 39 2024-03-22T10:42:55.401 INFO:tasks.workunit.client.0.smithi115.stdout:7/846: mknod d2/d7/d12/d10/d23/d33/c121 0 2024-03-22T10:42:55.405 INFO:tasks.workunit.client.0.smithi115.stdout:3/880: mknod d3/d82/d52/c12a 0 2024-03-22T10:42:55.405 INFO:tasks.workunit.client.0.smithi115.stdout:5/869: write d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f9b [1856495,17275] 0 2024-03-22T10:42:55.406 INFO:tasks.workunit.client.1.smithi150.stdout:8/491: truncate d0/d2/d19/d29/d31/f4b 1034136 0 2024-03-22T10:42:55.408 INFO:tasks.workunit.client.1.smithi150.stdout:7/473: mkdir d2/d5/d7/d9/db/d6c/d9c 0 2024-03-22T10:42:55.412 INFO:tasks.workunit.client.1.smithi150.stdout:8/492: creat d0/d2/d7e/fa9 x:0 0 0 2024-03-22T10:42:55.414 INFO:tasks.workunit.client.1.smithi150.stdout:4/492: getdents d0/d1b/d4d 0 2024-03-22T10:42:55.419 INFO:tasks.workunit.client.0.smithi115.stdout:5/870: rename d2/df/d3b/l21 to d2/df/d13/d8b/d93/da6/dbc/d50/l11b 0 2024-03-22T10:42:55.420 INFO:tasks.workunit.client.1.smithi150.stdout:7/474: creat d2/d5/d7/d9/db/d6c/d9c/f9d x:0 0 0 2024-03-22T10:42:55.421 INFO:tasks.workunit.client.1.smithi150.stdout:8/493: symlink d0/d3c/d71/d7a/d89/laa 0 2024-03-22T10:42:55.421 INFO:tasks.workunit.client.0.smithi115.stdout:3/881: link d3/d36/d41/c120 d3/d19/dda/c12b 0 2024-03-22T10:42:55.424 INFO:tasks.workunit.client.1.smithi150.stdout:8/494: symlink d0/d3c/d71/d7a/lab 0 2024-03-22T10:42:55.427 INFO:tasks.workunit.client.0.smithi115.stdout:3/882: write d3/d19/dda/f10c [218903,6128] 0 2024-03-22T10:42:55.429 INFO:tasks.workunit.client.1.smithi150.stdout:8/495: symlink d0/d2/d46/da2/lac 0 2024-03-22T10:42:55.432 INFO:tasks.workunit.client.0.smithi115.stdout:6/778: dwrite d4/d36/f53 [0,4194304] 0 2024-03-22T10:42:55.434 INFO:tasks.workunit.client.0.smithi115.stdout:3/883: creat d3/d89/f12c x:0 0 0 2024-03-22T10:42:55.435 INFO:tasks.workunit.client.0.smithi115.stdout:9/865: dread d6/d7/f17 [0,4194304] 0 2024-03-22T10:42:55.438 INFO:tasks.workunit.client.0.smithi115.stdout:0/805: write d3/d7/d1c/f5d [1421958,99317] 0 2024-03-22T10:42:55.442 INFO:tasks.workunit.client.1.smithi150.stdout:7/475: dwrite d2/d5/d7/d8/d57/d4a/f39 [4194304,4194304] 0 2024-03-22T10:42:55.444 INFO:tasks.workunit.client.1.smithi150.stdout:5/364: truncate d0/f3 1383136 0 2024-03-22T10:42:55.445 INFO:tasks.workunit.client.0.smithi115.stdout:0/806: write d3/d7/d1c/f5d [2047509,41994] 0 2024-03-22T10:42:55.451 INFO:tasks.workunit.client.1.smithi150.stdout:5/365: write d0/d8/dc/de/d14/d4f/d47/f6e [530105,13109] 0 2024-03-22T10:42:55.452 INFO:tasks.workunit.client.0.smithi115.stdout:1/885: dwrite d0/d22/fc6 [0,4194304] 0 2024-03-22T10:42:55.453 INFO:tasks.workunit.client.0.smithi115.stdout:1/886: readlink d0/d8/l69 0 2024-03-22T10:42:55.456 INFO:tasks.workunit.client.1.smithi150.stdout:7/476: fsync d2/d5/d7/d8/d57/d54/f5f 0 2024-03-22T10:42:55.457 INFO:tasks.workunit.client.0.smithi115.stdout:0/807: mknod d3/c106 0 2024-03-22T10:42:55.459 INFO:tasks.workunit.client.0.smithi115.stdout:0/808: chown d3/d7/d1b/c81 18726790 1 2024-03-22T10:42:55.460 INFO:tasks.workunit.client.0.smithi115.stdout:5/871: write d2/df/f10 [8947294,14646] 0 2024-03-22T10:42:55.460 INFO:tasks.workunit.client.1.smithi150.stdout:5/366: mknod d0/d8/dc/de/d14/d2e/d46/d4d/c74 0 2024-03-22T10:42:55.462 INFO:tasks.workunit.client.1.smithi150.stdout:7/477: fsync d2/d5/d7/d8/d2c/d45/f7d 0 2024-03-22T10:42:55.465 INFO:tasks.workunit.client.0.smithi115.stdout:5/872: creat d2/d15/d1b/d7c/daf/f11c x:0 0 0 2024-03-22T10:42:55.468 INFO:tasks.workunit.client.1.smithi150.stdout:5/367: rename d0/d8/dc/de/d14/d2e/d45 to d0/d8/dc/de/d14/d2e/d45/d75 22 2024-03-22T10:42:55.468 INFO:tasks.workunit.client.1.smithi150.stdout:4/493: dread d0/d32/d7c/f91 [0,4194304] 0 2024-03-22T10:42:55.469 INFO:tasks.workunit.client.0.smithi115.stdout:5/873: rmdir d2/d15/d1b/d7c/dee/d10c 0 2024-03-22T10:42:55.474 INFO:tasks.workunit.client.1.smithi150.stdout:7/478: mknod d2/d5/d7/d8/d2c/d45/c9e 0 2024-03-22T10:42:55.474 INFO:tasks.workunit.client.1.smithi150.stdout:5/368: dread - d0/d8/dc/de/d14/d2e/f62 zero size 2024-03-22T10:42:55.478 INFO:tasks.workunit.client.0.smithi115.stdout:9/866: write d6/d7/d48/d70/da9/dbd/ff3 [259680,118089] 0 2024-03-22T10:42:55.478 INFO:tasks.workunit.client.1.smithi150.stdout:7/479: read - d2/d5/d7/d88/f8f zero size 2024-03-22T10:42:55.481 INFO:tasks.workunit.client.1.smithi150.stdout:4/494: truncate d0/d32/f56 4868663 0 2024-03-22T10:42:55.482 INFO:tasks.workunit.client.1.smithi150.stdout:5/369: chown d0/d8/dc/de/d14/d4f/d47/c55 1081425 1 2024-03-22T10:42:55.483 INFO:tasks.workunit.client.1.smithi150.stdout:7/480: chown d2/d5/d7/d9/db/d6c/d9c 353 1 2024-03-22T10:42:55.484 INFO:tasks.workunit.client.1.smithi150.stdout:2/507: dread d12/d16/d2c/f33 [0,4194304] 0 2024-03-22T10:42:55.486 INFO:tasks.workunit.client.1.smithi150.stdout:8/496: dread d0/f34 [0,4194304] 0 2024-03-22T10:42:55.486 INFO:tasks.workunit.client.0.smithi115.stdout:3/884: dwrite d3/d6d/db4/d107/fac [0,4194304] 0 2024-03-22T10:42:55.486 INFO:tasks.workunit.client.0.smithi115.stdout:1/887: write d0/d8/db/d25/d9f/fd3 [291575,35805] 0 2024-03-22T10:42:55.488 INFO:tasks.workunit.client.1.smithi150.stdout:7/481: fdatasync d2/d5/d7/d9/db/d6c/d9c/f9d 0 2024-03-22T10:42:55.488 INFO:tasks.workunit.client.1.smithi150.stdout:4/495: mknod d0/d9b/ca5 0 2024-03-22T10:42:55.488 INFO:tasks.workunit.client.0.smithi115.stdout:1/888: chown d0/d22/d89 4548009 1 2024-03-22T10:42:55.489 INFO:tasks.workunit.client.0.smithi115.stdout:0/809: truncate d3/d7/d1c/f56 3820646 0 2024-03-22T10:42:55.489 INFO:tasks.workunit.client.0.smithi115.stdout:3/885: readlink d3/d19/dd8/l121 0 2024-03-22T10:42:55.489 INFO:tasks.workunit.client.1.smithi150.stdout:5/370: write d0/d8/dc/de/d14/d2e/f6d [923564,109591] 0 2024-03-22T10:42:55.491 INFO:tasks.workunit.client.0.smithi115.stdout:5/874: rmdir d2 39 2024-03-22T10:42:55.492 INFO:tasks.workunit.client.0.smithi115.stdout:3/886: rmdir d3/d82/d54/de7 39 2024-03-22T10:42:55.493 INFO:tasks.workunit.client.0.smithi115.stdout:6/779: dread d4/d13/d33/f42 [0,4194304] 0 2024-03-22T10:42:55.495 INFO:tasks.workunit.client.0.smithi115.stdout:3/887: chown d3/d89/dca/f88 3 1 2024-03-22T10:42:55.495 INFO:tasks.workunit.client.1.smithi150.stdout:4/496: unlink d0/d1b/d4d/d6c/d61/c9a 0 2024-03-22T10:42:55.497 INFO:tasks.workunit.client.0.smithi115.stdout:6/780: truncate d4/d36/d48/f62 2488833 0 2024-03-22T10:42:55.498 INFO:tasks.workunit.client.0.smithi115.stdout:5/875: truncate d2/df/fd1 647581 0 2024-03-22T10:42:55.499 INFO:tasks.workunit.client.1.smithi150.stdout:5/371: chown d0/d8/dc/d1d/d53/d31/l6a 496487362 1 2024-03-22T10:42:55.501 INFO:tasks.workunit.client.0.smithi115.stdout:6/781: chown d4/d36/d48/d72 1491623 1 2024-03-22T10:42:55.503 INFO:tasks.workunit.client.1.smithi150.stdout:4/497: dread d0/d47/f7e [0,4194304] 0 2024-03-22T10:42:55.503 INFO:tasks.workunit.client.1.smithi150.stdout:5/372: read d0/f22 [67583,17562] 0 2024-03-22T10:42:55.505 INFO:tasks.workunit.client.0.smithi115.stdout:0/810: creat d3/da/d4c/d74/d84/dac/dfc/f107 x:0 0 0 2024-03-22T10:42:55.505 INFO:tasks.workunit.client.0.smithi115.stdout:6/782: symlink d4/d5e/lfd 0 2024-03-22T10:42:55.508 INFO:tasks.workunit.client.0.smithi115.stdout:2/811: sync 2024-03-22T10:42:55.510 INFO:tasks.workunit.client.1.smithi150.stdout:4/498: dread - d0/d2f/f50 zero size 2024-03-22T10:42:55.514 INFO:tasks.workunit.client.1.smithi150.stdout:5/373: getdents d0/d8/dc/d1d/d53/d63 0 2024-03-22T10:42:55.515 INFO:tasks.workunit.client.1.smithi150.stdout:8/497: write d0/d2/f83 [444756,86243] 0 2024-03-22T10:42:55.515 INFO:tasks.workunit.client.0.smithi115.stdout:3/888: rename d3/d36/d41/d85/db8/l113 to d3/d16/d29/l12d 0 2024-03-22T10:42:55.516 INFO:tasks.workunit.client.1.smithi150.stdout:2/508: write d12/d19/d1a/f2b [1257017,112447] 0 2024-03-22T10:42:55.516 INFO:tasks.workunit.client.1.smithi150.stdout:5/374: write d0/f10 [4013338,70675] 0 2024-03-22T10:42:55.517 INFO:tasks.workunit.client.0.smithi115.stdout:5/876: symlink d2/df/d13/d8b/d93/da6/dbc/d50/df5/l11d 0 2024-03-22T10:42:55.518 INFO:tasks.workunit.client.0.smithi115.stdout:6/783: creat d4/d13/d33/d58/ffe x:0 0 0 2024-03-22T10:42:55.520 INFO:tasks.workunit.client.0.smithi115.stdout:3/889: fsync d3/d36/f3e 0 2024-03-22T10:42:55.522 INFO:tasks.workunit.client.1.smithi150.stdout:2/509: truncate d12/d48/f79 5107393 0 2024-03-22T10:42:55.524 INFO:tasks.workunit.client.1.smithi150.stdout:4/499: write d0/de/f3d [3690997,8503] 0 2024-03-22T10:42:55.525 INFO:tasks.workunit.client.0.smithi115.stdout:6/784: chown d4/d11/d91/ceb 7180246 1 2024-03-22T10:42:55.526 INFO:tasks.workunit.client.1.smithi150.stdout:5/375: read d0/d8/dc/f12 [1557578,78687] 0 2024-03-22T10:42:55.526 INFO:tasks.workunit.client.0.smithi115.stdout:1/889: dwrite d0/d12/d17/d37/d9e/de9/dc1/fc0 [0,4194304] 0 2024-03-22T10:42:55.527 INFO:tasks.workunit.client.1.smithi150.stdout:5/376: fdatasync d0/f38 0 2024-03-22T10:42:55.528 INFO:tasks.workunit.client.1.smithi150.stdout:1/411: sync 2024-03-22T10:42:55.528 INFO:tasks.workunit.client.1.smithi150.stdout:4/500: dread - d0/d1b/d4d/d6c/f9d zero size 2024-03-22T10:42:55.529 INFO:tasks.workunit.client.0.smithi115.stdout:0/811: rmdir d3/d7/d1b/d1f/d78/dd9/df9 0 2024-03-22T10:42:55.532 INFO:tasks.workunit.client.0.smithi115.stdout:0/812: chown d3/da/d4c/d74/d84/dd7/fbf 1623 1 2024-03-22T10:42:55.534 INFO:tasks.workunit.client.1.smithi150.stdout:1/412: creat d1/da/d2a/f80 x:0 0 0 2024-03-22T10:42:55.535 INFO:tasks.workunit.client.0.smithi115.stdout:2/812: dread da/d1c/d67/d78/d86/db4/fc3 [0,4194304] 0 2024-03-22T10:42:55.538 INFO:tasks.workunit.client.1.smithi150.stdout:1/413: write d1/f70 [595302,77038] 0 2024-03-22T10:42:55.539 INFO:tasks.workunit.client.0.smithi115.stdout:1/890: rename d0/d8/db to d0/d12/d17/d37/d119/d139 0 2024-03-22T10:42:55.539 INFO:tasks.workunit.client.1.smithi150.stdout:1/414: readlink d1/l3f 0 2024-03-22T10:42:55.542 INFO:tasks.workunit.client.0.smithi115.stdout:6/785: mknod d4/d13/d33/cff 0 2024-03-22T10:42:55.542 INFO:tasks.workunit.client.1.smithi150.stdout:1/415: readlink d1/da/d15/l30 0 2024-03-22T10:42:55.543 INFO:tasks.workunit.client.1.smithi150.stdout:5/377: dread d0/d8/dc/de/d14/d2e/f6d [0,4194304] 0 2024-03-22T10:42:55.544 INFO:tasks.workunit.client.0.smithi115.stdout:5/877: dread d2/d15/d1b/d7c/d38/d43/f82 [0,4194304] 0 2024-03-22T10:42:55.544 INFO:tasks.workunit.client.1.smithi150.stdout:7/482: dread d2/d5/d7/d8/d2c/d96/f73 [0,4194304] 0 2024-03-22T10:42:55.547 INFO:tasks.workunit.client.0.smithi115.stdout:3/890: dwrite d3/d16/d29/f125 [0,4194304] 0 2024-03-22T10:42:55.549 INFO:tasks.workunit.client.1.smithi150.stdout:1/416: creat d1/da/d2a/d65/d78/f81 x:0 0 0 2024-03-22T10:42:55.550 INFO:tasks.workunit.client.0.smithi115.stdout:6/786: read d4/d5e/f8b [2569343,94346] 0 2024-03-22T10:42:55.550 INFO:tasks.workunit.client.0.smithi115.stdout:2/813: getdents da/d1c/d67/d78/d86 0 2024-03-22T10:42:55.551 INFO:tasks.workunit.client.0.smithi115.stdout:1/891: dread d0/d12/f66 [0,4194304] 0 2024-03-22T10:42:55.552 INFO:tasks.workunit.client.0.smithi115.stdout:3/891: creat d3/d82/d57/f12e x:0 0 0 2024-03-22T10:42:55.554 INFO:tasks.workunit.client.0.smithi115.stdout:5/878: creat d2/d15/d1b/d7c/d38/d43/d49/d81/dbd/f11e x:0 0 0 2024-03-22T10:42:55.555 INFO:tasks.workunit.client.1.smithi150.stdout:4/501: truncate d0/d32/f4e 2814395 0 2024-03-22T10:42:55.555 INFO:tasks.workunit.client.0.smithi115.stdout:3/892: dread - d3/d36/d41/d85/fcc zero size 2024-03-22T10:42:55.556 INFO:tasks.workunit.client.0.smithi115.stdout:1/892: dread - d0/d22/fda zero size 2024-03-22T10:42:55.557 INFO:tasks.workunit.client.1.smithi150.stdout:0/488: sync 2024-03-22T10:42:55.557 INFO:tasks.workunit.client.1.smithi150.stdout:3/460: sync 2024-03-22T10:42:55.558 INFO:tasks.workunit.client.0.smithi115.stdout:5/879: chown d2/df/d13/d8b/d93/da6/dbc/d50/db1 9 1 2024-03-22T10:42:55.558 INFO:tasks.workunit.client.0.smithi115.stdout:0/813: dread d3/d7/d1b/d1f/f62 [0,4194304] 0 2024-03-22T10:42:55.560 INFO:tasks.workunit.client.0.smithi115.stdout:2/814: creat da/d25/d2a/ff6 x:0 0 0 2024-03-22T10:42:55.563 INFO:tasks.workunit.client.1.smithi150.stdout:7/483: rename d2/d5/d7/d8/d2c to d2/d5/d7/d9/db/d9f 0 2024-03-22T10:42:55.567 INFO:tasks.workunit.client.1.smithi150.stdout:1/417: dread d1/da/d2a/f7f [0,4194304] 0 2024-03-22T10:42:55.567 INFO:tasks.workunit.client.1.smithi150.stdout:4/502: unlink d0/d1b/d4d/d6c/c86 0 2024-03-22T10:42:55.567 INFO:tasks.workunit.client.0.smithi115.stdout:5/880: write d2/df/d13/f18 [11925694,16870] 0 2024-03-22T10:42:55.567 INFO:tasks.workunit.client.1.smithi150.stdout:8/498: dread d0/d2/f5f [0,4194304] 0 2024-03-22T10:42:55.567 INFO:tasks.workunit.client.1.smithi150.stdout:3/461: mkdir d5/da/d9d 0 2024-03-22T10:42:55.570 INFO:tasks.workunit.client.0.smithi115.stdout:1/893: write d0/d8/de/fa1 [3987822,70678] 0 2024-03-22T10:42:55.570 INFO:tasks.workunit.client.0.smithi115.stdout:2/815: mkdir da/d25/d3d/d57/da7/df7 0 2024-03-22T10:42:55.573 INFO:tasks.workunit.client.1.smithi150.stdout:2/510: dread d12/d16/f2a [0,4194304] 0 2024-03-22T10:42:55.574 INFO:tasks.workunit.client.0.smithi115.stdout:3/893: mknod d3/d16/c12f 0 2024-03-22T10:42:55.574 INFO:tasks.workunit.client.0.smithi115.stdout:5/881: symlink d2/d15/d1b/d7c/dee/l11f 0 2024-03-22T10:42:55.574 INFO:tasks.workunit.client.1.smithi150.stdout:7/484: unlink d2/d5/d7/d9/db/d9f/f47 0 2024-03-22T10:42:55.577 INFO:tasks.workunit.client.0.smithi115.stdout:5/882: write d2/df/d3b/dfb/fff [398267,36128] 0 2024-03-22T10:42:55.578 INFO:tasks.workunit.client.0.smithi115.stdout:3/894: write d3/d6d/db4/de1/f10b [754851,8596] 0 2024-03-22T10:42:55.578 INFO:tasks.workunit.client.1.smithi150.stdout:0/489: creat d0/d1/d84/d86/d42/f95 x:0 0 0 2024-03-22T10:42:55.579 INFO:tasks.workunit.client.1.smithi150.stdout:4/503: read - d0/d1b/d4d/d6c/d61/f9e zero size 2024-03-22T10:42:55.580 INFO:tasks.workunit.client.0.smithi115.stdout:5/883: readlink d2/d15/d1b/d7c/d38/l95 0 2024-03-22T10:42:55.581 INFO:tasks.workunit.client.1.smithi150.stdout:5/378: dwrite d0/d8/dc/de/d14/d2e/d46/d4d/f6c [0,4194304] 0 2024-03-22T10:42:55.587 INFO:tasks.workunit.client.1.smithi150.stdout:7/485: rename d2/d5/d7/d9/db/d9f/d45 to d2/d5/d7/d9/db/d9f/d45/da0 22 2024-03-22T10:42:55.588 INFO:tasks.workunit.client.1.smithi150.stdout:8/499: creat d0/d2/d19/d29/d2e/d70/fad x:0 0 0 2024-03-22T10:42:55.588 INFO:tasks.workunit.client.1.smithi150.stdout:3/462: symlink d5/d22/d2f/d9c/l9e 0 2024-03-22T10:42:55.591 INFO:tasks.workunit.client.1.smithi150.stdout:2/511: symlink d12/d16/d2c/d65/la3 0 2024-03-22T10:42:55.594 INFO:tasks.workunit.client.1.smithi150.stdout:5/379: write d0/d8/dc/de/d14/d2e/d46/d4d/f51 [580751,117130] 0 2024-03-22T10:42:55.598 INFO:tasks.workunit.client.1.smithi150.stdout:8/500: creat d0/d2/d19/d29/d31/fae x:0 0 0 2024-03-22T10:42:55.602 INFO:tasks.workunit.client.1.smithi150.stdout:4/504: creat d0/d2f/d63/d7b/d9c/fa6 x:0 0 0 2024-03-22T10:42:55.607 INFO:tasks.workunit.client.1.smithi150.stdout:8/501: mknod d0/d2/d19/d29/d2e/caf 0 2024-03-22T10:42:55.611 INFO:tasks.workunit.client.1.smithi150.stdout:3/463: dread d5/d53/f54 [0,4194304] 0 2024-03-22T10:42:55.611 INFO:tasks.workunit.client.1.smithi150.stdout:7/486: dread d2/d5/d7/d9/f28 [0,4194304] 0 2024-03-22T10:42:55.611 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:55 smithi115 ceph-mon[29111]: pgmap v16: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 54 MiB/s rd, 202 MiB/s wr, 356 op/s 2024-03-22T10:42:55.613 INFO:tasks.workunit.client.1.smithi150.stdout:4/505: mknod d0/d47/ca7 0 2024-03-22T10:42:55.614 INFO:tasks.workunit.client.1.smithi150.stdout:3/464: write d5/d53/f71 [1551971,26393] 0 2024-03-22T10:42:55.615 INFO:tasks.workunit.client.1.smithi150.stdout:1/418: truncate d1/da/d15/d18/f4a 4144840 0 2024-03-22T10:42:55.620 INFO:tasks.workunit.client.1.smithi150.stdout:2/512: creat d12/d19/fa4 x:0 0 0 2024-03-22T10:42:55.620 INFO:tasks.workunit.client.1.smithi150.stdout:1/419: chown d1/d19/d50/f5c 904219 1 2024-03-22T10:42:55.621 INFO:tasks.workunit.client.1.smithi150.stdout:9/497: sync 2024-03-22T10:42:55.621 INFO:tasks.workunit.client.1.smithi150.stdout:6/451: sync 2024-03-22T10:42:55.629 INFO:tasks.workunit.client.1.smithi150.stdout:4/506: rmdir d0 39 2024-03-22T10:42:55.632 INFO:tasks.workunit.client.1.smithi150.stdout:8/502: dwrite d0/d2/f86 [0,4194304] 0 2024-03-22T10:42:55.634 INFO:tasks.workunit.client.1.smithi150.stdout:0/490: dwrite d0/d1/d84/d86/d11/d66/d37/f3d [0,4194304] 0 2024-03-22T10:42:55.637 INFO:tasks.workunit.client.0.smithi115.stdout:1/894: getdents d0/d12 0 2024-03-22T10:42:55.638 INFO:tasks.workunit.client.1.smithi150.stdout:2/513: mknod d12/d48/d63/ca5 0 2024-03-22T10:42:55.638 INFO:tasks.workunit.client.0.smithi115.stdout:5/884: dread d2/df/d13/d8b/d93/da6/dbc/ff1 [0,4194304] 0 2024-03-22T10:42:55.639 INFO:tasks.workunit.client.0.smithi115.stdout:3/895: write d3/d19/dd8/f10a [263536,96175] 0 2024-03-22T10:42:55.639 INFO:tasks.workunit.client.1.smithi150.stdout:7/487: link d2/d5/d7/f2b d2/d5/d7/d9/db/d6c/fa1 0 2024-03-22T10:42:55.641 INFO:tasks.workunit.client.0.smithi115.stdout:6/787: dread d4/f30 [0,4194304] 0 2024-03-22T10:42:55.642 INFO:tasks.workunit.client.1.smithi150.stdout:0/491: stat d0/d1/d84/d86/d42/d10/d38/c3e 0 2024-03-22T10:42:55.642 INFO:tasks.workunit.client.1.smithi150.stdout:5/380: getdents d0/d8/dc/de/d14/d2e 0 2024-03-22T10:42:55.644 INFO:tasks.workunit.client.0.smithi115.stdout:1/895: rmdir d0/d12/d17/d67/de6 39 2024-03-22T10:42:55.645 INFO:tasks.workunit.client.1.smithi150.stdout:9/498: chown d1/d8/de/df/f18 1 1 2024-03-22T10:42:55.647 INFO:tasks.workunit.client.1.smithi150.stdout:7/488: chown d2/d5/d7/d8/d57/l9b 130960375 1 2024-03-22T10:42:55.647 INFO:tasks.workunit.client.1.smithi150.stdout:6/452: dread db/dd/d15/d60/f32 [0,4194304] 0 2024-03-22T10:42:55.648 INFO:tasks.workunit.client.1.smithi150.stdout:5/381: write d0/d8/dc/de/d14/d2e/f3e [2648592,85921] 0 2024-03-22T10:42:55.648 INFO:tasks.workunit.client.0.smithi115.stdout:3/896: unlink d3/d16/d29/d35/l116 0 2024-03-22T10:42:55.650 INFO:tasks.workunit.client.0.smithi115.stdout:6/788: creat d4/d11/d12/d1c/d8a/f100 x:0 0 0 2024-03-22T10:42:55.651 INFO:tasks.workunit.client.0.smithi115.stdout:1/896: fsync d0/d8/d4d/d56/db6/ddc/fa5 0 2024-03-22T10:42:55.653 INFO:tasks.workunit.client.0.smithi115.stdout:3/897: readlink d3/d82/la8 0 2024-03-22T10:42:55.654 INFO:tasks.workunit.client.1.smithi150.stdout:0/492: chown d0/d1/d84/d86/d42/f77 0 1 2024-03-22T10:42:55.654 INFO:tasks.workunit.client.1.smithi150.stdout:2/514: fsync d12/d16/f2a 0 2024-03-22T10:42:55.654 INFO:tasks.workunit.client.1.smithi150.stdout:6/453: fdatasync db/dd/d2e/f81 0 2024-03-22T10:42:55.655 INFO:tasks.workunit.client.0.smithi115.stdout:0/814: dwrite d3/d7/d1b/fa8 [0,4194304] 0 2024-03-22T10:42:55.655 INFO:tasks.workunit.client.1.smithi150.stdout:8/503: creat d0/d2/d19/d29/d37/d7d/d9e/fb0 x:0 0 0 2024-03-22T10:42:55.657 INFO:tasks.workunit.client.0.smithi115.stdout:6/789: write d4/d11/d12/d1c/d8a/d98/d9f/dc3/ff1 [291643,93205] 0 2024-03-22T10:42:55.657 INFO:tasks.workunit.client.0.smithi115.stdout:1/897: chown d0/d12/d17/d37/d9e/da0/lc3 568 1 2024-03-22T10:42:55.657 INFO:tasks.workunit.client.0.smithi115.stdout:3/898: read d3/d82/d52/fe6 [187169,26080] 0 2024-03-22T10:42:55.658 INFO:tasks.workunit.client.1.smithi150.stdout:4/507: rename d0/d32/f51 to d0/d1b/d4d/d6c/d61/fa8 0 2024-03-22T10:42:55.658 INFO:tasks.workunit.client.1.smithi150.stdout:9/499: rmdir d1/d8/de/df/d36 39 2024-03-22T10:42:55.659 INFO:tasks.workunit.client.1.smithi150.stdout:1/420: getdents d1/d19/d1a 0 2024-03-22T10:42:55.660 INFO:tasks.workunit.client.0.smithi115.stdout:0/815: creat d3/da/d4c/d69/f108 x:0 0 0 2024-03-22T10:42:55.661 INFO:tasks.workunit.client.1.smithi150.stdout:7/489: mkdir d2/d5/d7/d9/db/da2 0 2024-03-22T10:42:55.661 INFO:tasks.workunit.client.0.smithi115.stdout:1/898: mkdir d0/d8/d4d/d56/db6/ddc/d13a 0 2024-03-22T10:42:55.663 INFO:tasks.workunit.client.0.smithi115.stdout:0/816: symlink d3/l109 0 2024-03-22T10:42:55.663 INFO:tasks.workunit.client.1.smithi150.stdout:8/504: mknod d0/d5d/cb1 0 2024-03-22T10:42:55.665 INFO:tasks.workunit.client.1.smithi150.stdout:5/382: truncate d0/d8/dc/de/d14/d2e/f54 105332 0 2024-03-22T10:42:55.665 INFO:tasks.workunit.client.1.smithi150.stdout:6/454: symlink db/l9e 0 2024-03-22T10:42:55.666 INFO:tasks.workunit.client.0.smithi115.stdout:0/817: symlink d3/d7/d1b/d1f/d78/dc1/l10a 0 2024-03-22T10:42:55.669 INFO:tasks.workunit.client.0.smithi115.stdout:5/885: write d2/d15/d1b/d7c/d38/d56/fcd [4300438,34706] 0 2024-03-22T10:42:55.670 INFO:tasks.workunit.client.1.smithi150.stdout:2/515: rename d12/d19/d2d/d99 to d12/d16/d67/d8d/da6 0 2024-03-22T10:42:55.670 INFO:tasks.workunit.client.1.smithi150.stdout:9/500: creat d1/d8/de/df/d20/fb8 x:0 0 0 2024-03-22T10:42:55.671 INFO:tasks.workunit.client.0.smithi115.stdout:0/818: fsync d3/da/d4c/da3/fc5 0 2024-03-22T10:42:55.671 INFO:tasks.workunit.client.1.smithi150.stdout:8/505: write d0/d2/d19/d29/d37/da1/fa4 [721000,100296] 0 2024-03-22T10:42:55.675 INFO:tasks.workunit.client.1.smithi150.stdout:4/508: mknod d0/de/ca9 0 2024-03-22T10:42:55.675 INFO:tasks.workunit.client.1.smithi150.stdout:2/516: readlink d12/d16/d2c/d65/la3 0 2024-03-22T10:42:55.676 INFO:tasks.workunit.client.0.smithi115.stdout:5/886: read d2/d15/d1b/d7c/daf/ffc [435010,27119] 0 2024-03-22T10:42:55.678 INFO:tasks.workunit.client.1.smithi150.stdout:3/465: truncate d5/da/ff 3687484 0 2024-03-22T10:42:55.678 INFO:tasks.workunit.client.1.smithi150.stdout:9/501: write d1/d8/de/df/d20/d29/f9b [505725,103670] 0 2024-03-22T10:42:55.679 INFO:tasks.workunit.client.1.smithi150.stdout:6/455: mkdir db/d52/d9f 0 2024-03-22T10:42:55.682 INFO:tasks.workunit.client.0.smithi115.stdout:0/819: fsync d3/d4/f71 0 2024-03-22T10:42:55.682 INFO:tasks.workunit.client.1.smithi150.stdout:5/383: unlink d0/d8/dc/de/d14/d4f/d47/c55 0 2024-03-22T10:42:55.683 INFO:tasks.workunit.client.1.smithi150.stdout:3/466: write d5/d22/d37/f79 [328976,29518] 0 2024-03-22T10:42:55.683 INFO:tasks.workunit.client.1.smithi150.stdout:9/502: chown d1/d8/de/df/d24/c62 1436338 1 2024-03-22T10:42:55.684 INFO:tasks.workunit.client.0.smithi115.stdout:6/790: dwrite d4/d13/d33/f59 [0,4194304] 0 2024-03-22T10:42:55.684 INFO:tasks.workunit.client.0.smithi115.stdout:3/899: dwrite d3/d19/dd8/ddb/fdc [0,4194304] 0 2024-03-22T10:42:55.685 INFO:tasks.workunit.client.1.smithi150.stdout:1/421: creat d1/d19/d1a/d7d/f82 x:0 0 0 2024-03-22T10:42:55.685 INFO:tasks.workunit.client.1.smithi150.stdout:0/493: mkdir d0/d1/d84/d86/d11/d26/d76/d96 0 2024-03-22T10:42:55.686 INFO:tasks.workunit.client.1.smithi150.stdout:8/506: symlink d0/d2/d46/da2/lb2 0 2024-03-22T10:42:55.686 INFO:tasks.workunit.client.1.smithi150.stdout:5/384: fsync d0/d8/f23 0 2024-03-22T10:42:55.688 INFO:tasks.workunit.client.0.smithi115.stdout:5/887: creat d2/d15/f120 x:0 0 0 2024-03-22T10:42:55.689 INFO:tasks.workunit.client.0.smithi115.stdout:3/900: symlink d3/d19/df5/l130 0 2024-03-22T10:42:55.689 INFO:tasks.workunit.client.0.smithi115.stdout:1/899: dwrite d0/d12/d17/d67/dd8/de0/f128 [0,4194304] 0 2024-03-22T10:42:55.690 INFO:tasks.workunit.client.1.smithi150.stdout:4/509: unlink d0/de/f3d 0 2024-03-22T10:42:55.690 INFO:tasks.workunit.client.0.smithi115.stdout:0/820: write d3/d7/d1c/d6c/d87/dd6/ff3 [985578,44589] 0 2024-03-22T10:42:55.697 INFO:tasks.workunit.client.1.smithi150.stdout:7/490: dread d2/d5/d7/f50 [0,4194304] 0 2024-03-22T10:42:55.697 INFO:tasks.workunit.client.0.smithi115.stdout:3/901: rmdir d3/d19/d2b/d49 39 2024-03-22T10:42:55.698 INFO:tasks.workunit.client.1.smithi150.stdout:8/507: read d0/d2/d19/d29/d2f/f3b [782647,102864] 0 2024-03-22T10:42:55.701 INFO:tasks.workunit.client.1.smithi150.stdout:5/385: write d0/d8/dc/de/f13 [1060113,65324] 0 2024-03-22T10:42:55.705 INFO:tasks.workunit.client.1.smithi150.stdout:3/467: mkdir d5/da/d9d/d9f 0 2024-03-22T10:42:55.706 INFO:tasks.workunit.client.1.smithi150.stdout:1/422: fdatasync d1/da/d15/d18/d4b/f6a 0 2024-03-22T10:42:55.706 INFO:tasks.workunit.client.1.smithi150.stdout:8/508: read - d0/d2/d19/d29/d2f/df/f97 zero size 2024-03-22T10:42:55.706 INFO:tasks.workunit.client.0.smithi115.stdout:3/902: creat d3/d82/d57/dc5/f131 x:0 0 0 2024-03-22T10:42:55.706 INFO:tasks.workunit.client.1.smithi150.stdout:6/456: mknod db/dd/d15/ca0 0 2024-03-22T10:42:55.712 INFO:tasks.workunit.client.1.smithi150.stdout:4/510: read d0/d1b/d4d/d6c/d61/f48 [1635221,17693] 0 2024-03-22T10:42:55.712 INFO:tasks.workunit.client.1.smithi150.stdout:2/517: rename d12/d16/d2c/l3f to d12/d16/d67/d3c/la7 0 2024-03-22T10:42:55.713 INFO:tasks.workunit.client.1.smithi150.stdout:8/509: chown d0/d2/d19/d29/d2f/df/d1b/f1c 21747171 1 2024-03-22T10:42:55.714 INFO:tasks.workunit.client.0.smithi115.stdout:3/903: chown d3/d36/d41/lff 6514575 1 2024-03-22T10:42:55.714 INFO:tasks.workunit.client.0.smithi115.stdout:6/791: dwrite d4/fec [0,4194304] 0 2024-03-22T10:42:55.714 INFO:tasks.workunit.client.1.smithi150.stdout:5/386: truncate d0/d8/dc/de/d14/d2e/f62 477622 0 2024-03-22T10:42:55.715 INFO:tasks.workunit.client.0.smithi115.stdout:0/821: dwrite d3/d4/d6d/f9e [0,4194304] 0 2024-03-22T10:42:55.717 INFO:tasks.workunit.client.0.smithi115.stdout:3/904: truncate d3/d89/f12c 29382 0 2024-03-22T10:42:55.718 INFO:tasks.workunit.client.1.smithi150.stdout:9/503: symlink d1/d49/d98/lb9 0 2024-03-22T10:42:55.718 INFO:tasks.workunit.client.1.smithi150.stdout:5/387: fdatasync d0/f10 0 2024-03-22T10:42:55.721 INFO:tasks.workunit.client.1.smithi150.stdout:9/504: dread - d1/d8/d95/fa6 zero size 2024-03-22T10:42:55.725 INFO:tasks.workunit.client.1.smithi150.stdout:5/388: readlink d0/d8/dc/d1d/d53/d31/l65 0 2024-03-22T10:42:55.725 INFO:tasks.workunit.client.0.smithi115.stdout:6/792: getdents d4/d11/d91 0 2024-03-22T10:42:55.726 INFO:tasks.workunit.client.0.smithi115.stdout:0/822: dread d3/d7/d1b/d1f/f62 [0,4194304] 0 2024-03-22T10:42:55.726 INFO:tasks.workunit.client.0.smithi115.stdout:5/888: dwrite d2/d15/d1b/d7c/d38/d43/d49/d81/dbd/f11e [0,4194304] 0 2024-03-22T10:42:55.730 INFO:tasks.workunit.client.1.smithi150.stdout:7/491: dread d2/d5/d7/d9/db/f76 [0,4194304] 0 2024-03-22T10:42:55.731 INFO:tasks.workunit.client.1.smithi150.stdout:6/457: creat db/dd/d2e/fa1 x:0 0 0 2024-03-22T10:42:55.733 INFO:tasks.workunit.client.1.smithi150.stdout:4/511: creat d0/d47/faa x:0 0 0 2024-03-22T10:42:55.733 INFO:tasks.workunit.client.0.smithi115.stdout:1/900: dwrite d0/d22/fda [0,4194304] 0 2024-03-22T10:42:55.737 INFO:tasks.workunit.client.1.smithi150.stdout:8/510: readlink d0/d2/d19/d29/d37/l8f 0 2024-03-22T10:42:55.738 INFO:tasks.workunit.client.0.smithi115.stdout:6/793: creat d4/d13/f101 x:0 0 0 2024-03-22T10:42:55.738 INFO:tasks.workunit.client.0.smithi115.stdout:5/889: creat d2/df/d13/d8b/d93/da6/d10e/f121 x:0 0 0 2024-03-22T10:42:55.738 INFO:tasks.workunit.client.0.smithi115.stdout:1/901: chown d0/d8/d4d/d56/db6/ddc/d13a 27 1 2024-03-22T10:42:55.741 INFO:tasks.workunit.client.1.smithi150.stdout:0/494: dread d0/f45 [0,4194304] 0 2024-03-22T10:42:55.746 INFO:tasks.workunit.client.0.smithi115.stdout:1/902: symlink d0/d8/de/d1a/d91/da3/l13b 0 2024-03-22T10:42:55.749 INFO:tasks.workunit.client.1.smithi150.stdout:0/495: write d0/f63 [154992,50374] 0 2024-03-22T10:42:55.749 INFO:tasks.workunit.client.1.smithi150.stdout:3/468: link d5/da/f70 d5/d53/fa0 0 2024-03-22T10:42:55.751 INFO:tasks.workunit.client.1.smithi150.stdout:2/518: dwrite d12/d16/d61/f86 [0,4194304] 0 2024-03-22T10:42:55.752 INFO:tasks.workunit.client.0.smithi115.stdout:0/823: getdents d3/da/d4c/da3/da0 0 2024-03-22T10:42:55.752 INFO:tasks.workunit.client.0.smithi115.stdout:5/890: truncate d2/d15/d1b/d7c/d38/d56/f87 1311051 0 2024-03-22T10:42:55.753 INFO:tasks.workunit.client.0.smithi115.stdout:1/903: chown d0/d22/fda 0 1 2024-03-22T10:42:55.753 INFO:tasks.workunit.client.0.smithi115.stdout:3/905: dwrite d3/d36/d41/fa2 [0,4194304] 0 2024-03-22T10:42:55.754 INFO:tasks.workunit.client.1.smithi150.stdout:0/496: write d0/d1/d84/d86/d11/d66/f8b [939061,119385] 0 2024-03-22T10:42:55.756 INFO:tasks.workunit.client.0.smithi115.stdout:5/891: write d2/d15/d1b/d7c/f105 [359778,30932] 0 2024-03-22T10:42:55.757 INFO:tasks.workunit.client.1.smithi150.stdout:9/505: creat d1/d8/de/df/d20/d32/d75/d99/da7/fba x:0 0 0 2024-03-22T10:42:55.758 INFO:tasks.workunit.client.0.smithi115.stdout:0/824: creat d3/d4/d14/f10b x:0 0 0 2024-03-22T10:42:55.759 INFO:tasks.workunit.client.1.smithi150.stdout:5/389: dwrite d0/d8/dc/de/d14/d2e/d46/d4d/f51 [0,4194304] 0 2024-03-22T10:42:55.761 INFO:tasks.workunit.client.0.smithi115.stdout:1/904: unlink d0/d8/d4d/d56/dbd/fbf 0 2024-03-22T10:42:55.762 INFO:tasks.workunit.client.1.smithi150.stdout:0/497: write d0/d1/d84/d86/d11/d66/d35/f62 [4947475,53229] 0 2024-03-22T10:42:55.763 INFO:tasks.workunit.client.0.smithi115.stdout:5/892: stat d2/d15/d1b/d7c/l10d 0 2024-03-22T10:42:55.765 INFO:tasks.workunit.client.0.smithi115.stdout:3/906: creat d3/d16/d29/dc1/dfd/f132 x:0 0 0 2024-03-22T10:42:55.767 INFO:tasks.workunit.client.0.smithi115.stdout:6/794: dwrite d4/d11/d12/f9c [0,4194304] 0 2024-03-22T10:42:55.767 INFO:tasks.workunit.client.1.smithi150.stdout:6/458: rmdir db/dd/d82/d84 39 2024-03-22T10:42:55.771 INFO:tasks.workunit.client.1.smithi150.stdout:8/511: creat d0/d2/d19/d29/d37/fb3 x:0 0 0 2024-03-22T10:42:55.773 INFO:tasks.workunit.client.0.smithi115.stdout:1/905: mkdir d0/d8/d4d/d56/db6/d11b/d13c 0 2024-03-22T10:42:55.774 INFO:tasks.workunit.client.1.smithi150.stdout:2/519: write d12/d16/d67/d3c/f6b [1784848,114122] 0 2024-03-22T10:42:55.775 INFO:tasks.workunit.client.0.smithi115.stdout:0/825: mknod d3/da/d4c/d69/dd1/c10c 0 2024-03-22T10:42:55.776 INFO:tasks.workunit.client.0.smithi115.stdout:3/907: fdatasync d3/d19/d2b/d49/d91/f9f 0 2024-03-22T10:42:55.777 INFO:tasks.workunit.client.0.smithi115.stdout:1/906: mkdir d0/d12/d17/d37/d119/d13d 0 2024-03-22T10:42:55.777 INFO:tasks.workunit.client.1.smithi150.stdout:5/390: unlink d0/d8/l42 0 2024-03-22T10:42:55.780 INFO:tasks.workunit.client.1.smithi150.stdout:3/469: dwrite d5/d22/d2f/d38/d3e/d5b/f7f [0,4194304] 0 2024-03-22T10:42:55.780 INFO:tasks.workunit.client.1.smithi150.stdout:2/520: chown d12/d16/d2c/f57 2 1 2024-03-22T10:42:55.781 INFO:tasks.workunit.client.0.smithi115.stdout:0/826: dread - d3/da/d4c/d74/d84/dd7/fbf zero size 2024-03-22T10:42:55.783 INFO:tasks.workunit.client.0.smithi115.stdout:5/893: dread d2/d15/d1b/d7c/f105 [0,4194304] 0 2024-03-22T10:42:55.784 INFO:tasks.workunit.client.1.smithi150.stdout:7/492: mknod d2/d5/ca3 0 2024-03-22T10:42:55.784 INFO:tasks.workunit.client.1.smithi150.stdout:3/470: readlink d5/d53/l65 0 2024-03-22T10:42:55.784 INFO:tasks.workunit.client.1.smithi150.stdout:1/423: link d1/d19/d50/d6e/l7e d1/da/d15/d4f/l83 0 2024-03-22T10:42:55.785 INFO:tasks.workunit.client.0.smithi115.stdout:3/908: truncate d3/d6d/db4/de1/ffc 80403 0 2024-03-22T10:42:55.786 INFO:tasks.workunit.client.0.smithi115.stdout:0/827: write d3/da/d4c/d69/d6a/f86 [1862200,7809] 0 2024-03-22T10:42:55.787 INFO:tasks.workunit.client.0.smithi115.stdout:7/847: sync 2024-03-22T10:42:55.787 INFO:tasks.workunit.client.0.smithi115.stdout:4/858: sync 2024-03-22T10:42:55.788 INFO:tasks.workunit.client.1.smithi150.stdout:7/493: stat d2/d5/d7/d8/d57/d54 0 2024-03-22T10:42:55.788 INFO:tasks.workunit.client.0.smithi115.stdout:4/859: chown d1/dbb 153817687 1 2024-03-22T10:42:55.789 INFO:tasks.workunit.client.0.smithi115.stdout:6/795: dwrite d4/d13/d33/fba [0,4194304] 0 2024-03-22T10:42:55.789 INFO:tasks.workunit.client.1.smithi150.stdout:6/459: creat db/dd/d15/d28/d6c/fa2 x:0 0 0 2024-03-22T10:42:55.790 INFO:tasks.workunit.client.0.smithi115.stdout:1/907: read d0/d12/d17/f39 [1869686,7788] 0 2024-03-22T10:42:55.791 INFO:tasks.workunit.client.0.smithi115.stdout:0/828: write d3/d7/d1b/f47 [4563744,115089] 0 2024-03-22T10:42:55.792 INFO:tasks.workunit.client.0.smithi115.stdout:3/909: mknod d3/d6d/db4/d107/db3/c133 0 2024-03-22T10:42:55.792 INFO:tasks.workunit.client.0.smithi115.stdout:5/894: getdents d2/d15/d32 0 2024-03-22T10:42:55.792 INFO:tasks.workunit.client.0.smithi115.stdout:6/796: stat d4/d36/d48/d7a 0 2024-03-22T10:42:55.794 INFO:tasks.workunit.client.1.smithi150.stdout:8/512: rename d0/f6f to d0/d2/d19/d29/d31/fb4 0 2024-03-22T10:42:55.794 INFO:tasks.workunit.client.1.smithi150.stdout:9/506: truncate d1/d8/de/df/d36/d40/d41/f61 1473202 0 2024-03-22T10:42:55.798 INFO:tasks.workunit.client.0.smithi115.stdout:3/910: read d3/d36/d41/fa2 [1717266,80295] 0 2024-03-22T10:42:55.800 INFO:tasks.workunit.client.0.smithi115.stdout:1/908: chown d0/d12/d17/d37/d119/d139/c28 1825 1 2024-03-22T10:42:55.802 INFO:tasks.workunit.client.1.smithi150.stdout:9/507: write d1/d8/de/df/d36/f39 [3555376,12916] 0 2024-03-22T10:42:55.805 INFO:tasks.workunit.client.0.smithi115.stdout:5/895: dread d2/d15/d1b/d7c/d38/fd2 [0,4194304] 0 2024-03-22T10:42:55.806 INFO:tasks.workunit.client.0.smithi115.stdout:0/829: creat d3/d7/d1b/df8/f10d x:0 0 0 2024-03-22T10:42:55.809 INFO:tasks.workunit.client.0.smithi115.stdout:0/830: readlink d3/d7/d1b/d35/l53 0 2024-03-22T10:42:55.810 INFO:tasks.workunit.client.1.smithi150.stdout:8/513: mkdir d0/d3c/d71/d7a/d89/db5 0 2024-03-22T10:42:55.811 INFO:tasks.workunit.client.0.smithi115.stdout:3/911: unlink d3/d16/c74 0 2024-03-22T10:42:55.811 INFO:tasks.workunit.client.0.smithi115.stdout:4/860: link d1/d30/d4c/l85 d1/d30/d4c/d86/d9f/d75/d99/d98/l127 0 2024-03-22T10:42:55.813 INFO:tasks.workunit.client.1.smithi150.stdout:3/471: dwrite d5/d22/d2f/d38/d3e/f6f [0,4194304] 0 2024-03-22T10:42:55.815 INFO:tasks.workunit.client.1.smithi150.stdout:7/494: creat d2/d5/d7/fa4 x:0 0 0 2024-03-22T10:42:55.818 INFO:tasks.workunit.client.1.smithi150.stdout:2/521: getdents d12/d48 0 2024-03-22T10:42:55.821 INFO:tasks.workunit.client.0.smithi115.stdout:4/861: creat d1/f128 x:0 0 0 2024-03-22T10:42:55.825 INFO:tasks.workunit.client.0.smithi115.stdout:1/909: dwrite d0/d12/d17/f7d [0,4194304] 0 2024-03-22T10:42:55.828 INFO:tasks.workunit.client.1.smithi150.stdout:7/495: symlink d2/d5/d7/d38/la5 0 2024-03-22T10:42:55.830 INFO:tasks.workunit.client.1.smithi150.stdout:1/424: dwrite d1/da/d2a/f35 [0,4194304] 0 2024-03-22T10:42:55.830 INFO:tasks.workunit.client.1.smithi150.stdout:2/522: creat d12/d48/d63/fa8 x:0 0 0 2024-03-22T10:42:55.831 INFO:tasks.workunit.client.0.smithi115.stdout:4/862: creat d1/d66/d106/d68/f129 x:0 0 0 2024-03-22T10:42:55.833 INFO:tasks.workunit.client.0.smithi115.stdout:4/863: chown d1/d66/d106/d28 846894608 1 2024-03-22T10:42:55.834 INFO:tasks.workunit.client.1.smithi150.stdout:7/496: stat d2/d5/d7/d9/db/d9f/d45/c9e 0 2024-03-22T10:42:55.834 INFO:tasks.workunit.client.0.smithi115.stdout:1/910: fdatasync d0/d8/de/d1a/f61 0 2024-03-22T10:42:55.835 INFO:tasks.workunit.client.0.smithi115.stdout:5/896: dread d2/df/f67 [4194304,4194304] 0 2024-03-22T10:42:55.836 INFO:tasks.workunit.client.1.smithi150.stdout:4/512: rmdir d0/d47 39 2024-03-22T10:42:55.836 INFO:tasks.workunit.client.0.smithi115.stdout:3/912: dwrite d3/d16/d29/dc1/ff3 [0,4194304] 0 2024-03-22T10:42:55.848 INFO:tasks.workunit.client.1.smithi150.stdout:9/508: getdents d1/d8/de/d73 0 2024-03-22T10:42:55.852 INFO:tasks.workunit.client.1.smithi150.stdout:2/523: read d12/d19/d1a/f30 [1104740,21200] 0 2024-03-22T10:42:55.852 INFO:tasks.workunit.client.1.smithi150.stdout:3/472: dread d5/d53/f75 [0,4194304] 0 2024-03-22T10:42:55.852 INFO:tasks.workunit.client.1.smithi150.stdout:7/497: write d2/d5/d7/d38/f4b [4214739,70812] 0 2024-03-22T10:42:55.853 INFO:tasks.workunit.client.1.smithi150.stdout:5/391: dread d0/d8/dc/de/f15 [0,4194304] 0 2024-03-22T10:42:55.853 INFO:tasks.workunit.client.1.smithi150.stdout:9/509: write d1/d8/f88 [184927,46256] 0 2024-03-22T10:42:55.854 INFO:tasks.workunit.client.1.smithi150.stdout:1/425: rename d1/d19/d1a/c22 to d1/da/d15/d4f/c84 0 2024-03-22T10:42:55.855 INFO:tasks.workunit.client.0.smithi115.stdout:4/864: mkdir d1/d30/d4c/d86/d9f/d75/d99/da8/dea/d12a 0 2024-03-22T10:42:55.859 INFO:tasks.workunit.client.1.smithi150.stdout:6/460: truncate db/dd/d15/d60/f86 724846 0 2024-03-22T10:42:55.859 INFO:tasks.workunit.client.0.smithi115.stdout:1/911: truncate d0/d12/d17/d37/f49 1150964 0 2024-03-22T10:42:55.859 INFO:tasks.workunit.client.0.smithi115.stdout:2/816: sync 2024-03-22T10:42:55.860 INFO:tasks.workunit.client.0.smithi115.stdout:9/867: sync 2024-03-22T10:42:55.862 INFO:tasks.workunit.client.0.smithi115.stdout:5/897: link d2/df/d3b/c4e d2/d15/c122 0 2024-03-22T10:42:55.865 INFO:tasks.workunit.client.1.smithi150.stdout:5/392: chown d0/d8/c5d 37 1 2024-03-22T10:42:55.865 INFO:tasks.workunit.client.1.smithi150.stdout:0/498: dread d0/d1/d84/d86/f28 [0,4194304] 0 2024-03-22T10:42:55.865 INFO:tasks.workunit.client.0.smithi115.stdout:6/797: dread d4/d11/d12/f4e [0,4194304] 0 2024-03-22T10:42:55.866 INFO:tasks.workunit.client.1.smithi150.stdout:6/461: fdatasync db/dd/d15/d6a/d8d/f95 0 2024-03-22T10:42:55.868 INFO:tasks.workunit.client.1.smithi150.stdout:4/513: rename d0/d32/f4e to d0/d32/d7c/d99/fab 0 2024-03-22T10:42:55.869 INFO:tasks.workunit.client.0.smithi115.stdout:2/817: fsync da/d25/d3d/d7e/fea 0 2024-03-22T10:42:55.869 INFO:tasks.workunit.client.0.smithi115.stdout:1/912: write d0/d8/d4d/d56/dbd/f10c [377330,113156] 0 2024-03-22T10:42:55.870 INFO:tasks.workunit.client.1.smithi150.stdout:8/514: dwrite d0/f34 [4194304,4194304] 0 2024-03-22T10:42:55.871 INFO:tasks.workunit.client.0.smithi115.stdout:9/868: write d6/d7/d30/d37/f125 [110842,23900] 0 2024-03-22T10:42:55.872 INFO:tasks.workunit.client.1.smithi150.stdout:3/473: creat d5/d53/d7e/d95/fa1 x:0 0 0 2024-03-22T10:42:55.872 INFO:tasks.workunit.client.1.smithi150.stdout:5/393: read d0/d8/dc/de/d14/d2e/d46/d4d/f6c [728644,107781] 0 2024-03-22T10:42:55.873 INFO:tasks.workunit.client.1.smithi150.stdout:6/462: chown db/dd/d15/d57/f65 5 1 2024-03-22T10:42:55.873 INFO:tasks.workunit.client.0.smithi115.stdout:0/831: write d3/f33 [1324735,18157] 0 2024-03-22T10:42:55.874 INFO:tasks.workunit.client.1.smithi150.stdout:1/426: creat d1/d79/f85 x:0 0 0 2024-03-22T10:42:55.875 INFO:tasks.workunit.client.1.smithi150.stdout:4/514: readlink d0/d2f/d63/d7b/l97 0 2024-03-22T10:42:55.876 INFO:tasks.workunit.client.1.smithi150.stdout:7/498: rmdir d2/d5/d7/d9/db/d9f/d45 39 2024-03-22T10:42:55.880 INFO:tasks.workunit.client.1.smithi150.stdout:0/499: write d0/d1/d84/d86/d42/d10/d38/d52/f54 [1951004,26351] 0 2024-03-22T10:42:55.880 INFO:tasks.workunit.client.1.smithi150.stdout:1/427: dread - d1/d19/d50/f5c zero size 2024-03-22T10:42:55.880 INFO:tasks.workunit.client.0.smithi115.stdout:3/913: dwrite d3/d36/f51 [0,4194304] 0 2024-03-22T10:42:55.881 INFO:tasks.workunit.client.0.smithi115.stdout:1/913: symlink d0/d8/d4d/d56/db6/d11b/l13e 0 2024-03-22T10:42:55.882 INFO:tasks.workunit.client.0.smithi115.stdout:7/848: dread d2/d7/d12/d10/d19/d42/d76/d74/f113 [0,4194304] 0 2024-03-22T10:42:55.883 INFO:tasks.workunit.client.1.smithi150.stdout:2/524: rename d12/d16 to d12/d48/da9 0 2024-03-22T10:42:55.887 INFO:tasks.workunit.client.0.smithi115.stdout:6/798: write d4/d11/d12/d1c/d8a/d98/d9f/dc3/fd1 [1020444,67784] 0 2024-03-22T10:42:55.888 INFO:tasks.workunit.client.1.smithi150.stdout:1/428: chown d1/da/d15/d18/f5f 4524801 1 2024-03-22T10:42:55.890 INFO:tasks.workunit.client.0.smithi115.stdout:2/818: getdents da/d25/d3d/d57/d7d 0 2024-03-22T10:42:55.893 INFO:tasks.workunit.client.1.smithi150.stdout:2/525: write d12/d48/da9/d67/f74 [566462,118845] 0 2024-03-22T10:42:55.894 INFO:tasks.workunit.client.1.smithi150.stdout:4/515: unlink d0/f5f 0 2024-03-22T10:42:55.895 INFO:tasks.workunit.client.0.smithi115.stdout:2/819: dread - da/fe3 zero size 2024-03-22T10:42:55.895 INFO:tasks.workunit.client.0.smithi115.stdout:5/898: dwrite d2/df/f10 [4194304,4194304] 0 2024-03-22T10:42:55.896 INFO:tasks.workunit.client.0.smithi115.stdout:1/914: chown d0/d12/d17/d37/d9e/da0/dff/c122 96128217 1 2024-03-22T10:42:55.896 INFO:tasks.workunit.client.0.smithi115.stdout:9/869: rename d6/d8/d3c/d59/fb5 to d6/d7/d20/d22/d23/d106/da4/f131 0 2024-03-22T10:42:55.897 INFO:tasks.workunit.client.1.smithi150.stdout:8/515: dread d0/d2/f86 [0,4194304] 0 2024-03-22T10:42:55.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:55 smithi150 ceph-mon[34518]: pgmap v16: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 54 MiB/s rd, 202 MiB/s wr, 356 op/s 2024-03-22T10:42:55.902 INFO:tasks.workunit.client.1.smithi150.stdout:8/516: fsync d0/d2/d19/d29/d2f/df/d1b/fa0 0 2024-03-22T10:42:55.904 INFO:tasks.workunit.client.1.smithi150.stdout:6/463: unlink db/dd/d15/d28/l5d 0 2024-03-22T10:42:55.904 INFO:tasks.workunit.client.1.smithi150.stdout:3/474: read d5/d22/d2f/d38/d3e/f3f [516120,97904] 0 2024-03-22T10:42:55.904 INFO:tasks.workunit.client.0.smithi115.stdout:6/799: creat d4/d13/d33/dbe/f102 x:0 0 0 2024-03-22T10:42:55.905 INFO:tasks.workunit.client.0.smithi115.stdout:7/849: symlink d2/d7/d12/d10/d19/d42/d76/l122 0 2024-03-22T10:42:55.906 INFO:tasks.workunit.client.1.smithi150.stdout:7/499: rmdir d2/d5/d7/d38 39 2024-03-22T10:42:55.906 INFO:tasks.workunit.client.0.smithi115.stdout:2/820: chown da/d25/d30/l77 1857424 1 2024-03-22T10:42:55.907 INFO:tasks.workunit.client.0.smithi115.stdout:7/850: readlink d2/d7/d12/d10/d23/d3d/dd2/da3/ld8 0 2024-03-22T10:42:55.908 INFO:tasks.workunit.client.1.smithi150.stdout:3/475: chown d5/d22/d37/f79 91110435 1 2024-03-22T10:42:55.910 INFO:tasks.workunit.client.1.smithi150.stdout:7/500: stat d2/d5/d7/d8/d46/f5a 0 2024-03-22T10:42:55.914 INFO:tasks.workunit.client.0.smithi115.stdout:0/832: dread d3/d7/fd8 [0,4194304] 0 2024-03-22T10:42:55.916 INFO:tasks.workunit.client.0.smithi115.stdout:0/833: fdatasync d3/f33 0 2024-03-22T10:42:55.919 INFO:tasks.workunit.client.0.smithi115.stdout:1/915: symlink d0/d8/d4d/d56/db6/ddc/d135/l13f 0 2024-03-22T10:42:55.919 INFO:tasks.workunit.client.0.smithi115.stdout:0/834: fdatasync d3/da/d4c/d69/fe1 0 2024-03-22T10:42:55.919 INFO:tasks.workunit.client.0.smithi115.stdout:2/821: mkdir da/d1c/d67/df8 0 2024-03-22T10:42:55.919 INFO:tasks.workunit.client.1.smithi150.stdout:6/464: creat db/dd/d15/d28/fa3 x:0 0 0 2024-03-22T10:42:55.923 INFO:tasks.workunit.client.0.smithi115.stdout:3/914: write d3/d36/f5c [1734626,125787] 0 2024-03-22T10:42:55.925 INFO:tasks.workunit.client.1.smithi150.stdout:0/500: mknod d0/d1/d84/d86/d11/d66/d35/d92/c97 0 2024-03-22T10:42:55.927 INFO:tasks.workunit.client.1.smithi150.stdout:5/394: creat d0/d8/f76 x:0 0 0 2024-03-22T10:42:55.928 INFO:tasks.workunit.client.0.smithi115.stdout:6/800: mkdir d4/d13/d33/d103 0 2024-03-22T10:42:55.929 INFO:tasks.workunit.client.1.smithi150.stdout:9/510: truncate d1/d8/de/df/d20/d32/f2a 1507625 0 2024-03-22T10:42:55.929 INFO:tasks.workunit.client.1.smithi150.stdout:0/501: chown d0/d1/d84/d86/d11/d66 151916 1 2024-03-22T10:42:55.930 INFO:tasks.workunit.client.0.smithi115.stdout:9/870: write d6/d7/d20/d9c/fb0 [1375790,72010] 0 2024-03-22T10:42:55.932 INFO:tasks.workunit.client.0.smithi115.stdout:7/851: symlink d2/d7/l123 0 2024-03-22T10:42:55.933 INFO:tasks.workunit.client.0.smithi115.stdout:0/835: creat d3/d7/d1c/d6c/d87/dd6/f10e x:0 0 0 2024-03-22T10:42:55.933 INFO:tasks.workunit.client.1.smithi150.stdout:7/501: creat d2/d5/d7/d8/d57/d54/d91/fa6 x:0 0 0 2024-03-22T10:42:55.934 INFO:tasks.workunit.client.1.smithi150.stdout:2/526: rename d12/d19/f1b to d12/d19/d2d/faa 0 2024-03-22T10:42:55.934 INFO:tasks.workunit.client.0.smithi115.stdout:2/822: mkdir da/d25/d3d/d57/d7d/da1/df9 0 2024-03-22T10:42:55.934 INFO:tasks.workunit.client.0.smithi115.stdout:3/915: dread d3/d19/dd8/fb9 [0,4194304] 0 2024-03-22T10:42:55.935 INFO:tasks.workunit.client.0.smithi115.stdout:7/852: dread - d2/ffe zero size 2024-03-22T10:42:55.936 INFO:tasks.workunit.client.0.smithi115.stdout:0/836: fdatasync d3/df0/fef 0 2024-03-22T10:42:55.936 INFO:tasks.workunit.client.1.smithi150.stdout:5/395: creat d0/d8/dc/de/d14/d4f/d47/f77 x:0 0 0 2024-03-22T10:42:55.939 INFO:tasks.workunit.client.1.smithi150.stdout:9/511: symlink d1/d8/de/df/da2/d8c/lbb 0 2024-03-22T10:42:55.939 INFO:tasks.workunit.client.0.smithi115.stdout:6/801: rename d4/d13/d33/l57 to d4/d11/d12/d1c/d8a/d98/l104 0 2024-03-22T10:42:55.939 INFO:tasks.workunit.client.0.smithi115.stdout:4/865: dread d1/d30/d4c/d86/d9f/f34 [0,4194304] 0 2024-03-22T10:42:55.940 INFO:tasks.workunit.client.0.smithi115.stdout:1/916: mkdir d0/d12/d17/d37/d9e/de9/dc1/d12f/d140 0 2024-03-22T10:42:55.941 INFO:tasks.workunit.client.1.smithi150.stdout:4/516: link d0/d47/f96 d0/d1b/fac 0 2024-03-22T10:42:55.943 INFO:tasks.workunit.client.1.smithi150.stdout:8/517: getdents d0/d2/d19/d29/d37 0 2024-03-22T10:42:55.943 INFO:tasks.workunit.client.1.smithi150.stdout:5/396: write d0/d8/dc/de/d14/f16 [223343,85483] 0 2024-03-22T10:42:55.945 INFO:tasks.workunit.client.1.smithi150.stdout:4/517: readlink d0/d32/l3f 0 2024-03-22T10:42:55.945 INFO:tasks.workunit.client.1.smithi150.stdout:3/476: dwrite d5/d53/fa0 [0,4194304] 0 2024-03-22T10:42:55.945 INFO:tasks.workunit.client.0.smithi115.stdout:5/899: dread d2/df/d13/d8b/d93/da6/dbc/d50/db1/fdc [0,4194304] 0 2024-03-22T10:42:55.946 INFO:tasks.workunit.client.1.smithi150.stdout:9/512: chown d1/d8/de/df/d36/d40/d41/f61 5553 1 2024-03-22T10:42:55.947 INFO:tasks.workunit.client.1.smithi150.stdout:6/465: creat db/dd/fa4 x:0 0 0 2024-03-22T10:42:55.948 INFO:tasks.workunit.client.1.smithi150.stdout:4/518: chown d0/d47/faa 195252429 1 2024-03-22T10:42:55.951 INFO:tasks.workunit.client.1.smithi150.stdout:2/527: creat d12/d48/da9/d67/d3c/fab x:0 0 0 2024-03-22T10:42:55.951 INFO:tasks.workunit.client.1.smithi150.stdout:5/397: chown d0/d8/dc/de/d14/d2e/d5a/d66 295 1 2024-03-22T10:42:55.953 INFO:tasks.workunit.client.1.smithi150.stdout:9/513: chown d1/d8/de/df/f1f 217 1 2024-03-22T10:42:55.955 INFO:tasks.workunit.client.1.smithi150.stdout:7/502: mknod d2/d5/d7/d8/d46/ca7 0 2024-03-22T10:42:55.955 INFO:tasks.workunit.client.0.smithi115.stdout:3/916: unlink d3/d36/f51 0 2024-03-22T10:42:55.960 INFO:tasks.workunit.client.1.smithi150.stdout:2/528: truncate d12/d48/da9/d2c/d65/f9d 778462 0 2024-03-22T10:42:55.960 INFO:tasks.workunit.client.1.smithi150.stdout:9/514: write d1/d8/de/df/d36/f39 [3096590,21910] 0 2024-03-22T10:42:55.960 INFO:tasks.workunit.client.0.smithi115.stdout:7/853: truncate d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f91 372055 0 2024-03-22T10:42:55.962 INFO:tasks.workunit.client.0.smithi115.stdout:6/802: getdents d4/d13/d33/dbe 0 2024-03-22T10:42:55.962 INFO:tasks.workunit.client.1.smithi150.stdout:3/477: mknod d5/da/d78/ca2 0 2024-03-22T10:42:55.963 INFO:tasks.workunit.client.1.smithi150.stdout:9/515: dread - d1/d8/de/df/d36/f96 zero size 2024-03-22T10:42:55.964 INFO:tasks.workunit.client.0.smithi115.stdout:5/900: rename d2/df/d3b/dfb/fff to d2/df/d13/d8b/d93/da6/d10e/d106/f123 0 2024-03-22T10:42:55.965 INFO:tasks.workunit.client.0.smithi115.stdout:2/823: dwrite da/d25/d3d/d7e/d90/dbf/d54/fb7 [0,4194304] 0 2024-03-22T10:42:55.966 INFO:tasks.workunit.client.1.smithi150.stdout:1/429: rename d1/d19/d1a/c2d to d1/da/d15/c86 0 2024-03-22T10:42:55.970 INFO:tasks.workunit.client.0.smithi115.stdout:6/803: chown d4/d13/d33/d58/fb7 11 1 2024-03-22T10:42:55.972 INFO:tasks.workunit.client.0.smithi115.stdout:7/854: dread - d2/d7/d12/d10/d23/d3d/dd2/da3/de8/dc7/fff zero size 2024-03-22T10:42:55.974 INFO:tasks.workunit.client.1.smithi150.stdout:6/466: link db/dd/d15/d60/f9c db/dd/d25/fa5 0 2024-03-22T10:42:55.974 INFO:tasks.workunit.client.0.smithi115.stdout:0/837: write d3/f46 [1156707,76372] 0 2024-03-22T10:42:55.976 INFO:tasks.workunit.client.0.smithi115.stdout:4/866: dwrite d1/d66/d106/d35/d40/d5c/d55/fef [0,4194304] 0 2024-03-22T10:42:55.978 INFO:tasks.workunit.client.1.smithi150.stdout:1/430: symlink d1/d79/l87 0 2024-03-22T10:42:55.978 INFO:tasks.workunit.client.0.smithi115.stdout:0/838: dread - d3/da/d4c/d74/d84/dd7/fbf zero size 2024-03-22T10:42:55.979 INFO:tasks.workunit.client.1.smithi150.stdout:8/518: creat d0/fb6 x:0 0 0 2024-03-22T10:42:55.979 INFO:tasks.workunit.client.1.smithi150.stdout:0/502: rename d0/d1/d84/d86/d11/d26/d76/d96 to d0/d1/d84/d86/d42/d10/d38/d52/d98 0 2024-03-22T10:42:55.983 INFO:tasks.workunit.client.0.smithi115.stdout:0/839: write d3/d7/d1c/f5d [413628,11064] 0 2024-03-22T10:42:55.984 INFO:tasks.workunit.client.0.smithi115.stdout:4/867: chown d1/d66/d106/d35/d40/d54/cbf 437903 1 2024-03-22T10:42:55.987 INFO:tasks.workunit.client.0.smithi115.stdout:5/901: truncate d2/df/d13/d8b/d93/da6/fcf 721187 0 2024-03-22T10:42:55.988 INFO:tasks.workunit.client.0.smithi115.stdout:6/804: unlink d4/d36/c83 0 2024-03-22T10:42:55.988 INFO:tasks.workunit.client.1.smithi150.stdout:5/398: dwrite d0/d8/dc/d1d/d53/d31/f6b [0,4194304] 0 2024-03-22T10:42:55.991 INFO:tasks.workunit.client.1.smithi150.stdout:7/503: getdents d2/d5/d7/d8/d57 0 2024-03-22T10:42:55.992 INFO:tasks.workunit.client.0.smithi115.stdout:5/902: stat d2/d15/d1b/d7c/d38/d56 0 2024-03-22T10:42:55.993 INFO:tasks.workunit.client.0.smithi115.stdout:0/840: unlink d3/d52/lf2 0 2024-03-22T10:42:55.993 INFO:tasks.workunit.client.0.smithi115.stdout:3/917: write d3/d19/d2b/d49/fbd [740593,68598] 0 2024-03-22T10:42:55.999 INFO:tasks.workunit.client.0.smithi115.stdout:1/917: dwrite d0/d12/d17/d37/d9e/de9/dc1/f12d [0,4194304] 0 2024-03-22T10:42:56.002 INFO:tasks.workunit.client.0.smithi115.stdout:5/903: write d2/d15/d1b/d7c/d84/f118 [798096,33918] 0 2024-03-22T10:42:56.002 INFO:tasks.workunit.client.1.smithi150.stdout:9/516: dwrite d1/d8/de/df/d36/d40/d41/f94 [0,4194304] 0 2024-03-22T10:42:56.002 INFO:tasks.workunit.client.1.smithi150.stdout:4/519: dwrite d0/de/f7f [0,4194304] 0 2024-03-22T10:42:56.002 INFO:tasks.workunit.client.1.smithi150.stdout:1/431: write d1/da/d2a/f7f [904178,123639] 0 2024-03-22T10:42:56.004 INFO:tasks.workunit.client.1.smithi150.stdout:7/504: truncate d2/d5/d7/d8/d46/d49/f68 710264 0 2024-03-22T10:42:56.006 INFO:tasks.workunit.client.1.smithi150.stdout:3/478: rename d5/da/c6b to d5/ca3 0 2024-03-22T10:42:56.008 INFO:tasks.workunit.client.0.smithi115.stdout:5/904: dread - d2/d15/d1b/d7c/d38/d43/d49/fb7 zero size 2024-03-22T10:42:56.010 INFO:tasks.workunit.client.1.smithi150.stdout:4/520: unlink d0/d1b/d4d/d6c/d61/c7d 0 2024-03-22T10:42:56.011 INFO:tasks.workunit.client.0.smithi115.stdout:7/855: link d2/d7/d12/d10/d23/d3d/dd2/da3/de8/dc7/fe0 d2/d7/d12/d10/d19/dbd/f124 0 2024-03-22T10:42:56.014 INFO:tasks.workunit.client.1.smithi150.stdout:9/517: creat d1/d8/de/df/d20/d32/fbc x:0 0 0 2024-03-22T10:42:56.014 INFO:tasks.workunit.client.0.smithi115.stdout:9/871: dread d6/d7/d48/d70/da9/fac [0,4194304] 0 2024-03-22T10:42:56.015 INFO:tasks.workunit.client.1.smithi150.stdout:4/521: truncate d0/d32/f41 998622 0 2024-03-22T10:42:56.016 INFO:tasks.workunit.client.0.smithi115.stdout:0/841: creat d3/df0/dca/dce/f10f x:0 0 0 2024-03-22T10:42:56.021 INFO:tasks.workunit.client.0.smithi115.stdout:5/905: creat d2/d15/d1b/d7c/d38/d4a/d74/f124 x:0 0 0 2024-03-22T10:42:56.023 INFO:tasks.workunit.client.1.smithi150.stdout:5/399: dread d0/d8/dc/d1d/d53/d31/f6b [0,4194304] 0 2024-03-22T10:42:56.023 INFO:tasks.workunit.client.0.smithi115.stdout:5/906: write d2/df/d13/fef [741624,41211] 0 2024-03-22T10:42:56.025 INFO:tasks.workunit.client.1.smithi150.stdout:0/503: rename d0/d1/d84/d86/d42/l4e to d0/d1/d84/d86/l99 0 2024-03-22T10:42:56.025 INFO:tasks.workunit.client.1.smithi150.stdout:1/432: creat d1/d19/d50/d6e/f88 x:0 0 0 2024-03-22T10:42:56.028 INFO:tasks.workunit.client.0.smithi115.stdout:1/918: getdents d0/d8/de/d1a/d91 0 2024-03-22T10:42:56.030 INFO:tasks.workunit.client.1.smithi150.stdout:4/522: stat d0/de/l19 0 2024-03-22T10:42:56.031 INFO:tasks.workunit.client.0.smithi115.stdout:7/856: dread d2/d7/d12/d10/d23/fd4 [0,4194304] 0 2024-03-22T10:42:56.031 INFO:tasks.workunit.client.0.smithi115.stdout:4/868: dread d1/d66/d106/d35/d40/d5c/f6f [0,4194304] 0 2024-03-22T10:42:56.032 INFO:tasks.workunit.client.1.smithi150.stdout:5/400: fsync d0/d8/dc/de/d14/d2e/f62 0 2024-03-22T10:42:56.033 INFO:tasks.workunit.client.0.smithi115.stdout:5/907: creat d2/df/f125 x:0 0 0 2024-03-22T10:42:56.035 INFO:tasks.workunit.client.1.smithi150.stdout:2/529: truncate d12/d48/da9/d67/d3c/f77 473422 0 2024-03-22T10:42:56.036 INFO:tasks.workunit.client.1.smithi150.stdout:5/401: stat d0/d8/dc/d1d/d53 0 2024-03-22T10:42:56.036 INFO:tasks.workunit.client.0.smithi115.stdout:0/842: truncate d3/d4/d14/d64/fab 220194 0 2024-03-22T10:42:56.036 INFO:tasks.workunit.client.1.smithi150.stdout:0/504: mknod d0/d1/d84/d86/d42/d10/d38/d52/c9a 0 2024-03-22T10:42:56.039 INFO:tasks.workunit.client.0.smithi115.stdout:0/843: stat d3/d7/f45 0 2024-03-22T10:42:56.040 INFO:tasks.workunit.client.1.smithi150.stdout:2/530: truncate d12/d48/f44 1040987 0 2024-03-22T10:42:56.041 INFO:tasks.workunit.client.0.smithi115.stdout:5/908: stat d2/df/d13/d8b/d93/da6/dbc/d50/l11b 0 2024-03-22T10:42:56.043 INFO:tasks.workunit.client.0.smithi115.stdout:9/872: dwrite d6/d7/d48/faf [0,4194304] 0 2024-03-22T10:42:56.043 INFO:tasks.workunit.client.1.smithi150.stdout:4/523: write d0/de/f4b [2543892,102831] 0 2024-03-22T10:42:56.044 INFO:tasks.workunit.client.0.smithi115.stdout:1/919: truncate d0/d12/d17/d37/d9e/de9/f90 3452389 0 2024-03-22T10:42:56.046 INFO:tasks.workunit.client.0.smithi115.stdout:3/918: write d3/d36/d41/d85/fcc [847186,56199] 0 2024-03-22T10:42:56.049 INFO:tasks.workunit.client.0.smithi115.stdout:0/844: write d3/da/d4c/d69/d6a/fa4 [3570850,20337] 0 2024-03-22T10:42:56.049 INFO:tasks.workunit.client.1.smithi150.stdout:6/467: dread db/dd/d25/f6d [0,4194304] 0 2024-03-22T10:42:56.051 INFO:tasks.workunit.client.1.smithi150.stdout:8/519: truncate d0/d2/d19/d29/d37/f85 576387 0 2024-03-22T10:42:56.051 INFO:tasks.workunit.client.1.smithi150.stdout:3/479: write d5/d53/f54 [4812030,103165] 0 2024-03-22T10:42:56.052 INFO:tasks.workunit.client.0.smithi115.stdout:1/920: chown d0/d12/d17/f23 8569 1 2024-03-22T10:42:56.052 INFO:tasks.workunit.client.1.smithi150.stdout:9/518: dwrite d1/d8/de/df/d20/d29/f9b [0,4194304] 0 2024-03-22T10:42:56.055 INFO:tasks.workunit.client.0.smithi115.stdout:3/919: symlink d3/d6d/db4/l134 0 2024-03-22T10:42:56.057 INFO:tasks.workunit.client.0.smithi115.stdout:6/805: dwrite d4/d13/d33/fba [4194304,4194304] 0 2024-03-22T10:42:56.058 INFO:tasks.workunit.client.1.smithi150.stdout:4/524: unlink d0/de/l19 0 2024-03-22T10:42:56.064 INFO:tasks.workunit.client.1.smithi150.stdout:9/519: dread - d1/d8/de/df/d36/d40/d41/fac zero size 2024-03-22T10:42:56.064 INFO:tasks.workunit.client.0.smithi115.stdout:4/869: dwrite d1/d69/fe6 [0,4194304] 0 2024-03-22T10:42:56.065 INFO:tasks.workunit.client.1.smithi150.stdout:0/505: mkdir d0/d1/d84/d86/d42/d10/d38/d52/d9b 0 2024-03-22T10:42:56.066 INFO:tasks.workunit.client.1.smithi150.stdout:8/520: write d0/d2/d19/d29/d2f/df/d1b/f30 [3772526,97767] 0 2024-03-22T10:42:56.066 INFO:tasks.workunit.client.0.smithi115.stdout:9/873: rename d6/d7/d48/l11a to d6/d7/d30/d3b/l132 0 2024-03-22T10:42:56.068 INFO:tasks.workunit.client.0.smithi115.stdout:3/920: unlink d3/d6d/db4/de1/c114 0 2024-03-22T10:42:56.068 INFO:tasks.workunit.client.0.smithi115.stdout:4/870: dread - d1/d66/d82/da1/f112 zero size 2024-03-22T10:42:56.069 INFO:tasks.workunit.client.0.smithi115.stdout:6/806: creat d4/d11/d12/d1c/d8a/f105 x:0 0 0 2024-03-22T10:42:56.070 INFO:tasks.workunit.client.0.smithi115.stdout:9/874: chown d6/d7/d30/d37/fa0 10 1 2024-03-22T10:42:56.070 INFO:tasks.workunit.client.1.smithi150.stdout:5/402: dwrite d0/d8/f34 [0,4194304] 0 2024-03-22T10:42:56.071 INFO:tasks.workunit.client.1.smithi150.stdout:1/433: write d1/d19/d1a/f46 [674513,97733] 0 2024-03-22T10:42:56.072 INFO:tasks.workunit.client.0.smithi115.stdout:0/845: dread d3/d4/d6d/ff1 [0,4194304] 0 2024-03-22T10:42:56.074 INFO:tasks.workunit.client.0.smithi115.stdout:5/909: getdents d2/d15/d1b/d7c/d38/d43/d49/d81/dbd 0 2024-03-22T10:42:56.074 INFO:tasks.workunit.client.1.smithi150.stdout:7/505: dwrite d2/d5/d7/d8/d57/d54/f55 [0,4194304] 0 2024-03-22T10:42:56.077 INFO:tasks.workunit.client.0.smithi115.stdout:1/921: truncate d0/d12/d17/d37/d119/d139/fe5 431161 0 2024-03-22T10:42:56.081 INFO:tasks.workunit.client.0.smithi115.stdout:1/922: dread - d0/d12/d17/d37/d9e/de9/dc1/fa2 zero size 2024-03-22T10:42:56.081 INFO:tasks.workunit.client.0.smithi115.stdout:4/871: unlink d1/d30/d4c/d86/d9f/la4 0 2024-03-22T10:42:56.082 INFO:tasks.workunit.client.0.smithi115.stdout:9/875: rename d6/d7/d48/df4/c92 to d6/d7/d6a/d6f/c133 0 2024-03-22T10:42:56.086 INFO:tasks.workunit.client.0.smithi115.stdout:0/846: mknod d3/d7/d1c/d6c/d87/dd6/c110 0 2024-03-22T10:42:56.089 INFO:tasks.workunit.client.0.smithi115.stdout:0/847: stat d3/d7/d1b/d1f/lb5 0 2024-03-22T10:42:56.090 INFO:tasks.workunit.client.1.smithi150.stdout:3/480: dwrite d5/d22/d37/f6c [4194304,4194304] 0 2024-03-22T10:42:56.090 INFO:tasks.workunit.client.0.smithi115.stdout:5/910: mkdir d2/df/d13/d8b/d93/da6/d126 0 2024-03-22T10:42:56.092 INFO:tasks.workunit.client.0.smithi115.stdout:4/872: fsync d1/d66/d82/da1/fd3 0 2024-03-22T10:42:56.094 INFO:tasks.workunit.client.0.smithi115.stdout:9/876: unlink d6/d7/d20/d22/lee 0 2024-03-22T10:42:56.098 INFO:tasks.workunit.client.0.smithi115.stdout:5/911: unlink d2/fc 0 2024-03-22T10:42:56.098 INFO:tasks.workunit.client.1.smithi150.stdout:0/506: symlink d0/d1/d84/d86/l9c 0 2024-03-22T10:42:56.099 INFO:tasks.workunit.client.0.smithi115.stdout:1/923: read d0/d12/d2b/f35 [3661172,124523] 0 2024-03-22T10:42:56.100 INFO:tasks.workunit.client.0.smithi115.stdout:3/921: dwrite d3/d82/d52/fe6 [0,4194304] 0 2024-03-22T10:42:56.102 INFO:tasks.workunit.client.1.smithi150.stdout:6/468: write db/dd/d15/d60/f67 [2050448,110063] 0 2024-03-22T10:42:56.102 INFO:tasks.workunit.client.1.smithi150.stdout:1/434: truncate d1/f6c 3087815 0 2024-03-22T10:42:56.105 INFO:tasks.workunit.client.1.smithi150.stdout:7/506: readlink d2/d5/d7/d8/d57/d4a/l15 0 2024-03-22T10:42:56.106 INFO:tasks.workunit.client.0.smithi115.stdout:4/873: creat d1/d30/d4c/d86/d9f/d75/d99/f12b x:0 0 0 2024-03-22T10:42:56.106 INFO:tasks.workunit.client.0.smithi115.stdout:5/912: mkdir d2/df/d3b/dfb/d127 0 2024-03-22T10:42:56.106 INFO:tasks.workunit.client.1.smithi150.stdout:5/403: write d0/d8/dc/d1d/d53/d31/f6b [5242865,105751] 0 2024-03-22T10:42:56.108 INFO:tasks.workunit.client.0.smithi115.stdout:1/924: mkdir d0/d22/d89/d141 0 2024-03-22T10:42:56.109 INFO:tasks.workunit.client.1.smithi150.stdout:8/521: dwrite d0/d2/d19/d29/d2f/df/f45 [0,4194304] 0 2024-03-22T10:42:56.109 INFO:tasks.workunit.client.1.smithi150.stdout:6/469: write db/dd/f37 [293668,18595] 0 2024-03-22T10:42:56.111 INFO:tasks.workunit.client.1.smithi150.stdout:1/435: dread - d1/d19/d50/f69 zero size 2024-03-22T10:42:56.112 INFO:tasks.workunit.client.0.smithi115.stdout:0/848: getdents d3/d7/d6b/d94 0 2024-03-22T10:42:56.115 INFO:tasks.workunit.client.1.smithi150.stdout:0/507: symlink d0/d1/d84/d86/d11/d66/d37/l9d 0 2024-03-22T10:42:56.115 INFO:tasks.workunit.client.0.smithi115.stdout:9/877: rename d6/fcf to d6/d7/f134 0 2024-03-22T10:42:56.121 INFO:tasks.workunit.client.1.smithi150.stdout:5/404: write d0/d8/dc/de/f70 [51444,39290] 0 2024-03-22T10:42:56.122 INFO:tasks.workunit.client.0.smithi115.stdout:7/857: dread d2/d7/d12/d10/f17 [0,4194304] 0 2024-03-22T10:42:56.123 INFO:tasks.workunit.client.1.smithi150.stdout:0/508: write d0/d1/d84/d86/d42/f77 [1742679,49849] 0 2024-03-22T10:42:56.125 INFO:tasks.workunit.client.1.smithi150.stdout:8/522: creat d0/d2/d19/d29/d2e/fb7 x:0 0 0 2024-03-22T10:42:56.126 INFO:tasks.workunit.client.1.smithi150.stdout:5/405: write d0/f11 [4031225,84308] 0 2024-03-22T10:42:56.126 INFO:tasks.workunit.client.0.smithi115.stdout:4/874: getdents d1/d66/d106/d35/d40/d5c/d14 0 2024-03-22T10:42:56.126 INFO:tasks.workunit.client.0.smithi115.stdout:1/925: link d0/d12/d17/d67/dd8/de0/lf1 d0/d68/d81/l142 0 2024-03-22T10:42:56.127 INFO:tasks.workunit.client.0.smithi115.stdout:9/878: dread d6/d7/d48/f50 [0,4194304] 0 2024-03-22T10:42:56.128 INFO:tasks.workunit.client.1.smithi150.stdout:6/470: creat db/dd/d2e/fa6 x:0 0 0 2024-03-22T10:42:56.130 INFO:tasks.workunit.client.1.smithi150.stdout:4/525: dwrite d0/d47/f7e [0,4194304] 0 2024-03-22T10:42:56.131 INFO:tasks.workunit.client.1.smithi150.stdout:1/436: symlink d1/d19/d50/d51/l89 0 2024-03-22T10:42:56.131 INFO:tasks.workunit.client.0.smithi115.stdout:7/858: mknod d2/d7/d114/c125 0 2024-03-22T10:42:56.135 INFO:tasks.workunit.client.0.smithi115.stdout:4/875: creat d1/d30/d4c/d86/dde/d103/f12c x:0 0 0 2024-03-22T10:42:56.138 INFO:tasks.workunit.client.1.smithi150.stdout:8/523: write d0/d2/d19/d29/d2f/f7b [704456,52047] 0 2024-03-22T10:42:56.138 INFO:tasks.workunit.client.0.smithi115.stdout:7/859: mkdir d2/d62/d126 0 2024-03-22T10:42:56.139 INFO:tasks.workunit.client.1.smithi150.stdout:3/481: fdatasync d5/d53/f71 0 2024-03-22T10:42:56.143 INFO:tasks.workunit.client.1.smithi150.stdout:3/482: readlink d5/da/d78/d86/l73 0 2024-03-22T10:42:56.144 INFO:tasks.workunit.client.0.smithi115.stdout:1/926: truncate d0/d8/d4d/d56/db6/fee 46936 0 2024-03-22T10:42:56.144 INFO:tasks.workunit.client.0.smithi115.stdout:4/876: fdatasync d1/d69/f65 0 2024-03-22T10:42:56.145 INFO:tasks.workunit.client.0.smithi115.stdout:3/922: dwrite d3/d19/d2b/d49/d91/fa3 [0,4194304] 0 2024-03-22T10:42:56.146 INFO:tasks.workunit.client.0.smithi115.stdout:5/913: read d2/d15/d1b/d7c/d38/f3d [4863159,21341] 0 2024-03-22T10:42:56.148 INFO:tasks.workunit.client.0.smithi115.stdout:6/807: dwrite d4/d11/d12/f4e [0,4194304] 0 2024-03-22T10:42:56.150 INFO:tasks.workunit.client.0.smithi115.stdout:1/927: dread - d0/d12/d17/d67/f95 zero size 2024-03-22T10:42:56.154 INFO:tasks.workunit.client.1.smithi150.stdout:7/507: dwrite d2/d5/d7/d9/db/d9f/d45/f72 [0,4194304] 0 2024-03-22T10:42:56.155 INFO:tasks.workunit.client.0.smithi115.stdout:7/860: getdents d2/d7/d12/d10/d19 0 2024-03-22T10:42:56.158 INFO:tasks.workunit.client.0.smithi115.stdout:4/877: truncate d1/d66/d106/d28/fe8 792771 0 2024-03-22T10:42:56.158 INFO:tasks.workunit.client.0.smithi115.stdout:6/808: creat d4/d11/f106 x:0 0 0 2024-03-22T10:42:56.160 INFO:tasks.workunit.client.1.smithi150.stdout:7/508: fsync d2/d5/d7/d8/d46/d49/f68 0 2024-03-22T10:42:56.160 INFO:tasks.workunit.client.1.smithi150.stdout:6/471: symlink db/dd/d15/d57/la7 0 2024-03-22T10:42:56.161 INFO:tasks.workunit.client.1.smithi150.stdout:5/406: creat d0/d8/dc/de/d14/d2e/d45/f78 x:0 0 0 2024-03-22T10:42:56.163 INFO:tasks.workunit.client.0.smithi115.stdout:3/923: dread d3/d6d/db4/d107/da6/fc9 [0,4194304] 0 2024-03-22T10:42:56.164 INFO:tasks.workunit.client.0.smithi115.stdout:6/809: truncate d4/d13/d33/dbe/f102 996656 0 2024-03-22T10:42:56.166 INFO:tasks.workunit.client.1.smithi150.stdout:4/526: mknod d0/d32/cad 0 2024-03-22T10:42:56.166 INFO:tasks.workunit.client.1.smithi150.stdout:5/407: stat d0/d8/dc/de/d14/d2e/l69 0 2024-03-22T10:42:56.166 INFO:tasks.workunit.client.1.smithi150.stdout:9/520: dread d1/d8/de/df/d24/f4a [0,4194304] 0 2024-03-22T10:42:56.168 INFO:tasks.workunit.client.0.smithi115.stdout:5/914: rename d2/df/d13/d8b/d93/da6/dbc/d50/df5/l11d to d2/d15/d1b/d7c/d38/d4a/d74/l128 0 2024-03-22T10:42:56.170 INFO:tasks.workunit.client.1.smithi150.stdout:1/437: mkdir d1/da/d15/d18/d4b/d8a 0 2024-03-22T10:42:56.170 INFO:tasks.workunit.client.1.smithi150.stdout:2/531: truncate d12/d48/f4d 133030 0 2024-03-22T10:42:56.170 INFO:tasks.workunit.client.0.smithi115.stdout:4/878: unlink d1/d30/d4c/d86/d9f/d75/d99/f97 0 2024-03-22T10:42:56.171 INFO:tasks.workunit.client.0.smithi115.stdout:3/924: truncate d3/d19/d2b/d49/d91/dad/fc3 1011591 0 2024-03-22T10:42:56.172 INFO:tasks.workunit.client.0.smithi115.stdout:4/879: read d1/fd [162025,52107] 0 2024-03-22T10:42:56.175 INFO:tasks.workunit.client.0.smithi115.stdout:6/810: creat d4/d11/d12/d1c/d8a/d98/d9f/df0/f107 x:0 0 0 2024-03-22T10:42:56.175 INFO:tasks.workunit.client.0.smithi115.stdout:5/915: read d2/d15/d1b/d7c/d38/d56/fcd [3730978,7046] 0 2024-03-22T10:42:56.176 INFO:tasks.workunit.client.1.smithi150.stdout:7/509: write d2/d5/d7/f50 [3119029,21857] 0 2024-03-22T10:42:56.176 INFO:tasks.workunit.client.1.smithi150.stdout:9/521: truncate d1/d8/de/d73/f74 772240 0 2024-03-22T10:42:56.176 INFO:tasks.workunit.client.1.smithi150.stdout:4/527: read d0/f11 [7299307,15171] 0 2024-03-22T10:42:56.178 INFO:tasks.workunit.client.0.smithi115.stdout:6/811: chown d4/f24 34 1 2024-03-22T10:42:56.179 INFO:tasks.workunit.client.1.smithi150.stdout:2/532: write d12/d48/da9/d67/d3c/f6b [525659,96340] 0 2024-03-22T10:42:56.181 INFO:tasks.workunit.client.1.smithi150.stdout:8/524: link d0/d2/d7e/d96/f99 d0/d2/d19/d29/d2e/d70/fb8 0 2024-03-22T10:42:56.182 INFO:tasks.workunit.client.1.smithi150.stdout:5/408: fdatasync d0/d8/dc/f12 0 2024-03-22T10:42:56.183 INFO:tasks.workunit.client.0.smithi115.stdout:5/916: dread - d2/d15/f104 zero size 2024-03-22T10:42:56.185 INFO:tasks.workunit.client.1.smithi150.stdout:2/533: write d12/d21/f4f [1426134,56317] 0 2024-03-22T10:42:56.186 INFO:tasks.workunit.client.0.smithi115.stdout:7/861: rename d2/d7/d12/d10/d19/d42/d76/d74/fd0 to d2/d7/d12/d10/d23/d3d/dd2/f127 0 2024-03-22T10:42:56.186 INFO:tasks.workunit.client.0.smithi115.stdout:3/925: creat d3/d19/dd8/ddb/f135 x:0 0 0 2024-03-22T10:42:56.189 INFO:tasks.workunit.client.0.smithi115.stdout:3/926: stat d3/d82/f27 0 2024-03-22T10:42:56.190 INFO:tasks.workunit.client.1.smithi150.stdout:2/534: write d12/d48/da9/d67/d3c/f7f [1618302,57694] 0 2024-03-22T10:42:56.197 INFO:tasks.workunit.client.0.smithi115.stdout:5/917: mkdir d2/d15/d1b/d7c/d38/d129 0 2024-03-22T10:42:56.198 INFO:tasks.workunit.client.0.smithi115.stdout:6/812: dwrite d4/d11/d12/f9c [0,4194304] 0 2024-03-22T10:42:56.200 INFO:tasks.workunit.client.0.smithi115.stdout:4/880: dread d1/d30/d4c/d86/d9f/d75/d99/da8/fca [0,4194304] 0 2024-03-22T10:42:56.201 INFO:tasks.workunit.client.0.smithi115.stdout:7/862: mknod d2/d7/d12/d10/d23/d3d/c128 0 2024-03-22T10:42:56.202 INFO:tasks.workunit.client.1.smithi150.stdout:8/525: chown d0/d2/d19/d29/d37/f85 1 1 2024-03-22T10:42:56.204 INFO:tasks.workunit.client.1.smithi150.stdout:1/438: rename d1/da/fb to d1/d19/d71/f8b 0 2024-03-22T10:42:56.205 INFO:tasks.workunit.client.0.smithi115.stdout:5/918: creat d2/df/d13/d8b/d93/da6/dbc/d50/f12a x:0 0 0 2024-03-22T10:42:56.206 INFO:tasks.workunit.client.0.smithi115.stdout:9/879: write d6/d7/d19/f27 [1717316,124858] 0 2024-03-22T10:42:56.213 INFO:tasks.workunit.client.1.smithi150.stdout:0/509: dwrite d0/d1/d84/d86/d42/d10/d53/f57 [0,4194304] 0 2024-03-22T10:42:56.215 INFO:tasks.workunit.client.1.smithi150.stdout:7/510: dwrite d2/d5/d7/d8/d46/d49/f68 [0,4194304] 0 2024-03-22T10:42:56.215 INFO:tasks.workunit.client.0.smithi115.stdout:9/880: mkdir d6/d7/d19/d135 0 2024-03-22T10:42:56.218 INFO:tasks.workunit.client.0.smithi115.stdout:5/919: read d2/df/d13/d8b/d93/da6/dc3/fe3 [142352,67850] 0 2024-03-22T10:42:56.221 INFO:tasks.workunit.client.0.smithi115.stdout:3/927: dwrite d3/d82/d57/dc5/f131 [0,4194304] 0 2024-03-22T10:42:56.222 INFO:tasks.workunit.client.0.smithi115.stdout:9/881: chown d6/d5e/l6e 1961528 1 2024-03-22T10:42:56.226 INFO:tasks.workunit.client.0.smithi115.stdout:1/928: dwrite d0/d12/d2b/f97 [0,4194304] 0 2024-03-22T10:42:56.228 INFO:tasks.workunit.client.0.smithi115.stdout:4/881: dread d1/d30/d4c/d86/d9f/f34 [0,4194304] 0 2024-03-22T10:42:56.228 INFO:tasks.workunit.client.1.smithi150.stdout:9/522: fsync d1/d8/de/df/d36/d40/d41/fb2 0 2024-03-22T10:42:56.229 INFO:tasks.workunit.client.1.smithi150.stdout:5/409: mkdir d0/d33/d6f/d79 0 2024-03-22T10:42:56.232 INFO:tasks.workunit.client.1.smithi150.stdout:6/472: getdents db/dd/d15/d60/d50 0 2024-03-22T10:42:56.232 INFO:tasks.workunit.client.0.smithi115.stdout:1/929: stat d0/d8/d4d/c64 0 2024-03-22T10:42:56.233 INFO:tasks.workunit.client.1.smithi150.stdout:5/410: stat d0/d8/dc/d64 0 2024-03-22T10:42:56.233 INFO:tasks.workunit.client.0.smithi115.stdout:3/928: mknod d3/d19/d2b/d49/d91/dad/c136 0 2024-03-22T10:42:56.234 INFO:tasks.workunit.client.1.smithi150.stdout:8/526: dwrite d0/d3c/f66 [0,4194304] 0 2024-03-22T10:42:56.239 INFO:tasks.workunit.client.0.smithi115.stdout:1/930: mknod d0/d12/d17/d37/d9e/da0/c143 0 2024-03-22T10:42:56.240 INFO:tasks.workunit.client.1.smithi150.stdout:9/523: chown d1/d8/de/d73/c9d 9 1 2024-03-22T10:42:56.242 INFO:tasks.workunit.client.0.smithi115.stdout:4/882: truncate d1/d66/d106/d35/d40/d5c/d14/f3e 6582866 0 2024-03-22T10:42:56.246 INFO:tasks.workunit.client.1.smithi150.stdout:7/511: mkdir d2/d5/d7/d9/db/d6c/da8 0 2024-03-22T10:42:56.246 INFO:tasks.workunit.client.1.smithi150.stdout:9/524: write d1/d8/de/df/d36/d40/d41/fac [1027857,32285] 0 2024-03-22T10:42:56.248 INFO:tasks.workunit.client.0.smithi115.stdout:4/883: fdatasync d1/d30/d4c/d86/d9f/d32/fcf 0 2024-03-22T10:42:56.255 INFO:tasks.workunit.client.0.smithi115.stdout:4/884: chown d1/d66/d82/c126 261938 1 2024-03-22T10:42:56.255 INFO:tasks.workunit.client.1.smithi150.stdout:0/510: dwrite d0/d1/d84/d86/d5a/f7c [0,4194304] 0 2024-03-22T10:42:56.256 INFO:tasks.workunit.client.1.smithi150.stdout:6/473: truncate db/dd/f14 8554879 0 2024-03-22T10:42:56.257 INFO:tasks.workunit.client.1.smithi150.stdout:4/528: write d0/de/f3c [927757,73315] 0 2024-03-22T10:42:56.262 INFO:tasks.workunit.client.0.smithi115.stdout:7/863: dwrite d2/d7/d12/d10/d19/dbd/dce/fd1 [0,4194304] 0 2024-03-22T10:42:56.264 INFO:tasks.workunit.client.1.smithi150.stdout:9/525: write d1/d8/de/df/d24/d4e/f77 [672173,110121] 0 2024-03-22T10:42:56.265 INFO:tasks.workunit.client.1.smithi150.stdout:6/474: dread - db/dd/d25/f76 zero size 2024-03-22T10:42:56.266 INFO:tasks.workunit.client.1.smithi150.stdout:4/529: fdatasync d0/d1b/d4d/f8c 0 2024-03-22T10:42:56.267 INFO:tasks.workunit.client.1.smithi150.stdout:7/512: unlink d2/d5/d7/d9/f92 0 2024-03-22T10:42:56.268 INFO:tasks.workunit.client.0.smithi115.stdout:4/885: mknod d1/d30/d4c/d86/dde/d103/c12d 0 2024-03-22T10:42:56.271 INFO:tasks.workunit.client.1.smithi150.stdout:5/411: mknod d0/d8/dc/d1d/d53/d63/c7a 0 2024-03-22T10:42:56.274 INFO:tasks.workunit.client.1.smithi150.stdout:5/412: read d0/d8/f23 [1343067,80968] 0 2024-03-22T10:42:56.274 INFO:tasks.workunit.client.1.smithi150.stdout:8/527: dread d0/d2/d19/d29/d2f/df/f45 [0,4194304] 0 2024-03-22T10:42:56.274 INFO:tasks.workunit.client.0.smithi115.stdout:9/882: write d6/d7/d20/d22/d23/fc4 [3466725,70573] 0 2024-03-22T10:42:56.274 INFO:tasks.workunit.client.0.smithi115.stdout:3/929: dwrite d3/d16/d29/f125 [0,4194304] 0 2024-03-22T10:42:56.275 INFO:tasks.workunit.client.1.smithi150.stdout:6/475: chown db/dd/d15/d28/d6c/fa2 1029516753 1 2024-03-22T10:42:56.275 INFO:tasks.workunit.client.1.smithi150.stdout:9/526: mkdir d1/d8/de/dbd 0 2024-03-22T10:42:56.275 INFO:tasks.workunit.client.0.smithi115.stdout:1/931: dwrite d0/d12/d17/d37/d9e/de9/dc1/fc0 [0,4194304] 0 2024-03-22T10:42:56.277 INFO:tasks.workunit.client.1.smithi150.stdout:6/476: readlink db/dd/d15/d60/l78 0 2024-03-22T10:42:56.279 INFO:tasks.workunit.client.0.smithi115.stdout:5/920: dread d2/df/d13/d8b/d93/da6/fe0 [0,4194304] 0 2024-03-22T10:42:56.282 INFO:tasks.workunit.client.1.smithi150.stdout:2/535: truncate d12/d19/d2d/f53 787515 0 2024-03-22T10:42:56.284 INFO:tasks.workunit.client.1.smithi150.stdout:6/477: chown db/dd/d15/d60/f53 6252 1 2024-03-22T10:42:56.290 INFO:tasks.workunit.client.1.smithi150.stdout:8/528: truncate d0/d2/d19/d29/d31/fae 15214 0 2024-03-22T10:42:56.290 INFO:tasks.workunit.client.1.smithi150.stdout:1/439: creat d1/da/d15/d18/f8c x:0 0 0 2024-03-22T10:42:56.291 INFO:tasks.workunit.client.1.smithi150.stdout:7/513: mkdir d2/d5/d7/d8/d57/d4a/d8d/da9 0 2024-03-22T10:42:56.291 INFO:tasks.workunit.client.0.smithi115.stdout:4/886: symlink d1/d66/d106/d35/d40/d54/l12e 0 2024-03-22T10:42:56.292 INFO:tasks.workunit.client.1.smithi150.stdout:5/413: creat d0/d8/dc/d64/f7b x:0 0 0 2024-03-22T10:42:56.292 INFO:tasks.workunit.client.1.smithi150.stdout:2/536: symlink d12/d21/lac 0 2024-03-22T10:42:56.292 INFO:tasks.workunit.client.1.smithi150.stdout:9/527: truncate d1/d8/de/df/da2/d8c/fa0 4214807 0 2024-03-22T10:42:56.294 INFO:tasks.workunit.client.1.smithi150.stdout:6/478: chown db/dd/d15/d28/f41 1467 1 2024-03-22T10:42:56.295 INFO:tasks.workunit.client.1.smithi150.stdout:0/511: getdents d0/d1/d84/d86/d42 0 2024-03-22T10:42:56.295 INFO:tasks.workunit.client.0.smithi115.stdout:3/930: write d3/d82/d52/fe6 [1058419,6918] 0 2024-03-22T10:42:56.295 INFO:tasks.workunit.client.0.smithi115.stdout:7/864: truncate d2/d7/d12/d10/d23/d3d/dd2/da3/df3/f11e 431183 0 2024-03-22T10:42:56.296 INFO:tasks.workunit.client.1.smithi150.stdout:5/414: fsync d0/f10 0 2024-03-22T10:42:56.298 INFO:tasks.workunit.client.0.smithi115.stdout:6/813: dread d4/d11/d12/d1c/d8a/d98/fef [0,4194304] 0 2024-03-22T10:42:56.302 INFO:tasks.workunit.client.0.smithi115.stdout:6/814: write d4/d13/d33/dbe/fd3 [122567,123588] 0 2024-03-22T10:42:56.306 INFO:tasks.workunit.client.1.smithi150.stdout:8/529: rmdir d0/d2/d19/d29/d31 39 2024-03-22T10:42:56.308 INFO:tasks.workunit.client.1.smithi150.stdout:7/514: mkdir d2/d5/d7/d9/db/daa 0 2024-03-22T10:42:56.312 INFO:tasks.workunit.client.1.smithi150.stdout:9/528: write d1/d8/de/df/d36/d40/d41/fb2 [2657022,43425] 0 2024-03-22T10:42:56.316 INFO:tasks.workunit.client.1.smithi150.stdout:7/515: chown d2/d5/d7/d9/db/d9f/d96/c79 51698 1 2024-03-22T10:42:56.320 INFO:tasks.workunit.client.1.smithi150.stdout:7/516: write d2/d5/d7/d9/db/d6c/d9c/f9d [839606,98893] 0 2024-03-22T10:42:56.326 INFO:tasks.workunit.client.0.smithi115.stdout:9/883: dread d6/d8/d3c/d59/f90 [0,4194304] 0 2024-03-22T10:42:56.332 INFO:tasks.workunit.client.0.smithi115.stdout:5/921: creat d2/df/d13/d8b/d93/da6/d10e/d106/d10b/f12b x:0 0 0 2024-03-22T10:42:56.333 INFO:tasks.workunit.client.1.smithi150.stdout:1/440: mknod d1/da/d2a/d4e/c8d 0 2024-03-22T10:42:56.333 INFO:tasks.workunit.client.0.smithi115.stdout:5/922: chown d2/df 1090659503 1 2024-03-22T10:42:56.334 INFO:tasks.workunit.client.1.smithi150.stdout:4/530: dread d0/d32/d7c/d99/fab [0,4194304] 0 2024-03-22T10:42:56.335 INFO:tasks.workunit.client.1.smithi150.stdout:6/479: rename db/dd/d15/d60/l78 to db/dd/d15/d28/d6c/d9b/la8 0 2024-03-22T10:42:56.336 INFO:tasks.workunit.client.0.smithi115.stdout:1/932: creat d0/d8/d4d/d56/db6/ddc/d13a/f144 x:0 0 0 2024-03-22T10:42:56.337 INFO:tasks.workunit.client.1.smithi150.stdout:0/512: dwrite d0/f69 [0,4194304] 0 2024-03-22T10:42:56.337 INFO:tasks.workunit.client.1.smithi150.stdout:2/537: fsync d12/d19/d2d/faa 0 2024-03-22T10:42:56.339 INFO:tasks.workunit.client.0.smithi115.stdout:2/824: sync 2024-03-22T10:42:56.339 INFO:tasks.workunit.client.0.smithi115.stdout:0/849: sync 2024-03-22T10:42:56.340 INFO:tasks.workunit.client.0.smithi115.stdout:6/815: creat d4/d13/d33/d103/f108 x:0 0 0 2024-03-22T10:42:56.341 INFO:tasks.workunit.client.0.smithi115.stdout:5/923: truncate d2/df/d13/d8b/d93/da6/dbc/d50/f12a 265473 0 2024-03-22T10:42:56.342 INFO:tasks.workunit.client.1.smithi150.stdout:5/415: symlink d0/d8/l7c 0 2024-03-22T10:42:56.343 INFO:tasks.workunit.client.0.smithi115.stdout:1/933: creat d0/d12/d17/d37/d9e/da0/dff/f145 x:0 0 0 2024-03-22T10:42:56.343 INFO:tasks.workunit.client.1.smithi150.stdout:8/530: write d0/d2/d19/d29/d31/f49 [937338,71238] 0 2024-03-22T10:42:56.344 INFO:tasks.workunit.client.1.smithi150.stdout:4/531: fsync d0/de/f5b 0 2024-03-22T10:42:56.344 INFO:tasks.workunit.client.1.smithi150.stdout:2/538: read d12/d48/d54/f5b [211525,48292] 0 2024-03-22T10:42:56.347 INFO:tasks.workunit.client.1.smithi150.stdout:1/441: rename d1/d19/d1a/c23 to d1/d19/c8e 0 2024-03-22T10:42:56.348 INFO:tasks.workunit.client.0.smithi115.stdout:0/850: fsync d3/da/d4c/da3/f5f 0 2024-03-22T10:42:56.350 INFO:tasks.workunit.client.1.smithi150.stdout:7/517: mkdir d2/d5/d7/dab 0 2024-03-22T10:42:56.351 INFO:tasks.workunit.client.1.smithi150.stdout:5/416: fsync d0/d8/dc/de/f15 0 2024-03-22T10:42:56.352 INFO:tasks.workunit.client.1.smithi150.stdout:2/539: dread - d12/d48/da9/d67/d3c/fab zero size 2024-03-22T10:42:56.353 INFO:tasks.workunit.client.0.smithi115.stdout:2/825: dread da/d25/d51/fd4 [0,4194304] 0 2024-03-22T10:42:56.354 INFO:tasks.workunit.client.0.smithi115.stdout:5/924: dread - d2/d15/d1b/d7c/fea zero size 2024-03-22T10:42:56.356 INFO:tasks.workunit.client.1.smithi150.stdout:1/442: mknod d1/d19/d57/c8f 0 2024-03-22T10:42:56.356 INFO:tasks.workunit.client.1.smithi150.stdout:5/417: fsync d0/d8/dc/de/f70 0 2024-03-22T10:42:56.359 INFO:tasks.workunit.client.0.smithi115.stdout:3/931: dwrite d3/d36/d41/f128 [0,4194304] 0 2024-03-22T10:42:56.363 INFO:tasks.workunit.client.0.smithi115.stdout:1/934: symlink d0/d8/d4d/d56/dcf/d10b/de2/df8/l146 0 2024-03-22T10:42:56.364 INFO:tasks.workunit.client.0.smithi115.stdout:3/932: dread - d3/d19/d2b/f11e zero size 2024-03-22T10:42:56.365 INFO:tasks.workunit.client.0.smithi115.stdout:6/816: getdents d4/d36/d48/d72/daa/db9 0 2024-03-22T10:42:56.366 INFO:tasks.workunit.client.1.smithi150.stdout:2/540: rmdir d12/d48/da9/d61 39 2024-03-22T10:42:56.368 INFO:tasks.workunit.client.1.smithi150.stdout:7/518: symlink d2/d5/d7/d9/db/d9f/d96/lac 0 2024-03-22T10:42:56.369 INFO:tasks.workunit.client.0.smithi115.stdout:7/865: dread d2/d7/d12/d10/d19/d42/d76/f83 [0,4194304] 0 2024-03-22T10:42:56.369 INFO:tasks.workunit.client.1.smithi150.stdout:1/443: creat d1/d19/d50/f90 x:0 0 0 2024-03-22T10:42:56.372 INFO:tasks.workunit.client.1.smithi150.stdout:4/532: dwrite d0/d1b/d4d/f8c [0,4194304] 0 2024-03-22T10:42:56.373 INFO:tasks.workunit.client.0.smithi115.stdout:1/935: symlink d0/d12/d17/d37/d9e/de9/dde/l147 0 2024-03-22T10:42:56.373 INFO:tasks.workunit.client.0.smithi115.stdout:4/887: write d1/d66/d106/f64 [3204663,107512] 0 2024-03-22T10:42:56.373 INFO:tasks.workunit.client.1.smithi150.stdout:2/541: write d12/d48/da9/d67/d3c/f92 [660298,12212] 0 2024-03-22T10:42:56.375 INFO:tasks.workunit.client.1.smithi150.stdout:1/444: write d1/da/d15/d18/d4b/f53 [1427805,25477] 0 2024-03-22T10:42:56.377 INFO:tasks.workunit.client.0.smithi115.stdout:2/826: rename da/fe3 to da/d25/d3d/d57/d7d/da1/dba/ddf/ffa 0 2024-03-22T10:42:56.377 INFO:tasks.workunit.client.1.smithi150.stdout:6/480: rename c6 to db/dd/d15/ca9 0 2024-03-22T10:42:56.378 INFO:tasks.workunit.client.0.smithi115.stdout:6/817: creat d4/d11/d12/d1c/d8a/d98/d9f/dc3/f109 x:0 0 0 2024-03-22T10:42:56.379 INFO:tasks.workunit.client.0.smithi115.stdout:9/884: truncate d6/d7/d20/d22/d23/f85 2430693 0 2024-03-22T10:42:56.380 INFO:tasks.workunit.client.1.smithi150.stdout:7/519: readlink d2/d5/d7/d8/d57/d4a/l20 0 2024-03-22T10:42:56.380 INFO:tasks.workunit.client.1.smithi150.stdout:2/542: readlink d12/d19/d2d/d66/l7e 0 2024-03-22T10:42:56.382 INFO:tasks.workunit.client.1.smithi150.stdout:1/445: write d1/f73 [4379950,37118] 0 2024-03-22T10:42:56.383 INFO:tasks.workunit.client.1.smithi150.stdout:6/481: fsync db/dd/d25/f76 0 2024-03-22T10:42:56.387 INFO:tasks.workunit.client.0.smithi115.stdout:1/936: rmdir d0/d12/d17/d37/d119/d139 39 2024-03-22T10:42:56.387 INFO:tasks.workunit.client.1.smithi150.stdout:5/418: dwrite d0/d8/dc/de/d14/d2e/f54 [0,4194304] 0 2024-03-22T10:42:56.388 INFO:tasks.workunit.client.0.smithi115.stdout:9/885: creat d6/d7/d20/d9c/f136 x:0 0 0 2024-03-22T10:42:56.388 INFO:tasks.workunit.client.1.smithi150.stdout:6/482: read - db/dd/d15/d28/d6c/fa2 zero size 2024-03-22T10:42:56.389 INFO:tasks.workunit.client.0.smithi115.stdout:6/818: truncate d4/d11/d12/d1c/d8a/f100 189569 0 2024-03-22T10:42:56.389 INFO:tasks.workunit.client.0.smithi115.stdout:0/851: link d3/d7/d1b/d1f/d78/dc1/ce9 d3/d7/d1c/c111 0 2024-03-22T10:42:56.390 INFO:tasks.workunit.client.1.smithi150.stdout:2/543: truncate d12/d19/d2d/f82 573789 0 2024-03-22T10:42:56.394 INFO:tasks.workunit.client.1.smithi150.stdout:2/544: stat d12/d48/da9/d67/l7b 0 2024-03-22T10:42:56.394 INFO:tasks.workunit.client.1.smithi150.stdout:6/483: chown db/dd/d15/d28/f58 4909 1 2024-03-22T10:42:56.395 INFO:tasks.workunit.client.1.smithi150.stdout:2/545: fdatasync d12/d19/f1d 0 2024-03-22T10:42:56.395 INFO:tasks.workunit.client.0.smithi115.stdout:2/827: getdents da/d1c/d67/d78/d86/d9d 0 2024-03-22T10:42:56.397 INFO:tasks.workunit.client.1.smithi150.stdout:6/484: chown db/dd/f37 21255996 1 2024-03-22T10:42:56.398 INFO:tasks.workunit.client.0.smithi115.stdout:4/888: dwrite d1/d66/d106/d28/fe8 [0,4194304] 0 2024-03-22T10:42:56.398 INFO:tasks.workunit.client.0.smithi115.stdout:2/828: chown da/d25/d3d/d57/d7d/da1 994109 1 2024-03-22T10:42:56.399 INFO:tasks.workunit.client.1.smithi150.stdout:6/485: dread - db/dd/d25/f76 zero size 2024-03-22T10:42:56.399 INFO:tasks.workunit.client.1.smithi150.stdout:7/520: mkdir d2/d5/d7/d8/d57/d54/d91/dad 0 2024-03-22T10:42:56.400 INFO:tasks.workunit.client.1.smithi150.stdout:1/446: mkdir d1/d19/d1a/d91 0 2024-03-22T10:42:56.404 INFO:tasks.workunit.client.0.smithi115.stdout:0/852: mknod d3/d7/d1b/d1f/d5b/c112 0 2024-03-22T10:42:56.405 INFO:tasks.workunit.client.1.smithi150.stdout:2/546: mknod d12/d35/d7d/cad 0 2024-03-22T10:42:56.406 INFO:tasks.workunit.client.0.smithi115.stdout:6/819: mknod d4/d11/d12/d1c/d8a/d98/c10a 0 2024-03-22T10:42:56.406 INFO:tasks.workunit.client.1.smithi150.stdout:7/521: chown d2/d5/d7/d8/d57/d54/f55 8 1 2024-03-22T10:42:56.407 INFO:tasks.workunit.client.1.smithi150.stdout:1/447: truncate d1/da/d2a/d65/f76 1011547 0 2024-03-22T10:42:56.411 INFO:tasks.workunit.client.1.smithi150.stdout:0/513: rename d0/d1/d84/d86/d42/d10/d38/d52/l55 to d0/d1/l9e 0 2024-03-22T10:42:56.411 INFO:tasks.workunit.client.0.smithi115.stdout:1/937: rename d0/d12/d17/d37/d9e/la8 to d0/d12/d17/d37/d9e/de9/dc1/l148 0 2024-03-22T10:42:56.412 INFO:tasks.workunit.client.0.smithi115.stdout:3/933: write d3/d6d/db4/fed [251621,20606] 0 2024-03-22T10:42:56.413 INFO:tasks.workunit.client.0.smithi115.stdout:1/938: readlink d0/d8/l69 0 2024-03-22T10:42:56.413 INFO:tasks.workunit.client.0.smithi115.stdout:5/925: truncate d2/df/d13/fb3 553011 0 2024-03-22T10:42:56.415 INFO:tasks.workunit.client.1.smithi150.stdout:1/448: creat d1/d19/f92 x:0 0 0 2024-03-22T10:42:56.418 INFO:tasks.workunit.client.0.smithi115.stdout:6/820: mkdir d4/d11/d12/d1c/d8a/d98/d9f/dc3/d10b 0 2024-03-22T10:42:56.421 INFO:tasks.workunit.client.0.smithi115.stdout:1/939: read - d0/d12/d2b/fcb zero size 2024-03-22T10:42:56.424 INFO:tasks.workunit.client.0.smithi115.stdout:6/821: stat d4/d11/d12/d1c/d8a/d98/d9f/dc3/f109 0 2024-03-22T10:42:56.427 INFO:tasks.workunit.client.0.smithi115.stdout:2/829: getdents da/d1c/d67 0 2024-03-22T10:42:56.428 INFO:tasks.workunit.client.1.smithi150.stdout:4/533: dwrite d0/de/f5e [0,4194304] 0 2024-03-22T10:42:56.429 INFO:tasks.workunit.client.1.smithi150.stdout:0/514: creat d0/d1/d84/d86/d11/d26/f9f x:0 0 0 2024-03-22T10:42:56.430 INFO:tasks.workunit.client.0.smithi115.stdout:6/822: rmdir d4/d11/d12/d1c/d84 39 2024-03-22T10:42:56.434 INFO:tasks.workunit.client.1.smithi150.stdout:5/419: dwrite d0/d8/dc/de/d14/f25 [0,4194304] 0 2024-03-22T10:42:56.435 INFO:tasks.workunit.client.0.smithi115.stdout:9/886: dwrite d6/d7/d20/d22/d3f/fb1 [0,4194304] 0 2024-03-22T10:42:56.436 INFO:tasks.workunit.client.1.smithi150.stdout:1/449: creat d1/da/d15/f93 x:0 0 0 2024-03-22T10:42:56.438 INFO:tasks.workunit.client.1.smithi150.stdout:8/531: dwrite d0/d2/d19/d29/d2f/df/d1b/f1d [0,4194304] 0 2024-03-22T10:42:56.440 INFO:tasks.workunit.client.1.smithi150.stdout:4/534: creat d0/d32/fae x:0 0 0 2024-03-22T10:42:56.442 INFO:tasks.workunit.client.1.smithi150.stdout:2/547: dwrite d12/d21/f4f [0,4194304] 0 2024-03-22T10:42:56.450 INFO:tasks.workunit.client.1.smithi150.stdout:8/532: chown d0/fb6 38029458 1 2024-03-22T10:42:56.450 INFO:tasks.workunit.client.1.smithi150.stdout:1/450: write d1/d19/d1a/f46 [1084654,44190] 0 2024-03-22T10:42:56.451 INFO:tasks.workunit.client.0.smithi115.stdout:2/830: dwrite da/f16 [0,4194304] 0 2024-03-22T10:42:56.457 INFO:tasks.workunit.client.0.smithi115.stdout:6/823: dwrite d4/d5e/f8b [0,4194304] 0 2024-03-22T10:42:56.458 INFO:tasks.workunit.client.1.smithi150.stdout:8/533: write d0/d2/d7e/f82 [2479928,40813] 0 2024-03-22T10:42:56.459 INFO:tasks.workunit.client.1.smithi150.stdout:1/451: chown d1/da/d2a/d4e/l56 610858912 1 2024-03-22T10:42:56.460 INFO:tasks.workunit.client.0.smithi115.stdout:2/831: mkdir da/d25/dfb 0 2024-03-22T10:42:56.461 INFO:tasks.workunit.client.0.smithi115.stdout:6/824: symlink d4/d36/l10c 0 2024-03-22T10:42:56.462 INFO:tasks.workunit.client.1.smithi150.stdout:8/534: stat d0/d2/d19/d29/l54 0 2024-03-22T10:42:56.469 INFO:tasks.workunit.client.0.smithi115.stdout:6/825: stat d4/l17 0 2024-03-22T10:42:56.469 INFO:tasks.workunit.client.0.smithi115.stdout:2/832: truncate da/d25/d3d/d57/d7d/da1/ddc/fe2 972021 0 2024-03-22T10:42:56.469 INFO:tasks.workunit.client.1.smithi150.stdout:0/515: truncate d0/d1/d84/d86/d42/d10/d38/d52/f65 5070736 0 2024-03-22T10:42:56.470 INFO:tasks.workunit.client.1.smithi150.stdout:5/420: read d0/d8/dc/de/d14/d4f/d47/f6e [278190,118813] 0 2024-03-22T10:42:56.470 INFO:tasks.workunit.client.1.smithi150.stdout:7/522: dwrite d2/d5/d7/d38/f4b [0,4194304] 0 2024-03-22T10:42:56.473 INFO:tasks.workunit.client.1.smithi150.stdout:1/452: symlink d1/d19/d1a/d7d/l94 0 2024-03-22T10:42:56.474 INFO:tasks.workunit.client.1.smithi150.stdout:7/523: creat d2/d5/d7/d8/d46/d49/fae x:0 0 0 2024-03-22T10:42:56.475 INFO:tasks.workunit.client.1.smithi150.stdout:1/453: read - d1/d79/f85 zero size 2024-03-22T10:42:56.479 INFO:tasks.workunit.client.0.smithi115.stdout:2/833: creat da/d25/d3d/d57/d7d/da1/dba/ffc x:0 0 0 2024-03-22T10:42:56.481 INFO:tasks.workunit.client.0.smithi115.stdout:6/826: getdents d4/d11/d12/d1c/d8a 0 2024-03-22T10:42:56.483 INFO:tasks.workunit.client.1.smithi150.stdout:7/524: mkdir d2/d5/d7/d8/d57/d54/d91/daf 0 2024-03-22T10:42:56.484 INFO:tasks.workunit.client.1.smithi150.stdout:1/454: write d1/da/d15/d18/d4b/f6a [8817321,125360] 0 2024-03-22T10:42:56.484 INFO:tasks.workunit.client.0.smithi115.stdout:5/926: dread d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:56.485 INFO:tasks.workunit.client.1.smithi150.stdout:8/535: dwrite d0/d3c/f4a [4194304,4194304] 0 2024-03-22T10:42:56.487 INFO:tasks.workunit.client.0.smithi115.stdout:2/834: symlink da/d25/d3d/d57/d7d/da1/dba/lfd 0 2024-03-22T10:42:56.494 INFO:tasks.workunit.client.1.smithi150.stdout:0/516: mkdir d0/d1/d84/d86/d11/d66/d35/d5f/da0 0 2024-03-22T10:42:56.497 INFO:tasks.workunit.client.1.smithi150.stdout:1/455: truncate d1/d19/d1a/f2b 3109052 0 2024-03-22T10:42:56.498 INFO:tasks.workunit.client.0.smithi115.stdout:9/887: chown d6/d7/d20/d22/d23/f85 7673 1 2024-03-22T10:42:56.499 INFO:tasks.workunit.client.1.smithi150.stdout:5/421: dwrite d0/d8/dc/de/d14/d2e/d46/d4d/f51 [0,4194304] 0 2024-03-22T10:42:56.500 INFO:tasks.workunit.client.0.smithi115.stdout:9/888: stat d6/d7/d48 0 2024-03-22T10:42:56.500 INFO:tasks.workunit.client.0.smithi115.stdout:0/853: write d3/d7/f45 [3815592,107859] 0 2024-03-22T10:42:56.503 INFO:tasks.workunit.client.1.smithi150.stdout:1/456: dread - d1/da/d2a/d65/d78/f81 zero size 2024-03-22T10:42:56.503 INFO:tasks.workunit.client.1.smithi150.stdout:5/422: chown d0/d8/dc/d1d/d53/d63 110 1 2024-03-22T10:42:56.504 INFO:tasks.workunit.client.0.smithi115.stdout:4/889: dwrite d1/d66/d106/f9d [0,4194304] 0 2024-03-22T10:42:56.504 INFO:tasks.workunit.client.0.smithi115.stdout:5/927: mknod d2/d15/c12c 0 2024-03-22T10:42:56.505 INFO:tasks.workunit.client.1.smithi150.stdout:8/536: write d0/d2/d19/d29/d88/f9a [2298743,19212] 0 2024-03-22T10:42:56.509 INFO:tasks.workunit.client.1.smithi150.stdout:8/537: write d0/d2/d19/d29/d37/da1/fa4 [1222386,45516] 0 2024-03-22T10:42:56.523 INFO:tasks.workunit.client.1.smithi150.stdout:8/538: creat d0/d3c/d71/d7a/d89/fb9 x:0 0 0 2024-03-22T10:42:56.524 INFO:tasks.workunit.client.0.smithi115.stdout:1/940: dwrite d0/d8/d4d/d56/dcf/d10b/ff2 [0,4194304] 0 2024-03-22T10:42:56.524 INFO:tasks.workunit.client.0.smithi115.stdout:4/890: mknod d1/d66/d106/df0/c12f 0 2024-03-22T10:42:56.525 INFO:tasks.workunit.client.0.smithi115.stdout:3/934: dwrite d3/d36/d41/fba [0,4194304] 0 2024-03-22T10:42:56.527 INFO:tasks.workunit.client.1.smithi150.stdout:0/517: link d0/d1/c1d d0/d1/d84/d86/d42/d10/d38/d52/ca1 0 2024-03-22T10:42:56.529 INFO:tasks.workunit.client.1.smithi150.stdout:5/423: rmdir d0/d33/d6f/d79 0 2024-03-22T10:42:56.529 INFO:tasks.workunit.client.0.smithi115.stdout:0/854: mkdir d3/da/d4c/d74/d84/dac/df4/d113 0 2024-03-22T10:42:56.535 INFO:tasks.workunit.client.1.smithi150.stdout:4/535: dwrite d0/d1b/f24 [4194304,4194304] 0 2024-03-22T10:42:56.537 INFO:tasks.workunit.client.1.smithi150.stdout:5/424: dread d0/d8/dc/f5b [0,4194304] 0 2024-03-22T10:42:56.537 INFO:tasks.workunit.client.1.smithi150.stdout:2/548: dwrite d12/d19/d2d/faa [0,4194304] 0 2024-03-22T10:42:56.540 INFO:tasks.workunit.client.0.smithi115.stdout:0/855: read d3/d7/d1c/d6c/d87/dd6/ff3 [771245,76928] 0 2024-03-22T10:42:56.543 INFO:tasks.workunit.client.0.smithi115.stdout:5/928: mknod d2/d15/d1b/d7c/d38/d4a/c12d 0 2024-03-22T10:42:56.544 INFO:tasks.workunit.client.1.smithi150.stdout:2/549: rename d12/d19/d2d/l3a to d12/d21/d81/lae 0 2024-03-22T10:42:56.546 INFO:tasks.workunit.client.1.smithi150.stdout:5/425: creat d0/d8/dc/d1d/d53/f7d x:0 0 0 2024-03-22T10:42:56.547 INFO:tasks.workunit.client.1.smithi150.stdout:2/550: truncate d12/d48/f44 1399805 0 2024-03-22T10:42:56.549 INFO:tasks.workunit.client.1.smithi150.stdout:5/426: chown d0/d8/dc/de/d14/d2e/d46/c67 9024 1 2024-03-22T10:42:56.550 INFO:tasks.workunit.client.1.smithi150.stdout:2/551: fsync d12/d48/da9/d2c/f57 0 2024-03-22T10:42:56.552 INFO:tasks.workunit.client.1.smithi150.stdout:5/427: read d0/f2b [3920478,70034] 0 2024-03-22T10:42:56.555 INFO:tasks.workunit.client.1.smithi150.stdout:2/552: mknod d12/d19/caf 0 2024-03-22T10:42:56.556 INFO:tasks.workunit.client.1.smithi150.stdout:5/428: stat d0/d8/dc/d1d/d53 0 2024-03-22T10:42:56.556 INFO:tasks.workunit.client.0.smithi115.stdout:1/941: unlink d0/d12/d17/d37/d119/c12a 0 2024-03-22T10:42:56.557 INFO:tasks.workunit.client.1.smithi150.stdout:2/553: write d12/d21/f4b [495126,21792] 0 2024-03-22T10:42:56.558 INFO:tasks.workunit.client.0.smithi115.stdout:5/929: write d2/d15/d1b/d7c/f105 [519804,96463] 0 2024-03-22T10:42:56.559 INFO:tasks.workunit.client.0.smithi115.stdout:1/942: chown d0/d12/f48 77396 1 2024-03-22T10:42:56.559 INFO:tasks.workunit.client.1.smithi150.stdout:5/429: symlink d0/d8/dc/de/d14/d4f/d47/l7e 0 2024-03-22T10:42:56.560 INFO:tasks.workunit.client.0.smithi115.stdout:0/856: mknod d3/d7/d1b/c114 0 2024-03-22T10:42:56.561 INFO:tasks.workunit.client.0.smithi115.stdout:1/943: stat d0/d12/c40 0 2024-03-22T10:42:56.561 INFO:tasks.workunit.client.1.smithi150.stdout:1/457: dread d1/da/d15/d18/f3d [0,4194304] 0 2024-03-22T10:42:56.563 INFO:tasks.workunit.client.1.smithi150.stdout:1/458: mknod d1/da/d2a/d55/c95 0 2024-03-22T10:42:56.565 INFO:tasks.workunit.client.0.smithi115.stdout:5/930: creat d2/d15/d1b/d7c/d38/d43/d49/f12e x:0 0 0 2024-03-22T10:42:56.565 INFO:tasks.workunit.client.1.smithi150.stdout:1/459: mkdir d1/d79/d96 0 2024-03-22T10:42:56.566 INFO:tasks.workunit.client.1.smithi150.stdout:1/460: fsync d1/da/d15/d18/f5f 0 2024-03-22T10:42:56.570 INFO:tasks.workunit.client.1.smithi150.stdout:1/461: truncate d1/d19/d71/f8b 36898 0 2024-03-22T10:42:56.572 INFO:tasks.workunit.client.0.smithi115.stdout:5/931: creat d2/df/d13/d8b/d93/da6/dbc/d50/db1/f12f x:0 0 0 2024-03-22T10:42:56.573 INFO:tasks.workunit.client.0.smithi115.stdout:1/944: dread d0/d12/d17/d37/d119/d139/d25/d9f/fd3 [0,4194304] 0 2024-03-22T10:42:56.574 INFO:tasks.workunit.client.0.smithi115.stdout:3/935: dwrite d3/d89/fe5 [0,4194304] 0 2024-03-22T10:42:56.575 INFO:tasks.workunit.client.0.smithi115.stdout:1/945: chown d0/d8/f106 238 1 2024-03-22T10:42:56.576 INFO:tasks.workunit.client.0.smithi115.stdout:3/936: stat d3/d19/d2b/d49/fe4 0 2024-03-22T10:42:56.580 INFO:tasks.workunit.client.0.smithi115.stdout:1/946: truncate d0/d12/d2b/fcb 829231 0 2024-03-22T10:42:56.584 INFO:tasks.workunit.client.0.smithi115.stdout:1/947: creat d0/d8/d4d/d56/db6/ddc/d82/f149 x:0 0 0 2024-03-22T10:42:56.588 INFO:tasks.workunit.client.0.smithi115.stdout:1/948: creat d0/d8/de/d1a/d120/f14a x:0 0 0 2024-03-22T10:42:56.591 INFO:tasks.workunit.client.0.smithi115.stdout:1/949: chown d0/d12/d17/d37/d119/d139/d25/d11a/f136 96 1 2024-03-22T10:42:56.595 INFO:tasks.workunit.client.0.smithi115.stdout:1/950: rename d0/d8/de/d1a/f2f to d0/d12/d2b/d10e/f14b 0 2024-03-22T10:42:56.597 INFO:tasks.workunit.client.1.smithi150.stdout:1/462: dread d1/d19/d57/f5b [0,4194304] 0 2024-03-22T10:42:56.598 INFO:tasks.workunit.client.0.smithi115.stdout:1/951: mkdir d0/d12/d17/d67/d14c 0 2024-03-22T10:42:56.600 INFO:tasks.workunit.client.1.smithi150.stdout:1/463: mkdir d1/da/d2a/d55/d97 0 2024-03-22T10:42:56.605 INFO:tasks.workunit.client.1.smithi150.stdout:1/464: read d1/f6c [2496132,95861] 0 2024-03-22T10:42:56.610 INFO:tasks.workunit.client.1.smithi150.stdout:1/465: rename d1/da/f43 to d1/da/d2a/d65/d78/f98 0 2024-03-22T10:42:56.611 INFO:tasks.workunit.client.0.smithi115.stdout:1/952: dwrite d0/d8/de/fa1 [0,4194304] 0 2024-03-22T10:42:56.612 INFO:tasks.workunit.client.1.smithi150.stdout:1/466: chown d1/d19/d50/d51/f6b 6256 1 2024-03-22T10:42:56.613 INFO:tasks.workunit.client.0.smithi115.stdout:1/953: write d0/d12/f1c [276050,24530] 0 2024-03-22T10:42:56.629 INFO:tasks.workunit.client.0.smithi115.stdout:7/866: sync 2024-03-22T10:42:56.633 INFO:tasks.workunit.client.0.smithi115.stdout:7/867: unlink d2/d7/d12/d10/d19/d42/d46/d94/f11d 0 2024-03-22T10:42:56.639 INFO:tasks.workunit.client.0.smithi115.stdout:7/868: rename d2/d7/d12/d10/d19/fad to d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/f129 0 2024-03-22T10:42:56.642 INFO:tasks.workunit.client.0.smithi115.stdout:7/869: stat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/ffa 0 2024-03-22T10:42:56.721 INFO:tasks.workunit.client.1.smithi150.stdout:3/483: sync 2024-03-22T10:42:56.729 INFO:tasks.workunit.client.1.smithi150.stdout:3/484: unlink d5/d22/d2f/d38/d3e/d72/l91 0 2024-03-22T10:42:56.748 INFO:tasks.workunit.client.1.smithi150.stdout:3/485: dwrite d5/d53/d7e/d95/fa1 [0,4194304] 0 2024-03-22T10:42:56.749 INFO:tasks.workunit.client.0.smithi115.stdout:2/835: truncate da/d25/d3d/d7e/fa2 2860452 0 2024-03-22T10:42:56.750 INFO:tasks.workunit.client.1.smithi150.stdout:7/525: dread d2/d5/d7/f2b [0,4194304] 0 2024-03-22T10:42:56.752 INFO:tasks.workunit.client.1.smithi150.stdout:8/539: write d0/f5a [1093027,118757] 0 2024-03-22T10:42:56.753 INFO:tasks.workunit.client.0.smithi115.stdout:2/836: chown da/d1c/d67/c81 506574 1 2024-03-22T10:42:56.754 INFO:tasks.workunit.client.1.smithi150.stdout:8/540: read - d0/d2/d7e/fa9 zero size 2024-03-22T10:42:56.755 INFO:tasks.workunit.client.1.smithi150.stdout:7/526: symlink d2/d5/d7/lb0 0 2024-03-22T10:42:56.759 INFO:tasks.workunit.client.0.smithi115.stdout:4/891: write d1/d66/d106/d35/d40/f48 [229895,16363] 0 2024-03-22T10:42:56.763 INFO:tasks.workunit.client.1.smithi150.stdout:7/527: creat d2/d5/d7/d8/d57/d54/d91/dad/fb1 x:0 0 0 2024-03-22T10:42:56.765 INFO:tasks.workunit.client.0.smithi115.stdout:9/889: dwrite d6/d7/d6a/d6f/df0/ffd [0,4194304] 0 2024-03-22T10:42:56.766 INFO:tasks.workunit.client.1.smithi150.stdout:5/430: write d0/f6 [2320159,84812] 0 2024-03-22T10:42:56.767 INFO:tasks.workunit.client.0.smithi115.stdout:0/857: write d3/d4/d14/d64/fab [859086,105387] 0 2024-03-22T10:42:56.768 INFO:tasks.workunit.client.0.smithi115.stdout:9/890: chown d6/d7/d48/d70 10616 1 2024-03-22T10:42:56.768 INFO:tasks.workunit.client.1.smithi150.stdout:8/541: truncate d0/d2/d19/f47 408364 0 2024-03-22T10:42:56.771 INFO:tasks.workunit.client.1.smithi150.stdout:5/431: fsync d0/d8/dc/de/d14/d2e/f3e 0 2024-03-22T10:42:56.772 INFO:tasks.workunit.client.1.smithi150.stdout:7/528: mknod d2/d5/d7/d9/db/d9f/d45/cb2 0 2024-03-22T10:42:56.774 INFO:tasks.workunit.client.0.smithi115.stdout:3/937: write d3/d16/f58 [4134929,76797] 0 2024-03-22T10:42:56.775 INFO:tasks.workunit.client.1.smithi150.stdout:2/554: dread d12/f26 [0,4194304] 0 2024-03-22T10:42:56.776 INFO:tasks.workunit.client.1.smithi150.stdout:7/529: read d2/d5/f7c [57506,50293] 0 2024-03-22T10:42:56.776 INFO:tasks.workunit.client.1.smithi150.stdout:8/542: truncate d0/d2/d19/d29/d31/fae 421700 0 2024-03-22T10:42:56.776 INFO:tasks.workunit.client.0.smithi115.stdout:5/932: creat d2/d15/d1b/d7c/d38/d43/d49/d81/dbd/f130 x:0 0 0 2024-03-22T10:42:56.776 INFO:tasks.workunit.client.0.smithi115.stdout:3/938: fsync d3/d36/d41/d85/f11d 0 2024-03-22T10:42:56.777 INFO:tasks.workunit.client.0.smithi115.stdout:2/837: dwrite da/d25/d2a/f72 [0,4194304] 0 2024-03-22T10:42:56.778 INFO:tasks.workunit.client.0.smithi115.stdout:6/827: sync 2024-03-22T10:42:56.779 INFO:tasks.workunit.client.0.smithi115.stdout:2/838: chown da/d25/d3d/f2c 2 1 2024-03-22T10:42:56.779 INFO:tasks.workunit.client.0.smithi115.stdout:0/858: read d3/d7/f3b [3354535,54832] 0 2024-03-22T10:42:56.780 INFO:tasks.workunit.client.0.smithi115.stdout:3/939: write d3/d82/d52/f56 [3133286,79065] 0 2024-03-22T10:42:56.782 INFO:tasks.workunit.client.1.smithi150.stdout:1/467: stat d1/da/d15/d18/c20 0 2024-03-22T10:42:56.783 INFO:tasks.workunit.client.1.smithi150.stdout:5/432: rename d0/l9 to d0/d33/l7f 0 2024-03-22T10:42:56.784 INFO:tasks.workunit.client.0.smithi115.stdout:6/828: write d4/d11/d12/d1c/d8a/f105 [118850,97861] 0 2024-03-22T10:42:56.786 INFO:tasks.workunit.client.0.smithi115.stdout:5/933: dread d2/df/d13/d8b/d93/da6/fe0 [4194304,4194304] 0 2024-03-22T10:42:56.786 INFO:tasks.workunit.client.1.smithi150.stdout:3/486: write d5/f25 [7496287,32198] 0 2024-03-22T10:42:56.787 INFO:tasks.workunit.client.0.smithi115.stdout:3/940: chown d3/d6d/db4/d107/db3/l10e 148764 1 2024-03-22T10:42:56.790 INFO:tasks.workunit.client.1.smithi150.stdout:3/487: write d5/da/d78/d86/f93 [149257,86241] 0 2024-03-22T10:42:56.791 INFO:tasks.workunit.client.0.smithi115.stdout:0/859: creat d3/df0/dca/dce/f115 x:0 0 0 2024-03-22T10:42:56.792 INFO:tasks.workunit.client.1.smithi150.stdout:1/468: unlink d1/da/d15/c5e 0 2024-03-22T10:42:56.794 INFO:tasks.workunit.client.0.smithi115.stdout:1/954: dread d0/d12/d17/d37/d9e/de9/f90 [0,4194304] 0 2024-03-22T10:42:56.796 INFO:tasks.workunit.client.1.smithi150.stdout:2/555: rename d12/d19/l20 to d12/d48/da9/d61/lb0 0 2024-03-22T10:42:56.800 INFO:tasks.workunit.client.1.smithi150.stdout:1/469: mkdir d1/d19/d1a/d99 0 2024-03-22T10:42:56.800 INFO:tasks.workunit.client.1.smithi150.stdout:3/488: mknod d5/d22/d2f/d38/d3e/d5b/ca4 0 2024-03-22T10:42:56.802 INFO:tasks.workunit.client.0.smithi115.stdout:5/934: creat d2/d15/d1b/d7c/d38/d43/d49/d81/d83/f131 x:0 0 0 2024-03-22T10:42:56.804 INFO:tasks.workunit.client.1.smithi150.stdout:1/470: unlink d1/l63 0 2024-03-22T10:42:56.806 INFO:tasks.workunit.client.0.smithi115.stdout:2/839: rename da/d25/d3d/d57/l91 to da/lfe 0 2024-03-22T10:42:56.808 INFO:tasks.workunit.client.1.smithi150.stdout:3/489: symlink d5/d22/d2f/d38/d59/la5 0 2024-03-22T10:42:56.809 INFO:tasks.workunit.client.0.smithi115.stdout:1/955: creat d0/d8/de/d1a/d91/f14d x:0 0 0 2024-03-22T10:42:56.810 INFO:tasks.workunit.client.0.smithi115.stdout:4/892: truncate d1/d66/d82/feb 288047 0 2024-03-22T10:42:56.812 INFO:tasks.workunit.client.0.smithi115.stdout:5/935: symlink d2/d15/d1b/d7c/d38/d43/d49/d81/l132 0 2024-03-22T10:42:56.813 INFO:tasks.workunit.client.0.smithi115.stdout:6/829: creat d4/d13/d33/f10d x:0 0 0 2024-03-22T10:42:56.815 INFO:tasks.workunit.client.1.smithi150.stdout:5/433: truncate d0/f2b 3503770 0 2024-03-22T10:42:56.815 INFO:tasks.workunit.client.0.smithi115.stdout:1/956: truncate d0/d12/d17/d37/d5c/f8c 1220092 0 2024-03-22T10:42:56.816 INFO:tasks.workunit.client.0.smithi115.stdout:5/936: write d2/df/d13/fbb [2012553,28851] 0 2024-03-22T10:42:56.817 INFO:tasks.workunit.client.0.smithi115.stdout:3/941: write d3/d6d/f71 [1869294,81952] 0 2024-03-22T10:42:56.822 INFO:tasks.workunit.client.1.smithi150.stdout:5/434: creat d0/d8/dc/de/d14/d2e/d46/d4d/f80 x:0 0 0 2024-03-22T10:42:56.823 INFO:tasks.workunit.client.0.smithi115.stdout:2/840: unlink da/d25/d3d/ce5 0 2024-03-22T10:42:56.824 INFO:tasks.workunit.client.1.smithi150.stdout:7/530: dwrite d2/d5/d7/d8/d57/d4a/f33 [0,4194304] 0 2024-03-22T10:42:56.826 INFO:tasks.workunit.client.0.smithi115.stdout:6/830: creat d4/d11/d12/d1c/d8a/d98/d9f/f10e x:0 0 0 2024-03-22T10:42:56.826 INFO:tasks.workunit.client.0.smithi115.stdout:0/860: getdents d3/da 0 2024-03-22T10:42:56.827 INFO:tasks.workunit.client.0.smithi115.stdout:1/957: mkdir d0/d8/de/d14e 0 2024-03-22T10:42:56.828 INFO:tasks.workunit.client.0.smithi115.stdout:5/937: dread d2/df/d13/d8b/d93/da6/fa8 [0,4194304] 0 2024-03-22T10:42:56.831 INFO:tasks.workunit.client.1.smithi150.stdout:5/435: chown d0/d8/dc/d1d/d53/f7d 11183760 1 2024-03-22T10:42:56.834 INFO:tasks.workunit.client.1.smithi150.stdout:8/543: dread d0/d2/d7e/f82 [0,4194304] 0 2024-03-22T10:42:56.835 INFO:tasks.workunit.client.0.smithi115.stdout:9/891: dread d6/d5e/f60 [0,4194304] 0 2024-03-22T10:42:56.835 INFO:tasks.workunit.client.0.smithi115.stdout:1/958: mknod d0/d8/d4d/d56/db6/ddc/d82/c14f 0 2024-03-22T10:42:56.836 INFO:tasks.workunit.client.1.smithi150.stdout:2/556: dread d12/d48/f4c [0,4194304] 0 2024-03-22T10:42:56.837 INFO:tasks.workunit.client.0.smithi115.stdout:0/861: creat d3/df0/f116 x:0 0 0 2024-03-22T10:42:56.837 INFO:tasks.workunit.client.0.smithi115.stdout:4/893: dwrite d1/d30/d4c/d86/fac [0,4194304] 0 2024-03-22T10:42:56.838 INFO:tasks.workunit.client.1.smithi150.stdout:1/471: dread d1/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:42:56.839 INFO:tasks.workunit.client.1.smithi150.stdout:5/436: write d0/d8/dc/de/d14/d2e/f54 [2463491,121249] 0 2024-03-22T10:42:56.840 INFO:tasks.workunit.client.1.smithi150.stdout:9/529: sync 2024-03-22T10:42:56.840 INFO:tasks.workunit.client.1.smithi150.stdout:3/490: dwrite d5/da/f63 [0,4194304] 0 2024-03-22T10:42:56.842 INFO:tasks.workunit.client.0.smithi115.stdout:7/870: sync 2024-03-22T10:42:56.844 INFO:tasks.workunit.client.0.smithi115.stdout:5/938: creat d2/df/d13/d8b/d93/f133 x:0 0 0 2024-03-22T10:42:56.845 INFO:tasks.workunit.client.0.smithi115.stdout:1/959: truncate d0/d12/d17/d37/d9e/de9/f12c 971047 0 2024-03-22T10:42:56.846 INFO:tasks.workunit.client.0.smithi115.stdout:7/871: write d2/d7/d12/fb8 [793805,113666] 0 2024-03-22T10:42:56.847 INFO:tasks.workunit.client.1.smithi150.stdout:2/557: creat d12/d48/da9/d61/d98/fb1 x:0 0 0 2024-03-22T10:42:56.847 INFO:tasks.workunit.client.0.smithi115.stdout:9/892: unlink d6/d7/d48/df4/dfa/l112 0 2024-03-22T10:42:56.847 INFO:tasks.workunit.client.1.smithi150.stdout:8/544: rmdir d0/d2/d19/d29/d2e 39 2024-03-22T10:42:56.848 INFO:tasks.workunit.client.1.smithi150.stdout:3/491: write d5/d22/d37/f79 [49045,110359] 0 2024-03-22T10:42:56.850 INFO:tasks.workunit.client.1.smithi150.stdout:9/530: mknod d1/d8/de/d73/cbe 0 2024-03-22T10:42:56.850 INFO:tasks.workunit.client.0.smithi115.stdout:1/960: stat d0/d12/d17/d37/d119/d139/d25/d11a/f134 0 2024-03-22T10:42:56.853 INFO:tasks.workunit.client.0.smithi115.stdout:1/961: write d0/d12/d17/f80 [9339699,121400] 0 2024-03-22T10:42:56.853 INFO:tasks.workunit.client.1.smithi150.stdout:8/545: fsync d0/d2/d19/d29/d88/f9a 0 2024-03-22T10:42:56.853 INFO:tasks.workunit.client.1.smithi150.stdout:5/437: read d0/d8/dc/de/d14/f16 [4132860,69142] 0 2024-03-22T10:42:56.854 INFO:tasks.workunit.client.0.smithi115.stdout:1/962: stat d0/d22 0 2024-03-22T10:42:56.856 INFO:tasks.workunit.client.0.smithi115.stdout:4/894: creat d1/d66/d82/da1/d110/f130 x:0 0 0 2024-03-22T10:42:56.856 INFO:tasks.workunit.client.0.smithi115.stdout:5/939: creat d2/df/d13/d8b/d93/da6/dbc/d50/f134 x:0 0 0 2024-03-22T10:42:56.856 INFO:tasks.workunit.client.1.smithi150.stdout:3/492: fsync d5/f35 0 2024-03-22T10:42:56.859 INFO:tasks.workunit.client.0.smithi115.stdout:0/862: creat d3/da/d4c/f117 x:0 0 0 2024-03-22T10:42:56.859 INFO:tasks.workunit.client.1.smithi150.stdout:1/472: mknod d1/da/d2a/d4e/c9a 0 2024-03-22T10:42:56.859 INFO:tasks.workunit.client.0.smithi115.stdout:0/863: chown d3 81970732 1 2024-03-22T10:42:56.860 INFO:tasks.workunit.client.1.smithi150.stdout:7/531: dwrite d2/d5/d7/d9/db/d6c/fa1 [4194304,4194304] 0 2024-03-22T10:42:56.861 INFO:tasks.workunit.client.0.smithi115.stdout:7/872: symlink d2/d62/d126/l12a 0 2024-03-22T10:42:56.861 INFO:tasks.workunit.client.0.smithi115.stdout:5/940: dread - d2/df/d13/d8b/d93/da6/dbc/d50/db1/f12f zero size 2024-03-22T10:42:56.862 INFO:tasks.workunit.client.0.smithi115.stdout:3/942: truncate d3/d16/d29/f32 3339104 0 2024-03-22T10:42:56.865 INFO:tasks.workunit.client.1.smithi150.stdout:2/558: rename d12/d48/da9/d61/d98/fb1 to d12/d35/fb2 0 2024-03-22T10:42:56.865 INFO:tasks.workunit.client.1.smithi150.stdout:8/546: rmdir d0/d5d 39 2024-03-22T10:42:56.867 INFO:tasks.workunit.client.0.smithi115.stdout:2/841: write da/d25/d3d/f2c [3441111,71800] 0 2024-03-22T10:42:56.867 INFO:tasks.workunit.client.0.smithi115.stdout:0/864: write d3/d7/d6b/d94/fe4 [298708,81258] 0 2024-03-22T10:42:56.867 INFO:tasks.workunit.client.0.smithi115.stdout:3/943: fsync d3/d19/d2b/f11e 0 2024-03-22T10:42:56.868 INFO:tasks.workunit.client.1.smithi150.stdout:9/531: creat d1/d8/de/df/da2/fbf x:0 0 0 2024-03-22T10:42:56.870 INFO:tasks.workunit.client.0.smithi115.stdout:4/895: rename d1/dbb/dc6/dd8/df7/d10f to d1/dbb/dc6/dd8/df7/d10f/d131 22 2024-03-22T10:42:56.871 INFO:tasks.workunit.client.0.smithi115.stdout:3/944: fdatasync d3/d36/d41/f48 0 2024-03-22T10:42:56.872 INFO:tasks.workunit.client.0.smithi115.stdout:4/896: stat d1/d30/d4c/d86/c87 0 2024-03-22T10:42:56.872 INFO:tasks.workunit.client.1.smithi150.stdout:5/438: symlink d0/d8/dc/de/d14/d4f/d47/l81 0 2024-03-22T10:42:56.875 INFO:tasks.workunit.client.1.smithi150.stdout:7/532: rmdir d2/d5/d7/d8/d57/d54/d91 39 2024-03-22T10:42:56.876 INFO:tasks.workunit.client.0.smithi115.stdout:0/865: mkdir d3/df0/dca/dce/d118 0 2024-03-22T10:42:56.879 INFO:tasks.workunit.client.1.smithi150.stdout:9/532: mknod d1/d8/de/d73/cc0 0 2024-03-22T10:42:56.886 INFO:tasks.workunit.client.0.smithi115.stdout:0/866: symlink d3/da/l119 0 2024-03-22T10:42:56.887 INFO:tasks.workunit.client.1.smithi150.stdout:0/518: sync 2024-03-22T10:42:56.887 INFO:tasks.workunit.client.1.smithi150.stdout:6/486: sync 2024-03-22T10:42:56.887 INFO:tasks.workunit.client.1.smithi150.stdout:4/536: sync 2024-03-22T10:42:56.887 INFO:tasks.workunit.client.1.smithi150.stdout:2/559: link d12/d48/f4c d12/d21/fb3 0 2024-03-22T10:42:56.888 INFO:tasks.workunit.client.0.smithi115.stdout:2/842: truncate da/d25/d3d/d7e/fa2 1156679 0 2024-03-22T10:42:56.894 INFO:tasks.workunit.client.1.smithi150.stdout:0/519: readlink d0/d1/d84/d86/d42/l60 0 2024-03-22T10:42:56.894 INFO:tasks.workunit.client.1.smithi150.stdout:1/473: getdents d1/d19/d50/d51 0 2024-03-22T10:42:56.894 INFO:tasks.workunit.client.1.smithi150.stdout:4/537: readlink d0/d1b/d4d/d6c/l5a 0 2024-03-22T10:42:56.894 INFO:tasks.workunit.client.1.smithi150.stdout:2/560: fdatasync d12/d48/da9/d67/d3c/f7f 0 2024-03-22T10:42:56.895 INFO:tasks.workunit.client.0.smithi115.stdout:5/941: rename d2/d15/d1b/d7c/d38/d43/d49/d81 to d2/d15/d1b/d7c/d38/d135 0 2024-03-22T10:42:56.898 INFO:tasks.workunit.client.0.smithi115.stdout:5/942: mkdir d2/df/d13/d8b/d93/da6/dbc/d50/db1/d136 0 2024-03-22T10:42:56.903 INFO:tasks.workunit.client.1.smithi150.stdout:5/439: write d0/f3 [1217305,33170] 0 2024-03-22T10:42:56.904 INFO:tasks.workunit.client.1.smithi150.stdout:2/561: unlink d12/f13 0 2024-03-22T10:42:56.905 INFO:tasks.workunit.client.1.smithi150.stdout:7/533: dwrite d2/d5/d7/d8/d57/d4a/f22 [0,4194304] 0 2024-03-22T10:42:56.911 INFO:tasks.workunit.client.1.smithi150.stdout:6/487: creat db/dd/faa x:0 0 0 2024-03-22T10:42:56.913 INFO:tasks.workunit.client.0.smithi115.stdout:0/867: chown d3/da/d4c/d69/d6a/lcf 1443 1 2024-03-22T10:42:56.914 INFO:tasks.workunit.client.1.smithi150.stdout:9/533: dwrite d1/d8/de/df/d24/f6b [0,4194304] 0 2024-03-22T10:42:56.915 INFO:tasks.workunit.client.0.smithi115.stdout:2/843: dread da/d25/d3d/d7e/d90/dbf/d54/ff1 [0,4194304] 0 2024-03-22T10:42:56.915 INFO:tasks.workunit.client.0.smithi115.stdout:1/963: dwrite d0/d8/d4d/d56/db6/f101 [0,4194304] 0 2024-03-22T10:42:56.919 INFO:tasks.workunit.client.0.smithi115.stdout:3/945: dread d3/f8e [0,4194304] 0 2024-03-22T10:42:56.924 INFO:tasks.workunit.client.0.smithi115.stdout:1/964: chown d0/d8/de/d1a/d91/fd2 483267045 1 2024-03-22T10:42:56.926 INFO:tasks.workunit.client.0.smithi115.stdout:7/873: dwrite d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f91 [0,4194304] 0 2024-03-22T10:42:56.927 INFO:tasks.workunit.client.1.smithi150.stdout:8/547: dread d0/f1 [4194304,4194304] 0 2024-03-22T10:42:56.930 INFO:tasks.workunit.client.0.smithi115.stdout:2/844: symlink da/d25/d3d/d57/d7d/da1/dba/ddf/lff 0 2024-03-22T10:42:56.933 INFO:tasks.workunit.client.1.smithi150.stdout:5/440: link d0/f11 d0/d8/dc/de/f82 0 2024-03-22T10:42:56.933 INFO:tasks.workunit.client.0.smithi115.stdout:1/965: creat d0/d12/d17/d37/d9e/de9/f150 x:0 0 0 2024-03-22T10:42:56.935 INFO:tasks.workunit.client.0.smithi115.stdout:3/946: read d3/fb2 [1292707,116158] 0 2024-03-22T10:42:56.936 INFO:tasks.workunit.client.1.smithi150.stdout:6/488: creat db/dd/d15/d30/fab x:0 0 0 2024-03-22T10:42:56.936 INFO:tasks.workunit.client.1.smithi150.stdout:1/474: dwrite d1/da/d15/d4f/f60 [0,4194304] 0 2024-03-22T10:42:56.936 INFO:tasks.workunit.client.1.smithi150.stdout:3/493: dwrite d5/f10 [0,4194304] 0 2024-03-22T10:42:56.940 INFO:tasks.workunit.client.0.smithi115.stdout:1/966: write d0/d8/de/d1a/d91/feb [4729954,81513] 0 2024-03-22T10:42:56.941 INFO:tasks.workunit.client.0.smithi115.stdout:0/868: dwrite d3/d7/d1b/d35/f79 [0,4194304] 0 2024-03-22T10:42:56.941 INFO:tasks.workunit.client.0.smithi115.stdout:5/943: dwrite d2/df/d13/d8b/d93/da6/fe0 [0,4194304] 0 2024-03-22T10:42:56.944 INFO:tasks.workunit.client.0.smithi115.stdout:3/947: chown d3/d89/dca/c9c 4822307 1 2024-03-22T10:42:56.947 INFO:tasks.workunit.client.1.smithi150.stdout:7/534: creat d2/d5/d7/d8/d57/d54/d91/fb3 x:0 0 0 2024-03-22T10:42:56.948 INFO:tasks.workunit.client.0.smithi115.stdout:0/869: rmdir d3/d7/d1c 39 2024-03-22T10:42:56.949 INFO:tasks.workunit.client.0.smithi115.stdout:3/948: mknod d3/d82/d52/c137 0 2024-03-22T10:42:56.950 INFO:tasks.workunit.client.1.smithi150.stdout:8/548: mknod d0/d3c/d71/d7a/d89/db5/cba 0 2024-03-22T10:42:56.957 INFO:tasks.workunit.client.0.smithi115.stdout:5/944: creat d2/df/d13/d8b/f137 x:0 0 0 2024-03-22T10:42:56.959 INFO:tasks.workunit.client.1.smithi150.stdout:7/535: readlink d2/d5/d7/l48 0 2024-03-22T10:42:56.967 INFO:tasks.workunit.client.0.smithi115.stdout:5/945: read d2/df/d13/d8b/d93/da6/fcf [596683,15711] 0 2024-03-22T10:42:56.967 INFO:tasks.workunit.client.0.smithi115.stdout:7/874: dwrite d2/d7/d12/d10/d19/fc1 [0,4194304] 0 2024-03-22T10:42:56.968 INFO:tasks.workunit.client.0.smithi115.stdout:3/949: creat d3/d82/d54/de7/d104/f138 x:0 0 0 2024-03-22T10:42:56.968 INFO:tasks.workunit.client.0.smithi115.stdout:1/967: link d0/d12/d17/d37/d9e/da0/dff/f10d d0/d8/de/d1a/d120/f151 0 2024-03-22T10:42:56.968 INFO:tasks.workunit.client.1.smithi150.stdout:3/494: mkdir d5/d22/d2f/d38/d3e/d5b/da6 0 2024-03-22T10:42:56.969 INFO:tasks.workunit.client.1.smithi150.stdout:9/534: dwrite d1/d19/f79 [0,4194304] 0 2024-03-22T10:42:56.969 INFO:tasks.workunit.client.1.smithi150.stdout:4/538: write d0/d47/f96 [2485,3780] 0 2024-03-22T10:42:56.970 INFO:tasks.workunit.client.1.smithi150.stdout:7/536: fdatasync d2/d5/d7/d9/db/d9f/d45/f7d 0 2024-03-22T10:42:56.973 INFO:tasks.workunit.client.1.smithi150.stdout:3/495: write d5/da/f33 [221600,117963] 0 2024-03-22T10:42:56.979 INFO:tasks.workunit.client.1.smithi150.stdout:2/562: dread d12/d48/da9/d67/d3c/f7f [0,4194304] 0 2024-03-22T10:42:56.981 INFO:tasks.workunit.client.1.smithi150.stdout:0/520: dread d0/d1/d84/d86/d11/d66/d35/f62 [0,4194304] 0 2024-03-22T10:42:56.983 INFO:tasks.workunit.client.1.smithi150.stdout:7/537: chown d2/d5/d7/d8/d46/f66 36951374 1 2024-03-22T10:42:56.987 INFO:tasks.workunit.client.0.smithi115.stdout:3/950: dwrite d3/d19/dd8/ddb/f135 [0,4194304] 0 2024-03-22T10:42:56.987 INFO:tasks.workunit.client.1.smithi150.stdout:5/441: dwrite d0/d8/dc/de/d14/d2e/f62 [0,4194304] 0 2024-03-22T10:42:56.993 INFO:tasks.workunit.client.0.smithi115.stdout:7/875: dwrite d2/d7/d12/d10/d19/d42/d46/f115 [0,4194304] 0 2024-03-22T10:42:56.994 INFO:tasks.workunit.client.1.smithi150.stdout:1/475: link d1/d19/d50/d6e/l7e d1/d19/d50/d51/l9b 0 2024-03-22T10:42:56.995 INFO:tasks.workunit.client.1.smithi150.stdout:8/549: rename d0/c7c to d0/d2/d19/d29/d2f/cbb 0 2024-03-22T10:42:56.996 INFO:tasks.workunit.client.1.smithi150.stdout:7/538: truncate d2/d5/d7/d8/d57/d54/d91/fb3 323234 0 2024-03-22T10:42:57.002 INFO:tasks.workunit.client.1.smithi150.stdout:9/535: truncate d1/d8/de/df/da2/d8c/f8b 626335 0 2024-03-22T10:42:57.003 INFO:tasks.workunit.client.0.smithi115.stdout:3/951: dread d3/d19/d2b/d49/fbd [0,4194304] 0 2024-03-22T10:42:57.005 INFO:tasks.workunit.client.1.smithi150.stdout:9/536: chown d1/d49/f31 17232238 1 2024-03-22T10:42:57.009 INFO:tasks.workunit.client.1.smithi150.stdout:0/521: rename d0/d1/d84/d86/d42/d10/d38/d52/c9a to d0/d1/d84/d86/d11/d66/ca2 0 2024-03-22T10:42:57.012 INFO:tasks.workunit.client.1.smithi150.stdout:3/496: dwrite d5/da/f40 [0,4194304] 0 2024-03-22T10:42:57.012 INFO:tasks.workunit.client.1.smithi150.stdout:8/550: dread - d0/d2/d19/d29/d2e/d70/fb8 zero size 2024-03-22T10:42:57.014 INFO:tasks.workunit.client.1.smithi150.stdout:1/476: creat d1/d19/d50/d51/f9c x:0 0 0 2024-03-22T10:42:57.020 INFO:tasks.workunit.client.1.smithi150.stdout:3/497: truncate d5/da/d78/d86/d81/f9b 351006 0 2024-03-22T10:42:57.031 INFO:tasks.workunit.client.1.smithi150.stdout:3/498: mknod d5/d22/ca7 0 2024-03-22T10:42:57.037 INFO:tasks.workunit.client.1.smithi150.stdout:3/499: creat d5/da/d9d/fa8 x:0 0 0 2024-03-22T10:42:57.042 INFO:tasks.workunit.client.0.smithi115.stdout:1/968: rename d0/d12/d17/d37/l94 to d0/d12/l152 0 2024-03-22T10:42:57.043 INFO:tasks.workunit.client.1.smithi150.stdout:3/500: symlink d5/d22/d2f/d9c/la9 0 2024-03-22T10:42:57.046 INFO:tasks.workunit.client.1.smithi150.stdout:8/551: dwrite d0/d2/d19/d29/d2f/df/f26 [0,4194304] 0 2024-03-22T10:42:57.049 INFO:tasks.workunit.client.0.smithi115.stdout:7/876: write d2/d7/d12/d10/d23/d33/f9b [651165,109296] 0 2024-03-22T10:42:57.051 INFO:tasks.workunit.client.0.smithi115.stdout:1/969: creat d0/d8/d4d/d56/db6/d11b/f153 x:0 0 0 2024-03-22T10:42:57.052 INFO:tasks.workunit.client.0.smithi115.stdout:3/952: chown d3/d19/f1d 113532856 1 2024-03-22T10:42:57.054 INFO:tasks.workunit.client.1.smithi150.stdout:8/552: getdents d0/d2/d19/d29/d2e 0 2024-03-22T10:42:57.058 INFO:tasks.workunit.client.1.smithi150.stdout:4/539: dwrite d0/d1b/f74 [0,4194304] 0 2024-03-22T10:42:57.061 INFO:tasks.workunit.client.1.smithi150.stdout:8/553: creat d0/d5d/fbc x:0 0 0 2024-03-22T10:42:57.062 INFO:tasks.workunit.client.0.smithi115.stdout:7/877: creat d2/d7/d12/d10/d19/f12b x:0 0 0 2024-03-22T10:42:57.062 INFO:tasks.workunit.client.0.smithi115.stdout:1/970: creat d0/d8/d4d/d56/db6/ddc/d13a/f154 x:0 0 0 2024-03-22T10:42:57.062 INFO:tasks.workunit.client.0.smithi115.stdout:3/953: creat d3/d16/d29/dc1/f139 x:0 0 0 2024-03-22T10:42:57.064 INFO:tasks.workunit.client.0.smithi115.stdout:7/878: read d2/d7/d12/d10/f96 [19413,60097] 0 2024-03-22T10:42:57.064 INFO:tasks.workunit.client.1.smithi150.stdout:9/537: rename d1/d8/de/df/da2/d8c to d1/d19/d7d/dc1 0 2024-03-22T10:42:57.066 INFO:tasks.workunit.client.0.smithi115.stdout:3/954: symlink d3/d19/dd8/ddb/l13a 0 2024-03-22T10:42:57.068 INFO:tasks.workunit.client.1.smithi150.stdout:7/539: write d2/d5/d7/d8/d57/d4a/f7f [148083,42524] 0 2024-03-22T10:42:57.074 INFO:tasks.workunit.client.0.smithi115.stdout:0/870: write d3/d52/f57 [4261179,105922] 0 2024-03-22T10:42:57.074 INFO:tasks.workunit.client.1.smithi150.stdout:0/522: rename d0/d1/d84/d86/d42/l60 to d0/d1/d84/d86/d42/d10/d38/d52/la3 0 2024-03-22T10:42:57.075 INFO:tasks.workunit.client.0.smithi115.stdout:5/946: write d2/d15/d1b/d7c/d38/fd2 [872705,36191] 0 2024-03-22T10:42:57.077 INFO:tasks.workunit.client.1.smithi150.stdout:7/540: getdents d2/d5/d7/d8/d57/d54/d91/daf 0 2024-03-22T10:42:57.077 INFO:tasks.workunit.client.1.smithi150.stdout:8/554: dread d0/d2/d19/d29/d2f/df/f45 [0,4194304] 0 2024-03-22T10:42:57.078 INFO:tasks.workunit.client.1.smithi150.stdout:3/501: write d5/da/d62/f8e [392729,54583] 0 2024-03-22T10:42:57.080 INFO:tasks.workunit.client.1.smithi150.stdout:4/540: rename d0/d1b/d4d/d6c/d61/d98/la3 to d0/d2f/laf 0 2024-03-22T10:42:57.084 INFO:tasks.workunit.client.0.smithi115.stdout:7/879: link d2/d7/d12/d10/d23/d33/l43 d2/d7/d12/d10/d23/d33/l12c 0 2024-03-22T10:42:57.084 INFO:tasks.workunit.client.0.smithi115.stdout:5/947: creat d2/df/d13/d8b/d93/da6/dbc/d50/d96/f138 x:0 0 0 2024-03-22T10:42:57.085 INFO:tasks.workunit.client.1.smithi150.stdout:0/523: mknod d0/d1/d84/d86/d11/d66/d35/d5f/ca4 0 2024-03-22T10:42:57.089 INFO:tasks.workunit.client.1.smithi150.stdout:1/477: dwrite d1/d19/d1a/f49 [0,4194304] 0 2024-03-22T10:42:57.089 INFO:tasks.workunit.client.1.smithi150.stdout:7/541: rename d2/d5/d7/d38/la5 to d2/d5/d7/d9/db/d9f/d96/lb4 0 2024-03-22T10:42:57.090 INFO:tasks.workunit.client.1.smithi150.stdout:4/541: mknod d0/d1b/d4d/d6c/d61/d98/cb0 0 2024-03-22T10:42:57.090 INFO:tasks.workunit.client.0.smithi115.stdout:1/971: dwrite d0/d8/d4d/d56/dbd/f10c [0,4194304] 0 2024-03-22T10:42:57.091 INFO:tasks.workunit.client.1.smithi150.stdout:8/555: mknod d0/d2/d19/d29/d2f/df/cbd 0 2024-03-22T10:42:57.093 INFO:tasks.workunit.client.1.smithi150.stdout:1/478: chown d1/f29 94005120 1 2024-03-22T10:42:57.098 INFO:tasks.workunit.client.1.smithi150.stdout:8/556: read d0/d2/d46/f4e [3867988,13171] 0 2024-03-22T10:42:57.098 INFO:tasks.workunit.client.1.smithi150.stdout:7/542: chown d2/d5/d7/d9/f62 44231 1 2024-03-22T10:42:57.099 INFO:tasks.workunit.client.1.smithi150.stdout:9/538: dwrite d1/d8/de/df/d20/d32/d75/f8d [0,4194304] 0 2024-03-22T10:42:57.100 INFO:tasks.workunit.client.1.smithi150.stdout:1/479: rename d1/d19/d1a/f46 to d1/d19/f9d 0 2024-03-22T10:42:57.101 INFO:tasks.workunit.client.1.smithi150.stdout:9/539: stat d1/d8/de/df/da2 0 2024-03-22T10:42:57.103 INFO:tasks.workunit.client.1.smithi150.stdout:4/542: dread - d0/d2f/d63/d7b/d9c/fa6 zero size 2024-03-22T10:42:57.104 INFO:tasks.workunit.client.1.smithi150.stdout:9/540: chown d1/d19/l21 127 1 2024-03-22T10:42:57.106 INFO:tasks.workunit.client.0.smithi115.stdout:1/972: creat d0/d22/d89/d141/f155 x:0 0 0 2024-03-22T10:42:57.106 INFO:tasks.workunit.client.0.smithi115.stdout:0/871: dwrite d3/d7/d1c/d6c/d9a/fd4 [0,4194304] 0 2024-03-22T10:42:57.111 INFO:tasks.workunit.client.1.smithi150.stdout:8/557: creat d0/d2/d19/d29/d37/fbe x:0 0 0 2024-03-22T10:42:57.111 INFO:tasks.workunit.client.1.smithi150.stdout:9/541: chown d1/d8/de/df/l4b 63719905 1 2024-03-22T10:42:57.113 INFO:tasks.workunit.client.1.smithi150.stdout:1/480: truncate d1/da/d15/d18/f3d 4831437 0 2024-03-22T10:42:57.113 INFO:tasks.workunit.client.0.smithi115.stdout:7/880: dwrite d2/d7/d12/f99 [0,4194304] 0 2024-03-22T10:42:57.114 INFO:tasks.workunit.client.0.smithi115.stdout:0/872: chown d3/d4/d14/d64/f72 12 1 2024-03-22T10:42:57.117 INFO:tasks.workunit.client.1.smithi150.stdout:7/543: getdents d2/d5/d7/d9 0 2024-03-22T10:42:57.118 INFO:tasks.workunit.client.1.smithi150.stdout:5/442: sync 2024-03-22T10:42:57.119 INFO:tasks.workunit.client.1.smithi150.stdout:2/563: sync 2024-03-22T10:42:57.119 INFO:tasks.workunit.client.1.smithi150.stdout:6/489: sync 2024-03-22T10:42:57.120 INFO:tasks.workunit.client.0.smithi115.stdout:3/955: dread d3/d19/d2b/d49/d91/f9f [0,4194304] 0 2024-03-22T10:42:57.121 INFO:tasks.workunit.client.0.smithi115.stdout:1/973: creat d0/d12/d17/d37/d119/d13d/f156 x:0 0 0 2024-03-22T10:42:57.123 INFO:tasks.workunit.client.1.smithi150.stdout:9/542: chown d1/d8/de/df/d20/d32/c92 409599553 1 2024-03-22T10:42:57.124 INFO:tasks.workunit.client.1.smithi150.stdout:4/543: dread d0/d32/d7c/d99/fab [0,4194304] 0 2024-03-22T10:42:57.124 INFO:tasks.workunit.client.0.smithi115.stdout:3/956: dread - d3/d16/d29/dc1/f139 zero size 2024-03-22T10:42:57.128 INFO:tasks.workunit.client.0.smithi115.stdout:0/873: symlink d3/d7/d1c/d6c/d9a/l11a 0 2024-03-22T10:42:57.129 INFO:tasks.workunit.client.1.smithi150.stdout:5/443: chown d0/d8/dc/de/d14/d2e/d46/f72 3210555 1 2024-03-22T10:42:57.130 INFO:tasks.workunit.client.0.smithi115.stdout:6/831: sync 2024-03-22T10:42:57.130 INFO:tasks.workunit.client.1.smithi150.stdout:1/481: creat d1/d19/f9e x:0 0 0 2024-03-22T10:42:57.131 INFO:tasks.workunit.client.0.smithi115.stdout:1/974: mkdir d0/d8/de/d1a/d91/dc7/d125/d157 0 2024-03-22T10:42:57.133 INFO:tasks.workunit.client.0.smithi115.stdout:3/957: rmdir d3/d82/d54/de7 39 2024-03-22T10:42:57.136 INFO:tasks.workunit.client.1.smithi150.stdout:5/444: chown d0/f10 12305375 1 2024-03-22T10:42:57.137 INFO:tasks.workunit.client.0.smithi115.stdout:3/958: chown d3/d19/dd8/f10a 207573583 1 2024-03-22T10:42:57.137 INFO:tasks.workunit.client.0.smithi115.stdout:6/832: truncate d4/d36/d48/d72/daa/db9/fd7 965809 0 2024-03-22T10:42:57.140 INFO:tasks.workunit.client.1.smithi150.stdout:5/445: read d0/f11 [1521522,15476] 0 2024-03-22T10:42:57.143 INFO:tasks.workunit.client.0.smithi115.stdout:1/975: creat d0/d12/d17/d37/d119/d139/dd7/f158 x:0 0 0 2024-03-22T10:42:57.145 INFO:tasks.workunit.client.1.smithi150.stdout:5/446: stat d0/d8/dc/de/d14/d2e/d46/f72 0 2024-03-22T10:42:57.146 INFO:tasks.workunit.client.0.smithi115.stdout:5/948: write d2/d15/d1b/d7c/d84/d89/fb8 [3854447,67759] 0 2024-03-22T10:42:57.146 INFO:tasks.workunit.client.1.smithi150.stdout:9/543: creat d1/d19/d7d/fc2 x:0 0 0 2024-03-22T10:42:57.148 INFO:tasks.workunit.client.0.smithi115.stdout:6/833: mknod d4/d11/d12/d1c/d8a/d98/d9f/df0/df4/c10f 0 2024-03-22T10:42:57.148 INFO:tasks.workunit.client.1.smithi150.stdout:5/447: chown d0/f6 101010 1 2024-03-22T10:42:57.148 INFO:tasks.workunit.client.0.smithi115.stdout:7/881: dwrite d2/d7/d12/d10/d19/f110 [0,4194304] 0 2024-03-22T10:42:57.150 INFO:tasks.workunit.client.0.smithi115.stdout:1/976: write d0/d12/d17/d37/d119/d139/d25/d11a/f136 [240501,93855] 0 2024-03-22T10:42:57.152 INFO:tasks.workunit.client.1.smithi150.stdout:2/564: mkdir d12/d48/da9/d67/db4 0 2024-03-22T10:42:57.155 INFO:tasks.workunit.client.0.smithi115.stdout:1/977: chown d0/d8/d4d/f138 568582 1 2024-03-22T10:42:57.156 INFO:tasks.workunit.client.0.smithi115.stdout:6/834: mknod d4/d36/d48/d7a/c110 0 2024-03-22T10:42:57.157 INFO:tasks.workunit.client.1.smithi150.stdout:3/502: dread d5/d22/d2f/d38/f42 [4194304,4194304] 0 2024-03-22T10:42:57.158 INFO:tasks.workunit.client.0.smithi115.stdout:2/845: sync 2024-03-22T10:42:57.158 INFO:tasks.workunit.client.0.smithi115.stdout:4/897: sync 2024-03-22T10:42:57.158 INFO:tasks.workunit.client.0.smithi115.stdout:9/893: sync 2024-03-22T10:42:57.159 INFO:tasks.workunit.client.0.smithi115.stdout:7/882: mkdir d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/d12d 0 2024-03-22T10:42:57.161 INFO:tasks.workunit.client.0.smithi115.stdout:1/978: chown d0/d8/de/d1a/d120 12657912 1 2024-03-22T10:42:57.162 INFO:tasks.workunit.client.0.smithi115.stdout:6/835: mknod d4/d11/d12/d1c/d8a/c111 0 2024-03-22T10:42:57.163 INFO:tasks.workunit.client.0.smithi115.stdout:7/883: stat d2/d7/d12/d10/d19/d42/f85 0 2024-03-22T10:42:57.163 INFO:tasks.workunit.client.1.smithi150.stdout:6/490: link db/dd/d15/d6a/f6b db/dd/d25/fac 0 2024-03-22T10:42:57.164 INFO:tasks.workunit.client.0.smithi115.stdout:1/979: chown d0/d8/l79 2368001 1 2024-03-22T10:42:57.164 INFO:tasks.workunit.client.1.smithi150.stdout:1/482: dwrite d1/d19/d50/f69 [0,4194304] 0 2024-03-22T10:42:57.165 INFO:tasks.workunit.client.0.smithi115.stdout:6/836: fsync d4/d36/d48/d72/f81 0 2024-03-22T10:42:57.165 INFO:tasks.workunit.client.0.smithi115.stdout:7/884: chown d2/d62/l8f 235067 1 2024-03-22T10:42:57.166 INFO:tasks.workunit.client.0.smithi115.stdout:9/894: rmdir d6/d7/d20/d22/d23 39 2024-03-22T10:42:57.168 INFO:tasks.workunit.client.0.smithi115.stdout:6/837: fdatasync d4/d11/d12/f4e 0 2024-03-22T10:42:57.168 INFO:tasks.workunit.client.1.smithi150.stdout:5/448: fdatasync d0/d8/dc/de/d14/f50 0 2024-03-22T10:42:57.173 INFO:tasks.workunit.client.0.smithi115.stdout:4/898: creat d1/d66/d106/d35/d40/d54/d5e/df3/f132 x:0 0 0 2024-03-22T10:42:57.179 INFO:tasks.workunit.client.0.smithi115.stdout:6/838: symlink d4/d36/l112 0 2024-03-22T10:42:57.180 INFO:tasks.workunit.client.1.smithi150.stdout:0/524: rmdir d0/d1/d84/d86/d42/d10/d38/d52/d98 0 2024-03-22T10:42:57.181 INFO:tasks.workunit.client.1.smithi150.stdout:8/558: truncate d0/d2/d46/f4e 1485460 0 2024-03-22T10:42:57.186 INFO:tasks.workunit.client.1.smithi150.stdout:1/483: symlink d1/d19/d50/d51/l9f 0 2024-03-22T10:42:57.186 INFO:tasks.workunit.client.1.smithi150.stdout:7/544: dread d2/d5/d7/d9/db/d9f/d96/f83 [0,4194304] 0 2024-03-22T10:42:57.191 INFO:tasks.workunit.client.1.smithi150.stdout:8/559: stat d0/c27 0 2024-03-22T10:42:57.192 INFO:tasks.workunit.client.0.smithi115.stdout:4/899: mknod d1/dbb/ddf/df9/c133 0 2024-03-22T10:42:57.193 INFO:tasks.workunit.client.0.smithi115.stdout:9/895: creat d6/d7/d20/f137 x:0 0 0 2024-03-22T10:42:57.194 INFO:tasks.workunit.client.1.smithi150.stdout:6/491: creat db/fad x:0 0 0 2024-03-22T10:42:57.198 INFO:tasks.workunit.client.0.smithi115.stdout:7/885: creat d2/d7/d12/d10/d19/d42/d76/f12e x:0 0 0 2024-03-22T10:42:57.201 INFO:tasks.workunit.client.0.smithi115.stdout:6/839: dwrite d4/d13/d33/fba [4194304,4194304] 0 2024-03-22T10:42:57.202 INFO:tasks.workunit.client.0.smithi115.stdout:9/896: creat d6/d7/d19/d4e/f138 x:0 0 0 2024-03-22T10:42:57.202 INFO:tasks.workunit.client.1.smithi150.stdout:9/544: write d1/d8/de/df/da2/f91 [457443,64444] 0 2024-03-22T10:42:57.202 INFO:tasks.workunit.client.1.smithi150.stdout:5/449: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:42:57.202 INFO:tasks.workunit.client.1.smithi150.stdout:0/525: fsync d0/d1/d84/d86/d42/f72 0 2024-03-22T10:42:57.207 INFO:tasks.workunit.client.1.smithi150.stdout:7/545: symlink d2/d5/d7/d9/db/lb5 0 2024-03-22T10:42:57.209 INFO:tasks.workunit.client.0.smithi115.stdout:0/874: dread d3/d7/d1b/f47 [0,4194304] 0 2024-03-22T10:42:57.210 INFO:tasks.workunit.client.1.smithi150.stdout:3/503: truncate d5/da/d62/f8e 423628 0 2024-03-22T10:42:57.211 INFO:tasks.workunit.client.0.smithi115.stdout:4/900: truncate d1/d66/d82/da1/fc9 368560 0 2024-03-22T10:42:57.211 INFO:tasks.workunit.client.1.smithi150.stdout:4/544: dwrite d0/d1b/f2e [4194304,4194304] 0 2024-03-22T10:42:57.212 INFO:tasks.workunit.client.0.smithi115.stdout:9/897: rmdir d6/d7/d30/d3b 39 2024-03-22T10:42:57.212 INFO:tasks.workunit.client.1.smithi150.stdout:3/504: chown d5/da/f34 53483708 1 2024-03-22T10:42:57.213 INFO:tasks.workunit.client.0.smithi115.stdout:5/949: dread d2/d15/d1b/d7c/d38/d43/d49/fa9 [0,4194304] 0 2024-03-22T10:42:57.213 INFO:tasks.workunit.client.1.smithi150.stdout:6/492: fdatasync db/dd/d82/d84/f87 0 2024-03-22T10:42:57.214 INFO:tasks.workunit.client.0.smithi115.stdout:7/886: stat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae 0 2024-03-22T10:42:57.214 INFO:tasks.workunit.client.1.smithi150.stdout:8/560: mkdir d0/d2/dbf 0 2024-03-22T10:42:57.215 INFO:tasks.workunit.client.0.smithi115.stdout:1/980: dread d0/d8/d4d/f5a [0,4194304] 0 2024-03-22T10:42:57.216 INFO:tasks.workunit.client.1.smithi150.stdout:7/546: fsync d2/d5/d7/d9/db/d9f/f6f 0 2024-03-22T10:42:57.217 INFO:tasks.workunit.client.1.smithi150.stdout:2/565: link d12/d48/da9/d97/d78/c7a d12/d19/cb5 0 2024-03-22T10:42:57.218 INFO:tasks.workunit.client.1.smithi150.stdout:9/545: mkdir d1/d8/de/df/d20/d32/d75/dc3 0 2024-03-22T10:42:57.219 INFO:tasks.workunit.client.1.smithi150.stdout:6/493: write db/dd/d15/d60/f53 [576316,10841] 0 2024-03-22T10:42:57.219 INFO:tasks.workunit.client.1.smithi150.stdout:1/484: dread d1/d19/d50/f69 [0,4194304] 0 2024-03-22T10:42:57.220 INFO:tasks.workunit.client.1.smithi150.stdout:7/547: readlink d2/d5/d7/d8/d57/l37 0 2024-03-22T10:42:57.221 INFO:tasks.workunit.client.0.smithi115.stdout:9/898: dread - d6/d7/d6a/fdf zero size 2024-03-22T10:42:57.222 INFO:tasks.workunit.client.1.smithi150.stdout:5/450: truncate d0/d8/dc/de/d14/d4f/d47/f6e 1035479 0 2024-03-22T10:42:57.223 INFO:tasks.workunit.client.1.smithi150.stdout:0/526: read d0/d1/d84/d86/d11/d66/d37/f3d [11086,7454] 0 2024-03-22T10:42:57.223 INFO:tasks.workunit.client.0.smithi115.stdout:5/950: readlink d2/d15/d1b/d7c/d38/d135/d83/l6a 0 2024-03-22T10:42:57.224 INFO:tasks.workunit.client.0.smithi115.stdout:3/959: dread d3/f9 [0,4194304] 0 2024-03-22T10:42:57.224 INFO:tasks.workunit.client.0.smithi115.stdout:6/840: rmdir d4/d11/d12/d1c/d8a/d98/d9f/df0/df4 39 2024-03-22T10:42:57.224 INFO:tasks.workunit.client.1.smithi150.stdout:3/505: rmdir d5/d22/d2f 39 2024-03-22T10:42:57.227 INFO:tasks.workunit.client.1.smithi150.stdout:4/545: rename d0/d9b to d0/d2f/d63/d7b/d9c/db1 0 2024-03-22T10:42:57.228 INFO:tasks.workunit.client.0.smithi115.stdout:7/887: unlink d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f91 0 2024-03-22T10:42:57.229 INFO:tasks.workunit.client.1.smithi150.stdout:9/546: unlink d1/c1a 0 2024-03-22T10:42:57.229 INFO:tasks.workunit.client.0.smithi115.stdout:3/960: rmdir d3 39 2024-03-22T10:42:57.234 INFO:tasks.workunit.client.1.smithi150.stdout:2/566: unlink d12/d48/f4c 0 2024-03-22T10:42:57.235 INFO:tasks.workunit.client.0.smithi115.stdout:4/901: dread d1/f57 [0,4194304] 0 2024-03-22T10:42:57.235 INFO:tasks.workunit.client.1.smithi150.stdout:0/527: write d0/d1/d84/d86/f28 [3506610,107768] 0 2024-03-22T10:42:57.236 INFO:tasks.workunit.client.1.smithi150.stdout:7/548: creat d2/d5/d7/d9/db/d6c/fb6 x:0 0 0 2024-03-22T10:42:57.236 INFO:tasks.workunit.client.1.smithi150.stdout:1/485: mkdir d1/d19/d50/d51/da0 0 2024-03-22T10:42:57.239 INFO:tasks.workunit.client.0.smithi115.stdout:2/846: dread da/d1c/f1f [0,4194304] 0 2024-03-22T10:42:57.242 INFO:tasks.workunit.client.1.smithi150.stdout:1/486: dread - d1/d19/d50/d6e/f88 zero size 2024-03-22T10:42:57.242 INFO:tasks.workunit.client.1.smithi150.stdout:4/546: write d0/f93 [2234896,113940] 0 2024-03-22T10:42:57.243 INFO:tasks.workunit.client.0.smithi115.stdout:5/951: creat d2/df/d13/d8b/d93/da6/dbc/d50/d96/f139 x:0 0 0 2024-03-22T10:42:57.244 INFO:tasks.workunit.client.1.smithi150.stdout:6/494: symlink db/dd/d15/d60/lae 0 2024-03-22T10:42:57.245 INFO:tasks.workunit.client.0.smithi115.stdout:2/847: readlink da/d25/d3d/d57/l9c 0 2024-03-22T10:42:57.250 INFO:tasks.workunit.client.0.smithi115.stdout:2/848: chown da/d25/d51/cbb 42582586 1 2024-03-22T10:42:57.251 INFO:tasks.workunit.client.0.smithi115.stdout:0/875: dwrite d3/d7/d1c/f102 [0,4194304] 0 2024-03-22T10:42:57.252 INFO:tasks.workunit.client.1.smithi150.stdout:7/549: write d2/d5/d7/d9/db/d9f/d96/f83 [3426424,18325] 0 2024-03-22T10:42:57.252 INFO:tasks.workunit.client.1.smithi150.stdout:3/506: truncate d5/d53/f71 3573465 0 2024-03-22T10:42:57.255 INFO:tasks.workunit.client.0.smithi115.stdout:0/876: stat d3/da/d4c/d69 0 2024-03-22T10:42:57.255 INFO:tasks.workunit.client.1.smithi150.stdout:4/547: mknod d0/d2f/cb2 0 2024-03-22T10:42:57.256 INFO:tasks.workunit.client.1.smithi150.stdout:5/451: dwrite d0/d8/dc/de/d14/f25 [0,4194304] 0 2024-03-22T10:42:57.256 INFO:tasks.workunit.client.1.smithi150.stdout:3/507: write d5/da/f34 [3179972,106521] 0 2024-03-22T10:42:57.257 INFO:tasks.workunit.client.0.smithi115.stdout:7/888: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/f12f x:0 0 0 2024-03-22T10:42:57.261 INFO:tasks.workunit.client.0.smithi115.stdout:4/902: truncate d1/d30/f41 1809981 0 2024-03-22T10:42:57.265 INFO:tasks.workunit.client.1.smithi150.stdout:0/528: symlink d0/d1/d84/d86/d11/d66/d35/d5f/da0/la5 0 2024-03-22T10:42:57.266 INFO:tasks.workunit.client.0.smithi115.stdout:3/961: dwrite d3/d82/d57/f12e [0,4194304] 0 2024-03-22T10:42:57.268 INFO:tasks.workunit.client.0.smithi115.stdout:9/899: link d6/d7/d20/d22/d3f/fab d6/d7/d30/d37/dbe/d110/f139 0 2024-03-22T10:42:57.272 INFO:tasks.workunit.client.1.smithi150.stdout:2/567: rename d12/d48 to d12/d19/db6 0 2024-03-22T10:42:57.274 INFO:tasks.workunit.client.0.smithi115.stdout:1/981: write d0/d8/de/d1a/d91/fd2 [434081,49107] 0 2024-03-22T10:42:57.277 INFO:tasks.workunit.client.1.smithi150.stdout:6/495: symlink db/dd/d15/d70/laf 0 2024-03-22T10:42:57.278 INFO:tasks.workunit.client.0.smithi115.stdout:6/841: link d4/d13/caf d4/d11/d12/d1c/d8a/d98/d9f/dc3/d10b/c113 0 2024-03-22T10:42:57.279 INFO:tasks.workunit.client.1.smithi150.stdout:3/508: mkdir d5/d53/d7e/d95/daa 0 2024-03-22T10:42:57.282 INFO:tasks.workunit.client.1.smithi150.stdout:6/496: write db/dd/d15/d28/d6c/fa2 [133286,48669] 0 2024-03-22T10:42:57.283 INFO:tasks.workunit.client.0.smithi115.stdout:9/900: mkdir d6/d7/d20/d22/de5/d13a 0 2024-03-22T10:42:57.284 INFO:tasks.workunit.client.1.smithi150.stdout:5/452: truncate d0/d8/dc/de/d14/d2e/d45/f2a 3232094 0 2024-03-22T10:42:57.284 INFO:tasks.workunit.client.1.smithi150.stdout:7/550: mkdir d2/d5/d7/d8/d57/d4a/db7 0 2024-03-22T10:42:57.285 INFO:tasks.workunit.client.0.smithi115.stdout:3/962: chown d3/d16/d29/d35/cd4 393 1 2024-03-22T10:42:57.285 INFO:tasks.workunit.client.0.smithi115.stdout:6/842: read - d4/d11/d12/d1c/d8a/d98/fed zero size 2024-03-22T10:42:57.286 INFO:tasks.workunit.client.0.smithi115.stdout:1/982: write d0/d22/d89/d141/f155 [723712,118821] 0 2024-03-22T10:42:57.288 INFO:tasks.workunit.client.1.smithi150.stdout:1/487: truncate d1/d19/d57/f5b 7528493 0 2024-03-22T10:42:57.290 INFO:tasks.workunit.client.1.smithi150.stdout:3/509: fsync d5/f35 0 2024-03-22T10:42:57.298 INFO:tasks.workunit.client.0.smithi115.stdout:2/849: dwrite da/d25/d3d/d57/f65 [0,4194304] 0 2024-03-22T10:42:57.298 INFO:tasks.workunit.client.0.smithi115.stdout:9/901: dread - d6/d7/d6a/d6f/f11f zero size 2024-03-22T10:42:57.299 INFO:tasks.workunit.client.1.smithi150.stdout:1/488: mknod d1/d19/d57/ca1 0 2024-03-22T10:42:57.302 INFO:tasks.workunit.client.1.smithi150.stdout:6/497: unlink db/dd/d15/d30/f43 0 2024-03-22T10:42:57.302 INFO:tasks.workunit.client.0.smithi115.stdout:0/877: dwrite d3/f7f [4194304,4194304] 0 2024-03-22T10:42:57.302 INFO:tasks.workunit.client.0.smithi115.stdout:9/902: stat d6/d8/c61 0 2024-03-22T10:42:57.302 INFO:tasks.workunit.client.0.smithi115.stdout:7/889: dread d2/d7/d12/d10/d23/d3d/dd2/ff5 [0,4194304] 0 2024-03-22T10:42:57.303 INFO:tasks.workunit.client.1.smithi150.stdout:3/510: truncate d5/fb 4201829 0 2024-03-22T10:42:57.303 INFO:tasks.workunit.client.1.smithi150.stdout:9/547: dread d1/d8/de/df/d36/d40/d41/f94 [0,4194304] 0 2024-03-22T10:42:57.304 INFO:tasks.workunit.client.1.smithi150.stdout:7/551: rename d2/d5/d7/d8/d57/c56 to d2/d5/d7/d8/d57/d4a/d8d/cb8 0 2024-03-22T10:42:57.305 INFO:tasks.workunit.client.1.smithi150.stdout:4/548: dwrite d0/f26 [0,4194304] 0 2024-03-22T10:42:57.313 INFO:tasks.workunit.client.0.smithi115.stdout:5/952: write d2/d15/d1b/d7c/d38/d56/f87 [1730797,32241] 0 2024-03-22T10:42:57.317 INFO:tasks.workunit.client.1.smithi150.stdout:1/489: creat d1/da/d2a/d4e/fa2 x:0 0 0 2024-03-22T10:42:57.318 INFO:tasks.workunit.client.1.smithi150.stdout:6/498: creat db/dd/d15/d60/d50/fb0 x:0 0 0 2024-03-22T10:42:57.320 INFO:tasks.workunit.client.1.smithi150.stdout:2/568: truncate d12/d21/f4f 1234930 0 2024-03-22T10:42:57.321 INFO:tasks.workunit.client.0.smithi115.stdout:5/953: write d2/df/d13/d8b/d93/da6/dbc/fd0 [2719750,69856] 0 2024-03-22T10:42:57.325 INFO:tasks.workunit.client.0.smithi115.stdout:6/843: dwrite d4/d13/d33/dbe/fd9 [0,4194304] 0 2024-03-22T10:42:57.325 INFO:tasks.workunit.client.1.smithi150.stdout:6/499: truncate db/dd/d15/d30/fab 844639 0 2024-03-22T10:42:57.329 INFO:tasks.workunit.client.0.smithi115.stdout:6/844: chown d4/d36/d48/d7a/c86 972 1 2024-03-22T10:42:57.332 INFO:tasks.workunit.client.1.smithi150.stdout:6/500: creat db/d52/fb1 x:0 0 0 2024-03-22T10:42:57.334 INFO:tasks.workunit.client.1.smithi150.stdout:4/549: link d0/f26 d0/d2f/fb3 0 2024-03-22T10:42:57.336 INFO:tasks.workunit.client.1.smithi150.stdout:1/490: fsync d1/da/d15/d18/f3d 0 2024-03-22T10:42:57.337 INFO:tasks.workunit.client.1.smithi150.stdout:3/511: creat d5/d22/d2f/d38/d3e/d5b/da6/fab x:0 0 0 2024-03-22T10:42:57.343 INFO:tasks.workunit.client.1.smithi150.stdout:9/548: dread d1/d8/fa1 [0,4194304] 0 2024-03-22T10:42:57.345 INFO:tasks.workunit.client.1.smithi150.stdout:1/491: mknod d1/d19/d1a/d7d/ca3 0 2024-03-22T10:42:57.346 INFO:tasks.workunit.client.1.smithi150.stdout:2/569: dread d12/d19/db6/da9/d2c/d65/f9d [0,4194304] 0 2024-03-22T10:42:57.349 INFO:tasks.workunit.client.1.smithi150.stdout:1/492: write d1/da/d15/f74 [871479,30684] 0 2024-03-22T10:42:57.349 INFO:tasks.workunit.client.0.smithi115.stdout:4/903: dread d1/d30/f41 [0,4194304] 0 2024-03-22T10:42:57.351 INFO:tasks.workunit.client.1.smithi150.stdout:6/501: mknod db/dd/d15/d60/cb2 0 2024-03-22T10:42:57.351 INFO:tasks.workunit.client.0.smithi115.stdout:4/904: readlink d1/d66/d106/d35/d40/d5c/d14/l8d 0 2024-03-22T10:42:57.353 INFO:tasks.workunit.client.1.smithi150.stdout:1/493: chown d1/d19/d1a/d7d/ca3 5954681 1 2024-03-22T10:42:57.355 INFO:tasks.workunit.client.1.smithi150.stdout:6/502: chown db/dd/d15/d57 193105 1 2024-03-22T10:42:57.356 INFO:tasks.workunit.client.1.smithi150.stdout:9/549: truncate d1/d8/de/df/d36/d40/d41/f61 1668136 0 2024-03-22T10:42:57.360 INFO:tasks.workunit.client.1.smithi150.stdout:2/570: write d12/d19/db6/da9/d67/f74 [713580,123554] 0 2024-03-22T10:42:57.360 INFO:tasks.workunit.client.1.smithi150.stdout:4/550: creat d0/d2f/d63/fb4 x:0 0 0 2024-03-22T10:42:57.361 INFO:tasks.workunit.client.1.smithi150.stdout:6/503: write db/dd/d82/d84/f87 [77841,72763] 0 2024-03-22T10:42:57.361 INFO:tasks.workunit.client.1.smithi150.stdout:9/550: chown d1/d8/de/df/d20/d32/d75/d99 611472 1 2024-03-22T10:42:57.362 INFO:tasks.workunit.client.1.smithi150.stdout:7/552: unlink d2/d5/d7/d8/d57/d54/f55 0 2024-03-22T10:42:57.362 INFO:tasks.workunit.client.1.smithi150.stdout:3/512: dread d5/d53/d7e/d95/fa1 [0,4194304] 0 2024-03-22T10:42:57.364 INFO:tasks.workunit.client.1.smithi150.stdout:5/453: dread d0/f10 [0,4194304] 0 2024-03-22T10:42:57.366 INFO:tasks.workunit.client.0.smithi115.stdout:1/983: rmdir d0/d8/d4d/d56 39 2024-03-22T10:42:57.366 INFO:tasks.workunit.client.1.smithi150.stdout:3/513: stat d5/d53/d7e/l92 0 2024-03-22T10:42:57.368 INFO:tasks.workunit.client.0.smithi115.stdout:2/850: write da/d1c/f1f [1958998,30419] 0 2024-03-22T10:42:57.369 INFO:tasks.workunit.client.0.smithi115.stdout:9/903: chown d6/d7/d6a/d6f/c133 9163 1 2024-03-22T10:42:57.371 INFO:tasks.workunit.client.0.smithi115.stdout:0/878: mknod d3/d7/d1b/d93/d104/c11b 0 2024-03-22T10:42:57.371 INFO:tasks.workunit.client.0.smithi115.stdout:9/904: stat d6/d7/d20 0 2024-03-22T10:42:57.374 INFO:tasks.workunit.client.0.smithi115.stdout:6/845: creat d4/d11/d12/d1c/d8a/d98/d9f/dc3/f114 x:0 0 0 2024-03-22T10:42:57.374 INFO:tasks.workunit.client.1.smithi150.stdout:6/504: mknod db/dd/d15/d28/d6c/d9b/cb3 0 2024-03-22T10:42:57.376 INFO:tasks.workunit.client.0.smithi115.stdout:4/905: symlink d1/d30/d4c/d86/d9f/d73/l134 0 2024-03-22T10:42:57.376 INFO:tasks.workunit.client.1.smithi150.stdout:9/551: unlink d1/d8/de/df/d24/f6b 0 2024-03-22T10:42:57.378 INFO:tasks.workunit.client.0.smithi115.stdout:3/963: mkdir d3/d19/d2b/d49/d13b 0 2024-03-22T10:42:57.380 INFO:tasks.workunit.client.0.smithi115.stdout:1/984: creat d0/d68/f159 x:0 0 0 2024-03-22T10:42:57.381 INFO:tasks.workunit.client.1.smithi150.stdout:4/551: creat d0/d1b/d4d/d9f/fb5 x:0 0 0 2024-03-22T10:42:57.381 INFO:tasks.workunit.client.0.smithi115.stdout:2/851: mkdir da/d25/d51/d100 0 2024-03-22T10:42:57.382 INFO:tasks.workunit.client.1.smithi150.stdout:8/561: sync 2024-03-22T10:42:57.383 INFO:tasks.workunit.client.0.smithi115.stdout:7/890: fdatasync d2/d62/fcb 0 2024-03-22T10:42:57.384 INFO:tasks.workunit.client.1.smithi150.stdout:6/505: mkdir db/dd/d2e/db4 0 2024-03-22T10:42:57.385 INFO:tasks.workunit.client.0.smithi115.stdout:5/954: rename d2/d15/d1b/f7f to d2/d15/d1b/d7c/d38/d56/f13a 0 2024-03-22T10:42:57.386 INFO:tasks.workunit.client.1.smithi150.stdout:8/562: write d0/d2/d19/f20 [2124643,19928] 0 2024-03-22T10:42:57.387 INFO:tasks.workunit.client.0.smithi115.stdout:9/905: creat d6/d7/d48/f13b x:0 0 0 2024-03-22T10:42:57.387 INFO:tasks.workunit.client.1.smithi150.stdout:6/506: fsync db/dd/d15/d30/f51 0 2024-03-22T10:42:57.388 INFO:tasks.workunit.client.1.smithi150.stdout:3/514: dwrite d5/f52 [0,4194304] 0 2024-03-22T10:42:57.389 INFO:tasks.workunit.client.0.smithi115.stdout:4/906: mkdir d1/d30/d4c/d86/d9f/de9/d135 0 2024-03-22T10:42:57.391 INFO:tasks.workunit.client.0.smithi115.stdout:4/907: fsync d1/d30/d4c/d86/d9f/fd6 0 2024-03-22T10:42:57.392 INFO:tasks.workunit.client.1.smithi150.stdout:8/563: stat d0/d3c/d71/d7a/d89/db5/cba 0 2024-03-22T10:42:57.393 INFO:tasks.workunit.client.1.smithi150.stdout:2/571: creat d12/fb7 x:0 0 0 2024-03-22T10:42:57.393 INFO:tasks.workunit.client.1.smithi150.stdout:3/515: read - d5/d22/d2f/d38/f85 zero size 2024-03-22T10:42:57.400 INFO:tasks.workunit.client.0.smithi115.stdout:2/852: creat da/d1c/d67/d78/f101 x:0 0 0 2024-03-22T10:42:57.400 INFO:tasks.workunit.client.1.smithi150.stdout:4/552: mkdir d0/d47/d72/db6 0 2024-03-22T10:42:57.402 INFO:tasks.workunit.client.1.smithi150.stdout:7/553: dwrite d2/d5/d7/d9/db/d9f/d96/f73 [0,4194304] 0 2024-03-22T10:42:57.403 INFO:tasks.workunit.client.0.smithi115.stdout:7/891: symlink d2/d7/d12/l130 0 2024-03-22T10:42:57.407 INFO:tasks.workunit.client.0.smithi115.stdout:0/879: creat d3/df0/dca/dce/d118/f11c x:0 0 0 2024-03-22T10:42:57.408 INFO:tasks.workunit.client.1.smithi150.stdout:2/572: symlink d12/d19/db6/da9/d67/d3c/lb8 0 2024-03-22T10:42:57.408 INFO:tasks.workunit.client.1.smithi150.stdout:9/552: dwrite d1/d8/de/df/d36/f96 [0,4194304] 0 2024-03-22T10:42:57.409 INFO:tasks.workunit.client.1.smithi150.stdout:6/507: dread db/dd/f5b [0,4194304] 0 2024-03-22T10:42:57.409 INFO:tasks.workunit.client.1.smithi150.stdout:5/454: dread d0/d8/dc/de/d14/d2e/f3e [0,4194304] 0 2024-03-22T10:42:57.411 INFO:tasks.workunit.client.0.smithi115.stdout:0/880: chown d3/d7/d1c/d6c/d9a/fbe 23059 1 2024-03-22T10:42:57.412 INFO:tasks.workunit.client.1.smithi150.stdout:5/455: dread - d0/d33/f73 zero size 2024-03-22T10:42:57.415 INFO:tasks.workunit.client.0.smithi115.stdout:4/908: write d1/d66/d106/d35/d40/d5c/d55/f50 [3018443,9521] 0 2024-03-22T10:42:57.417 INFO:tasks.workunit.client.0.smithi115.stdout:1/985: rmdir d0/d8/d4d/d56/db6/ddc/d135 39 2024-03-22T10:42:57.419 INFO:tasks.workunit.client.0.smithi115.stdout:2/853: mkdir da/d1c/d67/d78/d102 0 2024-03-22T10:42:57.420 INFO:tasks.workunit.client.1.smithi150.stdout:7/554: write d2/d5/d7/d8/d46/f66 [3355391,120347] 0 2024-03-22T10:42:57.420 INFO:tasks.workunit.client.0.smithi115.stdout:5/955: mkdir d2/d15/d1b/d7c/d38/d129/d13b 0 2024-03-22T10:42:57.425 INFO:tasks.workunit.client.0.smithi115.stdout:7/892: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/f131 x:0 0 0 2024-03-22T10:42:57.426 INFO:tasks.workunit.client.1.smithi150.stdout:1/494: dwrite d1/d19/d57/f5b [4194304,4194304] 0 2024-03-22T10:42:57.428 INFO:tasks.workunit.client.0.smithi115.stdout:9/906: creat d6/d7/d20/d22/d23/d106/da4/f13c x:0 0 0 2024-03-22T10:42:57.429 INFO:tasks.workunit.client.1.smithi150.stdout:6/508: unlink db/dd/d15/d28/f2b 0 2024-03-22T10:42:57.429 INFO:tasks.workunit.client.1.smithi150.stdout:9/553: write d1/d8/de/df/da2/f91 [983963,93815] 0 2024-03-22T10:42:57.430 INFO:tasks.workunit.client.0.smithi115.stdout:5/956: fsync d2/df/d13/d8b/d93/da6/fe0 0 2024-03-22T10:42:57.431 INFO:tasks.workunit.client.0.smithi115.stdout:2/854: symlink da/d25/d3d/d7e/d90/d96/l103 0 2024-03-22T10:42:57.432 INFO:tasks.workunit.client.1.smithi150.stdout:1/495: dread - d1/d19/d1a/d7d/f82 zero size 2024-03-22T10:42:57.433 INFO:tasks.workunit.client.0.smithi115.stdout:4/909: rename d1/d66/d106/d3b/fb7 to d1/d66/d106/d68/f136 0 2024-03-22T10:42:57.434 INFO:tasks.workunit.client.1.smithi150.stdout:9/554: chown d1/d8/de/df/d20/d29/f9b 831544 1 2024-03-22T10:42:57.436 INFO:tasks.workunit.client.1.smithi150.stdout:3/516: rmdir d5/da/d62/d84 0 2024-03-22T10:42:57.438 INFO:tasks.workunit.client.1.smithi150.stdout:0/529: sync 2024-03-22T10:42:57.439 INFO:tasks.workunit.client.1.smithi150.stdout:5/456: mknod d0/d8/dc/de/d14/d4f/c83 0 2024-03-22T10:42:57.441 INFO:tasks.workunit.client.1.smithi150.stdout:9/555: read d1/d8/de/df/d24/f4a [354344,105785] 0 2024-03-22T10:42:57.441 INFO:tasks.workunit.client.0.smithi115.stdout:2/855: truncate da/d25/d51/fef 858512 0 2024-03-22T10:42:57.442 INFO:tasks.workunit.client.1.smithi150.stdout:7/555: rmdir d2/d5/d7/d8/d46/d49 39 2024-03-22T10:42:57.443 INFO:tasks.workunit.client.1.smithi150.stdout:5/457: readlink d0/d8/dc/de/d14/d4f/l5f 0 2024-03-22T10:42:57.444 INFO:tasks.workunit.client.0.smithi115.stdout:7/893: rmdir d2/d7/d12/d10/d19 39 2024-03-22T10:42:57.445 INFO:tasks.workunit.client.1.smithi150.stdout:8/564: dread d0/d2/d19/d29/d2f/df/d1b/f30 [0,4194304] 0 2024-03-22T10:42:57.446 INFO:tasks.workunit.client.1.smithi150.stdout:6/509: creat db/dd/d82/d84/fb5 x:0 0 0 2024-03-22T10:42:57.446 INFO:tasks.workunit.client.0.smithi115.stdout:3/964: link d3/d82/d54/de7/l118 d3/d82/d54/l13c 0 2024-03-22T10:42:57.448 INFO:tasks.workunit.client.0.smithi115.stdout:1/986: truncate d0/d8/d4d/d56/db6/fee 510863 0 2024-03-22T10:42:57.449 INFO:tasks.workunit.client.1.smithi150.stdout:3/517: dread - d5/f4f zero size 2024-03-22T10:42:57.450 INFO:tasks.workunit.client.1.smithi150.stdout:4/553: link d0/d2f/d63/c7a d0/d47/d72/db6/cb7 0 2024-03-22T10:42:57.450 INFO:tasks.workunit.client.0.smithi115.stdout:4/910: truncate d1/d30/d4c/d86/f8a 1855745 0 2024-03-22T10:42:57.455 INFO:tasks.workunit.client.1.smithi150.stdout:7/556: unlink d2/d5/d7/d8/d57/d54/d91/dad/fb1 0 2024-03-22T10:42:57.455 INFO:tasks.workunit.client.1.smithi150.stdout:5/458: mknod d0/d8/dc/de/d14/d4f/c84 0 2024-03-22T10:42:57.456 INFO:tasks.workunit.client.1.smithi150.stdout:6/510: symlink db/d52/lb6 0 2024-03-22T10:42:57.457 INFO:tasks.workunit.client.0.smithi115.stdout:3/965: mkdir d3/d19/d2b/d49/d91/dad/d13d 0 2024-03-22T10:42:57.458 INFO:tasks.workunit.client.0.smithi115.stdout:2/856: getdents da/d25/dfb 0 2024-03-22T10:42:57.458 INFO:tasks.workunit.client.0.smithi115.stdout:1/987: mknod d0/d8/d4d/d56/dcf/d10b/c15a 0 2024-03-22T10:42:57.459 INFO:tasks.workunit.client.1.smithi150.stdout:3/518: chown d5/c12 5161802 1 2024-03-22T10:42:57.460 INFO:tasks.workunit.client.1.smithi150.stdout:0/530: rename d0/d1/d81 to d0/d1/d84/d86/da6 0 2024-03-22T10:42:57.461 INFO:tasks.workunit.client.1.smithi150.stdout:3/519: write d5/f10 [450441,7299] 0 2024-03-22T10:42:57.461 INFO:tasks.workunit.client.1.smithi150.stdout:8/565: mkdir d0/d2/d19/dc0 0 2024-03-22T10:42:57.462 INFO:tasks.workunit.client.0.smithi115.stdout:2/857: chown da/d25/d3d/d7e/d90/dbf/cca 75 1 2024-03-22T10:42:57.463 INFO:tasks.workunit.client.0.smithi115.stdout:4/911: fdatasync d1/d30/d4c/d86/d9f/d4f/f10a 0 2024-03-22T10:42:57.466 INFO:tasks.workunit.client.0.smithi115.stdout:7/894: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f132 x:0 0 0 2024-03-22T10:42:57.466 INFO:tasks.workunit.client.0.smithi115.stdout:6/846: truncate d4/d36/d48/d72/daa/fc4 2100976 0 2024-03-22T10:42:57.467 INFO:tasks.workunit.client.1.smithi150.stdout:3/520: chown d5/d22/d2f/d38/f42 29146603 1 2024-03-22T10:42:57.469 INFO:tasks.workunit.client.1.smithi150.stdout:7/557: truncate d2/d5/d7/d8/d57/d54/d91/fb3 416570 0 2024-03-22T10:42:57.470 INFO:tasks.workunit.client.0.smithi115.stdout:1/988: creat d0/d12/d2b/d10e/f15b x:0 0 0 2024-03-22T10:42:57.473 INFO:tasks.workunit.client.1.smithi150.stdout:2/573: dwrite d12/d19/db6/da9/d2c/f57 [0,4194304] 0 2024-03-22T10:42:57.473 INFO:tasks.workunit.client.1.smithi150.stdout:9/556: getdents d1/d8/de/df/d20/d29 0 2024-03-22T10:42:57.474 INFO:tasks.workunit.client.0.smithi115.stdout:4/912: chown d1/d66/d106/d35/d40/f48 1 1 2024-03-22T10:42:57.480 INFO:tasks.workunit.client.0.smithi115.stdout:0/881: truncate d3/d4/d6d/f9e 2093541 0 2024-03-22T10:42:57.481 INFO:tasks.workunit.client.1.smithi150.stdout:8/566: mkdir d0/d2/d19/d29/d37/d7d/dc1 0 2024-03-22T10:42:57.481 INFO:tasks.workunit.client.1.smithi150.stdout:3/521: read d5/d53/f75 [1206218,6169] 0 2024-03-22T10:42:57.483 INFO:tasks.workunit.client.1.smithi150.stdout:7/558: unlink d2/f67 0 2024-03-22T10:42:57.484 INFO:tasks.workunit.client.0.smithi115.stdout:0/882: fsync d3/d4/d14/d64/fab 0 2024-03-22T10:42:57.485 INFO:tasks.workunit.client.1.smithi150.stdout:3/522: mknod d5/d53/d7e/d95/cac 0 2024-03-22T10:42:57.488 INFO:tasks.workunit.client.0.smithi115.stdout:4/913: mknod d1/dbb/d102/c137 0 2024-03-22T10:42:57.488 INFO:tasks.workunit.client.1.smithi150.stdout:2/574: write d12/d19/db6/da9/d61/d98/f6f [1774983,68420] 0 2024-03-22T10:42:57.488 INFO:tasks.workunit.client.1.smithi150.stdout:1/496: dwrite d1/f6c [0,4194304] 0 2024-03-22T10:42:57.490 INFO:tasks.workunit.client.0.smithi115.stdout:6/847: getdents d4/d11/d12 0 2024-03-22T10:42:57.491 INFO:tasks.workunit.client.0.smithi115.stdout:7/895: rename d2/d7/d12/d10/d23/d3d/dd2/da3/ld8 to d2/d7/l133 0 2024-03-22T10:42:57.496 INFO:tasks.workunit.client.1.smithi150.stdout:3/523: mkdir d5/da/dad 0 2024-03-22T10:42:57.497 INFO:tasks.workunit.client.1.smithi150.stdout:7/559: symlink d2/d5/d7/d8/lb9 0 2024-03-22T10:42:57.498 INFO:tasks.workunit.client.0.smithi115.stdout:0/883: read d3/d7/f5c [3877073,29704] 0 2024-03-22T10:42:57.498 INFO:tasks.workunit.client.0.smithi115.stdout:3/966: dwrite d3/d19/dd8/f10a [0,4194304] 0 2024-03-22T10:42:57.499 INFO:tasks.workunit.client.0.smithi115.stdout:9/907: dread d6/d7/d19/f31 [0,4194304] 0 2024-03-22T10:42:57.502 INFO:tasks.workunit.client.1.smithi150.stdout:4/554: rename d0/d2f/d63/d7b/c84 to d0/cb8 0 2024-03-22T10:42:57.502 INFO:tasks.workunit.client.1.smithi150.stdout:8/567: fsync d0/d2/d19/d29/d31/f4b 0 2024-03-22T10:42:57.503 INFO:tasks.workunit.client.1.smithi150.stdout:0/531: write d0/d1/d84/d86/d11/d66/d37/f3d [4211230,5852] 0 2024-03-22T10:42:57.508 INFO:tasks.workunit.client.1.smithi150.stdout:4/555: chown d0/d1b/f21 37 1 2024-03-22T10:42:57.509 INFO:tasks.workunit.client.0.smithi115.stdout:4/914: mknod d1/d66/d106/d35/d40/d54/c138 0 2024-03-22T10:42:57.509 INFO:tasks.workunit.client.0.smithi115.stdout:0/884: readlink d3/d4/d14/d64/lcb 0 2024-03-22T10:42:57.511 INFO:tasks.workunit.client.1.smithi150.stdout:5/459: dwrite d0/f22 [0,4194304] 0 2024-03-22T10:42:57.519 INFO:tasks.workunit.client.0.smithi115.stdout:7/896: fdatasync d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/f129 0 2024-03-22T10:42:57.519 INFO:tasks.workunit.client.1.smithi150.stdout:3/524: symlink d5/d22/d2f/d38/d3e/d5b/lae 0 2024-03-22T10:42:57.520 INFO:tasks.workunit.client.1.smithi150.stdout:6/511: truncate db/dd/d15/f77 1537993 0 2024-03-22T10:42:57.521 INFO:tasks.workunit.client.0.smithi115.stdout:3/967: mknod d3/d19/d2b/d49/d13b/c13e 0 2024-03-22T10:42:57.526 INFO:tasks.workunit.client.1.smithi150.stdout:3/525: chown d5/d53/d7e/d95/daa 30771 1 2024-03-22T10:42:57.527 INFO:tasks.workunit.client.1.smithi150.stdout:8/568: mkdir d0/d2/d19/d29/d37/dc2 0 2024-03-22T10:42:57.527 INFO:tasks.workunit.client.0.smithi115.stdout:6/848: link d4/d13/d33/c66 d4/d11/dc7/c115 0 2024-03-22T10:42:57.530 INFO:tasks.workunit.client.1.smithi150.stdout:5/460: write d0/d8/dc/de/d14/f50 [2173961,23641] 0 2024-03-22T10:42:57.530 INFO:tasks.workunit.client.0.smithi115.stdout:4/915: truncate d1/fd 2000224 0 2024-03-22T10:42:57.532 INFO:tasks.workunit.client.1.smithi150.stdout:3/526: read d5/da/f7a [140052,24240] 0 2024-03-22T10:42:57.532 INFO:tasks.workunit.client.1.smithi150.stdout:8/569: readlink d0/d3c/d71/l6a 0 2024-03-22T10:42:57.532 INFO:tasks.workunit.client.1.smithi150.stdout:4/556: write d0/d1b/d4d/f65 [756290,45208] 0 2024-03-22T10:42:57.535 INFO:tasks.workunit.client.1.smithi150.stdout:0/532: creat d0/d1/d84/d86/d42/d10/fa7 x:0 0 0 2024-03-22T10:42:57.536 INFO:tasks.workunit.client.0.smithi115.stdout:9/908: creat d6/d7/d20/d22/d3f/dba/f13d x:0 0 0 2024-03-22T10:42:57.536 INFO:tasks.workunit.client.0.smithi115.stdout:2/858: dwrite da/d25/f84 [0,4194304] 0 2024-03-22T10:42:57.537 INFO:tasks.workunit.client.1.smithi150.stdout:1/497: dwrite d1/d79/f85 [0,4194304] 0 2024-03-22T10:42:57.538 INFO:tasks.workunit.client.0.smithi115.stdout:6/849: mknod d4/d36/d48/d7a/c116 0 2024-03-22T10:42:57.538 INFO:tasks.workunit.client.1.smithi150.stdout:2/575: dwrite d12/d19/d1a/f30 [4194304,4194304] 0 2024-03-22T10:42:57.541 INFO:tasks.workunit.client.1.smithi150.stdout:5/461: write d0/f3 [2379365,65237] 0 2024-03-22T10:42:57.543 INFO:tasks.workunit.client.1.smithi150.stdout:8/570: write d0/d2/d19/d29/d37/fb3 [795903,16340] 0 2024-03-22T10:42:57.549 INFO:tasks.workunit.client.1.smithi150.stdout:0/533: chown d0/d1/d84/d86/d42 10352 1 2024-03-22T10:42:57.551 INFO:tasks.workunit.client.0.smithi115.stdout:4/916: creat d1/d30/d4c/d86/d9f/d4f/f139 x:0 0 0 2024-03-22T10:42:57.556 INFO:tasks.workunit.client.0.smithi115.stdout:2/859: stat da/d25/d3d/d7e/d90/d96/l103 0 2024-03-22T10:42:57.556 INFO:tasks.workunit.client.1.smithi150.stdout:9/557: dread d1/d8/de/df/f18 [0,4194304] 0 2024-03-22T10:42:57.560 INFO:tasks.workunit.client.0.smithi115.stdout:2/860: chown da/f82 7498986 1 2024-03-22T10:42:57.561 INFO:tasks.workunit.client.0.smithi115.stdout:5/957: dread d2/d15/fac [0,4194304] 0 2024-03-22T10:42:57.565 INFO:tasks.workunit.client.0.smithi115.stdout:6/850: truncate d4/d11/d12/d1c/f2d 5564448 0 2024-03-22T10:42:57.566 INFO:tasks.workunit.client.1.smithi150.stdout:9/558: fdatasync d1/d8/de/df/d20/d29/fb1 0 2024-03-22T10:42:57.572 INFO:tasks.workunit.client.0.smithi115.stdout:3/968: dwrite d3/d19/dd8/f115 [0,4194304] 0 2024-03-22T10:42:57.572 INFO:tasks.workunit.client.0.smithi115.stdout:4/917: mknod d1/d30/d4c/d86/d9f/d75/d99/da8/c13a 0 2024-03-22T10:42:57.575 INFO:tasks.workunit.client.0.smithi115.stdout:0/885: dwrite d3/d7/d1b/df8/f10d [0,4194304] 0 2024-03-22T10:42:57.575 INFO:tasks.workunit.client.1.smithi150.stdout:3/527: dwrite d5/d22/d2f/d38/d3e/d5b/f76 [0,4194304] 0 2024-03-22T10:42:57.577 INFO:tasks.workunit.client.1.smithi150.stdout:3/528: chown d5/da/f34 3039 1 2024-03-22T10:42:57.586 INFO:tasks.workunit.client.0.smithi115.stdout:5/958: rename d2/d15/d1b/d7c/daf/lbe to d2/df/d13/d8b/d93/da6/dbc/l13c 0 2024-03-22T10:42:57.586 INFO:tasks.workunit.client.0.smithi115.stdout:2/861: truncate da/d25/d3d/d57/d7d/da1/dba/ddf/ffa 945517 0 2024-03-22T10:42:57.588 INFO:tasks.workunit.client.0.smithi115.stdout:4/918: mkdir d1/d30/d4c/d86/d9f/d32/d13b 0 2024-03-22T10:42:57.591 INFO:tasks.workunit.client.0.smithi115.stdout:6/851: getdents d4/d11/dc7 0 2024-03-22T10:42:57.593 INFO:tasks.workunit.client.0.smithi115.stdout:9/909: write d6/d7/d20/d22/d23/d106/f10c [1032972,24934] 0 2024-03-22T10:42:57.599 INFO:tasks.workunit.client.0.smithi115.stdout:4/919: dread - d1/d66/d82/da1/d110/f130 zero size 2024-03-22T10:42:57.600 INFO:tasks.workunit.client.0.smithi115.stdout:0/886: rmdir d3/da/d4c/d74/deb 0 2024-03-22T10:42:57.602 INFO:tasks.workunit.client.0.smithi115.stdout:6/852: write d4/d11/d12/d1c/f7f [5730131,38575] 0 2024-03-22T10:42:57.603 INFO:tasks.workunit.client.1.smithi150.stdout:4/557: dread d0/f93 [4194304,4194304] 0 2024-03-22T10:42:57.605 INFO:tasks.workunit.client.0.smithi115.stdout:2/862: creat da/d25/d3d/d57/d7d/da1/f104 x:0 0 0 2024-03-22T10:42:57.606 INFO:tasks.workunit.client.0.smithi115.stdout:7/897: dwrite d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f11c [0,4194304] 0 2024-03-22T10:42:57.607 INFO:tasks.workunit.client.0.smithi115.stdout:3/969: link d3/d19/dda/c12b d3/d6d/db4/d107/da6/c13f 0 2024-03-22T10:42:57.607 INFO:tasks.workunit.client.1.smithi150.stdout:1/498: creat d1/da/d15/d4f/fa4 x:0 0 0 2024-03-22T10:42:57.612 INFO:tasks.workunit.client.1.smithi150.stdout:8/571: symlink d0/d2/d7e/lc3 0 2024-03-22T10:42:57.612 INFO:tasks.workunit.client.1.smithi150.stdout:6/512: link db/dd/l13 db/dd/d15/d30/lb7 0 2024-03-22T10:42:57.616 INFO:tasks.workunit.client.1.smithi150.stdout:6/513: write db/dd/d25/f45 [2127711,84686] 0 2024-03-22T10:42:57.616 INFO:tasks.workunit.client.1.smithi150.stdout:8/572: fdatasync d0/d2/d19/d29/d2f/df/f4f 0 2024-03-22T10:42:57.618 INFO:tasks.workunit.client.1.smithi150.stdout:0/534: creat d0/d1/d84/d86/d11/d66/d35/d5f/fa8 x:0 0 0 2024-03-22T10:42:57.619 INFO:tasks.workunit.client.0.smithi115.stdout:6/853: truncate d4/d11/d12/d1c/d8a/d98/d9f/df0/f107 427936 0 2024-03-22T10:42:57.620 INFO:tasks.workunit.client.0.smithi115.stdout:9/910: rmdir d6/d7/d20/d9c/d108 0 2024-03-22T10:42:57.620 INFO:tasks.workunit.client.0.smithi115.stdout:3/970: mkdir d3/d140 0 2024-03-22T10:42:57.620 INFO:tasks.workunit.client.0.smithi115.stdout:7/898: symlink d2/d7/d12/d10/d23/d3d/da4/l134 0 2024-03-22T10:42:57.620 INFO:tasks.workunit.client.0.smithi115.stdout:6/854: chown d4/d13/d33/cff 12 1 2024-03-22T10:42:57.621 INFO:tasks.workunit.client.1.smithi150.stdout:6/514: fsync db/dd/d15/d57/f65 0 2024-03-22T10:42:57.626 INFO:tasks.workunit.client.0.smithi115.stdout:4/920: rename d1/d30/d4c/d86/d9f/d75/d99/d98/d9b/fe2 to d1/d30/d4c/d86/d9f/d75/d99/da8/dea/f13c 0 2024-03-22T10:42:57.627 INFO:tasks.workunit.client.1.smithi150.stdout:7/560: dread d2/d5/d7/f4d [4194304,4194304] 0 2024-03-22T10:42:57.627 INFO:tasks.workunit.client.0.smithi115.stdout:7/899: mknod d2/d62/d126/c135 0 2024-03-22T10:42:57.628 INFO:tasks.workunit.client.1.smithi150.stdout:8/573: chown d0/d2/d19/f78 35123 1 2024-03-22T10:42:57.629 INFO:tasks.workunit.client.0.smithi115.stdout:2/863: read da/d25/d30/dcd/ff0 [342102,8377] 0 2024-03-22T10:42:57.630 INFO:tasks.workunit.client.1.smithi150.stdout:7/561: fsync d2/d5/d7/fa4 0 2024-03-22T10:42:57.631 INFO:tasks.workunit.client.0.smithi115.stdout:4/921: truncate d1/d66/d82/f9a 4717437 0 2024-03-22T10:42:57.633 INFO:tasks.workunit.client.1.smithi150.stdout:1/499: dwrite d1/f70 [0,4194304] 0 2024-03-22T10:42:57.633 INFO:tasks.workunit.client.1.smithi150.stdout:8/574: fsync d0/d2/d19/d29/d37/f9c 0 2024-03-22T10:42:57.637 INFO:tasks.workunit.client.0.smithi115.stdout:6/855: creat d4/d13/d33/d103/f117 x:0 0 0 2024-03-22T10:42:57.638 INFO:tasks.workunit.client.0.smithi115.stdout:9/911: mkdir d6/d7/d20/d22/d23/d106/da4/d13e 0 2024-03-22T10:42:57.640 INFO:tasks.workunit.client.0.smithi115.stdout:2/864: truncate da/d25/d3d/d7e/fa2 212862 0 2024-03-22T10:42:57.641 INFO:tasks.workunit.client.0.smithi115.stdout:9/912: chown d6/d8/d3c/d59/d68/c81 0 1 2024-03-22T10:42:57.644 INFO:tasks.workunit.client.0.smithi115.stdout:4/922: read - d1/d30/d4c/d86/d9f/d75/d99/d98/fc5 zero size 2024-03-22T10:42:57.647 INFO:tasks.workunit.client.1.smithi150.stdout:0/535: mknod d0/d1/d84/d86/d11/d66/d35/d5f/ca9 0 2024-03-22T10:42:57.648 INFO:tasks.workunit.client.0.smithi115.stdout:5/959: write d2/d15/d1b/d7c/d38/d135/d83/f65 [533322,86400] 0 2024-03-22T10:42:57.650 INFO:tasks.workunit.client.1.smithi150.stdout:6/515: mknod db/dd/d15/d28/d6c/d9b/cb8 0 2024-03-22T10:42:57.652 INFO:tasks.workunit.client.0.smithi115.stdout:6/856: creat d4/d11/d12/d1c/d8a/f118 x:0 0 0 2024-03-22T10:42:57.655 INFO:tasks.workunit.client.1.smithi150.stdout:1/500: write d1/da/f72 [1647029,117871] 0 2024-03-22T10:42:57.655 INFO:tasks.workunit.client.1.smithi150.stdout:8/575: write d0/d3c/f56 [869618,67702] 0 2024-03-22T10:42:57.656 INFO:tasks.workunit.client.0.smithi115.stdout:3/971: getdents d3/d19/dda 0 2024-03-22T10:42:57.657 INFO:tasks.workunit.client.0.smithi115.stdout:0/887: dread d3/d4/d6d/f98 [0,4194304] 0 2024-03-22T10:42:57.660 INFO:tasks.workunit.client.1.smithi150.stdout:7/562: creat d2/d5/d7/d9/db/d6c/da8/fba x:0 0 0 2024-03-22T10:42:57.660 INFO:tasks.workunit.client.1.smithi150.stdout:8/576: write d0/d2/d19/d29/d2f/f73 [845368,119977] 0 2024-03-22T10:42:57.662 INFO:tasks.workunit.client.1.smithi150.stdout:6/516: mkdir db/dd/d2e/db9 0 2024-03-22T10:42:57.662 INFO:tasks.workunit.client.0.smithi115.stdout:3/972: chown d3/d6d/db4/d107/db3/c133 0 1 2024-03-22T10:42:57.662 INFO:tasks.workunit.client.0.smithi115.stdout:5/960: read d2/d15/d1b/d7c/d38/f78 [199892,1145] 0 2024-03-22T10:42:57.663 INFO:tasks.workunit.client.0.smithi115.stdout:6/857: rename d4/d11/d99/la8 to d4/d11/d12/d1c/d8a/l119 0 2024-03-22T10:42:57.664 INFO:tasks.workunit.client.1.smithi150.stdout:0/536: creat d0/d12/faa x:0 0 0 2024-03-22T10:42:57.664 INFO:tasks.workunit.client.1.smithi150.stdout:7/563: chown d2/cc 14708 1 2024-03-22T10:42:57.667 INFO:tasks.workunit.client.1.smithi150.stdout:9/559: getdents d1/d49 0 2024-03-22T10:42:57.668 INFO:tasks.workunit.client.0.smithi115.stdout:2/865: dwrite da/d25/d51/f63 [0,4194304] 0 2024-03-22T10:42:57.672 INFO:tasks.workunit.client.1.smithi150.stdout:8/577: symlink d0/d2/d19/d29/d88/lc4 0 2024-03-22T10:42:57.673 INFO:tasks.workunit.client.0.smithi115.stdout:2/866: chown da/d1c/c46 1468249 1 2024-03-22T10:42:57.674 INFO:tasks.workunit.client.0.smithi115.stdout:6/858: creat d4/d5e/dd0/f11a x:0 0 0 2024-03-22T10:42:57.676 INFO:tasks.workunit.client.0.smithi115.stdout:6/859: dread - d4/d11/f106 zero size 2024-03-22T10:42:57.677 INFO:tasks.workunit.client.1.smithi150.stdout:8/578: fsync d0/d2/d19/d29/d37/d7d/d9e/fb0 0 2024-03-22T10:42:57.677 INFO:tasks.workunit.client.0.smithi115.stdout:3/973: symlink d3/d19/d2b/l141 0 2024-03-22T10:42:57.679 INFO:tasks.workunit.client.1.smithi150.stdout:8/579: readlink d0/d2/l8b 0 2024-03-22T10:42:57.681 INFO:tasks.workunit.client.0.smithi115.stdout:2/867: creat da/d25/d3d/d7e/f105 x:0 0 0 2024-03-22T10:42:57.681 INFO:tasks.workunit.client.1.smithi150.stdout:4/558: dread d0/d1b/d4d/d6c/d61/f48 [0,4194304] 0 2024-03-22T10:42:57.683 INFO:tasks.workunit.client.1.smithi150.stdout:6/517: creat db/dd/d2e/db9/fba x:0 0 0 2024-03-22T10:42:57.684 INFO:tasks.workunit.client.0.smithi115.stdout:3/974: write d3/d6d/db4/de1/f10b [777269,97292] 0 2024-03-22T10:42:57.684 INFO:tasks.workunit.client.0.smithi115.stdout:4/923: write d1/d30/d4c/d86/d9f/f45 [5039536,18057] 0 2024-03-22T10:42:57.685 INFO:tasks.workunit.client.0.smithi115.stdout:6/860: readlink d4/ld 0 2024-03-22T10:42:57.691 INFO:tasks.workunit.client.0.smithi115.stdout:2/868: chown da/d25/d30/f2e 499 1 2024-03-22T10:42:57.692 INFO:tasks.workunit.client.1.smithi150.stdout:4/559: read d0/f9 [112943,68286] 0 2024-03-22T10:42:57.693 INFO:tasks.workunit.client.0.smithi115.stdout:5/961: write d2/df/d13/d8b/d93/da6/f90 [859707,112467] 0 2024-03-22T10:42:57.693 INFO:tasks.workunit.client.1.smithi150.stdout:0/537: creat d0/d1/d84/d86/d42/d10/fab x:0 0 0 2024-03-22T10:42:57.695 INFO:tasks.workunit.client.0.smithi115.stdout:3/975: mkdir d3/d82/d57/d142 0 2024-03-22T10:42:57.695 INFO:tasks.workunit.client.1.smithi150.stdout:1/501: getdents d1/d79 0 2024-03-22T10:42:57.697 INFO:tasks.workunit.client.1.smithi150.stdout:6/518: creat db/dd/d2e/fbb x:0 0 0 2024-03-22T10:42:57.698 INFO:tasks.workunit.client.1.smithi150.stdout:2/576: dwrite d12/d19/d2d/f36 [8388608,4194304] 0 2024-03-22T10:42:57.702 INFO:tasks.workunit.client.1.smithi150.stdout:7/564: link d2/d5/d7/c14 d2/d5/d7/d9/db/d6c/d9c/cbb 0 2024-03-22T10:42:57.702 INFO:tasks.workunit.client.1.smithi150.stdout:5/462: dwrite d0/d8/dc/de/d14/d4f/f3d [0,4194304] 0 2024-03-22T10:42:57.704 INFO:tasks.workunit.client.0.smithi115.stdout:5/962: chown d2/d15/d1b/d7c/f105 103 1 2024-03-22T10:42:57.704 INFO:tasks.workunit.client.1.smithi150.stdout:6/519: stat db/dd/l11 0 2024-03-22T10:42:57.705 INFO:tasks.workunit.client.1.smithi150.stdout:1/502: write d1/da/d15/f74 [1588613,62409] 0 2024-03-22T10:42:57.706 INFO:tasks.workunit.client.0.smithi115.stdout:9/913: dwrite d6/d7/d48/d70/fc0 [0,4194304] 0 2024-03-22T10:42:57.706 INFO:tasks.workunit.client.0.smithi115.stdout:3/976: creat d3/d19/f143 x:0 0 0 2024-03-22T10:42:57.711 INFO:tasks.workunit.client.0.smithi115.stdout:4/924: link d1/d66/d106/d35/d40/l8b d1/dbb/dc6/dd8/l13d 0 2024-03-22T10:42:57.712 INFO:tasks.workunit.client.0.smithi115.stdout:5/963: mkdir d2/d15/d1b/d7c/d84/d13d 0 2024-03-22T10:42:57.712 INFO:tasks.workunit.client.0.smithi115.stdout:0/888: dwrite d3/d4/d14/f15 [4194304,4194304] 0 2024-03-22T10:42:57.712 INFO:tasks.workunit.client.1.smithi150.stdout:3/529: write d5/fb [1551236,72274] 0 2024-03-22T10:42:57.713 INFO:tasks.workunit.client.1.smithi150.stdout:6/520: truncate db/dd/f2c 2993713 0 2024-03-22T10:42:57.713 INFO:tasks.workunit.client.1.smithi150.stdout:2/577: mknod d12/d19/cb9 0 2024-03-22T10:42:57.714 INFO:tasks.workunit.client.1.smithi150.stdout:3/530: chown d5/d22 114 1 2024-03-22T10:42:57.718 INFO:tasks.workunit.client.1.smithi150.stdout:1/503: write d1/d19/d50/d51/f6b [453259,121577] 0 2024-03-22T10:42:57.720 INFO:tasks.workunit.client.1.smithi150.stdout:5/463: mknod d0/d8/dc/de/d14/d2e/d45/d56/c85 0 2024-03-22T10:42:57.721 INFO:tasks.workunit.client.0.smithi115.stdout:3/977: rename d3/d19/f4e to d3/d140/f144 0 2024-03-22T10:42:57.722 INFO:tasks.workunit.client.1.smithi150.stdout:8/580: dread d0/d2/d19/d29/d2f/df/f26 [0,4194304] 0 2024-03-22T10:42:57.723 INFO:tasks.workunit.client.1.smithi150.stdout:1/504: chown d1/d19/d57/c61 212012 1 2024-03-22T10:42:57.723 INFO:tasks.workunit.client.0.smithi115.stdout:2/869: write da/d1c/f52 [260448,39496] 0 2024-03-22T10:42:57.724 INFO:tasks.workunit.client.0.smithi115.stdout:4/925: chown d1/d30/d4c/d86/d9f/d75/d99/da8/ldb 58 1 2024-03-22T10:42:57.724 INFO:tasks.workunit.client.0.smithi115.stdout:5/964: symlink d2/df/d13/d8b/d93/da6/dbc/d50/db1/d136/l13e 0 2024-03-22T10:42:57.725 INFO:tasks.workunit.client.1.smithi150.stdout:6/521: readlink db/dd/d15/l42 0 2024-03-22T10:42:57.727 INFO:tasks.workunit.client.1.smithi150.stdout:1/505: chown d1/da/d2a/d4e 215667026 1 2024-03-22T10:42:57.728 INFO:tasks.workunit.client.0.smithi115.stdout:0/889: fdatasync d3/da/d4c/d74/d84/dd7/fbf 0 2024-03-22T10:42:57.728 INFO:tasks.workunit.client.1.smithi150.stdout:6/522: fdatasync db/dd/d2e/fa1 0 2024-03-22T10:42:57.729 INFO:tasks.workunit.client.0.smithi115.stdout:3/978: chown d3/d16/d29/d35/cd4 4 1 2024-03-22T10:42:57.729 INFO:tasks.workunit.client.0.smithi115.stdout:2/870: write da/d25/d3d/f3c [4865817,8874] 0 2024-03-22T10:42:57.730 INFO:tasks.workunit.client.0.smithi115.stdout:6/861: dwrite d4/d13/d33/f6c [0,4194304] 0 2024-03-22T10:42:57.731 INFO:tasks.workunit.client.0.smithi115.stdout:9/914: rename d6/d7/d30/d37/fa0 to d6/d7/d19/d135/f13f 0 2024-03-22T10:42:57.732 INFO:tasks.workunit.client.0.smithi115.stdout:0/890: readlink d3/l109 0 2024-03-22T10:42:57.733 INFO:tasks.workunit.client.1.smithi150.stdout:3/531: mkdir d5/daf 0 2024-03-22T10:42:57.733 INFO:tasks.workunit.client.0.smithi115.stdout:5/965: unlink d2/f5e 0 2024-03-22T10:42:57.735 INFO:tasks.workunit.client.0.smithi115.stdout:4/926: unlink d1/d66/d106/d35/d40/d54/f7a 0 2024-03-22T10:42:57.735 INFO:tasks.workunit.client.0.smithi115.stdout:0/891: chown d3/d7/d1c/d6c/d9a 12261349 1 2024-03-22T10:42:57.738 INFO:tasks.workunit.client.0.smithi115.stdout:3/979: symlink d3/d19/d2b/l145 0 2024-03-22T10:42:57.739 INFO:tasks.workunit.client.1.smithi150.stdout:5/464: dread d0/f22 [0,4194304] 0 2024-03-22T10:42:57.740 INFO:tasks.workunit.client.0.smithi115.stdout:4/927: chown d1/d66/d106/d28/fe5 602456189 1 2024-03-22T10:42:57.740 INFO:tasks.workunit.client.0.smithi115.stdout:0/892: write d3/f7f [5246219,112640] 0 2024-03-22T10:42:57.742 INFO:tasks.workunit.client.1.smithi150.stdout:5/465: chown d0/d8/dc/de/d14/d4f/c83 323228181 1 2024-03-22T10:42:57.743 INFO:tasks.workunit.client.1.smithi150.stdout:5/466: chown d0/d8/dc/d1d 24956149 1 2024-03-22T10:42:57.744 INFO:tasks.workunit.client.0.smithi115.stdout:9/915: creat d6/d7/d30/d37/dbe/d110/f140 x:0 0 0 2024-03-22T10:42:57.744 INFO:tasks.workunit.client.0.smithi115.stdout:6/862: symlink d4/df3/l11b 0 2024-03-22T10:42:57.745 INFO:tasks.workunit.client.0.smithi115.stdout:2/871: rename da/d25/d2a/l4d to da/d25/d51/l106 0 2024-03-22T10:42:57.746 INFO:tasks.workunit.client.1.smithi150.stdout:1/506: rename d1/l3f to d1/d19/d1a/la5 0 2024-03-22T10:42:57.746 INFO:tasks.workunit.client.1.smithi150.stdout:5/467: chown d0/d8/dc/de/d14/d4f/d47 277114014 1 2024-03-22T10:42:57.746 INFO:tasks.workunit.client.1.smithi150.stdout:7/565: rmdir d2/d5/d7/d8/d57/d4a/db7 0 2024-03-22T10:42:57.748 INFO:tasks.workunit.client.0.smithi115.stdout:4/928: chown d1/d66/d106/d35/d40/d5c/d14 16271100 1 2024-03-22T10:42:57.749 INFO:tasks.workunit.client.0.smithi115.stdout:3/980: creat d3/d6d/db4/d107/f146 x:0 0 0 2024-03-22T10:42:57.749 INFO:tasks.workunit.client.0.smithi115.stdout:5/966: mknod d2/df/d13/d8b/d93/c13f 0 2024-03-22T10:42:57.751 INFO:tasks.workunit.client.1.smithi150.stdout:5/468: write d0/d8/dc/de/f70 [692457,30043] 0 2024-03-22T10:42:57.751 INFO:tasks.workunit.client.0.smithi115.stdout:2/872: chown da/d1c/ce9 27621684 1 2024-03-22T10:42:57.752 INFO:tasks.workunit.client.0.smithi115.stdout:6/863: stat d4/d11/d12/d1c/d8a/d98/d9f/dc3/le0 0 2024-03-22T10:42:57.754 INFO:tasks.workunit.client.0.smithi115.stdout:9/916: unlink d6/d7/d6a/d6f/le0 0 2024-03-22T10:42:57.755 INFO:tasks.workunit.client.1.smithi150.stdout:8/581: getdents d0/d3c 0 2024-03-22T10:42:57.756 INFO:tasks.workunit.client.1.smithi150.stdout:1/507: chown d1/d19/d50/d6e/l7e 510434628 1 2024-03-22T10:42:57.759 INFO:tasks.workunit.client.0.smithi115.stdout:9/917: stat d6/d8/d3c 0 2024-03-22T10:42:57.759 INFO:tasks.workunit.client.0.smithi115.stdout:6/864: mkdir d4/d11/dc7/d11c 0 2024-03-22T10:42:57.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:57 smithi115 ceph-mon[29111]: pgmap v17: 65 pgs: 65 active+clean; 2.7 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 103 MiB/s rd, 316 MiB/s wr, 599 op/s 2024-03-22T10:42:57.761 INFO:tasks.workunit.client.0.smithi115.stdout:3/981: mknod d3/d19/d2b/d49/d91/dad/d13d/c147 0 2024-03-22T10:42:57.762 INFO:tasks.workunit.client.1.smithi150.stdout:3/532: getdents d5/d22/d2f/d38/d3e/d5b/da6 0 2024-03-22T10:42:57.762 INFO:tasks.workunit.client.0.smithi115.stdout:0/893: rename d3/d7/cec to d3/d7/d6b/c11d 0 2024-03-22T10:42:57.764 INFO:tasks.workunit.client.0.smithi115.stdout:5/967: truncate d2/d15/d1b/d7c/d38/f61 6842534 0 2024-03-22T10:42:57.764 INFO:tasks.workunit.client.0.smithi115.stdout:6/865: creat d4/d11/d91/f11d x:0 0 0 2024-03-22T10:42:57.765 INFO:tasks.workunit.client.1.smithi150.stdout:9/560: dread d1/d8/de/df/f11 [0,4194304] 0 2024-03-22T10:42:57.767 INFO:tasks.workunit.client.1.smithi150.stdout:5/469: creat d0/d8/dc/de/d14/d2e/d5a/f86 x:0 0 0 2024-03-22T10:42:57.767 INFO:tasks.workunit.client.1.smithi150.stdout:0/538: write d0/d1/d84/d86/d42/f82 [201931,30963] 0 2024-03-22T10:42:57.767 INFO:tasks.workunit.client.0.smithi115.stdout:0/894: chown d3/d52/f57 0 1 2024-03-22T10:42:57.767 INFO:tasks.workunit.client.0.smithi115.stdout:4/929: getdents d1/d66/d106/d28 0 2024-03-22T10:42:57.768 INFO:tasks.workunit.client.0.smithi115.stdout:3/982: dread - d3/d6d/db4/d107/daa/ff6 zero size 2024-03-22T10:42:57.768 INFO:tasks.workunit.client.1.smithi150.stdout:2/578: truncate d12/d19/db6/da9/d2c/f57 2058375 0 2024-03-22T10:42:57.770 INFO:tasks.workunit.client.0.smithi115.stdout:9/918: creat d6/d7/d48/d70/da9/dbd/de9/f141 x:0 0 0 2024-03-22T10:42:57.770 INFO:tasks.workunit.client.1.smithi150.stdout:0/539: stat d0/d12/c3f 0 2024-03-22T10:42:57.773 INFO:tasks.workunit.client.0.smithi115.stdout:5/968: mknod d2/df/d3b/d8e/dd3/c140 0 2024-03-22T10:42:57.776 INFO:tasks.workunit.client.1.smithi150.stdout:7/566: write d2/d5/f7c [369970,115093] 0 2024-03-22T10:42:57.778 INFO:tasks.workunit.client.0.smithi115.stdout:6/866: chown d4/d11/d12/d1c/d8a/d98/d9f/df0/lf7 26042 1 2024-03-22T10:42:57.778 INFO:tasks.workunit.client.0.smithi115.stdout:0/895: chown d3/d7/d1b/d1f/l43 3809537 1 2024-03-22T10:42:57.779 INFO:tasks.workunit.client.1.smithi150.stdout:4/560: dread d0/de/f4b [0,4194304] 0 2024-03-22T10:42:57.779 INFO:tasks.workunit.client.0.smithi115.stdout:5/969: chown d2/df/d13 95890570 1 2024-03-22T10:42:57.781 INFO:tasks.workunit.client.1.smithi150.stdout:5/470: creat d0/d8/dc/de/d14/d2e/d45/f87 x:0 0 0 2024-03-22T10:42:57.781 INFO:tasks.workunit.client.1.smithi150.stdout:2/579: read d12/d19/db6/da9/d2c/d65/f9d [606956,13465] 0 2024-03-22T10:42:57.781 INFO:tasks.workunit.client.0.smithi115.stdout:6/867: dread - d4/f92 zero size 2024-03-22T10:42:57.782 INFO:tasks.workunit.client.1.smithi150.stdout:9/561: mknod d1/d8/de/df/d36/d40/d44/db5/cc4 0 2024-03-22T10:42:57.785 INFO:tasks.workunit.client.0.smithi115.stdout:2/873: dwrite da/d25/d3d/d7e/d90/d96/fdb [0,4194304] 0 2024-03-22T10:42:57.786 INFO:tasks.workunit.client.1.smithi150.stdout:9/562: stat d1/d8/de/d73/cbe 0 2024-03-22T10:42:57.787 INFO:tasks.workunit.client.0.smithi115.stdout:9/919: creat d6/d7/d19/d4e/f142 x:0 0 0 2024-03-22T10:42:57.789 INFO:tasks.workunit.client.0.smithi115.stdout:2/874: write da/d25/d3d/f2c [2604668,87701] 0 2024-03-22T10:42:57.794 INFO:tasks.workunit.client.1.smithi150.stdout:1/508: dwrite d1/d19/d1a/f1e [0,4194304] 0 2024-03-22T10:42:57.796 INFO:tasks.workunit.client.1.smithi150.stdout:7/567: creat d2/d5/d7/d8/d46/fbc x:0 0 0 2024-03-22T10:42:57.802 INFO:tasks.workunit.client.1.smithi150.stdout:7/568: dread - d2/d5/d7/d8/d46/f74 zero size 2024-03-22T10:42:57.805 INFO:tasks.workunit.client.1.smithi150.stdout:1/509: creat d1/fa6 x:0 0 0 2024-03-22T10:42:57.808 INFO:tasks.workunit.client.1.smithi150.stdout:4/561: dwrite d0/d1b/d4d/f92 [0,4194304] 0 2024-03-22T10:42:57.808 INFO:tasks.workunit.client.1.smithi150.stdout:0/540: dwrite d0/d1/d84/d86/d5a/f7c [0,4194304] 0 2024-03-22T10:42:57.809 INFO:tasks.workunit.client.1.smithi150.stdout:7/569: readlink d2/d5/d7/d8/d57/l9b 0 2024-03-22T10:42:57.810 INFO:tasks.workunit.client.1.smithi150.stdout:1/510: rmdir d1/da 39 2024-03-22T10:42:57.812 INFO:tasks.workunit.client.0.smithi115.stdout:0/896: read d3/d7/d1c/f56 [211251,61517] 0 2024-03-22T10:42:57.812 INFO:tasks.workunit.client.0.smithi115.stdout:3/983: getdents d3/d6d/db4/d107/daa 0 2024-03-22T10:42:57.814 INFO:tasks.workunit.client.0.smithi115.stdout:6/868: rename d4/c26 to d4/c11e 0 2024-03-22T10:42:57.817 INFO:tasks.workunit.client.1.smithi150.stdout:7/570: chown d2/d5/d7/d9/db/d9f/d45/l90 16427 1 2024-03-22T10:42:57.820 INFO:tasks.workunit.client.0.smithi115.stdout:6/869: write d4/d11/d12/f4e [5123634,44530] 0 2024-03-22T10:42:57.824 INFO:tasks.workunit.client.0.smithi115.stdout:3/984: symlink d3/d82/l148 0 2024-03-22T10:42:57.827 INFO:tasks.workunit.client.1.smithi150.stdout:8/582: dread d0/f5b [0,4194304] 0 2024-03-22T10:42:57.827 INFO:tasks.workunit.client.1.smithi150.stdout:7/571: write d2/d5/d7/d8/d57/d54/d91/fb3 [1106049,114039] 0 2024-03-22T10:42:57.828 INFO:tasks.workunit.client.0.smithi115.stdout:1/989: sync 2024-03-22T10:42:57.828 INFO:tasks.workunit.client.0.smithi115.stdout:7/900: sync 2024-03-22T10:42:57.834 INFO:tasks.workunit.client.0.smithi115.stdout:6/870: rmdir d4/d36/d48/d72/daa/db9 39 2024-03-22T10:42:57.836 INFO:tasks.workunit.client.1.smithi150.stdout:4/562: link d0/de/c1c d0/d1b/d4d/d6c/d61/cb9 0 2024-03-22T10:42:57.838 INFO:tasks.workunit.client.1.smithi150.stdout:0/541: dwrite d0/d1/d84/d86/d42/d10/d38/f4c [4194304,4194304] 0 2024-03-22T10:42:57.840 INFO:tasks.workunit.client.0.smithi115.stdout:6/871: stat d4/d36/l112 0 2024-03-22T10:42:57.840 INFO:tasks.workunit.client.0.smithi115.stdout:0/897: creat d3/d7/f11e x:0 0 0 2024-03-22T10:42:57.842 INFO:tasks.workunit.client.0.smithi115.stdout:3/985: unlink d3/d82/d54/ffa 0 2024-03-22T10:42:57.842 INFO:tasks.workunit.client.1.smithi150.stdout:7/572: mknod d2/d5/d7/dab/cbd 0 2024-03-22T10:42:57.843 INFO:tasks.workunit.client.1.smithi150.stdout:4/563: creat d0/d32/fba x:0 0 0 2024-03-22T10:42:57.844 INFO:tasks.workunit.client.0.smithi115.stdout:7/901: readlink d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/l86 0 2024-03-22T10:42:57.845 INFO:tasks.workunit.client.0.smithi115.stdout:9/920: dwrite d6/d7/d30/d37/f125 [0,4194304] 0 2024-03-22T10:42:57.849 INFO:tasks.workunit.client.1.smithi150.stdout:5/471: dwrite d0/f2b [0,4194304] 0 2024-03-22T10:42:57.849 INFO:tasks.workunit.client.1.smithi150.stdout:4/564: write d0/d47/faa [646330,33123] 0 2024-03-22T10:42:57.852 INFO:tasks.workunit.client.0.smithi115.stdout:1/990: dread - d0/d12/d17/d37/d9e/da0/dff/f145 zero size 2024-03-22T10:42:57.853 INFO:tasks.workunit.client.0.smithi115.stdout:2/875: dwrite da/d25/d3d/d57/f5d [0,4194304] 0 2024-03-22T10:42:57.854 INFO:tasks.workunit.client.0.smithi115.stdout:3/986: write d3/d82/d52/f56 [3387294,45211] 0 2024-03-22T10:42:57.854 INFO:tasks.workunit.client.0.smithi115.stdout:4/930: dwrite d1/d66/d106/d35/d40/d5c/f23 [0,4194304] 0 2024-03-22T10:42:57.855 INFO:tasks.workunit.client.1.smithi150.stdout:8/583: dread d0/f5b [0,4194304] 0 2024-03-22T10:42:57.856 INFO:tasks.workunit.client.1.smithi150.stdout:0/542: mkdir d0/d1/d84/d86/d42/d10/d38/d52/d9b/dac 0 2024-03-22T10:42:57.856 INFO:tasks.workunit.client.1.smithi150.stdout:4/565: fsync d0/de/f5e 0 2024-03-22T10:42:57.858 INFO:tasks.workunit.client.1.smithi150.stdout:5/472: read d0/f3 [2542772,55442] 0 2024-03-22T10:42:57.865 INFO:tasks.workunit.client.1.smithi150.stdout:2/580: dread d12/d19/db6/da9/d2c/f33 [0,4194304] 0 2024-03-22T10:42:57.865 INFO:tasks.workunit.client.1.smithi150.stdout:8/584: truncate d0/d5d/fbc 544321 0 2024-03-22T10:42:57.869 INFO:tasks.workunit.client.1.smithi150.stdout:9/563: dwrite d1/d8/de/df/d36/f43 [0,4194304] 0 2024-03-22T10:42:57.873 INFO:tasks.workunit.client.1.smithi150.stdout:8/585: creat d0/d2/d19/d29/d37/fc5 x:0 0 0 2024-03-22T10:42:57.875 INFO:tasks.workunit.client.1.smithi150.stdout:2/581: mknod d12/d19/db6/da9/d97/cba 0 2024-03-22T10:42:57.876 INFO:tasks.workunit.client.0.smithi115.stdout:9/921: chown d6/d8/ce 433497039 1 2024-03-22T10:42:57.876 INFO:tasks.workunit.client.1.smithi150.stdout:9/564: fdatasync d1/d8/de/df/d20/d32/fbc 0 2024-03-22T10:42:57.882 INFO:tasks.workunit.client.1.smithi150.stdout:8/586: mkdir d0/d2/d19/d29/d37/dc6 0 2024-03-22T10:42:57.885 INFO:tasks.workunit.client.0.smithi115.stdout:7/902: chown d2/d7/d12/d10/d23/d3d/da4/dc0/d116 575099085 1 2024-03-22T10:42:57.887 INFO:tasks.workunit.client.0.smithi115.stdout:4/931: mknod d1/d66/d106/d35/d40/d5c/c13e 0 2024-03-22T10:42:57.890 INFO:tasks.workunit.client.1.smithi150.stdout:9/565: link d1/d8/de/df/f11 d1/d8/fc5 0 2024-03-22T10:42:57.891 INFO:tasks.workunit.client.0.smithi115.stdout:5/970: dread d2/df/d3b/f7b [0,4194304] 0 2024-03-22T10:42:57.893 INFO:tasks.workunit.client.0.smithi115.stdout:2/876: creat da/d25/d3d/d57/da7/f107 x:0 0 0 2024-03-22T10:42:57.893 INFO:tasks.workunit.client.1.smithi150.stdout:5/473: dwrite d0/d8/dc/de/d14/d4f/d47/f77 [0,4194304] 0 2024-03-22T10:42:57.895 INFO:tasks.workunit.client.1.smithi150.stdout:8/587: creat d0/d3c/d71/d7a/d89/db5/fc7 x:0 0 0 2024-03-22T10:42:57.896 INFO:tasks.workunit.client.0.smithi115.stdout:1/991: rename d0/d8/l69 to d0/d8/de/d1a/d91/da3/l15c 0 2024-03-22T10:42:57.897 INFO:tasks.workunit.client.1.smithi150.stdout:5/474: symlink d0/d33/d6f/l88 0 2024-03-22T10:42:57.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:57 smithi150 ceph-mon[34518]: pgmap v17: 65 pgs: 65 active+clean; 2.7 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 103 MiB/s rd, 316 MiB/s wr, 599 op/s 2024-03-22T10:42:57.900 INFO:tasks.workunit.client.0.smithi115.stdout:5/971: truncate d2/df/d3b/fb0 2390819 0 2024-03-22T10:42:57.900 INFO:tasks.workunit.client.0.smithi115.stdout:4/932: creat d1/d66/d82/da1/d110/f13f x:0 0 0 2024-03-22T10:42:57.900 INFO:tasks.workunit.client.1.smithi150.stdout:8/588: write d0/d2/d19/d29/d37/da1/fa4 [1240966,47152] 0 2024-03-22T10:42:57.903 INFO:tasks.workunit.client.0.smithi115.stdout:1/992: read d0/d22/fc6 [3066057,69485] 0 2024-03-22T10:42:57.904 INFO:tasks.workunit.client.1.smithi150.stdout:5/475: mkdir d0/d8/dc/d1d/d89 0 2024-03-22T10:42:57.907 INFO:tasks.workunit.client.1.smithi150.stdout:5/476: symlink d0/d37/l8a 0 2024-03-22T10:42:57.911 INFO:tasks.workunit.client.1.smithi150.stdout:5/477: chown d0/d8/dc/de/d14/d2e/d45/f78 1 1 2024-03-22T10:42:57.913 INFO:tasks.workunit.client.0.smithi115.stdout:9/922: rename d6/d7/d48/d70/da9/fff to d6/d7/d6a/d6f/f143 0 2024-03-22T10:42:57.918 INFO:tasks.workunit.client.0.smithi115.stdout:7/903: mkdir d2/d7/d12/d10/d19/dbd/dce/d136 0 2024-03-22T10:42:57.919 INFO:tasks.workunit.client.1.smithi150.stdout:0/543: dwrite d0/d1/d84/d86/d42/fc [0,4194304] 0 2024-03-22T10:42:57.920 INFO:tasks.workunit.client.1.smithi150.stdout:2/582: dwrite d12/d19/db6/f4d [0,4194304] 0 2024-03-22T10:42:57.924 INFO:tasks.workunit.client.1.smithi150.stdout:0/544: write d0/d1/d84/d86/d42/d10/d38/d52/f54 [2539905,83017] 0 2024-03-22T10:42:57.926 INFO:tasks.workunit.client.1.smithi150.stdout:5/478: symlink d0/d8/dc/de/d14/d2e/d46/d4d/l8b 0 2024-03-22T10:42:57.930 INFO:tasks.workunit.client.1.smithi150.stdout:9/566: truncate d1/d8/de/df/d36/f43 2873861 0 2024-03-22T10:42:57.931 INFO:tasks.workunit.client.1.smithi150.stdout:2/583: creat d12/d19/db6/da9/d61/d98/fbb x:0 0 0 2024-03-22T10:42:57.932 INFO:tasks.workunit.client.1.smithi150.stdout:2/584: fdatasync d12/d19/db6/d63/fa8 0 2024-03-22T10:42:57.934 INFO:tasks.workunit.client.0.smithi115.stdout:3/987: getdents d3/d6d/db4/de1 0 2024-03-22T10:42:57.936 INFO:tasks.workunit.client.0.smithi115.stdout:5/972: mkdir d2/df/d3b/dfb/d127/d141 0 2024-03-22T10:42:57.940 INFO:tasks.workunit.client.0.smithi115.stdout:9/923: truncate d6/d7/d48/d70/da9/fac 921108 0 2024-03-22T10:42:57.940 INFO:tasks.workunit.client.0.smithi115.stdout:2/877: dread da/d1c/d67/f6f [0,4194304] 0 2024-03-22T10:42:57.944 INFO:tasks.workunit.client.0.smithi115.stdout:7/904: readlink d2/d7/d12/d10/d23/d3d/dd2/lc8 0 2024-03-22T10:42:57.945 INFO:tasks.workunit.client.0.smithi115.stdout:1/993: dwrite d0/d12/d17/d37/d119/d13d/f156 [0,4194304] 0 2024-03-22T10:42:57.951 INFO:tasks.workunit.client.0.smithi115.stdout:4/933: dwrite d1/d30/d4c/d86/dde/d103/f12c [0,4194304] 0 2024-03-22T10:42:57.951 INFO:tasks.workunit.client.0.smithi115.stdout:6/872: dread d4/f30 [0,4194304] 0 2024-03-22T10:42:57.957 INFO:tasks.workunit.client.0.smithi115.stdout:3/988: mknod d3/d16/c149 0 2024-03-22T10:42:57.957 INFO:tasks.workunit.client.0.smithi115.stdout:9/924: read - d6/d7/d6a/d6f/f143 zero size 2024-03-22T10:42:57.957 INFO:tasks.workunit.client.0.smithi115.stdout:2/878: fdatasync da/d25/d51/fd4 0 2024-03-22T10:42:57.959 INFO:tasks.workunit.client.1.smithi150.stdout:0/545: creat d0/d1/d84/d86/d11/d66/d35/d92/fad x:0 0 0 2024-03-22T10:42:57.960 INFO:tasks.workunit.client.0.smithi115.stdout:7/905: chown d2/d7/d12/d10/d23/d3d/dd2/da3/df3/f119 27074 1 2024-03-22T10:42:57.961 INFO:tasks.workunit.client.0.smithi115.stdout:6/873: fsync d4/d36/d48/f67 0 2024-03-22T10:42:57.964 INFO:tasks.workunit.client.1.smithi150.stdout:9/567: link d1/d8/d95/fa6 d1/d8/de/dbd/fc6 0 2024-03-22T10:42:57.965 INFO:tasks.workunit.client.1.smithi150.stdout:0/546: mkdir d0/d1/d84/d86/da6/dae 0 2024-03-22T10:42:57.967 INFO:tasks.workunit.client.1.smithi150.stdout:5/479: link d0/d8/dc/de/d14/d2e/d46/c67 d0/d8/dc/de/d14/d2e/d46/d4d/c8c 0 2024-03-22T10:42:57.967 INFO:tasks.workunit.client.1.smithi150.stdout:2/585: creat d12/fbc x:0 0 0 2024-03-22T10:42:57.969 INFO:tasks.workunit.client.0.smithi115.stdout:4/934: truncate d1/d30/d4c/d86/d9f/d75/d99/da8/dea/f13c 646966 0 2024-03-22T10:42:57.970 INFO:tasks.workunit.client.1.smithi150.stdout:5/480: mknod d0/d33/d6f/c8d 0 2024-03-22T10:42:57.972 INFO:tasks.workunit.client.0.smithi115.stdout:2/879: rmdir da/d1c 39 2024-03-22T10:42:57.973 INFO:tasks.workunit.client.0.smithi115.stdout:5/973: write d2/d15/d1b/d7c/daf/ff6 [935729,80987] 0 2024-03-22T10:42:57.974 INFO:tasks.workunit.client.0.smithi115.stdout:3/989: mknod d3/d6d/db4/c14a 0 2024-03-22T10:42:57.976 INFO:tasks.workunit.client.1.smithi150.stdout:2/586: getdents d12/d35/d7d 0 2024-03-22T10:42:57.977 INFO:tasks.workunit.client.0.smithi115.stdout:4/935: symlink d1/dbb/dc6/dd8/df7/l140 0 2024-03-22T10:42:57.978 INFO:tasks.workunit.client.1.smithi150.stdout:2/587: chown d12/d19/d1a/f30 27924931 1 2024-03-22T10:42:57.979 INFO:tasks.workunit.client.1.smithi150.stdout:9/568: link d1/d8/de/df/d36/d40/d44/f6e d1/d8/d95/fc7 0 2024-03-22T10:42:57.980 INFO:tasks.workunit.client.1.smithi150.stdout:2/588: dread - d12/fb7 zero size 2024-03-22T10:42:57.983 INFO:tasks.workunit.client.1.smithi150.stdout:9/569: write d1/d8/de/df/d36/d40/d41/fac [332843,26845] 0 2024-03-22T10:42:57.984 INFO:tasks.workunit.client.1.smithi150.stdout:2/589: truncate d12/d19/db6/da9/d61/f8c 352114 0 2024-03-22T10:42:57.986 INFO:tasks.workunit.client.1.smithi150.stdout:9/570: dread - d1/d8/d95/fa6 zero size 2024-03-22T10:42:57.987 INFO:tasks.workunit.client.0.smithi115.stdout:4/936: truncate d1/d66/d106/d35/f9c 1056488 0 2024-03-22T10:42:57.987 INFO:tasks.workunit.client.0.smithi115.stdout:5/974: mknod d2/d15/d1b/d7c/d84/d13d/c142 0 2024-03-22T10:42:57.989 INFO:tasks.workunit.client.0.smithi115.stdout:3/990: creat d3/d16/f14b x:0 0 0 2024-03-22T10:42:57.989 INFO:tasks.workunit.client.0.smithi115.stdout:9/925: getdents d6/d7/d20/d22 0 2024-03-22T10:42:57.990 INFO:tasks.workunit.client.0.smithi115.stdout:1/994: dwrite d0/d12/d2b/d10e/f14b [0,4194304] 0 2024-03-22T10:42:57.990 INFO:tasks.workunit.client.1.smithi150.stdout:0/547: dread d0/d1/f48 [0,4194304] 0 2024-03-22T10:42:57.992 INFO:tasks.workunit.client.0.smithi115.stdout:2/880: mknod da/d1c/d67/d78/d102/c108 0 2024-03-22T10:42:57.992 INFO:tasks.workunit.client.1.smithi150.stdout:2/590: mkdir d12/d19/db6/da9/d97/d78/dbd 0 2024-03-22T10:42:57.994 INFO:tasks.workunit.client.1.smithi150.stdout:5/481: dwrite d0/d8/f34 [0,4194304] 0 2024-03-22T10:42:57.994 INFO:tasks.workunit.client.0.smithi115.stdout:4/937: write d1/d66/d106/d35/d40/d5c/d14/f58 [323181,75946] 0 2024-03-22T10:42:57.995 INFO:tasks.workunit.client.0.smithi115.stdout:3/991: chown d3/d89/dca/f124 19528 1 2024-03-22T10:42:57.996 INFO:tasks.workunit.client.0.smithi115.stdout:9/926: creat d6/d7/d48/d53/f144 x:0 0 0 2024-03-22T10:42:57.998 INFO:tasks.workunit.client.0.smithi115.stdout:4/938: chown d1/d30/d4c/d86 4127981 1 2024-03-22T10:42:57.999 INFO:tasks.workunit.client.0.smithi115.stdout:5/975: write d2/df/d13/d8b/d93/da6/dbc/d50/db1/fdc [5166152,65949] 0 2024-03-22T10:42:58.000 INFO:tasks.workunit.client.0.smithi115.stdout:2/881: mknod da/d25/d3d/dad/db3/c109 0 2024-03-22T10:42:58.001 INFO:tasks.workunit.client.0.smithi115.stdout:1/995: truncate d0/d22/f41 497939 0 2024-03-22T10:42:58.003 INFO:tasks.workunit.client.1.smithi150.stdout:2/591: creat d12/d35/d7d/fbe x:0 0 0 2024-03-22T10:42:58.004 INFO:tasks.workunit.client.0.smithi115.stdout:5/976: readlink d2/df/d3b/ldb 0 2024-03-22T10:42:58.008 INFO:tasks.workunit.client.1.smithi150.stdout:5/482: creat d0/d8/dc/de/d14/d2e/d5a/f8e x:0 0 0 2024-03-22T10:42:58.008 INFO:tasks.workunit.client.1.smithi150.stdout:0/548: mknod d0/d1/d84/d86/d11/d66/d35/d92/caf 0 2024-03-22T10:42:58.009 INFO:tasks.workunit.client.0.smithi115.stdout:2/882: mkdir da/d25/d51/d10a 0 2024-03-22T10:42:58.011 INFO:tasks.workunit.client.0.smithi115.stdout:5/977: rmdir d2/df/d13/d8b/d93/da6 39 2024-03-22T10:42:58.011 INFO:tasks.workunit.client.1.smithi150.stdout:2/592: truncate d12/d19/d2d/f53 671639 0 2024-03-22T10:42:58.012 INFO:tasks.workunit.client.0.smithi115.stdout:7/906: dread d2/d7/d12/d10/f16 [0,4194304] 0 2024-03-22T10:42:58.013 INFO:tasks.workunit.client.0.smithi115.stdout:1/996: creat d0/d12/d17/d37/d9e/da0/db9/f15d x:0 0 0 2024-03-22T10:42:58.014 INFO:tasks.workunit.client.1.smithi150.stdout:0/549: rename d0/d1/d84/d86/f28 to d0/d1/d84/d86/d42/d10/d38/d52/d68/fb0 0 2024-03-22T10:42:58.015 INFO:tasks.workunit.client.0.smithi115.stdout:4/939: creat d1/d30/d4c/d86/d9f/d75/d99/da8/dea/d12a/f141 x:0 0 0 2024-03-22T10:42:58.017 INFO:tasks.workunit.client.0.smithi115.stdout:4/940: readlink d1/d66/d106/d28/lcd 0 2024-03-22T10:42:58.017 INFO:tasks.workunit.client.0.smithi115.stdout:5/978: truncate d2/df/f125 219523 0 2024-03-22T10:42:58.018 INFO:tasks.workunit.client.1.smithi150.stdout:0/550: rename d0/d1/d84/c87 to d0/d12/cb1 0 2024-03-22T10:42:58.020 INFO:tasks.workunit.client.0.smithi115.stdout:1/997: fsync d0/d12/d2b/fca 0 2024-03-22T10:42:58.023 INFO:tasks.workunit.client.0.smithi115.stdout:7/907: rename d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f11c to d2/d62/d126/f137 0 2024-03-22T10:42:58.024 INFO:tasks.workunit.client.1.smithi150.stdout:2/593: read d12/d19/db6/d63/f6c [3397721,21155] 0 2024-03-22T10:42:58.027 INFO:tasks.workunit.client.0.smithi115.stdout:7/908: rmdir d2/d7/d12/d10/d19/dbd/dce/d105 39 2024-03-22T10:42:58.028 INFO:tasks.workunit.client.0.smithi115.stdout:5/979: truncate d2/d15/d1b/d7c/d38/d43/fca 938111 0 2024-03-22T10:42:58.028 INFO:tasks.workunit.client.1.smithi150.stdout:2/594: unlink d12/d19/db6/da9/d67/d3c/lb8 0 2024-03-22T10:42:58.029 INFO:tasks.workunit.client.0.smithi115.stdout:1/998: rmdir d0/d12/d17/d37/d119/d139/d8e 39 2024-03-22T10:42:58.030 INFO:tasks.workunit.client.0.smithi115.stdout:4/941: link d1/d30/l115 d1/d66/dd9/d10e/l142 0 2024-03-22T10:42:58.031 INFO:tasks.workunit.client.1.smithi150.stdout:2/595: write d12/d19/d1a/f2b [681248,75564] 0 2024-03-22T10:42:58.033 INFO:tasks.workunit.client.0.smithi115.stdout:5/980: fsync d2/d15/d1b/d7c/d38/d135/d83/f9b 0 2024-03-22T10:42:58.033 INFO:tasks.workunit.client.1.smithi150.stdout:2/596: chown d12/d19/db6/f9f 4869252 1 2024-03-22T10:42:58.033 INFO:tasks.workunit.client.1.smithi150.stdout:0/551: dwrite d0/d1/d84/d86/d42/f82 [0,4194304] 0 2024-03-22T10:42:58.038 INFO:tasks.workunit.client.1.smithi150.stdout:2/597: write d12/d35/fb2 [190007,129117] 0 2024-03-22T10:42:58.041 INFO:tasks.workunit.client.1.smithi150.stdout:2/598: fsync d12/d19/db6/da9/d61/f86 0 2024-03-22T10:42:58.044 INFO:tasks.workunit.client.0.smithi115.stdout:9/927: write d6/d7/d87/f8c [1302015,115704] 0 2024-03-22T10:42:58.046 INFO:tasks.workunit.client.1.smithi150.stdout:2/599: read d12/d35/fb2 [53687,88239] 0 2024-03-22T10:42:58.046 INFO:tasks.workunit.client.0.smithi115.stdout:2/883: truncate da/d25/d51/fe6 3716957 0 2024-03-22T10:42:58.048 INFO:tasks.workunit.client.0.smithi115.stdout:3/992: dread d3/d16/d29/f125 [0,4194304] 0 2024-03-22T10:42:58.053 INFO:tasks.workunit.client.1.smithi150.stdout:2/600: link d12/d19/db6/da9/l90 d12/d21/d81/lbf 0 2024-03-22T10:42:58.056 INFO:tasks.workunit.client.1.smithi150.stdout:2/601: chown d12/d19/db6/da9/d67/f3d 47338 1 2024-03-22T10:42:58.062 INFO:tasks.workunit.client.1.smithi150.stdout:2/602: getdents d12/d19/db6/da9/d97/d78 0 2024-03-22T10:42:58.064 INFO:tasks.workunit.client.1.smithi150.stdout:2/603: mknod d12/d21/d81/cc0 0 2024-03-22T10:42:58.066 INFO:tasks.workunit.client.0.smithi115.stdout:0/898: sync 2024-03-22T10:42:58.067 INFO:tasks.workunit.client.0.smithi115.stdout:3/993: read d3/d140/f144 [404844,87101] 0 2024-03-22T10:42:58.067 INFO:tasks.workunit.client.0.smithi115.stdout:9/928: fsync d6/d8/d3c/d59/d68/da8/fb4 0 2024-03-22T10:42:58.068 INFO:tasks.workunit.client.0.smithi115.stdout:2/884: chown da/d25/d3d/d57/d7d/da1/ddc 1451 1 2024-03-22T10:42:58.071 INFO:tasks.workunit.client.0.smithi115.stdout:5/981: dread d2/d15/d1b/f116 [0,4194304] 0 2024-03-22T10:42:58.071 INFO:tasks.workunit.client.0.smithi115.stdout:1/999: dread d0/d8/d4d/d56/db6/ddc/d82/fe8 [0,4194304] 0 2024-03-22T10:42:58.071 INFO:tasks.workunit.client.0.smithi115.stdout:0/899: chown d3/da/d4c/d69/d6a 156222078 1 2024-03-22T10:42:58.073 INFO:tasks.workunit.client.0.smithi115.stdout:9/929: mkdir d6/d7/d145 0 2024-03-22T10:42:58.075 INFO:tasks.workunit.client.0.smithi115.stdout:3/994: read d3/d6d/db4/d107/f79 [371275,35737] 0 2024-03-22T10:42:58.076 INFO:tasks.workunit.client.0.smithi115.stdout:2/885: write da/d25/d3d/d7e/d90/d96/fee [356080,42641] 0 2024-03-22T10:42:58.076 INFO:tasks.workunit.client.1.smithi150.stdout:2/604: dwrite d12/d19/db6/da9/d67/f74 [0,4194304] 0 2024-03-22T10:42:58.078 INFO:tasks.workunit.client.0.smithi115.stdout:0/900: write d3/da/d4c/d74/d84/ff7 [2594537,68100] 0 2024-03-22T10:42:58.079 INFO:tasks.workunit.client.0.smithi115.stdout:9/930: mknod d6/d8/c146 0 2024-03-22T10:42:58.081 INFO:tasks.workunit.client.0.smithi115.stdout:5/982: creat d2/df/d13/d8b/d93/da6/d10e/f143 x:0 0 0 2024-03-22T10:42:58.083 INFO:tasks.workunit.client.0.smithi115.stdout:3/995: mkdir d3/d6d/db4/d107/daa/d14c 0 2024-03-22T10:42:58.083 INFO:tasks.workunit.client.0.smithi115.stdout:7/909: dwrite d2/d7/d12/d10/d23/d3d/f5a [0,4194304] 0 2024-03-22T10:42:58.085 INFO:tasks.workunit.client.0.smithi115.stdout:9/931: dread - d6/d7/d48/d70/f12a zero size 2024-03-22T10:42:58.088 INFO:tasks.workunit.client.0.smithi115.stdout:2/886: chown da/d25/d51/c61 1 1 2024-03-22T10:42:58.093 INFO:tasks.workunit.client.0.smithi115.stdout:7/910: chown d2/d7/d12/d10/d19/l73 114152 1 2024-03-22T10:42:58.098 INFO:tasks.workunit.client.0.smithi115.stdout:5/983: mknod d2/df/d3b/dfb/d127/d141/c144 0 2024-03-22T10:42:58.099 INFO:tasks.workunit.client.1.smithi150.stdout:2/605: dread d12/f37 [0,4194304] 0 2024-03-22T10:42:58.103 INFO:tasks.workunit.client.1.smithi150.stdout:2/606: creat d12/d19/db6/da9/d97/fc1 x:0 0 0 2024-03-22T10:42:58.106 INFO:tasks.workunit.client.0.smithi115.stdout:0/901: creat d3/d7/d1b/f11f x:0 0 0 2024-03-22T10:42:58.106 INFO:tasks.workunit.client.0.smithi115.stdout:5/984: chown d2/d15/d1b/d7c/d38/fd2 131684 1 2024-03-22T10:42:58.106 INFO:tasks.workunit.client.0.smithi115.stdout:7/911: mknod d2/d7/d12/d10/c138 0 2024-03-22T10:42:58.112 INFO:tasks.workunit.client.0.smithi115.stdout:7/912: chown d2/d7/d12/c2c 22069 1 2024-03-22T10:42:58.114 INFO:tasks.workunit.client.0.smithi115.stdout:0/902: dread d3/da/d4c/d69/f76 [0,4194304] 0 2024-03-22T10:42:58.118 INFO:tasks.workunit.client.1.smithi150.stdout:2/607: dwrite d12/d35/fb2 [0,4194304] 0 2024-03-22T10:42:58.119 INFO:tasks.workunit.client.0.smithi115.stdout:7/913: rmdir d2/d3a 39 2024-03-22T10:42:58.119 INFO:tasks.workunit.client.0.smithi115.stdout:2/887: dwrite da/d1c/f4c [4194304,4194304] 0 2024-03-22T10:42:58.120 INFO:tasks.workunit.client.1.smithi150.stdout:2/608: write d12/d19/d2d/f36 [2661356,26539] 0 2024-03-22T10:42:58.122 INFO:tasks.workunit.client.1.smithi150.stdout:2/609: dread - d12/d19/db6/da9/d61/d98/fbb zero size 2024-03-22T10:42:58.123 INFO:tasks.workunit.client.0.smithi115.stdout:0/903: creat d3/d7/d1c/d6c/d87/dd6/f120 x:0 0 0 2024-03-22T10:42:58.126 INFO:tasks.workunit.client.1.smithi150.stdout:2/610: mkdir d12/d19/db6/da9/d61/d98/dc2 0 2024-03-22T10:42:58.128 INFO:tasks.workunit.client.1.smithi150.stdout:2/611: write d12/d35/d7d/fbe [20807,93466] 0 2024-03-22T10:42:58.129 INFO:tasks.workunit.client.0.smithi115.stdout:2/888: unlink da/d25/f49 0 2024-03-22T10:42:58.131 INFO:tasks.workunit.client.0.smithi115.stdout:0/904: mkdir d3/d7/db7/d121 0 2024-03-22T10:42:58.131 INFO:tasks.workunit.client.1.smithi150.stdout:2/612: rename d12/d19/f34 to d12/d21/fc3 0 2024-03-22T10:42:58.132 INFO:tasks.workunit.client.1.smithi150.stdout:2/613: dread - d12/d19/db6/f9f zero size 2024-03-22T10:42:58.133 INFO:tasks.workunit.client.1.smithi150.stdout:2/614: stat d12/d19/db6/da9/d67/db4 0 2024-03-22T10:42:58.135 INFO:tasks.workunit.client.0.smithi115.stdout:7/914: truncate d2/d7/d12/d10/d19/d42/d46/d94/fa6 661596 0 2024-03-22T10:42:58.135 INFO:tasks.workunit.client.1.smithi150.stdout:2/615: fsync d12/d19/db6/da9/d61/f86 0 2024-03-22T10:42:58.139 INFO:tasks.workunit.client.0.smithi115.stdout:0/905: chown d3/d52/l8c 1 1 2024-03-22T10:42:58.140 INFO:tasks.workunit.client.0.smithi115.stdout:9/932: dwrite d6/d7/d19/d4e/f6d [0,4194304] 0 2024-03-22T10:42:58.140 INFO:tasks.workunit.client.0.smithi115.stdout:5/985: dwrite d2/d15/d1b/d7c/d38/d43/d49/f12e [0,4194304] 0 2024-03-22T10:42:58.142 INFO:tasks.workunit.client.0.smithi115.stdout:7/915: fsync d2/d62/fcb 0 2024-03-22T10:42:58.146 INFO:tasks.workunit.client.0.smithi115.stdout:2/889: truncate da/d1c/d67/d78/d86/db4/fc3 958858 0 2024-03-22T10:42:58.148 INFO:tasks.workunit.client.0.smithi115.stdout:3/996: dread d3/d89/fe5 [0,4194304] 0 2024-03-22T10:42:58.149 INFO:tasks.workunit.client.1.smithi150.stdout:2/616: dwrite d12/d35/d7d/fbe [0,4194304] 0 2024-03-22T10:42:58.150 INFO:tasks.workunit.client.1.smithi150.stdout:2/617: chown d12/d19 66120 1 2024-03-22T10:42:58.153 INFO:tasks.workunit.client.0.smithi115.stdout:0/906: creat d3/da/d4c/d69/dd1/f122 x:0 0 0 2024-03-22T10:42:58.154 INFO:tasks.workunit.client.1.smithi150.stdout:2/618: rmdir d12/d19/db6/da9/d61 39 2024-03-22T10:42:58.154 INFO:tasks.workunit.client.0.smithi115.stdout:9/933: mknod d6/d7/d48/d53/dc9/c147 0 2024-03-22T10:42:58.157 INFO:tasks.workunit.client.1.smithi150.stdout:2/619: write d12/d19/db6/f4d [1957156,110400] 0 2024-03-22T10:42:58.157 INFO:tasks.workunit.client.0.smithi115.stdout:5/986: creat d2/d15/d1b/d7c/d84/d13d/f145 x:0 0 0 2024-03-22T10:42:58.161 INFO:tasks.workunit.client.1.smithi150.stdout:2/620: stat d12/d19/db6/da9/d2c/c6a 0 2024-03-22T10:42:58.164 INFO:tasks.workunit.client.1.smithi150.stdout:2/621: symlink d12/d19/db6/da9/d97/lc4 0 2024-03-22T10:42:58.167 INFO:tasks.workunit.client.0.smithi115.stdout:2/890: write da/d25/d30/f6e [4852225,38785] 0 2024-03-22T10:42:58.167 INFO:tasks.workunit.client.0.smithi115.stdout:0/907: mknod d3/df0/dca/dce/c123 0 2024-03-22T10:42:58.168 INFO:tasks.workunit.client.0.smithi115.stdout:7/916: creat d2/d7/d12/d10/d19/d42/d76/f139 x:0 0 0 2024-03-22T10:42:58.169 INFO:tasks.workunit.client.1.smithi150.stdout:2/622: symlink d12/d19/db6/da9/d2c/lc5 0 2024-03-22T10:42:58.169 INFO:tasks.workunit.client.0.smithi115.stdout:5/987: rename d2/d15/d1b/d7c/d38/f3d to d2/df/d13/d8b/d93/da6/dbc/f146 0 2024-03-22T10:42:58.172 INFO:tasks.workunit.client.0.smithi115.stdout:2/891: rmdir da/db1 39 2024-03-22T10:42:58.175 INFO:tasks.workunit.client.0.smithi115.stdout:6/874: sync 2024-03-22T10:42:58.175 INFO:tasks.workunit.client.0.smithi115.stdout:0/908: dread d3/d4/f60 [0,4194304] 0 2024-03-22T10:42:58.176 INFO:tasks.workunit.client.1.smithi150.stdout:6/523: sync 2024-03-22T10:42:58.178 INFO:tasks.workunit.client.1.smithi150.stdout:6/524: dread - db/dd/d15/d60/d50/f64 zero size 2024-03-22T10:42:58.179 INFO:tasks.workunit.client.0.smithi115.stdout:2/892: creat da/d25/d51/d100/f10b x:0 0 0 2024-03-22T10:42:58.180 INFO:tasks.workunit.client.1.smithi150.stdout:6/525: fdatasync db/dd/d15/d57/f65 0 2024-03-22T10:42:58.181 INFO:tasks.workunit.client.0.smithi115.stdout:5/988: truncate d2/d15/d1b/d7c/d38/f61 86150 0 2024-03-22T10:42:58.182 INFO:tasks.workunit.client.0.smithi115.stdout:0/909: truncate d3/d4/d6d/f9c 5172215 0 2024-03-22T10:42:58.183 INFO:tasks.workunit.client.0.smithi115.stdout:6/875: mkdir d4/d13/d33/d103/d11f 0 2024-03-22T10:42:58.184 INFO:tasks.workunit.client.1.smithi150.stdout:2/623: dwrite d12/d19/db6/da9/f2a [0,4194304] 0 2024-03-22T10:42:58.185 INFO:tasks.workunit.client.0.smithi115.stdout:5/989: fsync d2/df/d13/d8b/d93/da6/dbc/fd0 0 2024-03-22T10:42:58.185 INFO:tasks.workunit.client.0.smithi115.stdout:2/893: symlink da/d1c/l10c 0 2024-03-22T10:42:58.186 INFO:tasks.workunit.client.0.smithi115.stdout:0/910: dread - d3/d7/d1b/d1f/d78/dd9/fe3 zero size 2024-03-22T10:42:58.188 INFO:tasks.workunit.client.1.smithi150.stdout:2/624: creat d12/d19/db6/da9/d67/db4/fc6 x:0 0 0 2024-03-22T10:42:58.188 INFO:tasks.workunit.client.1.smithi150.stdout:6/526: dread db/dd/d15/d28/f41 [0,4194304] 0 2024-03-22T10:42:58.190 INFO:tasks.workunit.client.0.smithi115.stdout:6/876: symlink d4/d36/d48/d72/l120 0 2024-03-22T10:42:58.191 INFO:tasks.workunit.client.0.smithi115.stdout:5/990: creat d2/df/d13/d8b/d93/da6/d10e/d106/f147 x:0 0 0 2024-03-22T10:42:58.193 INFO:tasks.workunit.client.0.smithi115.stdout:2/894: write da/d25/d3d/d57/f85 [4262964,116866] 0 2024-03-22T10:42:58.195 INFO:tasks.workunit.client.0.smithi115.stdout:5/991: chown d2/d15/c1a 1917416 1 2024-03-22T10:42:58.196 INFO:tasks.workunit.client.0.smithi115.stdout:0/911: creat d3/d7/db7/d121/f124 x:0 0 0 2024-03-22T10:42:58.203 INFO:tasks.workunit.client.0.smithi115.stdout:0/912: symlink d3/da/d4c/da3/l125 0 2024-03-22T10:42:58.205 INFO:tasks.workunit.client.1.smithi150.stdout:6/527: dwrite db/dd/fa4 [0,4194304] 0 2024-03-22T10:42:58.207 INFO:tasks.workunit.client.1.smithi150.stdout:6/528: chown db/dd/d15/d30/fab 5275 1 2024-03-22T10:42:58.208 INFO:tasks.workunit.client.1.smithi150.stdout:6/529: chown db/dd/d82/d84/f96 2370060 1 2024-03-22T10:42:58.214 INFO:tasks.workunit.client.1.smithi150.stdout:2/625: write d12/d19/db6/da9/d2c/d65/f9d [789752,65364] 0 2024-03-22T10:42:58.215 INFO:tasks.workunit.client.1.smithi150.stdout:6/530: creat db/dd/d2e/fbc x:0 0 0 2024-03-22T10:42:58.216 INFO:tasks.workunit.client.1.smithi150.stdout:2/626: fdatasync d12/d19/db6/da9/d67/f5a 0 2024-03-22T10:42:58.217 INFO:tasks.workunit.client.1.smithi150.stdout:6/531: dread - db/dd/faa zero size 2024-03-22T10:42:58.218 INFO:tasks.workunit.client.0.smithi115.stdout:3/997: dread d3/d6d/db4/d107/fac [0,4194304] 0 2024-03-22T10:42:58.220 INFO:tasks.workunit.client.0.smithi115.stdout:6/877: write d4/d11/d12/d1c/d8a/d98/d9f/fbb [419065,43778] 0 2024-03-22T10:42:58.220 INFO:tasks.workunit.client.0.smithi115.stdout:2/895: dwrite da/d25/d3d/d7e/d90/d96/fcb [0,4194304] 0 2024-03-22T10:42:58.221 INFO:tasks.workunit.client.0.smithi115.stdout:4/942: sync 2024-03-22T10:42:58.223 INFO:tasks.workunit.client.1.smithi150.stdout:6/532: getdents db/dd 0 2024-03-22T10:42:58.223 INFO:tasks.workunit.client.1.smithi150.stdout:2/627: rename d12/d19/db6/da9/d67/d3c/d75/c8a to d12/d19/db6/da9/d67/d8d/cc7 0 2024-03-22T10:42:58.225 INFO:tasks.workunit.client.1.smithi150.stdout:6/533: write db/dd/d15/f35 [2255359,21992] 0 2024-03-22T10:42:58.226 INFO:tasks.workunit.client.0.smithi115.stdout:2/896: fsync da/d25/d3d/d57/d7d/da1/f104 0 2024-03-22T10:42:58.228 INFO:tasks.workunit.client.0.smithi115.stdout:9/934: dread d6/d7/d48/df4/f11c [0,4194304] 0 2024-03-22T10:42:58.228 INFO:tasks.workunit.client.0.smithi115.stdout:7/917: dread d2/d7/d12/d10/f40 [0,4194304] 0 2024-03-22T10:42:58.228 INFO:tasks.workunit.client.1.smithi150.stdout:2/628: creat d12/d19/db6/da9/d67/d3c/d75/fc8 x:0 0 0 2024-03-22T10:42:58.230 INFO:tasks.workunit.client.1.smithi150.stdout:2/629: write d12/d19/f1d [604587,104251] 0 2024-03-22T10:42:58.231 INFO:tasks.workunit.client.0.smithi115.stdout:4/943: readlink d1/d66/d106/d35/lad 0 2024-03-22T10:42:58.231 INFO:tasks.workunit.client.0.smithi115.stdout:3/998: rmdir d3/d19/d2b/d49/d91/dad/d13d 39 2024-03-22T10:42:58.232 INFO:tasks.workunit.client.1.smithi150.stdout:6/534: link db/f44 db/dd/d15/d6a/d8d/fbd 0 2024-03-22T10:42:58.233 INFO:tasks.workunit.client.1.smithi150.stdout:2/630: mkdir d12/d35/dc9 0 2024-03-22T10:42:58.236 INFO:tasks.workunit.client.1.smithi150.stdout:6/535: creat db/dd/d2e/db9/fbe x:0 0 0 2024-03-22T10:42:58.236 INFO:tasks.workunit.client.0.smithi115.stdout:0/913: write d3/d7/d1c/d6c/d9a/fa5 [1618683,14981] 0 2024-03-22T10:42:58.238 INFO:tasks.workunit.client.1.smithi150.stdout:6/536: mknod db/cbf 0 2024-03-22T10:42:58.240 INFO:tasks.workunit.client.0.smithi115.stdout:9/935: mknod d6/d7/d20/d22/d23/c148 0 2024-03-22T10:42:58.241 INFO:tasks.workunit.client.1.smithi150.stdout:2/631: rename d12/d19/db6/da9/f42 to d12/d19/db6/da9/d67/d9a/fca 0 2024-03-22T10:42:58.246 INFO:tasks.workunit.client.0.smithi115.stdout:4/944: readlink d1/d30/d4c/d86/d9f/d75/d99/d98/l127 0 2024-03-22T10:42:58.246 INFO:tasks.workunit.client.0.smithi115.stdout:0/914: symlink d3/d4/d14/l126 0 2024-03-22T10:42:58.250 INFO:tasks.workunit.client.0.smithi115.stdout:4/945: rmdir d1/d30/d4c 39 2024-03-22T10:42:58.253 INFO:tasks.workunit.client.1.smithi150.stdout:2/632: dwrite d12/f26 [0,4194304] 0 2024-03-22T10:42:58.255 INFO:tasks.workunit.client.1.smithi150.stdout:2/633: stat d12/d19/db6/da9/d67/db4/fc6 0 2024-03-22T10:42:58.258 INFO:tasks.workunit.client.0.smithi115.stdout:3/999: write d3/d82/fa4 [1106275,35012] 0 2024-03-22T10:42:58.258 INFO:tasks.workunit.client.0.smithi115.stdout:4/946: read d1/d66/f6d [724649,45443] 0 2024-03-22T10:42:58.262 INFO:tasks.workunit.client.1.smithi150.stdout:2/634: rename d12/d19/db6/f9f to d12/d19/db6/da9/d97/fcb 0 2024-03-22T10:42:58.264 INFO:tasks.workunit.client.1.smithi150.stdout:2/635: symlink d12/d19/lcc 0 2024-03-22T10:42:58.268 INFO:tasks.workunit.client.1.smithi150.stdout:2/636: rename d12/d19/db6/ca0 to d12/d19/db6/da9/d67/d8d/ccd 0 2024-03-22T10:42:58.269 INFO:tasks.workunit.client.0.smithi115.stdout:4/947: write d1/d66/d106/d35/d40/f48 [664211,10065] 0 2024-03-22T10:42:58.271 INFO:tasks.workunit.client.1.smithi150.stdout:2/637: creat d12/d19/db6/da9/d67/d3c/fce x:0 0 0 2024-03-22T10:42:58.274 INFO:tasks.workunit.client.0.smithi115.stdout:6/878: dwrite d4/d11/d12/d1c/d8a/d98/fc8 [0,4194304] 0 2024-03-22T10:42:58.275 INFO:tasks.workunit.client.0.smithi115.stdout:5/992: dread d2/df/d3b/f36 [0,4194304] 0 2024-03-22T10:42:58.277 INFO:tasks.workunit.client.0.smithi115.stdout:7/918: dwrite d2/d7/d12/d10/d23/d3d/f2a [4194304,4194304] 0 2024-03-22T10:42:58.282 INFO:tasks.workunit.client.0.smithi115.stdout:4/948: write d1/d30/d4c/d86/d9f/db8/f121 [1033565,50655] 0 2024-03-22T10:42:58.282 INFO:tasks.workunit.client.0.smithi115.stdout:9/936: dwrite d6/fdd [0,4194304] 0 2024-03-22T10:42:58.286 INFO:tasks.workunit.client.0.smithi115.stdout:6/879: mkdir d4/d11/d121 0 2024-03-22T10:42:58.286 INFO:tasks.workunit.client.0.smithi115.stdout:7/919: mkdir d2/d7/d114/d13a 0 2024-03-22T10:42:58.288 INFO:tasks.workunit.client.0.smithi115.stdout:5/993: mkdir d2/df/d13/d8b/d93/d148 0 2024-03-22T10:42:58.289 INFO:tasks.workunit.client.0.smithi115.stdout:6/880: write d4/d13/d33/dbe/fd3 [309945,87544] 0 2024-03-22T10:42:58.295 INFO:tasks.workunit.client.1.smithi150.stdout:2/638: truncate d12/d19/db6/da9/f2a 3253718 0 2024-03-22T10:42:58.296 INFO:tasks.workunit.client.0.smithi115.stdout:9/937: mkdir d6/d7/d30/d3b/d149 0 2024-03-22T10:42:58.296 INFO:tasks.workunit.client.1.smithi150.stdout:2/639: chown d12/d19/db6/da9/d67/d3c/f92 712577 1 2024-03-22T10:42:58.299 INFO:tasks.workunit.client.1.smithi150.stdout:2/640: chown d12/d19/db6/da9/l17 129 1 2024-03-22T10:42:58.302 INFO:tasks.workunit.client.1.smithi150.stdout:2/641: chown d12/d19/db6/da9/d67/c5c 114757222 1 2024-03-22T10:42:58.305 INFO:tasks.workunit.client.1.smithi150.stdout:2/642: creat d12/d19/d2d/fcf x:0 0 0 2024-03-22T10:42:58.307 INFO:tasks.workunit.client.1.smithi150.stdout:2/643: fsync d12/d19/db6/da9/d67/d3c/fce 0 2024-03-22T10:42:58.307 INFO:tasks.workunit.client.0.smithi115.stdout:7/920: read d2/d7/d12/d10/d19/d42/d76/f83 [1530819,87199] 0 2024-03-22T10:42:58.307 INFO:tasks.workunit.client.0.smithi115.stdout:5/994: rmdir d2/df/d13/d8b/d93/da6/d10e/d106/d10b 39 2024-03-22T10:42:58.308 INFO:tasks.workunit.client.1.smithi150.stdout:2/644: chown d12/d21/f4b 7454609 1 2024-03-22T10:42:58.310 INFO:tasks.workunit.client.1.smithi150.stdout:2/645: mknod d12/d19/db6/da9/d67/d8d/da6/cd0 0 2024-03-22T10:42:58.312 INFO:tasks.workunit.client.1.smithi150.stdout:2/646: write d12/d19/db6/f79 [2556866,48625] 0 2024-03-22T10:42:58.314 INFO:tasks.workunit.client.1.smithi150.stdout:2/647: fdatasync d12/d19/db6/da9/d97/fc1 0 2024-03-22T10:42:58.316 INFO:tasks.workunit.client.0.smithi115.stdout:5/995: write d2/df/d13/f18 [1855201,38621] 0 2024-03-22T10:42:58.316 INFO:tasks.workunit.client.0.smithi115.stdout:2/897: dread da/d25/d30/f31 [0,4194304] 0 2024-03-22T10:42:58.316 INFO:tasks.workunit.client.0.smithi115.stdout:0/915: dread d3/d7/d1c/f1e [0,4194304] 0 2024-03-22T10:42:58.318 INFO:tasks.workunit.client.0.smithi115.stdout:2/898: write da/d25/d3d/d57/f5d [3691502,126219] 0 2024-03-22T10:42:58.319 INFO:tasks.workunit.client.0.smithi115.stdout:0/916: chown d3/d7/d1b/c32 139639 1 2024-03-22T10:42:58.320 INFO:tasks.workunit.client.1.smithi150.stdout:2/648: dread d12/d19/db6/d54/f5b [0,4194304] 0 2024-03-22T10:42:58.326 INFO:tasks.workunit.client.0.smithi115.stdout:4/949: write d1/d66/d82/da1/fd3 [1491651,76174] 0 2024-03-22T10:42:58.328 INFO:tasks.workunit.client.1.smithi150.stdout:2/649: chown d12/d19/db6/da9/d61/c6d 10392 1 2024-03-22T10:42:58.330 INFO:tasks.workunit.client.1.smithi150.stdout:2/650: mknod d12/d19/db6/d54/cd1 0 2024-03-22T10:42:58.333 INFO:tasks.workunit.client.0.smithi115.stdout:9/938: dwrite d6/d7/d30/d37/dbe/d110/f140 [0,4194304] 0 2024-03-22T10:42:58.335 INFO:tasks.workunit.client.0.smithi115.stdout:9/939: dread - d6/d7/d48/d70/f12a zero size 2024-03-22T10:42:58.337 INFO:tasks.workunit.client.0.smithi115.stdout:9/940: readlink d6/d7/d30/d3b/lf9 0 2024-03-22T10:42:58.342 INFO:tasks.workunit.client.0.smithi115.stdout:9/941: write d6/d7/d48/faf [4111372,31204] 0 2024-03-22T10:42:58.343 INFO:tasks.workunit.client.1.smithi150.stdout:2/651: dread d12/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:42:58.348 INFO:tasks.workunit.client.1.smithi150.stdout:3/533: sync 2024-03-22T10:42:58.348 INFO:tasks.workunit.client.1.smithi150.stdout:8/589: sync 2024-03-22T10:42:58.348 INFO:tasks.workunit.client.1.smithi150.stdout:7/573: sync 2024-03-22T10:42:58.348 INFO:tasks.workunit.client.1.smithi150.stdout:4/566: sync 2024-03-22T10:42:58.348 INFO:tasks.workunit.client.1.smithi150.stdout:1/511: sync 2024-03-22T10:42:58.349 INFO:tasks.workunit.client.0.smithi115.stdout:5/996: rename d2/d15/d1b/d7c/d38/d56/d70 to d2/d15/d1b/d7c/d38/d135/d149 0 2024-03-22T10:42:58.350 INFO:tasks.workunit.client.1.smithi150.stdout:2/652: truncate d12/d19/db6/da9/d67/d9a/fca 2454176 0 2024-03-22T10:42:58.352 INFO:tasks.workunit.client.1.smithi150.stdout:3/534: mknod d5/d53/d7e/cb0 0 2024-03-22T10:42:58.352 INFO:tasks.workunit.client.0.smithi115.stdout:4/950: creat d1/d66/d106/d35/d40/d54/d5e/f143 x:0 0 0 2024-03-22T10:42:58.353 INFO:tasks.workunit.client.1.smithi150.stdout:1/512: symlink d1/d19/d50/la7 0 2024-03-22T10:42:58.356 INFO:tasks.workunit.client.1.smithi150.stdout:2/653: rmdir d12/d19/db6/d63 39 2024-03-22T10:42:58.357 INFO:tasks.workunit.client.1.smithi150.stdout:8/590: symlink d0/d2/d19/d29/d2f/df/lc8 0 2024-03-22T10:42:58.358 INFO:tasks.workunit.client.0.smithi115.stdout:6/881: getdents d4/d13/d33 0 2024-03-22T10:42:58.358 INFO:tasks.workunit.client.1.smithi150.stdout:1/513: write d1/fa6 [60592,68696] 0 2024-03-22T10:42:58.358 INFO:tasks.workunit.client.1.smithi150.stdout:7/574: creat d2/d5/d7/d9/db/da2/fbe x:0 0 0 2024-03-22T10:42:58.359 INFO:tasks.workunit.client.0.smithi115.stdout:9/942: creat d6/d7/d30/f14a x:0 0 0 2024-03-22T10:42:58.360 INFO:tasks.workunit.client.1.smithi150.stdout:2/654: symlink d12/d19/d1a/ld2 0 2024-03-22T10:42:58.363 INFO:tasks.workunit.client.0.smithi115.stdout:4/951: getdents d1/d69 0 2024-03-22T10:42:58.363 INFO:tasks.workunit.client.1.smithi150.stdout:8/591: mknod d0/d2/d19/dc0/cc9 0 2024-03-22T10:42:58.364 INFO:tasks.workunit.client.0.smithi115.stdout:6/882: creat d4/d11/d12/d1c/d8a/d98/f122 x:0 0 0 2024-03-22T10:42:58.366 INFO:tasks.workunit.client.1.smithi150.stdout:2/655: chown d12/d21/d81/lbf 415873 1 2024-03-22T10:42:58.366 INFO:tasks.workunit.client.1.smithi150.stdout:7/575: mkdir d2/d5/d7/d9/db/d9f/d96/dbf 0 2024-03-22T10:42:58.367 INFO:tasks.workunit.client.0.smithi115.stdout:9/943: fdatasync d6/d8/d3c/d59/f90 0 2024-03-22T10:42:58.367 INFO:tasks.workunit.client.1.smithi150.stdout:2/656: chown d12/d19/l24 3 1 2024-03-22T10:42:58.368 INFO:tasks.workunit.client.0.smithi115.stdout:6/883: chown d4/d13/d33/l87 852421456 1 2024-03-22T10:42:58.369 INFO:tasks.workunit.client.1.smithi150.stdout:2/657: chown d12/d19/d1a 53 1 2024-03-22T10:42:58.373 INFO:tasks.workunit.client.1.smithi150.stdout:2/658: fsync d12/d19/db6/f4d 0 2024-03-22T10:42:58.373 INFO:tasks.workunit.client.1.smithi150.stdout:7/576: creat d2/d5/d7/d9/db/d6c/d9c/fc0 x:0 0 0 2024-03-22T10:42:58.374 INFO:tasks.workunit.client.1.smithi150.stdout:3/535: dwrite d5/fd [0,4194304] 0 2024-03-22T10:42:58.376 INFO:tasks.workunit.client.1.smithi150.stdout:2/659: readlink d12/d19/db6/da9/d2c/d65/la3 0 2024-03-22T10:42:58.376 INFO:tasks.workunit.client.0.smithi115.stdout:5/997: dwrite d2/d15/d1b/d7c/d38/d135/dbd/f11e [0,4194304] 0 2024-03-22T10:42:58.376 INFO:tasks.workunit.client.1.smithi150.stdout:7/577: dread - d2/d5/d7/d9/db/d6c/fb6 zero size 2024-03-22T10:42:58.379 INFO:tasks.workunit.client.0.smithi115.stdout:7/921: write d2/d7/d12/d10/d19/d42/d76/f83 [2103066,23265] 0 2024-03-22T10:42:58.380 INFO:tasks.workunit.client.1.smithi150.stdout:4/567: write d0/d32/d7c/d99/fab [3632831,21023] 0 2024-03-22T10:42:58.382 INFO:tasks.workunit.client.1.smithi150.stdout:1/514: write d1/d19/d71/f8b [1072369,84361] 0 2024-03-22T10:42:58.383 INFO:tasks.workunit.client.1.smithi150.stdout:6/537: sync 2024-03-22T10:42:58.383 INFO:tasks.workunit.client.1.smithi150.stdout:0/552: sync 2024-03-22T10:42:58.383 INFO:tasks.workunit.client.1.smithi150.stdout:9/571: sync 2024-03-22T10:42:58.383 INFO:tasks.workunit.client.1.smithi150.stdout:5/483: sync 2024-03-22T10:42:58.384 INFO:tasks.workunit.client.0.smithi115.stdout:2/899: dread da/f15 [0,4194304] 0 2024-03-22T10:42:58.386 INFO:tasks.workunit.client.0.smithi115.stdout:0/917: truncate d3/d7/d1b/d93/fcc 19566 0 2024-03-22T10:42:58.386 INFO:tasks.workunit.client.0.smithi115.stdout:2/900: chown da/da5 27 1 2024-03-22T10:42:58.387 INFO:tasks.workunit.client.1.smithi150.stdout:2/660: mknod d12/d35/dc9/cd3 0 2024-03-22T10:42:58.388 INFO:tasks.workunit.client.1.smithi150.stdout:1/515: chown d1/d19/d50/f5c 21559 1 2024-03-22T10:42:58.389 INFO:tasks.workunit.client.0.smithi115.stdout:5/998: symlink d2/df/d13/d8b/d93/da6/d10e/l14a 0 2024-03-22T10:42:58.391 INFO:tasks.workunit.client.1.smithi150.stdout:5/484: fsync d0/f5 0 2024-03-22T10:42:58.391 INFO:tasks.workunit.client.1.smithi150.stdout:8/592: write d0/d2/d7e/d96/f99 [925315,53039] 0 2024-03-22T10:42:58.392 INFO:tasks.workunit.client.0.smithi115.stdout:7/922: truncate d2/d7/d12/d10/d19/dbd/f124 4872811 0 2024-03-22T10:42:58.393 INFO:tasks.workunit.client.1.smithi150.stdout:9/572: mkdir d1/d8/de/dc8 0 2024-03-22T10:42:58.394 INFO:tasks.workunit.client.0.smithi115.stdout:2/901: chown da/d25/d3d/d57/d7d 23995650 1 2024-03-22T10:42:58.394 INFO:tasks.workunit.client.1.smithi150.stdout:4/568: read d0/d1b/f39 [2477275,21582] 0 2024-03-22T10:42:58.395 INFO:tasks.workunit.client.0.smithi115.stdout:4/952: dwrite d1/d30/d78/fb4 [0,4194304] 0 2024-03-22T10:42:58.395 INFO:tasks.workunit.client.0.smithi115.stdout:0/918: creat d3/f127 x:0 0 0 2024-03-22T10:42:58.396 INFO:tasks.workunit.client.1.smithi150.stdout:7/578: truncate d2/d5/d7/d8/d46/f66 5483155 0 2024-03-22T10:42:58.396 INFO:tasks.workunit.client.1.smithi150.stdout:2/661: unlink d12/d19/db6/f79 0 2024-03-22T10:42:58.398 INFO:tasks.workunit.client.0.smithi115.stdout:9/944: write d6/d7/d30/d37/dbe/d110/f139 [1974752,126931] 0 2024-03-22T10:42:58.402 INFO:tasks.workunit.client.0.smithi115.stdout:6/884: truncate d4/d5e/f8b 1646236 0 2024-03-22T10:42:58.403 INFO:tasks.workunit.client.0.smithi115.stdout:0/919: write d3/f127 [757408,114373] 0 2024-03-22T10:42:58.406 INFO:tasks.workunit.client.0.smithi115.stdout:7/923: chown d2/d7/d12/d10/d23/d3d/dd2/fba 2751 1 2024-03-22T10:42:58.407 INFO:tasks.workunit.client.0.smithi115.stdout:5/999: truncate d2/d15/d1b/d7c/d84/f97 364844 0 2024-03-22T10:42:58.410 INFO:tasks.workunit.client.0.smithi115.stdout:2/902: mkdir da/d25/d30/d10d 0 2024-03-22T10:42:58.415 INFO:tasks.workunit.client.1.smithi150.stdout:1/516: getdents d1/d7c 0 2024-03-22T10:42:58.417 INFO:tasks.workunit.client.1.smithi150.stdout:9/573: rename d1/d8/de/df/d20/l34 to d1/d8/de/df/da2/lc9 0 2024-03-22T10:42:58.421 INFO:tasks.workunit.client.0.smithi115.stdout:2/903: creat da/d25/dfb/f10e x:0 0 0 2024-03-22T10:42:58.424 INFO:tasks.workunit.client.1.smithi150.stdout:6/538: dwrite db/dd/d15/f35 [0,4194304] 0 2024-03-22T10:42:58.425 INFO:tasks.workunit.client.0.smithi115.stdout:4/953: dread d1/d66/d106/d35/d40/d5c/f6f [0,4194304] 0 2024-03-22T10:42:58.429 INFO:tasks.workunit.client.1.smithi150.stdout:0/553: dwrite d0/d1/d84/d86/d42/f77 [0,4194304] 0 2024-03-22T10:42:58.429 INFO:tasks.workunit.client.0.smithi115.stdout:2/904: link da/d25/d3d/d57/f65 da/d25/d3d/d57/d7d/da1/f10f 0 2024-03-22T10:42:58.429 INFO:tasks.workunit.client.0.smithi115.stdout:4/954: write d1/d66/d106/d35/d40/d54/d5e/df3/f132 [147440,95022] 0 2024-03-22T10:42:58.432 INFO:tasks.workunit.client.0.smithi115.stdout:4/955: mknod d1/d66/d106/d3b/c144 0 2024-03-22T10:42:58.432 INFO:tasks.workunit.client.1.smithi150.stdout:5/485: write d0/d8/dc/d1d/f24 [90138,109776] 0 2024-03-22T10:42:58.433 INFO:tasks.workunit.client.1.smithi150.stdout:4/569: dwrite d0/d1b/f24 [8388608,4194304] 0 2024-03-22T10:42:58.435 INFO:tasks.workunit.client.0.smithi115.stdout:2/905: unlink da/d25/d2a/c3a 0 2024-03-22T10:42:58.439 INFO:tasks.workunit.client.1.smithi150.stdout:6/539: stat db/dd/d25/f6d 0 2024-03-22T10:42:58.445 INFO:tasks.workunit.client.1.smithi150.stdout:8/593: dwrite d0/d3c/f44 [4194304,4194304] 0 2024-03-22T10:42:58.447 INFO:tasks.workunit.client.1.smithi150.stdout:5/486: chown d0/d8/dc/de/d14/d4f/f3d 6439 1 2024-03-22T10:42:58.449 INFO:tasks.workunit.client.1.smithi150.stdout:5/487: readlink d0/d8/dc/de/l4a 0 2024-03-22T10:42:58.451 INFO:tasks.workunit.client.1.smithi150.stdout:9/574: fdatasync d1/d8/de/df/d36/f5c 0 2024-03-22T10:42:58.452 INFO:tasks.workunit.client.1.smithi150.stdout:8/594: chown d0/d2/d19/d29/d69 302906 1 2024-03-22T10:42:58.456 INFO:tasks.workunit.client.1.smithi150.stdout:4/570: rmdir d0/d1b/d4d/d6c 39 2024-03-22T10:42:58.459 INFO:tasks.workunit.client.1.smithi150.stdout:9/575: read d1/d8/de/df/da2/f91 [968963,9566] 0 2024-03-22T10:42:58.461 INFO:tasks.workunit.client.1.smithi150.stdout:1/517: getdents d1/d19/d71 0 2024-03-22T10:42:58.461 INFO:tasks.workunit.client.1.smithi150.stdout:9/576: chown d1/c7 2 1 2024-03-22T10:42:58.462 INFO:tasks.workunit.client.1.smithi150.stdout:3/536: dread d5/da/ff [0,4194304] 0 2024-03-22T10:42:58.463 INFO:tasks.workunit.client.1.smithi150.stdout:8/595: symlink d0/d3c/d71/d7a/d89/db5/lca 0 2024-03-22T10:42:58.464 INFO:tasks.workunit.client.1.smithi150.stdout:4/571: read - d0/d2f/f52 zero size 2024-03-22T10:42:58.466 INFO:tasks.workunit.client.1.smithi150.stdout:0/554: getdents d0 0 2024-03-22T10:42:58.469 INFO:tasks.workunit.client.1.smithi150.stdout:9/577: mkdir d1/d19/d7d/dc1/dca 0 2024-03-22T10:42:58.470 INFO:tasks.workunit.client.1.smithi150.stdout:3/537: mknod d5/da/d62/cb1 0 2024-03-22T10:42:58.471 INFO:tasks.workunit.client.1.smithi150.stdout:0/555: stat d0/d12/l7f 0 2024-03-22T10:42:58.474 INFO:tasks.workunit.client.1.smithi150.stdout:4/572: stat d0/d1b/d4d/d6c/d61/f9e 0 2024-03-22T10:42:58.477 INFO:tasks.workunit.client.1.smithi150.stdout:3/538: creat d5/d53/d7e/d95/fb2 x:0 0 0 2024-03-22T10:42:58.478 INFO:tasks.workunit.client.1.smithi150.stdout:7/579: write d2/d5/d7/d8/f5e [2318777,38770] 0 2024-03-22T10:42:58.482 INFO:tasks.workunit.client.1.smithi150.stdout:8/596: read d0/d2/f5f [3250970,130788] 0 2024-03-22T10:42:58.482 INFO:tasks.workunit.client.1.smithi150.stdout:5/488: dwrite d0/d8/dc/de/d14/d2e/d5a/f8e [0,4194304] 0 2024-03-22T10:42:58.483 INFO:tasks.workunit.client.0.smithi115.stdout:6/885: truncate d4/d11/d12/f31 1667939 0 2024-03-22T10:42:58.485 INFO:tasks.workunit.client.1.smithi150.stdout:3/539: fsync d5/f3c 0 2024-03-22T10:42:58.486 INFO:tasks.workunit.client.0.smithi115.stdout:2/906: chown da/ca9 27129095 1 2024-03-22T10:42:58.488 INFO:tasks.workunit.client.1.smithi150.stdout:0/556: rmdir d0/d1/d84/d86/da6/dae 0 2024-03-22T10:42:58.491 INFO:tasks.workunit.client.0.smithi115.stdout:6/886: creat d4/d5e/f123 x:0 0 0 2024-03-22T10:42:58.491 INFO:tasks.workunit.client.0.smithi115.stdout:4/956: truncate d1/dbb/dc6/dd8/df7/f100 596867 0 2024-03-22T10:42:58.492 INFO:tasks.workunit.client.1.smithi150.stdout:7/580: write d2/d5/d7/d9/db/d6c/fb6 [549721,61260] 0 2024-03-22T10:42:58.495 INFO:tasks.workunit.client.1.smithi150.stdout:5/489: creat d0/d8/dc/d1d/d89/f8f x:0 0 0 2024-03-22T10:42:58.498 INFO:tasks.workunit.client.1.smithi150.stdout:2/662: dwrite d12/d21/fb3 [0,4194304] 0 2024-03-22T10:42:58.500 INFO:tasks.workunit.client.0.smithi115.stdout:6/887: write d4/d11/d12/d1c/d8a/f100 [246882,107401] 0 2024-03-22T10:42:58.501 INFO:tasks.workunit.client.1.smithi150.stdout:8/597: creat d0/d2/d19/d29/d37/d7d/d9e/fcb x:0 0 0 2024-03-22T10:42:58.502 INFO:tasks.workunit.client.0.smithi115.stdout:9/945: dwrite d6/d7/d20/d22/f2b [0,4194304] 0 2024-03-22T10:42:58.503 INFO:tasks.workunit.client.1.smithi150.stdout:9/578: dread d1/d8/f82 [0,4194304] 0 2024-03-22T10:42:58.504 INFO:tasks.workunit.client.0.smithi115.stdout:7/924: dwrite d2/d7/d12/d10/d23/d3d/dd2/f92 [0,4194304] 0 2024-03-22T10:42:58.509 INFO:tasks.workunit.client.1.smithi150.stdout:1/518: truncate d1/da/d15/f3e 600018 0 2024-03-22T10:42:58.512 INFO:tasks.workunit.client.0.smithi115.stdout:4/957: mknod d1/d30/d4c/d86/d9f/db8/c145 0 2024-03-22T10:42:58.512 INFO:tasks.workunit.client.0.smithi115.stdout:6/888: symlink d4/d11/d121/l124 0 2024-03-22T10:42:58.515 INFO:tasks.workunit.client.1.smithi150.stdout:4/573: dwrite d0/d1b/d4d/f92 [0,4194304] 0 2024-03-22T10:42:58.516 INFO:tasks.workunit.client.1.smithi150.stdout:6/540: dwrite db/dd/d25/fa5 [0,4194304] 0 2024-03-22T10:42:58.519 INFO:tasks.workunit.client.1.smithi150.stdout:6/541: chown db/dd/d25/c56 3236 1 2024-03-22T10:42:58.522 INFO:tasks.workunit.client.0.smithi115.stdout:6/889: fsync d4/d11/d91/fd5 0 2024-03-22T10:42:58.527 INFO:tasks.workunit.client.0.smithi115.stdout:6/890: dread - d4/d13/d33/f10d zero size 2024-03-22T10:42:58.529 INFO:tasks.workunit.client.1.smithi150.stdout:1/519: unlink d1/da/d2a/c2c 0 2024-03-22T10:42:58.532 INFO:tasks.workunit.client.0.smithi115.stdout:7/925: creat d2/d7/d12/d10/d19/f13b x:0 0 0 2024-03-22T10:42:58.535 INFO:tasks.workunit.client.0.smithi115.stdout:6/891: stat d4/d36/d48/d72/daa/db9/fd7 0 2024-03-22T10:42:58.541 INFO:tasks.workunit.client.1.smithi150.stdout:9/579: unlink d1/d8/de/df/d36/d40/d41/lb7 0 2024-03-22T10:42:58.547 INFO:tasks.workunit.client.0.smithi115.stdout:2/907: dwrite da/d25/f34 [0,4194304] 0 2024-03-22T10:42:58.547 INFO:tasks.workunit.client.0.smithi115.stdout:6/892: link d4/d11/d12/d1c/d8a/d98/d9f/ff6 d4/d11/d121/f125 0 2024-03-22T10:42:58.548 INFO:tasks.workunit.client.1.smithi150.stdout:2/663: dwrite d12/d19/f1d [0,4194304] 0 2024-03-22T10:42:58.548 INFO:tasks.workunit.client.1.smithi150.stdout:1/520: unlink d1/l6 0 2024-03-22T10:42:58.551 INFO:tasks.workunit.client.1.smithi150.stdout:0/557: dwrite d0/d1/d84/d86/d42/f2b [0,4194304] 0 2024-03-22T10:42:58.552 INFO:tasks.workunit.client.0.smithi115.stdout:7/926: dwrite d2/d7/d12/d10/d19/f110 [0,4194304] 0 2024-03-22T10:42:58.554 INFO:tasks.workunit.client.0.smithi115.stdout:2/908: unlink da/d1c/d67/f6f 0 2024-03-22T10:42:58.555 INFO:tasks.workunit.client.0.smithi115.stdout:6/893: mknod d4/d13/c126 0 2024-03-22T10:42:58.555 INFO:tasks.workunit.client.0.smithi115.stdout:2/909: chown da/d1c/d67/d78 1002050351 1 2024-03-22T10:42:58.556 INFO:tasks.workunit.client.1.smithi150.stdout:0/558: fsync d0/f56 0 2024-03-22T10:42:58.557 INFO:tasks.workunit.client.1.smithi150.stdout:6/542: getdents db/d52/d9f 0 2024-03-22T10:42:58.561 INFO:tasks.workunit.client.0.smithi115.stdout:2/910: mknod da/d25/d3d/dad/db3/c110 0 2024-03-22T10:42:58.564 INFO:tasks.workunit.client.0.smithi115.stdout:6/894: write d4/d13/d33/f6c [1852875,100923] 0 2024-03-22T10:42:58.565 INFO:tasks.workunit.client.0.smithi115.stdout:7/927: fsync d2/d7/d12/d10/d19/d42/d76/d74/f77 0 2024-03-22T10:42:58.566 INFO:tasks.workunit.client.1.smithi150.stdout:0/559: write d0/f7b [4707118,50635] 0 2024-03-22T10:42:58.568 INFO:tasks.workunit.client.1.smithi150.stdout:2/664: creat d12/d19/db6/da9/d61/d98/fd4 x:0 0 0 2024-03-22T10:42:58.570 INFO:tasks.workunit.client.0.smithi115.stdout:2/911: creat da/d25/d3d/d57/da7/f111 x:0 0 0 2024-03-22T10:42:58.573 INFO:tasks.workunit.client.0.smithi115.stdout:6/895: rmdir d4/d13/d33/d58 39 2024-03-22T10:42:58.576 INFO:tasks.workunit.client.1.smithi150.stdout:7/581: dwrite d2/d5/d7/d8/d46/f74 [0,4194304] 0 2024-03-22T10:42:58.579 INFO:tasks.workunit.client.0.smithi115.stdout:2/912: mknod da/d25/d3d/d7e/c112 0 2024-03-22T10:42:58.581 INFO:tasks.workunit.client.0.smithi115.stdout:4/958: write d1/d30/d4c/d86/d9f/d4f/f10a [612344,29647] 0 2024-03-22T10:42:58.582 INFO:tasks.workunit.client.0.smithi115.stdout:9/946: write d6/d7/f134 [656955,123872] 0 2024-03-22T10:42:58.586 INFO:tasks.workunit.client.1.smithi150.stdout:9/580: dwrite d1/d19/d7d/dc1/fa0 [0,4194304] 0 2024-03-22T10:42:58.588 INFO:tasks.workunit.client.0.smithi115.stdout:7/928: getdents d2/d7/d12/d10/d19/d42/d76 0 2024-03-22T10:42:58.588 INFO:tasks.workunit.client.0.smithi115.stdout:2/913: fdatasync da/d25/d83/fd0 0 2024-03-22T10:42:58.589 INFO:tasks.workunit.client.1.smithi150.stdout:8/598: dread d0/d2/d19/d29/d2f/df/d1b/f1d [0,4194304] 0 2024-03-22T10:42:58.590 INFO:tasks.workunit.client.1.smithi150.stdout:0/560: creat d0/fb2 x:0 0 0 2024-03-22T10:42:58.590 INFO:tasks.workunit.client.1.smithi150.stdout:9/581: dread - d1/d19/d7d/fc2 zero size 2024-03-22T10:42:58.591 INFO:tasks.workunit.client.0.smithi115.stdout:4/959: read d1/d66/d82/feb [45102,54397] 0 2024-03-22T10:42:58.592 INFO:tasks.workunit.client.0.smithi115.stdout:2/914: chown da/d25/d3d/d57/d7d 13693 1 2024-03-22T10:42:58.594 INFO:tasks.workunit.client.1.smithi150.stdout:6/543: creat db/d52/d9f/fc0 x:0 0 0 2024-03-22T10:42:58.594 INFO:tasks.workunit.client.0.smithi115.stdout:0/920: sync 2024-03-22T10:42:58.594 INFO:tasks.workunit.client.0.smithi115.stdout:4/960: read - d1/d66/d82/ffe zero size 2024-03-22T10:42:58.597 INFO:tasks.workunit.client.1.smithi150.stdout:9/582: stat d1/d19/l27 0 2024-03-22T10:42:58.599 INFO:tasks.workunit.client.1.smithi150.stdout:4/574: dread d0/d1b/f23 [0,4194304] 0 2024-03-22T10:42:58.602 INFO:tasks.workunit.client.1.smithi150.stdout:5/490: truncate d0/d8/dc/de/d14/d4f/d47/f77 3543032 0 2024-03-22T10:42:58.603 INFO:tasks.workunit.client.0.smithi115.stdout:6/896: dwrite d4/d11/d91/fb0 [0,4194304] 0 2024-03-22T10:42:58.607 INFO:tasks.workunit.client.0.smithi115.stdout:4/961: rename d1/d30/d4c/d86/c87 to d1/d66/d106/d28/c146 0 2024-03-22T10:42:58.607 INFO:tasks.workunit.client.1.smithi150.stdout:7/582: dwrite d2/d5/d7/d9/f28 [0,4194304] 0 2024-03-22T10:42:58.609 INFO:tasks.workunit.client.0.smithi115.stdout:9/947: dwrite d6/d7/d30/d37/dbe/fd4 [0,4194304] 0 2024-03-22T10:42:58.610 INFO:tasks.workunit.client.1.smithi150.stdout:1/521: mknod d1/da/d15/d18/ca8 0 2024-03-22T10:42:58.611 INFO:tasks.workunit.client.0.smithi115.stdout:7/929: creat d2/d7/d12/d10/d23/d3d/f13c x:0 0 0 2024-03-22T10:42:58.612 INFO:tasks.workunit.client.1.smithi150.stdout:3/540: dread d5/d22/d37/f6c [0,4194304] 0 2024-03-22T10:42:58.617 INFO:tasks.workunit.client.0.smithi115.stdout:7/930: chown d2/f5 4195 1 2024-03-22T10:42:58.617 INFO:tasks.workunit.client.1.smithi150.stdout:8/599: unlink d0/d2/d19/d29/d37/da1/fa4 0 2024-03-22T10:42:58.618 INFO:tasks.workunit.client.1.smithi150.stdout:3/541: write d5/da/f40 [1842494,9775] 0 2024-03-22T10:42:58.618 INFO:tasks.workunit.client.1.smithi150.stdout:6/544: rename db/dd/d2e/db9 to db/dd/d15/d60/d50/dc1 0 2024-03-22T10:42:58.618 INFO:tasks.workunit.client.0.smithi115.stdout:9/948: symlink d6/d7/d6a/d6f/l14b 0 2024-03-22T10:42:58.622 INFO:tasks.workunit.client.1.smithi150.stdout:5/491: unlink d0/d8/dc/de/d14/d2e/f6d 0 2024-03-22T10:42:58.623 INFO:tasks.workunit.client.0.smithi115.stdout:7/931: chown d2/d7/d12/d10/d19/d42/d76/f83 199960 1 2024-03-22T10:42:58.625 INFO:tasks.workunit.client.1.smithi150.stdout:9/583: dread d1/d49/f31 [0,4194304] 0 2024-03-22T10:42:58.627 INFO:tasks.workunit.client.1.smithi150.stdout:7/583: write d2/d5/d7/d8/d46/d49/fae [342064,19226] 0 2024-03-22T10:42:58.627 INFO:tasks.workunit.client.0.smithi115.stdout:4/962: getdents d1/d30/d4c/d86/d9f/d75/d99/da8/dea 0 2024-03-22T10:42:58.630 INFO:tasks.workunit.client.1.smithi150.stdout:1/522: link d1/da/d2a/d55/c95 d1/d19/d50/ca9 0 2024-03-22T10:42:58.633 INFO:tasks.workunit.client.0.smithi115.stdout:9/949: getdents d6/d7/d30/d3b 0 2024-03-22T10:42:58.634 INFO:tasks.workunit.client.1.smithi150.stdout:6/545: symlink db/dd/d15/d60/d8b/lc2 0 2024-03-22T10:42:58.637 INFO:tasks.workunit.client.1.smithi150.stdout:7/584: read d2/d5/d7/d8/d57/d54/d91/fb3 [383059,4334] 0 2024-03-22T10:42:58.637 INFO:tasks.workunit.client.1.smithi150.stdout:6/546: dread - db/dd/d2e/fa6 zero size 2024-03-22T10:42:58.641 INFO:tasks.workunit.client.0.smithi115.stdout:7/932: rename d2/d7/d12/d10/d19/dbd/f124 to d2/d7/d12/d10/d23/d3d/da4/dc0/f13d 0 2024-03-22T10:42:58.642 INFO:tasks.workunit.client.1.smithi150.stdout:9/584: creat d1/d8/de/df/d36/fcb x:0 0 0 2024-03-22T10:42:58.645 INFO:tasks.workunit.client.0.smithi115.stdout:6/897: dwrite d4/d11/d12/d1c/d8a/f100 [0,4194304] 0 2024-03-22T10:42:58.645 INFO:tasks.workunit.client.0.smithi115.stdout:7/933: fdatasync d2/d7/d12/d10/d19/d42/d76/f98 0 2024-03-22T10:42:58.646 INFO:tasks.workunit.client.0.smithi115.stdout:9/950: unlink d6/d7/d30/d37/f4d 0 2024-03-22T10:42:58.647 INFO:tasks.workunit.client.1.smithi150.stdout:1/523: creat d1/da/d2a/faa x:0 0 0 2024-03-22T10:42:58.647 INFO:tasks.workunit.client.1.smithi150.stdout:9/585: readlink d1/d8/de/df/d36/d40/d41/l56 0 2024-03-22T10:42:58.650 INFO:tasks.workunit.client.0.smithi115.stdout:9/951: stat d6/d7/d20/d22/d3f/l45 0 2024-03-22T10:42:58.652 INFO:tasks.workunit.client.1.smithi150.stdout:1/524: mknod d1/d19/cab 0 2024-03-22T10:42:58.654 INFO:tasks.workunit.client.0.smithi115.stdout:6/898: mknod d4/d11/d12/d1c/d8a/d98/d9f/c127 0 2024-03-22T10:42:58.656 INFO:tasks.workunit.client.1.smithi150.stdout:8/600: dwrite d0/d2/d19/d29/d2f/df/f97 [0,4194304] 0 2024-03-22T10:42:58.661 INFO:tasks.workunit.client.1.smithi150.stdout:7/585: getdents d2/d5/d7/d8/d57/d54/d91/dad 0 2024-03-22T10:42:58.662 INFO:tasks.workunit.client.0.smithi115.stdout:4/963: rename d1/d30/d4c/d86/dde/d103 to d1/d66/d147 0 2024-03-22T10:42:58.671 INFO:tasks.workunit.client.1.smithi150.stdout:3/542: dwrite d5/fb [0,4194304] 0 2024-03-22T10:42:58.672 INFO:tasks.workunit.client.0.smithi115.stdout:9/952: symlink d6/d8/d3c/d59/d68/da8/l14c 0 2024-03-22T10:42:58.672 INFO:tasks.workunit.client.1.smithi150.stdout:5/492: dwrite d0/d8/dc/d1d/d53/d31/f6b [0,4194304] 0 2024-03-22T10:42:58.672 INFO:tasks.workunit.client.1.smithi150.stdout:7/586: creat d2/d5/d7/fc1 x:0 0 0 2024-03-22T10:42:58.679 INFO:tasks.workunit.client.0.smithi115.stdout:9/953: truncate d6/d7/d19/d4e/f109 1539613 0 2024-03-22T10:42:58.680 INFO:tasks.workunit.client.1.smithi150.stdout:1/525: dread d1/d79/f85 [0,4194304] 0 2024-03-22T10:42:58.685 INFO:tasks.workunit.client.0.smithi115.stdout:0/921: dread d3/d7/d1b/f24 [0,4194304] 0 2024-03-22T10:42:58.685 INFO:tasks.workunit.client.0.smithi115.stdout:9/954: mknod d6/d7/d20/d22/de5/c14d 0 2024-03-22T10:42:58.686 INFO:tasks.workunit.client.0.smithi115.stdout:4/964: dwrite d1/d66/d106/f64 [4194304,4194304] 0 2024-03-22T10:42:58.687 INFO:tasks.workunit.client.0.smithi115.stdout:9/955: dread - d6/d7/d20/f55 zero size 2024-03-22T10:42:58.690 INFO:tasks.workunit.client.0.smithi115.stdout:0/922: mknod d3/df0/dca/c128 0 2024-03-22T10:42:58.693 INFO:tasks.workunit.client.1.smithi150.stdout:9/586: link d1/d8/de/d73/ca5 d1/d8/de/df/d20/d32/ccc 0 2024-03-22T10:42:58.700 INFO:tasks.workunit.client.0.smithi115.stdout:9/956: creat d6/d8/d3c/d59/f14e x:0 0 0 2024-03-22T10:42:58.702 INFO:tasks.workunit.client.0.smithi115.stdout:0/923: creat d3/d7/db7/f129 x:0 0 0 2024-03-22T10:42:58.705 INFO:tasks.workunit.client.0.smithi115.stdout:9/957: creat d6/d7/d6a/d6f/f14f x:0 0 0 2024-03-22T10:42:58.708 INFO:tasks.workunit.client.1.smithi150.stdout:1/526: unlink d1/d19/d1a/d7d/l94 0 2024-03-22T10:42:58.708 INFO:tasks.workunit.client.0.smithi115.stdout:0/924: mkdir d3/d4/d14/d64/d12a 0 2024-03-22T10:42:58.708 INFO:tasks.workunit.client.0.smithi115.stdout:4/965: dwrite d1/d66/d106/f64 [0,4194304] 0 2024-03-22T10:42:58.709 INFO:tasks.workunit.client.0.smithi115.stdout:2/915: dwrite da/d1c/f69 [0,4194304] 0 2024-03-22T10:42:58.714 INFO:tasks.workunit.client.1.smithi150.stdout:7/587: rename d2/d5/d7/d9/db/d9f/d96/lb4 to d2/d5/d7/d88/lc2 0 2024-03-22T10:42:58.716 INFO:tasks.workunit.client.1.smithi150.stdout:9/587: dwrite d1/d8/de/d73/f74 [0,4194304] 0 2024-03-22T10:42:58.717 INFO:tasks.workunit.client.0.smithi115.stdout:9/958: creat d6/d7/d30/d3b/f150 x:0 0 0 2024-03-22T10:42:58.718 INFO:tasks.workunit.client.0.smithi115.stdout:4/966: write d1/d30/d4c/d86/d9f/d75/d99/f12b [967253,84355] 0 2024-03-22T10:42:58.718 INFO:tasks.workunit.client.1.smithi150.stdout:7/588: chown d2/d5/d7/d8/d57/d54/d91/daf 32841 1 2024-03-22T10:42:58.720 INFO:tasks.workunit.client.1.smithi150.stdout:2/665: dwrite d12/d19/f59 [4194304,4194304] 0 2024-03-22T10:42:58.722 INFO:tasks.workunit.client.0.smithi115.stdout:0/925: write d3/d7/d1c/d6c/d87/dd6/f120 [709168,94498] 0 2024-03-22T10:42:58.725 INFO:tasks.workunit.client.0.smithi115.stdout:9/959: write d6/d7/d19/d4e/f109 [531608,88743] 0 2024-03-22T10:42:58.727 INFO:tasks.workunit.client.1.smithi150.stdout:2/666: chown d12/d19/fa4 987309 1 2024-03-22T10:42:58.727 INFO:tasks.workunit.client.0.smithi115.stdout:7/934: dwrite d2/d7/d12/d10/d23/d3d/dd2/ffd [0,4194304] 0 2024-03-22T10:42:58.729 INFO:tasks.workunit.client.0.smithi115.stdout:2/916: getdents da/d25/d51 0 2024-03-22T10:42:58.729 INFO:tasks.workunit.client.1.smithi150.stdout:7/589: write d2/d5/d7/d9/db/d6c/fb6 [1058135,118273] 0 2024-03-22T10:42:58.732 INFO:tasks.workunit.client.0.smithi115.stdout:9/960: write d6/d7/d19/d4e/fd7 [1766025,47366] 0 2024-03-22T10:42:58.732 INFO:tasks.workunit.client.0.smithi115.stdout:2/917: write da/d25/d51/f56 [4921259,67142] 0 2024-03-22T10:42:58.737 INFO:tasks.workunit.client.0.smithi115.stdout:0/926: rename d3/d7/d1b/d35/f7e to d3/d52/f12b 0 2024-03-22T10:42:58.742 INFO:tasks.workunit.client.0.smithi115.stdout:2/918: mknod da/d25/d51/c113 0 2024-03-22T10:42:58.743 INFO:tasks.workunit.client.0.smithi115.stdout:7/935: creat d2/d7/d114/d13a/f13e x:0 0 0 2024-03-22T10:42:58.745 INFO:tasks.workunit.client.0.smithi115.stdout:2/919: dread - da/d25/fb6 zero size 2024-03-22T10:42:58.746 INFO:tasks.workunit.client.1.smithi150.stdout:5/493: getdents d0/d8/dc/de/d14/d4f/d47 0 2024-03-22T10:42:58.752 INFO:tasks.workunit.client.1.smithi150.stdout:0/561: fdatasync d0/d1/d84/d86/d11/d66/f8b 0 2024-03-22T10:42:58.754 INFO:tasks.workunit.client.0.smithi115.stdout:4/967: rename d1/d66/d106/d35/d40/d54/d5e/df3 to d1/d148 0 2024-03-22T10:42:58.754 INFO:tasks.workunit.client.0.smithi115.stdout:7/936: mknod d2/d7/d12/d10/d23/d3d/da4/dc0/d116/c13f 0 2024-03-22T10:42:58.756 INFO:tasks.workunit.client.1.smithi150.stdout:2/667: mkdir d12/d21/dd5 0 2024-03-22T10:42:58.757 INFO:tasks.workunit.client.0.smithi115.stdout:0/927: rename d3 to d3/d7/d1c/d12c 22 2024-03-22T10:42:58.761 INFO:tasks.workunit.client.0.smithi115.stdout:7/937: stat d2/d7/d12/d10/d19/le4 0 2024-03-22T10:42:58.762 INFO:tasks.workunit.client.0.smithi115.stdout:4/968: creat d1/dbb/dc6/dd8/df7/f149 x:0 0 0 2024-03-22T10:42:58.764 INFO:tasks.workunit.client.1.smithi150.stdout:1/527: dwrite d1/da/f75 [0,4194304] 0 2024-03-22T10:42:58.764 INFO:tasks.workunit.client.1.smithi150.stdout:5/494: creat d0/d8/dc/d1d/d53/d31/f90 x:0 0 0 2024-03-22T10:42:58.764 INFO:tasks.workunit.client.1.smithi150.stdout:4/575: dwrite d0/d2f/f50 [0,4194304] 0 2024-03-22T10:42:58.769 INFO:tasks.workunit.client.0.smithi115.stdout:2/920: rename da/d1c/d67/d78/l9e to da/d25/d3d/d57/d7d/da1/ddc/l114 0 2024-03-22T10:42:58.769 INFO:tasks.workunit.client.1.smithi150.stdout:3/543: dread d5/d22/d2f/d38/f39 [0,4194304] 0 2024-03-22T10:42:58.769 INFO:tasks.workunit.client.1.smithi150.stdout:0/562: creat d0/d1/d84/d86/d11/d66/fb3 x:0 0 0 2024-03-22T10:42:58.772 INFO:tasks.workunit.client.0.smithi115.stdout:0/928: symlink d3/d7/d1b/d1f/db9/l12d 0 2024-03-22T10:42:58.773 INFO:tasks.workunit.client.1.smithi150.stdout:4/576: write d0/d47/f64 [4391726,34007] 0 2024-03-22T10:42:58.774 INFO:tasks.workunit.client.1.smithi150.stdout:1/528: mknod d1/d19/d1a/d99/cac 0 2024-03-22T10:42:58.776 INFO:tasks.workunit.client.1.smithi150.stdout:0/563: dread - d0/d12/faa zero size 2024-03-22T10:42:58.778 INFO:tasks.workunit.client.1.smithi150.stdout:4/577: write d0/d1b/d68/f6a [496549,20875] 0 2024-03-22T10:42:58.780 INFO:tasks.workunit.client.0.smithi115.stdout:6/899: dwrite d4/d11/d12/d1c/fa7 [0,4194304] 0 2024-03-22T10:42:58.782 INFO:tasks.workunit.client.1.smithi150.stdout:0/564: chown d0/d1/d84/d86/d5a/l6a 233792 1 2024-03-22T10:42:58.782 INFO:tasks.workunit.client.1.smithi150.stdout:4/578: chown d0/ld 40631 1 2024-03-22T10:42:58.786 INFO:tasks.workunit.client.1.smithi150.stdout:2/668: dwrite d12/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:42:58.786 INFO:tasks.workunit.client.0.smithi115.stdout:0/929: truncate d3/da/d4c/f67 614561 0 2024-03-22T10:42:58.791 INFO:tasks.workunit.client.1.smithi150.stdout:0/565: read d0/f56 [2645865,29610] 0 2024-03-22T10:42:58.794 INFO:tasks.workunit.client.0.smithi115.stdout:0/930: truncate d3/d4/d14/d64/f72 749075 0 2024-03-22T10:42:58.795 INFO:tasks.workunit.client.1.smithi150.stdout:8/601: write d0/d2/f3f [8322027,97609] 0 2024-03-22T10:42:58.796 INFO:tasks.workunit.client.1.smithi150.stdout:4/579: truncate d0/d1b/d4d/d6c/d61/f48 4127831 0 2024-03-22T10:42:58.801 INFO:tasks.workunit.client.0.smithi115.stdout:6/900: rename d4/d11/d12/d1c/c79 to d4/d13/d33/c128 0 2024-03-22T10:42:58.804 INFO:tasks.workunit.client.0.smithi115.stdout:0/931: creat d3/da/d4c/da3/f12e x:0 0 0 2024-03-22T10:42:58.808 INFO:tasks.workunit.client.1.smithi150.stdout:1/529: dwrite d1/fa6 [0,4194304] 0 2024-03-22T10:42:58.810 INFO:tasks.workunit.client.1.smithi150.stdout:1/530: write d1/da/d2a/f35 [3814748,37066] 0 2024-03-22T10:42:58.812 INFO:tasks.workunit.client.0.smithi115.stdout:4/969: dread d1/d66/d106/d35/fbd [0,4194304] 0 2024-03-22T10:42:58.814 INFO:tasks.workunit.client.0.smithi115.stdout:0/932: fsync d3/d7/d1c/f77 0 2024-03-22T10:42:58.816 INFO:tasks.workunit.client.0.smithi115.stdout:7/938: dread d2/d7/d12/d10/d23/d3d/dd2/da3/faf [0,4194304] 0 2024-03-22T10:42:58.818 INFO:tasks.workunit.client.1.smithi150.stdout:3/544: write d5/da/ff [523508,92551] 0 2024-03-22T10:42:58.821 INFO:tasks.workunit.client.0.smithi115.stdout:6/901: dread d4/d11/d12/d1c/d8a/d98/fc8 [0,4194304] 0 2024-03-22T10:42:58.822 INFO:tasks.workunit.client.1.smithi150.stdout:1/531: truncate d1/da/d2a/d65/d78/f81 909949 0 2024-03-22T10:42:58.824 INFO:tasks.workunit.client.0.smithi115.stdout:7/939: unlink d2/d7/d12/c2c 0 2024-03-22T10:42:58.827 INFO:tasks.workunit.client.1.smithi150.stdout:3/545: chown d5/d53/l80 34182196 1 2024-03-22T10:42:58.827 INFO:tasks.workunit.client.0.smithi115.stdout:0/933: mknod d3/d7/d1b/d1f/db9/c12f 0 2024-03-22T10:42:58.830 INFO:tasks.workunit.client.1.smithi150.stdout:3/546: chown d5/f31 386 1 2024-03-22T10:42:58.833 INFO:tasks.workunit.client.1.smithi150.stdout:9/588: dwrite d1/d8/de/dbd/fc6 [0,4194304] 0 2024-03-22T10:42:58.834 INFO:tasks.workunit.client.1.smithi150.stdout:4/580: dwrite d0/d1b/d4d/d6c/d61/d71/fa4 [0,4194304] 0 2024-03-22T10:42:58.843 INFO:tasks.workunit.client.1.smithi150.stdout:4/581: write d0/de/f5b [1755325,116787] 0 2024-03-22T10:42:58.844 INFO:tasks.workunit.client.1.smithi150.stdout:2/669: truncate d12/d35/d7d/fbe 4004286 0 2024-03-22T10:42:58.845 INFO:tasks.workunit.client.1.smithi150.stdout:8/602: write d0/d2/d19/f47 [546571,59733] 0 2024-03-22T10:42:58.855 INFO:tasks.workunit.client.1.smithi150.stdout:0/566: dread d0/d1/d84/d86/d5a/f73 [0,4194304] 0 2024-03-22T10:42:58.857 INFO:tasks.workunit.client.1.smithi150.stdout:4/582: creat d0/d1b/d4d/d6c/d61/fbb x:0 0 0 2024-03-22T10:42:58.863 INFO:tasks.workunit.client.0.smithi115.stdout:9/961: truncate d6/d7/d20/d22/d23/f83 2630160 0 2024-03-22T10:42:58.868 INFO:tasks.workunit.client.1.smithi150.stdout:1/532: dwrite d1/d19/d1a/d7d/f82 [0,4194304] 0 2024-03-22T10:42:58.870 INFO:tasks.workunit.client.1.smithi150.stdout:9/589: unlink d1/d8/fc5 0 2024-03-22T10:42:58.880 INFO:tasks.workunit.client.1.smithi150.stdout:8/603: dwrite d0/d2/f67 [0,4194304] 0 2024-03-22T10:42:58.882 INFO:tasks.workunit.client.1.smithi150.stdout:0/567: mkdir d0/d1/d84/d86/d42/d10/d38/d52/d9b/db4 0 2024-03-22T10:42:58.883 INFO:tasks.workunit.client.1.smithi150.stdout:2/670: fdatasync d12/d19/db6/da9/d67/d3c/f77 0 2024-03-22T10:42:58.888 INFO:tasks.workunit.client.1.smithi150.stdout:1/533: symlink d1/d19/d1a/lad 0 2024-03-22T10:42:58.890 INFO:tasks.workunit.client.1.smithi150.stdout:0/568: chown d0/d1/d84/d86/d11/d66/c67 0 1 2024-03-22T10:42:58.893 INFO:tasks.workunit.client.1.smithi150.stdout:1/534: rename d1/da to d1/da/d15/dae 22 2024-03-22T10:42:58.898 INFO:tasks.workunit.client.1.smithi150.stdout:9/590: creat d1/d8/de/df/d20/d32/d75/dc3/fcd x:0 0 0 2024-03-22T10:42:58.900 INFO:tasks.workunit.client.0.smithi115.stdout:7/940: dread d2/d7/d12/d10/d19/dbd/f104 [0,4194304] 0 2024-03-22T10:42:58.901 INFO:tasks.workunit.client.1.smithi150.stdout:9/591: chown d1/d8/de/df/d36/f96 1944 1 2024-03-22T10:42:58.902 INFO:tasks.workunit.client.0.smithi115.stdout:2/921: write da/d25/d3d/d57/f65 [1733445,68587] 0 2024-03-22T10:42:58.902 INFO:tasks.workunit.client.1.smithi150.stdout:8/604: unlink d0/l9f 0 2024-03-22T10:42:58.910 INFO:tasks.workunit.client.0.smithi115.stdout:9/962: mkdir d6/d7/d48/d70/d151 0 2024-03-22T10:42:58.911 INFO:tasks.workunit.client.0.smithi115.stdout:9/963: dread - d6/d8/d3c/d59/d68/d8e/f11b zero size 2024-03-22T10:42:58.916 INFO:tasks.workunit.client.1.smithi150.stdout:0/569: dwrite d0/d1/d84/d86/d11/f7e [0,4194304] 0 2024-03-22T10:42:58.916 INFO:tasks.workunit.client.0.smithi115.stdout:2/922: dread da/f15 [0,4194304] 0 2024-03-22T10:42:58.918 INFO:tasks.workunit.client.0.smithi115.stdout:7/941: mkdir d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/d12d/d140 0 2024-03-22T10:42:58.918 INFO:tasks.workunit.client.0.smithi115.stdout:2/923: chown da/d1c/d67/d78/d86/db4 2 1 2024-03-22T10:42:58.920 INFO:tasks.workunit.client.1.smithi150.stdout:2/671: getdents d12/d19/db6/da9/d67 0 2024-03-22T10:42:58.922 INFO:tasks.workunit.client.1.smithi150.stdout:9/592: dread d1/d8/de/df/d24/d4e/f77 [0,4194304] 0 2024-03-22T10:42:58.922 INFO:tasks.workunit.client.0.smithi115.stdout:9/964: symlink d6/d7/d30/d37/dbe/d110/d126/l152 0 2024-03-22T10:42:58.923 INFO:tasks.workunit.client.1.smithi150.stdout:9/593: readlink d1/l9a 0 2024-03-22T10:42:58.923 INFO:tasks.workunit.client.0.smithi115.stdout:2/924: mknod da/d25/dfb/c115 0 2024-03-22T10:42:58.927 INFO:tasks.workunit.client.0.smithi115.stdout:7/942: fsync d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/fd9 0 2024-03-22T10:42:58.931 INFO:tasks.workunit.client.0.smithi115.stdout:2/925: chown da/d25/d30/c47 126603322 1 2024-03-22T10:42:58.933 INFO:tasks.workunit.client.1.smithi150.stdout:8/605: dwrite d0/d2/d19/d29/d31/f6d [0,4194304] 0 2024-03-22T10:42:58.935 INFO:tasks.workunit.client.1.smithi150.stdout:9/594: truncate d1/d8/de/df/f18 411239 0 2024-03-22T10:42:58.938 INFO:tasks.workunit.client.0.smithi115.stdout:2/926: creat da/d25/d3d/d57/d7d/da1/dba/ddf/f116 x:0 0 0 2024-03-22T10:42:58.939 INFO:tasks.workunit.client.0.smithi115.stdout:9/965: dwrite d6/d7/d48/df4/dfa/f10b [0,4194304] 0 2024-03-22T10:42:58.944 INFO:tasks.workunit.client.1.smithi150.stdout:8/606: mknod d0/d2/d19/d29/d37/d7d/d9e/ccc 0 2024-03-22T10:42:58.949 INFO:tasks.workunit.client.1.smithi150.stdout:0/570: symlink d0/d1/d84/d86/d11/lb5 0 2024-03-22T10:42:58.950 INFO:tasks.workunit.client.1.smithi150.stdout:8/607: write d0/d2/fa8 [4452324,128635] 0 2024-03-22T10:42:58.956 INFO:tasks.workunit.client.1.smithi150.stdout:4/583: write d0/d32/f56 [4640086,2543] 0 2024-03-22T10:42:58.958 INFO:tasks.workunit.client.1.smithi150.stdout:9/595: dwrite d1/d8/de/df/d36/f39 [0,4194304] 0 2024-03-22T10:42:58.960 INFO:tasks.workunit.client.0.smithi115.stdout:9/966: read d6/d7/d48/d70/da9/fac [348694,61804] 0 2024-03-22T10:42:58.960 INFO:tasks.workunit.client.1.smithi150.stdout:8/608: unlink d0/d2/d19/d29/d37/d7d/la7 0 2024-03-22T10:42:58.961 INFO:tasks.workunit.client.0.smithi115.stdout:9/967: stat d6/d7/d48/f119 0 2024-03-22T10:42:58.980 INFO:tasks.workunit.client.1.smithi150.stdout:0/571: link d0/d1/d84/d86/d42/d10/d53/f5e d0/d1/d84/d86/d42/d10/d38/d52/d9b/fb6 0 2024-03-22T10:42:58.984 INFO:tasks.workunit.client.1.smithi150.stdout:4/584: mknod d0/d2f/d63/d7b/cbc 0 2024-03-22T10:42:58.985 INFO:tasks.workunit.client.0.smithi115.stdout:4/970: dwrite d1/fd [0,4194304] 0 2024-03-22T10:42:58.994 INFO:tasks.workunit.client.0.smithi115.stdout:0/934: mknod d3/da/d4c/d69/c130 0 2024-03-22T10:42:58.995 INFO:tasks.workunit.client.0.smithi115.stdout:4/971: symlink d1/d69/l14a 0 2024-03-22T10:42:58.998 INFO:tasks.workunit.client.0.smithi115.stdout:0/935: symlink d3/d4/d14/l131 0 2024-03-22T10:42:58.998 INFO:tasks.workunit.client.1.smithi150.stdout:9/596: dread d1/d8/de/df/d20/d32/d75/f84 [0,4194304] 0 2024-03-22T10:42:58.999 INFO:tasks.workunit.client.0.smithi115.stdout:4/972: dread - d1/d66/d106/d35/d40/d54/d5e/f143 zero size 2024-03-22T10:42:59.000 INFO:tasks.workunit.client.0.smithi115.stdout:0/936: chown d3/d7/d1c/f56 96 1 2024-03-22T10:42:59.003 INFO:tasks.workunit.client.0.smithi115.stdout:0/937: truncate d3/d7/d1c/d6c/d87/dd6/f10e 963704 0 2024-03-22T10:42:59.006 INFO:tasks.workunit.client.0.smithi115.stdout:4/973: symlink d1/d66/d106/d28/l14b 0 2024-03-22T10:42:59.007 INFO:tasks.workunit.client.1.smithi150.stdout:9/597: dread d1/d8/f88 [0,4194304] 0 2024-03-22T10:42:59.009 INFO:tasks.workunit.client.1.smithi150.stdout:0/572: dread d0/d1/d84/d86/f31 [0,4194304] 0 2024-03-22T10:42:59.009 INFO:tasks.workunit.client.0.smithi115.stdout:4/974: write d1/d66/d106/f9d [632483,12068] 0 2024-03-22T10:42:59.011 INFO:tasks.workunit.client.0.smithi115.stdout:4/975: chown d1/d66/d106/d68/fae 227494 1 2024-03-22T10:42:59.013 INFO:tasks.workunit.client.0.smithi115.stdout:0/938: fdatasync d3/da/f90 0 2024-03-22T10:42:59.013 INFO:tasks.workunit.client.1.smithi150.stdout:8/609: getdents d0/d2/d19/d29/d37/d7d/d9e 0 2024-03-22T10:42:59.015 INFO:tasks.workunit.client.1.smithi150.stdout:0/573: chown d0/d1/d84/d86/d42/f82 5735 1 2024-03-22T10:42:59.018 INFO:tasks.workunit.client.1.smithi150.stdout:4/585: dwrite d0/d47/faa [0,4194304] 0 2024-03-22T10:42:59.018 INFO:tasks.workunit.client.1.smithi150.stdout:8/610: chown d0/d2/d19/d29/d2f/df/cbd 0 1 2024-03-22T10:42:59.020 INFO:tasks.workunit.client.1.smithi150.stdout:0/574: fsync d0/d1/d84/d86/d42/d10/d38/d52/f54 0 2024-03-22T10:42:59.021 INFO:tasks.workunit.client.0.smithi115.stdout:4/976: link d1/d66/f6d d1/d30/d4c/d86/d9f/d4f/f14c 0 2024-03-22T10:42:59.027 INFO:tasks.workunit.client.0.smithi115.stdout:0/939: link d3/d7/db7/cbb d3/d7/d1b/d35/c132 0 2024-03-22T10:42:59.027 INFO:tasks.workunit.client.0.smithi115.stdout:4/977: creat d1/dbb/dc6/dd8/f14d x:0 0 0 2024-03-22T10:42:59.029 INFO:tasks.workunit.client.1.smithi150.stdout:0/575: chown d0/d1/d84/d86/l9c 2 1 2024-03-22T10:42:59.031 INFO:tasks.workunit.client.1.smithi150.stdout:8/611: symlink d0/d2/d19/d29/d37/d7d/lcd 0 2024-03-22T10:42:59.031 INFO:tasks.workunit.client.0.smithi115.stdout:4/978: chown d1/dbb/dc6/dd8/df7/l140 119485299 1 2024-03-22T10:42:59.036 INFO:tasks.workunit.client.0.smithi115.stdout:0/940: mknod d3/da/d4c/da3/c133 0 2024-03-22T10:42:59.037 INFO:tasks.workunit.client.1.smithi150.stdout:8/612: write d0/d2/d19/d29/d2e/d70/fb8 [1964206,72130] 0 2024-03-22T10:42:59.038 INFO:tasks.workunit.client.1.smithi150.stdout:9/598: dwrite d1/d19/d7d/dc1/fa0 [0,4194304] 0 2024-03-22T10:42:59.039 INFO:tasks.workunit.client.0.smithi115.stdout:4/979: truncate d1/d66/d106/d68/f129 292156 0 2024-03-22T10:42:59.041 INFO:tasks.workunit.client.1.smithi150.stdout:8/613: readlink d0/d2/d19/d29/d2f/df/d1b/l79 0 2024-03-22T10:42:59.041 INFO:tasks.workunit.client.1.smithi150.stdout:4/586: link d0/f11 d0/d2f/d63/fbd 0 2024-03-22T10:42:59.042 INFO:tasks.workunit.client.0.smithi115.stdout:6/902: dwrite d4/d11/d12/f31 [0,4194304] 0 2024-03-22T10:42:59.042 INFO:tasks.workunit.client.0.smithi115.stdout:0/941: creat d3/df0/dca/dce/f134 x:0 0 0 2024-03-22T10:42:59.043 INFO:tasks.workunit.client.1.smithi150.stdout:0/576: creat d0/d1/d84/d86/d42/d10/fb7 x:0 0 0 2024-03-22T10:42:59.044 INFO:tasks.workunit.client.0.smithi115.stdout:4/980: creat d1/dbb/ddf/df9/f14e x:0 0 0 2024-03-22T10:42:59.044 INFO:tasks.workunit.client.1.smithi150.stdout:9/599: mknod d1/d8/de/df/d24/cce 0 2024-03-22T10:42:59.046 INFO:tasks.workunit.client.0.smithi115.stdout:0/942: chown d3/da/d4c/d69/d6a 28679570 1 2024-03-22T10:42:59.047 INFO:tasks.workunit.client.1.smithi150.stdout:0/577: dread - d0/d1/d84/d86/d11/d66/d35/d5f/fa8 zero size 2024-03-22T10:42:59.048 INFO:tasks.workunit.client.1.smithi150.stdout:9/600: read d1/d49/f31 [2589,73542] 0 2024-03-22T10:42:59.051 INFO:tasks.workunit.client.0.smithi115.stdout:0/943: chown d3/d7/d1b/c7a 2 1 2024-03-22T10:42:59.051 INFO:tasks.workunit.client.0.smithi115.stdout:0/944: fsync d3/d7/db7/d121/f124 0 2024-03-22T10:42:59.053 INFO:tasks.workunit.client.1.smithi150.stdout:9/601: rmdir d1/d19/d7d/dc1 39 2024-03-22T10:42:59.054 INFO:tasks.workunit.client.0.smithi115.stdout:6/903: mkdir d4/d11/d12/d1c/d8a/d98/d9f/df0/d129 0 2024-03-22T10:42:59.058 INFO:tasks.workunit.client.1.smithi150.stdout:9/602: readlink d1/d8/de/df/d24/d4e/l64 0 2024-03-22T10:42:59.058 INFO:tasks.workunit.client.1.smithi150.stdout:0/578: creat d0/d1/d84/d86/d42/d10/d38/d52/d68/fb8 x:0 0 0 2024-03-22T10:42:59.059 INFO:tasks.workunit.client.0.smithi115.stdout:0/945: creat d3/da/d4c/d74/f135 x:0 0 0 2024-03-22T10:42:59.061 INFO:tasks.workunit.client.1.smithi150.stdout:8/614: getdents d0/d2/d19/d29/d2e/d70 0 2024-03-22T10:42:59.065 INFO:tasks.workunit.client.1.smithi150.stdout:0/579: write d0/d1/d84/d86/d42/d10/d38/d52/f65 [6073927,115386] 0 2024-03-22T10:42:59.067 INFO:tasks.workunit.client.0.smithi115.stdout:0/946: creat d3/d7/d1c/d6c/f136 x:0 0 0 2024-03-22T10:42:59.068 INFO:tasks.workunit.client.1.smithi150.stdout:8/615: write d0/d2/d19/d29/d37/d7d/d9e/fcb [665007,15159] 0 2024-03-22T10:42:59.072 INFO:tasks.workunit.client.0.smithi115.stdout:2/927: dwrite da/f16 [4194304,4194304] 0 2024-03-22T10:42:59.073 INFO:tasks.workunit.client.1.smithi150.stdout:0/580: chown d0/d1/d84/d86/d11/d66/d35/d92/c97 2 1 2024-03-22T10:42:59.074 INFO:tasks.workunit.client.0.smithi115.stdout:9/968: rmdir d6/d7/d48 39 2024-03-22T10:42:59.078 INFO:tasks.workunit.client.0.smithi115.stdout:2/928: fdatasync da/d25/d3d/f9f 0 2024-03-22T10:42:59.079 INFO:tasks.workunit.client.1.smithi150.stdout:4/587: dwrite d0/d1b/d4d/d6c/f62 [0,4194304] 0 2024-03-22T10:42:59.080 INFO:tasks.workunit.client.0.smithi115.stdout:9/969: unlink d6/d7/d6a/d6f/ce8 0 2024-03-22T10:42:59.082 INFO:tasks.workunit.client.1.smithi150.stdout:8/616: getdents d0/d2/d19/d29/d2e/d70 0 2024-03-22T10:42:59.083 INFO:tasks.workunit.client.1.smithi150.stdout:5/495: sync 2024-03-22T10:42:59.083 INFO:tasks.workunit.client.1.smithi150.stdout:6/547: sync 2024-03-22T10:42:59.083 INFO:tasks.workunit.client.0.smithi115.stdout:2/929: truncate da/d1c/f1e 2550172 0 2024-03-22T10:42:59.083 INFO:tasks.workunit.client.1.smithi150.stdout:7/590: sync 2024-03-22T10:42:59.085 INFO:tasks.workunit.client.0.smithi115.stdout:2/930: truncate da/f82 4307 0 2024-03-22T10:42:59.087 INFO:tasks.workunit.client.0.smithi115.stdout:6/904: mkdir d4/d12a 0 2024-03-22T10:42:59.091 INFO:tasks.workunit.client.1.smithi150.stdout:8/617: unlink d0/d2/d19/d29/d37/f85 0 2024-03-22T10:42:59.093 INFO:tasks.workunit.client.1.smithi150.stdout:6/548: mkdir db/d52/dc3 0 2024-03-22T10:42:59.099 INFO:tasks.workunit.client.0.smithi115.stdout:2/931: rename da/d25/d30/dcd/ld8 to da/d25/d3d/d57/da7/df7/l117 0 2024-03-22T10:42:59.102 INFO:tasks.workunit.client.1.smithi150.stdout:0/581: dread d0/d1/d84/d86/d42/f77 [0,4194304] 0 2024-03-22T10:42:59.105 INFO:tasks.workunit.client.1.smithi150.stdout:7/591: symlink d2/d5/d7/d8/d57/d4a/d8d/lc3 0 2024-03-22T10:42:59.105 INFO:tasks.workunit.client.0.smithi115.stdout:9/970: dwrite d6/d7/d19/d4e/fd7 [0,4194304] 0 2024-03-22T10:42:59.105 INFO:tasks.workunit.client.0.smithi115.stdout:2/932: write da/d25/d3d/d7e/fa2 [3992218,41106] 0 2024-03-22T10:42:59.108 INFO:tasks.workunit.client.0.smithi115.stdout:2/933: chown da/d1c/f88 1283 1 2024-03-22T10:42:59.112 INFO:tasks.workunit.client.1.smithi150.stdout:5/496: write d0/d8/dc/de/d14/d2e/d5a/f8e [3519905,42688] 0 2024-03-22T10:42:59.112 INFO:tasks.workunit.client.0.smithi115.stdout:9/971: creat d6/d7/d20/d22/d3f/f153 x:0 0 0 2024-03-22T10:42:59.115 INFO:tasks.workunit.client.0.smithi115.stdout:2/934: symlink da/d25/d3d/d7e/l118 0 2024-03-22T10:42:59.115 INFO:tasks.workunit.client.0.smithi115.stdout:9/972: stat d6/d7/d30 0 2024-03-22T10:42:59.115 INFO:tasks.workunit.client.1.smithi150.stdout:6/549: dread db/dd/d15/f35 [0,4194304] 0 2024-03-22T10:42:59.116 INFO:tasks.workunit.client.0.smithi115.stdout:6/905: dwrite d4/f30 [0,4194304] 0 2024-03-22T10:42:59.122 INFO:tasks.workunit.client.0.smithi115.stdout:4/981: write d1/d66/d106/d35/d40/d5c/f25 [6753706,51083] 0 2024-03-22T10:42:59.123 INFO:tasks.workunit.client.1.smithi150.stdout:5/497: mknod d0/d8/dc/d1d/d53/d31/c91 0 2024-03-22T10:42:59.124 INFO:tasks.workunit.client.0.smithi115.stdout:2/935: write da/d25/d30/dcd/f87 [3099997,40550] 0 2024-03-22T10:42:59.124 INFO:tasks.workunit.client.0.smithi115.stdout:6/906: dread - d4/d11/f9a zero size 2024-03-22T10:42:59.125 INFO:tasks.workunit.client.1.smithi150.stdout:5/498: chown d0/d8/dc/f12 8082 1 2024-03-22T10:42:59.125 INFO:tasks.workunit.client.1.smithi150.stdout:9/603: write d1/d8/de/df/d36/f5c [2217298,66512] 0 2024-03-22T10:42:59.127 INFO:tasks.workunit.client.0.smithi115.stdout:2/936: chown da/d25/d30/f59 459991765 1 2024-03-22T10:42:59.128 INFO:tasks.workunit.client.1.smithi150.stdout:6/550: creat db/dd/d15/d6a/d8d/fc4 x:0 0 0 2024-03-22T10:42:59.129 INFO:tasks.workunit.client.0.smithi115.stdout:0/947: write d3/d4/d6d/f98 [808772,115629] 0 2024-03-22T10:42:59.130 INFO:tasks.workunit.client.0.smithi115.stdout:9/973: truncate d6/d8/d3c/d59/d68/f7f 830105 0 2024-03-22T10:42:59.133 INFO:tasks.workunit.client.0.smithi115.stdout:4/982: dread - d1/d30/d4c/d86/d9f/d75/d99/d98/d9b/fa0 zero size 2024-03-22T10:42:59.133 INFO:tasks.workunit.client.1.smithi150.stdout:1/535: sync 2024-03-22T10:42:59.133 INFO:tasks.workunit.client.1.smithi150.stdout:3/547: sync 2024-03-22T10:42:59.133 INFO:tasks.workunit.client.1.smithi150.stdout:2/672: sync 2024-03-22T10:42:59.134 INFO:tasks.workunit.client.1.smithi150.stdout:7/592: rmdir d2/d5/d9a 0 2024-03-22T10:42:59.139 INFO:tasks.workunit.client.1.smithi150.stdout:4/588: write d0/f9 [528474,60502] 0 2024-03-22T10:42:59.139 INFO:tasks.workunit.client.1.smithi150.stdout:0/582: truncate d0/d1/d84/d86/f6b 603848 0 2024-03-22T10:42:59.139 INFO:tasks.workunit.client.0.smithi115.stdout:6/907: truncate d4/d11/d12/d1c/d8a/d98/d9f/f10e 304848 0 2024-03-22T10:42:59.140 INFO:tasks.workunit.client.1.smithi150.stdout:1/536: write d1/da/d15/d18/d4b/f53 [4567430,26372] 0 2024-03-22T10:42:59.141 INFO:tasks.workunit.client.1.smithi150.stdout:7/593: read - d2/d5/d7/d8/d57/d54/f5f zero size 2024-03-22T10:42:59.143 INFO:tasks.workunit.client.1.smithi150.stdout:3/548: fdatasync d5/d22/d2f/d38/d59/d88/f8a 0 2024-03-22T10:42:59.145 INFO:tasks.workunit.client.1.smithi150.stdout:5/499: rename d0/d8/c3a to d0/d8/dc/de/d14/d68/c92 0 2024-03-22T10:42:59.147 INFO:tasks.workunit.client.1.smithi150.stdout:6/551: mknod db/dd/cc5 0 2024-03-22T10:42:59.147 INFO:tasks.workunit.client.0.smithi115.stdout:6/908: readlink d4/d36/d48/d72/lc2 0 2024-03-22T10:42:59.147 INFO:tasks.workunit.client.0.smithi115.stdout:6/909: chown d4/d36/l112 7291 1 2024-03-22T10:42:59.148 INFO:tasks.workunit.client.1.smithi150.stdout:7/594: read d2/d5/d7/d9/db/d6c/fb6 [515887,25230] 0 2024-03-22T10:42:59.154 INFO:tasks.workunit.client.0.smithi115.stdout:2/937: dwrite da/d25/d3d/d57/d7d/da1/ddc/fe2 [0,4194304] 0 2024-03-22T10:42:59.156 INFO:tasks.workunit.client.1.smithi150.stdout:1/537: chown d1/da/c48 0 1 2024-03-22T10:42:59.157 INFO:tasks.workunit.client.1.smithi150.stdout:2/673: mkdir d12/da1/dd6 0 2024-03-22T10:42:59.157 INFO:tasks.workunit.client.1.smithi150.stdout:4/589: symlink d0/d2f/d63/d7b/d9c/lbe 0 2024-03-22T10:42:59.157 INFO:tasks.workunit.client.0.smithi115.stdout:4/983: creat d1/d30/d4c/d86/f14f x:0 0 0 2024-03-22T10:42:59.159 INFO:tasks.workunit.client.1.smithi150.stdout:8/618: dwrite d0/d2/d19/d29/d2e/f58 [0,4194304] 0 2024-03-22T10:42:59.159 INFO:tasks.workunit.client.1.smithi150.stdout:7/595: fdatasync d2/d5/d7/d9/db/d9f/f40 0 2024-03-22T10:42:59.162 INFO:tasks.workunit.client.1.smithi150.stdout:4/590: write d0/d1b/f2b [946569,107206] 0 2024-03-22T10:42:59.162 INFO:tasks.workunit.client.0.smithi115.stdout:2/938: write da/f82 [992256,45166] 0 2024-03-22T10:42:59.163 INFO:tasks.workunit.client.1.smithi150.stdout:6/552: mkdir db/d52/d9f/dc6 0 2024-03-22T10:42:59.167 INFO:tasks.workunit.client.1.smithi150.stdout:9/604: write d1/d8/de/df/da2/f91 [765956,109264] 0 2024-03-22T10:42:59.170 INFO:tasks.workunit.client.1.smithi150.stdout:5/500: mkdir d0/d8/dc/de/d14/d4f/d93 0 2024-03-22T10:42:59.171 INFO:tasks.workunit.client.0.smithi115.stdout:2/939: read da/d1c/f48 [1301315,2810] 0 2024-03-22T10:42:59.171 INFO:tasks.workunit.client.1.smithi150.stdout:2/674: fdatasync d12/d19/db6/da9/d67/d3c/fce 0 2024-03-22T10:42:59.172 INFO:tasks.workunit.client.0.smithi115.stdout:9/974: dwrite d6/f10e [0,4194304] 0 2024-03-22T10:42:59.176 INFO:tasks.workunit.client.1.smithi150.stdout:2/675: write d12/d19/f59 [6149286,108082] 0 2024-03-22T10:42:59.178 INFO:tasks.workunit.client.0.smithi115.stdout:2/940: readlink da/d25/d30/l77 0 2024-03-22T10:42:59.179 INFO:tasks.workunit.client.1.smithi150.stdout:0/583: symlink d0/d1/d84/d86/d11/d66/d35/d92/lb9 0 2024-03-22T10:42:59.179 INFO:tasks.workunit.client.1.smithi150.stdout:3/549: dwrite d5/da/f33 [4194304,4194304] 0 2024-03-22T10:42:59.179 INFO:tasks.workunit.client.0.smithi115.stdout:0/948: dwrite d3/d7/d1b/d35/f80 [4194304,4194304] 0 2024-03-22T10:42:59.180 INFO:tasks.workunit.client.1.smithi150.stdout:2/676: write d12/d19/db6/da9/d2c/d65/f9d [1366755,124594] 0 2024-03-22T10:42:59.182 INFO:tasks.workunit.client.1.smithi150.stdout:2/677: write f11 [1397372,126671] 0 2024-03-22T10:42:59.186 INFO:tasks.workunit.client.0.smithi115.stdout:9/975: chown d6/d7/d30/ca2 3 1 2024-03-22T10:42:59.190 INFO:tasks.workunit.client.0.smithi115.stdout:7/943: dwrite d2/d7/d12/d10/f17 [4194304,4194304] 0 2024-03-22T10:42:59.196 INFO:tasks.workunit.client.0.smithi115.stdout:2/941: rmdir da/d25/d3d/d57/d7d/da1/ddc 39 2024-03-22T10:42:59.196 INFO:tasks.workunit.client.1.smithi150.stdout:1/538: unlink d1/d19/d1a/d99/cac 0 2024-03-22T10:42:59.197 INFO:tasks.workunit.client.1.smithi150.stdout:5/501: unlink d0/d33/f73 0 2024-03-22T10:42:59.199 INFO:tasks.workunit.client.0.smithi115.stdout:0/949: truncate d3/d7/d1b/d93/fee 157542 0 2024-03-22T10:42:59.200 INFO:tasks.workunit.client.1.smithi150.stdout:9/605: rmdir d1/d49/d98 39 2024-03-22T10:42:59.203 INFO:tasks.workunit.client.1.smithi150.stdout:7/596: fdatasync d2/d5/d7/d8/d57/d4a/f63 0 2024-03-22T10:42:59.204 INFO:tasks.workunit.client.1.smithi150.stdout:5/502: chown d0/d8/dc/lf 1060 1 2024-03-22T10:42:59.205 INFO:tasks.workunit.client.0.smithi115.stdout:7/944: fsync d2/d62/d126/f137 0 2024-03-22T10:42:59.205 INFO:tasks.workunit.client.1.smithi150.stdout:4/591: dwrite d0/d1b/f39 [0,4194304] 0 2024-03-22T10:42:59.207 INFO:tasks.workunit.client.0.smithi115.stdout:2/942: rename da/d1c/d67/d78/d86/db4/fc3 to da/d25/d3d/d57/d7d/f119 0 2024-03-22T10:42:59.210 INFO:tasks.workunit.client.1.smithi150.stdout:3/550: rename d5/d53/d7e/d95/fb2 to d5/d53/fb3 0 2024-03-22T10:42:59.212 INFO:tasks.workunit.client.0.smithi115.stdout:4/984: dread d1/d66/f77 [0,4194304] 0 2024-03-22T10:42:59.213 INFO:tasks.workunit.client.0.smithi115.stdout:0/950: dread d3/d7/d1b/d93/fcc [0,4194304] 0 2024-03-22T10:42:59.216 INFO:tasks.workunit.client.0.smithi115.stdout:0/951: write d3/f7f [5260248,27109] 0 2024-03-22T10:42:59.217 INFO:tasks.workunit.client.1.smithi150.stdout:9/606: fsync d1/d8/de/df/d36/f43 0 2024-03-22T10:42:59.217 INFO:tasks.workunit.client.0.smithi115.stdout:7/945: symlink d2/d7/d102/l141 0 2024-03-22T10:42:59.218 INFO:tasks.workunit.client.1.smithi150.stdout:3/551: rmdir d5/d22/d2f/d38/d3e 39 2024-03-22T10:42:59.219 INFO:tasks.workunit.client.1.smithi150.stdout:1/539: getdents d1/da/d2a/d55/d97 0 2024-03-22T10:42:59.219 INFO:tasks.workunit.client.1.smithi150.stdout:0/584: mkdir d0/d1/d84/d86/d42/d10/d38/dba 0 2024-03-22T10:42:59.221 INFO:tasks.workunit.client.1.smithi150.stdout:4/592: chown d0/d1b/c1f 30575 1 2024-03-22T10:42:59.223 INFO:tasks.workunit.client.0.smithi115.stdout:6/910: dread d4/d11/f54 [4194304,4194304] 0 2024-03-22T10:42:59.224 INFO:tasks.workunit.client.1.smithi150.stdout:5/503: write d0/d8/dc/de/d14/d2e/d46/d4d/f51 [2016978,53962] 0 2024-03-22T10:42:59.224 INFO:tasks.workunit.client.1.smithi150.stdout:2/678: getdents d12/d19/db6/da9/d97/d78 0 2024-03-22T10:42:59.228 INFO:tasks.workunit.client.1.smithi150.stdout:4/593: chown d0/de/ca9 10548713 1 2024-03-22T10:42:59.229 INFO:tasks.workunit.client.0.smithi115.stdout:6/911: write d4/d11/d12/d1c/d8a/d98/d9f/dc3/ff1 [455706,43172] 0 2024-03-22T10:42:59.238 INFO:tasks.workunit.client.1.smithi150.stdout:0/585: stat d0/d1/d84/d86/d42/d10/d38/d79/f8f 0 2024-03-22T10:42:59.239 INFO:tasks.workunit.client.1.smithi150.stdout:6/553: getdents db/dd/d82/d84 0 2024-03-22T10:42:59.243 INFO:tasks.workunit.client.1.smithi150.stdout:0/586: read d0/d1/d84/d86/f31 [2536526,115974] 0 2024-03-22T10:42:59.243 INFO:tasks.workunit.client.1.smithi150.stdout:4/594: rmdir d0/d1b 39 2024-03-22T10:42:59.245 INFO:tasks.workunit.client.1.smithi150.stdout:5/504: symlink d0/d8/dc/de/d14/d2e/d46/l94 0 2024-03-22T10:42:59.247 INFO:tasks.workunit.client.0.smithi115.stdout:0/952: creat d3/da/f137 x:0 0 0 2024-03-22T10:42:59.248 INFO:tasks.workunit.client.0.smithi115.stdout:4/985: write d1/d30/d4c/d86/d9f/d32/f60 [2983803,56302] 0 2024-03-22T10:42:59.251 INFO:tasks.workunit.client.0.smithi115.stdout:0/953: truncate d3/d4/f60 548782 0 2024-03-22T10:42:59.252 INFO:tasks.workunit.client.1.smithi150.stdout:6/554: readlink db/dd/l5e 0 2024-03-22T10:42:59.253 INFO:tasks.workunit.client.0.smithi115.stdout:4/986: write d1/d30/d4c/d86/d9f/d32/f5b [2013820,15511] 0 2024-03-22T10:42:59.253 INFO:tasks.workunit.client.0.smithi115.stdout:2/943: rename da/d25/d3d/c6d to da/d25/d3d/d57/d7d/da1/dba/c11a 0 2024-03-22T10:42:59.254 INFO:tasks.workunit.client.1.smithi150.stdout:9/607: dwrite d1/d8/d95/fa6 [0,4194304] 0 2024-03-22T10:42:59.254 INFO:tasks.workunit.client.0.smithi115.stdout:6/912: mknod d4/d13/d33/c12b 0 2024-03-22T10:42:59.259 INFO:tasks.workunit.client.1.smithi150.stdout:8/619: dwrite d0/d3c/d71/f59 [0,4194304] 0 2024-03-22T10:42:59.259 INFO:tasks.workunit.client.1.smithi150.stdout:2/679: mkdir d12/d19/d2d/dd7 0 2024-03-22T10:42:59.260 INFO:tasks.workunit.client.0.smithi115.stdout:6/913: write d4/d11/d12/d1c/f7f [2177352,58571] 0 2024-03-22T10:42:59.261 INFO:tasks.workunit.client.0.smithi115.stdout:6/914: readlink d4/lb3 0 2024-03-22T10:42:59.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:59 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:59.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:59 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:59.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:42:59 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:59.262 INFO:tasks.workunit.client.1.smithi150.stdout:4/595: rmdir d0/d32 39 2024-03-22T10:42:59.264 INFO:tasks.workunit.client.1.smithi150.stdout:5/505: symlink d0/l95 0 2024-03-22T10:42:59.264 INFO:tasks.workunit.client.1.smithi150.stdout:0/587: write d0/f63 [1127942,100333] 0 2024-03-22T10:42:59.265 INFO:tasks.workunit.client.1.smithi150.stdout:3/552: rename d5/f4e to d5/d22/d2f/d38/fb4 0 2024-03-22T10:42:59.267 INFO:tasks.workunit.client.0.smithi115.stdout:7/946: write d2/d7/d12/d10/d19/f8b [2596257,36033] 0 2024-03-22T10:42:59.271 INFO:tasks.workunit.client.1.smithi150.stdout:3/553: readlink d5/d22/l2a 0 2024-03-22T10:42:59.272 INFO:tasks.workunit.client.1.smithi150.stdout:9/608: creat d1/d49/d98/fcf x:0 0 0 2024-03-22T10:42:59.274 INFO:tasks.workunit.client.0.smithi115.stdout:4/987: rename d1/d66/d82/da1/c43 to d1/d66/d106/d35/c150 0 2024-03-22T10:42:59.277 INFO:tasks.workunit.client.1.smithi150.stdout:7/597: dread d2/d5/d7/d9/db/d9f/f40 [0,4194304] 0 2024-03-22T10:42:59.277 INFO:tasks.workunit.client.0.smithi115.stdout:4/988: chown d1/d30/d4c/d86/d9f/d4f/d93 1008 1 2024-03-22T10:42:59.280 INFO:tasks.workunit.client.0.smithi115.stdout:2/944: dwrite da/d25/d3d/d57/d7d/da1/ddc/fe2 [0,4194304] 0 2024-03-22T10:42:59.286 INFO:tasks.workunit.client.0.smithi115.stdout:4/989: unlink d1/dbb/ddf/df9/c133 0 2024-03-22T10:42:59.288 INFO:tasks.workunit.client.0.smithi115.stdout:4/990: chown d1/d30/d4c/d86/d9f/d4f/f139 3529 1 2024-03-22T10:42:59.289 INFO:tasks.workunit.client.0.smithi115.stdout:0/954: truncate d3/d7/d1c/f5d 1858106 0 2024-03-22T10:42:59.289 INFO:tasks.workunit.client.1.smithi150.stdout:5/506: rename d0/d8/dc/de/d14/d4f/d93 to d0/d8/dc/de/d96 0 2024-03-22T10:42:59.289 INFO:tasks.workunit.client.1.smithi150.stdout:3/554: write d5/d53/d7e/d95/fa1 [2483143,74735] 0 2024-03-22T10:42:59.291 INFO:tasks.workunit.client.1.smithi150.stdout:1/540: dwrite d1/da/d15/f3e [0,4194304] 0 2024-03-22T10:42:59.291 INFO:tasks.workunit.client.0.smithi115.stdout:7/947: dread d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dc9/ffa [0,4194304] 0 2024-03-22T10:42:59.293 INFO:tasks.workunit.client.0.smithi115.stdout:2/945: rename da/d25/d51/c62 to da/d25/d30/c11b 0 2024-03-22T10:42:59.297 INFO:tasks.workunit.client.0.smithi115.stdout:0/955: rename d3/f33 to d3/d7/d1b/d1f/d78/dd9/f138 0 2024-03-22T10:42:59.298 INFO:tasks.workunit.client.1.smithi150.stdout:9/609: symlink d1/d8/de/df/d20/d32/d75/d99/da7/ld0 0 2024-03-22T10:42:59.302 INFO:tasks.workunit.client.0.smithi115.stdout:0/956: mknod d3/d7/d1c/d6c/d9a/c139 0 2024-03-22T10:42:59.302 INFO:tasks.workunit.client.0.smithi115.stdout:2/946: fdatasync da/d25/d3d/d57/d7d/f119 0 2024-03-22T10:42:59.303 INFO:tasks.workunit.client.1.smithi150.stdout:3/555: unlink d5/da/f34 0 2024-03-22T10:42:59.304 INFO:tasks.workunit.client.1.smithi150.stdout:1/541: read d1/da/d15/d18/f3d [4098208,8343] 0 2024-03-22T10:42:59.305 INFO:tasks.workunit.client.0.smithi115.stdout:0/957: truncate d3/d7/db7/f129 430633 0 2024-03-22T10:42:59.305 INFO:tasks.workunit.client.0.smithi115.stdout:7/948: rename d2/d7/d12/d10/d19/dbd/fdb to d2/d7/d12/d10/d19/dbd/f142 0 2024-03-22T10:42:59.306 INFO:tasks.workunit.client.1.smithi150.stdout:2/680: dwrite d12/d21/f4b [0,4194304] 0 2024-03-22T10:42:59.312 INFO:tasks.workunit.client.0.smithi115.stdout:2/947: symlink da/d25/d30/d10d/l11c 0 2024-03-22T10:42:59.314 INFO:tasks.workunit.client.1.smithi150.stdout:5/507: write d0/d8/dc/d1d/f24 [719643,47166] 0 2024-03-22T10:42:59.315 INFO:tasks.workunit.client.0.smithi115.stdout:4/991: write d1/d66/d106/d35/fb0 [901471,4783] 0 2024-03-22T10:42:59.327 INFO:tasks.workunit.client.1.smithi150.stdout:8/620: dwrite d0/d2/d19/d29/d37/fbe [0,4194304] 0 2024-03-22T10:42:59.330 INFO:tasks.workunit.client.1.smithi150.stdout:3/556: rename d5/c17 to d5/d22/d2f/cb5 0 2024-03-22T10:42:59.336 INFO:tasks.workunit.client.1.smithi150.stdout:0/588: dwrite d0/d1/d84/d86/f6b [0,4194304] 0 2024-03-22T10:42:59.338 INFO:tasks.workunit.client.0.smithi115.stdout:0/958: dwrite d3/d7/d1b/df8/f10d [4194304,4194304] 0 2024-03-22T10:42:59.341 INFO:tasks.workunit.client.1.smithi150.stdout:5/508: read d0/d8/dc/d1d/f24 [612017,114946] 0 2024-03-22T10:42:59.343 INFO:tasks.workunit.client.0.smithi115.stdout:0/959: truncate d3/d7/f11e 987332 0 2024-03-22T10:42:59.349 INFO:tasks.workunit.client.1.smithi150.stdout:4/596: dread d0/f11 [0,4194304] 0 2024-03-22T10:42:59.350 INFO:tasks.workunit.client.1.smithi150.stdout:9/610: dwrite d1/d8/fa1 [0,4194304] 0 2024-03-22T10:42:59.351 INFO:tasks.workunit.client.0.smithi115.stdout:7/949: dwrite d2/d7/d12/d10/d23/d3d/f2a [0,4194304] 0 2024-03-22T10:42:59.360 INFO:tasks.workunit.client.0.smithi115.stdout:2/948: creat da/d1c/d67/d78/d86/d9d/f11d x:0 0 0 2024-03-22T10:42:59.362 INFO:tasks.workunit.client.1.smithi150.stdout:7/598: dwrite d2/d5/d7/d8/d46/d49/f58 [0,4194304] 0 2024-03-22T10:42:59.363 INFO:tasks.workunit.client.0.smithi115.stdout:4/992: creat d1/d30/d4c/d86/d9f/d75/d99/da8/dea/f151 x:0 0 0 2024-03-22T10:42:59.367 INFO:tasks.workunit.client.1.smithi150.stdout:5/509: rmdir d0/d8 39 2024-03-22T10:42:59.369 INFO:tasks.workunit.client.0.smithi115.stdout:7/950: mkdir d2/d7/d12/d10/d23/d3d/dd2/d143 0 2024-03-22T10:42:59.369 INFO:tasks.workunit.client.0.smithi115.stdout:4/993: mkdir d1/d30/d4c/d86/d9f/db8/d152 0 2024-03-22T10:42:59.372 INFO:tasks.workunit.client.1.smithi150.stdout:1/542: dwrite d1/f29 [0,4194304] 0 2024-03-22T10:42:59.372 INFO:tasks.workunit.client.0.smithi115.stdout:0/960: dwrite d3/d7/d1c/d6c/f136 [0,4194304] 0 2024-03-22T10:42:59.378 INFO:tasks.workunit.client.1.smithi150.stdout:8/621: rename d0/d2/d19/f20 to d0/d3c/fce 0 2024-03-22T10:42:59.378 INFO:tasks.workunit.client.1.smithi150.stdout:4/597: mknod d0/d2f/d63/d7b/d9c/db1/cbf 0 2024-03-22T10:42:59.379 INFO:tasks.workunit.client.1.smithi150.stdout:1/543: write d1/da/d2a/d65/f76 [1668989,65206] 0 2024-03-22T10:42:59.379 INFO:tasks.workunit.client.1.smithi150.stdout:3/557: creat d5/d22/d2f/d38/d3e/fb6 x:0 0 0 2024-03-22T10:42:59.381 INFO:tasks.workunit.client.0.smithi115.stdout:0/961: mknod d3/df0/dca/dce/c13a 0 2024-03-22T10:42:59.381 INFO:tasks.workunit.client.0.smithi115.stdout:4/994: mknod d1/d30/d4c/d86/c153 0 2024-03-22T10:42:59.382 INFO:tasks.workunit.client.1.smithi150.stdout:9/611: mkdir d1/d8/de/df/d20/d32/dd1 0 2024-03-22T10:42:59.385 INFO:tasks.workunit.client.1.smithi150.stdout:8/622: chown d0/d2/d19/d29/d2e 1629 1 2024-03-22T10:42:59.385 INFO:tasks.workunit.client.1.smithi150.stdout:2/681: getdents d12/d19 0 2024-03-22T10:42:59.386 INFO:tasks.workunit.client.1.smithi150.stdout:0/589: creat d0/d1/fbb x:0 0 0 2024-03-22T10:42:59.391 INFO:tasks.workunit.client.1.smithi150.stdout:2/682: write d12/d19/db6/da9/d67/d3c/d75/fc8 [387538,89121] 0 2024-03-22T10:42:59.391 INFO:tasks.workunit.client.1.smithi150.stdout:5/510: creat d0/d33/f97 x:0 0 0 2024-03-22T10:42:59.392 INFO:tasks.workunit.client.1.smithi150.stdout:8/623: read d0/d2/d19/d29/d37/fb3 [48501,90416] 0 2024-03-22T10:42:59.393 INFO:tasks.workunit.client.0.smithi115.stdout:0/962: mknod d3/d7/d1b/d1f/d78/dd9/c13b 0 2024-03-22T10:42:59.395 INFO:tasks.workunit.client.1.smithi150.stdout:7/599: rmdir d2/d5/d7/d9/db/d6c/d9c 39 2024-03-22T10:42:59.397 INFO:tasks.workunit.client.0.smithi115.stdout:7/951: dwrite d2/d7/d114/d13a/f13e [0,4194304] 0 2024-03-22T10:42:59.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:59 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:59.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:59 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:42:59.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:42:59 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:42:59.399 INFO:tasks.workunit.client.0.smithi115.stdout:0/963: chown d3/da/c44 1838870 1 2024-03-22T10:42:59.401 INFO:tasks.workunit.client.1.smithi150.stdout:4/598: write d0/d1b/f2e [8932800,106904] 0 2024-03-22T10:42:59.408 INFO:tasks.workunit.client.1.smithi150.stdout:9/612: write d1/d8/de/df/d20/f50 [1046289,92214] 0 2024-03-22T10:42:59.408 INFO:tasks.workunit.client.0.smithi115.stdout:7/952: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f144 x:0 0 0 2024-03-22T10:42:59.409 INFO:tasks.workunit.client.1.smithi150.stdout:1/544: rmdir d1/d19/d57 39 2024-03-22T10:42:59.412 INFO:tasks.workunit.client.1.smithi150.stdout:1/545: write d1/d19/d50/f90 [485540,84756] 0 2024-03-22T10:42:59.414 INFO:tasks.workunit.client.0.smithi115.stdout:7/953: mknod d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/dbb/c145 0 2024-03-22T10:42:59.415 INFO:tasks.workunit.client.1.smithi150.stdout:0/590: write d0/d1/d84/d86/d42/d10/d38/d52/d68/fb0 [968818,30191] 0 2024-03-22T10:42:59.415 INFO:tasks.workunit.client.1.smithi150.stdout:1/546: chown d1/d19/f9e 254636 1 2024-03-22T10:42:59.416 INFO:tasks.workunit.client.1.smithi150.stdout:1/547: fsync d1/d19/d50/f5c 0 2024-03-22T10:42:59.419 INFO:tasks.workunit.client.1.smithi150.stdout:2/683: unlink d12/d19/db6/da9/d97/fc1 0 2024-03-22T10:42:59.423 INFO:tasks.workunit.client.1.smithi150.stdout:2/684: write d12/d19/db6/da9/d61/d98/fbb [906063,47721] 0 2024-03-22T10:42:59.425 INFO:tasks.workunit.client.1.smithi150.stdout:6/555: sync 2024-03-22T10:42:59.430 INFO:tasks.workunit.client.1.smithi150.stdout:5/511: chown d0/d8/dc/de/d14/d2e/d45/d56 1496333 1 2024-03-22T10:42:59.431 INFO:tasks.workunit.client.1.smithi150.stdout:9/613: creat d1/d8/de/df/d20/d32/d75/dc3/fd2 x:0 0 0 2024-03-22T10:42:59.432 INFO:tasks.workunit.client.1.smithi150.stdout:5/512: truncate d0/f38 959878 0 2024-03-22T10:42:59.433 INFO:tasks.workunit.client.1.smithi150.stdout:0/591: readlink d0/d1/d84/d86/d11/d66/l7d 0 2024-03-22T10:42:59.435 INFO:tasks.workunit.client.0.smithi115.stdout:7/954: write d2/d7/d12/d10/d23/f64 [318634,33619] 0 2024-03-22T10:42:59.438 INFO:tasks.workunit.client.1.smithi150.stdout:7/600: mknod d2/d5/d7/d8/d57/d4a/d8d/da9/cc4 0 2024-03-22T10:42:59.439 INFO:tasks.workunit.client.0.smithi115.stdout:7/955: creat d2/d62/f146 x:0 0 0 2024-03-22T10:42:59.443 INFO:tasks.workunit.client.0.smithi115.stdout:7/956: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/dc7/d11f/f147 x:0 0 0 2024-03-22T10:42:59.444 INFO:tasks.workunit.client.1.smithi150.stdout:1/548: creat d1/d79/d96/faf x:0 0 0 2024-03-22T10:42:59.445 INFO:tasks.workunit.client.1.smithi150.stdout:0/592: fsync d0/d1/d84/d86/d5a/f73 0 2024-03-22T10:42:59.447 INFO:tasks.workunit.client.1.smithi150.stdout:2/685: mkdir d12/d19/db6/d63/dd8 0 2024-03-22T10:42:59.449 INFO:tasks.workunit.client.0.smithi115.stdout:4/995: dread d1/d66/d106/d68/f89 [0,4194304] 0 2024-03-22T10:42:59.449 INFO:tasks.workunit.client.1.smithi150.stdout:2/686: fdatasync d12/d19/db6/da9/d61/d98/fbb 0 2024-03-22T10:42:59.451 INFO:tasks.workunit.client.0.smithi115.stdout:7/957: link d2/d7/l133 d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/l148 0 2024-03-22T10:42:59.451 INFO:tasks.workunit.client.1.smithi150.stdout:4/599: creat d0/d1b/fc0 x:0 0 0 2024-03-22T10:42:59.454 INFO:tasks.workunit.client.1.smithi150.stdout:6/556: rmdir db/dd/d15 39 2024-03-22T10:42:59.455 INFO:tasks.workunit.client.0.smithi115.stdout:7/958: truncate d2/d7/d12/d10/d23/d3d/dd2/da3/df3/f11e 1261699 0 2024-03-22T10:42:59.456 INFO:tasks.workunit.client.1.smithi150.stdout:7/601: dwrite d2/d5/d7/d8/d46/f66 [4194304,4194304] 0 2024-03-22T10:42:59.456 INFO:tasks.workunit.client.1.smithi150.stdout:5/513: mknod d0/d8/c98 0 2024-03-22T10:42:59.457 INFO:tasks.workunit.client.1.smithi150.stdout:1/549: symlink d1/d19/d71/lb0 0 2024-03-22T10:42:59.457 INFO:tasks.workunit.client.0.smithi115.stdout:7/959: read - d2/d7/d12/d10/d23/d3d/ff6 zero size 2024-03-22T10:42:59.459 INFO:tasks.workunit.client.1.smithi150.stdout:5/514: fdatasync d0/d8/dc/de/d14/f50 0 2024-03-22T10:42:59.460 INFO:tasks.workunit.client.1.smithi150.stdout:3/558: write d5/da/d62/f8e [1394645,41326] 0 2024-03-22T10:42:59.462 INFO:tasks.workunit.client.1.smithi150.stdout:2/687: rmdir d12/d19/db6/da9/d2c/d65 39 2024-03-22T10:42:59.465 INFO:tasks.workunit.client.1.smithi150.stdout:3/559: chown d5/d22/d2f/d38/d59/d88/f8a 906 1 2024-03-22T10:42:59.465 INFO:tasks.workunit.client.1.smithi150.stdout:9/614: creat d1/d8/de/df/fd3 x:0 0 0 2024-03-22T10:42:59.467 INFO:tasks.workunit.client.1.smithi150.stdout:0/593: fsync d0/f6d 0 2024-03-22T10:42:59.472 INFO:tasks.workunit.client.1.smithi150.stdout:8/624: link d0/d2/f86 d0/d3c/d71/d7a/d89/fcf 0 2024-03-22T10:42:59.472 INFO:tasks.workunit.client.1.smithi150.stdout:5/515: mknod d0/d8/dc/d1d/d89/c99 0 2024-03-22T10:42:59.474 INFO:tasks.workunit.client.1.smithi150.stdout:5/516: chown d0/d37/l3c 1935016 1 2024-03-22T10:42:59.476 INFO:tasks.workunit.client.1.smithi150.stdout:4/600: mkdir d0/d32/d7c/dc1 0 2024-03-22T10:42:59.476 INFO:tasks.workunit.client.1.smithi150.stdout:6/557: creat db/dd/d2e/fc7 x:0 0 0 2024-03-22T10:42:59.478 INFO:tasks.workunit.client.1.smithi150.stdout:2/688: chown d12/d19/db6/da9/d67/d8d/ccd 0 1 2024-03-22T10:42:59.478 INFO:tasks.workunit.client.1.smithi150.stdout:0/594: unlink d0/d1/d84/d86/d11/d66/l22 0 2024-03-22T10:42:59.479 INFO:tasks.workunit.client.1.smithi150.stdout:8/625: mkdir d0/d2/d46/da2/dd0 0 2024-03-22T10:42:59.482 INFO:tasks.workunit.client.1.smithi150.stdout:5/517: write d0/f22 [2440252,105917] 0 2024-03-22T10:42:59.484 INFO:tasks.workunit.client.1.smithi150.stdout:9/615: creat d1/d19/d7d/dc1/fd4 x:0 0 0 2024-03-22T10:42:59.488 INFO:tasks.workunit.client.1.smithi150.stdout:1/550: dwrite d1/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:42:59.494 INFO:tasks.workunit.client.1.smithi150.stdout:8/626: chown d0/d2/d19/d29/d2f/df/cbd 455569748 1 2024-03-22T10:42:59.497 INFO:tasks.workunit.client.1.smithi150.stdout:6/558: chown db/dd/d15/d70/c75 4022 1 2024-03-22T10:42:59.497 INFO:tasks.workunit.client.1.smithi150.stdout:9/616: stat d1/d8/de/df/d36/d40/f78 0 2024-03-22T10:42:59.498 INFO:tasks.workunit.client.1.smithi150.stdout:5/518: chown d0/d33/c57 790715425 1 2024-03-22T10:42:59.499 INFO:tasks.workunit.client.1.smithi150.stdout:7/602: getdents d2/d5/d7/d9/db/d9f 0 2024-03-22T10:42:59.500 INFO:tasks.workunit.client.1.smithi150.stdout:2/689: read d12/d19/d2d/f53 [174694,36412] 0 2024-03-22T10:42:59.502 INFO:tasks.workunit.client.1.smithi150.stdout:8/627: mkdir d0/d2/d19/dd1 0 2024-03-22T10:42:59.529 INFO:tasks.workunit.client.1.smithi150.stdout:5/519: creat d0/d8/f9a x:0 0 0 2024-03-22T10:42:59.531 INFO:tasks.workunit.client.1.smithi150.stdout:6/559: mkdir db/dd/d82/d84/dc8 0 2024-03-22T10:42:59.532 INFO:tasks.workunit.client.1.smithi150.stdout:8/628: fsync d0/d2/fe 0 2024-03-22T10:42:59.534 INFO:tasks.workunit.client.1.smithi150.stdout:2/690: mknod d12/d19/db6/da9/d67/cd9 0 2024-03-22T10:42:59.534 INFO:tasks.workunit.client.1.smithi150.stdout:5/520: fsync d0/d8/dc/de/f13 0 2024-03-22T10:42:59.536 INFO:tasks.workunit.client.1.smithi150.stdout:3/560: dread d5/f52 [0,4194304] 0 2024-03-22T10:42:59.539 INFO:tasks.workunit.client.1.smithi150.stdout:6/560: creat db/d52/fc9 x:0 0 0 2024-03-22T10:42:59.542 INFO:tasks.workunit.client.1.smithi150.stdout:0/595: dwrite d0/d1/d84/d86/d42/d10/d38/d52/d68/fb0 [0,4194304] 0 2024-03-22T10:42:59.544 INFO:tasks.workunit.client.1.smithi150.stdout:3/561: unlink d5/d22/d2f/d38/f39 0 2024-03-22T10:42:59.545 INFO:tasks.workunit.client.1.smithi150.stdout:5/521: dread - d0/d8/dc/de/d14/d2e/d46/d4d/f80 zero size 2024-03-22T10:42:59.545 INFO:tasks.workunit.client.1.smithi150.stdout:4/601: dwrite d0/d2f/f76 [0,4194304] 0 2024-03-22T10:42:59.547 INFO:tasks.workunit.client.1.smithi150.stdout:3/562: write d5/da/f45 [5699772,95999] 0 2024-03-22T10:42:59.548 INFO:tasks.workunit.client.1.smithi150.stdout:4/602: fsync d0/d47/f64 0 2024-03-22T10:42:59.550 INFO:tasks.workunit.client.1.smithi150.stdout:9/617: dread d1/d8/de/dbd/fc6 [0,4194304] 0 2024-03-22T10:42:59.552 INFO:tasks.workunit.client.1.smithi150.stdout:4/603: fsync d0/d1b/d4d/f92 0 2024-03-22T10:42:59.556 INFO:tasks.workunit.client.1.smithi150.stdout:0/596: creat d0/d1/d84/d86/da6/fbc x:0 0 0 2024-03-22T10:42:59.563 INFO:tasks.workunit.client.1.smithi150.stdout:7/603: dread d2/d5/d7/f4d [0,4194304] 0 2024-03-22T10:42:59.565 INFO:tasks.workunit.client.1.smithi150.stdout:4/604: write d0/d1b/d4d/d6c/d61/d71/fa4 [3431561,30013] 0 2024-03-22T10:42:59.567 INFO:tasks.workunit.client.1.smithi150.stdout:9/618: rmdir d1/d8/de/dc8 0 2024-03-22T10:42:59.570 INFO:tasks.workunit.client.1.smithi150.stdout:0/597: chown d0/d1/d84/d86/d42/c2d 29414 1 2024-03-22T10:42:59.571 INFO:tasks.workunit.client.1.smithi150.stdout:2/691: rename d12/d19/db6/da9/d97/d78/c7a to d12/d19/db6/cda 0 2024-03-22T10:42:59.579 INFO:tasks.workunit.client.1.smithi150.stdout:4/605: fsync d0/d1b/f23 0 2024-03-22T10:42:59.579 INFO:tasks.workunit.client.1.smithi150.stdout:5/522: dwrite d0/d8/dc/d1d/d53/f7d [0,4194304] 0 2024-03-22T10:42:59.583 INFO:tasks.workunit.client.1.smithi150.stdout:9/619: write d1/d49/f31 [19260,124869] 0 2024-03-22T10:42:59.589 INFO:tasks.workunit.client.1.smithi150.stdout:0/598: read d0/d1/d84/d86/d42/f4a [482111,74529] 0 2024-03-22T10:42:59.590 INFO:tasks.workunit.client.1.smithi150.stdout:3/563: truncate d5/d53/d7e/d95/fa1 1676890 0 2024-03-22T10:42:59.597 INFO:tasks.workunit.client.1.smithi150.stdout:8/629: dread d0/d2/d19/d29/d2f/df/f26 [0,4194304] 0 2024-03-22T10:42:59.599 INFO:tasks.workunit.client.1.smithi150.stdout:3/564: mkdir d5/da/d78/db7 0 2024-03-22T10:42:59.604 INFO:tasks.workunit.client.1.smithi150.stdout:7/604: link d2/d5/d7/d8/d57/d4a/d8d/cb8 d2/d5/d7/d8/d57/d54/d91/daf/cc5 0 2024-03-22T10:42:59.604 INFO:tasks.workunit.client.1.smithi150.stdout:6/561: dwrite db/dd/d15/d60/f86 [0,4194304] 0 2024-03-22T10:42:59.604 INFO:tasks.workunit.client.1.smithi150.stdout:9/620: unlink d1/d8/de/df/d20/d32/ccc 0 2024-03-22T10:42:59.606 INFO:tasks.workunit.client.1.smithi150.stdout:4/606: link d0/de/c59 d0/d47/d72/cc2 0 2024-03-22T10:42:59.611 INFO:tasks.workunit.client.1.smithi150.stdout:6/562: readlink db/dd/l13 0 2024-03-22T10:42:59.612 INFO:tasks.workunit.client.1.smithi150.stdout:4/607: read - d0/d1b/d4d/d6c/f9d zero size 2024-03-22T10:42:59.616 INFO:tasks.workunit.client.1.smithi150.stdout:0/599: getdents d0/d1/d84/d86/d11 0 2024-03-22T10:42:59.618 INFO:tasks.workunit.client.1.smithi150.stdout:8/630: write d0/d2/d19/d29/d2f/df/f26 [1902769,101332] 0 2024-03-22T10:42:59.621 INFO:tasks.workunit.client.1.smithi150.stdout:2/692: dwrite d12/d19/db6/da9/d67/d3c/f92 [0,4194304] 0 2024-03-22T10:42:59.631 INFO:tasks.workunit.client.1.smithi150.stdout:8/631: write d0/d2/d19/d29/d2e/f3a [4705759,79111] 0 2024-03-22T10:42:59.635 INFO:tasks.workunit.client.1.smithi150.stdout:4/608: symlink d0/d2f/d63/d7b/lc3 0 2024-03-22T10:42:59.637 INFO:tasks.workunit.client.1.smithi150.stdout:0/600: rename d0/d1/d84/d86/d11/d66/f8b to d0/d1/d84/d86/d42/d10/d38/d52/fbd 0 2024-03-22T10:42:59.638 INFO:tasks.workunit.client.0.smithi115.stdout:9/976: sync 2024-03-22T10:42:59.638 INFO:tasks.workunit.client.0.smithi115.stdout:6/915: sync 2024-03-22T10:42:59.638 INFO:tasks.workunit.client.0.smithi115.stdout:2/949: sync 2024-03-22T10:42:59.638 INFO:tasks.workunit.client.0.smithi115.stdout:0/964: sync 2024-03-22T10:42:59.643 INFO:tasks.workunit.client.1.smithi150.stdout:5/523: dwrite d0/d8/dc/de/d14/d2e/d46/d4d/f6c [4194304,4194304] 0 2024-03-22T10:42:59.644 INFO:tasks.workunit.client.1.smithi150.stdout:3/565: dwrite d5/f31 [0,4194304] 0 2024-03-22T10:42:59.648 INFO:tasks.workunit.client.0.smithi115.stdout:9/977: link d6/d7/d87/f8c d6/d7/d6a/d6f/f154 0 2024-03-22T10:42:59.649 INFO:tasks.workunit.client.1.smithi150.stdout:8/632: rename d0/d2/d19/d29/d31 to d0/d5d/dd2 0 2024-03-22T10:42:59.649 INFO:tasks.workunit.client.1.smithi150.stdout:3/566: readlink d5/l16 0 2024-03-22T10:42:59.650 INFO:tasks.workunit.client.0.smithi115.stdout:0/965: link d3/d4/d6d/f9c d3/f13c 0 2024-03-22T10:42:59.650 INFO:tasks.workunit.client.1.smithi150.stdout:5/524: chown d0/d8/f23 0 1 2024-03-22T10:42:59.652 INFO:tasks.workunit.client.1.smithi150.stdout:6/563: dread db/dd/d15/d60/f67 [0,4194304] 0 2024-03-22T10:42:59.652 INFO:tasks.workunit.client.1.smithi150.stdout:3/567: chown d5/da/f40 0 1 2024-03-22T10:42:59.653 INFO:tasks.workunit.client.1.smithi150.stdout:0/601: unlink d0/d1/d84/d86/d11/d66/c91 0 2024-03-22T10:42:59.655 INFO:tasks.workunit.client.0.smithi115.stdout:9/978: creat d6/d7/d20/d22/d23/d106/f155 x:0 0 0 2024-03-22T10:42:59.656 INFO:tasks.workunit.client.1.smithi150.stdout:7/605: truncate d2/d5/d7/d9/db/f76 3336939 0 2024-03-22T10:42:59.658 INFO:tasks.workunit.client.1.smithi150.stdout:5/525: creat d0/d8/dc/d1d/d89/f9b x:0 0 0 2024-03-22T10:42:59.663 INFO:tasks.workunit.client.1.smithi150.stdout:3/568: write d5/d53/f75 [1302797,12322] 0 2024-03-22T10:42:59.664 INFO:tasks.workunit.client.1.smithi150.stdout:6/564: readlink db/dd/d15/d60/l69 0 2024-03-22T10:42:59.664 INFO:tasks.workunit.client.0.smithi115.stdout:2/950: dwrite da/d25/d3d/d57/d7d/da1/dba/ddf/ffa [0,4194304] 0 2024-03-22T10:42:59.665 INFO:tasks.workunit.client.1.smithi150.stdout:7/606: creat d2/d5/d7/d8/d57/d54/d91/fc6 x:0 0 0 2024-03-22T10:42:59.670 INFO:tasks.workunit.client.1.smithi150.stdout:0/602: creat d0/d1/d84/d86/d42/d10/d38/d79/fbe x:0 0 0 2024-03-22T10:42:59.670 INFO:tasks.workunit.client.1.smithi150.stdout:6/565: symlink db/d52/lca 0 2024-03-22T10:42:59.671 INFO:tasks.workunit.client.1.smithi150.stdout:4/609: dread d0/f8 [0,4194304] 0 2024-03-22T10:42:59.673 INFO:tasks.workunit.client.0.smithi115.stdout:2/951: dread - da/d25/d30/fd2 zero size 2024-03-22T10:42:59.678 INFO:tasks.workunit.client.1.smithi150.stdout:0/603: write d0/d1/d84/d86/d11/d66/d37/f3d [6975966,15100] 0 2024-03-22T10:42:59.678 INFO:tasks.workunit.client.1.smithi150.stdout:9/621: dwrite d1/d8/de/df/d20/d32/f2a [0,4194304] 0 2024-03-22T10:42:59.679 INFO:tasks.workunit.client.1.smithi150.stdout:3/569: mknod d5/d22/d2f/d38/d59/d88/cb8 0 2024-03-22T10:42:59.679 INFO:tasks.workunit.client.1.smithi150.stdout:1/551: sync 2024-03-22T10:42:59.681 INFO:tasks.workunit.client.0.smithi115.stdout:2/952: mknod da/d1c/d67/d78/d86/c11e 0 2024-03-22T10:42:59.684 INFO:tasks.workunit.client.0.smithi115.stdout:0/966: dwrite d3/d7/d1b/d1f/d78/dd9/fe3 [0,4194304] 0 2024-03-22T10:42:59.686 INFO:tasks.workunit.client.1.smithi150.stdout:7/607: symlink d2/d5/d7/d8/d57/d4a/d8d/da9/lc7 0 2024-03-22T10:42:59.689 INFO:tasks.workunit.client.0.smithi115.stdout:0/967: symlink d3/d7/d1b/d35/l13d 0 2024-03-22T10:42:59.689 INFO:tasks.workunit.client.0.smithi115.stdout:6/916: dread d4/d36/f5c [0,4194304] 0 2024-03-22T10:42:59.690 INFO:tasks.workunit.client.0.smithi115.stdout:0/968: fdatasync d3/d52/f57 0 2024-03-22T10:42:59.691 INFO:tasks.workunit.client.1.smithi150.stdout:6/566: read db/dd/d15/d30/f51 [2295280,123260] 0 2024-03-22T10:42:59.691 INFO:tasks.workunit.client.1.smithi150.stdout:1/552: rename d1/da/f72 to d1/da/d2a/d65/d78/fb1 0 2024-03-22T10:42:59.691 INFO:tasks.workunit.client.0.smithi115.stdout:2/953: mknod da/d25/d3d/d57/d7d/da1/c11f 0 2024-03-22T10:42:59.694 INFO:tasks.workunit.client.1.smithi150.stdout:5/526: dread d0/d8/dc/d1d/f24 [0,4194304] 0 2024-03-22T10:42:59.694 INFO:tasks.workunit.client.1.smithi150.stdout:6/567: read - db/dd/d2e/fbc zero size 2024-03-22T10:42:59.694 INFO:tasks.workunit.client.1.smithi150.stdout:7/608: write d2/d5/d7/d9/f62 [578399,47988] 0 2024-03-22T10:42:59.694 INFO:tasks.workunit.client.0.smithi115.stdout:6/917: fdatasync d4/d11/d12/d1c/f46 0 2024-03-22T10:42:59.695 INFO:tasks.workunit.client.0.smithi115.stdout:2/954: mknod da/d1c/d67/d78/d86/d9d/c120 0 2024-03-22T10:42:59.697 INFO:tasks.workunit.client.1.smithi150.stdout:6/568: dread - db/dd/d15/d60/d50/dc1/fba zero size 2024-03-22T10:42:59.697 INFO:tasks.workunit.client.0.smithi115.stdout:6/918: mknod d4/d11/c12c 0 2024-03-22T10:42:59.700 INFO:tasks.workunit.client.1.smithi150.stdout:2/693: dread d12/d19/db6/da9/d61/d98/f6f [0,4194304] 0 2024-03-22T10:42:59.700 INFO:tasks.workunit.client.0.smithi115.stdout:0/969: mknod d3/da/d4c/d69/c13e 0 2024-03-22T10:42:59.700 INFO:tasks.workunit.client.0.smithi115.stdout:2/955: unlink da/d25/d51/cbb 0 2024-03-22T10:42:59.702 INFO:tasks.workunit.client.0.smithi115.stdout:0/970: chown d3/d4/d6d/fb0 2015184081 1 2024-03-22T10:42:59.704 INFO:tasks.workunit.client.0.smithi115.stdout:6/919: link d4/d36/d48/lcb d4/d36/d48/l12d 0 2024-03-22T10:42:59.706 INFO:tasks.workunit.client.0.smithi115.stdout:2/956: creat da/d25/d3d/d57/d7d/da1/df9/f121 x:0 0 0 2024-03-22T10:42:59.707 INFO:tasks.workunit.client.1.smithi150.stdout:8/633: truncate d0/d3c/f56 20495 0 2024-03-22T10:42:59.707 INFO:tasks.workunit.client.0.smithi115.stdout:6/920: chown d4/d11/f54 442 1 2024-03-22T10:42:59.708 INFO:tasks.workunit.client.1.smithi150.stdout:9/622: dwrite d1/d8/de/df/d24/d4e/f93 [4194304,4194304] 0 2024-03-22T10:42:59.709 INFO:tasks.workunit.client.0.smithi115.stdout:0/971: unlink d3/d7/d1c/f5d 0 2024-03-22T10:42:59.712 INFO:tasks.workunit.client.0.smithi115.stdout:0/972: dread - d3/da/d4c/d74/d84/dac/dfc/f107 zero size 2024-03-22T10:42:59.715 INFO:tasks.workunit.client.1.smithi150.stdout:9/623: chown d1/d8/de/df/d24/cce 292109988 1 2024-03-22T10:42:59.716 INFO:tasks.workunit.client.1.smithi150.stdout:7/609: readlink d2/d5/l6b 0 2024-03-22T10:42:59.721 INFO:tasks.workunit.client.1.smithi150.stdout:5/527: mkdir d0/d8/dc/de/d14/d2e/d46/d9c 0 2024-03-22T10:42:59.725 INFO:tasks.workunit.client.1.smithi150.stdout:0/604: dwrite d0/d1/d84/d86/d42/d10/d38/d79/f8f [0,4194304] 0 2024-03-22T10:42:59.725 INFO:tasks.workunit.client.1.smithi150.stdout:2/694: chown d12/d19/db6/da9/f2a 92043 1 2024-03-22T10:42:59.725 INFO:tasks.workunit.client.1.smithi150.stdout:7/610: write d2/d5/d7/d8/d57/d4a/f7f [489390,109933] 0 2024-03-22T10:42:59.726 INFO:tasks.workunit.client.1.smithi150.stdout:7/611: fdatasync d2/d5/d7/d8/d46/f4e 0 2024-03-22T10:42:59.732 INFO:tasks.workunit.client.1.smithi150.stdout:7/612: chown d2/d5/d7/d9/db/d9f/d45/cb2 40183033 1 2024-03-22T10:42:59.734 INFO:tasks.workunit.client.1.smithi150.stdout:6/569: rename db/dd/d15/d60/lae to db/dd/d15/d6a/lcb 0 2024-03-22T10:42:59.737 INFO:tasks.workunit.client.1.smithi150.stdout:4/610: getdents d0/d1b 0 2024-03-22T10:42:59.741 INFO:tasks.workunit.client.1.smithi150.stdout:9/624: unlink d1/d19/d7d/dc1/fd4 0 2024-03-22T10:42:59.745 INFO:tasks.workunit.client.1.smithi150.stdout:0/605: dwrite d0/d1/d84/d86/d42/d10/f43 [0,4194304] 0 2024-03-22T10:42:59.747 INFO:tasks.workunit.client.1.smithi150.stdout:3/570: getdents d5/d22/d2f/d9c 0 2024-03-22T10:42:59.748 INFO:tasks.workunit.client.0.smithi115.stdout:2/957: truncate da/f14 553391 0 2024-03-22T10:42:59.749 INFO:tasks.workunit.client.0.smithi115.stdout:0/973: write d3/d4/d6d/fb0 [3365920,69013] 0 2024-03-22T10:42:59.751 INFO:tasks.workunit.client.1.smithi150.stdout:2/695: stat d12/d19/db6/da9/d67/d8d/da6 0 2024-03-22T10:42:59.752 INFO:tasks.workunit.client.0.smithi115.stdout:2/958: symlink da/d25/d3d/d7e/l122 0 2024-03-22T10:42:59.753 INFO:tasks.workunit.client.0.smithi115.stdout:0/974: mknod d3/d7/d1c/d6c/c13f 0 2024-03-22T10:42:59.755 INFO:tasks.workunit.client.1.smithi150.stdout:5/528: creat d0/d8/dc/d1d/d53/d63/f9d x:0 0 0 2024-03-22T10:42:59.756 INFO:tasks.workunit.client.0.smithi115.stdout:0/975: mknod d3/d4/d14/d64/c140 0 2024-03-22T10:42:59.757 INFO:tasks.workunit.client.0.smithi115.stdout:0/976: fdatasync d3/d7/d1c/f102 0 2024-03-22T10:42:59.760 INFO:tasks.workunit.client.1.smithi150.stdout:7/613: unlink d2/d5/d7/d9/db/f1a 0 2024-03-22T10:42:59.764 INFO:tasks.workunit.client.1.smithi150.stdout:8/634: rename d0/d2/d19/d29/d69/l81 to d0/d2/d19/dd1/ld3 0 2024-03-22T10:42:59.764 INFO:tasks.workunit.client.1.smithi150.stdout:5/529: rename d0/d8/dc/de/d14/d68 to d0/d8/dc/de/d14/d68/d9e 22 2024-03-22T10:42:59.766 INFO:tasks.workunit.client.1.smithi150.stdout:1/553: rmdir d1/d19/d1a/d91 0 2024-03-22T10:42:59.771 INFO:tasks.workunit.client.1.smithi150.stdout:0/606: unlink d0/d1/d84/d86/da6/fbc 0 2024-03-22T10:42:59.771 INFO:tasks.workunit.client.1.smithi150.stdout:3/571: rmdir d5/d22/d2f 39 2024-03-22T10:42:59.773 INFO:tasks.workunit.client.1.smithi150.stdout:2/696: creat d12/d19/db6/da9/d67/d8d/da6/fdb x:0 0 0 2024-03-22T10:42:59.773 INFO:tasks.workunit.client.0.smithi115.stdout:0/977: dwrite d3/d7/f1d [0,4194304] 0 2024-03-22T10:42:59.775 INFO:tasks.workunit.client.0.smithi115.stdout:2/959: dwrite da/f14 [0,4194304] 0 2024-03-22T10:42:59.775 INFO:tasks.workunit.client.1.smithi150.stdout:6/570: symlink db/dd/lcc 0 2024-03-22T10:42:59.779 INFO:tasks.workunit.client.1.smithi150.stdout:0/607: symlink d0/d1/d84/d86/d11/d66/lbf 0 2024-03-22T10:42:59.781 INFO:tasks.workunit.client.0.smithi115.stdout:2/960: mkdir da/d25/d3d/d7e/d90/dbf/d54/d123 0 2024-03-22T10:42:59.782 INFO:tasks.workunit.client.0.smithi115.stdout:0/978: link d3/d4/c25 d3/d7/d1c/c141 0 2024-03-22T10:42:59.783 INFO:tasks.workunit.client.1.smithi150.stdout:5/530: dread d0/d8/dc/de/d14/f58 [0,4194304] 0 2024-03-22T10:42:59.784 INFO:tasks.workunit.client.0.smithi115.stdout:2/961: readlink da/d25/d3d/d57/l58 0 2024-03-22T10:42:59.786 INFO:tasks.workunit.client.1.smithi150.stdout:1/554: symlink d1/da/d15/d18/lb2 0 2024-03-22T10:42:59.786 INFO:tasks.workunit.client.0.smithi115.stdout:0/979: mknod d3/d7/d1b/df8/c142 0 2024-03-22T10:42:59.789 INFO:tasks.workunit.client.0.smithi115.stdout:2/962: creat da/d25/d3d/d7e/d90/d96/f124 x:0 0 0 2024-03-22T10:42:59.790 INFO:tasks.workunit.client.1.smithi150.stdout:3/572: rename d5/ca3 to d5/da/d78/cb9 0 2024-03-22T10:42:59.791 INFO:tasks.workunit.client.0.smithi115.stdout:4/996: sync 2024-03-22T10:42:59.791 INFO:tasks.workunit.client.0.smithi115.stdout:7/960: sync 2024-03-22T10:42:59.792 INFO:tasks.workunit.client.1.smithi150.stdout:2/697: unlink d12/d19/db6/d63/l89 0 2024-03-22T10:42:59.792 INFO:tasks.workunit.client.0.smithi115.stdout:2/963: chown da/d25/d3d/dad 279905011 1 2024-03-22T10:42:59.793 INFO:tasks.workunit.client.1.smithi150.stdout:0/608: creat d0/d1/d84/d86/d11/d66/d35/d5f/fc0 x:0 0 0 2024-03-22T10:42:59.793 INFO:tasks.workunit.client.0.smithi115.stdout:4/997: stat d1/d66/d106/d68/f6c 0 2024-03-22T10:42:59.794 INFO:tasks.workunit.client.0.smithi115.stdout:2/964: chown da/d25/d3d/d57/d7d/da1/df9 0 1 2024-03-22T10:42:59.795 INFO:tasks.workunit.client.1.smithi150.stdout:4/611: getdents d0/d2f 0 2024-03-22T10:42:59.797 INFO:tasks.workunit.client.1.smithi150.stdout:9/625: getdents d1/d8/d95 0 2024-03-22T10:42:59.797 INFO:tasks.workunit.client.0.smithi115.stdout:4/998: creat d1/d30/d4c/d86/d9f/d4f/f154 x:0 0 0 2024-03-22T10:42:59.801 INFO:tasks.workunit.client.1.smithi150.stdout:6/571: dwrite db/dd/d82/f9a [0,4194304] 0 2024-03-22T10:42:59.802 INFO:tasks.workunit.client.0.smithi115.stdout:4/999: chown d1/d30/d4c/d86/d9f/d75/d99/da8/cbc 348270 1 2024-03-22T10:42:59.803 INFO:tasks.workunit.client.0.smithi115.stdout:2/965: unlink da/d25/d3d/d57/da7/df7/l117 0 2024-03-22T10:42:59.803 INFO:tasks.workunit.client.1.smithi150.stdout:6/572: fdatasync db/dd/d15/d60/d50/fb0 0 2024-03-22T10:42:59.803 INFO:tasks.workunit.client.1.smithi150.stdout:3/573: mkdir d5/da/d62/dba 0 2024-03-22T10:42:59.805 INFO:tasks.workunit.client.1.smithi150.stdout:9/626: truncate d1/d8/de/df/d20/d32/d75/d99/da7/fba 550423 0 2024-03-22T10:42:59.806 INFO:tasks.workunit.client.0.smithi115.stdout:2/966: chown da/d25/d51/c113 5 1 2024-03-22T10:42:59.807 INFO:tasks.workunit.client.1.smithi150.stdout:9/627: truncate d1/d19/f79 4410094 0 2024-03-22T10:42:59.808 INFO:tasks.workunit.client.0.smithi115.stdout:0/980: dwrite d3/f127 [0,4194304] 0 2024-03-22T10:42:59.810 INFO:tasks.workunit.client.0.smithi115.stdout:2/967: stat da/d25/d3d/dad/db3/c109 0 2024-03-22T10:42:59.812 INFO:tasks.workunit.client.1.smithi150.stdout:4/612: creat d0/d1b/d68/fc4 x:0 0 0 2024-03-22T10:42:59.812 INFO:tasks.workunit.client.1.smithi150.stdout:1/555: getdents d1/da/d15/d18/d4b/d8a 0 2024-03-22T10:42:59.815 INFO:tasks.workunit.client.0.smithi115.stdout:0/981: fsync d3/d7/d1b/f47 0 2024-03-22T10:42:59.815 INFO:tasks.workunit.client.0.smithi115.stdout:7/961: dwrite d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/f131 [0,4194304] 0 2024-03-22T10:42:59.817 INFO:tasks.workunit.client.1.smithi150.stdout:4/613: rmdir d0/d32 39 2024-03-22T10:42:59.818 INFO:tasks.workunit.client.1.smithi150.stdout:2/698: truncate d12/d19/db6/da9/d67/f51 208357 0 2024-03-22T10:42:59.821 INFO:tasks.workunit.client.1.smithi150.stdout:0/609: link d0/d1/d84/d86/d42/d10/fab d0/d1/fc1 0 2024-03-22T10:42:59.824 INFO:tasks.workunit.client.1.smithi150.stdout:6/573: creat db/dd/d15/d60/fcd x:0 0 0 2024-03-22T10:42:59.826 INFO:tasks.workunit.client.1.smithi150.stdout:0/610: chown d0/d1/d84/d86/d42/d10/d38/d52/d9b/dac 14477 1 2024-03-22T10:42:59.829 INFO:tasks.workunit.client.0.smithi115.stdout:2/968: rename da/d25/d51/lec to da/d25/d3d/dad/l125 0 2024-03-22T10:42:59.831 INFO:tasks.workunit.client.1.smithi150.stdout:8/635: dread d0/d3c/f56 [0,4194304] 0 2024-03-22T10:42:59.832 INFO:tasks.workunit.client.0.smithi115.stdout:2/969: dread - da/d25/d3d/d57/da7/f111 zero size 2024-03-22T10:42:59.833 INFO:tasks.workunit.client.1.smithi150.stdout:4/614: mknod d0/d47/d72/db6/cc5 0 2024-03-22T10:42:59.833 INFO:tasks.workunit.client.0.smithi115.stdout:2/970: chown da/d25/d3d 11816487 1 2024-03-22T10:42:59.836 INFO:tasks.workunit.client.1.smithi150.stdout:7/614: truncate d2/d5/d7/d8/d57/d4a/f39 7594780 0 2024-03-22T10:42:59.838 INFO:tasks.workunit.client.1.smithi150.stdout:6/574: symlink db/dd/d15/d28/d6c/d9b/lce 0 2024-03-22T10:42:59.839 INFO:tasks.workunit.client.0.smithi115.stdout:2/971: link da/l17 da/d25/d30/dcd/l126 0 2024-03-22T10:42:59.840 INFO:tasks.workunit.client.0.smithi115.stdout:2/972: chown da/d1c/d67/d78/d86/cc1 1341 1 2024-03-22T10:42:59.841 INFO:tasks.workunit.client.1.smithi150.stdout:0/611: mkdir d0/d1/d84/d86/d42/d10/d38/d52/d9b/dc2 0 2024-03-22T10:42:59.843 INFO:tasks.workunit.client.1.smithi150.stdout:5/531: truncate d0/f2b 2252842 0 2024-03-22T10:42:59.843 INFO:tasks.workunit.client.0.smithi115.stdout:0/982: truncate d3/d7/d1c/d6c/d9a/ffb 1497671 0 2024-03-22T10:42:59.845 INFO:tasks.workunit.client.0.smithi115.stdout:2/973: creat da/d25/d3d/d57/d7d/da1/dba/f127 x:0 0 0 2024-03-22T10:42:59.848 INFO:tasks.workunit.client.1.smithi150.stdout:6/575: mknod db/dd/d15/d28/d6c/ccf 0 2024-03-22T10:42:59.849 INFO:tasks.workunit.client.0.smithi115.stdout:0/983: creat d3/da/d4c/da3/da0/f143 x:0 0 0 2024-03-22T10:42:59.851 INFO:tasks.workunit.client.1.smithi150.stdout:9/628: truncate d1/d8/de/df/d36/f96 302475 0 2024-03-22T10:42:59.853 INFO:tasks.workunit.client.0.smithi115.stdout:0/984: creat d3/d4/f144 x:0 0 0 2024-03-22T10:42:59.854 INFO:tasks.workunit.client.1.smithi150.stdout:4/615: dwrite d0/f14 [0,4194304] 0 2024-03-22T10:42:59.856 INFO:tasks.workunit.client.1.smithi150.stdout:7/615: symlink d2/d5/d7/d8/d57/d4a/lc8 0 2024-03-22T10:42:59.861 INFO:tasks.workunit.client.0.smithi115.stdout:0/985: dread d3/d4/f60 [0,4194304] 0 2024-03-22T10:42:59.862 INFO:tasks.workunit.client.1.smithi150.stdout:8/636: getdents d0/d3c/d71 0 2024-03-22T10:42:59.864 INFO:tasks.workunit.client.1.smithi150.stdout:1/556: truncate d1/d19/d57/f62 3205100 0 2024-03-22T10:42:59.864 INFO:tasks.workunit.client.1.smithi150.stdout:6/576: mkdir db/d52/d9f/dc6/dd0 0 2024-03-22T10:42:59.865 INFO:tasks.workunit.client.1.smithi150.stdout:4/616: mkdir d0/d1b/d4d/d9f/dc6 0 2024-03-22T10:42:59.868 INFO:tasks.workunit.client.0.smithi115.stdout:2/974: dwrite da/d25/d83/f8a [0,4194304] 0 2024-03-22T10:42:59.869 INFO:tasks.workunit.client.1.smithi150.stdout:0/612: creat d0/d1/d84/d86/d42/d10/d38/fc3 x:0 0 0 2024-03-22T10:42:59.873 INFO:tasks.workunit.client.0.smithi115.stdout:7/962: dread d2/d7/d12/d10/d19/d42/d46/f115 [0,4194304] 0 2024-03-22T10:42:59.873 INFO:tasks.workunit.client.0.smithi115.stdout:6/921: sync 2024-03-22T10:42:59.874 INFO:tasks.workunit.client.0.smithi115.stdout:9/979: sync 2024-03-22T10:42:59.874 INFO:tasks.workunit.client.0.smithi115.stdout:0/986: rename d3/d4/d14/l126 to d3/da/d4c/d74/d84/dac/df4/l145 0 2024-03-22T10:42:59.879 INFO:tasks.workunit.client.0.smithi115.stdout:2/975: truncate da/d25/d30/fd2 990597 0 2024-03-22T10:42:59.880 INFO:tasks.workunit.client.1.smithi150.stdout:5/532: dwrite d0/d8/dc/de/d14/d2e/d45/f78 [0,4194304] 0 2024-03-22T10:42:59.881 INFO:tasks.workunit.client.1.smithi150.stdout:7/616: rename d2/d5/d7/d9/db/d9f/d96/c79 to d2/d5/d7/d8/d57/d54/d91/daf/cc9 0 2024-03-22T10:42:59.882 INFO:tasks.workunit.client.0.smithi115.stdout:0/987: creat d3/d7/d6b/d94/f146 x:0 0 0 2024-03-22T10:42:59.884 INFO:tasks.workunit.client.0.smithi115.stdout:9/980: mknod d6/d7/d48/d53/dc9/c156 0 2024-03-22T10:42:59.884 INFO:tasks.workunit.client.1.smithi150.stdout:8/637: truncate d0/d2/d19/d29/d37/f9c 283831 0 2024-03-22T10:42:59.886 INFO:tasks.workunit.client.1.smithi150.stdout:2/699: dread d12/d19/d2d/f36 [0,4194304] 0 2024-03-22T10:42:59.887 INFO:tasks.workunit.client.0.smithi115.stdout:0/988: dread d3/d7/d1b/d93/fee [0,4194304] 0 2024-03-22T10:42:59.887 INFO:tasks.workunit.client.1.smithi150.stdout:6/577: mknod db/dd/d15/cd1 0 2024-03-22T10:42:59.891 INFO:tasks.workunit.client.0.smithi115.stdout:6/922: mkdir d4/d13/d33/d103/d12e 0 2024-03-22T10:42:59.893 INFO:tasks.workunit.client.1.smithi150.stdout:4/617: mkdir d0/d32/d7c/dc7 0 2024-03-22T10:42:59.893 INFO:tasks.workunit.client.0.smithi115.stdout:0/989: write d3/d7/db7/f129 [894713,67211] 0 2024-03-22T10:42:59.894 INFO:tasks.workunit.client.1.smithi150.stdout:9/629: dwrite d1/d8/fa1 [4194304,4194304] 0 2024-03-22T10:42:59.895 INFO:tasks.workunit.client.0.smithi115.stdout:9/981: unlink d6/d7/d20/d22/d23/c12f 0 2024-03-22T10:42:59.897 INFO:tasks.workunit.client.0.smithi115.stdout:0/990: fdatasync d3/da/d4c/d69/fbd 0 2024-03-22T10:42:59.898 INFO:tasks.workunit.client.0.smithi115.stdout:6/923: truncate d4/d13/f25 2485580 0 2024-03-22T10:42:59.898 INFO:tasks.workunit.client.1.smithi150.stdout:9/630: read d1/d8/fa1 [3204218,32347] 0 2024-03-22T10:42:59.903 INFO:tasks.workunit.client.0.smithi115.stdout:6/924: unlink d4/d11/ld2 0 2024-03-22T10:42:59.905 INFO:tasks.workunit.client.0.smithi115.stdout:0/991: write d3/d7/d1b/d1f/d78/dd9/f138 [800562,56597] 0 2024-03-22T10:42:59.908 INFO:tasks.workunit.client.0.smithi115.stdout:6/925: mkdir d4/d11/dc7/d11c/d12f 0 2024-03-22T10:42:59.910 INFO:tasks.workunit.client.0.smithi115.stdout:0/992: chown d3/d7/d1b/d1f/d78/dc1/l10a 347935611 1 2024-03-22T10:42:59.911 INFO:tasks.workunit.client.1.smithi150.stdout:4/618: dwrite d0/d32/f56 [0,4194304] 0 2024-03-22T10:42:59.912 INFO:tasks.workunit.client.1.smithi150.stdout:6/578: creat db/dd/d2e/fd2 x:0 0 0 2024-03-22T10:42:59.912 INFO:tasks.workunit.client.1.smithi150.stdout:2/700: chown d12/d21/f4f 24901 1 2024-03-22T10:42:59.912 INFO:tasks.workunit.client.0.smithi115.stdout:6/926: read d4/d11/d91/fb0 [430244,93562] 0 2024-03-22T10:42:59.915 INFO:tasks.workunit.client.1.smithi150.stdout:0/613: link d0/d1/d84/d86/d42/d10/d38/d52/fbd d0/d1/d84/d86/d11/fc4 0 2024-03-22T10:42:59.915 INFO:tasks.workunit.client.0.smithi115.stdout:9/982: dwrite d6/d7/d30/d37/dbe/d110/f140 [0,4194304] 0 2024-03-22T10:42:59.922 INFO:tasks.workunit.client.1.smithi150.stdout:9/631: rename d1/d8/de/df/d20/d32/c92 to d1/d8/de/df/da2/cd5 0 2024-03-22T10:42:59.923 INFO:tasks.workunit.client.0.smithi115.stdout:7/963: write d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/dbb/fdd [49265,4847] 0 2024-03-22T10:42:59.924 INFO:tasks.workunit.client.1.smithi150.stdout:9/632: stat d1/d8/de/df/d24/f4a 0 2024-03-22T10:42:59.926 INFO:tasks.workunit.client.1.smithi150.stdout:9/633: fdatasync d1/d8/de/d73/f74 0 2024-03-22T10:42:59.927 INFO:tasks.workunit.client.1.smithi150.stdout:7/617: creat d2/d5/fca x:0 0 0 2024-03-22T10:42:59.929 INFO:tasks.workunit.client.1.smithi150.stdout:1/557: truncate d1/f3c 3571575 0 2024-03-22T10:42:59.930 INFO:tasks.workunit.client.0.smithi115.stdout:9/983: rmdir d6/d7/d20/d22/de5/d13a 0 2024-03-22T10:42:59.933 INFO:tasks.workunit.client.1.smithi150.stdout:1/558: write d1/da/d15/f93 [730194,57872] 0 2024-03-22T10:42:59.934 INFO:tasks.workunit.client.1.smithi150.stdout:7/618: read d2/d5/d7/d9/db/d9f/d45/f72 [3938992,9564] 0 2024-03-22T10:42:59.935 INFO:tasks.workunit.client.0.smithi115.stdout:7/964: symlink d2/d7/dc6/l149 0 2024-03-22T10:42:59.936 INFO:tasks.workunit.client.1.smithi150.stdout:8/638: write d0/d2/f5f [1076901,18028] 0 2024-03-22T10:42:59.936 INFO:tasks.workunit.client.1.smithi150.stdout:9/634: mkdir d1/d8/dd6 0 2024-03-22T10:42:59.937 INFO:tasks.workunit.client.1.smithi150.stdout:1/559: read - d1/da/f67 zero size 2024-03-22T10:42:59.941 INFO:tasks.workunit.client.0.smithi115.stdout:6/927: write d4/d36/d48/d72/fdc [476033,74767] 0 2024-03-22T10:42:59.942 INFO:tasks.workunit.client.0.smithi115.stdout:2/976: dwrite da/d25/d3d/d7e/d90/fd9 [0,4194304] 0 2024-03-22T10:42:59.946 INFO:tasks.workunit.client.0.smithi115.stdout:0/993: dwrite d3/da/d4c/d69/d6a/f86 [0,4194304] 0 2024-03-22T10:42:59.947 INFO:tasks.workunit.client.1.smithi150.stdout:1/560: unlink d1/d19/c6f 0 2024-03-22T10:42:59.947 INFO:tasks.workunit.client.0.smithi115.stdout:9/984: truncate d6/d7/d48/df4/f11c 3146079 0 2024-03-22T10:42:59.948 INFO:tasks.workunit.client.1.smithi150.stdout:7/619: mkdir d2/d5/d7/d8/d57/d4a/d8d/da9/dcb 0 2024-03-22T10:42:59.949 INFO:tasks.workunit.client.0.smithi115.stdout:0/994: stat d3/d7/db7/d121 0 2024-03-22T10:42:59.949 INFO:tasks.workunit.client.0.smithi115.stdout:9/985: readlink d6/d7/d20/d22/d3f/ld0 0 2024-03-22T10:42:59.954 INFO:tasks.workunit.client.1.smithi150.stdout:1/561: write d1/da/d15/d18/f4a [3365462,62650] 0 2024-03-22T10:42:59.957 INFO:tasks.workunit.client.0.smithi115.stdout:0/995: rename d3/d7/d1b/c114 to d3/d7/d1c/d6c/c147 0 2024-03-22T10:42:59.957 INFO:tasks.workunit.client.0.smithi115.stdout:2/977: symlink da/d25/d3d/d57/d7d/da1/ddc/l128 0 2024-03-22T10:42:59.958 INFO:tasks.workunit.client.1.smithi150.stdout:1/562: write d1/d19/d50/f5c [881066,111213] 0 2024-03-22T10:42:59.959 INFO:tasks.workunit.client.0.smithi115.stdout:2/978: dread - da/d25/d83/fd0 zero size 2024-03-22T10:42:59.959 INFO:tasks.workunit.client.0.smithi115.stdout:0/996: readlink d3/d7/d1b/d35/l53 0 2024-03-22T10:42:59.963 INFO:tasks.workunit.client.0.smithi115.stdout:0/997: creat d3/d7/db7/f148 x:0 0 0 2024-03-22T10:42:59.964 INFO:tasks.workunit.client.1.smithi150.stdout:0/614: symlink d0/d1/d84/d86/d11/d66/d35/d5f/da0/lc5 0 2024-03-22T10:42:59.964 INFO:tasks.workunit.client.0.smithi115.stdout:2/979: rename da/d1c/d67/l7c to da/d25/d51/d10a/l129 0 2024-03-22T10:42:59.964 INFO:tasks.workunit.client.1.smithi150.stdout:7/620: creat d2/d5/d7/d8/fcc x:0 0 0 2024-03-22T10:42:59.965 INFO:tasks.workunit.client.1.smithi150.stdout:2/701: truncate d12/d19/db6/da9/d67/d3c/f92 3631800 0 2024-03-22T10:42:59.968 INFO:tasks.workunit.client.1.smithi150.stdout:1/563: fdatasync d1/d19/d50/f90 0 2024-03-22T10:42:59.969 INFO:tasks.workunit.client.0.smithi115.stdout:2/980: truncate da/d1c/d67/d78/fb9 26342 0 2024-03-22T10:42:59.973 INFO:tasks.workunit.client.0.smithi115.stdout:2/981: getdents da/d1c/d67/df8 0 2024-03-22T10:42:59.977 INFO:tasks.workunit.client.0.smithi115.stdout:7/965: truncate d2/d7/d12/d10/d19/d42/d46/fb5 17538 0 2024-03-22T10:42:59.980 INFO:tasks.workunit.client.0.smithi115.stdout:2/982: rmdir da/d25/d3d/d7e 39 2024-03-22T10:42:59.981 INFO:tasks.workunit.client.0.smithi115.stdout:7/966: dread - d2/d7/d12/d10/d19/f13b zero size 2024-03-22T10:42:59.982 INFO:tasks.workunit.client.1.smithi150.stdout:4/619: dread d0/d1b/f24 [4194304,4194304] 0 2024-03-22T10:42:59.986 INFO:tasks.workunit.client.1.smithi150.stdout:7/621: rmdir d2/d5/d7/d8/d57 39 2024-03-22T10:42:59.988 INFO:tasks.workunit.client.1.smithi150.stdout:2/702: dread - d12/d19/db6/da9/d67/d3c/fce zero size 2024-03-22T10:42:59.990 INFO:tasks.workunit.client.1.smithi150.stdout:6/579: dwrite db/dd/f14 [0,4194304] 0 2024-03-22T10:42:59.992 INFO:tasks.workunit.client.1.smithi150.stdout:4/620: write d0/d1b/fac [311551,108299] 0 2024-03-22T10:42:59.993 INFO:tasks.workunit.client.1.smithi150.stdout:8/639: dwrite d0/d2/d19/d29/d37/f9c [0,4194304] 0 2024-03-22T10:42:59.994 INFO:tasks.workunit.client.1.smithi150.stdout:1/564: write d1/da/d2a/f7a [1124060,72274] 0 2024-03-22T10:42:59.994 INFO:tasks.workunit.client.1.smithi150.stdout:9/635: dwrite d1/d8/de/df/d24/d4e/f77 [0,4194304] 0 2024-03-22T10:42:59.995 INFO:tasks.workunit.client.1.smithi150.stdout:4/621: chown d0/c5c 5709485 1 2024-03-22T10:42:59.999 INFO:tasks.workunit.client.1.smithi150.stdout:1/565: write d1/d19/d50/d6e/f88 [447165,109931] 0 2024-03-22T10:43:00.000 INFO:tasks.workunit.client.0.smithi115.stdout:6/928: dwrite d4/d36/d48/f67 [4194304,4194304] 0 2024-03-22T10:43:00.001 INFO:tasks.workunit.client.0.smithi115.stdout:9/986: dwrite d6/d8/d3c/fd6 [0,4194304] 0 2024-03-22T10:43:00.004 INFO:tasks.workunit.client.0.smithi115.stdout:7/967: write d2/f5 [4004294,21898] 0 2024-03-22T10:43:00.006 INFO:tasks.workunit.client.0.smithi115.stdout:6/929: readlink d4/d11/d91/le5 0 2024-03-22T10:43:00.008 INFO:tasks.workunit.client.0.smithi115.stdout:9/987: truncate d6/d7/d20/d22/d3f/f153 255076 0 2024-03-22T10:43:00.010 INFO:tasks.workunit.client.1.smithi150.stdout:6/580: write db/dd/d25/f3d [1677662,111100] 0 2024-03-22T10:43:00.012 INFO:tasks.workunit.client.1.smithi150.stdout:9/636: fsync d1/d8/f65 0 2024-03-22T10:43:00.012 INFO:tasks.workunit.client.1.smithi150.stdout:1/566: symlink d1/d79/lb3 0 2024-03-22T10:43:00.017 INFO:tasks.workunit.client.1.smithi150.stdout:6/581: chown db/dd/d2e/fa6 3 1 2024-03-22T10:43:00.017 INFO:tasks.workunit.client.1.smithi150.stdout:5/533: sync 2024-03-22T10:43:00.018 INFO:tasks.workunit.client.1.smithi150.stdout:3/574: sync 2024-03-22T10:43:00.019 INFO:tasks.workunit.client.1.smithi150.stdout:8/640: write d0/f5b [1288644,93188] 0 2024-03-22T10:43:00.021 INFO:tasks.workunit.client.0.smithi115.stdout:7/968: read - d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f132 zero size 2024-03-22T10:43:00.023 INFO:tasks.workunit.client.1.smithi150.stdout:7/622: creat d2/fcd x:0 0 0 2024-03-22T10:43:00.024 INFO:tasks.workunit.client.1.smithi150.stdout:8/641: readlink d0/d3c/d71/l5e 0 2024-03-22T10:43:00.024 INFO:tasks.workunit.client.0.smithi115.stdout:0/998: dread d3/d7/d1c/f77 [0,4194304] 0 2024-03-22T10:43:00.026 INFO:tasks.workunit.client.0.smithi115.stdout:9/988: truncate d6/d8/d3c/d59/d68/f7f 1022325 0 2024-03-22T10:43:00.027 INFO:tasks.workunit.client.1.smithi150.stdout:2/703: dwrite d12/d19/db6/da9/d67/d8d/da6/fdb [0,4194304] 0 2024-03-22T10:43:00.028 INFO:tasks.workunit.client.1.smithi150.stdout:7/623: write d2/d5/d7/d9/db/d9f/d96/f8c [76260,75993] 0 2024-03-22T10:43:00.028 INFO:tasks.workunit.client.0.smithi115.stdout:6/930: write d4/d11/f23 [2247947,91231] 0 2024-03-22T10:43:00.031 INFO:tasks.workunit.client.0.smithi115.stdout:7/969: truncate d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f144 649371 0 2024-03-22T10:43:00.032 INFO:tasks.workunit.client.0.smithi115.stdout:6/931: fdatasync d4/d11/d12/f9c 0 2024-03-22T10:43:00.045 INFO:tasks.workunit.client.1.smithi150.stdout:0/615: truncate d0/d1/d84/d86/d11/f5d 284285 0 2024-03-22T10:43:00.047 INFO:tasks.workunit.client.0.smithi115.stdout:6/932: link d4/fc1 d4/d11/dc7/f130 0 2024-03-22T10:43:00.050 INFO:tasks.workunit.client.0.smithi115.stdout:9/989: dwrite d6/d7/d48/d70/fc0 [0,4194304] 0 2024-03-22T10:43:00.053 INFO:tasks.workunit.client.0.smithi115.stdout:0/999: truncate d3/d7/d1c/d6c/d9a/ffb 1798255 0 2024-03-22T10:43:00.055 INFO:tasks.workunit.client.1.smithi150.stdout:4/622: truncate d0/d1b/fac 119587 0 2024-03-22T10:43:00.056 INFO:tasks.workunit.client.1.smithi150.stdout:3/575: dwrite d5/fd [0,4194304] 0 2024-03-22T10:43:00.058 INFO:tasks.workunit.client.1.smithi150.stdout:8/642: truncate d0/d2/d19/d29/d2f/df/d1b/f75 293542 0 2024-03-22T10:43:00.059 INFO:tasks.workunit.client.1.smithi150.stdout:5/534: dwrite d0/f6 [0,4194304] 0 2024-03-22T10:43:00.069 INFO:tasks.workunit.client.1.smithi150.stdout:8/643: write d0/d3c/d71/d7a/d89/db5/fc7 [813924,92404] 0 2024-03-22T10:43:00.071 INFO:tasks.workunit.client.1.smithi150.stdout:7/624: dwrite d2/d5/d7/d8/f5e [0,4194304] 0 2024-03-22T10:43:00.072 INFO:tasks.workunit.client.0.smithi115.stdout:9/990: dwrite d6/d7/d48/faf [4194304,4194304] 0 2024-03-22T10:43:00.074 INFO:tasks.workunit.client.1.smithi150.stdout:6/582: rename db/dd/d15/d28/d6c/ccf to db/cd3 0 2024-03-22T10:43:00.076 INFO:tasks.workunit.client.1.smithi150.stdout:4/623: write d0/d1b/d4d/d6c/d61/f48 [2598257,79354] 0 2024-03-22T10:43:00.080 INFO:tasks.workunit.client.1.smithi150.stdout:7/625: read - d2/d5/d7/d8/fcc zero size 2024-03-22T10:43:00.081 INFO:tasks.workunit.client.1.smithi150.stdout:9/637: getdents d1/d49 0 2024-03-22T10:43:00.083 INFO:tasks.workunit.client.1.smithi150.stdout:2/704: dwrite d12/d19/db6/da9/d67/db4/fc6 [0,4194304] 0 2024-03-22T10:43:00.084 INFO:tasks.workunit.client.0.smithi115.stdout:6/933: dwrite d4/d11/d12/f80 [0,4194304] 0 2024-03-22T10:43:00.085 INFO:tasks.workunit.client.1.smithi150.stdout:4/624: write d0/d1b/d4d/d6c/f9d [815924,22980] 0 2024-03-22T10:43:00.089 INFO:tasks.workunit.client.0.smithi115.stdout:7/970: dread d2/f60 [0,4194304] 0 2024-03-22T10:43:00.090 INFO:tasks.workunit.client.1.smithi150.stdout:5/535: rename d0/d33/d6f/c8d to d0/d8/dc/de/d14/d4f/d47/c9f 0 2024-03-22T10:43:00.091 INFO:tasks.workunit.client.0.smithi115.stdout:7/971: readlink d2/d7/l123 0 2024-03-22T10:43:00.091 INFO:tasks.workunit.client.1.smithi150.stdout:2/705: mkdir d12/d21/ddc 0 2024-03-22T10:43:00.098 INFO:tasks.workunit.client.1.smithi150.stdout:8/644: creat d0/d2/d19/d29/d37/dc6/fd4 x:0 0 0 2024-03-22T10:43:00.099 INFO:tasks.workunit.client.0.smithi115.stdout:7/972: fdatasync d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/f111 0 2024-03-22T10:43:00.101 INFO:tasks.workunit.client.1.smithi150.stdout:3/576: rename d5/d22/d2f/d38/d3e/d72/f83 to d5/da/d8d/fbb 0 2024-03-22T10:43:00.103 INFO:tasks.workunit.client.0.smithi115.stdout:7/973: readlink d2/d7/d12/d10/d23/d3d/dd2/l63 0 2024-03-22T10:43:00.105 INFO:tasks.workunit.client.1.smithi150.stdout:8/645: write d0/d2/d19/d29/d2f/df/f26 [3842614,69379] 0 2024-03-22T10:43:00.108 INFO:tasks.workunit.client.1.smithi150.stdout:5/536: unlink d0/d37/l8a 0 2024-03-22T10:43:00.108 INFO:tasks.workunit.client.0.smithi115.stdout:6/934: dwrite d4/d36/d48/d72/daa/fac [0,4194304] 0 2024-03-22T10:43:00.113 INFO:tasks.workunit.client.1.smithi150.stdout:0/616: link d0/d1/d84/d86/d11/d66/ca2 d0/d1/cc6 0 2024-03-22T10:43:00.114 INFO:tasks.workunit.client.0.smithi115.stdout:6/935: truncate d4/d11/f65 907934 0 2024-03-22T10:43:00.116 INFO:tasks.workunit.client.0.smithi115.stdout:7/974: dread d2/d7/d12/d10/d19/d42/fc4 [0,4194304] 0 2024-03-22T10:43:00.119 INFO:tasks.workunit.client.1.smithi150.stdout:7/626: mknod d2/d5/d7/d8/d57/d54/d91/daf/cce 0 2024-03-22T10:43:00.119 INFO:tasks.workunit.client.1.smithi150.stdout:0/617: chown d0/d1/d84/d86/d42/d10/fab 22 1 2024-03-22T10:43:00.123 INFO:tasks.workunit.client.0.smithi115.stdout:2/983: sync 2024-03-22T10:43:00.123 INFO:tasks.workunit.client.0.smithi115.stdout:6/936: symlink d4/d11/d12/d1c/d8a/d98/d9f/dc3/l131 0 2024-03-22T10:43:00.127 INFO:tasks.workunit.client.0.smithi115.stdout:2/984: symlink da/d1c/d67/d78/l12a 0 2024-03-22T10:43:00.128 INFO:tasks.workunit.client.1.smithi150.stdout:3/577: symlink d5/da/d78/d86/lbc 0 2024-03-22T10:43:00.128 INFO:tasks.workunit.client.1.smithi150.stdout:9/638: truncate d1/d8/de/df/d36/f43 1141830 0 2024-03-22T10:43:00.130 INFO:tasks.workunit.client.1.smithi150.stdout:2/706: dwrite d12/d19/d2d/fcf [0,4194304] 0 2024-03-22T10:43:00.132 INFO:tasks.workunit.client.1.smithi150.stdout:8/646: mknod d0/d3c/d71/cd5 0 2024-03-22T10:43:00.133 INFO:tasks.workunit.client.1.smithi150.stdout:2/707: chown d12/d19/db6/da9/d2c 1572 1 2024-03-22T10:43:00.134 INFO:tasks.workunit.client.0.smithi115.stdout:7/975: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f14a x:0 0 0 2024-03-22T10:43:00.137 INFO:tasks.workunit.client.1.smithi150.stdout:6/583: link db/dd/d15/d60/d50/dc1/fbe db/dd/fd4 0 2024-03-22T10:43:00.139 INFO:tasks.workunit.client.0.smithi115.stdout:7/976: creat d2/d7/d12/d10/d19/d42/d76/d74/f14b x:0 0 0 2024-03-22T10:43:00.140 INFO:tasks.workunit.client.1.smithi150.stdout:6/584: write db/dd/d25/f3b [1661501,104385] 0 2024-03-22T10:43:00.140 INFO:tasks.workunit.client.1.smithi150.stdout:5/537: rename d0/d8/dc/d1d/d53/f7d to d0/d8/dc/de/d14/d4f/d47/fa0 0 2024-03-22T10:43:00.142 INFO:tasks.workunit.client.1.smithi150.stdout:1/567: dread d1/fa6 [0,4194304] 0 2024-03-22T10:43:00.144 INFO:tasks.workunit.client.0.smithi115.stdout:7/977: symlink d2/d7/l14c 0 2024-03-22T10:43:00.146 INFO:tasks.workunit.client.1.smithi150.stdout:6/585: dread - db/dd/d15/d30/f7c zero size 2024-03-22T10:43:00.146 INFO:tasks.workunit.client.0.smithi115.stdout:2/985: dwrite da/d25/d3d/d57/d7d/da1/dba/ddf/ffa [0,4194304] 0 2024-03-22T10:43:00.147 INFO:tasks.workunit.client.0.smithi115.stdout:7/978: chown d2/d7/d12/d10/d23/d3d/dd2/d143 2010 1 2024-03-22T10:43:00.148 INFO:tasks.workunit.client.0.smithi115.stdout:2/986: stat da/d25/f37 0 2024-03-22T10:43:00.158 INFO:tasks.workunit.client.1.smithi150.stdout:5/538: symlink d0/d33/la1 0 2024-03-22T10:43:00.158 INFO:tasks.workunit.client.1.smithi150.stdout:7/627: unlink d2/d5/c27 0 2024-03-22T10:43:00.158 INFO:tasks.workunit.client.1.smithi150.stdout:5/539: fdatasync d0/d8/dc/d64/f7b 0 2024-03-22T10:43:00.159 INFO:tasks.workunit.client.1.smithi150.stdout:9/639: mkdir d1/d19/d7d/dc1/dca/dd7 0 2024-03-22T10:43:00.160 INFO:tasks.workunit.client.1.smithi150.stdout:5/540: stat d0/c20 0 2024-03-22T10:43:00.163 INFO:tasks.workunit.client.1.smithi150.stdout:7/628: chown d2/d5/d7/d8/d57/d4a/d8d 22220 1 2024-03-22T10:43:00.165 INFO:tasks.workunit.client.1.smithi150.stdout:6/586: rename db/dd/d15/d60/d50/dc1/fbe to db/dd/d15/d30/fd5 0 2024-03-22T10:43:00.167 INFO:tasks.workunit.client.1.smithi150.stdout:5/541: write d0/d8/dc/de/d14/d2e/d45/f78 [511086,65934] 0 2024-03-22T10:43:00.167 INFO:tasks.workunit.client.1.smithi150.stdout:7/629: fsync d2/d5/d7/d8/d46/d49/f68 0 2024-03-22T10:43:00.169 INFO:tasks.workunit.client.1.smithi150.stdout:0/618: rename d0/f63 to d0/fc7 0 2024-03-22T10:43:00.170 INFO:tasks.workunit.client.0.smithi115.stdout:9/991: dwrite d6/d7/d19/f31 [0,4194304] 0 2024-03-22T10:43:00.174 INFO:tasks.workunit.client.0.smithi115.stdout:7/979: truncate d2/d7/d12/d10/d23/d3d/dd2/ffd 3750588 0 2024-03-22T10:43:00.176 INFO:tasks.workunit.client.1.smithi150.stdout:2/708: dread d12/d19/db6/da9/d61/d98/fbb [0,4194304] 0 2024-03-22T10:43:00.176 INFO:tasks.workunit.client.1.smithi150.stdout:1/568: creat d1/d19/d50/d51/da0/fb4 x:0 0 0 2024-03-22T10:43:00.176 INFO:tasks.workunit.client.1.smithi150.stdout:6/587: read db/dd/d25/fa5 [2723277,92933] 0 2024-03-22T10:43:00.178 INFO:tasks.workunit.client.1.smithi150.stdout:9/640: fdatasync d1/d8/de/df/d36/d40/d41/fa9 0 2024-03-22T10:43:00.181 INFO:tasks.workunit.client.0.smithi115.stdout:7/980: stat d2/d7/d12/c2b 0 2024-03-22T10:43:00.184 INFO:tasks.workunit.client.0.smithi115.stdout:6/937: dwrite d4/d11/d12/d1c/d8a/d98/d9f/ff6 [0,4194304] 0 2024-03-22T10:43:00.190 INFO:tasks.workunit.client.0.smithi115.stdout:9/992: dread d6/d7/d48/faf [4194304,4194304] 0 2024-03-22T10:43:00.193 INFO:tasks.workunit.client.1.smithi150.stdout:1/569: unlink d1/f70 0 2024-03-22T10:43:00.196 INFO:tasks.workunit.client.1.smithi150.stdout:5/542: read d0/d8/dc/de/d14/d4f/d47/f77 [407256,72326] 0 2024-03-22T10:43:00.197 INFO:tasks.workunit.client.0.smithi115.stdout:6/938: creat d4/d11/d12/d1c/d8a/d98/d9f/dc3/d10b/f132 x:0 0 0 2024-03-22T10:43:00.200 INFO:tasks.workunit.client.1.smithi150.stdout:9/641: symlink d1/d8/de/ld8 0 2024-03-22T10:43:00.200 INFO:tasks.workunit.client.0.smithi115.stdout:7/981: rename d2/d7/dc6/f107 to d2/d7/d12/d10/d23/d3d/dd2/da3/de8/f14d 0 2024-03-22T10:43:00.204 INFO:tasks.workunit.client.0.smithi115.stdout:6/939: rename d4/d11/d99/lcc to d4/d11/d121/l133 0 2024-03-22T10:43:00.205 INFO:tasks.workunit.client.0.smithi115.stdout:9/993: write d6/d7/d20/d22/d3f/fc8 [371940,101401] 0 2024-03-22T10:43:00.207 INFO:tasks.workunit.client.1.smithi150.stdout:7/630: dwrite d2/d5/d7/d38/f86 [0,4194304] 0 2024-03-22T10:43:00.207 INFO:tasks.workunit.client.1.smithi150.stdout:8/647: getdents d0/d3c/d71 0 2024-03-22T10:43:00.209 INFO:tasks.workunit.client.0.smithi115.stdout:6/940: mkdir d4/d11/d91/dee/d134 0 2024-03-22T10:43:00.210 INFO:tasks.workunit.client.0.smithi115.stdout:9/994: truncate d6/d7/d20/f137 333229 0 2024-03-22T10:43:00.212 INFO:tasks.workunit.client.1.smithi150.stdout:4/625: dwrite d0/de/f81 [4194304,4194304] 0 2024-03-22T10:43:00.217 INFO:tasks.workunit.client.1.smithi150.stdout:6/588: creat db/dd/d2e/db4/fd6 x:0 0 0 2024-03-22T10:43:00.219 INFO:tasks.workunit.client.0.smithi115.stdout:9/995: creat d6/d7/d19/d4e/f157 x:0 0 0 2024-03-22T10:43:00.220 INFO:tasks.workunit.client.1.smithi150.stdout:5/543: creat d0/d8/dc/de/d14/d2e/d45/fa2 x:0 0 0 2024-03-22T10:43:00.220 INFO:tasks.workunit.client.1.smithi150.stdout:1/570: creat d1/d7c/fb5 x:0 0 0 2024-03-22T10:43:00.222 INFO:tasks.workunit.client.1.smithi150.stdout:3/578: write d5/d53/d7e/d95/fa1 [2208539,45089] 0 2024-03-22T10:43:00.222 INFO:tasks.workunit.client.0.smithi115.stdout:2/987: dread da/d25/d51/f56 [0,4194304] 0 2024-03-22T10:43:00.223 INFO:tasks.workunit.client.1.smithi150.stdout:8/648: unlink d0/c80 0 2024-03-22T10:43:00.225 INFO:tasks.workunit.client.0.smithi115.stdout:9/996: truncate d6/d7/d6a/d6f/f14f 793055 0 2024-03-22T10:43:00.229 INFO:tasks.workunit.client.1.smithi150.stdout:6/589: creat db/dd/d15/d30/fd7 x:0 0 0 2024-03-22T10:43:00.232 INFO:tasks.workunit.client.0.smithi115.stdout:7/982: dwrite d2/d7/d12/d10/d23/d3d/dd2/da3/de8/fd5 [0,4194304] 0 2024-03-22T10:43:00.232 INFO:tasks.workunit.client.1.smithi150.stdout:9/642: truncate d1/d8/de/df/f18 780835 0 2024-03-22T10:43:00.234 INFO:tasks.workunit.client.1.smithi150.stdout:9/643: chown d1/d8/de/df 15 1 2024-03-22T10:43:00.235 INFO:tasks.workunit.client.1.smithi150.stdout:5/544: mknod d0/d33/ca3 0 2024-03-22T10:43:00.237 INFO:tasks.workunit.client.1.smithi150.stdout:7/631: dwrite d2/d5/d7/d8/d46/d49/f68 [0,4194304] 0 2024-03-22T10:43:00.237 INFO:tasks.workunit.client.1.smithi150.stdout:4/626: dwrite d0/d32/f56 [0,4194304] 0 2024-03-22T10:43:00.237 INFO:tasks.workunit.client.1.smithi150.stdout:1/571: mkdir d1/d19/d50/d6e/db6 0 2024-03-22T10:43:00.238 INFO:tasks.workunit.client.0.smithi115.stdout:9/997: creat d6/d7/d6a/d6f/f158 x:0 0 0 2024-03-22T10:43:00.238 INFO:tasks.workunit.client.0.smithi115.stdout:2/988: rename da/d25/d3d/d57/d7d/da1/ddc/l114 to da/d25/d3d/d57/da7/l12b 0 2024-03-22T10:43:00.239 INFO:tasks.workunit.client.0.smithi115.stdout:7/983: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/f14e x:0 0 0 2024-03-22T10:43:00.243 INFO:tasks.workunit.client.0.smithi115.stdout:2/989: chown da/d25/d3d/d7e/d90/dbf/cca 469695570 1 2024-03-22T10:43:00.243 INFO:tasks.workunit.client.1.smithi150.stdout:3/579: creat d5/d22/d2f/d38/d3e/d5b/da6/fbd x:0 0 0 2024-03-22T10:43:00.247 INFO:tasks.workunit.client.1.smithi150.stdout:3/580: fdatasync d5/f31 0 2024-03-22T10:43:00.247 INFO:tasks.workunit.client.0.smithi115.stdout:2/990: write da/d25/d83/f8a [1673673,44644] 0 2024-03-22T10:43:00.248 INFO:tasks.workunit.client.1.smithi150.stdout:2/709: write d12/d19/db6/d54/f5b [157021,116376] 0 2024-03-22T10:43:00.253 INFO:tasks.workunit.client.1.smithi150.stdout:5/545: unlink d0/d8/dc/de/d14/d4f/d47/f77 0 2024-03-22T10:43:00.253 INFO:tasks.workunit.client.1.smithi150.stdout:4/627: mknod d0/d2f/cc8 0 2024-03-22T10:43:00.254 INFO:tasks.workunit.client.1.smithi150.stdout:6/590: write db/dd/f1b [2791379,34496] 0 2024-03-22T10:43:00.256 INFO:tasks.workunit.client.0.smithi115.stdout:2/991: creat da/db1/f12c x:0 0 0 2024-03-22T10:43:00.259 INFO:tasks.workunit.client.1.smithi150.stdout:0/619: truncate d0/d1/d84/d86/d11/d66/d35/d5f/fa8 1045003 0 2024-03-22T10:43:00.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:00 smithi115 ceph-mon[29111]: pgmap v18: 65 pgs: 65 active+clean; 2.7 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 79 MiB/s rd, 224 MiB/s wr, 447 op/s 2024-03-22T10:43:00.262 INFO:tasks.workunit.client.0.smithi115.stdout:2/992: creat da/d1c/d67/d78/d86/db4/f12d x:0 0 0 2024-03-22T10:43:00.262 INFO:tasks.workunit.client.1.smithi150.stdout:1/572: creat d1/da/d2a/d65/d78/fb7 x:0 0 0 2024-03-22T10:43:00.262 INFO:tasks.workunit.client.1.smithi150.stdout:8/649: dwrite d0/d2/d7e/d96/f99 [0,4194304] 0 2024-03-22T10:43:00.264 INFO:tasks.workunit.client.0.smithi115.stdout:2/993: getdents da/d1c/d67/d78 0 2024-03-22T10:43:00.266 INFO:tasks.workunit.client.1.smithi150.stdout:6/591: chown db/dd/d2e/fd2 226977 1 2024-03-22T10:43:00.266 INFO:tasks.workunit.client.0.smithi115.stdout:2/994: write da/d1c/f48 [4268403,85249] 0 2024-03-22T10:43:00.268 INFO:tasks.workunit.client.0.smithi115.stdout:2/995: stat da/d25/d3d/d7e/d90/d96/f124 0 2024-03-22T10:43:00.270 INFO:tasks.workunit.client.1.smithi150.stdout:2/710: mknod d12/d19/db6/da9/d97/d78/cdd 0 2024-03-22T10:43:00.270 INFO:tasks.workunit.client.1.smithi150.stdout:9/644: chown d1/d8/de/df/f18 212 1 2024-03-22T10:43:00.271 INFO:tasks.workunit.client.1.smithi150.stdout:0/620: dread - d0/d1/d84/d86/d11/d66/d35/d5f/fc0 zero size 2024-03-22T10:43:00.273 INFO:tasks.workunit.client.0.smithi115.stdout:2/996: mkdir da/d25/d3d/d57/d7d/dc6/d12e 0 2024-03-22T10:43:00.273 INFO:tasks.workunit.client.1.smithi150.stdout:1/573: mkdir d1/da/d2a/d55/db8 0 2024-03-22T10:43:00.278 INFO:tasks.workunit.client.1.smithi150.stdout:6/592: dread - db/dd/d15/d6a/d8d/fc4 zero size 2024-03-22T10:43:00.278 INFO:tasks.workunit.client.1.smithi150.stdout:0/621: fdatasync d0/d1/fbb 0 2024-03-22T10:43:00.279 INFO:tasks.workunit.client.1.smithi150.stdout:4/628: mknod d0/d32/d7c/dc1/cc9 0 2024-03-22T10:43:00.280 INFO:tasks.workunit.client.0.smithi115.stdout:2/997: creat da/d25/d3d/d57/d7d/f12f x:0 0 0 2024-03-22T10:43:00.280 INFO:tasks.workunit.client.0.smithi115.stdout:9/998: write d6/d7/d48/d70/da9/dbd/ff7 [856089,7988] 0 2024-03-22T10:43:00.281 INFO:tasks.workunit.client.0.smithi115.stdout:7/984: write d2/d7/d12/d10/d19/d42/d46/f52 [409451,41198] 0 2024-03-22T10:43:00.281 INFO:tasks.workunit.client.1.smithi150.stdout:8/650: fdatasync d0/d2/d19/d29/d37/fb3 0 2024-03-22T10:43:00.282 INFO:tasks.workunit.client.0.smithi115.stdout:2/998: readlink da/d25/d30/l77 0 2024-03-22T10:43:00.283 INFO:tasks.workunit.client.1.smithi150.stdout:2/711: read d12/d19/db6/da9/d67/db4/fc6 [3406565,50250] 0 2024-03-22T10:43:00.284 INFO:tasks.workunit.client.1.smithi150.stdout:5/546: rename d0/d8/c98 to d0/d8/dc/de/d96/ca4 0 2024-03-22T10:43:00.284 INFO:tasks.workunit.client.0.smithi115.stdout:7/985: creat d2/d7/dc6/f14f x:0 0 0 2024-03-22T10:43:00.286 INFO:tasks.workunit.client.1.smithi150.stdout:7/632: dwrite d2/d5/d7/fa4 [0,4194304] 0 2024-03-22T10:43:00.286 INFO:tasks.workunit.client.0.smithi115.stdout:2/999: symlink da/d1c/d67/d78/d86/d9d/l130 0 2024-03-22T10:43:00.286 INFO:tasks.workunit.client.1.smithi150.stdout:1/574: chown d1/d79/d96/faf 390318 1 2024-03-22T10:43:00.287 INFO:tasks.workunit.client.0.smithi115.stdout:7/986: creat d2/d7/dc6/f150 x:0 0 0 2024-03-22T10:43:00.289 INFO:tasks.workunit.client.1.smithi150.stdout:7/633: chown d2/d5/d7/d9/db/d6c/da8 8122632 1 2024-03-22T10:43:00.290 INFO:tasks.workunit.client.0.smithi115.stdout:7/987: chown d2/d7/d12/d10/d19/d42/d46/f68 220773302 1 2024-03-22T10:43:00.292 INFO:tasks.workunit.client.1.smithi150.stdout:0/622: fdatasync d0/d1/d84/d86/d11/f7e 0 2024-03-22T10:43:00.293 INFO:tasks.workunit.client.0.smithi115.stdout:7/988: read d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/f144 [590412,87993] 0 2024-03-22T10:43:00.296 INFO:tasks.workunit.client.1.smithi150.stdout:3/581: dwrite d5/fd [0,4194304] 0 2024-03-22T10:43:00.296 INFO:tasks.workunit.client.0.smithi115.stdout:9/999: dwrite d6/d7/d6a/d6f/f158 [0,4194304] 0 2024-03-22T10:43:00.297 INFO:tasks.workunit.client.1.smithi150.stdout:6/593: write db/dd/d15/d60/d50/dc1/fba [202123,97699] 0 2024-03-22T10:43:00.297 INFO:tasks.workunit.client.1.smithi150.stdout:3/582: write d5/da/f70 [4217372,9030] 0 2024-03-22T10:43:00.299 INFO:tasks.workunit.client.1.smithi150.stdout:9/645: truncate d1/d8/de/df/d20/fb8 217295 0 2024-03-22T10:43:00.300 INFO:tasks.workunit.client.1.smithi150.stdout:3/583: chown d5/da/f57 295 1 2024-03-22T10:43:00.300 INFO:tasks.workunit.client.0.smithi115.stdout:7/989: creat d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/d12d/d140/f151 x:0 0 0 2024-03-22T10:43:00.301 INFO:tasks.workunit.client.1.smithi150.stdout:8/651: read - d0/d2/d19/d29/d88/f91 zero size 2024-03-22T10:43:00.303 INFO:tasks.workunit.client.0.smithi115.stdout:7/990: dread - d2/d7/d12/d10/d19/d42/d46/ffc zero size 2024-03-22T10:43:00.306 INFO:tasks.workunit.client.1.smithi150.stdout:5/547: mkdir d0/d8/dc/de/d96/da5 0 2024-03-22T10:43:00.307 INFO:tasks.workunit.client.1.smithi150.stdout:3/584: read - d5/d22/d2f/d38/d3e/d5b/f9a zero size 2024-03-22T10:43:00.311 INFO:tasks.workunit.client.0.smithi115.stdout:7/991: dread - d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/fe2 zero size 2024-03-22T10:43:00.314 INFO:tasks.workunit.client.1.smithi150.stdout:7/634: creat d2/d5/d7/d8/d57/d54/d91/dad/fcf x:0 0 0 2024-03-22T10:43:00.316 INFO:tasks.workunit.client.1.smithi150.stdout:0/623: chown d0/d1/d84/d86/d42/f95 4180917 1 2024-03-22T10:43:00.318 INFO:tasks.workunit.client.1.smithi150.stdout:4/629: dwrite d0/d1b/d68/f6a [4194304,4194304] 0 2024-03-22T10:43:00.319 INFO:tasks.workunit.client.1.smithi150.stdout:6/594: dread db/dd/d15/d60/f86 [0,4194304] 0 2024-03-22T10:43:00.319 INFO:tasks.workunit.client.1.smithi150.stdout:0/624: chown d0/d1/d84/d86/d42/d10/d53/c75 6615 1 2024-03-22T10:43:00.321 INFO:tasks.workunit.client.1.smithi150.stdout:8/652: creat d0/d2/d19/d29/d37/fd6 x:0 0 0 2024-03-22T10:43:00.323 INFO:tasks.workunit.client.1.smithi150.stdout:0/625: dread - d0/d1/d84/d86/f8d zero size 2024-03-22T10:43:00.324 INFO:tasks.workunit.client.1.smithi150.stdout:5/548: rename d0/d33/c57 to d0/d33/d6f/ca6 0 2024-03-22T10:43:00.327 INFO:tasks.workunit.client.0.smithi115.stdout:6/941: sync 2024-03-22T10:43:00.332 INFO:tasks.workunit.client.0.smithi115.stdout:6/942: write d4/d11/d12/d1c/f2d [68040,77342] 0 2024-03-22T10:43:00.332 INFO:tasks.workunit.client.1.smithi150.stdout:6/595: mknod db/dd/d82/d84/cd8 0 2024-03-22T10:43:00.332 INFO:tasks.workunit.client.1.smithi150.stdout:2/712: link d12/d21/lac d12/d19/db6/da9/d67/d3c/d75/lde 0 2024-03-22T10:43:00.336 INFO:tasks.workunit.client.1.smithi150.stdout:8/653: read d0/d3c/d71/f59 [441305,124435] 0 2024-03-22T10:43:00.336 INFO:tasks.workunit.client.1.smithi150.stdout:2/713: stat d12/d21/d81/lbf 0 2024-03-22T10:43:00.346 INFO:tasks.workunit.client.1.smithi150.stdout:3/585: dwrite d5/f3c [0,4194304] 0 2024-03-22T10:43:00.350 INFO:tasks.workunit.client.0.smithi115.stdout:7/992: symlink d2/d7/d12/d10/d23/d3d/da4/dc0/l152 0 2024-03-22T10:43:00.361 INFO:tasks.workunit.client.1.smithi150.stdout:0/626: rename d0/d1/d84/d86/d5a/d90 to d0/d1/d84/d86/da6/dc8 0 2024-03-22T10:43:00.363 INFO:tasks.workunit.client.1.smithi150.stdout:5/549: fdatasync d0/f2b 0 2024-03-22T10:43:00.364 INFO:tasks.workunit.client.1.smithi150.stdout:2/714: creat d12/d21/d81/fdf x:0 0 0 2024-03-22T10:43:00.367 INFO:tasks.workunit.client.1.smithi150.stdout:3/586: creat d5/d22/d2f/d38/d59/d88/fbe x:0 0 0 2024-03-22T10:43:00.368 INFO:tasks.workunit.client.1.smithi150.stdout:7/635: creat d2/d5/d7/fd0 x:0 0 0 2024-03-22T10:43:00.368 INFO:tasks.workunit.client.1.smithi150.stdout:5/550: dread - d0/d8/dc/de/d14/d2e/d5a/f86 zero size 2024-03-22T10:43:00.371 INFO:tasks.workunit.client.1.smithi150.stdout:8/654: rename d0/d2/d19/d29/d37/d7d/d9e/fb0 to d0/d2/d19/d29/d37/d7d/dc1/fd7 0 2024-03-22T10:43:00.372 INFO:tasks.workunit.client.0.smithi115.stdout:6/943: truncate d4/d36/d48/d72/daa/db9/fd7 783072 0 2024-03-22T10:43:00.373 INFO:tasks.workunit.client.1.smithi150.stdout:3/587: truncate d5/f35 4626500 0 2024-03-22T10:43:00.374 INFO:tasks.workunit.client.1.smithi150.stdout:0/627: mknod d0/d1/d84/d86/d42/d10/d38/d52/d9b/dac/cc9 0 2024-03-22T10:43:00.376 INFO:tasks.workunit.client.0.smithi115.stdout:6/944: mkdir d4/d13/d33/dbe/d135 0 2024-03-22T10:43:00.377 INFO:tasks.workunit.client.1.smithi150.stdout:6/596: getdents db/dd 0 2024-03-22T10:43:00.378 INFO:tasks.workunit.client.1.smithi150.stdout:3/588: mknod d5/d22/d2f/d38/d3e/d5b/da6/cbf 0 2024-03-22T10:43:00.379 INFO:tasks.workunit.client.1.smithi150.stdout:8/655: mkdir d0/dd8 0 2024-03-22T10:43:00.380 INFO:tasks.workunit.client.0.smithi115.stdout:6/945: mkdir d4/d13/d33/d103/d11f/d136 0 2024-03-22T10:43:00.380 INFO:tasks.workunit.client.0.smithi115.stdout:7/993: dread d2/d7/d12/d10/d23/d3d/dd2/da3/df3/f100 [4194304,4194304] 0 2024-03-22T10:43:00.382 INFO:tasks.workunit.client.1.smithi150.stdout:0/628: write d0/d1/d84/d86/d42/f82 [4551334,22990] 0 2024-03-22T10:43:00.382 INFO:tasks.workunit.client.0.smithi115.stdout:6/946: dread - d4/d11/d12/d1c/d8a/d98/d9f/dc3/f109 zero size 2024-03-22T10:43:00.382 INFO:tasks.workunit.client.1.smithi150.stdout:6/597: creat db/dd/d15/d6a/fd9 x:0 0 0 2024-03-22T10:43:00.383 INFO:tasks.workunit.client.1.smithi150.stdout:8/656: read d0/d2/f67 [2577723,39006] 0 2024-03-22T10:43:00.385 INFO:tasks.workunit.client.0.smithi115.stdout:7/994: mknod d2/c153 0 2024-03-22T10:43:00.386 INFO:tasks.workunit.client.1.smithi150.stdout:2/715: dwrite d12/d21/d81/fdf [0,4194304] 0 2024-03-22T10:43:00.386 INFO:tasks.workunit.client.1.smithi150.stdout:5/551: rename d0/d8/dc/lf to d0/d8/dc/de/la7 0 2024-03-22T10:43:00.391 INFO:tasks.workunit.client.1.smithi150.stdout:8/657: write d0/d3c/d71/d7a/d89/fb9 [434864,58055] 0 2024-03-22T10:43:00.395 INFO:tasks.workunit.client.0.smithi115.stdout:6/947: rename d4/d11/d12/d1c/d8a/d98/d9f/dc3/f109 to d4/d11/d12/d1c/d8a/d98/d9f/dc3/d10b/f137 0 2024-03-22T10:43:00.397 INFO:tasks.workunit.client.1.smithi150.stdout:2/716: mkdir d12/da1/dd6/de0 0 2024-03-22T10:43:00.397 INFO:tasks.workunit.client.1.smithi150.stdout:0/629: read d0/f89 [100848,119399] 0 2024-03-22T10:43:00.397 INFO:tasks.workunit.client.1.smithi150.stdout:5/552: mknod d0/d8/dc/d64/ca8 0 2024-03-22T10:43:00.398 INFO:tasks.workunit.client.0.smithi115.stdout:6/948: fsync d4/d36/d48/d72/f81 0 2024-03-22T10:43:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:00 smithi150 ceph-mon[34518]: pgmap v18: 65 pgs: 65 active+clean; 2.7 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 79 MiB/s rd, 224 MiB/s wr, 447 op/s 2024-03-22T10:43:00.400 INFO:tasks.workunit.client.0.smithi115.stdout:6/949: readlink d4/d11/l4f 0 2024-03-22T10:43:00.402 INFO:tasks.workunit.client.1.smithi150.stdout:4/630: write d0/de/f70 [631581,44970] 0 2024-03-22T10:43:00.404 INFO:tasks.workunit.client.0.smithi115.stdout:6/950: creat d4/d36/d48/f138 x:0 0 0 2024-03-22T10:43:00.411 INFO:tasks.workunit.client.0.smithi115.stdout:6/951: mkdir d4/d11/d12/d1c/d84/d139 0 2024-03-22T10:43:00.414 INFO:tasks.workunit.client.1.smithi150.stdout:4/631: symlink d0/d1b/d4d/d9f/dc6/lca 0 2024-03-22T10:43:00.414 INFO:tasks.workunit.client.0.smithi115.stdout:6/952: fdatasync d4/d11/d12/d1c/d8a/f118 0 2024-03-22T10:43:00.415 INFO:tasks.workunit.client.1.smithi150.stdout:4/632: chown d0/d32/fba 22865 1 2024-03-22T10:43:00.416 INFO:tasks.workunit.client.0.smithi115.stdout:7/995: dwrite d2/d7/d12/d10/d23/d3d/dd2/da3/de8/d78/dbf/dae/f111 [0,4194304] 0 2024-03-22T10:43:00.416 INFO:tasks.workunit.client.1.smithi150.stdout:0/630: rename d0/c4d to d0/d1/d84/d86/d11/d66/d37/cca 0 2024-03-22T10:43:00.420 INFO:tasks.workunit.client.1.smithi150.stdout:2/717: write d12/d19/db6/da9/d67/d3c/f77 [1134177,31371] 0 2024-03-22T10:43:00.422 INFO:tasks.workunit.client.1.smithi150.stdout:4/633: mknod d0/d32/d7c/ccb 0 2024-03-22T10:43:00.423 INFO:tasks.workunit.client.1.smithi150.stdout:8/658: truncate d0/d2/f3f 7932257 0 2024-03-22T10:43:00.427 INFO:tasks.workunit.client.1.smithi150.stdout:6/598: dread db/f44 [0,4194304] 0 2024-03-22T10:43:00.431 INFO:tasks.workunit.client.0.smithi115.stdout:7/996: dwrite d2/d7/d12/d10/d19/d42/d76/f98 [0,4194304] 0 2024-03-22T10:43:00.431 INFO:tasks.workunit.client.1.smithi150.stdout:3/589: dwrite d5/d22/d2f/f68 [4194304,4194304] 0 2024-03-22T10:43:00.435 INFO:tasks.workunit.client.1.smithi150.stdout:4/634: creat d0/d32/d7c/dc1/fcc x:0 0 0 2024-03-22T10:43:00.435 INFO:tasks.workunit.client.0.smithi115.stdout:7/997: mknod d2/d62/d126/c154 0 2024-03-22T10:43:00.435 INFO:tasks.workunit.client.1.smithi150.stdout:6/599: rmdir db/dd/d15/d28/d6c/d9b 39 2024-03-22T10:43:00.445 INFO:tasks.workunit.client.1.smithi150.stdout:2/718: getdents d12/da1 0 2024-03-22T10:43:00.448 INFO:tasks.workunit.client.1.smithi150.stdout:6/600: link db/c85 db/dd/cda 0 2024-03-22T10:43:00.453 INFO:tasks.workunit.client.1.smithi150.stdout:0/631: creat d0/d1/d84/d86/d42/d10/d38/d52/d9b/dc2/fcb x:0 0 0 2024-03-22T10:43:00.454 INFO:tasks.workunit.client.1.smithi150.stdout:5/553: dread d0/d8/dc/de/d14/f25 [0,4194304] 0 2024-03-22T10:43:00.454 INFO:tasks.workunit.client.1.smithi150.stdout:2/719: unlink d12/d19/db6/da9/d67/d8d/c94 0 2024-03-22T10:43:00.456 INFO:tasks.workunit.client.1.smithi150.stdout:5/554: readlink d0/d8/dc/de/l4a 0 2024-03-22T10:43:00.456 INFO:tasks.workunit.client.1.smithi150.stdout:2/720: chown d12/d21/d81/c84 45818076 1 2024-03-22T10:43:00.456 INFO:tasks.workunit.client.1.smithi150.stdout:8/659: dwrite d0/d2/f83 [0,4194304] 0 2024-03-22T10:43:00.458 INFO:tasks.workunit.client.1.smithi150.stdout:0/632: creat d0/d1/d84/d86/fcc x:0 0 0 2024-03-22T10:43:00.462 INFO:tasks.workunit.client.1.smithi150.stdout:8/660: dread - d0/d3c/d71/d7a/f94 zero size 2024-03-22T10:43:00.463 INFO:tasks.workunit.client.1.smithi150.stdout:0/633: read d0/fc7 [855761,41820] 0 2024-03-22T10:43:00.469 INFO:tasks.workunit.client.1.smithi150.stdout:5/555: mkdir d0/d8/dc/de/d14/d2e/d5a/d66/da9 0 2024-03-22T10:43:00.470 INFO:tasks.workunit.client.1.smithi150.stdout:8/661: mkdir d0/d5d/dd9 0 2024-03-22T10:43:00.473 INFO:tasks.workunit.client.1.smithi150.stdout:3/590: truncate d5/fb 1069915 0 2024-03-22T10:43:00.475 INFO:tasks.workunit.client.1.smithi150.stdout:4/635: truncate d0/d47/faa 3201899 0 2024-03-22T10:43:00.476 INFO:tasks.workunit.client.0.smithi115.stdout:6/953: dread d4/d11/d12/d1c/f46 [0,4194304] 0 2024-03-22T10:43:00.477 INFO:tasks.workunit.client.1.smithi150.stdout:4/636: stat d0/d1b/d68/fc4 0 2024-03-22T10:43:00.479 INFO:tasks.workunit.client.1.smithi150.stdout:4/637: readlink d0/d32/d7c/l89 0 2024-03-22T10:43:00.480 INFO:tasks.workunit.client.1.smithi150.stdout:5/556: mknod d0/d8/dc/de/d96/caa 0 2024-03-22T10:43:00.480 INFO:tasks.workunit.client.0.smithi115.stdout:6/954: truncate d4/d13/f25 772027 0 2024-03-22T10:43:00.481 INFO:tasks.workunit.client.1.smithi150.stdout:8/662: mkdir d0/d2/d19/dc0/dda 0 2024-03-22T10:43:00.481 INFO:tasks.workunit.client.1.smithi150.stdout:6/601: write db/dd/d2e/fbc [691305,49412] 0 2024-03-22T10:43:00.482 INFO:tasks.workunit.client.0.smithi115.stdout:6/955: write d4/d11/d12/d1c/d8a/d98/d9f/dc3/ff1 [944752,125057] 0 2024-03-22T10:43:00.485 INFO:tasks.workunit.client.1.smithi150.stdout:0/634: getdents d0/d1/d84/d86/d11/d66/d35/d5f/da0 0 2024-03-22T10:43:00.485 INFO:tasks.workunit.client.0.smithi115.stdout:6/956: chown d4/d13/d33/d58/cde 103083 1 2024-03-22T10:43:00.485 INFO:tasks.workunit.client.1.smithi150.stdout:4/638: mknod d0/ccd 0 2024-03-22T10:43:00.486 INFO:tasks.workunit.client.1.smithi150.stdout:6/602: dread - db/dd/d2e/fa1 zero size 2024-03-22T10:43:00.489 INFO:tasks.workunit.client.0.smithi115.stdout:6/957: creat d4/d11/d91/dee/d134/f13a x:0 0 0 2024-03-22T10:43:00.489 INFO:tasks.workunit.client.1.smithi150.stdout:8/663: rename d0 to d0/d2/d19/d29/d2e/d70/ddb 22 2024-03-22T10:43:00.490 INFO:tasks.workunit.client.1.smithi150.stdout:5/557: getdents d0/d33 0 2024-03-22T10:43:00.490 INFO:tasks.workunit.client.1.smithi150.stdout:2/721: dwrite d12/d35/d7d/fbe [0,4194304] 0 2024-03-22T10:43:00.492 INFO:tasks.workunit.client.1.smithi150.stdout:4/639: mkdir d0/dce 0 2024-03-22T10:43:00.492 INFO:tasks.workunit.client.1.smithi150.stdout:3/591: dread d5/fd [0,4194304] 0 2024-03-22T10:43:00.494 INFO:tasks.workunit.client.1.smithi150.stdout:6/603: truncate db/dd/d2e/fc7 787606 0 2024-03-22T10:43:00.495 INFO:tasks.workunit.client.0.smithi115.stdout:6/958: rename d4/d11/d12/f4e to d4/d11/d12/d1c/d8a/f13b 0 2024-03-22T10:43:00.496 INFO:tasks.workunit.client.1.smithi150.stdout:0/635: unlink d0/d1/d84/d86/d42/d10/fab 0 2024-03-22T10:43:00.497 INFO:tasks.workunit.client.0.smithi115.stdout:6/959: chown d4/d13/d33/d103/d11f 556414541 1 2024-03-22T10:43:00.499 INFO:tasks.workunit.client.1.smithi150.stdout:8/664: mknod d0/d2/d46/cdc 0 2024-03-22T10:43:00.509 INFO:tasks.workunit.client.1.smithi150.stdout:5/558: mknod d0/d8/dc/de/d14/d2e/d46/d4d/cab 0 2024-03-22T10:43:00.509 INFO:tasks.workunit.client.1.smithi150.stdout:5/559: readlink d0/d8/dc/de/l4a 0 2024-03-22T10:43:00.511 INFO:tasks.workunit.client.1.smithi150.stdout:2/722: mknod d12/d19/db6/da9/d67/d3c/d75/ce1 0 2024-03-22T10:43:00.515 INFO:tasks.workunit.client.0.smithi115.stdout:6/960: dwrite d4/d13/d33/f42 [0,4194304] 0 2024-03-22T10:43:00.517 INFO:tasks.workunit.client.1.smithi150.stdout:0/636: unlink d0/f45 0 2024-03-22T10:43:00.519 INFO:tasks.workunit.client.1.smithi150.stdout:8/665: mkdir d0/ddd 0 2024-03-22T10:43:00.519 INFO:tasks.workunit.client.1.smithi150.stdout:2/723: readlink d12/d19/db6/da9/l3e 0 2024-03-22T10:43:00.520 INFO:tasks.workunit.client.0.smithi115.stdout:6/961: creat d4/d11/d12/d1c/d8a/d98/d9f/df0/d129/f13c x:0 0 0 2024-03-22T10:43:00.521 INFO:tasks.workunit.client.1.smithi150.stdout:4/640: dwrite d0/d1b/d4d/f92 [0,4194304] 0 2024-03-22T10:43:00.523 INFO:tasks.workunit.client.0.smithi115.stdout:6/962: mknod d4/d13/d33/d103/c13d 0 2024-03-22T10:43:00.526 INFO:tasks.workunit.client.0.smithi115.stdout:6/963: read d4/d11/d12/d1c/f6a [1586043,35003] 0 2024-03-22T10:43:00.529 INFO:tasks.workunit.client.1.smithi150.stdout:8/666: fdatasync d0/d2/d19/f47 0 2024-03-22T10:43:00.531 INFO:tasks.workunit.client.1.smithi150.stdout:8/667: fsync d0/d2/d19/d29/d2f/f7b 0 2024-03-22T10:43:00.531 INFO:tasks.workunit.client.0.smithi115.stdout:6/964: symlink d4/d11/dc7/l13e 0 2024-03-22T10:43:00.533 INFO:tasks.workunit.client.1.smithi150.stdout:6/604: creat db/dd/d15/d60/fdb x:0 0 0 2024-03-22T10:43:00.534 INFO:tasks.workunit.client.1.smithi150.stdout:4/641: mknod d0/d47/ccf 0 2024-03-22T10:43:00.536 INFO:tasks.workunit.client.1.smithi150.stdout:5/560: dwrite d0/d8/dc/de/d14/d2e/d46/d4d/f80 [0,4194304] 0 2024-03-22T10:43:00.536 INFO:tasks.workunit.client.1.smithi150.stdout:4/642: readlink d0/l15 0 2024-03-22T10:43:00.536 INFO:tasks.workunit.client.1.smithi150.stdout:3/592: dwrite d5/d53/d7e/f99 [0,4194304] 0 2024-03-22T10:43:00.541 INFO:tasks.workunit.client.1.smithi150.stdout:4/643: read d0/d1b/d4d/f92 [2525156,124022] 0 2024-03-22T10:43:00.542 INFO:tasks.workunit.client.1.smithi150.stdout:5/561: chown d0/d8/dc/de/d14/f58 57847397 1 2024-03-22T10:43:00.544 INFO:tasks.workunit.client.1.smithi150.stdout:8/668: unlink d0/d2/d19/d29/d2f/ca5 0 2024-03-22T10:43:00.548 INFO:tasks.workunit.client.1.smithi150.stdout:6/605: symlink db/ldc 0 2024-03-22T10:43:00.550 INFO:tasks.workunit.client.1.smithi150.stdout:2/724: dwrite d12/d19/db6/da9/d61/d98/fd4 [0,4194304] 0 2024-03-22T10:43:00.553 INFO:tasks.workunit.client.1.smithi150.stdout:4/644: creat d0/d1b/d4d/d6c/d61/d71/fd0 x:0 0 0 2024-03-22T10:43:00.558 INFO:tasks.workunit.client.1.smithi150.stdout:5/562: truncate d0/d8/dc/f40 2746877 0 2024-03-22T10:43:00.560 INFO:tasks.workunit.client.1.smithi150.stdout:3/593: mknod d5/d22/d2f/d38/cc0 0 2024-03-22T10:43:00.566 INFO:tasks.workunit.client.1.smithi150.stdout:3/594: read - d5/d22/d2f/d38/d3e/d5b/da6/fab zero size 2024-03-22T10:43:00.566 INFO:tasks.workunit.client.1.smithi150.stdout:6/606: chown db/dd/f5b 915 1 2024-03-22T10:43:00.567 INFO:tasks.workunit.client.1.smithi150.stdout:5/563: mkdir d0/d8/dc/de/d14/d2e/d45/dac 0 2024-03-22T10:43:00.568 INFO:tasks.workunit.client.1.smithi150.stdout:3/595: write d5/f31 [5136594,83038] 0 2024-03-22T10:43:00.570 INFO:tasks.workunit.client.1.smithi150.stdout:4/645: chown d0/d2f/d63/d7b/l97 0 1 2024-03-22T10:43:00.572 INFO:tasks.workunit.client.1.smithi150.stdout:9/646: sync 2024-03-22T10:43:00.573 INFO:tasks.workunit.client.1.smithi150.stdout:0/637: getdents d0/d1/d84/d86/d11/d26 0 2024-03-22T10:43:00.579 INFO:tasks.workunit.client.1.smithi150.stdout:5/564: mkdir d0/d8/dc/de/d14/d2e/d46/d4d/dad 0 2024-03-22T10:43:00.580 INFO:tasks.workunit.client.1.smithi150.stdout:4/646: mkdir d0/d1b/d4d/d6c/d61/dd1 0 2024-03-22T10:43:00.583 INFO:tasks.workunit.client.1.smithi150.stdout:4/647: write d0/d1b/f39 [6643986,79647] 0 2024-03-22T10:43:00.584 INFO:tasks.workunit.client.1.smithi150.stdout:3/596: getdents d5/da/d78/d86/d81 0 2024-03-22T10:43:00.584 INFO:tasks.workunit.client.1.smithi150.stdout:9/647: creat d1/d8/de/df/d36/d40/d44/db5/fd9 x:0 0 0 2024-03-22T10:43:00.585 INFO:tasks.workunit.client.1.smithi150.stdout:4/648: readlink d0/l6d 0 2024-03-22T10:43:00.587 INFO:tasks.workunit.client.1.smithi150.stdout:3/597: rmdir d5/d22/d37 39 2024-03-22T10:43:00.588 INFO:tasks.workunit.client.1.smithi150.stdout:4/649: readlink d0/d2f/d63/d7b/l83 0 2024-03-22T10:43:00.589 INFO:tasks.workunit.client.1.smithi150.stdout:9/648: mkdir d1/d8/de/df/d36/d40/d44/db5/dda 0 2024-03-22T10:43:00.589 INFO:tasks.workunit.client.0.smithi115.stdout:6/965: dread d4/d11/d12/d1c/f7f [0,4194304] 0 2024-03-22T10:43:00.591 INFO:tasks.workunit.client.1.smithi150.stdout:3/598: chown d5/fb 6963761 1 2024-03-22T10:43:00.593 INFO:tasks.workunit.client.1.smithi150.stdout:9/649: unlink d1/d8/de/l66 0 2024-03-22T10:43:00.595 INFO:tasks.workunit.client.1.smithi150.stdout:0/638: link d0/d1/d84/d86/fe d0/d12/fcd 0 2024-03-22T10:43:00.595 INFO:tasks.workunit.client.0.smithi115.stdout:7/998: sync 2024-03-22T10:43:00.600 INFO:tasks.workunit.client.1.smithi150.stdout:5/565: dwrite d0/d8/dc/de/f15 [4194304,4194304] 0 2024-03-22T10:43:00.602 INFO:tasks.workunit.client.1.smithi150.stdout:8/669: write d0/d2/d19/d29/d2f/df/f42 [1811117,112422] 0 2024-03-22T10:43:00.604 INFO:tasks.workunit.client.0.smithi115.stdout:7/999: mkdir d2/d7/d12/d10/d19/d42/d155 0 2024-03-22T10:43:00.608 INFO:tasks.workunit.client.1.smithi150.stdout:2/725: truncate d12/d19/f59 2924373 0 2024-03-22T10:43:00.612 INFO:tasks.workunit.client.1.smithi150.stdout:8/670: rmdir d0/d2/d46 39 2024-03-22T10:43:00.612 INFO:tasks.workunit.client.1.smithi150.stdout:3/599: dwrite d5/f3c [0,4194304] 0 2024-03-22T10:43:00.615 INFO:tasks.workunit.client.0.smithi115.stdout:6/966: dwrite d4/d36/d48/d72/fa9 [0,4194304] 0 2024-03-22T10:43:00.616 INFO:tasks.workunit.client.1.smithi150.stdout:0/639: rename d0/d1/d84/d86/d42/d10/c6e to d0/d1/d84/d86/d11/d66/cce 0 2024-03-22T10:43:00.618 INFO:tasks.workunit.client.1.smithi150.stdout:2/726: symlink d12/d21/d81/le2 0 2024-03-22T10:43:00.621 INFO:tasks.workunit.client.1.smithi150.stdout:5/566: mkdir d0/dae 0 2024-03-22T10:43:00.622 INFO:tasks.workunit.client.1.smithi150.stdout:2/727: dread d12/d19/db6/da9/d61/f8c [0,4194304] 0 2024-03-22T10:43:00.628 INFO:tasks.workunit.client.1.smithi150.stdout:4/650: truncate d0/d1b/f39 1654342 0 2024-03-22T10:43:00.631 INFO:tasks.workunit.client.1.smithi150.stdout:3/600: mkdir d5/da/d9d/d9f/dc1 0 2024-03-22T10:43:00.634 INFO:tasks.workunit.client.1.smithi150.stdout:3/601: write d5/da/d62/f8e [779805,52333] 0 2024-03-22T10:43:00.635 INFO:tasks.workunit.client.1.smithi150.stdout:5/567: creat d0/d8/dc/de/d14/d2e/d45/dac/faf x:0 0 0 2024-03-22T10:43:00.637 INFO:tasks.workunit.client.1.smithi150.stdout:5/568: chown d0/d8/dc/l71 327 1 2024-03-22T10:43:00.691 INFO:tasks.workunit.client.1.smithi150.stdout:9/650: dwrite d1/d8/de/df/f18 [0,4194304] 0 2024-03-22T10:43:00.691 INFO:tasks.workunit.client.1.smithi150.stdout:0/640: write d0/d1/f39 [40392,31681] 0 2024-03-22T10:43:00.691 INFO:tasks.workunit.client.1.smithi150.stdout:2/728: truncate f11 3275223 0 2024-03-22T10:43:00.697 INFO:tasks.workunit.client.1.smithi150.stdout:5/569: write d0/d8/dc/de/d14/d4f/d47/fa0 [689473,107806] 0 2024-03-22T10:43:00.697 INFO:tasks.workunit.client.1.smithi150.stdout:2/729: rename d12 to d12/d19/d2d/de3 22 2024-03-22T10:43:00.699 INFO:tasks.workunit.client.1.smithi150.stdout:2/730: readlink d12/d19/db6/d54/l76 0 2024-03-22T10:43:00.700 INFO:tasks.workunit.client.1.smithi150.stdout:9/651: creat d1/d8/de/df/d36/d40/d44/db5/fdb x:0 0 0 2024-03-22T10:43:00.702 INFO:tasks.workunit.client.1.smithi150.stdout:5/570: symlink d0/dae/lb0 0 2024-03-22T10:43:00.703 INFO:tasks.workunit.client.1.smithi150.stdout:8/671: dwrite d0/d2/f12 [0,4194304] 0 2024-03-22T10:43:00.705 INFO:tasks.workunit.client.1.smithi150.stdout:0/641: rename d0/d1/d84/d86/d11/f5d to d0/d1/d84/fcf 0 2024-03-22T10:43:00.707 INFO:tasks.workunit.client.1.smithi150.stdout:2/731: mknod d12/d19/d2d/d66/ce4 0 2024-03-22T10:43:00.709 INFO:tasks.workunit.client.1.smithi150.stdout:5/571: symlink d0/lb1 0 2024-03-22T10:43:00.714 INFO:tasks.workunit.client.1.smithi150.stdout:3/602: dwrite d5/d22/d37/f6c [0,4194304] 0 2024-03-22T10:43:00.714 INFO:tasks.workunit.client.0.smithi115.stdout:6/967: sync 2024-03-22T10:43:00.718 INFO:tasks.workunit.client.0.smithi115.stdout:6/968: mknod d4/d11/d12/d1c/d8a/d98/d9f/df0/d129/c13f 0 2024-03-22T10:43:00.719 INFO:tasks.workunit.client.1.smithi150.stdout:5/572: fsync d0/d8/dc/de/d14/d2e/d46/d4d/f51 0 2024-03-22T10:43:00.719 INFO:tasks.workunit.client.1.smithi150.stdout:2/732: rename d12/d19/d2d/f82 to d12/d19/db6/d54/fe5 0 2024-03-22T10:43:00.720 INFO:tasks.workunit.client.1.smithi150.stdout:2/733: stat d12/d19/db6/da9/d2c 0 2024-03-22T10:43:00.721 INFO:tasks.workunit.client.1.smithi150.stdout:0/642: truncate d0/d1/d84/d86/fe 2132868 0 2024-03-22T10:43:00.723 INFO:tasks.workunit.client.1.smithi150.stdout:5/573: write d0/d8/dc/de/d14/d2e/d5a/f86 [922928,32541] 0 2024-03-22T10:43:00.728 INFO:tasks.workunit.client.1.smithi150.stdout:2/734: dread - d12/d19/db6/da9/d67/d3c/fce zero size 2024-03-22T10:43:00.731 INFO:tasks.workunit.client.1.smithi150.stdout:2/735: read - d12/d19/db6/d63/fa8 zero size 2024-03-22T10:43:00.733 INFO:tasks.workunit.client.1.smithi150.stdout:9/652: dwrite d1/d8/de/df/d20/d32/d75/f8d [0,4194304] 0 2024-03-22T10:43:00.737 INFO:tasks.workunit.client.1.smithi150.stdout:8/672: dread d0/d5d/dd2/f4b [0,4194304] 0 2024-03-22T10:43:00.738 INFO:tasks.workunit.client.1.smithi150.stdout:9/653: chown d1/d8/de/df/d36/d40/f78 302507537 1 2024-03-22T10:43:00.739 INFO:tasks.workunit.client.1.smithi150.stdout:9/654: chown d1/d8/de/df/d24 2524136 1 2024-03-22T10:43:00.740 INFO:tasks.workunit.client.1.smithi150.stdout:5/574: rename d0/d8/dc/d1d/d53/d31 to d0/dae/db2 0 2024-03-22T10:43:00.744 INFO:tasks.workunit.client.1.smithi150.stdout:2/736: creat d12/d35/d7d/fe6 x:0 0 0 2024-03-22T10:43:00.745 INFO:tasks.workunit.client.1.smithi150.stdout:9/655: truncate d1/d8/de/df/d36/f5c 2683227 0 2024-03-22T10:43:00.746 INFO:tasks.workunit.client.1.smithi150.stdout:2/737: chown d12/d21/f4f 100709000 1 2024-03-22T10:43:00.748 INFO:tasks.workunit.client.1.smithi150.stdout:8/673: creat d0/d2/d19/d29/d37/d7d/d9e/fde x:0 0 0 2024-03-22T10:43:00.750 INFO:tasks.workunit.client.1.smithi150.stdout:5/575: rename d0/d8/dc/de/d14/d2e/d45/d56 to d0/d8/dc/de/d14/d2e/d45/db3 0 2024-03-22T10:43:00.752 INFO:tasks.workunit.client.1.smithi150.stdout:8/674: read d0/d2/d19/d29/d2f/df/d1b/f75 [114952,94767] 0 2024-03-22T10:43:00.753 INFO:tasks.workunit.client.1.smithi150.stdout:9/656: mkdir d1/d8/de/dbd/ddc 0 2024-03-22T10:43:00.754 INFO:tasks.workunit.client.1.smithi150.stdout:5/576: mkdir d0/d37/db4 0 2024-03-22T10:43:00.754 INFO:tasks.workunit.client.0.smithi115.stdout:6/969: dread d4/d13/f3d [0,4194304] 0 2024-03-22T10:43:00.756 INFO:tasks.workunit.client.1.smithi150.stdout:5/577: dread - d0/d8/dc/de/d14/d2e/d45/f87 zero size 2024-03-22T10:43:00.756 INFO:tasks.workunit.client.0.smithi115.stdout:6/970: fsync d4/d11/d12/d1c/f7f 0 2024-03-22T10:43:00.757 INFO:tasks.workunit.client.1.smithi150.stdout:9/657: truncate d1/d19/fb4 861481 0 2024-03-22T10:43:00.759 INFO:tasks.workunit.client.1.smithi150.stdout:9/658: creat d1/d49/d98/fdd x:0 0 0 2024-03-22T10:43:00.760 INFO:tasks.workunit.client.1.smithi150.stdout:5/578: fsync d0/d8/dc/de/d14/d2e/d46/f72 0 2024-03-22T10:43:00.762 INFO:tasks.workunit.client.0.smithi115.stdout:6/971: rename d4/lb3 to d4/d11/l140 0 2024-03-22T10:43:00.763 INFO:tasks.workunit.client.1.smithi150.stdout:5/579: write d0/d8/dc/de/d14/d2e/d5a/f8e [3783799,42060] 0 2024-03-22T10:43:00.765 INFO:tasks.workunit.client.1.smithi150.stdout:9/659: link d1/d8/de/df/d24/d4e/f77 d1/d8/fde 0 2024-03-22T10:43:00.765 INFO:tasks.workunit.client.1.smithi150.stdout:5/580: read d0/d8/dc/de/f13 [951253,35965] 0 2024-03-22T10:43:00.769 INFO:tasks.workunit.client.1.smithi150.stdout:0/643: truncate d0/d1/d84/d86/d42/f82 3224376 0 2024-03-22T10:43:00.769 INFO:tasks.workunit.client.1.smithi150.stdout:9/660: write d1/d8/de/df/d20/d32/d75/dc3/fcd [63518,4505] 0 2024-03-22T10:43:00.770 INFO:tasks.workunit.client.1.smithi150.stdout:0/644: dread - d0/d1/d84/d86/fcc zero size 2024-03-22T10:43:00.773 INFO:tasks.workunit.client.1.smithi150.stdout:8/675: write d0/d2/f86 [909663,59830] 0 2024-03-22T10:43:00.777 INFO:tasks.workunit.client.0.smithi115.stdout:6/972: dwrite d4/d11/d91/f11d [0,4194304] 0 2024-03-22T10:43:00.778 INFO:tasks.workunit.client.1.smithi150.stdout:0/645: rmdir d0/d1/d84/d86/d42/d10/d38/d52/d9b 39 2024-03-22T10:43:00.779 INFO:tasks.workunit.client.0.smithi115.stdout:6/973: write d4/d36/d48/f138 [1038476,98536] 0 2024-03-22T10:43:00.783 INFO:tasks.workunit.client.1.smithi150.stdout:0/646: creat d0/d1/d84/d86/da6/fd0 x:0 0 0 2024-03-22T10:43:00.784 INFO:tasks.workunit.client.1.smithi150.stdout:5/581: dwrite d0/f10 [0,4194304] 0 2024-03-22T10:43:00.786 INFO:tasks.workunit.client.0.smithi115.stdout:6/974: rename d4/d13/d33/f10d to d4/d11/dc7/d11c/d12f/f141 0 2024-03-22T10:43:00.787 INFO:tasks.workunit.client.0.smithi115.stdout:6/975: chown d4/d36/d48/d72/daa 38431 1 2024-03-22T10:43:00.789 INFO:tasks.workunit.client.0.smithi115.stdout:6/976: stat d4/d13/d33/lad 0 2024-03-22T10:43:00.792 INFO:tasks.workunit.client.0.smithi115.stdout:6/977: chown d4/d13/d33/cff 426782345 1 2024-03-22T10:43:00.793 INFO:tasks.workunit.client.1.smithi150.stdout:5/582: rename d0/d8/dc/de/d96/caa to d0/d37/db4/cb5 0 2024-03-22T10:43:00.798 INFO:tasks.workunit.client.1.smithi150.stdout:5/583: chown d0/d8/dc/de/d14/d2e/d45/f87 3980958 1 2024-03-22T10:43:00.798 INFO:tasks.workunit.client.1.smithi150.stdout:9/661: dread d1/d8/f65 [0,4194304] 0 2024-03-22T10:43:00.798 INFO:tasks.workunit.client.1.smithi150.stdout:0/647: creat d0/d1/d84/d86/d11/d66/d37/fd1 x:0 0 0 2024-03-22T10:43:00.799 INFO:tasks.workunit.client.1.smithi150.stdout:7/636: sync 2024-03-22T10:43:00.799 INFO:tasks.workunit.client.1.smithi150.stdout:1/575: sync 2024-03-22T10:43:00.801 INFO:tasks.workunit.client.1.smithi150.stdout:5/584: dread - d0/d8/dc/d1d/d89/f8f zero size 2024-03-22T10:43:00.801 INFO:tasks.workunit.client.1.smithi150.stdout:9/662: rmdir d1/d8/de/df/d36/d40 39 2024-03-22T10:43:00.803 INFO:tasks.workunit.client.1.smithi150.stdout:1/576: creat d1/d19/d50/d6e/fb9 x:0 0 0 2024-03-22T10:43:00.804 INFO:tasks.workunit.client.1.smithi150.stdout:0/648: mkdir d0/d1/d84/d86/d11/d26/dd2 0 2024-03-22T10:43:00.808 INFO:tasks.workunit.client.1.smithi150.stdout:9/663: creat d1/d8/de/df/d20/d32/d75/d99/fdf x:0 0 0 2024-03-22T10:43:00.809 INFO:tasks.workunit.client.1.smithi150.stdout:1/577: mkdir d1/d19/d1a/dba 0 2024-03-22T10:43:00.810 INFO:tasks.workunit.client.0.smithi115.stdout:6/978: dwrite d4/d13/f70 [8388608,4194304] 0 2024-03-22T10:43:00.811 INFO:tasks.workunit.client.1.smithi150.stdout:5/585: link d0/f10 d0/d8/dc/de/d14/fb6 0 2024-03-22T10:43:00.813 INFO:tasks.workunit.client.1.smithi150.stdout:9/664: creat d1/d8/de/df/d20/d32/d75/dc3/fe0 x:0 0 0 2024-03-22T10:43:00.814 INFO:tasks.workunit.client.0.smithi115.stdout:6/979: mknod d4/d13/d33/c142 0 2024-03-22T10:43:00.816 INFO:tasks.workunit.client.1.smithi150.stdout:1/578: rename d1/d19/d50/la7 to d1/lbb 0 2024-03-22T10:43:00.817 INFO:tasks.workunit.client.1.smithi150.stdout:9/665: symlink d1/le1 0 2024-03-22T10:43:00.822 INFO:tasks.workunit.client.1.smithi150.stdout:9/666: truncate d1/d19/d7d/fc2 103435 0 2024-03-22T10:43:00.822 INFO:tasks.workunit.client.1.smithi150.stdout:5/586: link d0/f2b d0/d8/dc/de/d14/d2e/d46/d4d/fb7 0 2024-03-22T10:43:00.823 INFO:tasks.workunit.client.0.smithi115.stdout:6/980: write d4/d11/d91/fd5 [1003922,119547] 0 2024-03-22T10:43:00.825 INFO:tasks.workunit.client.1.smithi150.stdout:1/579: getdents d1/d19/d1a/d99 0 2024-03-22T10:43:00.825 INFO:tasks.workunit.client.1.smithi150.stdout:0/649: dwrite d0/d12/faa [0,4194304] 0 2024-03-22T10:43:00.827 INFO:tasks.workunit.client.0.smithi115.stdout:6/981: rmdir d4/d11 39 2024-03-22T10:43:00.828 INFO:tasks.workunit.client.1.smithi150.stdout:5/587: mkdir d0/d8/dc/de/d96/db8 0 2024-03-22T10:43:00.829 INFO:tasks.workunit.client.0.smithi115.stdout:6/982: creat d4/d13/d33/d103/f143 x:0 0 0 2024-03-22T10:43:00.830 INFO:tasks.workunit.client.1.smithi150.stdout:5/588: chown d0/d8/dc/de/d14/d2e/d46/d4d/dad 235331695 1 2024-03-22T10:43:00.835 INFO:tasks.workunit.client.1.smithi150.stdout:9/667: rename d1/d8/de/df/d20/l13 to d1/d8/de/df/d36/d40/le2 0 2024-03-22T10:43:00.843 INFO:tasks.workunit.client.1.smithi150.stdout:5/589: rename d0/d8/dc/de/l4a to d0/d37/lb9 0 2024-03-22T10:43:00.843 INFO:tasks.workunit.client.1.smithi150.stdout:1/580: dwrite d1/f29 [4194304,4194304] 0 2024-03-22T10:43:00.848 INFO:tasks.workunit.client.1.smithi150.stdout:9/668: link d1/d8/de/d73/f74 d1/d8/de/df/d24/fe3 0 2024-03-22T10:43:00.849 INFO:tasks.workunit.client.0.smithi115.stdout:6/983: dwrite d4/d11/d12/d1c/f2d [0,4194304] 0 2024-03-22T10:43:00.857 INFO:tasks.workunit.client.1.smithi150.stdout:1/581: creat d1/d19/fbc x:0 0 0 2024-03-22T10:43:00.860 INFO:tasks.workunit.client.1.smithi150.stdout:9/669: link d1/d8/de/df/d20/d32/d75/dc3/fe0 d1/d8/de/df/d24/d4e/fe4 0 2024-03-22T10:43:00.863 INFO:tasks.workunit.client.1.smithi150.stdout:0/650: truncate d0/fc7 924961 0 2024-03-22T10:43:00.864 INFO:tasks.workunit.client.0.smithi115.stdout:6/984: truncate d4/d11/d12/d1c/d8a/d98/fef 1397020 0 2024-03-22T10:43:00.865 INFO:tasks.workunit.client.1.smithi150.stdout:9/670: unlink d1/d8/de/df/d20/d32/d75/lad 0 2024-03-22T10:43:00.867 INFO:tasks.workunit.client.1.smithi150.stdout:0/651: rmdir d0/d1/d84/d86/d11/d66/d37 39 2024-03-22T10:43:00.873 INFO:tasks.workunit.client.1.smithi150.stdout:5/590: dread d0/d8/dc/de/d14/d2e/d46/d4d/fb7 [0,4194304] 0 2024-03-22T10:43:00.877 INFO:tasks.workunit.client.1.smithi150.stdout:5/591: symlink d0/lba 0 2024-03-22T10:43:00.882 INFO:tasks.workunit.client.1.smithi150.stdout:5/592: rename d0/d8/dc/de/d14/f50 to d0/d8/dc/de/d96/fbb 0 2024-03-22T10:43:00.886 INFO:tasks.workunit.client.0.smithi115.stdout:6/985: dwrite d4/d11/d12/d1c/d8a/d98/fef [0,4194304] 0 2024-03-22T10:43:00.891 INFO:tasks.workunit.client.0.smithi115.stdout:6/986: creat d4/d13/d33/dbe/d135/f144 x:0 0 0 2024-03-22T10:43:00.895 INFO:tasks.workunit.client.1.smithi150.stdout:9/671: write d1/d8/de/df/d20/d32/f5e [656320,37790] 0 2024-03-22T10:43:00.896 INFO:tasks.workunit.client.0.smithi115.stdout:6/987: creat d4/d11/d12/d1c/d8a/f145 x:0 0 0 2024-03-22T10:43:00.898 INFO:tasks.workunit.client.1.smithi150.stdout:9/672: chown d1/l3a 53 1 2024-03-22T10:43:00.898 INFO:tasks.workunit.client.0.smithi115.stdout:6/988: write d4/d36/d48/d72/daa/fac [3411421,83936] 0 2024-03-22T10:43:00.902 INFO:tasks.workunit.client.0.smithi115.stdout:6/989: unlink d4/d36/d48/f67 0 2024-03-22T10:43:00.904 INFO:tasks.workunit.client.1.smithi150.stdout:9/673: write d1/d8/de/df/d20/d29/f9b [4351731,6517] 0 2024-03-22T10:43:00.904 INFO:tasks.workunit.client.0.smithi115.stdout:6/990: dread - d4/d11/d12/d1c/d8a/f118 zero size 2024-03-22T10:43:00.907 INFO:tasks.workunit.client.0.smithi115.stdout:6/991: dread - d4/d11/d12/d1c/d8a/d98/d9f/dc3/d10b/f137 zero size 2024-03-22T10:43:00.907 INFO:tasks.workunit.client.1.smithi150.stdout:0/652: dwrite d0/f83 [0,4194304] 0 2024-03-22T10:43:00.908 INFO:tasks.workunit.client.1.smithi150.stdout:1/582: dread d1/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:43:00.914 INFO:tasks.workunit.client.1.smithi150.stdout:0/653: dread - d0/d12/f58 zero size 2024-03-22T10:43:00.914 INFO:tasks.workunit.client.1.smithi150.stdout:9/674: write d1/d8/de/df/d36/d40/d44/db5/fd9 [362024,116316] 0 2024-03-22T10:43:00.914 INFO:tasks.workunit.client.1.smithi150.stdout:1/583: write d1/d19/d1a/d7d/f82 [1640566,108002] 0 2024-03-22T10:43:00.917 INFO:tasks.workunit.client.1.smithi150.stdout:1/584: readlink d1/d19/d50/d51/l89 0 2024-03-22T10:43:00.919 INFO:tasks.workunit.client.1.smithi150.stdout:5/593: dread d0/d8/dc/de/d14/d2e/d46/d4d/f6c [4194304,4194304] 0 2024-03-22T10:43:00.924 INFO:tasks.workunit.client.1.smithi150.stdout:9/675: symlink d1/d8/de/df/d20/d32/d75/d99/da7/le5 0 2024-03-22T10:43:00.924 INFO:tasks.workunit.client.1.smithi150.stdout:5/594: fsync d0/d8/dc/de/d14/f16 0 2024-03-22T10:43:00.934 INFO:tasks.workunit.client.1.smithi150.stdout:1/585: dread d1/d19/d50/f5c [0,4194304] 0 2024-03-22T10:43:00.935 INFO:tasks.workunit.client.0.smithi115.stdout:6/992: dread d4/fec [0,4194304] 0 2024-03-22T10:43:00.937 INFO:tasks.workunit.client.1.smithi150.stdout:5/595: dread d0/d8/dc/de/f13 [0,4194304] 0 2024-03-22T10:43:00.937 INFO:tasks.workunit.client.0.smithi115.stdout:6/993: rmdir d4/d11/d91/dee 39 2024-03-22T10:43:00.938 INFO:tasks.workunit.client.0.smithi115.stdout:6/994: chown d4/d13/f101 1058 1 2024-03-22T10:43:00.938 INFO:tasks.workunit.client.1.smithi150.stdout:1/586: mkdir d1/da/d2a/d65/dbd 0 2024-03-22T10:43:00.941 INFO:tasks.workunit.client.1.smithi150.stdout:1/587: dread - d1/da/d15/d4f/fa4 zero size 2024-03-22T10:43:00.941 INFO:tasks.workunit.client.1.smithi150.stdout:5/596: read d0/d8/dc/de/f82 [1222528,60831] 0 2024-03-22T10:43:00.941 INFO:tasks.workunit.client.1.smithi150.stdout:0/654: dread d0/d1/d84/d86/d11/d26/f2e [0,4194304] 0 2024-03-22T10:43:00.942 INFO:tasks.workunit.client.0.smithi115.stdout:6/995: creat d4/d11/d91/f146 x:0 0 0 2024-03-22T10:43:00.942 INFO:tasks.workunit.client.1.smithi150.stdout:9/676: dwrite d1/d8/de/d73/f74 [4194304,4194304] 0 2024-03-22T10:43:00.945 INFO:tasks.workunit.client.1.smithi150.stdout:9/677: fdatasync d1/d8/de/df/f18 0 2024-03-22T10:43:00.947 INFO:tasks.workunit.client.0.smithi115.stdout:6/996: creat d4/d11/d91/dee/d134/f147 x:0 0 0 2024-03-22T10:43:00.950 INFO:tasks.workunit.client.0.smithi115.stdout:6/997: creat d4/d36/d48/d72/f148 x:0 0 0 2024-03-22T10:43:00.951 INFO:tasks.workunit.client.1.smithi150.stdout:9/678: mknod d1/d49/d98/ce6 0 2024-03-22T10:43:00.953 INFO:tasks.workunit.client.1.smithi150.stdout:5/597: mkdir d0/d8/dc/de/d14/d2e/dbc 0 2024-03-22T10:43:00.954 INFO:tasks.workunit.client.1.smithi150.stdout:0/655: write d0/d1/d84/d86/d42/d10/f71 [285867,3301] 0 2024-03-22T10:43:00.956 INFO:tasks.workunit.client.1.smithi150.stdout:1/588: link d1/d19/l5a d1/d19/d50/d51/da0/lbe 0 2024-03-22T10:43:00.957 INFO:tasks.workunit.client.1.smithi150.stdout:5/598: dread d0/d8/dc/f5b [0,4194304] 0 2024-03-22T10:43:00.958 INFO:tasks.workunit.client.0.smithi115.stdout:6/998: rename d4/d36/d48/f62 to d4/d11/d12/d1c/d8a/d98/d9f/dc3/f149 0 2024-03-22T10:43:00.961 INFO:tasks.workunit.client.1.smithi150.stdout:5/599: write d0/d8/dc/de/f13 [2657244,628] 0 2024-03-22T10:43:00.961 INFO:tasks.workunit.client.1.smithi150.stdout:1/589: mknod d1/d7c/cbf 0 2024-03-22T10:43:00.964 INFO:tasks.workunit.client.1.smithi150.stdout:0/656: rename d0/d1/d84/d86/d11/d66/fb3 to d0/d1/d84/d86/da6/fd3 0 2024-03-22T10:43:00.966 INFO:tasks.workunit.client.1.smithi150.stdout:0/657: chown d0/d12/c2c 12180678 1 2024-03-22T10:43:00.971 INFO:tasks.workunit.client.0.smithi115.stdout:6/999: dwrite d4/d13/d33/d58/ffe [0,4194304] 0 2024-03-22T10:43:00.972 INFO:tasks.workunit.client.1.smithi150.stdout:0/658: write d0/d1/d84/d86/d11/f7e [3493235,124712] 0 2024-03-22T10:43:00.973 INFO:tasks.workunit.client.1.smithi150.stdout:9/679: dwrite d1/d8/de/f17 [0,4194304] 0 2024-03-22T10:43:00.974 INFO:tasks.workunit.client.1.smithi150.stdout:6/607: sync 2024-03-22T10:43:00.977 INFO:tasks.workunit.client.1.smithi150.stdout:0/659: write d0/d1/d84/d86/da6/fd0 [339380,128209] 0 2024-03-22T10:43:00.982 INFO:tasks.workunit.client.0.smithi115.stderr:+ rm -rf -- ./tmp.JwwKOLKHxY 2024-03-22T10:43:00.985 INFO:tasks.workunit.client.1.smithi150.stdout:9/680: creat d1/d8/de/df/d20/d32/d75/d99/da7/fe7 x:0 0 0 2024-03-22T10:43:00.992 INFO:tasks.workunit.client.1.smithi150.stdout:9/681: creat d1/d8/de/dbd/ddc/fe8 x:0 0 0 2024-03-22T10:43:00.993 INFO:tasks.workunit.client.1.smithi150.stdout:5/600: dwrite d0/d8/dc/de/d14/d2e/f3e [0,4194304] 0 2024-03-22T10:43:00.997 INFO:tasks.workunit.client.1.smithi150.stdout:1/590: dread d1/d19/d1a/f1e [0,4194304] 0 2024-03-22T10:43:00.999 INFO:tasks.workunit.client.1.smithi150.stdout:6/608: dwrite db/dd/d2e/f81 [0,4194304] 0 2024-03-22T10:43:01.002 INFO:tasks.workunit.client.1.smithi150.stdout:1/591: mkdir d1/da/d2a/dc0 0 2024-03-22T10:43:01.008 INFO:tasks.workunit.client.1.smithi150.stdout:1/592: rmdir d1/d7c 39 2024-03-22T10:43:01.011 INFO:tasks.workunit.client.1.smithi150.stdout:0/660: dwrite d0/d1/d84/d86/d42/d10/f43 [0,4194304] 0 2024-03-22T10:43:01.013 INFO:tasks.workunit.client.1.smithi150.stdout:1/593: write d1/da/d15/d18/f8c [513514,92716] 0 2024-03-22T10:43:01.014 INFO:tasks.workunit.client.1.smithi150.stdout:5/601: creat d0/d8/dc/de/d14/d2e/d45/dac/fbd x:0 0 0 2024-03-22T10:43:01.018 INFO:tasks.workunit.client.1.smithi150.stdout:9/682: dwrite d1/d19/d7d/fc2 [0,4194304] 0 2024-03-22T10:43:01.023 INFO:tasks.workunit.client.1.smithi150.stdout:6/609: rename db/dd/d15/d6a/lcb to db/dd/d15/d30/ldd 0 2024-03-22T10:43:01.024 INFO:tasks.workunit.client.1.smithi150.stdout:0/661: rename d0/d1/d84/d86/d11/d66/d35 to d0/d1/d84/d86/d11/d66/d35/d92/dd4 22 2024-03-22T10:43:01.025 INFO:tasks.workunit.client.1.smithi150.stdout:9/683: mkdir d1/d8/de/df/d24/d4e/de9 0 2024-03-22T10:43:01.027 INFO:tasks.workunit.client.1.smithi150.stdout:1/594: truncate d1/d19/f26 252948 0 2024-03-22T10:43:01.034 INFO:tasks.workunit.client.1.smithi150.stdout:5/602: rename d0/d8/dc/de/d14/d2e/f3e to d0/d8/dc/de/d14/d2e/d45/dac/fbe 0 2024-03-22T10:43:01.039 INFO:tasks.workunit.client.1.smithi150.stdout:9/684: truncate d1/d19/d7d/dc1/f8b 1132032 0 2024-03-22T10:43:01.042 INFO:tasks.workunit.client.1.smithi150.stdout:6/610: getdents db/dd/d15/d57 0 2024-03-22T10:43:01.046 INFO:tasks.workunit.client.1.smithi150.stdout:9/685: creat d1/d19/fea x:0 0 0 2024-03-22T10:43:01.047 INFO:tasks.workunit.client.1.smithi150.stdout:6/611: chown db/dd/d15/d6a/f6b 96710282 1 2024-03-22T10:43:01.052 INFO:tasks.workunit.client.1.smithi150.stdout:6/612: unlink db/ldc 0 2024-03-22T10:43:01.054 INFO:tasks.workunit.client.1.smithi150.stdout:5/603: dread d0/d8/dc/de/f15 [4194304,4194304] 0 2024-03-22T10:43:01.062 INFO:tasks.workunit.client.1.smithi150.stdout:9/686: rename d1/d8/de/df/d20/d32/d75/d99/laa to d1/d8/de/df/d20/d32/d75/leb 0 2024-03-22T10:43:01.065 INFO:tasks.workunit.client.1.smithi150.stdout:0/662: dread d0/f6d [0,4194304] 0 2024-03-22T10:43:01.067 INFO:tasks.workunit.client.1.smithi150.stdout:9/687: stat d1/d8/de/df/d20/d32/c9c 0 2024-03-22T10:43:01.067 INFO:tasks.workunit.client.1.smithi150.stdout:1/595: dwrite d1/d19/d50/d6e/f77 [0,4194304] 0 2024-03-22T10:43:01.069 INFO:tasks.workunit.client.1.smithi150.stdout:5/604: link d0/d8/dc/de/d14/d2e/d46/d4d/l8b d0/d37/db4/lbf 0 2024-03-22T10:43:01.073 INFO:tasks.workunit.client.1.smithi150.stdout:0/663: truncate d0/d1/d84/d86/d11/fc4 190068 0 2024-03-22T10:43:01.086 INFO:tasks.workunit.client.1.smithi150.stdout:0/664: creat d0/d1/d84/d86/d11/fd5 x:0 0 0 2024-03-22T10:43:01.089 INFO:tasks.workunit.client.1.smithi150.stdout:0/665: write d0/d1/f39 [4492167,17936] 0 2024-03-22T10:43:01.092 INFO:tasks.workunit.client.1.smithi150.stdout:1/596: dwrite d1/d19/d50/d51/f6b [0,4194304] 0 2024-03-22T10:43:01.097 INFO:tasks.workunit.client.1.smithi150.stdout:9/688: dwrite d1/d19/f9f [0,4194304] 0 2024-03-22T10:43:01.098 INFO:tasks.workunit.client.1.smithi150.stdout:0/666: write d0/d1/d84/d86/d11/d66/d35/d5f/fc0 [580437,58409] 0 2024-03-22T10:43:01.106 INFO:tasks.workunit.client.1.smithi150.stdout:1/597: creat d1/da/d15/d18/d4b/fc1 x:0 0 0 2024-03-22T10:43:01.109 INFO:tasks.workunit.client.1.smithi150.stdout:0/667: mkdir d0/d12/dd6 0 2024-03-22T10:43:01.110 INFO:tasks.workunit.client.1.smithi150.stdout:9/689: symlink d1/d8/de/dbd/ddc/lec 0 2024-03-22T10:43:01.113 INFO:tasks.workunit.client.1.smithi150.stdout:1/598: creat d1/da/d2a/d4e/fc2 x:0 0 0 2024-03-22T10:43:01.115 INFO:tasks.workunit.client.1.smithi150.stdout:1/599: stat d1/da/d2a/d65/d78 0 2024-03-22T10:43:01.116 INFO:tasks.workunit.client.1.smithi150.stdout:1/600: stat d1/d79/d96 0 2024-03-22T10:43:01.118 INFO:tasks.workunit.client.1.smithi150.stdout:0/668: mknod d0/d1/d84/d86/d42/d10/d38/d52/d9b/dc2/cd7 0 2024-03-22T10:43:01.120 INFO:tasks.workunit.client.1.smithi150.stdout:1/601: rename d1/d19/l45 to d1/da/d15/d4f/lc3 0 2024-03-22T10:43:01.124 INFO:tasks.workunit.client.1.smithi150.stdout:9/690: write d1/d8/de/df/d36/f43 [2110576,29050] 0 2024-03-22T10:43:01.124 INFO:tasks.workunit.client.1.smithi150.stdout:0/669: dread d0/d1/d84/d86/d11/d66/d35/d5f/fc0 [0,4194304] 0 2024-03-22T10:43:01.129 INFO:tasks.workunit.client.1.smithi150.stdout:0/670: getdents d0/d1/d84/d86/da6/dc8 0 2024-03-22T10:43:01.131 INFO:tasks.workunit.client.1.smithi150.stdout:9/691: rename d1/d8/de/df/d36/l60 to d1/d8/de/df/d20/led 0 2024-03-22T10:43:01.135 INFO:tasks.workunit.client.1.smithi150.stdout:9/692: creat d1/d8/d95/fee x:0 0 0 2024-03-22T10:43:01.138 INFO:tasks.workunit.client.1.smithi150.stdout:1/602: dwrite d1/da/d2a/faa [0,4194304] 0 2024-03-22T10:43:01.139 INFO:tasks.workunit.client.1.smithi150.stdout:9/693: dread - d1/d8/d95/fc7 zero size 2024-03-22T10:43:01.142 INFO:tasks.workunit.client.1.smithi150.stdout:1/603: rmdir d1/da/d15/d18/d4b 39 2024-03-22T10:43:01.146 INFO:tasks.workunit.client.1.smithi150.stdout:1/604: mkdir d1/d19/d50/d51/da0/dc4 0 2024-03-22T10:43:01.153 INFO:tasks.workunit.client.1.smithi150.stdout:2/738: sync 2024-03-22T10:43:01.153 INFO:tasks.workunit.client.1.smithi150.stdout:8/676: sync 2024-03-22T10:43:01.153 INFO:tasks.workunit.client.1.smithi150.stdout:4/651: sync 2024-03-22T10:43:01.154 INFO:tasks.workunit.client.1.smithi150.stdout:3/603: sync 2024-03-22T10:43:01.154 INFO:tasks.workunit.client.1.smithi150.stdout:7/637: sync 2024-03-22T10:43:01.156 INFO:tasks.workunit.client.1.smithi150.stdout:7/638: chown d2/d5/l87 1 1 2024-03-22T10:43:01.159 INFO:tasks.workunit.client.1.smithi150.stdout:7/639: chown d2/d5/d7/d8/d57/d4a/f7f 14281 1 2024-03-22T10:43:01.160 INFO:tasks.workunit.client.1.smithi150.stdout:8/677: chown d0/d2/d46/f52 0 1 2024-03-22T10:43:01.161 INFO:tasks.workunit.client.1.smithi150.stdout:9/694: write d1/d8/d95/fc7 [631396,21979] 0 2024-03-22T10:43:01.162 INFO:tasks.workunit.client.1.smithi150.stdout:2/739: dread d12/d19/db6/f44 [0,4194304] 0 2024-03-22T10:43:01.168 INFO:tasks.workunit.client.1.smithi150.stdout:7/640: write d2/d5/d7/d9/db/da2/fbe [584736,60657] 0 2024-03-22T10:43:01.168 INFO:tasks.workunit.client.1.smithi150.stdout:9/695: chown d1/d8/de/df/d24/d4e/f77 32 1 2024-03-22T10:43:01.172 INFO:tasks.workunit.client.1.smithi150.stdout:0/671: dread d0/d1/d84/d86/d11/f7e [0,4194304] 0 2024-03-22T10:43:01.173 INFO:tasks.workunit.client.1.smithi150.stdout:9/696: chown d1/d8/de/df/d36 103698819 1 2024-03-22T10:43:01.174 INFO:tasks.workunit.client.1.smithi150.stdout:2/740: creat d12/d19/db6/da9/d2c/fe7 x:0 0 0 2024-03-22T10:43:01.177 INFO:tasks.workunit.client.1.smithi150.stdout:1/605: dwrite d1/da/d15/d18/f38 [0,4194304] 0 2024-03-22T10:43:01.178 INFO:tasks.workunit.client.1.smithi150.stdout:2/741: chown d12/d19/db6/da9/d61 66 1 2024-03-22T10:43:01.179 INFO:tasks.workunit.client.1.smithi150.stdout:0/672: creat d0/d12/fd8 x:0 0 0 2024-03-22T10:43:01.183 INFO:tasks.workunit.client.1.smithi150.stdout:2/742: fdatasync d12/d19/d1a/f30 0 2024-03-22T10:43:01.184 INFO:tasks.workunit.client.1.smithi150.stdout:1/606: chown d1/da/d2a/d55/db8 0 1 2024-03-22T10:43:01.187 INFO:tasks.workunit.client.1.smithi150.stdout:4/652: dread d0/d1b/d4d/d6c/d61/d71/fa4 [0,4194304] 0 2024-03-22T10:43:01.189 INFO:tasks.workunit.client.1.smithi150.stdout:7/641: rename d2/d5/d7/f4d to d2/d5/d7/d9/db/d6c/d9c/fd1 0 2024-03-22T10:43:01.190 INFO:tasks.workunit.client.1.smithi150.stdout:9/697: mknod d1/d8/de/df/d20/d32/d75/dc3/cef 0 2024-03-22T10:43:01.193 INFO:tasks.workunit.client.1.smithi150.stdout:1/607: symlink d1/d79/d96/lc5 0 2024-03-22T10:43:01.194 INFO:tasks.workunit.client.1.smithi150.stdout:2/743: read d12/d19/db6/d54/fe5 [180459,122093] 0 2024-03-22T10:43:01.195 INFO:tasks.workunit.client.1.smithi150.stdout:9/698: readlink d1/d8/de/df/l3e 0 2024-03-22T10:43:01.195 INFO:tasks.workunit.client.1.smithi150.stdout:8/678: dwrite d0/d3c/f56 [0,4194304] 0 2024-03-22T10:43:01.198 INFO:tasks.workunit.client.1.smithi150.stdout:4/653: symlink d0/dce/ld2 0 2024-03-22T10:43:01.205 INFO:tasks.workunit.client.1.smithi150.stdout:4/654: write d0/de/f5e [5168173,78432] 0 2024-03-22T10:43:01.206 INFO:tasks.workunit.client.1.smithi150.stdout:0/673: rename d0/d1/d84/d86/d42/f44 to d0/d1/d84/d86/d11/d66/d35/d5f/fd9 0 2024-03-22T10:43:01.208 INFO:tasks.workunit.client.1.smithi150.stdout:1/608: unlink d1/da/d2a/d4e/l56 0 2024-03-22T10:43:01.209 INFO:tasks.workunit.client.1.smithi150.stdout:8/679: truncate d0/d5d/f6c 507544 0 2024-03-22T10:43:01.211 INFO:tasks.workunit.client.1.smithi150.stdout:2/744: unlink d12/d21/l64 0 2024-03-22T10:43:01.214 INFO:tasks.workunit.client.1.smithi150.stdout:8/680: write d0/f5b [277992,88822] 0 2024-03-22T10:43:01.216 INFO:tasks.workunit.client.1.smithi150.stdout:9/699: dwrite d1/d8/de/df/da2/fbf [0,4194304] 0 2024-03-22T10:43:01.216 INFO:tasks.workunit.client.1.smithi150.stdout:7/642: link d2/d5/d7/fd0 d2/d5/d7/d8/d46/fd2 0 2024-03-22T10:43:01.224 INFO:tasks.workunit.client.1.smithi150.stdout:2/745: creat d12/d19/db6/d63/dd8/fe8 x:0 0 0 2024-03-22T10:43:01.225 INFO:tasks.workunit.client.1.smithi150.stdout:0/674: dread d0/d1/d84/d86/d11/d26/f2e [0,4194304] 0 2024-03-22T10:43:01.234 INFO:tasks.workunit.client.1.smithi150.stdout:0/675: chown d0/d1/d84/d86/d11/d26/l30 0 1 2024-03-22T10:43:01.238 INFO:tasks.workunit.client.1.smithi150.stdout:4/655: dwrite d0/d32/d7c/f91 [4194304,4194304] 0 2024-03-22T10:43:01.241 INFO:tasks.workunit.client.1.smithi150.stdout:4/656: fsync d0/de/f7f 0 2024-03-22T10:43:01.246 INFO:tasks.workunit.client.1.smithi150.stdout:0/676: write d0/d1/d84/d86/d42/fc [485748,126012] 0 2024-03-22T10:43:01.249 INFO:tasks.workunit.client.1.smithi150.stdout:9/700: write d1/d8/de/df/d20/f5d [508908,116614] 0 2024-03-22T10:43:01.257 INFO:tasks.workunit.client.1.smithi150.stdout:2/746: dwrite d12/d21/fb3 [4194304,4194304] 0 2024-03-22T10:43:01.259 INFO:tasks.workunit.client.1.smithi150.stdout:4/657: getdents d0/de 0 2024-03-22T10:43:01.262 INFO:tasks.workunit.client.1.smithi150.stdout:7/643: dread d2/d5/d7/d8/d57/d4a/f33 [0,4194304] 0 2024-03-22T10:43:01.262 INFO:tasks.workunit.client.1.smithi150.stdout:1/609: dread d1/f7 [4194304,4194304] 0 2024-03-22T10:43:01.267 INFO:tasks.workunit.client.1.smithi150.stdout:6/613: sync 2024-03-22T10:43:01.267 INFO:tasks.workunit.client.1.smithi150.stdout:0/677: mknod d0/d1/d84/d86/d42/d10/d38/cda 0 2024-03-22T10:43:01.267 INFO:tasks.workunit.client.1.smithi150.stdout:1/610: stat d1/d19/d1a/d99 0 2024-03-22T10:43:01.267 INFO:tasks.workunit.client.1.smithi150.stdout:9/701: truncate d1/d8/de/df/d20/d32/d75/dc3/fe0 601318 0 2024-03-22T10:43:01.269 INFO:tasks.workunit.client.1.smithi150.stdout:8/681: dwrite d0/d2/f14 [0,4194304] 0 2024-03-22T10:43:01.270 INFO:tasks.workunit.client.1.smithi150.stdout:1/611: readlink d1/da/d2a/d4e/l7b 0 2024-03-22T10:43:01.273 INFO:tasks.workunit.client.1.smithi150.stdout:4/658: rename d0/d2f/fb3 to d0/d1b/d4d/d6c/d61/d71/fd3 0 2024-03-22T10:43:01.273 INFO:tasks.workunit.client.1.smithi150.stdout:1/612: read - d1/d79/d96/faf zero size 2024-03-22T10:43:01.274 INFO:tasks.workunit.client.1.smithi150.stdout:6/614: mkdir db/dd/d15/d6a/dde 0 2024-03-22T10:43:01.277 INFO:tasks.workunit.client.1.smithi150.stdout:7/644: write d2/f82 [106241,56330] 0 2024-03-22T10:43:01.281 INFO:tasks.workunit.client.1.smithi150.stdout:4/659: mkdir d0/d32/d7c/dd4 0 2024-03-22T10:43:01.283 INFO:tasks.workunit.client.1.smithi150.stdout:9/702: mkdir d1/d19/d7d/dc1/dca/dd7/df0 0 2024-03-22T10:43:01.284 INFO:tasks.workunit.client.1.smithi150.stdout:6/615: mkdir db/dd/d15/d30/ddf 0 2024-03-22T10:43:01.285 INFO:tasks.workunit.client.1.smithi150.stdout:4/660: chown d0/d32/d7c/dc1/cc9 85 1 2024-03-22T10:43:01.286 INFO:tasks.workunit.client.1.smithi150.stdout:6/616: fdatasync db/d52/fb1 0 2024-03-22T10:43:01.287 INFO:tasks.workunit.client.1.smithi150.stdout:9/703: chown d1/d8/de/l8e 90861 1 2024-03-22T10:43:01.292 INFO:tasks.workunit.client.1.smithi150.stdout:1/613: creat d1/d19/d50/d6e/fc6 x:0 0 0 2024-03-22T10:43:01.295 INFO:tasks.workunit.client.1.smithi150.stdout:7/645: creat d2/d5/d7/d38/fd3 x:0 0 0 2024-03-22T10:43:01.296 INFO:tasks.workunit.client.1.smithi150.stdout:6/617: truncate db/dd/d2e/fd2 940252 0 2024-03-22T10:43:01.301 INFO:tasks.workunit.client.1.smithi150.stdout:8/682: dwrite d0/d2/f12 [0,4194304] 0 2024-03-22T10:43:01.302 INFO:tasks.workunit.client.1.smithi150.stdout:6/618: symlink db/dd/d2e/db4/le0 0 2024-03-22T10:43:01.304 INFO:tasks.workunit.client.1.smithi150.stdout:9/704: mknod d1/d8/de/df/d36/d40/d41/cf1 0 2024-03-22T10:43:01.304 INFO:tasks.workunit.client.1.smithi150.stdout:7/646: symlink d2/d5/d7/ld4 0 2024-03-22T10:43:01.307 INFO:tasks.workunit.client.1.smithi150.stdout:0/678: dwrite d0/d1/d84/d86/d11/f7e [0,4194304] 0 2024-03-22T10:43:01.309 INFO:tasks.workunit.client.1.smithi150.stdout:6/619: creat db/d52/d9f/fe1 x:0 0 0 2024-03-22T10:43:01.309 INFO:tasks.workunit.client.1.smithi150.stdout:1/614: getdents d1/da/d15/d18 0 2024-03-22T10:43:01.318 INFO:tasks.workunit.client.1.smithi150.stdout:9/705: creat d1/d8/de/df/d24/d4e/de9/ff2 x:0 0 0 2024-03-22T10:43:01.318 INFO:tasks.workunit.client.1.smithi150.stdout:7/647: dread d2/d5/d7/d8/d57/f5b [0,4194304] 0 2024-03-22T10:43:01.320 INFO:tasks.workunit.client.1.smithi150.stdout:0/679: symlink d0/d1/d84/d86/d42/d10/d38/d52/d9b/dac/ldb 0 2024-03-22T10:43:01.323 INFO:tasks.workunit.client.1.smithi150.stdout:4/661: dwrite d0/de/f5b [0,4194304] 0 2024-03-22T10:43:01.324 INFO:tasks.workunit.client.1.smithi150.stdout:2/747: dread d12/d35/fb2 [0,4194304] 0 2024-03-22T10:43:01.329 INFO:tasks.workunit.client.1.smithi150.stdout:4/662: dread - d0/d32/fae zero size 2024-03-22T10:43:01.329 INFO:tasks.workunit.client.1.smithi150.stdout:1/615: write d1/d19/d50/f66 [1353677,4694] 0 2024-03-22T10:43:01.346 INFO:tasks.workunit.client.1.smithi150.stdout:9/706: unlink d1/d8/de/df/da2/cd5 0 2024-03-22T10:43:01.347 INFO:tasks.workunit.client.1.smithi150.stdout:2/748: creat d12/d19/db6/da9/d61/fe9 x:0 0 0 2024-03-22T10:43:01.348 INFO:tasks.workunit.client.1.smithi150.stdout:1/616: write d1/d19/d1a/f49 [3763995,16820] 0 2024-03-22T10:43:01.352 INFO:tasks.workunit.client.1.smithi150.stdout:1/617: readlink d1/d79/lb3 0 2024-03-22T10:43:01.355 INFO:tasks.workunit.client.1.smithi150.stdout:8/683: dread d0/f21 [0,4194304] 0 2024-03-22T10:43:01.355 INFO:tasks.workunit.client.1.smithi150.stdout:9/707: truncate d1/d8/de/df/d20/d32/fbc 756641 0 2024-03-22T10:43:01.356 INFO:tasks.workunit.client.1.smithi150.stdout:7/648: dwrite d2/d5/d7/fa4 [0,4194304] 0 2024-03-22T10:43:01.361 INFO:tasks.workunit.client.1.smithi150.stdout:4/663: creat d0/d1b/d4d/d6c/d61/fd5 x:0 0 0 2024-03-22T10:43:01.366 INFO:tasks.workunit.client.1.smithi150.stdout:9/708: symlink d1/d8/de/df/d24/d4e/de9/lf3 0 2024-03-22T10:43:01.370 INFO:tasks.workunit.client.1.smithi150.stdout:2/749: unlink d12/d19/db6/da9/d67/l7b 0 2024-03-22T10:43:01.371 INFO:tasks.workunit.client.1.smithi150.stdout:7/649: mknod d2/d5/d7/d9/db/d9f/d96/d71/cd5 0 2024-03-22T10:43:01.372 INFO:tasks.workunit.client.1.smithi150.stdout:0/680: getdents d0/d1/d84/d86/d11/d66/d35/d5f 0 2024-03-22T10:43:01.374 INFO:tasks.workunit.client.1.smithi150.stdout:9/709: symlink d1/d8/de/df/d20/d29/lf4 0 2024-03-22T10:43:01.377 INFO:tasks.workunit.client.1.smithi150.stdout:9/710: fsync d1/d8/de/df/d36/f39 0 2024-03-22T10:43:01.378 INFO:tasks.workunit.client.1.smithi150.stdout:1/618: getdents d1/d19/d50 0 2024-03-22T10:43:01.379 INFO:tasks.workunit.client.1.smithi150.stdout:0/681: write d0/d1/d84/d86/d42/d10/f43 [3604705,3998] 0 2024-03-22T10:43:01.379 INFO:tasks.workunit.client.1.smithi150.stdout:4/664: getdents d0/d1b/d4d/d9f/dc6 0 2024-03-22T10:43:01.381 INFO:tasks.workunit.client.1.smithi150.stdout:4/665: stat d0/d1b/d68/f6a 0 2024-03-22T10:43:01.391 INFO:tasks.workunit.client.1.smithi150.stdout:2/750: dread d12/d19/db6/da9/d67/d3c/f77 [0,4194304] 0 2024-03-22T10:43:01.391 INFO:tasks.workunit.client.1.smithi150.stdout:7/650: creat d2/d5/d7/d9/db/d6c/fd6 x:0 0 0 2024-03-22T10:43:01.394 INFO:tasks.workunit.client.1.smithi150.stdout:2/751: chown d12/d21/f4f 85 1 2024-03-22T10:43:01.399 INFO:tasks.workunit.client.1.smithi150.stdout:2/752: symlink d12/d19/db6/d54/lea 0 2024-03-22T10:43:01.399 INFO:tasks.workunit.client.1.smithi150.stdout:7/651: creat d2/d5/d7/d9/db/d9f/d96/fd7 x:0 0 0 2024-03-22T10:43:01.401 INFO:tasks.workunit.client.1.smithi150.stdout:9/711: dwrite d1/d8/de/df/fd3 [0,4194304] 0 2024-03-22T10:43:01.406 INFO:tasks.workunit.client.1.smithi150.stdout:9/712: stat d1/d8/de/df/d20/fb8 0 2024-03-22T10:43:01.410 INFO:tasks.workunit.client.1.smithi150.stdout:7/652: write d2/d5/d7/d9/db/d9f/d96/f73 [2180971,117698] 0 2024-03-22T10:43:01.413 INFO:tasks.workunit.client.1.smithi150.stdout:0/682: dwrite d0/d1/d84/d86/d42/d10/d38/f4b [0,4194304] 0 2024-03-22T10:43:01.413 INFO:tasks.workunit.client.1.smithi150.stdout:2/753: rename d12/d35/d7d/l9e to d12/d19/db6/da9/d97/d78/leb 0 2024-03-22T10:43:01.415 INFO:tasks.workunit.client.1.smithi150.stdout:5/605: sync 2024-03-22T10:43:01.415 INFO:tasks.workunit.client.1.smithi150.stdout:3/604: sync 2024-03-22T10:43:01.415 INFO:tasks.workunit.client.1.smithi150.stdout:7/653: write d2/d5/d7/fa4 [4508213,67078] 0 2024-03-22T10:43:01.419 INFO:tasks.workunit.client.1.smithi150.stdout:9/713: creat d1/d19/d7d/dc1/dca/dd7/ff5 x:0 0 0 2024-03-22T10:43:01.425 INFO:tasks.workunit.client.1.smithi150.stdout:9/714: write d1/d19/d7d/dc1/dca/dd7/ff5 [464031,100395] 0 2024-03-22T10:43:01.429 INFO:tasks.workunit.client.1.smithi150.stdout:5/606: chown d0/d8/dc/de/d14/d2e/d46/c67 218 1 2024-03-22T10:43:01.430 INFO:tasks.workunit.client.1.smithi150.stdout:3/605: truncate d5/d22/d2f/d38/d3e/f4d 810913 0 2024-03-22T10:43:01.434 INFO:tasks.workunit.client.1.smithi150.stdout:9/715: fsync d1/d8/de/df/d24/f89 0 2024-03-22T10:43:01.438 INFO:tasks.workunit.client.1.smithi150.stdout:3/606: link d5/d53/f75 d5/d53/fc2 0 2024-03-22T10:43:01.442 INFO:tasks.workunit.client.1.smithi150.stdout:3/607: write d5/d22/d2f/d38/d59/d88/fbe [228683,48043] 0 2024-03-22T10:43:01.442 INFO:tasks.workunit.client.1.smithi150.stdout:5/607: mkdir d0/d8/dc/de/d14/d2e/d46/d9c/dc0 0 2024-03-22T10:43:01.446 INFO:tasks.workunit.client.1.smithi150.stdout:9/716: read d1/d8/de/df/da2/f91 [5471703,117379] 0 2024-03-22T10:43:01.452 INFO:tasks.workunit.client.1.smithi150.stdout:3/608: symlink d5/d53/d7e/lc3 0 2024-03-22T10:43:01.456 INFO:tasks.workunit.client.1.smithi150.stdout:3/609: write d5/d22/d2f/f68 [6946963,114213] 0 2024-03-22T10:43:01.457 INFO:tasks.workunit.client.1.smithi150.stdout:9/717: symlink d1/d8/de/df/d20/d32/lf6 0 2024-03-22T10:43:01.461 INFO:tasks.workunit.client.1.smithi150.stdout:2/754: truncate d12/d35/fb2 4056761 0 2024-03-22T10:43:01.465 INFO:tasks.workunit.client.1.smithi150.stdout:5/608: dread d0/d8/dc/de/d14/d2e/d46/d4d/fb7 [0,4194304] 0 2024-03-22T10:43:01.471 INFO:tasks.workunit.client.1.smithi150.stdout:5/609: write d0/d8/dc/de/d14/d2e/d46/d4d/f80 [4612460,7064] 0 2024-03-22T10:43:01.473 INFO:tasks.workunit.client.1.smithi150.stdout:9/718: link d1/d8/de/df/d36/f43 d1/d8/de/df/ff7 0 2024-03-22T10:43:01.476 INFO:tasks.workunit.client.1.smithi150.stdout:5/610: read d0/d8/dc/de/d14/d4f/f3d [509024,44574] 0 2024-03-22T10:43:01.477 INFO:tasks.workunit.client.1.smithi150.stdout:7/654: dread d2/d5/d7/d8/d46/d49/f68 [0,4194304] 0 2024-03-22T10:43:01.479 INFO:tasks.workunit.client.1.smithi150.stdout:2/755: creat d12/d19/db6/da9/d97/d78/dbd/fec x:0 0 0 2024-03-22T10:43:01.485 INFO:tasks.workunit.client.1.smithi150.stdout:5/611: symlink d0/d33/lc1 0 2024-03-22T10:43:01.486 INFO:tasks.workunit.client.1.smithi150.stdout:0/683: dread d0/d1/f39 [0,4194304] 0 2024-03-22T10:43:01.488 INFO:tasks.workunit.client.1.smithi150.stdout:3/610: dwrite d5/d53/fc2 [0,4194304] 0 2024-03-22T10:43:01.488 INFO:tasks.workunit.client.1.smithi150.stdout:7/655: mknod d2/d5/d7/d9/db/cd8 0 2024-03-22T10:43:01.492 INFO:tasks.workunit.client.1.smithi150.stdout:2/756: chown d12/d19/db6/da9/d67/d9a 7578041 1 2024-03-22T10:43:01.493 INFO:tasks.workunit.client.1.smithi150.stdout:5/612: truncate d0/f11 2414409 0 2024-03-22T10:43:01.495 INFO:tasks.workunit.client.1.smithi150.stdout:0/684: fsync d0/d1/f70 0 2024-03-22T10:43:01.496 INFO:tasks.workunit.client.1.smithi150.stdout:7/656: mkdir d2/d5/d7/d8/d57/d4a/d8d/dd9 0 2024-03-22T10:43:01.502 INFO:tasks.workunit.client.1.smithi150.stdout:9/719: dwrite d1/d8/de/df/d36/f7c [0,4194304] 0 2024-03-22T10:43:01.502 INFO:tasks.workunit.client.1.smithi150.stdout:2/757: getdents d12/d19/db6/da9/d2c/d73 0 2024-03-22T10:43:01.504 INFO:tasks.workunit.client.1.smithi150.stdout:0/685: truncate d0/d1/d84/d86/d11/fd5 923978 0 2024-03-22T10:43:01.506 INFO:tasks.workunit.client.1.smithi150.stdout:2/758: write d12/d35/d7d/fbe [4961578,104811] 0 2024-03-22T10:43:01.506 INFO:tasks.workunit.client.1.smithi150.stdout:9/720: chown d1/d19/d7d/dc1/dca/dd7/df0 3 1 2024-03-22T10:43:01.510 INFO:tasks.workunit.client.1.smithi150.stdout:5/613: mknod d0/d8/dc/de/d96/db8/cc2 0 2024-03-22T10:43:01.515 INFO:tasks.workunit.client.1.smithi150.stdout:9/721: mkdir d1/d8/de/dbd/df8 0 2024-03-22T10:43:01.519 INFO:tasks.workunit.client.1.smithi150.stdout:2/759: rename d12/d35/d7d/fbe to d12/d19/db6/da9/d67/d3c/d75/fed 0 2024-03-22T10:43:01.521 INFO:tasks.workunit.client.1.smithi150.stdout:7/657: dread d2/d5/d7/d8/d57/d4a/f22 [4194304,4194304] 0 2024-03-22T10:43:01.524 INFO:tasks.workunit.client.1.smithi150.stdout:3/611: write d5/d53/f54 [5605971,80808] 0 2024-03-22T10:43:01.533 INFO:tasks.workunit.client.1.smithi150.stdout:7/658: creat d2/d5/d7/d9/db/d6c/d9c/fda x:0 0 0 2024-03-22T10:43:01.537 INFO:tasks.workunit.client.1.smithi150.stdout:3/612: mknod d5/d22/d2f/d38/d3e/d5b/cc4 0 2024-03-22T10:43:01.538 INFO:tasks.workunit.client.1.smithi150.stdout:9/722: link d1/d8/de/df/d20/d32/d75/f84 d1/d8/de/dbd/ddc/ff9 0 2024-03-22T10:43:01.540 INFO:tasks.workunit.client.1.smithi150.stdout:3/613: truncate d5/f52 4637988 0 2024-03-22T10:43:01.541 INFO:tasks.workunit.client.1.smithi150.stdout:7/659: creat d2/d5/d7/d88/fdb x:0 0 0 2024-03-22T10:43:01.541 INFO:tasks.workunit.client.1.smithi150.stdout:3/614: stat d5/da/f63 0 2024-03-22T10:43:01.543 INFO:tasks.workunit.client.1.smithi150.stdout:9/723: mkdir d1/d19/dfa 0 2024-03-22T10:43:01.544 INFO:tasks.workunit.client.1.smithi150.stdout:7/660: fsync d2/d5/d7/d9/db/d9f/f40 0 2024-03-22T10:43:01.547 INFO:tasks.workunit.client.1.smithi150.stdout:3/615: chown d5/d22/d2f/d38/d3e/d5b/da6/cbf 0 1 2024-03-22T10:43:01.550 INFO:tasks.workunit.client.1.smithi150.stdout:5/614: dread d0/f22 [0,4194304] 0 2024-03-22T10:43:01.554 INFO:tasks.workunit.client.1.smithi150.stdout:0/686: dwrite d0/d1/d84/d86/d11/fc4 [0,4194304] 0 2024-03-22T10:43:01.558 INFO:tasks.workunit.client.1.smithi150.stdout:2/760: dread f11 [0,4194304] 0 2024-03-22T10:43:01.563 INFO:tasks.workunit.client.1.smithi150.stdout:9/724: truncate d1/d8/de/df/d36/d40/f78 338814 0 2024-03-22T10:43:01.563 INFO:tasks.workunit.client.1.smithi150.stdout:6/620: sync 2024-03-22T10:43:01.570 INFO:tasks.workunit.client.1.smithi150.stdout:5/615: creat d0/d8/dc/de/d14/d2e/dbc/fc3 x:0 0 0 2024-03-22T10:43:01.570 INFO:tasks.workunit.client.1.smithi150.stdout:2/761: creat d12/d19/db6/da9/d97/d78/dbd/fee x:0 0 0 2024-03-22T10:43:01.571 INFO:tasks.workunit.client.1.smithi150.stdout:7/661: link d2/cc d2/d5/d7/d9/db/d9f/d96/d71/cdc 0 2024-03-22T10:43:01.571 INFO:tasks.workunit.client.1.smithi150.stdout:3/616: dwrite d5/fb [0,4194304] 0 2024-03-22T10:43:01.577 INFO:tasks.workunit.client.1.smithi150.stdout:5/616: truncate d0/d8/dc/d1d/d53/d63/f9d 266535 0 2024-03-22T10:43:01.578 INFO:tasks.workunit.client.1.smithi150.stdout:2/762: write d12/d19/db6/da9/d61/d98/fbb [1809021,40805] 0 2024-03-22T10:43:01.578 INFO:tasks.workunit.client.1.smithi150.stdout:3/617: truncate d5/d22/d37/f79 460884 0 2024-03-22T10:43:01.579 INFO:tasks.workunit.client.1.smithi150.stdout:9/725: creat d1/d8/de/df/d20/d32/d75/ffb x:0 0 0 2024-03-22T10:43:01.583 INFO:tasks.workunit.client.1.smithi150.stdout:2/763: stat d12/d19/d1a/f7c 0 2024-03-22T10:43:01.586 INFO:tasks.workunit.client.1.smithi150.stdout:3/618: dread - d5/d22/d2f/d38/d3e/fb6 zero size 2024-03-22T10:43:01.587 INFO:tasks.workunit.client.1.smithi150.stdout:7/662: symlink d2/d5/d7/d8/d46/d93/ldd 0 2024-03-22T10:43:01.587 INFO:tasks.workunit.client.1.smithi150.stdout:6/621: link db/dd/d2e/fa6 db/dd/d82/d84/dc8/fe2 0 2024-03-22T10:43:01.592 INFO:tasks.workunit.client.1.smithi150.stdout:5/617: link d0/d8/f9a d0/d8/dc/d1d/d89/fc4 0 2024-03-22T10:43:01.593 INFO:tasks.workunit.client.1.smithi150.stdout:6/622: chown db/dd/d82/d84/dc8/fe2 7659 1 2024-03-22T10:43:01.594 INFO:tasks.workunit.client.1.smithi150.stdout:3/619: creat d5/da/d9d/fc5 x:0 0 0 2024-03-22T10:43:01.594 INFO:tasks.workunit.client.1.smithi150.stdout:7/663: fdatasync d2/d5/d7/d8/d57/d4a/f7f 0 2024-03-22T10:43:01.595 INFO:tasks.workunit.client.1.smithi150.stdout:0/687: dwrite d0/f89 [0,4194304] 0 2024-03-22T10:43:01.601 INFO:tasks.workunit.client.1.smithi150.stdout:5/618: write d0/d8/dc/de/f13 [3195500,22714] 0 2024-03-22T10:43:01.609 INFO:tasks.workunit.client.1.smithi150.stdout:5/619: mknod d0/d37/db4/cc5 0 2024-03-22T10:43:01.611 INFO:tasks.workunit.client.1.smithi150.stdout:0/688: write d0/d1/d84/d86/d11/d66/d35/d5f/fa8 [1924489,51190] 0 2024-03-22T10:43:01.613 INFO:tasks.workunit.client.1.smithi150.stdout:7/664: rmdir d2/d5/d7/d8/d57/d4a/d8d/dd9 0 2024-03-22T10:43:01.614 INFO:tasks.workunit.client.1.smithi150.stdout:9/726: dread d1/d8/de/df/da2/fbf [0,4194304] 0 2024-03-22T10:43:01.616 INFO:tasks.workunit.client.1.smithi150.stdout:7/665: chown d2/d5/d7/d9/db/l70 1 1 2024-03-22T10:43:01.712 INFO:tasks.workunit.client.1.smithi150.stdout:8/684: sync 2024-03-22T10:43:01.712 INFO:tasks.workunit.client.1.smithi150.stdout:1/619: sync 2024-03-22T10:43:01.712 INFO:tasks.workunit.client.1.smithi150.stdout:4/666: sync 2024-03-22T10:43:01.714 INFO:tasks.workunit.client.1.smithi150.stdout:3/620: dwrite d5/d53/d7e/f99 [0,4194304] 0 2024-03-22T10:43:01.718 INFO:tasks.workunit.client.1.smithi150.stdout:9/727: write d1/d8/de/df/f11 [1524012,24025] 0 2024-03-22T10:43:01.720 INFO:tasks.workunit.client.1.smithi150.stdout:6/623: dwrite db/dd/d2e/fc7 [0,4194304] 0 2024-03-22T10:43:01.727 INFO:tasks.workunit.client.1.smithi150.stdout:8/685: creat d0/d2/d7e/d96/fdf x:0 0 0 2024-03-22T10:43:01.729 INFO:tasks.workunit.client.1.smithi150.stdout:6/624: truncate db/d52/fb1 233074 0 2024-03-22T10:43:01.729 INFO:tasks.workunit.client.1.smithi150.stdout:1/620: unlink d1/da/d2a/d4e/l7b 0 2024-03-22T10:43:01.735 INFO:tasks.workunit.client.1.smithi150.stdout:8/686: creat d0/d2/d46/fe0 x:0 0 0 2024-03-22T10:43:01.736 INFO:tasks.workunit.client.1.smithi150.stdout:9/728: readlink d1/d8/de/df/da2/lc9 0 2024-03-22T10:43:01.739 INFO:tasks.workunit.client.1.smithi150.stdout:9/729: chown d1/d49/d98 48511322 1 2024-03-22T10:43:01.739 INFO:tasks.workunit.client.1.smithi150.stdout:1/621: fsync d1/d79/f85 0 2024-03-22T10:43:01.742 INFO:tasks.workunit.client.1.smithi150.stdout:6/625: creat db/dd/d15/d6a/fe3 x:0 0 0 2024-03-22T10:43:01.744 INFO:tasks.workunit.client.1.smithi150.stdout:1/622: stat d1/d19/d57 0 2024-03-22T10:43:01.750 INFO:tasks.workunit.client.1.smithi150.stdout:1/623: write d1/d19/d1a/f49 [3511909,100292] 0 2024-03-22T10:43:01.751 INFO:tasks.workunit.client.1.smithi150.stdout:7/666: dwrite d2/d5/d7/d8/d57/d4a/f7f [0,4194304] 0 2024-03-22T10:43:01.752 INFO:tasks.workunit.client.1.smithi150.stdout:5/620: dread d0/f10 [0,4194304] 0 2024-03-22T10:43:01.756 INFO:tasks.workunit.client.1.smithi150.stdout:5/621: write d0/d8/dc/de/f70 [752215,124978] 0 2024-03-22T10:43:01.758 INFO:tasks.workunit.client.1.smithi150.stdout:8/687: link d0/d2/fa8 d0/d2/d19/d29/fe1 0 2024-03-22T10:43:01.762 INFO:tasks.workunit.client.1.smithi150.stdout:6/626: mknod db/d52/dc3/ce4 0 2024-03-22T10:43:01.762 INFO:tasks.workunit.client.1.smithi150.stdout:1/624: rmdir d1/d19/d1a/d7d 39 2024-03-22T10:43:01.762 INFO:tasks.workunit.client.1.smithi150.stdout:6/627: dread - db/dd/d2e/fbb zero size 2024-03-22T10:43:01.763 INFO:tasks.workunit.client.1.smithi150.stdout:7/667: mknod d2/d5/d7/d9/db/d9f/d96/cde 0 2024-03-22T10:43:01.764 INFO:tasks.workunit.client.1.smithi150.stdout:9/730: dread d1/d8/f82 [0,4194304] 0 2024-03-22T10:43:01.770 INFO:tasks.workunit.client.1.smithi150.stdout:1/625: mkdir d1/d19/d50/dc7 0 2024-03-22T10:43:01.773 INFO:tasks.workunit.client.1.smithi150.stdout:4/667: dread d0/d1b/d4d/d6c/f62 [0,4194304] 0 2024-03-22T10:43:01.774 INFO:tasks.workunit.client.1.smithi150.stdout:7/668: write d2/d5/d7/d9/db/d6c/d9c/fc0 [564390,30212] 0 2024-03-22T10:43:01.775 INFO:tasks.workunit.client.1.smithi150.stdout:8/688: creat d0/ddd/fe2 x:0 0 0 2024-03-22T10:43:01.780 INFO:tasks.workunit.client.1.smithi150.stdout:1/626: mkdir d1/da/d15/d18/d4b/dc8 0 2024-03-22T10:43:01.780 INFO:tasks.workunit.client.1.smithi150.stdout:8/689: write d0/f5a [349452,6074] 0 2024-03-22T10:43:01.781 INFO:tasks.workunit.client.1.smithi150.stdout:6/628: rename db/dd/d15/d60/l40 to db/dd/d15/d70/le5 0 2024-03-22T10:43:01.782 INFO:tasks.workunit.client.1.smithi150.stdout:3/621: dread d5/d22/d37/f6c [0,4194304] 0 2024-03-22T10:43:01.782 INFO:tasks.workunit.client.1.smithi150.stdout:4/668: read d0/d1b/f23 [3508919,127749] 0 2024-03-22T10:43:01.782 INFO:tasks.workunit.client.1.smithi150.stdout:7/669: chown d2/d5/d7/d9/db/d9f/d96/lac 2868 1 2024-03-22T10:43:01.785 INFO:tasks.workunit.client.1.smithi150.stdout:9/731: write d1/d8/de/df/d36/f43 [1312330,91242] 0 2024-03-22T10:43:01.785 INFO:tasks.workunit.client.1.smithi150.stdout:4/669: mknod d0/d1b/d4d/d9f/dc6/cd6 0 2024-03-22T10:43:01.786 INFO:tasks.workunit.client.1.smithi150.stdout:3/622: write d5/fd [4033614,69098] 0 2024-03-22T10:43:01.792 INFO:tasks.workunit.client.1.smithi150.stdout:9/732: chown d1/d8/de/df/d20/fb8 57478 1 2024-03-22T10:43:01.792 INFO:tasks.workunit.client.1.smithi150.stdout:7/670: symlink d2/d5/d7/d8/d57/d54/d91/ldf 0 2024-03-22T10:43:01.794 INFO:tasks.workunit.client.1.smithi150.stdout:6/629: getdents db/dd/d15/d28 0 2024-03-22T10:43:01.795 INFO:tasks.workunit.client.1.smithi150.stdout:8/690: write d0/d2/d46/f77 [142212,108082] 0 2024-03-22T10:43:01.797 INFO:tasks.workunit.client.1.smithi150.stdout:6/630: chown db/d52/d9f/fe1 654 1 2024-03-22T10:43:01.798 INFO:tasks.workunit.client.1.smithi150.stdout:8/691: stat d0/d2/d7e/fa9 0 2024-03-22T10:43:01.799 INFO:tasks.workunit.client.1.smithi150.stdout:5/622: dread d0/dae/db2/f6b [0,4194304] 0 2024-03-22T10:43:01.800 INFO:tasks.workunit.client.1.smithi150.stdout:9/733: write d1/d8/de/df/d20/d32/d75/dc3/fcd [431258,83506] 0 2024-03-22T10:43:01.801 INFO:tasks.workunit.client.1.smithi150.stdout:4/670: mkdir d0/d2f/d63/d7b/d9c/db1/dd7 0 2024-03-22T10:43:01.802 INFO:tasks.workunit.client.1.smithi150.stdout:5/623: readlink d0/d33/d6f/l88 0 2024-03-22T10:43:01.802 INFO:tasks.workunit.client.1.smithi150.stdout:6/631: rmdir db/dd/d15/d60/d8b 39 2024-03-22T10:43:01.802 INFO:tasks.workunit.client.1.smithi150.stdout:8/692: chown d0/d2/d7e/c95 7 1 2024-03-22T10:43:01.802 INFO:tasks.workunit.client.1.smithi150.stdout:7/671: fsync d2/d5/d7/d8/d57/d54/f5f 0 2024-03-22T10:43:01.806 INFO:tasks.workunit.client.1.smithi150.stdout:9/734: fdatasync d1/d8/de/df/d20/d32/d75/ffb 0 2024-03-22T10:43:01.807 INFO:tasks.workunit.client.1.smithi150.stdout:8/693: mkdir d0/d2/d46/de3 0 2024-03-22T10:43:01.809 INFO:tasks.workunit.client.1.smithi150.stdout:7/672: mknod d2/d5/d7/d9/ce0 0 2024-03-22T10:43:01.811 INFO:tasks.workunit.client.1.smithi150.stdout:5/624: unlink d0/d8/dc/de/d14/d2e/d46/d4d/f51 0 2024-03-22T10:43:01.815 INFO:tasks.workunit.client.1.smithi150.stdout:7/673: unlink d2/d5/ca3 0 2024-03-22T10:43:01.816 INFO:tasks.workunit.client.1.smithi150.stdout:7/674: dread - d2/d5/d7/fc1 zero size 2024-03-22T10:43:01.816 INFO:tasks.workunit.client.1.smithi150.stdout:3/623: dwrite d5/da/f7a [0,4194304] 0 2024-03-22T10:43:01.820 INFO:tasks.workunit.client.1.smithi150.stdout:4/671: rename d0/d2f/cc8 to d0/d2f/d63/d7b/cd8 0 2024-03-22T10:43:01.821 INFO:tasks.workunit.client.1.smithi150.stdout:5/625: symlink d0/d37/db4/lc6 0 2024-03-22T10:43:01.823 INFO:tasks.workunit.client.1.smithi150.stdout:9/735: mknod d1/d8/de/df/d36/cfc 0 2024-03-22T10:43:01.828 INFO:tasks.workunit.client.1.smithi150.stdout:9/736: chown d1/d8/de/df/d24/f53 0 1 2024-03-22T10:43:01.830 INFO:tasks.workunit.client.1.smithi150.stdout:7/675: creat d2/d5/d7/d9/db/d9f/fe1 x:0 0 0 2024-03-22T10:43:01.834 INFO:tasks.workunit.client.1.smithi150.stdout:5/626: rename d0/d8/dc/de/d96/fbb to d0/d8/dc/de/d14/d2e/dbc/fc7 0 2024-03-22T10:43:01.836 INFO:tasks.workunit.client.1.smithi150.stdout:4/672: creat d0/d2f/d63/d7b/d9c/db1/dd7/fd9 x:0 0 0 2024-03-22T10:43:01.836 INFO:tasks.workunit.client.1.smithi150.stdout:3/624: link c4 d5/da/d9d/d9f/cc6 0 2024-03-22T10:43:01.837 INFO:tasks.workunit.client.1.smithi150.stdout:6/632: dwrite db/d52/f6f [0,4194304] 0 2024-03-22T10:43:01.840 INFO:tasks.workunit.client.1.smithi150.stdout:8/694: dread d0/d2/f3f [4194304,4194304] 0 2024-03-22T10:43:01.841 INFO:tasks.workunit.client.1.smithi150.stdout:4/673: creat d0/d1b/d4d/d6c/d61/d98/fda x:0 0 0 2024-03-22T10:43:01.842 INFO:tasks.workunit.client.1.smithi150.stdout:8/695: write d0/d2/f5f [3834329,117423] 0 2024-03-22T10:43:01.844 INFO:tasks.workunit.client.1.smithi150.stdout:9/737: link d1/d8/de/df/d20/d29/c3d d1/d8/de/df/d24/cfd 0 2024-03-22T10:43:01.850 INFO:tasks.workunit.client.1.smithi150.stdout:6/633: write db/dd/d2e/fa1 [694300,63326] 0 2024-03-22T10:43:01.850 INFO:tasks.workunit.client.1.smithi150.stdout:9/738: chown d1/d8/de/df/d36/d40/d41/fa9 265200888 1 2024-03-22T10:43:01.855 INFO:tasks.workunit.client.1.smithi150.stdout:6/634: chown db/dd/d15/d28/d6c/l89 26812 1 2024-03-22T10:43:01.859 INFO:tasks.workunit.client.1.smithi150.stdout:9/739: truncate d1/d8/f65 4795052 0 2024-03-22T10:43:01.862 INFO:tasks.workunit.client.1.smithi150.stdout:5/627: dwrite d0/d8/f34 [0,4194304] 0 2024-03-22T10:43:01.863 INFO:tasks.workunit.client.1.smithi150.stdout:6/635: mkdir db/d52/de6 0 2024-03-22T10:43:01.866 INFO:tasks.workunit.client.1.smithi150.stdout:2/764: sync 2024-03-22T10:43:01.866 INFO:tasks.workunit.client.1.smithi150.stdout:0/689: sync 2024-03-22T10:43:01.868 INFO:tasks.workunit.client.1.smithi150.stdout:2/765: fdatasync d12/d21/f4b 0 2024-03-22T10:43:01.869 INFO:tasks.workunit.client.1.smithi150.stdout:4/674: dwrite d0/d32/fae [0,4194304] 0 2024-03-22T10:43:01.875 INFO:tasks.workunit.client.1.smithi150.stdout:9/740: mkdir d1/d8/de/df/d20/d32/dfe 0 2024-03-22T10:43:01.877 INFO:tasks.workunit.client.1.smithi150.stdout:2/766: readlink d12/d19/db6/da9/d61/d98/l96 0 2024-03-22T10:43:01.877 INFO:tasks.workunit.client.1.smithi150.stdout:5/628: symlink d0/d33/lc8 0 2024-03-22T10:43:01.880 INFO:tasks.workunit.client.1.smithi150.stdout:0/690: creat d0/d1/d84/d86/d42/d10/d53/fdc x:0 0 0 2024-03-22T10:43:01.888 INFO:tasks.workunit.client.1.smithi150.stdout:9/741: creat d1/d8/de/df/d20/d32/d75/d99/da7/fff x:0 0 0 2024-03-22T10:43:01.891 INFO:tasks.workunit.client.1.smithi150.stdout:3/625: dwrite d5/d22/d2f/d38/d3e/d5b/f7f [0,4194304] 0 2024-03-22T10:43:01.891 INFO:tasks.workunit.client.1.smithi150.stdout:0/691: mknod d0/d12/cdd 0 2024-03-22T10:43:01.895 INFO:tasks.workunit.client.1.smithi150.stdout:0/692: chown d0/d12 30 1 2024-03-22T10:43:01.895 INFO:tasks.workunit.client.1.smithi150.stdout:9/742: chown d1/d8/de/df/d20/d32/dfe 57295271 1 2024-03-22T10:43:01.895 INFO:tasks.workunit.client.1.smithi150.stdout:6/636: truncate db/dd/d82/f9a 2953078 0 2024-03-22T10:43:01.902 INFO:tasks.workunit.client.1.smithi150.stdout:8/696: dread d0/d3c/f44 [4194304,4194304] 0 2024-03-22T10:43:01.904 INFO:tasks.workunit.client.1.smithi150.stdout:9/743: fsync d1/d8/de/df/d20/d32/d75/dc3/fd2 0 2024-03-22T10:43:01.906 INFO:tasks.workunit.client.1.smithi150.stdout:4/675: dread d0/d1b/d4d/d6c/f62 [0,4194304] 0 2024-03-22T10:43:01.908 INFO:tasks.workunit.client.1.smithi150.stdout:3/626: dread d5/d22/d2f/d38/d59/d88/fbe [0,4194304] 0 2024-03-22T10:43:01.908 INFO:tasks.workunit.client.1.smithi150.stdout:6/637: mkdir db/dd/d15/d60/d8b/de7 0 2024-03-22T10:43:01.911 INFO:tasks.workunit.client.1.smithi150.stdout:8/697: mknod d0/d2/d19/d29/d37/ce4 0 2024-03-22T10:43:01.912 INFO:tasks.workunit.client.1.smithi150.stdout:9/744: fdatasync d1/d8/de/df/d24/f89 0 2024-03-22T10:43:01.915 INFO:tasks.workunit.client.1.smithi150.stdout:4/676: creat d0/d2f/fdb x:0 0 0 2024-03-22T10:43:01.916 INFO:tasks.workunit.client.1.smithi150.stdout:6/638: dread - db/dd/d15/d6a/fd9 zero size 2024-03-22T10:43:01.917 INFO:tasks.workunit.client.1.smithi150.stdout:3/627: chown d5/f35 21177 1 2024-03-22T10:43:01.917 INFO:tasks.workunit.client.1.smithi150.stdout:0/693: link d0/f89 d0/d1/d84/d86/d11/d66/d35/fde 0 2024-03-22T10:43:01.920 INFO:tasks.workunit.client.1.smithi150.stdout:6/639: dread - db/d52/fc9 zero size 2024-03-22T10:43:01.923 INFO:tasks.workunit.client.1.smithi150.stdout:5/629: dwrite d0/d8/dc/de/d14/d2e/dbc/fc3 [0,4194304] 0 2024-03-22T10:43:01.936 INFO:tasks.workunit.client.1.smithi150.stdout:2/767: dwrite d12/d19/db6/da9/d67/f51 [0,4194304] 0 2024-03-22T10:43:01.942 INFO:tasks.workunit.client.1.smithi150.stdout:6/640: unlink db/dd/d15/d6a/fe3 0 2024-03-22T10:43:01.944 INFO:tasks.workunit.client.1.smithi150.stdout:5/630: truncate d0/d8/dc/f40 493386 0 2024-03-22T10:43:01.950 INFO:tasks.workunit.client.1.smithi150.stdout:6/641: dread - db/dd/d15/d30/fd5 zero size 2024-03-22T10:43:01.951 INFO:tasks.workunit.client.1.smithi150.stdout:0/694: getdents d0/d1/d84/d86/d11/d66/d37 0 2024-03-22T10:43:01.951 INFO:tasks.workunit.client.1.smithi150.stdout:9/745: dwrite d1/f71 [4194304,4194304] 0 2024-03-22T10:43:01.953 INFO:tasks.workunit.client.1.smithi150.stdout:5/631: write d0/d8/dc/de/d14/f25 [4115106,36778] 0 2024-03-22T10:43:01.957 INFO:tasks.workunit.client.1.smithi150.stdout:4/677: dwrite d0/d1b/d4d/f92 [0,4194304] 0 2024-03-22T10:43:01.959 INFO:tasks.workunit.client.1.smithi150.stdout:6/642: write db/dd/d2e/fa1 [1359715,88143] 0 2024-03-22T10:43:01.962 INFO:tasks.workunit.client.1.smithi150.stdout:9/746: mkdir d1/d19/d7d/dc1/dca/dd7/d100 0 2024-03-22T10:43:01.965 INFO:tasks.workunit.client.1.smithi150.stdout:4/678: chown d0/d1b/d4d/d6c/d61/dd1 3 1 2024-03-22T10:43:01.965 INFO:tasks.workunit.client.1.smithi150.stdout:0/695: creat d0/d1/d84/d86/d11/d66/d35/d5f/fdf x:0 0 0 2024-03-22T10:43:01.966 INFO:tasks.workunit.client.1.smithi150.stdout:5/632: stat d0/d8/dc/d1d/d53/d63/f9d 0 2024-03-22T10:43:01.969 INFO:tasks.workunit.client.1.smithi150.stdout:3/628: dwrite d5/d22/d2f/f41 [0,4194304] 0 2024-03-22T10:43:01.971 INFO:tasks.workunit.client.1.smithi150.stdout:8/698: dread d0/d2/d19/d29/d2f/df/f45 [0,4194304] 0 2024-03-22T10:43:01.971 INFO:tasks.workunit.client.1.smithi150.stdout:4/679: creat d0/d47/d72/fdc x:0 0 0 2024-03-22T10:43:01.972 INFO:tasks.workunit.client.1.smithi150.stdout:5/633: creat d0/d37/fc9 x:0 0 0 2024-03-22T10:43:01.973 INFO:tasks.workunit.client.1.smithi150.stdout:8/699: write d0/d2/d7e/d96/fdf [860292,114114] 0 2024-03-22T10:43:01.973 INFO:tasks.workunit.client.1.smithi150.stdout:5/634: stat d0/f3 0 2024-03-22T10:43:01.977 INFO:tasks.workunit.client.1.smithi150.stdout:8/700: mkdir d0/d2/d19/d29/d88/de5 0 2024-03-22T10:43:02.001 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:01 smithi115 ceph-mon[29111]: pgmap v19: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 121 MiB/s rd, 335 MiB/s wr, 677 op/s 2024-03-22T10:43:02.001 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:01 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:02.001 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:01 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:02.048 INFO:tasks.workunit.client.1.smithi150.stdout:5/635: unlink d0/d8/dc/de/f70 0 2024-03-22T10:43:02.050 INFO:tasks.workunit.client.1.smithi150.stdout:6/643: write db/dd/d15/d28/fa3 [472796,52228] 0 2024-03-22T10:43:02.052 INFO:tasks.workunit.client.1.smithi150.stdout:4/680: write d0/f8 [207299,118998] 0 2024-03-22T10:43:02.055 INFO:tasks.workunit.client.1.smithi150.stdout:8/701: read - d0/d2/d46/fe0 zero size 2024-03-22T10:43:02.059 INFO:tasks.workunit.client.1.smithi150.stdout:4/681: symlink d0/de/ldd 0 2024-03-22T10:43:02.060 INFO:tasks.workunit.client.1.smithi150.stdout:3/629: dread d5/da/f57 [0,4194304] 0 2024-03-22T10:43:02.062 INFO:tasks.workunit.client.1.smithi150.stdout:4/682: chown d0/d1b/c25 112971 1 2024-03-22T10:43:02.062 INFO:tasks.workunit.client.1.smithi150.stdout:8/702: write d0/d2/d19/d29/d37/fd6 [141797,26825] 0 2024-03-22T10:43:02.065 INFO:tasks.workunit.client.1.smithi150.stdout:2/768: dwrite d12/d35/fb2 [0,4194304] 0 2024-03-22T10:43:02.066 INFO:tasks.workunit.client.1.smithi150.stdout:3/630: read d5/d22/d37/f79 [66175,34769] 0 2024-03-22T10:43:02.066 INFO:tasks.workunit.client.1.smithi150.stdout:9/747: dwrite d1/d8/de/df/d20/d32/f5e [0,4194304] 0 2024-03-22T10:43:02.077 INFO:tasks.workunit.client.1.smithi150.stdout:8/703: dread d0/d2/d19/d29/d37/fd6 [0,4194304] 0 2024-03-22T10:43:02.078 INFO:tasks.workunit.client.1.smithi150.stdout:6/644: truncate db/dd/d15/d60/f53 1579909 0 2024-03-22T10:43:02.083 INFO:tasks.workunit.client.1.smithi150.stdout:2/769: mknod d12/d19/db6/da9/d61/cef 0 2024-03-22T10:43:02.083 INFO:tasks.workunit.client.1.smithi150.stdout:3/631: creat d5/d22/d2f/d38/d3e/d5b/fc7 x:0 0 0 2024-03-22T10:43:02.083 INFO:tasks.workunit.client.1.smithi150.stdout:5/636: dwrite d0/d8/dc/de/f15 [4194304,4194304] 0 2024-03-22T10:43:02.084 INFO:tasks.workunit.client.1.smithi150.stdout:8/704: truncate d0/d2/d46/f77 303699 0 2024-03-22T10:43:02.086 INFO:tasks.workunit.client.1.smithi150.stdout:3/632: chown d5/da/f63 5167 1 2024-03-22T10:43:02.087 INFO:tasks.workunit.client.1.smithi150.stdout:9/748: rename d1/d8/de/df/d20 to d1/d8/de/d73/d101 0 2024-03-22T10:43:02.087 INFO:tasks.workunit.client.1.smithi150.stdout:6/645: write db/dd/d2e/fc7 [477549,126722] 0 2024-03-22T10:43:02.089 INFO:tasks.workunit.client.1.smithi150.stdout:4/683: mknod d0/d1b/d4d/d6c/d61/d71/cde 0 2024-03-22T10:43:02.096 INFO:tasks.workunit.client.1.smithi150.stdout:5/637: creat d0/d8/dc/de/d14/fca x:0 0 0 2024-03-22T10:43:02.099 INFO:tasks.workunit.client.1.smithi150.stdout:6/646: rmdir db/dd/d15/d6a/d8d 39 2024-03-22T10:43:02.100 INFO:tasks.workunit.client.1.smithi150.stdout:3/633: mkdir d5/d22/dc8 0 2024-03-22T10:43:02.102 INFO:tasks.workunit.client.1.smithi150.stdout:3/634: stat d5/d53/c7c 0 2024-03-22T10:43:02.104 INFO:tasks.workunit.client.1.smithi150.stdout:4/684: mknod d0/d32/d7c/dc1/cdf 0 2024-03-22T10:43:02.105 INFO:tasks.workunit.client.1.smithi150.stdout:6/647: dread db/dd/d15/d30/fab [0,4194304] 0 2024-03-22T10:43:02.105 INFO:tasks.workunit.client.1.smithi150.stdout:9/749: mkdir d1/d8/de/d73/d101/d32/d75/d102 0 2024-03-22T10:43:02.107 INFO:tasks.workunit.client.1.smithi150.stdout:1/627: sync 2024-03-22T10:43:02.109 INFO:tasks.workunit.client.1.smithi150.stdout:8/705: dwrite d0/d5d/dd2/fae [0,4194304] 0 2024-03-22T10:43:02.110 INFO:tasks.workunit.client.1.smithi150.stdout:3/635: symlink d5/d22/d2f/d38/d3e/lc9 0 2024-03-22T10:43:02.111 INFO:tasks.workunit.client.1.smithi150.stdout:4/685: fsync d0/f11 0 2024-03-22T10:43:02.112 INFO:tasks.workunit.client.1.smithi150.stdout:1/628: write d1/da/d15/d4f/f60 [2383460,13791] 0 2024-03-22T10:43:02.117 INFO:tasks.workunit.client.1.smithi150.stdout:9/750: creat d1/d8/de/d73/d101/d29/f103 x:0 0 0 2024-03-22T10:43:02.118 INFO:tasks.workunit.client.1.smithi150.stdout:8/706: dread d0/d2/d19/d29/d37/fd6 [0,4194304] 0 2024-03-22T10:43:02.120 INFO:tasks.workunit.client.1.smithi150.stdout:2/770: truncate d12/d19/db6/da9/d67/f51 367773 0 2024-03-22T10:43:02.121 INFO:tasks.workunit.client.1.smithi150.stdout:9/751: chown d1/d8/de/df/d36/f39 249917894 1 2024-03-22T10:43:02.122 INFO:tasks.workunit.client.1.smithi150.stdout:8/707: write d0/d2/d7e/d96/f99 [984225,74739] 0 2024-03-22T10:43:02.123 INFO:tasks.workunit.client.1.smithi150.stdout:4/686: mkdir d0/d1b/d68/de0 0 2024-03-22T10:43:02.137 INFO:tasks.workunit.client.1.smithi150.stdout:7/676: sync 2024-03-22T10:43:02.147 INFO:tasks.workunit.client.1.smithi150.stdout:1/629: mkdir d1/da/d15/d4f/dc9 0 2024-03-22T10:43:02.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:01 smithi150 ceph-mon[34518]: pgmap v19: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 121 MiB/s rd, 335 MiB/s wr, 677 op/s 2024-03-22T10:43:02.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:01 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:02.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:01 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:02.148 INFO:tasks.workunit.client.1.smithi150.stdout:4/687: chown d0/ccd 7105 1 2024-03-22T10:43:02.148 INFO:tasks.workunit.client.1.smithi150.stdout:5/638: truncate d0/d8/dc/de/d14/d2e/d5a/f8e 3089373 0 2024-03-22T10:43:02.150 INFO:tasks.workunit.client.1.smithi150.stdout:2/771: rmdir d12/d19/db6/da9/d67/d3c/d75 39 2024-03-22T10:43:02.151 INFO:tasks.workunit.client.1.smithi150.stdout:4/688: fdatasync d0/d32/f56 0 2024-03-22T10:43:02.152 INFO:tasks.workunit.client.1.smithi150.stdout:7/677: dread - d2/d5/d7/d38/fd3 zero size 2024-03-22T10:43:02.156 INFO:tasks.workunit.client.1.smithi150.stdout:4/689: chown d0/d2f/d63/d7b/d9c/lbe 12 1 2024-03-22T10:43:02.158 INFO:tasks.workunit.client.1.smithi150.stdout:8/708: rename d0/d3c/d71/d7a/d89/db5/cba to d0/d2/d19/d29/d37/ce6 0 2024-03-22T10:43:02.159 INFO:tasks.workunit.client.1.smithi150.stdout:2/772: rmdir d12/d19/db6/d63/dd8 39 2024-03-22T10:43:02.160 INFO:tasks.workunit.client.1.smithi150.stdout:4/690: write d0/d1b/d68/fc4 [737480,92710] 0 2024-03-22T10:43:02.166 INFO:tasks.workunit.client.1.smithi150.stdout:6/648: dwrite db/dd/d25/f6d [0,4194304] 0 2024-03-22T10:43:02.170 INFO:tasks.workunit.client.1.smithi150.stdout:5/639: rename d0/d8/dc/de/d14/d2e/d45/db3/c5e to d0/d8/dc/de/d14/d2e/d5a/d66/da9/ccb 0 2024-03-22T10:43:02.171 INFO:tasks.workunit.client.1.smithi150.stdout:1/630: link d1/lbb d1/d19/d50/d6e/lca 0 2024-03-22T10:43:02.171 INFO:tasks.workunit.client.1.smithi150.stdout:7/678: write d2/d5/d7/d9/db/d6c/fb6 [202408,119627] 0 2024-03-22T10:43:02.174 INFO:tasks.workunit.client.1.smithi150.stdout:6/649: read db/f4a [4061809,81718] 0 2024-03-22T10:43:02.178 INFO:tasks.workunit.client.1.smithi150.stdout:9/752: dwrite d1/d8/de/df/d36/d40/d41/fac [0,4194304] 0 2024-03-22T10:43:02.179 INFO:tasks.workunit.client.1.smithi150.stdout:5/640: dread - d0/d8/dc/d1d/d89/f8f zero size 2024-03-22T10:43:02.180 INFO:tasks.workunit.client.1.smithi150.stdout:2/773: creat d12/d19/db6/da9/d67/d3c/d75/ff0 x:0 0 0 2024-03-22T10:43:02.182 INFO:tasks.workunit.client.1.smithi150.stdout:1/631: write d1/da/d15/d18/d4b/f6a [6413660,3624] 0 2024-03-22T10:43:02.189 INFO:tasks.workunit.client.1.smithi150.stdout:8/709: dread d0/d3c/f44 [4194304,4194304] 0 2024-03-22T10:43:02.193 INFO:tasks.workunit.client.1.smithi150.stdout:7/679: symlink d2/d5/d7/d9/db/d9f/le2 0 2024-03-22T10:43:02.194 INFO:tasks.workunit.client.1.smithi150.stdout:5/641: creat d0/d8/dc/d64/fcc x:0 0 0 2024-03-22T10:43:02.195 INFO:tasks.workunit.client.1.smithi150.stdout:2/774: mkdir d12/d19/db6/df1 0 2024-03-22T10:43:02.195 INFO:tasks.workunit.client.1.smithi150.stdout:6/650: write db/dd/d15/d60/f53 [1895270,64092] 0 2024-03-22T10:43:02.196 INFO:tasks.workunit.client.1.smithi150.stdout:4/691: dwrite d0/f26 [4194304,4194304] 0 2024-03-22T10:43:02.199 INFO:tasks.workunit.client.1.smithi150.stdout:2/775: write d12/d35/fb2 [3250536,119044] 0 2024-03-22T10:43:02.201 INFO:tasks.workunit.client.1.smithi150.stdout:7/680: fsync d2/d5/d7/d9/db/d6c/d9c/fda 0 2024-03-22T10:43:02.206 INFO:tasks.workunit.client.1.smithi150.stdout:1/632: getdents d1/d19/d57 0 2024-03-22T10:43:02.207 INFO:tasks.workunit.client.1.smithi150.stdout:4/692: symlink d0/d32/d7c/le1 0 2024-03-22T10:43:02.207 INFO:tasks.workunit.client.1.smithi150.stdout:2/776: chown d12/d19/db6/da9/d61/d98/fbb 8245001 1 2024-03-22T10:43:02.207 INFO:tasks.workunit.client.1.smithi150.stdout:6/651: write db/dd/d25/f3b [946902,70302] 0 2024-03-22T10:43:02.208 INFO:tasks.workunit.client.1.smithi150.stdout:4/693: chown d0/d32/d7c 14 1 2024-03-22T10:43:02.210 INFO:tasks.workunit.client.1.smithi150.stdout:5/642: mknod d0/d8/dc/de/d14/d2e/d46/d4d/dad/ccd 0 2024-03-22T10:43:02.213 INFO:tasks.workunit.client.1.smithi150.stdout:9/753: write d1/d8/de/df/d24/f5f [657755,45605] 0 2024-03-22T10:43:02.215 INFO:tasks.workunit.client.1.smithi150.stdout:6/652: symlink db/dd/d25/le8 0 2024-03-22T10:43:02.218 INFO:tasks.workunit.client.1.smithi150.stdout:9/754: fdatasync d1/d8/de/d73/d101/f50 0 2024-03-22T10:43:02.218 INFO:tasks.workunit.client.1.smithi150.stdout:4/694: rename d0/d1b/d68/c95 to d0/d1b/d4d/d9f/ce2 0 2024-03-22T10:43:02.219 INFO:tasks.workunit.client.1.smithi150.stdout:2/777: creat d12/d19/db6/da9/d2c/d73/ff2 x:0 0 0 2024-03-22T10:43:02.223 INFO:tasks.workunit.client.1.smithi150.stdout:7/681: rename d2/d5/d7/d9/f62 to d2/d5/d7/d9/db/d9f/d45/fe3 0 2024-03-22T10:43:02.224 INFO:tasks.workunit.client.1.smithi150.stdout:6/653: mkdir db/dd/d82/de9 0 2024-03-22T10:43:02.230 INFO:tasks.workunit.client.1.smithi150.stdout:2/778: mknod d12/d19/db6/da9/d67/d8d/cf3 0 2024-03-22T10:43:02.233 INFO:tasks.workunit.client.1.smithi150.stdout:2/779: write d12/d21/fb3 [922012,104481] 0 2024-03-22T10:43:02.233 INFO:tasks.workunit.client.1.smithi150.stdout:6/654: rename db/dd/d15/d60/c24 to db/dd/d15/d30/ddf/cea 0 2024-03-22T10:43:02.239 INFO:tasks.workunit.client.1.smithi150.stdout:1/633: dwrite d1/d7c/fb5 [0,4194304] 0 2024-03-22T10:43:02.240 INFO:tasks.workunit.client.1.smithi150.stdout:6/655: chown db/dd/d15/d60/d8b/de7 3 1 2024-03-22T10:43:02.241 INFO:tasks.workunit.client.1.smithi150.stdout:2/780: dread - d12/d19/db6/d63/dd8/fe8 zero size 2024-03-22T10:43:02.241 INFO:tasks.workunit.client.1.smithi150.stdout:8/710: dread d0/f34 [4194304,4194304] 0 2024-03-22T10:43:02.248 INFO:tasks.workunit.client.1.smithi150.stdout:1/634: rename d1/d79/d96 to d1/da/d15/d18/d4b/dcb 0 2024-03-22T10:43:02.251 INFO:tasks.workunit.client.1.smithi150.stdout:1/635: write d1/d7c/fb5 [3766472,17549] 0 2024-03-22T10:43:02.253 INFO:tasks.workunit.client.1.smithi150.stdout:8/711: mknod d0/d2/d46/da2/dd0/ce7 0 2024-03-22T10:43:02.254 INFO:tasks.workunit.client.1.smithi150.stdout:6/656: read db/dd/d15/d6a/d8d/fbd [2436441,3335] 0 2024-03-22T10:43:02.256 INFO:tasks.workunit.client.1.smithi150.stdout:4/695: dread d0/d1b/f24 [8388608,4194304] 0 2024-03-22T10:43:02.256 INFO:tasks.workunit.client.1.smithi150.stdout:2/781: rename d12/d19/db6/da9/f43 to d12/d19/db6/da9/d67/db4/ff4 0 2024-03-22T10:43:02.258 INFO:tasks.workunit.client.1.smithi150.stdout:9/755: dwrite d1/d8/de/d73/d101/d32/d75/f84 [0,4194304] 0 2024-03-22T10:43:02.261 INFO:tasks.workunit.client.1.smithi150.stdout:9/756: fdatasync d1/d8/de/d73/d101/d29/f9b 0 2024-03-22T10:43:02.262 INFO:tasks.workunit.client.1.smithi150.stdout:8/712: getdents d0/d2/d46/de3 0 2024-03-22T10:43:02.263 INFO:tasks.workunit.client.1.smithi150.stdout:6/657: symlink db/dd/d15/d60/d50/leb 0 2024-03-22T10:43:02.264 INFO:tasks.workunit.client.1.smithi150.stdout:6/658: dread - db/dd/d15/d30/f7c zero size 2024-03-22T10:43:02.269 INFO:tasks.workunit.client.1.smithi150.stdout:8/713: read d0/d3c/d71/d7a/d89/db5/fc7 [687803,87231] 0 2024-03-22T10:43:02.273 INFO:tasks.workunit.client.1.smithi150.stdout:1/636: rename d1/d19/d1a/c31 to d1/da/ccc 0 2024-03-22T10:43:02.276 INFO:tasks.workunit.client.1.smithi150.stdout:6/659: rmdir db/dd/d15/d6a/d8d 39 2024-03-22T10:43:02.277 INFO:tasks.workunit.client.1.smithi150.stdout:8/714: read - d0/d2/d19/d29/f9d zero size 2024-03-22T10:43:02.278 INFO:tasks.workunit.client.1.smithi150.stdout:4/696: link d0/d1b/d4d/d6c/d61/l55 d0/d1b/d4d/d6c/le3 0 2024-03-22T10:43:02.281 INFO:tasks.workunit.client.1.smithi150.stdout:8/715: rmdir d0/d2/d19/d29/d37/d7d/d9e 39 2024-03-22T10:43:02.285 INFO:tasks.workunit.client.1.smithi150.stdout:9/757: dwrite d1/d8/d95/fa6 [0,4194304] 0 2024-03-22T10:43:02.289 INFO:tasks.workunit.client.1.smithi150.stdout:2/782: dwrite d12/d19/db6/da9/d67/d3c/f92 [0,4194304] 0 2024-03-22T10:43:02.294 INFO:tasks.workunit.client.1.smithi150.stdout:4/697: symlink d0/d2f/d63/d7b/le4 0 2024-03-22T10:43:02.295 INFO:tasks.workunit.client.1.smithi150.stdout:0/696: sync 2024-03-22T10:43:02.298 INFO:tasks.workunit.client.1.smithi150.stdout:8/716: symlink d0/d2/d46/le8 0 2024-03-22T10:43:02.301 INFO:tasks.workunit.client.1.smithi150.stdout:9/758: rename d1/d8/de/d73/d101/d32/d75 to d1/d8/de/df/d36/d40/d44/db5/dda/d104 0 2024-03-22T10:43:02.306 INFO:tasks.workunit.client.1.smithi150.stdout:8/717: creat d0/d5d/dd9/fe9 x:0 0 0 2024-03-22T10:43:02.307 INFO:tasks.workunit.client.1.smithi150.stdout:9/759: fsync d1/d8/de/d73/d101/d29/f54 0 2024-03-22T10:43:02.307 INFO:tasks.workunit.client.1.smithi150.stdout:4/698: getdents d0/d32/d7c/dd4 0 2024-03-22T10:43:02.313 INFO:tasks.workunit.client.1.smithi150.stdout:0/697: dread d0/f7b [4194304,4194304] 0 2024-03-22T10:43:02.314 INFO:tasks.workunit.client.1.smithi150.stdout:4/699: creat d0/d1b/d68/fe5 x:0 0 0 2024-03-22T10:43:02.316 INFO:tasks.workunit.client.1.smithi150.stdout:9/760: write d1/d8/de/df/da2/fbf [5205046,50618] 0 2024-03-22T10:43:02.319 INFO:tasks.workunit.client.1.smithi150.stdout:2/783: dwrite d12/d19/d1a/f30 [0,4194304] 0 2024-03-22T10:43:02.321 INFO:tasks.workunit.client.1.smithi150.stdout:4/700: write d0/d1b/d4d/f65 [1431160,39044] 0 2024-03-22T10:43:02.321 INFO:tasks.workunit.client.1.smithi150.stdout:6/660: dread db/dd/d15/d30/f51 [0,4194304] 0 2024-03-22T10:43:02.323 INFO:tasks.workunit.client.1.smithi150.stdout:0/698: fdatasync d0/d1/d84/d86/d42/f95 0 2024-03-22T10:43:02.326 INFO:tasks.workunit.client.1.smithi150.stdout:9/761: mkdir d1/d8/de/d105 0 2024-03-22T10:43:02.330 INFO:tasks.workunit.client.1.smithi150.stdout:4/701: read - d0/d32/f8e zero size 2024-03-22T10:43:02.330 INFO:tasks.workunit.client.1.smithi150.stdout:0/699: write d0/d1/d84/d86/d11/d66/d37/fd1 [402426,73930] 0 2024-03-22T10:43:02.334 INFO:tasks.workunit.client.1.smithi150.stdout:2/784: truncate d12/d19/db6/da9/d67/db4/ff4 3491429 0 2024-03-22T10:43:02.334 INFO:tasks.workunit.client.1.smithi150.stdout:8/718: dwrite d0/d5d/fbc [0,4194304] 0 2024-03-22T10:43:02.337 INFO:tasks.workunit.client.1.smithi150.stdout:4/702: write d0/d2f/d63/d7b/d9c/fa6 [269597,78522] 0 2024-03-22T10:43:02.343 INFO:tasks.workunit.client.1.smithi150.stdout:8/719: mkdir d0/d2/d19/d29/d2f/df/d1b/dea 0 2024-03-22T10:43:02.344 INFO:tasks.workunit.client.1.smithi150.stdout:4/703: chown d0/d2f/f52 56723922 1 2024-03-22T10:43:02.356 INFO:tasks.workunit.client.1.smithi150.stdout:8/720: fdatasync d0/d2/d19/d29/d2f/df/f42 0 2024-03-22T10:43:02.364 INFO:tasks.workunit.client.1.smithi150.stdout:0/700: dwrite d0/fb2 [0,4194304] 0 2024-03-22T10:43:02.364 INFO:tasks.workunit.client.1.smithi150.stdout:9/762: dwrite d1/d8/de/df/da2/fbf [4194304,4194304] 0 2024-03-22T10:43:02.367 INFO:tasks.workunit.client.1.smithi150.stdout:6/661: dread db/dd/d15/d60/f9c [0,4194304] 0 2024-03-22T10:43:02.368 INFO:tasks.workunit.client.1.smithi150.stdout:0/701: chown d0/d12/c25 9551 1 2024-03-22T10:43:02.376 INFO:tasks.workunit.client.1.smithi150.stdout:2/785: dwrite d12/d19/db6/f44 [0,4194304] 0 2024-03-22T10:43:02.381 INFO:tasks.workunit.client.1.smithi150.stdout:0/702: chown d0/d1/d84/d86/d11/d66/d35/d5f/fd9 389 1 2024-03-22T10:43:02.389 INFO:tasks.workunit.client.1.smithi150.stdout:8/721: dwrite d0/d3c/d71/d7a/d89/fcf [4194304,4194304] 0 2024-03-22T10:43:02.391 INFO:tasks.workunit.client.1.smithi150.stdout:6/662: dread db/dd/d2e/fbc [0,4194304] 0 2024-03-22T10:43:02.392 INFO:tasks.workunit.client.1.smithi150.stdout:9/763: mkdir d1/d8/de/d73/d101/d32/d106 0 2024-03-22T10:43:02.402 INFO:tasks.workunit.client.1.smithi150.stdout:0/703: rename d0/d1/d84/d86/d42/d10/d53/c8e to d0/d1/d84/d86/d42/d10/d38/d52/d9b/dac/ce0 0 2024-03-22T10:43:02.403 INFO:tasks.workunit.client.1.smithi150.stdout:6/663: truncate db/d52/f93 325613 0 2024-03-22T10:43:02.404 INFO:tasks.workunit.client.1.smithi150.stdout:2/786: link d12/d19/db6/da9/d67/d3c/c52 d12/d19/db6/d63/dd8/cf5 0 2024-03-22T10:43:02.412 INFO:tasks.workunit.client.1.smithi150.stdout:9/764: creat d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/f107 x:0 0 0 2024-03-22T10:43:02.414 INFO:tasks.workunit.client.1.smithi150.stdout:0/704: creat d0/d1/d84/d86/d42/d10/d38/d52/d68/fe1 x:0 0 0 2024-03-22T10:43:02.418 INFO:tasks.workunit.client.1.smithi150.stdout:0/705: stat d0/d1/d84/d86/d42/d10/d38/d52/d68/fe1 0 2024-03-22T10:43:02.419 INFO:tasks.workunit.client.1.smithi150.stdout:9/765: write d1/d8/d95/fee [3992,45572] 0 2024-03-22T10:43:02.430 INFO:tasks.workunit.client.1.smithi150.stdout:9/766: creat d1/d8/de/df/d36/d40/d44/db5/f108 x:0 0 0 2024-03-22T10:43:02.430 INFO:tasks.workunit.client.1.smithi150.stdout:2/787: write d12/d19/db6/da9/d61/f8c [286323,103381] 0 2024-03-22T10:43:02.432 INFO:tasks.workunit.client.1.smithi150.stdout:8/722: dwrite d0/d3c/d71/d7a/d89/db5/fc7 [0,4194304] 0 2024-03-22T10:43:02.436 INFO:tasks.workunit.client.1.smithi150.stdout:8/723: chown d0/d2/d46/f77 1 1 2024-03-22T10:43:02.442 INFO:tasks.workunit.client.1.smithi150.stdout:6/664: dwrite db/dd/d15/d60/f86 [4194304,4194304] 0 2024-03-22T10:43:02.445 INFO:tasks.workunit.client.1.smithi150.stdout:0/706: dwrite d0/d1/d84/d86/d42/d10/d38/d52/f65 [0,4194304] 0 2024-03-22T10:43:02.448 INFO:tasks.workunit.client.1.smithi150.stdout:6/665: dread - db/dd/fd4 zero size 2024-03-22T10:43:02.457 INFO:tasks.workunit.client.1.smithi150.stdout:2/788: dwrite d12/d19/d2d/f36 [0,4194304] 0 2024-03-22T10:43:02.459 INFO:tasks.workunit.client.1.smithi150.stdout:6/666: read db/dd/d25/f45 [282244,98552] 0 2024-03-22T10:43:02.463 INFO:tasks.workunit.client.1.smithi150.stdout:3/636: sync 2024-03-22T10:43:02.464 INFO:tasks.workunit.client.1.smithi150.stdout:8/724: dwrite d0/d2/d19/d29/d2e/d70/fb8 [0,4194304] 0 2024-03-22T10:43:02.469 INFO:tasks.workunit.client.1.smithi150.stdout:8/725: mknod d0/ddd/ceb 0 2024-03-22T10:43:02.471 INFO:tasks.workunit.client.1.smithi150.stdout:3/637: mknod d5/d22/cca 0 2024-03-22T10:43:02.472 INFO:tasks.workunit.client.1.smithi150.stdout:6/667: getdents db/dd/d82/d84 0 2024-03-22T10:43:02.474 INFO:tasks.workunit.client.1.smithi150.stdout:8/726: mkdir d0/d2/d19/d29/d88/dec 0 2024-03-22T10:43:02.475 INFO:tasks.workunit.client.1.smithi150.stdout:6/668: dread - db/dd/d82/d84/dc8/fe2 zero size 2024-03-22T10:43:02.477 INFO:tasks.workunit.client.1.smithi150.stdout:8/727: readlink d0/d2/d19/d29/d37/d7d/lcd 0 2024-03-22T10:43:02.490 INFO:tasks.workunit.client.1.smithi150.stdout:3/638: dwrite d5/d22/d2f/f41 [0,4194304] 0 2024-03-22T10:43:02.493 INFO:tasks.workunit.client.1.smithi150.stdout:8/728: truncate d0/d2/d19/d29/d2f/df/f8a 565366 0 2024-03-22T10:43:02.493 INFO:tasks.workunit.client.1.smithi150.stdout:3/639: dread - d5/da/d9d/fc5 zero size 2024-03-22T10:43:02.498 INFO:tasks.workunit.client.1.smithi150.stdout:3/640: mkdir d5/d22/dcb 0 2024-03-22T10:43:02.500 INFO:tasks.workunit.client.1.smithi150.stdout:8/729: creat d0/d3c/fed x:0 0 0 2024-03-22T10:43:02.500 INFO:tasks.workunit.client.1.smithi150.stdout:3/641: stat d5/d53/d7e/d95 0 2024-03-22T10:43:02.505 INFO:tasks.workunit.client.1.smithi150.stdout:3/642: write d5/d22/d2f/d38/d3e/d5b/da6/fab [954563,85318] 0 2024-03-22T10:43:02.512 INFO:tasks.workunit.client.1.smithi150.stdout:3/643: mknod d5/d22/d2f/d38/d3e/d5b/ccc 0 2024-03-22T10:43:02.516 INFO:tasks.workunit.client.1.smithi150.stdout:6/669: dread db/dd/d15/d30/f4b [0,4194304] 0 2024-03-22T10:43:02.517 INFO:tasks.workunit.client.1.smithi150.stdout:2/789: dread d12/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:43:02.517 INFO:tasks.workunit.client.1.smithi150.stdout:3/644: symlink d5/d22/d2f/d38/d3e/lcd 0 2024-03-22T10:43:02.519 INFO:tasks.workunit.client.1.smithi150.stdout:8/730: write d0/d5d/dd2/f4b [1703213,128331] 0 2024-03-22T10:43:02.523 INFO:tasks.workunit.client.1.smithi150.stdout:2/790: fsync d12/d19/db6/da9/d97/d78/dbd/fec 0 2024-03-22T10:43:02.524 INFO:tasks.workunit.client.1.smithi150.stdout:3/645: mknod d5/da/d62/dba/cce 0 2024-03-22T10:43:02.526 INFO:tasks.workunit.client.1.smithi150.stdout:2/791: dread - d12/d19/db6/da9/d2c/fe7 zero size 2024-03-22T10:43:02.529 INFO:tasks.workunit.client.1.smithi150.stdout:8/731: symlink d0/d2/d7e/d96/lee 0 2024-03-22T10:43:02.532 INFO:tasks.workunit.client.1.smithi150.stdout:3/646: creat d5/fcf x:0 0 0 2024-03-22T10:43:02.536 INFO:tasks.workunit.client.1.smithi150.stdout:3/647: write d5/d22/d2f/d38/d3e/fb6 [964811,82517] 0 2024-03-22T10:43:02.536 INFO:tasks.workunit.client.1.smithi150.stdout:2/792: mknod d12/d19/db6/da9/d2c/cf6 0 2024-03-22T10:43:02.542 INFO:tasks.workunit.client.1.smithi150.stdout:3/648: rename d5/d22/d2f/d38/f42 to d5/da/d78/db7/fd0 0 2024-03-22T10:43:02.544 INFO:tasks.workunit.client.1.smithi150.stdout:3/649: fsync d5/d53/d7e/f99 0 2024-03-22T10:43:02.544 INFO:tasks.workunit.client.1.smithi150.stdout:2/793: creat d12/d19/db6/da9/d67/ff7 x:0 0 0 2024-03-22T10:43:02.593 INFO:tasks.workunit.client.1.smithi150.stdout:2/794: readlink d12/d21/d81/l83 0 2024-03-22T10:43:02.600 INFO:tasks.workunit.client.1.smithi150.stdout:2/795: chown d12/d21/c6e 11 1 2024-03-22T10:43:02.601 INFO:tasks.workunit.client.1.smithi150.stdout:2/796: dread - d12/fb7 zero size 2024-03-22T10:43:02.608 INFO:tasks.workunit.client.1.smithi150.stdout:8/732: dwrite d0/d2/d19/f78 [0,4194304] 0 2024-03-22T10:43:02.621 INFO:tasks.workunit.client.1.smithi150.stdout:2/797: dread d12/d19/db6/f44 [0,4194304] 0 2024-03-22T10:43:02.623 INFO:tasks.workunit.client.1.smithi150.stdout:2/798: stat d12/d19/db6/da9/l90 0 2024-03-22T10:43:02.625 INFO:tasks.workunit.client.1.smithi150.stdout:3/650: dread d5/d22/d2f/d38/d3e/f6f [0,4194304] 0 2024-03-22T10:43:02.626 INFO:tasks.workunit.client.1.smithi150.stdout:2/799: mkdir d12/d35/df8 0 2024-03-22T10:43:02.628 INFO:tasks.workunit.client.1.smithi150.stdout:3/651: truncate d5/d53/fb3 323963 0 2024-03-22T10:43:02.632 INFO:tasks.workunit.client.1.smithi150.stdout:3/652: write d5/fb [3930013,83791] 0 2024-03-22T10:43:02.637 INFO:tasks.workunit.client.1.smithi150.stdout:3/653: mknod d5/d22/d37/cd1 0 2024-03-22T10:43:02.637 INFO:tasks.workunit.client.1.smithi150.stdout:2/800: dread d12/d19/db6/da9/d67/f4e [0,4194304] 0 2024-03-22T10:43:02.639 INFO:tasks.workunit.client.1.smithi150.stdout:3/654: chown d5/d22/d2f/d38/d3e/d5b/da6/fab 0 1 2024-03-22T10:43:02.641 INFO:tasks.workunit.client.1.smithi150.stdout:2/801: write d12/d19/db6/f44 [326854,15723] 0 2024-03-22T10:43:02.645 INFO:tasks.workunit.client.1.smithi150.stdout:7/682: sync 2024-03-22T10:43:02.646 INFO:tasks.workunit.client.1.smithi150.stdout:5/643: sync 2024-03-22T10:43:02.651 INFO:tasks.workunit.client.1.smithi150.stdout:7/683: dread - d2/d5/d7/d88/f8f zero size 2024-03-22T10:43:02.657 INFO:tasks.workunit.client.1.smithi150.stdout:7/684: creat d2/d5/d7/d9/db/d9f/d96/fe4 x:0 0 0 2024-03-22T10:43:02.683 INFO:tasks.workunit.client.1.smithi150.stdout:8/733: dwrite d0/d5d/f6c [0,4194304] 0 2024-03-22T10:43:02.685 INFO:tasks.workunit.client.1.smithi150.stdout:2/802: dread d12/d19/db6/da9/d67/f74 [0,4194304] 0 2024-03-22T10:43:02.685 INFO:tasks.workunit.client.1.smithi150.stdout:2/803: fdatasync d12/d21/fb3 0 2024-03-22T10:43:02.686 INFO:tasks.workunit.client.1.smithi150.stdout:7/685: unlink d2/d5/d7/d8/l3a 0 2024-03-22T10:43:02.688 INFO:tasks.workunit.client.1.smithi150.stdout:8/734: creat d0/d2/d19/d29/d2e/fef x:0 0 0 2024-03-22T10:43:02.690 INFO:tasks.workunit.client.1.smithi150.stdout:3/655: truncate d5/d22/d2f/d38/d3e/d5b/f7f 2046290 0 2024-03-22T10:43:02.691 INFO:tasks.workunit.client.1.smithi150.stdout:5/644: write d0/d8/dc/de/d14/d2e/d46/f72 [181874,7475] 0 2024-03-22T10:43:02.692 INFO:tasks.workunit.client.1.smithi150.stdout:2/804: symlink d12/d19/db6/da9/d67/d3c/lf9 0 2024-03-22T10:43:02.695 INFO:tasks.workunit.client.1.smithi150.stdout:2/805: chown d12/d19/db6/da9/d67/d8d/da6 15710309 1 2024-03-22T10:43:02.695 INFO:tasks.workunit.client.1.smithi150.stdout:8/735: rename d0/c2d to d0/d2/d19/d29/d88/cf0 0 2024-03-22T10:43:02.697 INFO:tasks.workunit.client.1.smithi150.stdout:5/645: mkdir d0/d8/dc/d64/dce 0 2024-03-22T10:43:02.698 INFO:tasks.workunit.client.1.smithi150.stdout:2/806: write d12/d19/db6/f44 [1436130,33702] 0 2024-03-22T10:43:02.701 INFO:tasks.workunit.client.1.smithi150.stdout:8/736: write d0/d2/d19/d29/d37/fc5 [1030264,58565] 0 2024-03-22T10:43:02.705 INFO:tasks.workunit.client.1.smithi150.stdout:3/656: creat d5/d22/fd2 x:0 0 0 2024-03-22T10:43:02.708 INFO:tasks.workunit.client.1.smithi150.stdout:8/737: read d0/d3c/d71/f59 [167174,27822] 0 2024-03-22T10:43:02.708 INFO:tasks.workunit.client.1.smithi150.stdout:2/807: stat d12/d19/db6/da9/d67/f51 0 2024-03-22T10:43:02.709 INFO:tasks.workunit.client.1.smithi150.stdout:5/646: creat d0/d8/dc/de/d14/d2e/d46/d9c/dc0/fcf x:0 0 0 2024-03-22T10:43:02.709 INFO:tasks.workunit.client.1.smithi150.stdout:7/686: dwrite d2/d5/d7/d9/db/d9f/d96/fe4 [0,4194304] 0 2024-03-22T10:43:02.717 INFO:tasks.workunit.client.1.smithi150.stdout:8/738: chown d0/d2/d19/d29/d2e/d70/c53 503 1 2024-03-22T10:43:02.717 INFO:tasks.workunit.client.1.smithi150.stdout:3/657: symlink d5/da/d78/db7/ld3 0 2024-03-22T10:43:02.720 INFO:tasks.workunit.client.1.smithi150.stdout:8/739: write d0/d2/d19/d29/d37/d7d/d9e/fcb [1582540,77268] 0 2024-03-22T10:43:02.724 INFO:tasks.workunit.client.1.smithi150.stdout:3/658: creat d5/d22/d2f/d38/d3e/d5b/fd4 x:0 0 0 2024-03-22T10:43:02.724 INFO:tasks.workunit.client.1.smithi150.stdout:2/808: symlink d12/d21/ddc/lfa 0 2024-03-22T10:43:02.724 INFO:tasks.workunit.client.1.smithi150.stdout:5/647: truncate d0/d8/dc/de/d14/d2e/d45/f2a 106044 0 2024-03-22T10:43:02.725 INFO:tasks.workunit.client.1.smithi150.stdout:8/740: creat d0/d2/d46/ff1 x:0 0 0 2024-03-22T10:43:02.727 INFO:tasks.workunit.client.1.smithi150.stdout:7/687: rename d2/d5/d7/d9/db/d9f/d45/c9e to d2/d5/d7/d8/d46/ce5 0 2024-03-22T10:43:02.727 INFO:tasks.workunit.client.1.smithi150.stdout:2/809: readlink d12/d19/db6/da9/d61/la2 0 2024-03-22T10:43:02.727 INFO:tasks.workunit.client.1.smithi150.stdout:3/659: mkdir d5/da/d62/dba/dd5 0 2024-03-22T10:43:02.728 INFO:tasks.workunit.client.1.smithi150.stdout:5/648: stat d0/d8/dc/de/d14/d2e/d45/dac/faf 0 2024-03-22T10:43:02.730 INFO:tasks.workunit.client.1.smithi150.stdout:2/810: creat d12/da1/ffb x:0 0 0 2024-03-22T10:43:02.732 INFO:tasks.workunit.client.1.smithi150.stdout:5/649: symlink d0/d8/dc/de/d14/d2e/d45/ld0 0 2024-03-22T10:43:02.734 INFO:tasks.workunit.client.1.smithi150.stdout:2/811: mkdir d12/d19/db6/da9/d97/d78/dbd/dfc 0 2024-03-22T10:43:02.738 INFO:tasks.workunit.client.1.smithi150.stdout:5/650: chown d0/d8/dc/de/d14/d2e/d5a/f8e 22143 1 2024-03-22T10:43:02.742 INFO:tasks.workunit.client.1.smithi150.stdout:5/651: creat d0/d8/dc/de/d14/d4f/fd1 x:0 0 0 2024-03-22T10:43:02.746 INFO:tasks.workunit.client.1.smithi150.stdout:8/741: dwrite d0/d2/f14 [4194304,4194304] 0 2024-03-22T10:43:02.746 INFO:tasks.workunit.client.1.smithi150.stdout:3/660: dwrite d5/d22/d2f/f41 [0,4194304] 0 2024-03-22T10:43:02.749 INFO:tasks.workunit.client.1.smithi150.stdout:1/637: sync 2024-03-22T10:43:02.750 INFO:tasks.workunit.client.1.smithi150.stdout:4/704: sync 2024-03-22T10:43:02.753 INFO:tasks.workunit.client.1.smithi150.stdout:5/652: creat d0/d8/dc/de/d14/d2e/d46/d9c/dc0/fd2 x:0 0 0 2024-03-22T10:43:02.757 INFO:tasks.workunit.client.1.smithi150.stdout:4/705: stat d0/d2f/d63/c7a 0 2024-03-22T10:43:02.759 INFO:tasks.workunit.client.1.smithi150.stdout:7/688: truncate d2/d5/d7/d9/db/d9f/d96/fe4 1968835 0 2024-03-22T10:43:02.763 INFO:tasks.workunit.client.1.smithi150.stdout:4/706: symlink d0/d2f/d63/d7b/d9c/le6 0 2024-03-22T10:43:02.765 INFO:tasks.workunit.client.1.smithi150.stdout:4/707: chown d0/d2f/l34 21158360 1 2024-03-22T10:43:02.768 INFO:tasks.workunit.client.1.smithi150.stdout:1/638: rename d1/d19/d50/f69 to d1/d19/d1a/fcd 0 2024-03-22T10:43:02.775 INFO:tasks.workunit.client.1.smithi150.stdout:8/742: truncate d0/f5a 1024880 0 2024-03-22T10:43:02.776 INFO:tasks.workunit.client.1.smithi150.stdout:1/639: creat d1/da/d2a/fce x:0 0 0 2024-03-22T10:43:02.777 INFO:tasks.workunit.client.1.smithi150.stdout:3/661: write d5/d22/d2f/d38/d3e/d5b/f7f [2292880,32615] 0 2024-03-22T10:43:02.779 INFO:tasks.workunit.client.1.smithi150.stdout:5/653: rename d0/d8/dc/de/d14/d2e/d45/ld0 to d0/d8/dc/de/ld3 0 2024-03-22T10:43:02.784 INFO:tasks.workunit.client.1.smithi150.stdout:3/662: dread d5/d22/d2f/d38/fb4 [0,4194304] 0 2024-03-22T10:43:02.785 INFO:tasks.workunit.client.1.smithi150.stdout:7/689: rename d2/d5/d7/d9/l4c to d2/d5/d7/d9/db/da2/le6 0 2024-03-22T10:43:02.785 INFO:tasks.workunit.client.1.smithi150.stdout:4/708: truncate d0/d1b/f24 1773097 0 2024-03-22T10:43:02.787 INFO:tasks.workunit.client.1.smithi150.stdout:1/640: mkdir d1/d19/d50/d51/da0/dcf 0 2024-03-22T10:43:02.788 INFO:tasks.workunit.client.1.smithi150.stdout:7/690: fdatasync d2/d5/f7c 0 2024-03-22T10:43:02.791 INFO:tasks.workunit.client.1.smithi150.stdout:1/641: write d1/d19/fbc [383747,39434] 0 2024-03-22T10:43:02.791 INFO:tasks.workunit.client.1.smithi150.stdout:4/709: mkdir d0/d47/d72/de7 0 2024-03-22T10:43:02.794 INFO:tasks.workunit.client.1.smithi150.stdout:5/654: link d0/c4 d0/d37/cd4 0 2024-03-22T10:43:02.794 INFO:tasks.workunit.client.1.smithi150.stdout:4/710: chown d0/l10 610 1 2024-03-22T10:43:02.795 INFO:tasks.workunit.client.1.smithi150.stdout:8/743: dread d0/d2/d19/d29/d2e/d70/fb8 [0,4194304] 0 2024-03-22T10:43:02.795 INFO:tasks.workunit.client.1.smithi150.stdout:5/655: stat d0/d8/dc/de/d96 0 2024-03-22T10:43:02.796 INFO:tasks.workunit.client.1.smithi150.stdout:1/642: unlink d1/da/d2a/f7f 0 2024-03-22T10:43:02.796 INFO:tasks.workunit.client.1.smithi150.stdout:8/744: chown d0/d3c 59838 1 2024-03-22T10:43:02.796 INFO:tasks.workunit.client.1.smithi150.stdout:4/711: chown d0/c5c 325 1 2024-03-22T10:43:02.802 INFO:tasks.workunit.client.1.smithi150.stdout:8/745: symlink d0/d2/d19/d29/d88/lf2 0 2024-03-22T10:43:02.804 INFO:tasks.workunit.client.1.smithi150.stdout:4/712: link d0/d2f/fdb d0/d2f/fe8 0 2024-03-22T10:43:02.804 INFO:tasks.workunit.client.1.smithi150.stdout:4/713: chown d0/d47 22 1 2024-03-22T10:43:02.806 INFO:tasks.workunit.client.1.smithi150.stdout:5/656: rename d0/lb1 to d0/d8/dc/de/d14/d2e/d46/d4d/ld5 0 2024-03-22T10:43:02.810 INFO:tasks.workunit.client.1.smithi150.stdout:5/657: fsync d0/d8/dc/de/f15 0 2024-03-22T10:43:02.810 INFO:tasks.workunit.client.1.smithi150.stdout:4/714: symlink d0/d47/d72/de7/le9 0 2024-03-22T10:43:02.812 INFO:tasks.workunit.client.1.smithi150.stdout:3/663: dread d5/da/ff [0,4194304] 0 2024-03-22T10:43:02.813 INFO:tasks.workunit.client.1.smithi150.stdout:5/658: creat d0/d8/dc/d1d/d53/d63/fd6 x:0 0 0 2024-03-22T10:43:02.821 INFO:tasks.workunit.client.1.smithi150.stdout:1/643: dwrite d1/d19/d1a/d7d/f82 [4194304,4194304] 0 2024-03-22T10:43:02.890 INFO:tasks.workunit.client.1.smithi150.stdout:7/691: dread d2/d5/d7/f2b [0,4194304] 0 2024-03-22T10:43:02.890 INFO:tasks.workunit.client.1.smithi150.stdout:5/659: getdents d0/d8/dc/de/d14/d2e/dbc 0 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:4/715: dread d0/d1b/f39 [0,4194304] 0 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:7/692: rmdir d2/d5/d7/dab 39 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:4/716: read d0/d2f/d63/d7b/d9c/fa6 [318984,25272] 0 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:5/660: mkdir d0/d8/dc/de/d14/d2e/d45/dac/dd7 0 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:4/717: chown d0/d1b/d4d/d6c/d61/fa8 19135638 1 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:4/718: mknod d0/d47/d72/cea 0 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:4/719: write d0/d2f/f50 [299116,67295] 0 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:5/661: mknod d0/d8/dc/de/d14/d2e/d46/d9c/dc0/cd8 0 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:7/693: fdatasync d2/d5/d7/d9/db/d9f/f8b 0 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:4/720: readlink d0/d2f/d63/d7b/l87 0 2024-03-22T10:43:02.891 INFO:tasks.workunit.client.1.smithi150.stdout:3/664: dread d5/d53/f54 [0,4194304] 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:5/662: creat d0/d8/dc/de/d96/fd9 x:0 0 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:5/663: symlink d0/d8/dc/de/d96/db8/lda 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:5/664: dread - d0/dae/db2/f90 zero size 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:7/694: link d2/d5/d7/d8/d57/d4a/d8d/da9/cc4 d2/d5/d7/dab/ce7 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:5/665: fdatasync d0/d8/f34 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:3/665: getdents d5/d22/d37 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:4/721: dwrite d0/d2f/f50 [0,4194304] 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:4/722: fdatasync d0/d2f/fdb 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:5/666: rename d0/d8/dc/de/d14/d2e/d46/d9c/dc0/fd2 to d0/d8/dc/de/d14/d2e/d5a/d66/fdb 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:4/723: write d0/d1b/d4d/d6c/d61/d71/fd0 [25245,77403] 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:4/724: chown d0/d2f/d63/d7b/l83 70564703 1 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:7/695: dwrite d2/d5/f7c [0,4194304] 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:5/667: stat d0/d8/dc/de/d14/d2e/d5a/d66/da9/ccb 0 2024-03-22T10:43:02.892 INFO:tasks.workunit.client.1.smithi150.stdout:4/725: creat d0/d2f/d63/d7b/d9c/db1/dd7/feb x:0 0 0 2024-03-22T10:43:02.893 INFO:tasks.workunit.client.1.smithi150.stdout:7/696: symlink d2/d5/d7/d88/le8 0 2024-03-22T10:43:02.893 INFO:tasks.workunit.client.1.smithi150.stdout:4/726: truncate d0/d1b/d4d/d9f/fb5 344302 0 2024-03-22T10:43:02.895 INFO:tasks.workunit.client.1.smithi150.stdout:7/697: rmdir d2/d5/d7/d8/d57/d54/d91/dad 39 2024-03-22T10:43:02.897 INFO:tasks.workunit.client.1.smithi150.stdout:4/727: creat d0/d1b/d4d/d6c/d61/dd1/fec x:0 0 0 2024-03-22T10:43:02.901 INFO:tasks.workunit.client.1.smithi150.stdout:7/698: mknod d2/d5/d7/d8/d57/d54/d91/daf/ce9 0 2024-03-22T10:43:02.902 INFO:tasks.workunit.client.1.smithi150.stdout:4/728: creat d0/d2f/d63/d7b/d9c/fed x:0 0 0 2024-03-22T10:43:02.904 INFO:tasks.workunit.client.1.smithi150.stdout:4/729: write d0/d2f/d63/d7b/d9c/db1/dd7/fd9 [183883,9452] 0 2024-03-22T10:43:02.910 INFO:tasks.workunit.client.1.smithi150.stdout:7/699: dread d2/d5/d7/d8/d57/d54/d91/fb3 [0,4194304] 0 2024-03-22T10:43:02.912 INFO:tasks.workunit.client.1.smithi150.stdout:7/700: chown d2/d5/d7/d88/fdb 2 1 2024-03-22T10:43:02.920 INFO:tasks.workunit.client.1.smithi150.stdout:7/701: write d2/d5/d7/fd0 [854760,87948] 0 2024-03-22T10:43:02.921 INFO:tasks.workunit.client.1.smithi150.stdout:5/668: dread d0/d8/dc/de/d14/d4f/d47/fa0 [0,4194304] 0 2024-03-22T10:43:02.923 INFO:tasks.workunit.client.1.smithi150.stdout:6/670: sync 2024-03-22T10:43:02.923 INFO:tasks.workunit.client.1.smithi150.stdout:0/707: sync 2024-03-22T10:43:02.923 INFO:tasks.workunit.client.1.smithi150.stdout:9/767: sync 2024-03-22T10:43:02.924 INFO:tasks.workunit.client.1.smithi150.stdout:7/702: creat d2/d5/d7/d9/db/da2/fea x:0 0 0 2024-03-22T10:43:02.927 INFO:tasks.workunit.client.1.smithi150.stdout:9/768: chown d1/ca4 1572278939 1 2024-03-22T10:43:02.927 INFO:tasks.workunit.client.1.smithi150.stdout:6/671: rmdir db/d52 39 2024-03-22T10:43:02.933 INFO:tasks.workunit.client.1.smithi150.stdout:4/730: dread d0/d1b/f2e [0,4194304] 0 2024-03-22T10:43:02.936 INFO:tasks.workunit.client.1.smithi150.stdout:4/731: chown d0/d1b/d4d/d6c/c5d 3444177 1 2024-03-22T10:43:02.936 INFO:tasks.workunit.client.1.smithi150.stdout:5/669: rename d0/d8/dc/de/d14/d2e/d46/d4d/dad to d0/ddc 0 2024-03-22T10:43:02.939 INFO:tasks.workunit.client.1.smithi150.stdout:7/703: dread d2/d5/d7/d9/db/d9f/d45/fe3 [0,4194304] 0 2024-03-22T10:43:02.941 INFO:tasks.workunit.client.1.smithi150.stdout:7/704: chown d2/d5/d7/d8 29430 1 2024-03-22T10:43:02.943 INFO:tasks.workunit.client.1.smithi150.stdout:0/708: dwrite d0/d1/d84/d86/d11/d66/d35/d5f/fd9 [0,4194304] 0 2024-03-22T10:43:02.945 INFO:tasks.workunit.client.1.smithi150.stdout:0/709: dread - d0/d1/d84/d86/d42/d10/d38/d79/fbe zero size 2024-03-22T10:43:02.946 INFO:tasks.workunit.client.1.smithi150.stdout:0/710: stat d0/d12/c3f 0 2024-03-22T10:43:02.947 INFO:tasks.workunit.client.1.smithi150.stdout:6/672: creat db/dd/d15/d60/d50/dc1/fec x:0 0 0 2024-03-22T10:43:02.954 INFO:tasks.workunit.client.1.smithi150.stdout:4/732: rmdir d0/d2f 39 2024-03-22T10:43:02.954 INFO:tasks.workunit.client.1.smithi150.stdout:5/670: creat d0/ddc/fdd x:0 0 0 2024-03-22T10:43:02.957 INFO:tasks.workunit.client.1.smithi150.stdout:9/769: dwrite d1/d8/de/df/f1f [0,4194304] 0 2024-03-22T10:43:02.960 INFO:tasks.workunit.client.1.smithi150.stdout:5/671: dread - d0/d8/dc/de/d14/d4f/fd1 zero size 2024-03-22T10:43:02.960 INFO:tasks.workunit.client.1.smithi150.stdout:4/733: write d0/d2f/f76 [4057681,53068] 0 2024-03-22T10:43:02.965 INFO:tasks.workunit.client.1.smithi150.stdout:0/711: dread d0/d1/d84/d86/d42/f2b [0,4194304] 0 2024-03-22T10:43:02.965 INFO:tasks.workunit.client.1.smithi150.stdout:7/705: mknod d2/d5/d7/d8/d57/d54/d91/dad/ceb 0 2024-03-22T10:43:02.969 INFO:tasks.workunit.client.1.smithi150.stdout:9/770: creat d1/d49/f109 x:0 0 0 2024-03-22T10:43:02.971 INFO:tasks.workunit.client.1.smithi150.stdout:5/672: chown d0/d8/dc/de/d14/d2e/d46/d4d/ld5 11 1 2024-03-22T10:43:02.971 INFO:tasks.workunit.client.1.smithi150.stdout:6/673: getdents db/dd/d2e 0 2024-03-22T10:43:02.973 INFO:tasks.workunit.client.1.smithi150.stdout:7/706: mkdir d2/d5/d7/dab/dec 0 2024-03-22T10:43:02.975 INFO:tasks.workunit.client.1.smithi150.stdout:4/734: creat d0/d1b/d68/de0/fee x:0 0 0 2024-03-22T10:43:02.975 INFO:tasks.workunit.client.1.smithi150.stdout:5/673: stat d0/d8/dc/l4b 0 2024-03-22T10:43:02.975 INFO:tasks.workunit.client.1.smithi150.stdout:6/674: dread - db/dd/d15/f7b zero size 2024-03-22T10:43:02.976 INFO:tasks.workunit.client.1.smithi150.stdout:9/771: getdents d1/d19/d7d/dc1/dca/dd7/df0 0 2024-03-22T10:43:02.978 INFO:tasks.workunit.client.1.smithi150.stdout:6/675: chown db/dd/d15/d60/fcd 904713193 1 2024-03-22T10:43:02.979 INFO:tasks.workunit.client.1.smithi150.stdout:0/712: link d0/d1/d84/d86/d11/d66/d35/d5f/fdf d0/d1/d84/d86/d11/d66/d35/fe2 0 2024-03-22T10:43:02.979 INFO:tasks.workunit.client.1.smithi150.stdout:4/735: truncate d0/d32/d7c/dc1/fcc 782444 0 2024-03-22T10:43:02.981 INFO:tasks.workunit.client.1.smithi150.stdout:5/674: dread - d0/d8/dc/de/d14/d2e/d45/dac/fbd zero size 2024-03-22T10:43:02.989 INFO:tasks.workunit.client.1.smithi150.stdout:0/713: creat d0/d1/d84/d86/d42/fe3 x:0 0 0 2024-03-22T10:43:02.990 INFO:tasks.workunit.client.1.smithi150.stdout:5/675: mknod d0/d8/dc/cde 0 2024-03-22T10:43:02.991 INFO:tasks.workunit.client.1.smithi150.stdout:7/707: dread d2/d5/f7c [0,4194304] 0 2024-03-22T10:43:02.999 INFO:tasks.workunit.client.1.smithi150.stdout:0/714: dread d0/fc7 [0,4194304] 0 2024-03-22T10:43:03.000 INFO:tasks.workunit.client.1.smithi150.stdout:6/676: dwrite db/dd/d15/d6a/f6b [0,4194304] 0 2024-03-22T10:43:03.007 INFO:tasks.workunit.client.1.smithi150.stdout:9/772: dread d1/d8/de/df/f5b [0,4194304] 0 2024-03-22T10:43:03.024 INFO:tasks.workunit.client.1.smithi150.stdout:0/715: dread d0/d1/d84/d86/d11/d66/d35/fde [0,4194304] 0 2024-03-22T10:43:03.026 INFO:tasks.workunit.client.1.smithi150.stdout:0/716: chown d0/d1/d84/d86/d42/d10/d38/d52/f65 1291724 1 2024-03-22T10:43:03.064 INFO:tasks.workunit.client.1.smithi150.stdout:5/676: link d0/d8/dc/de/d96/fd9 d0/d37/fdf 0 2024-03-22T10:43:03.069 INFO:tasks.workunit.client.1.smithi150.stdout:9/773: creat d1/d19/d7d/dc1/dca/dd7/df0/f10a x:0 0 0 2024-03-22T10:43:03.070 INFO:tasks.workunit.client.1.smithi150.stdout:6/677: symlink db/dd/d82/led 0 2024-03-22T10:43:03.071 INFO:tasks.workunit.client.1.smithi150.stdout:0/717: symlink d0/le4 0 2024-03-22T10:43:03.072 INFO:tasks.workunit.client.1.smithi150.stdout:9/774: chown d1/d8/de/d73/d101/d29/fb1 396943276 1 2024-03-22T10:43:03.075 INFO:tasks.workunit.client.1.smithi150.stdout:7/708: write d2/d5/d7/d8/d57/d54/d91/fa6 [133417,117301] 0 2024-03-22T10:43:03.078 INFO:tasks.workunit.client.1.smithi150.stdout:9/775: symlink d1/d19/d7d/dc1/dca/l10b 0 2024-03-22T10:43:03.080 INFO:tasks.workunit.client.1.smithi150.stdout:7/709: mknod d2/d5/d7/d38/ced 0 2024-03-22T10:43:03.081 INFO:tasks.workunit.client.1.smithi150.stdout:9/776: readlink d1/d8/de/d73/d101/l22 0 2024-03-22T10:43:03.083 INFO:tasks.workunit.client.1.smithi150.stdout:5/677: dwrite d0/f3 [0,4194304] 0 2024-03-22T10:43:03.083 INFO:tasks.workunit.client.1.smithi150.stdout:7/710: mknod d2/d5/d7/d9/cee 0 2024-03-22T10:43:03.084 INFO:tasks.workunit.client.1.smithi150.stdout:6/678: write db/dd/d15/d28/d6c/fa2 [563578,86375] 0 2024-03-22T10:43:03.085 INFO:tasks.workunit.client.1.smithi150.stdout:9/777: write d1/d8/de/d73/d101/d29/fb1 [807703,31505] 0 2024-03-22T10:43:03.086 INFO:tasks.workunit.client.1.smithi150.stdout:5/678: chown d0/d8/dc/d64/f7b 10720663 1 2024-03-22T10:43:03.087 INFO:tasks.workunit.client.1.smithi150.stdout:0/718: rename d0/d12/c2c to d0/d1/d84/d86/d42/ce5 0 2024-03-22T10:43:03.091 INFO:tasks.workunit.client.1.smithi150.stdout:7/711: mknod d2/d5/d7/cef 0 2024-03-22T10:43:03.094 INFO:tasks.workunit.client.1.smithi150.stdout:5/679: truncate d0/d8/dc/de/d14/d4f/fd1 420027 0 2024-03-22T10:43:03.094 INFO:tasks.workunit.client.1.smithi150.stdout:0/719: mknod d0/d1/d84/d86/d42/d10/ce6 0 2024-03-22T10:43:03.101 INFO:tasks.workunit.client.1.smithi150.stdout:5/680: rename d0/d8/dc/d64 to d0/dae/de0 0 2024-03-22T10:43:03.102 INFO:tasks.workunit.client.1.smithi150.stdout:0/720: dread d0/d1/d84/d86/d11/d66/d37/fd1 [0,4194304] 0 2024-03-22T10:43:03.103 INFO:tasks.workunit.client.1.smithi150.stdout:9/778: dread d1/d8/de/df/d36/d40/d41/fac [0,4194304] 0 2024-03-22T10:43:03.106 INFO:tasks.workunit.client.1.smithi150.stdout:6/679: dwrite db/dd/d25/f34 [4194304,4194304] 0 2024-03-22T10:43:03.108 INFO:tasks.workunit.client.1.smithi150.stdout:7/712: link d2/d5/d7/d38/f4b d2/d5/d7/d8/d57/d4a/d8d/ff0 0 2024-03-22T10:43:03.109 INFO:tasks.workunit.client.1.smithi150.stdout:9/779: chown d1/d8/de/df/d36/d40/c46 894075175 1 2024-03-22T10:43:03.110 INFO:tasks.workunit.client.1.smithi150.stdout:5/681: unlink d0/d8/dc/de/d14/d4f/d47/l59 0 2024-03-22T10:43:03.110 INFO:tasks.workunit.client.1.smithi150.stdout:9/780: stat d1/d8 0 2024-03-22T10:43:03.113 INFO:tasks.workunit.client.1.smithi150.stdout:9/781: chown d1/d8/de/d73/d101/l5 1 1 2024-03-22T10:43:03.113 INFO:tasks.workunit.client.1.smithi150.stdout:0/721: symlink d0/d1/d84/d86/d42/le7 0 2024-03-22T10:43:03.115 INFO:tasks.workunit.client.1.smithi150.stdout:6/680: symlink db/dd/d15/d6a/dde/lee 0 2024-03-22T10:43:03.116 INFO:tasks.workunit.client.1.smithi150.stdout:5/682: creat d0/d8/dc/de/d96/db8/fe1 x:0 0 0 2024-03-22T10:43:03.116 INFO:tasks.workunit.client.1.smithi150.stdout:7/713: mknod d2/d5/d7/cf1 0 2024-03-22T10:43:03.117 INFO:tasks.workunit.client.1.smithi150.stdout:9/782: mknod d1/d8/de/d73/d101/c10c 0 2024-03-22T10:43:03.118 INFO:tasks.workunit.client.1.smithi150.stdout:0/722: dread - d0/d1/fc1 zero size 2024-03-22T10:43:03.119 INFO:tasks.workunit.client.1.smithi150.stdout:5/683: chown d0/d8/dc/d1d/d53/d63/f9d 1006 1 2024-03-22T10:43:03.124 INFO:tasks.workunit.client.1.smithi150.stdout:6/681: rename db/cbf to db/d52/cef 0 2024-03-22T10:43:03.124 INFO:tasks.workunit.client.1.smithi150.stdout:5/684: mknod d0/d8/dc/de/d14/d2e/d45/db3/ce2 0 2024-03-22T10:43:03.129 INFO:tasks.workunit.client.1.smithi150.stdout:0/723: truncate d0/d1/d84/d86/d42/d10/d38/d52/d68/f80 2219309 0 2024-03-22T10:43:03.129 INFO:tasks.workunit.client.1.smithi150.stdout:9/783: rename l0 to d1/d8/de/df/d36/d40/d41/l10d 0 2024-03-22T10:43:03.132 INFO:tasks.workunit.client.1.smithi150.stdout:0/724: rename d0/d1 to d0/d1/d84/d86/d42/d10/d38/d52/d9b/db4/de8 22 2024-03-22T10:43:03.134 INFO:tasks.workunit.client.1.smithi150.stdout:9/784: creat d1/d8/de/dbd/df8/f10e x:0 0 0 2024-03-22T10:43:03.135 INFO:tasks.workunit.client.1.smithi150.stdout:0/725: write d0/d1/d84/d86/d42/d10/d38/f4c [193336,22838] 0 2024-03-22T10:43:03.138 INFO:tasks.workunit.client.1.smithi150.stdout:0/726: chown d0/d1/d84/d86/l9c 205952483 1 2024-03-22T10:43:03.140 INFO:tasks.workunit.client.1.smithi150.stdout:9/785: readlink d1/d8/de/df/d36/d40/le2 0 2024-03-22T10:43:03.143 INFO:tasks.workunit.client.1.smithi150.stdout:7/714: write d2/d5/d7/d8/d46/f5a [1130646,106378] 0 2024-03-22T10:43:03.148 INFO:tasks.workunit.client.1.smithi150.stdout:5/685: dwrite d0/d8/dc/de/d14/d2e/d46/d9c/dc0/fcf [0,4194304] 0 2024-03-22T10:43:03.149 INFO:tasks.workunit.client.1.smithi150.stdout:2/812: sync 2024-03-22T10:43:03.149 INFO:tasks.workunit.client.1.smithi150.stdout:8/746: sync 2024-03-22T10:43:03.150 INFO:tasks.workunit.client.1.smithi150.stdout:6/682: dwrite db/dd/d82/d84/fb5 [0,4194304] 0 2024-03-22T10:43:03.153 INFO:tasks.workunit.client.1.smithi150.stdout:7/715: creat d2/d5/d7/d9/db/d9f/d45/ff2 x:0 0 0 2024-03-22T10:43:03.154 INFO:tasks.workunit.client.1.smithi150.stdout:9/786: rmdir d1/d19/d7d/dc1/dca/dd7/d100 0 2024-03-22T10:43:03.160 INFO:tasks.workunit.client.1.smithi150.stdout:9/787: readlink d1/d8/de/d73/d101/led 0 2024-03-22T10:43:03.160 INFO:tasks.workunit.client.1.smithi150.stdout:8/747: truncate d0/d3c/fed 428323 0 2024-03-22T10:43:03.160 INFO:tasks.workunit.client.1.smithi150.stdout:7/716: dread - d2/d5/d7/d8/d57/d54/d91/dad/fcf zero size 2024-03-22T10:43:03.168 INFO:tasks.workunit.client.1.smithi150.stdout:0/727: dwrite d0/d12/fd8 [0,4194304] 0 2024-03-22T10:43:03.170 INFO:tasks.workunit.client.1.smithi150.stdout:2/813: rename d12/d19/d1a/ld2 to d12/d21/d81/lfd 0 2024-03-22T10:43:03.174 INFO:tasks.workunit.client.1.smithi150.stdout:7/717: creat d2/d5/d7/d9/db/d6c/d9c/ff3 x:0 0 0 2024-03-22T10:43:03.175 INFO:tasks.workunit.client.1.smithi150.stdout:5/686: link d0/d8/dc/de/d14/d2e/d46/l94 d0/d8/dc/de/d14/d2e/d5a/d66/le3 0 2024-03-22T10:43:03.177 INFO:tasks.workunit.client.1.smithi150.stdout:7/718: read - d2/d5/d7/fc1 zero size 2024-03-22T10:43:03.179 INFO:tasks.workunit.client.1.smithi150.stdout:6/683: dwrite db/dd/d15/d60/fdb [0,4194304] 0 2024-03-22T10:43:03.183 INFO:tasks.workunit.client.1.smithi150.stdout:2/814: write d12/d19/db6/da9/d2c/d65/f9d [2505868,114817] 0 2024-03-22T10:43:03.183 INFO:tasks.workunit.client.1.smithi150.stdout:8/748: getdents d0/ddd 0 2024-03-22T10:43:03.184 INFO:tasks.workunit.client.1.smithi150.stdout:5/687: truncate d0/d8/dc/d1d/d53/d63/fd6 355764 0 2024-03-22T10:43:03.185 INFO:tasks.workunit.client.1.smithi150.stdout:7/719: rmdir d2/d5/d7/d8/d57/d54 39 2024-03-22T10:43:03.192 INFO:tasks.workunit.client.1.smithi150.stdout:7/720: write d2/d5/d7/d9/db/d6c/fd6 [319158,61936] 0 2024-03-22T10:43:03.194 INFO:tasks.workunit.client.1.smithi150.stdout:8/749: mkdir d0/ddd/df3 0 2024-03-22T10:43:03.195 INFO:tasks.workunit.client.1.smithi150.stdout:0/728: link d0/f85 d0/d1/d84/fe9 0 2024-03-22T10:43:03.195 INFO:tasks.workunit.client.1.smithi150.stdout:7/721: chown d2/d5/d7/d8/d57/d4a/l20 4262 1 2024-03-22T10:43:03.199 INFO:tasks.workunit.client.1.smithi150.stdout:8/750: stat d0/d2/d46/le8 0 2024-03-22T10:43:03.201 INFO:tasks.workunit.client.1.smithi150.stdout:7/722: symlink d2/d5/d7/d38/lf4 0 2024-03-22T10:43:03.205 INFO:tasks.workunit.client.1.smithi150.stdout:8/751: chown d0/d2/d19/l98 301500 1 2024-03-22T10:43:03.209 INFO:tasks.workunit.client.1.smithi150.stdout:2/815: dwrite d12/d19/d2d/f36 [0,4194304] 0 2024-03-22T10:43:03.212 INFO:tasks.workunit.client.1.smithi150.stdout:0/729: rename d0/d1/d84/d86/d42/d10/d38/d52/c5b to d0/d1/d84/d86/d42/d10/d38/cea 0 2024-03-22T10:43:03.215 INFO:tasks.workunit.client.1.smithi150.stdout:8/752: creat d0/d2/d19/d29/d37/d7d/dc1/ff4 x:0 0 0 2024-03-22T10:43:03.216 INFO:tasks.workunit.client.1.smithi150.stdout:2/816: creat d12/d19/db6/da9/d67/d8d/da6/ffe x:0 0 0 2024-03-22T10:43:03.217 INFO:tasks.workunit.client.1.smithi150.stdout:6/684: dwrite db/dd/d25/f76 [0,4194304] 0 2024-03-22T10:43:03.221 INFO:tasks.workunit.client.1.smithi150.stdout:2/817: symlink d12/d35/dc9/lff 0 2024-03-22T10:43:03.226 INFO:tasks.workunit.client.1.smithi150.stdout:6/685: creat db/dd/d15/d28/ff0 x:0 0 0 2024-03-22T10:43:03.226 INFO:tasks.workunit.client.1.smithi150.stdout:7/723: dwrite d2/d5/d7/d8/d57/d4a/f63 [0,4194304] 0 2024-03-22T10:43:03.227 INFO:tasks.workunit.client.1.smithi150.stdout:8/753: symlink d0/d2/d19/d29/d37/dc6/lf5 0 2024-03-22T10:43:03.227 INFO:tasks.workunit.client.1.smithi150.stdout:6/686: readlink db/dd/d15/d30/lb7 0 2024-03-22T10:43:03.227 INFO:tasks.workunit.client.1.smithi150.stdout:2/818: mkdir d12/d19/d100 0 2024-03-22T10:43:03.228 INFO:tasks.workunit.client.1.smithi150.stdout:7/724: write d2/fcd [634162,5619] 0 2024-03-22T10:43:03.230 INFO:tasks.workunit.client.1.smithi150.stdout:2/819: chown d12/d35 3585 1 2024-03-22T10:43:03.231 INFO:tasks.workunit.client.1.smithi150.stdout:7/725: chown d2/d5/d7/dab 6860 1 2024-03-22T10:43:03.233 INFO:tasks.workunit.client.1.smithi150.stdout:7/726: chown d2/d5/d7/d9/db/d9f 1 1 2024-03-22T10:43:03.237 INFO:tasks.workunit.client.1.smithi150.stdout:0/730: dwrite d0/d1/d84/d86/d42/d10/d38/d52/f54 [4194304,4194304] 0 2024-03-22T10:43:03.240 INFO:tasks.workunit.client.1.smithi150.stdout:6/687: rename db/dd/d2e to db/d52/d9f/dc6/df1 0 2024-03-22T10:43:03.241 INFO:tasks.workunit.client.1.smithi150.stdout:5/688: dread d0/d8/dc/de/d14/d4f/f3d [0,4194304] 0 2024-03-22T10:43:03.242 INFO:tasks.workunit.client.1.smithi150.stdout:0/731: creat d0/d1/d84/d86/d42/d10/d38/d52/d9b/dac/feb x:0 0 0 2024-03-22T10:43:03.242 INFO:tasks.workunit.client.1.smithi150.stdout:5/689: write d0/f38 [890054,101819] 0 2024-03-22T10:43:03.244 INFO:tasks.workunit.client.1.smithi150.stdout:5/690: chown d0/d8/dc/l71 30461 1 2024-03-22T10:43:03.246 INFO:tasks.workunit.client.1.smithi150.stdout:0/732: fdatasync d0/d1/d84/d86/f31 0 2024-03-22T10:43:03.249 INFO:tasks.workunit.client.1.smithi150.stdout:5/691: dread - d0/d8/dc/de/d96/db8/fe1 zero size 2024-03-22T10:43:03.296 INFO:tasks.workunit.client.1.smithi150.stdout:4/736: sync 2024-03-22T10:43:03.296 INFO:tasks.workunit.client.1.smithi150.stdout:3/666: sync 2024-03-22T10:43:03.296 INFO:tasks.workunit.client.1.smithi150.stdout:1/644: sync 2024-03-22T10:43:03.313 INFO:tasks.workunit.client.1.smithi150.stdout:8/754: write d0/f1 [1462770,46448] 0 2024-03-22T10:43:03.313 INFO:tasks.workunit.client.1.smithi150.stdout:3/667: write d5/fd [87844,68127] 0 2024-03-22T10:43:03.314 INFO:tasks.workunit.client.1.smithi150.stdout:5/692: mknod d0/d37/ce4 0 2024-03-22T10:43:03.316 INFO:tasks.workunit.client.1.smithi150.stdout:5/693: dread - d0/d8/f9a zero size 2024-03-22T10:43:03.318 INFO:tasks.workunit.client.1.smithi150.stdout:1/645: mkdir d1/da/dd0 0 2024-03-22T10:43:03.319 INFO:tasks.workunit.client.1.smithi150.stdout:2/820: write d12/d19/db6/da9/d2c/d65/f9d [2368015,74315] 0 2024-03-22T10:43:03.319 INFO:tasks.workunit.client.1.smithi150.stdout:4/737: creat d0/d1b/d4d/d9f/fef x:0 0 0 2024-03-22T10:43:03.324 INFO:tasks.workunit.client.1.smithi150.stdout:1/646: write d1/d19/d50/f66 [1802536,78921] 0 2024-03-22T10:43:03.325 INFO:tasks.workunit.client.1.smithi150.stdout:3/668: mknod d5/da/dad/cd6 0 2024-03-22T10:43:03.326 INFO:tasks.workunit.client.1.smithi150.stdout:6/688: write db/dd/d15/d28/d6c/fa2 [158285,79928] 0 2024-03-22T10:43:03.327 INFO:tasks.workunit.client.1.smithi150.stdout:1/647: chown d1/d19/d50/dc7 32020534 1 2024-03-22T10:43:03.328 INFO:tasks.workunit.client.1.smithi150.stdout:5/694: creat d0/d8/dc/de/d14/d2e/d5a/d66/da9/fe5 x:0 0 0 2024-03-22T10:43:03.329 INFO:tasks.workunit.client.1.smithi150.stdout:2/821: truncate d12/d19/db6/d54/f5b 378801 0 2024-03-22T10:43:03.330 INFO:tasks.workunit.client.1.smithi150.stdout:0/733: write d0/d1/d84/d86/d11/d66/d37/fd1 [44813,11706] 0 2024-03-22T10:43:03.333 INFO:tasks.workunit.client.1.smithi150.stdout:8/755: rmdir d0/ddd/df3 0 2024-03-22T10:43:03.339 INFO:tasks.workunit.client.1.smithi150.stdout:3/669: creat d5/d22/d37/fd7 x:0 0 0 2024-03-22T10:43:03.339 INFO:tasks.workunit.client.1.smithi150.stdout:0/734: read d0/d1/d84/d86/d42/fc [1315499,38598] 0 2024-03-22T10:43:03.340 INFO:tasks.workunit.client.1.smithi150.stdout:6/689: symlink db/dd/d15/d30/ddf/lf2 0 2024-03-22T10:43:03.340 INFO:tasks.workunit.client.1.smithi150.stdout:1/648: dread d1/d19/d50/f90 [0,4194304] 0 2024-03-22T10:43:03.346 INFO:tasks.workunit.client.1.smithi150.stdout:8/756: creat d0/d2/d19/d29/d37/d7d/dc1/ff6 x:0 0 0 2024-03-22T10:43:03.347 INFO:tasks.workunit.client.1.smithi150.stdout:3/670: creat d5/d22/d37/fd8 x:0 0 0 2024-03-22T10:43:03.347 INFO:tasks.workunit.client.1.smithi150.stdout:0/735: chown d0/d1/l4 15 1 2024-03-22T10:43:03.350 INFO:tasks.workunit.client.1.smithi150.stdout:6/690: creat db/d52/d79/ff3 x:0 0 0 2024-03-22T10:43:03.352 INFO:tasks.workunit.client.1.smithi150.stdout:8/757: readlink d0/d2/d19/d29/l43 0 2024-03-22T10:43:03.352 INFO:tasks.workunit.client.1.smithi150.stdout:4/738: dread d0/d2f/f76 [0,4194304] 0 2024-03-22T10:43:03.358 INFO:tasks.workunit.client.1.smithi150.stdout:5/695: dwrite d0/d8/dc/d1d/d53/d63/fd6 [0,4194304] 0 2024-03-22T10:43:03.359 INFO:tasks.workunit.client.1.smithi150.stdout:3/671: creat d5/daf/fd9 x:0 0 0 2024-03-22T10:43:03.360 INFO:tasks.workunit.client.1.smithi150.stdout:3/672: stat d5/d22/dc8 0 2024-03-22T10:43:03.373 INFO:tasks.workunit.client.1.smithi150.stdout:2/822: dwrite d12/d19/db6/da9/d67/f4e [0,4194304] 0 2024-03-22T10:43:03.374 INFO:tasks.workunit.client.1.smithi150.stdout:0/736: dread d0/d1/d84/d86/d42/d10/d38/d52/f54 [4194304,4194304] 0 2024-03-22T10:43:03.377 INFO:tasks.workunit.client.1.smithi150.stdout:1/649: dwrite d1/da/f16 [0,4194304] 0 2024-03-22T10:43:03.379 INFO:tasks.workunit.client.1.smithi150.stdout:0/737: chown d0/d1/d84/d86/d11/d66/d35/d92/c97 117253101 1 2024-03-22T10:43:03.385 INFO:tasks.workunit.client.1.smithi150.stdout:4/739: chown d0/de/f3c 60692 1 2024-03-22T10:43:03.387 INFO:tasks.workunit.client.1.smithi150.stdout:8/758: rename d0/d3c/d71/l6a to d0/d2/d19/dc0/dda/lf7 0 2024-03-22T10:43:03.391 INFO:tasks.workunit.client.1.smithi150.stdout:9/788: sync 2024-03-22T10:43:03.394 INFO:tasks.workunit.client.1.smithi150.stdout:9/789: chown d1/d8/de/df/fd3 7320 1 2024-03-22T10:43:03.394 INFO:tasks.workunit.client.1.smithi150.stdout:2/823: rename d12/l5e to d12/d19/db6/da9/d97/d78/dbd/l101 0 2024-03-22T10:43:03.395 INFO:tasks.workunit.client.1.smithi150.stdout:1/650: symlink d1/ld1 0 2024-03-22T10:43:03.399 INFO:tasks.workunit.client.1.smithi150.stdout:4/740: creat d0/d2f/d63/d7b/d9c/ff0 x:0 0 0 2024-03-22T10:43:03.401 INFO:tasks.workunit.client.1.smithi150.stdout:4/741: dread - d0/d2f/fdb zero size 2024-03-22T10:43:03.403 INFO:tasks.workunit.client.1.smithi150.stdout:9/790: symlink d1/d19/d7d/dc1/l10f 0 2024-03-22T10:43:03.403 INFO:tasks.workunit.client.1.smithi150.stdout:8/759: write d0/d2/d7e/d96/f99 [3978479,65796] 0 2024-03-22T10:43:03.404 INFO:tasks.workunit.client.1.smithi150.stdout:3/673: truncate d5/f46 1453930 0 2024-03-22T10:43:03.407 INFO:tasks.workunit.client.1.smithi150.stdout:6/691: getdents db/dd/d15/d70 0 2024-03-22T10:43:03.408 INFO:tasks.workunit.client.1.smithi150.stdout:2/824: symlink d12/d19/db6/da9/d2c/d65/l102 0 2024-03-22T10:43:03.409 INFO:tasks.workunit.client.1.smithi150.stdout:1/651: truncate d1/f1d 2629749 0 2024-03-22T10:43:03.409 INFO:tasks.workunit.client.1.smithi150.stdout:5/696: dwrite d0/d8/dc/de/d14/d2e/d45/dac/fbe [0,4194304] 0 2024-03-22T10:43:03.410 INFO:tasks.workunit.client.1.smithi150.stdout:6/692: fsync db/dd/d15/d60/fcd 0 2024-03-22T10:43:03.414 INFO:tasks.workunit.client.1.smithi150.stdout:8/760: truncate d0/d2/d46/ff1 726695 0 2024-03-22T10:43:03.417 INFO:tasks.workunit.client.1.smithi150.stdout:3/674: rename d5/d53/d7e to d5/d22/dc8/dda 0 2024-03-22T10:43:03.419 INFO:tasks.workunit.client.1.smithi150.stdout:9/791: unlink d1/d8/de/df/d36/d40/d41/l72 0 2024-03-22T10:43:03.422 INFO:tasks.workunit.client.1.smithi150.stdout:4/742: truncate d0/d1b/fac 668703 0 2024-03-22T10:43:03.423 INFO:tasks.workunit.client.1.smithi150.stdout:6/693: rmdir db/dd/d15/d60/d8b 39 2024-03-22T10:43:03.425 INFO:tasks.workunit.client.1.smithi150.stdout:2/825: truncate d12/d19/d1a/f7c 4697943 0 2024-03-22T10:43:03.431 INFO:tasks.workunit.client.1.smithi150.stdout:8/761: symlink d0/d2/d19/d29/d37/d7d/d9e/lf8 0 2024-03-22T10:43:03.434 INFO:tasks.workunit.client.1.smithi150.stdout:1/652: mknod d1/da/d2a/d55/db8/cd2 0 2024-03-22T10:43:03.438 INFO:tasks.workunit.client.1.smithi150.stdout:3/675: rename d5/da/d78/db7/ld3 to d5/d22/dc8/dda/d95/ldb 0 2024-03-22T10:43:03.440 INFO:tasks.workunit.client.1.smithi150.stdout:4/743: creat d0/d1b/d4d/d9f/ff1 x:0 0 0 2024-03-22T10:43:03.442 INFO:tasks.workunit.client.1.smithi150.stdout:0/738: write d0/d1/d84/fe9 [550853,23407] 0 2024-03-22T10:43:03.442 INFO:tasks.workunit.client.1.smithi150.stdout:7/727: sync 2024-03-22T10:43:03.443 INFO:tasks.workunit.client.1.smithi150.stdout:8/762: readlink d0/l22 0 2024-03-22T10:43:03.443 INFO:tasks.workunit.client.1.smithi150.stdout:6/694: dwrite db/d52/d9f/dc6/df1/fbb [0,4194304] 0 2024-03-22T10:43:03.445 INFO:tasks.workunit.client.1.smithi150.stdout:9/792: symlink d1/d8/de/df/l110 0 2024-03-22T10:43:03.449 INFO:tasks.workunit.client.1.smithi150.stdout:6/695: chown db/dd/d15/d28 390 1 2024-03-22T10:43:03.449 INFO:tasks.workunit.client.1.smithi150.stdout:1/653: creat d1/d19/d50/d6e/fd3 x:0 0 0 2024-03-22T10:43:03.455 INFO:tasks.workunit.client.1.smithi150.stdout:4/744: truncate d0/de/f3c 412733 0 2024-03-22T10:43:03.457 INFO:tasks.workunit.client.1.smithi150.stdout:7/728: creat d2/d5/d7/d8/d57/d4a/d8d/ff5 x:0 0 0 2024-03-22T10:43:03.457 INFO:tasks.workunit.client.1.smithi150.stdout:4/745: write d0/f26 [4079337,111835] 0 2024-03-22T10:43:03.459 INFO:tasks.workunit.client.1.smithi150.stdout:9/793: mkdir d1/d19/d7d/dc1/dca/d111 0 2024-03-22T10:43:03.462 INFO:tasks.workunit.client.1.smithi150.stdout:3/676: symlink d5/da/d9d/d9f/dc1/ldc 0 2024-03-22T10:43:03.463 INFO:tasks.workunit.client.1.smithi150.stdout:4/746: chown d0/d1b/d4d/d6c/d61/c85 2717 1 2024-03-22T10:43:03.465 INFO:tasks.workunit.client.1.smithi150.stdout:2/826: creat d12/d19/db6/da9/f103 x:0 0 0 2024-03-22T10:43:03.467 INFO:tasks.workunit.client.1.smithi150.stdout:6/696: symlink db/dd/d15/d60/d50/dc1/lf4 0 2024-03-22T10:43:03.467 INFO:tasks.workunit.client.1.smithi150.stdout:5/697: dwrite d0/d8/dc/d1d/d89/fc4 [0,4194304] 0 2024-03-22T10:43:03.470 INFO:tasks.workunit.client.1.smithi150.stdout:0/739: creat d0/d1/d84/d86/d42/d10/d38/dba/fec x:0 0 0 2024-03-22T10:43:03.472 INFO:tasks.workunit.client.1.smithi150.stdout:6/697: fdatasync db/dd/d15/d60/f53 0 2024-03-22T10:43:03.473 INFO:tasks.workunit.client.1.smithi150.stdout:0/740: chown d0/d1/d84/d86/d42/d10/d38/f4b 63046156 1 2024-03-22T10:43:03.476 INFO:tasks.workunit.client.1.smithi150.stdout:5/698: write d0/d8/dc/de/d14/d2e/d45/dac/faf [852186,129647] 0 2024-03-22T10:43:03.479 INFO:tasks.workunit.client.1.smithi150.stdout:3/677: mkdir d5/d22/d37/ddd 0 2024-03-22T10:43:03.486 INFO:tasks.workunit.client.1.smithi150.stdout:2/827: chown d12/d19/d2d/faa 2481516 1 2024-03-22T10:43:03.489 INFO:tasks.workunit.client.1.smithi150.stdout:7/729: mkdir d2/d5/d7/d8/d57/d4a/df6 0 2024-03-22T10:43:03.489 INFO:tasks.workunit.client.1.smithi150.stdout:9/794: dwrite d1/d8/de/d73/d101/d32/fbc [0,4194304] 0 2024-03-22T10:43:03.494 INFO:tasks.workunit.client.1.smithi150.stdout:6/698: creat db/dd/d15/d70/ff5 x:0 0 0 2024-03-22T10:43:03.496 INFO:tasks.workunit.client.1.smithi150.stdout:8/763: rename d0/d2/d19/d29/d88/f91 to d0/d2/d19/d29/d2f/ff9 0 2024-03-22T10:43:03.501 INFO:tasks.workunit.client.1.smithi150.stdout:3/678: symlink d5/d22/d2f/d38/d3e/d5b/lde 0 2024-03-22T10:43:03.506 INFO:tasks.workunit.client.1.smithi150.stdout:5/699: dwrite d0/d8/dc/d1d/d89/f9b [0,4194304] 0 2024-03-22T10:43:03.507 INFO:tasks.workunit.client.1.smithi150.stdout:2/828: read d12/d19/f1d [3577788,57388] 0 2024-03-22T10:43:03.515 INFO:tasks.workunit.client.1.smithi150.stdout:1/654: dwrite d1/da/d15/f74 [0,4194304] 0 2024-03-22T10:43:03.516 INFO:tasks.workunit.client.1.smithi150.stdout:3/679: unlink d5/ce 0 2024-03-22T10:43:03.517 INFO:tasks.workunit.client.1.smithi150.stdout:2/829: chown d12/d19/db6/da9/d2c/d65/la3 455 1 2024-03-22T10:43:03.523 INFO:tasks.workunit.client.1.smithi150.stdout:4/747: write d0/d1b/f23 [1837553,37639] 0 2024-03-22T10:43:03.527 INFO:tasks.workunit.client.1.smithi150.stdout:8/764: dwrite d0/f1 [0,4194304] 0 2024-03-22T10:43:03.528 INFO:tasks.workunit.client.1.smithi150.stdout:6/699: creat db/dd/d15/d28/d6c/d9b/ff6 x:0 0 0 2024-03-22T10:43:03.532 INFO:tasks.workunit.client.1.smithi150.stdout:1/655: rmdir d1/da/d2a/d65 39 2024-03-22T10:43:03.534 INFO:tasks.workunit.client.1.smithi150.stdout:7/730: dwrite d2/d5/f7c [0,4194304] 0 2024-03-22T10:43:03.538 INFO:tasks.workunit.client.1.smithi150.stdout:2/830: mkdir d12/d19/db6/da9/d67/db4/d104 0 2024-03-22T10:43:03.541 INFO:tasks.workunit.client.1.smithi150.stdout:9/795: creat d1/d49/d98/f112 x:0 0 0 2024-03-22T10:43:03.543 INFO:tasks.workunit.client.1.smithi150.stdout:5/700: symlink d0/d8/dc/de/d14/d2e/d45/dac/dd7/le6 0 2024-03-22T10:43:03.547 INFO:tasks.workunit.client.1.smithi150.stdout:4/748: creat d0/d2f/d63/d7b/d9c/db1/dd7/ff2 x:0 0 0 2024-03-22T10:43:03.550 INFO:tasks.workunit.client.1.smithi150.stdout:1/656: symlink d1/d79/ld4 0 2024-03-22T10:43:03.555 INFO:tasks.workunit.client.1.smithi150.stdout:9/796: rmdir d1/d8/de/df/d36/d40 39 2024-03-22T10:43:03.561 INFO:tasks.workunit.client.1.smithi150.stdout:0/741: dwrite d0/d12/f58 [0,4194304] 0 2024-03-22T10:43:03.561 INFO:tasks.workunit.client.1.smithi150.stdout:3/680: dwrite d5/d22/d37/fd7 [0,4194304] 0 2024-03-22T10:43:03.563 INFO:tasks.workunit.client.1.smithi150.stdout:7/731: mknod d2/d5/d7/d8/d57/d4a/d8d/da9/cf7 0 2024-03-22T10:43:03.572 INFO:tasks.workunit.client.1.smithi150.stdout:0/742: chown d0/l94 229405 1 2024-03-22T10:43:03.572 INFO:tasks.workunit.client.1.smithi150.stdout:7/732: mknod d2/d5/d7/d9/db/da2/cf8 0 2024-03-22T10:43:03.575 INFO:tasks.workunit.client.1.smithi150.stdout:2/831: dread d12/d19/d2d/f53 [0,4194304] 0 2024-03-22T10:43:03.582 INFO:tasks.workunit.client.1.smithi150.stdout:3/681: symlink d5/da/d62/ldf 0 2024-03-22T10:43:03.584 INFO:tasks.workunit.client.1.smithi150.stdout:2/832: write d12/d19/db6/d63/dd8/fe8 [940645,92732] 0 2024-03-22T10:43:03.585 INFO:tasks.workunit.client.1.smithi150.stdout:8/765: link d0/d2/d46/da2/lb2 d0/d2/d19/d29/d37/da1/lfa 0 2024-03-22T10:43:03.587 INFO:tasks.workunit.client.1.smithi150.stdout:8/766: stat d0/f34 0 2024-03-22T10:43:03.589 INFO:tasks.workunit.client.1.smithi150.stdout:5/701: dwrite d0/d8/dc/de/d14/d2e/d45/f78 [0,4194304] 0 2024-03-22T10:43:03.591 INFO:tasks.workunit.client.1.smithi150.stdout:3/682: unlink d5/da/c49 0 2024-03-22T10:43:03.595 INFO:tasks.workunit.client.1.smithi150.stdout:9/797: dwrite d1/d8/de/d73/d101/d29/fb1 [0,4194304] 0 2024-03-22T10:43:03.596 INFO:tasks.workunit.client.1.smithi150.stdout:7/733: truncate d2/d5/d7/d8/d57/d54/d91/dad/fcf 611375 0 2024-03-22T10:43:03.604 INFO:tasks.workunit.client.1.smithi150.stdout:4/749: write d0/d47/f7e [4165444,57297] 0 2024-03-22T10:43:03.606 INFO:tasks.workunit.client.1.smithi150.stdout:1/657: write d1/da/d15/d18/f3d [184499,42019] 0 2024-03-22T10:43:03.612 INFO:tasks.workunit.client.1.smithi150.stdout:1/658: readlink d1/d19/d1a/lad 0 2024-03-22T10:43:03.612 INFO:tasks.workunit.client.1.smithi150.stdout:3/683: dread d5/d22/d2f/d38/d59/d88/f8a [0,4194304] 0 2024-03-22T10:43:03.616 INFO:tasks.workunit.client.1.smithi150.stdout:7/734: symlink d2/d5/lf9 0 2024-03-22T10:43:03.618 INFO:tasks.workunit.client.1.smithi150.stdout:4/750: symlink d0/d32/d7c/dc1/lf3 0 2024-03-22T10:43:03.619 INFO:tasks.workunit.client.1.smithi150.stdout:8/767: link d0/d2/d19/d29/d2f/df/d1b/l60 d0/d2/d19/d29/d2e/d70/lfb 0 2024-03-22T10:43:03.621 INFO:tasks.workunit.client.1.smithi150.stdout:5/702: dwrite d0/f5 [0,4194304] 0 2024-03-22T10:43:03.622 INFO:tasks.workunit.client.1.smithi150.stdout:1/659: dread - d1/d19/d50/d51/f9c zero size 2024-03-22T10:43:03.622 INFO:tasks.workunit.client.1.smithi150.stdout:9/798: chown d1/d8/de/df/d36/d40/le2 1539038493 1 2024-03-22T10:43:03.622 INFO:tasks.workunit.client.1.smithi150.stdout:3/684: chown d5/d22/d2f/d38/d3e 97 1 2024-03-22T10:43:03.625 INFO:tasks.workunit.client.1.smithi150.stdout:7/735: creat d2/d5/d7/d9/db/d6c/da8/ffa x:0 0 0 2024-03-22T10:43:03.631 INFO:tasks.workunit.client.1.smithi150.stdout:8/768: unlink d0/c76 0 2024-03-22T10:43:03.637 INFO:tasks.workunit.client.1.smithi150.stdout:3/685: rename d5/da/d78/d86/f93 to d5/da/d62/dba/dd5/fe0 0 2024-03-22T10:43:03.639 INFO:tasks.workunit.client.1.smithi150.stdout:3/686: dread - d5/d22/fd2 zero size 2024-03-22T10:43:03.649 INFO:tasks.workunit.client.1.smithi150.stdout:9/799: rename d1/d8/de/df/ff7 to d1/d8/de/df/d36/d40/d44/db5/f113 0 2024-03-22T10:43:03.658 INFO:tasks.workunit.client.1.smithi150.stdout:4/751: dwrite d0/d32/fae [0,4194304] 0 2024-03-22T10:43:03.658 INFO:tasks.workunit.client.1.smithi150.stdout:1/660: dwrite d1/d19/d1a/f1e [4194304,4194304] 0 2024-03-22T10:43:03.658 INFO:tasks.workunit.client.1.smithi150.stdout:3/687: rename d5/da/f70 to d5/d22/d2f/d9c/fe1 0 2024-03-22T10:43:03.665 INFO:tasks.workunit.client.1.smithi150.stdout:1/661: write d1/da/d15/d18/f8c [12223,86829] 0 2024-03-22T10:43:03.670 INFO:tasks.workunit.client.1.smithi150.stdout:9/800: fdatasync d1/d19/fb4 0 2024-03-22T10:43:03.670 INFO:tasks.workunit.client.1.smithi150.stdout:4/752: symlink d0/d1b/d68/de0/lf4 0 2024-03-22T10:43:03.672 INFO:tasks.workunit.client.1.smithi150.stdout:3/688: mknod d5/da/dad/ce2 0 2024-03-22T10:43:03.679 INFO:tasks.workunit.client.1.smithi150.stdout:8/769: dwrite d0/d2/d19/d29/d2f/f64 [0,4194304] 0 2024-03-22T10:43:03.682 INFO:tasks.workunit.client.1.smithi150.stdout:8/770: chown d0/d2/d19/d29/d2f/c65 55238 1 2024-03-22T10:43:03.685 INFO:tasks.workunit.client.1.smithi150.stdout:1/662: read d1/d7c/fb5 [4031112,118237] 0 2024-03-22T10:43:03.687 INFO:tasks.workunit.client.1.smithi150.stdout:4/753: dread d0/d2f/d63/d7b/d9c/db1/dd7/fd9 [0,4194304] 0 2024-03-22T10:43:03.691 INFO:tasks.workunit.client.1.smithi150.stdout:3/689: write d5/f35 [5073474,19830] 0 2024-03-22T10:43:03.698 INFO:tasks.workunit.client.1.smithi150.stdout:4/754: creat d0/d1b/d4d/d9f/dc6/ff5 x:0 0 0 2024-03-22T10:43:03.703 INFO:tasks.workunit.client.1.smithi150.stdout:3/690: dread - d5/da/d8d/fbb zero size 2024-03-22T10:43:03.704 INFO:tasks.workunit.client.1.smithi150.stdout:4/755: symlink d0/d2f/d63/d7b/d9c/db1/dd7/lf6 0 2024-03-22T10:43:03.707 INFO:tasks.workunit.client.1.smithi150.stdout:8/771: dread d0/d2/d19/d29/d37/fc5 [0,4194304] 0 2024-03-22T10:43:03.710 INFO:tasks.workunit.client.1.smithi150.stdout:4/756: mkdir d0/d32/d7c/dc7/df7 0 2024-03-22T10:43:03.710 INFO:tasks.workunit.client.1.smithi150.stdout:8/772: mkdir d0/ddd/dfc 0 2024-03-22T10:43:03.712 INFO:tasks.workunit.client.1.smithi150.stdout:4/757: rmdir d0 39 2024-03-22T10:43:03.716 INFO:tasks.workunit.client.1.smithi150.stdout:3/691: write d5/d22/d2f/d38/d3e/f55 [1367263,43922] 0 2024-03-22T10:43:03.717 INFO:tasks.workunit.client.1.smithi150.stdout:5/703: dread d0/d8/dc/de/d14/fb6 [0,4194304] 0 2024-03-22T10:43:03.717 INFO:tasks.workunit.client.1.smithi150.stdout:8/773: creat d0/d5d/ffd x:0 0 0 2024-03-22T10:43:03.721 INFO:tasks.workunit.client.1.smithi150.stdout:3/692: unlink d5/f3c 0 2024-03-22T10:43:03.721 INFO:tasks.workunit.client.1.smithi150.stdout:4/758: read d0/d2f/f44 [3656332,48928] 0 2024-03-22T10:43:03.726 INFO:tasks.workunit.client.1.smithi150.stdout:1/663: dread d1/d19/d1a/f1e [0,4194304] 0 2024-03-22T10:43:03.731 INFO:tasks.workunit.client.1.smithi150.stdout:8/774: creat d0/d2/d19/d29/d88/dec/ffe x:0 0 0 2024-03-22T10:43:03.733 INFO:tasks.workunit.client.1.smithi150.stdout:1/664: write d1/d19/d50/d6e/fd3 [1024251,85425] 0 2024-03-22T10:43:03.734 INFO:tasks.workunit.client.1.smithi150.stdout:4/759: rename d0/d2f/d63/d7b/cd8 to d0/d2f/d63/cf8 0 2024-03-22T10:43:03.739 INFO:tasks.workunit.client.1.smithi150.stdout:1/665: readlink d1/d19/d50/d6e/lca 0 2024-03-22T10:43:03.744 INFO:tasks.workunit.client.1.smithi150.stdout:5/704: dwrite d0/d8/dc/de/d14/f16 [0,4194304] 0 2024-03-22T10:43:03.744 INFO:tasks.workunit.client.1.smithi150.stdout:4/760: mknod d0/d32/d7c/dd4/cf9 0 2024-03-22T10:43:03.746 INFO:tasks.workunit.client.1.smithi150.stdout:8/775: mkdir d0/d2/d19/d29/d2f/df/dff 0 2024-03-22T10:43:03.750 INFO:tasks.workunit.client.1.smithi150.stdout:4/761: chown d0/d1b/d4d/d6c/c5d 192 1 2024-03-22T10:43:03.752 INFO:tasks.workunit.client.1.smithi150.stdout:1/666: mkdir d1/d19/dd5 0 2024-03-22T10:43:03.752 INFO:tasks.workunit.client.1.smithi150.stdout:5/705: mknod d0/d8/dc/de/d14/d2e/d5a/d66/ce7 0 2024-03-22T10:43:03.758 INFO:tasks.workunit.client.1.smithi150.stdout:1/667: rename d1/da/d15/d18/d4b/dcb to d1/d19/d50/d51/da0/dcf/dd6 0 2024-03-22T10:43:03.759 INFO:tasks.workunit.client.1.smithi150.stdout:5/706: mkdir d0/ddc/de8 0 2024-03-22T10:43:03.762 INFO:tasks.workunit.client.1.smithi150.stdout:1/668: fsync d1/d7c/fb5 0 2024-03-22T10:43:03.764 INFO:tasks.workunit.client.1.smithi150.stdout:8/776: creat d0/d2/d19/d29/d37/d7d/f100 x:0 0 0 2024-03-22T10:43:03.767 INFO:tasks.workunit.client.1.smithi150.stdout:5/707: creat d0/d8/dc/de/d14/d2e/d5a/fe9 x:0 0 0 2024-03-22T10:43:03.769 INFO:tasks.workunit.client.1.smithi150.stdout:1/669: unlink d1/d19/d57/c8f 0 2024-03-22T10:43:03.772 INFO:tasks.workunit.client.1.smithi150.stdout:5/708: write d0/d8/f9a [875979,96681] 0 2024-03-22T10:43:03.773 INFO:tasks.workunit.client.1.smithi150.stdout:8/777: chown d0/d3c/f44 156716 1 2024-03-22T10:43:03.777 INFO:tasks.workunit.client.1.smithi150.stdout:8/778: dread d0/d3c/fed [0,4194304] 0 2024-03-22T10:43:03.780 INFO:tasks.workunit.client.1.smithi150.stdout:5/709: mknod d0/d8/dc/de/d14/d2e/d5a/cea 0 2024-03-22T10:43:03.782 INFO:tasks.workunit.client.1.smithi150.stdout:5/710: chown d0/d8/dc/de/d96 26831506 1 2024-03-22T10:43:03.785 INFO:tasks.workunit.client.1.smithi150.stdout:1/670: rename d1/da/d2a/d65/d78/f81 to d1/da/d2a/d4e/fd7 0 2024-03-22T10:43:03.788 INFO:tasks.workunit.client.1.smithi150.stdout:4/762: write d0/de/f3c [479076,64347] 0 2024-03-22T10:43:03.790 INFO:tasks.workunit.client.1.smithi150.stdout:5/711: fsync d0/d37/fdf 0 2024-03-22T10:43:03.791 INFO:tasks.workunit.client.1.smithi150.stdout:1/671: write d1/d19/d50/d6e/f77 [1941480,82787] 0 2024-03-22T10:43:03.791 INFO:tasks.workunit.client.1.smithi150.stdout:3/693: dread d5/d53/fc2 [0,4194304] 0 2024-03-22T10:43:03.793 INFO:tasks.workunit.client.1.smithi150.stdout:8/779: link d0/d2/d19/d29/d37/d7d/f100 d0/d2/d19/dd1/f101 0 2024-03-22T10:43:03.794 INFO:tasks.workunit.client.1.smithi150.stdout:5/712: stat d0/d8/dc/de/d14/d4f 0 2024-03-22T10:43:03.797 INFO:tasks.workunit.client.1.smithi150.stdout:8/780: dread - d0/d2/d19/d29/f9d zero size 2024-03-22T10:43:03.797 INFO:tasks.workunit.client.1.smithi150.stdout:4/763: truncate d0/d2f/d63/d7b/d9c/fed 439452 0 2024-03-22T10:43:03.799 INFO:tasks.workunit.client.1.smithi150.stdout:5/713: creat d0/dae/feb x:0 0 0 2024-03-22T10:43:03.804 INFO:tasks.workunit.client.1.smithi150.stdout:1/672: rmdir d1/da/dd0 0 2024-03-22T10:43:03.807 INFO:tasks.workunit.client.1.smithi150.stdout:5/714: mkdir d0/d8/dc/de/d14/d2e/d45/dac/dec 0 2024-03-22T10:43:03.808 INFO:tasks.workunit.client.1.smithi150.stdout:8/781: mknod d0/ddd/dfc/c102 0 2024-03-22T10:43:03.812 INFO:tasks.workunit.client.1.smithi150.stdout:4/764: link d0/d1b/d4d/d6c/d61/c66 d0/d32/d7c/d99/cfa 0 2024-03-22T10:43:03.813 INFO:tasks.workunit.client.1.smithi150.stdout:5/715: mknod d0/d8/dc/de/d14/d2e/dbc/ced 0 2024-03-22T10:43:03.814 INFO:tasks.workunit.client.1.smithi150.stdout:1/673: creat d1/da/d15/d18/d4b/dc8/fd8 x:0 0 0 2024-03-22T10:43:03.818 INFO:tasks.workunit.client.1.smithi150.stdout:1/674: dread - d1/da/d2a/d65/d78/fb7 zero size 2024-03-22T10:43:03.820 INFO:tasks.workunit.client.1.smithi150.stdout:3/694: dwrite d5/d22/d2f/d38/d3e/d5b/f76 [0,4194304] 0 2024-03-22T10:43:03.825 INFO:tasks.workunit.client.1.smithi150.stdout:8/782: dwrite d0/d5d/dd2/f6d [0,4194304] 0 2024-03-22T10:43:03.825 INFO:tasks.workunit.client.1.smithi150.stdout:6/700: sync 2024-03-22T10:43:03.826 INFO:tasks.workunit.client.1.smithi150.stdout:2/833: sync 2024-03-22T10:43:03.827 INFO:tasks.workunit.client.1.smithi150.stdout:0/743: sync 2024-03-22T10:43:03.828 INFO:tasks.workunit.client.1.smithi150.stdout:3/695: creat d5/da/d62/dba/fe3 x:0 0 0 2024-03-22T10:43:03.830 INFO:tasks.workunit.client.1.smithi150.stdout:1/675: truncate d1/d19/fbc 50835 0 2024-03-22T10:43:03.831 INFO:tasks.workunit.client.1.smithi150.stdout:8/783: write d0/d2/d46/fe0 [452142,58132] 0 2024-03-22T10:43:03.832 INFO:tasks.workunit.client.1.smithi150.stdout:4/765: dread d0/d1b/f2e [0,4194304] 0 2024-03-22T10:43:03.840 INFO:tasks.workunit.client.1.smithi150.stdout:3/696: creat d5/da/d62/dba/dd5/fe4 x:0 0 0 2024-03-22T10:43:03.840 INFO:tasks.workunit.client.1.smithi150.stdout:2/834: creat d12/d19/db6/da9/d67/db4/f105 x:0 0 0 2024-03-22T10:43:03.844 INFO:tasks.workunit.client.1.smithi150.stdout:4/766: rename d0/d2f/d63/d7b/cbc to d0/dce/cfb 0 2024-03-22T10:43:03.844 INFO:tasks.workunit.client.1.smithi150.stdout:5/716: dwrite d0/d8/dc/de/d14/d2e/d45/dac/fbe [0,4194304] 0 2024-03-22T10:43:03.846 INFO:tasks.workunit.client.1.smithi150.stdout:1/676: mkdir d1/d19/d50/d51/da0/dcf/dd9 0 2024-03-22T10:43:03.847 INFO:tasks.workunit.client.1.smithi150.stdout:8/784: mknod d0/d2/d19/d29/d37/d7d/d9e/c103 0 2024-03-22T10:43:03.848 INFO:tasks.workunit.client.1.smithi150.stdout:4/767: write d0/d1b/f2c [3224451,60893] 0 2024-03-22T10:43:03.852 INFO:tasks.workunit.client.1.smithi150.stdout:3/697: mkdir d5/da/d8d/de5 0 2024-03-22T10:43:03.858 INFO:tasks.workunit.client.1.smithi150.stdout:1/677: mknod d1/da/d2a/d55/db8/cda 0 2024-03-22T10:43:03.859 INFO:tasks.workunit.client.1.smithi150.stdout:4/768: symlink d0/d1b/d4d/d9f/lfc 0 2024-03-22T10:43:03.865 INFO:tasks.workunit.client.1.smithi150.stdout:1/678: write d1/d19/d50/d6e/fc6 [109443,125746] 0 2024-03-22T10:43:03.867 INFO:tasks.workunit.client.1.smithi150.stdout:8/785: unlink d0/d2/d19/d29/d2e/fef 0 2024-03-22T10:43:03.869 INFO:tasks.workunit.client.1.smithi150.stdout:6/701: dwrite db/dd/d15/d60/d50/dc1/fba [0,4194304] 0 2024-03-22T10:43:03.875 INFO:tasks.workunit.client.1.smithi150.stdout:3/698: creat d5/d22/d37/fe6 x:0 0 0 2024-03-22T10:43:03.877 INFO:tasks.workunit.client.1.smithi150.stdout:4/769: write d0/d1b/d68/f6a [2007484,65122] 0 2024-03-22T10:43:03.877 INFO:tasks.workunit.client.1.smithi150.stdout:1/679: rmdir d1/da/d15/d18/d4b/dc8 39 2024-03-22T10:43:03.879 INFO:tasks.workunit.client.1.smithi150.stdout:2/835: dwrite d12/d19/db6/da9/d67/d3c/d75/fed [4194304,4194304] 0 2024-03-22T10:43:03.890 INFO:tasks.workunit.client.1.smithi150.stdout:4/770: fdatasync d0/d1b/f21 0 2024-03-22T10:43:03.891 INFO:tasks.workunit.client.1.smithi150.stdout:6/702: fsync db/dd/d25/f6d 0 2024-03-22T10:43:03.891 INFO:tasks.workunit.client.1.smithi150.stdout:1/680: write d1/da/d15/d18/d4b/f68 [1696118,107317] 0 2024-03-22T10:43:03.895 INFO:tasks.workunit.client.1.smithi150.stdout:6/703: stat db/dd/d15/d28/d6c/d9b/cb8 0 2024-03-22T10:43:03.902 INFO:tasks.workunit.client.1.smithi150.stdout:3/699: creat d5/da/d78/fe7 x:0 0 0 2024-03-22T10:43:03.904 INFO:tasks.workunit.client.1.smithi150.stdout:8/786: dwrite d0/d2/d19/d29/d2e/d70/fb8 [0,4194304] 0 2024-03-22T10:43:03.905 INFO:tasks.workunit.client.1.smithi150.stdout:3/700: chown d5/d22/d2f/d38/d3e/lc9 2469 1 2024-03-22T10:43:03.905 INFO:tasks.workunit.client.1.smithi150.stdout:4/771: unlink d0/d1b/d4d/d6c/d61/d98/cb0 0 2024-03-22T10:43:03.908 INFO:tasks.workunit.client.1.smithi150.stdout:2/836: mknod d12/d19/db6/da9/d97/d78/c106 0 2024-03-22T10:43:03.912 INFO:tasks.workunit.client.1.smithi150.stdout:2/837: fsync d12/d19/db6/da9/d61/fe9 0 2024-03-22T10:43:03.916 INFO:tasks.workunit.client.1.smithi150.stdout:4/772: mkdir d0/dce/dfd 0 2024-03-22T10:43:03.917 INFO:tasks.workunit.client.1.smithi150.stdout:6/704: dread db/dd/d15/d60/d50/dc1/fba [0,4194304] 0 2024-03-22T10:43:03.917 INFO:tasks.workunit.client.1.smithi150.stdout:1/681: symlink d1/da/d15/d18/d4b/dc8/ldb 0 2024-03-22T10:43:03.917 INFO:tasks.workunit.client.1.smithi150.stdout:2/838: write d12/d19/db6/da9/d67/d8d/da6/fdb [3496963,53801] 0 2024-03-22T10:43:03.919 INFO:tasks.workunit.client.1.smithi150.stdout:1/682: fsync d1/da/f16 0 2024-03-22T10:43:03.921 INFO:tasks.workunit.client.1.smithi150.stdout:1/683: chown d1/da/d15/d18/l33 254156 1 2024-03-22T10:43:03.931 INFO:tasks.workunit.client.1.smithi150.stdout:8/787: write d0/d3c/d71/d7a/f94 [477386,5193] 0 2024-03-22T10:43:03.931 INFO:tasks.workunit.client.1.smithi150.stdout:2/839: mkdir d12/d19/db6/da9/d67/d8d/d107 0 2024-03-22T10:43:03.946 INFO:tasks.workunit.client.1.smithi150.stdout:8/788: fdatasync d0/d2/d19/d29/d2f/f3b 0 2024-03-22T10:43:03.949 INFO:tasks.workunit.client.1.smithi150.stdout:6/705: link db/dd/d15/d60/f67 db/dd/d15/ff7 0 2024-03-22T10:43:03.952 INFO:tasks.workunit.client.1.smithi150.stdout:1/684: link d1/da/d15/c86 d1/da/d2a/d55/db8/cdc 0 2024-03-22T10:43:03.955 INFO:tasks.workunit.client.1.smithi150.stdout:2/840: link d12/d19/db6/da9/d67/d3c/f92 d12/d19/db6/df1/f108 0 2024-03-22T10:43:03.958 INFO:tasks.workunit.client.1.smithi150.stdout:3/701: dread d5/da/ff [0,4194304] 0 2024-03-22T10:43:03.959 INFO:tasks.workunit.client.1.smithi150.stdout:8/789: mknod d0/d3c/d71/d7a/c104 0 2024-03-22T10:43:03.959 INFO:tasks.workunit.client.1.smithi150.stdout:4/773: getdents d0/d2f/d63/d7b 0 2024-03-22T10:43:03.963 INFO:tasks.workunit.client.1.smithi150.stdout:4/774: chown d0/d1b/d4d/d9f/fb5 13399230 1 2024-03-22T10:43:03.963 INFO:tasks.workunit.client.1.smithi150.stdout:4/775: chown d0/d1b/d4d/l80 343238 1 2024-03-22T10:43:03.964 INFO:tasks.workunit.client.1.smithi150.stdout:2/841: mkdir d12/d35/d109 0 2024-03-22T10:43:03.969 INFO:tasks.workunit.client.1.smithi150.stdout:1/685: rename d1/da/d2a/d65/d78/f98 to d1/d79/fdd 0 2024-03-22T10:43:03.970 INFO:tasks.workunit.client.1.smithi150.stdout:6/706: dread db/dd/d25/f3d [0,4194304] 0 2024-03-22T10:43:03.974 INFO:tasks.workunit.client.1.smithi150.stdout:3/702: unlink d5/da/d78/d86/c67 0 2024-03-22T10:43:03.976 INFO:tasks.workunit.client.1.smithi150.stdout:6/707: read db/dd/d15/ff7 [1139767,95129] 0 2024-03-22T10:43:03.978 INFO:tasks.workunit.client.1.smithi150.stdout:6/708: chown db/dd/d15/d30/f4b 91539989 1 2024-03-22T10:43:03.979 INFO:tasks.workunit.client.1.smithi150.stdout:4/776: symlink d0/d2f/d63/d7b/d9c/db1/dd7/lfe 0 2024-03-22T10:43:03.979 INFO:tasks.workunit.client.1.smithi150.stdout:6/709: fdatasync db/dd/d25/fac 0 2024-03-22T10:43:03.980 INFO:tasks.workunit.client.1.smithi150.stdout:7/736: sync 2024-03-22T10:43:03.980 INFO:tasks.workunit.client.1.smithi150.stdout:9/801: sync 2024-03-22T10:43:03.984 INFO:tasks.workunit.client.1.smithi150.stdout:2/842: mknod d12/d19/db6/da9/d61/d98/c10a 0 2024-03-22T10:43:03.987 INFO:tasks.workunit.client.1.smithi150.stdout:2/843: chown d12/d19/db6/da9/d67/d3c/d75/ce1 0 1 2024-03-22T10:43:03.991 INFO:tasks.workunit.client.1.smithi150.stdout:2/844: chown d12/d19/d2d/f36 597 1 2024-03-22T10:43:03.992 INFO:tasks.workunit.client.1.smithi150.stdout:8/790: dread d0/d2/f14 [4194304,4194304] 0 2024-03-22T10:43:03.994 INFO:tasks.workunit.client.1.smithi150.stdout:1/686: dread d1/d19/f26 [0,4194304] 0 2024-03-22T10:43:04.000 INFO:tasks.workunit.client.1.smithi150.stdout:7/737: rmdir d2/d5/d7/dab 39 2024-03-22T10:43:04.001 INFO:tasks.workunit.client.1.smithi150.stdout:6/710: symlink db/dd/d15/d57/lf8 0 2024-03-22T10:43:04.005 INFO:tasks.workunit.client.1.smithi150.stdout:6/711: dread - db/dd/d15/d60/fcd zero size 2024-03-22T10:43:04.007 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:03 smithi115 ceph-mon[29111]: pgmap v20: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 91 MiB/s rd, 225 MiB/s wr, 473 op/s 2024-03-22T10:43:04.012 INFO:tasks.workunit.client.1.smithi150.stdout:7/738: unlink d2/d5/d7/d9/db/d9f/d96/f83 0 2024-03-22T10:43:04.018 INFO:tasks.workunit.client.1.smithi150.stdout:7/739: write d2/d5/d7/d9/db/d9f/fe1 [606854,27949] 0 2024-03-22T10:43:04.018 INFO:tasks.workunit.client.1.smithi150.stdout:3/703: dread d5/da/f7a [0,4194304] 0 2024-03-22T10:43:04.020 INFO:tasks.workunit.client.1.smithi150.stdout:1/687: rename d1/da/l54 to d1/d19/d50/d51/da0/dcf/dd9/lde 0 2024-03-22T10:43:04.022 INFO:tasks.workunit.client.1.smithi150.stdout:4/777: creat d0/d1b/d4d/fff x:0 0 0 2024-03-22T10:43:04.026 INFO:tasks.workunit.client.1.smithi150.stdout:9/802: dread d1/d8/de/df/d36/d40/d41/f61 [0,4194304] 0 2024-03-22T10:43:04.026 INFO:tasks.workunit.client.1.smithi150.stdout:3/704: dread - d5/d22/d2f/d38/d3e/d5b/fc7 zero size 2024-03-22T10:43:04.029 INFO:tasks.workunit.client.1.smithi150.stdout:6/712: write db/dd/d15/d60/f32 [1861601,56264] 0 2024-03-22T10:43:04.032 INFO:tasks.workunit.client.1.smithi150.stdout:3/705: read d5/fb [1609837,120333] 0 2024-03-22T10:43:04.042 INFO:tasks.workunit.client.1.smithi150.stdout:4/778: creat d0/d2f/d63/d7b/d9c/db1/f100 x:0 0 0 2024-03-22T10:43:04.043 INFO:tasks.workunit.client.1.smithi150.stdout:1/688: write d1/da/d2a/d65/d78/fb1 [585229,32031] 0 2024-03-22T10:43:04.043 INFO:tasks.workunit.client.1.smithi150.stdout:2/845: getdents d12/d19/db6/da9/d67/d8d/da6 0 2024-03-22T10:43:04.043 INFO:tasks.workunit.client.1.smithi150.stdout:6/713: rename db/f4f to db/d52/d9f/dc6/df1/ff9 0 2024-03-22T10:43:04.046 INFO:tasks.workunit.client.1.smithi150.stdout:8/791: dwrite d0/f34 [0,4194304] 0 2024-03-22T10:43:04.046 INFO:tasks.workunit.client.1.smithi150.stdout:3/706: chown d5/da/d78/fe7 498794 1 2024-03-22T10:43:04.051 INFO:tasks.workunit.client.1.smithi150.stdout:7/740: truncate d2/d5/d7/d9/db/f76 1486917 0 2024-03-22T10:43:04.056 INFO:tasks.workunit.client.1.smithi150.stdout:1/689: creat d1/d7c/fdf x:0 0 0 2024-03-22T10:43:04.061 INFO:tasks.workunit.client.1.smithi150.stdout:4/779: dread d0/d1b/d4d/d6c/d61/d71/fd0 [0,4194304] 0 2024-03-22T10:43:04.062 INFO:tasks.workunit.client.1.smithi150.stdout:0/744: sync 2024-03-22T10:43:04.062 INFO:tasks.workunit.client.1.smithi150.stdout:5/717: sync 2024-03-22T10:43:04.066 INFO:tasks.workunit.client.1.smithi150.stdout:6/714: write db/dd/d82/d84/fb5 [2104343,107975] 0 2024-03-22T10:43:04.072 INFO:tasks.workunit.client.1.smithi150.stdout:0/745: dread - d0/d1/d84/d86/d42/d10/fb7 zero size 2024-03-22T10:43:04.080 INFO:tasks.workunit.client.1.smithi150.stdout:3/707: dread d5/d22/d2f/d38/d59/d88/fbe [0,4194304] 0 2024-03-22T10:43:04.080 INFO:tasks.workunit.client.1.smithi150.stdout:9/803: dwrite d1/d8/de/df/d36/d40/d44/db5/fd9 [0,4194304] 0 2024-03-22T10:43:04.085 INFO:tasks.workunit.client.1.smithi150.stdout:5/718: symlink d0/ddc/de8/lee 0 2024-03-22T10:43:04.088 INFO:tasks.workunit.client.1.smithi150.stdout:0/746: write d0/d12/f58 [3786467,119306] 0 2024-03-22T10:43:04.091 INFO:tasks.workunit.client.1.smithi150.stdout:3/708: readlink d5/d22/d2f/d38/d3e/d5b/lae 0 2024-03-22T10:43:04.097 INFO:tasks.workunit.client.1.smithi150.stdout:6/715: rename db/dd/d82/f9a to db/dd/d15/d60/d50/ffa 0 2024-03-22T10:43:04.100 INFO:tasks.workunit.client.1.smithi150.stdout:7/741: dwrite d2/d5/d7/d8/d46/fd2 [0,4194304] 0 2024-03-22T10:43:04.102 INFO:tasks.workunit.client.1.smithi150.stdout:8/792: dwrite d0/d3c/d71/f7f [0,4194304] 0 2024-03-22T10:43:04.102 INFO:tasks.workunit.client.1.smithi150.stdout:2/846: dwrite d12/d19/db6/d63/fa8 [0,4194304] 0 2024-03-22T10:43:04.107 INFO:tasks.workunit.client.1.smithi150.stdout:0/747: rmdir d0/d1/d84 39 2024-03-22T10:43:04.114 INFO:tasks.workunit.client.1.smithi150.stdout:3/709: mkdir d5/da/d78/d86/de8 0 2024-03-22T10:43:04.117 INFO:tasks.workunit.client.1.smithi150.stdout:1/690: dwrite d1/d19/d1a/d7d/f82 [0,4194304] 0 2024-03-22T10:43:04.118 INFO:tasks.workunit.client.1.smithi150.stdout:7/742: chown d2/d5/d7/d8/d46/fd2 204826089 1 2024-03-22T10:43:04.119 INFO:tasks.workunit.client.1.smithi150.stdout:9/804: fsync d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/f107 0 2024-03-22T10:43:04.121 INFO:tasks.workunit.client.1.smithi150.stdout:5/719: symlink d0/d8/dc/de/d14/d2e/d5a/d66/da9/lef 0 2024-03-22T10:43:04.130 INFO:tasks.workunit.client.1.smithi150.stdout:8/793: creat d0/d2/d19/d29/d88/f105 x:0 0 0 2024-03-22T10:43:04.132 INFO:tasks.workunit.client.1.smithi150.stdout:4/780: dwrite d0/d1b/d4d/f8c [0,4194304] 0 2024-03-22T10:43:04.140 INFO:tasks.workunit.client.1.smithi150.stdout:0/748: read d0/d1/d84/d86/d42/d10/f71 [60666,23779] 0 2024-03-22T10:43:04.140 INFO:tasks.workunit.client.1.smithi150.stdout:2/847: creat d12/d19/db6/d54/f10b x:0 0 0 2024-03-22T10:43:04.140 INFO:tasks.workunit.client.1.smithi150.stdout:6/716: symlink db/dd/d15/lfb 0 2024-03-22T10:43:04.145 INFO:tasks.workunit.client.1.smithi150.stdout:3/710: rename d5/d22/d2f/d38/d59/d88/c90 to d5/d22/d2f/d38/d3e/ce9 0 2024-03-22T10:43:04.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:03 smithi150 ceph-mon[34518]: pgmap v20: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 91 MiB/s rd, 225 MiB/s wr, 473 op/s 2024-03-22T10:43:04.148 INFO:tasks.workunit.client.1.smithi150.stdout:5/720: chown d0/d8/dc/de/d14/d68/c92 3 1 2024-03-22T10:43:04.152 INFO:tasks.workunit.client.1.smithi150.stdout:9/805: mknod d1/d8/de/df/d36/d40/d44/db5/dda/d104/dc3/c114 0 2024-03-22T10:43:04.155 INFO:tasks.workunit.client.1.smithi150.stdout:9/806: read d1/d8/de/d73/d101/d29/fb1 [2850014,119631] 0 2024-03-22T10:43:04.155 INFO:tasks.workunit.client.1.smithi150.stdout:8/794: mkdir d0/d3c/d71/d7a/d89/db5/d106 0 2024-03-22T10:43:04.156 INFO:tasks.workunit.client.1.smithi150.stdout:0/749: creat d0/d1/d84/d86/fed x:0 0 0 2024-03-22T10:43:04.157 INFO:tasks.workunit.client.1.smithi150.stdout:9/807: chown d1/d8/de/df/d36/d40 12 1 2024-03-22T10:43:04.158 INFO:tasks.workunit.client.1.smithi150.stdout:1/691: rename d1/d19/d50/d51/c59 to d1/da/d2a/d65/d78/ce0 0 2024-03-22T10:43:04.159 INFO:tasks.workunit.client.1.smithi150.stdout:2/848: symlink d12/d19/db6/da9/d67/d8d/da6/l10c 0 2024-03-22T10:43:04.161 INFO:tasks.workunit.client.1.smithi150.stdout:9/808: truncate d1/d8/f88 1035661 0 2024-03-22T10:43:04.171 INFO:tasks.workunit.client.1.smithi150.stdout:6/717: write db/dd/d82/d84/dc8/fe2 [832331,124735] 0 2024-03-22T10:43:04.173 INFO:tasks.workunit.client.1.smithi150.stdout:3/711: rename d5/d22/d37/f79 to d5/d22/dc8/dda/d95/fea 0 2024-03-22T10:43:04.175 INFO:tasks.workunit.client.1.smithi150.stdout:2/849: read - d12/fbc zero size 2024-03-22T10:43:04.175 INFO:tasks.workunit.client.1.smithi150.stdout:5/721: write d0/d8/f23 [1546281,6499] 0 2024-03-22T10:43:04.179 INFO:tasks.workunit.client.1.smithi150.stdout:4/781: creat d0/d2f/d63/f101 x:0 0 0 2024-03-22T10:43:04.180 INFO:tasks.workunit.client.1.smithi150.stdout:8/795: write d0/d2/d19/d29/d37/fb3 [858776,77621] 0 2024-03-22T10:43:04.184 INFO:tasks.workunit.client.1.smithi150.stdout:0/750: rename d0/d1/d84/fe9 to d0/d1/d84/d86/d11/d66/d35/d5f/da0/fee 0 2024-03-22T10:43:04.184 INFO:tasks.workunit.client.1.smithi150.stdout:1/692: mkdir d1/d19/d50/d6e/db6/de1 0 2024-03-22T10:43:04.185 INFO:tasks.workunit.client.1.smithi150.stdout:5/722: fdatasync d0/f10 0 2024-03-22T10:43:04.188 INFO:tasks.workunit.client.1.smithi150.stdout:3/712: mknod d5/d22/d2f/d38/d3e/d5b/da6/ceb 0 2024-03-22T10:43:04.188 INFO:tasks.workunit.client.1.smithi150.stdout:2/850: dread - d12/d19/db6/da9/d67/f5a zero size 2024-03-22T10:43:04.188 INFO:tasks.workunit.client.1.smithi150.stdout:7/743: dwrite d2/d5/d7/d9/db/f76 [0,4194304] 0 2024-03-22T10:43:04.190 INFO:tasks.workunit.client.1.smithi150.stdout:8/796: fsync d0/d3c/f56 0 2024-03-22T10:43:04.192 INFO:tasks.workunit.client.1.smithi150.stdout:2/851: chown d12/d19/db6/da9/d67 538 1 2024-03-22T10:43:04.195 INFO:tasks.workunit.client.1.smithi150.stdout:4/782: creat d0/d2f/d63/d7b/d9c/f102 x:0 0 0 2024-03-22T10:43:04.196 INFO:tasks.workunit.client.1.smithi150.stdout:9/809: rmdir d1/d8/dd6 0 2024-03-22T10:43:04.204 INFO:tasks.workunit.client.1.smithi150.stdout:7/744: mkdir d2/d5/d7/d9/db/dfb 0 2024-03-22T10:43:04.207 INFO:tasks.workunit.client.1.smithi150.stdout:1/693: symlink d1/d19/d1a/le2 0 2024-03-22T10:43:04.207 INFO:tasks.workunit.client.1.smithi150.stdout:2/852: rmdir d12/d19/db6/da9/d2c 39 2024-03-22T10:43:04.210 INFO:tasks.workunit.client.1.smithi150.stdout:4/783: mknod d0/d32/d7c/c103 0 2024-03-22T10:43:04.210 INFO:tasks.workunit.client.1.smithi150.stdout:5/723: dread d0/d8/dc/de/d14/d2e/d46/d4d/fb7 [0,4194304] 0 2024-03-22T10:43:04.211 INFO:tasks.workunit.client.1.smithi150.stdout:9/810: mknod d1/d8/de/df/d36/d40/d44/db5/dda/c115 0 2024-03-22T10:43:04.214 INFO:tasks.workunit.client.1.smithi150.stdout:4/784: write d0/d32/fae [3658079,32680] 0 2024-03-22T10:43:04.216 INFO:tasks.workunit.client.1.smithi150.stdout:5/724: truncate d0/dae/feb 740187 0 2024-03-22T10:43:04.216 INFO:tasks.workunit.client.1.smithi150.stdout:5/725: readlink d0/d37/l3c 0 2024-03-22T10:43:04.217 INFO:tasks.workunit.client.1.smithi150.stdout:6/718: rename db/dd/d82/d84/dc8/fe2 to db/ffc 0 2024-03-22T10:43:04.218 INFO:tasks.workunit.client.1.smithi150.stdout:8/797: mkdir d0/d2/d19/d29/d37/dc2/d107 0 2024-03-22T10:43:04.222 INFO:tasks.workunit.client.1.smithi150.stdout:5/726: creat d0/dae/db2/ff0 x:0 0 0 2024-03-22T10:43:04.225 INFO:tasks.workunit.client.1.smithi150.stdout:0/751: rename d0/d1/d84/d86/d11/d66/d37/cca to d0/d1/d84/d86/d42/d10/d38/d52/d9b/cef 0 2024-03-22T10:43:04.226 INFO:tasks.workunit.client.1.smithi150.stdout:3/713: dread d5/d22/d2f/d38/d3e/d5b/f7f [0,4194304] 0 2024-03-22T10:43:04.227 INFO:tasks.workunit.client.1.smithi150.stdout:2/853: rmdir d12/d35/df8 0 2024-03-22T10:43:04.230 INFO:tasks.workunit.client.1.smithi150.stdout:1/694: creat d1/d19/fe3 x:0 0 0 2024-03-22T10:43:04.232 INFO:tasks.workunit.client.1.smithi150.stdout:2/854: readlink d12/d19/db6/da9/d67/d8d/l8f 0 2024-03-22T10:43:04.236 INFO:tasks.workunit.client.1.smithi150.stdout:3/714: mkdir d5/da/d78/db7/dec 0 2024-03-22T10:43:04.237 INFO:tasks.workunit.client.1.smithi150.stdout:4/785: dread d0/f7 [0,4194304] 0 2024-03-22T10:43:04.240 INFO:tasks.workunit.client.1.smithi150.stdout:2/855: truncate d12/d19/db6/da9/f103 28286 0 2024-03-22T10:43:04.240 INFO:tasks.workunit.client.1.smithi150.stdout:5/727: truncate d0/d8/dc/de/d14/d2e/f54 3429135 0 2024-03-22T10:43:04.242 INFO:tasks.workunit.client.1.smithi150.stdout:7/745: rename d2/d5/d7/d8/d57/d4a/d8d/ff0 to d2/ffc 0 2024-03-22T10:43:04.247 INFO:tasks.workunit.client.1.smithi150.stdout:3/715: mkdir d5/da/d9d/ded 0 2024-03-22T10:43:04.252 INFO:tasks.workunit.client.1.smithi150.stdout:6/719: dread db/dd/d25/f45 [0,4194304] 0 2024-03-22T10:43:04.254 INFO:tasks.workunit.client.1.smithi150.stdout:4/786: getdents d0/d32/d7c/dc7/df7 0 2024-03-22T10:43:04.255 INFO:tasks.workunit.client.1.smithi150.stdout:0/752: dwrite d0/d1/d84/d86/d42/d10/d38/f4c [4194304,4194304] 0 2024-03-22T10:43:04.258 INFO:tasks.workunit.client.1.smithi150.stdout:3/716: chown d5/da/d8d/c94 39 1 2024-03-22T10:43:04.262 INFO:tasks.workunit.client.1.smithi150.stdout:7/746: write d2/d5/d7/d9/db/d9f/d96/fe4 [1075927,25443] 0 2024-03-22T10:43:04.265 INFO:tasks.workunit.client.1.smithi150.stdout:8/798: dwrite d0/d2/d19/d29/d2f/df/f4f [0,4194304] 0 2024-03-22T10:43:04.268 INFO:tasks.workunit.client.1.smithi150.stdout:4/787: fsync d0/de/f81 0 2024-03-22T10:43:04.268 INFO:tasks.workunit.client.1.smithi150.stdout:1/695: truncate d1/da/d15/d18/f4a 97752 0 2024-03-22T10:43:04.268 INFO:tasks.workunit.client.1.smithi150.stdout:2/856: dwrite d12/d19/d2d/fcf [0,4194304] 0 2024-03-22T10:43:04.268 INFO:tasks.workunit.client.1.smithi150.stdout:6/720: mkdir db/d52/d9f/dc6/df1/db4/dfd 0 2024-03-22T10:43:04.274 INFO:tasks.workunit.client.1.smithi150.stdout:8/799: chown d0/d2/d19/d29/d2f/df/dff 9504 1 2024-03-22T10:43:04.278 INFO:tasks.workunit.client.1.smithi150.stdout:0/753: mkdir d0/d1/d84/d86/d11/d26/dd2/df0 0 2024-03-22T10:43:04.282 INFO:tasks.workunit.client.1.smithi150.stdout:3/717: getdents d5/d22/d2f 0 2024-03-22T10:43:04.286 INFO:tasks.workunit.client.1.smithi150.stdout:2/857: write d12/d19/db6/da9/d61/d98/fd4 [3404972,107170] 0 2024-03-22T10:43:04.286 INFO:tasks.workunit.client.1.smithi150.stdout:3/718: read - d5/da/d78/fe7 zero size 2024-03-22T10:43:04.289 INFO:tasks.workunit.client.1.smithi150.stdout:6/721: rename db/dd/d15/d60/fdb to db/dd/ffe 0 2024-03-22T10:43:04.289 INFO:tasks.workunit.client.1.smithi150.stdout:4/788: write d0/d1b/d4d/d6c/d61/d71/fd0 [709587,74171] 0 2024-03-22T10:43:04.291 INFO:tasks.workunit.client.1.smithi150.stdout:5/728: dwrite d0/dae/feb [0,4194304] 0 2024-03-22T10:43:04.293 INFO:tasks.workunit.client.1.smithi150.stdout:7/747: write d2/d5/d7/d8/d57/f5b [550244,53526] 0 2024-03-22T10:43:04.294 INFO:tasks.workunit.client.1.smithi150.stdout:4/789: chown d0/d2f/f50 352756 1 2024-03-22T10:43:04.295 INFO:tasks.workunit.client.1.smithi150.stdout:8/800: rmdir d0/d2/d19/d29/d69 0 2024-03-22T10:43:04.296 INFO:tasks.workunit.client.1.smithi150.stdout:2/858: stat d12/d19/db6/da9/d67/d3c/d75/fc8 0 2024-03-22T10:43:04.300 INFO:tasks.workunit.client.1.smithi150.stdout:4/790: creat d0/d2f/d63/d7b/d9c/db1/dd7/f104 x:0 0 0 2024-03-22T10:43:04.300 INFO:tasks.workunit.client.1.smithi150.stdout:6/722: symlink db/d52/d9f/dc6/df1/lff 0 2024-03-22T10:43:04.307 INFO:tasks.workunit.client.1.smithi150.stdout:5/729: truncate d0/f11 58687 0 2024-03-22T10:43:04.308 INFO:tasks.workunit.client.1.smithi150.stdout:4/791: creat d0/d32/f105 x:0 0 0 2024-03-22T10:43:04.309 INFO:tasks.workunit.client.1.smithi150.stdout:8/801: creat d0/d2/d19/d29/d2f/f108 x:0 0 0 2024-03-22T10:43:04.312 INFO:tasks.workunit.client.1.smithi150.stdout:2/859: truncate d12/d19/db6/da9/d2c/d73/ff2 616227 0 2024-03-22T10:43:04.315 INFO:tasks.workunit.client.1.smithi150.stdout:6/723: mknod db/dd/d15/d60/d8b/c100 0 2024-03-22T10:43:04.316 INFO:tasks.workunit.client.1.smithi150.stdout:0/754: dwrite d0/d12/f58 [4194304,4194304] 0 2024-03-22T10:43:04.318 INFO:tasks.workunit.client.1.smithi150.stdout:1/696: truncate d1/da/d15/f93 709826 0 2024-03-22T10:43:04.329 INFO:tasks.workunit.client.1.smithi150.stdout:0/755: read d0/d1/d84/d86/d42/d10/d38/d52/d9b/fb6 [329636,26698] 0 2024-03-22T10:43:04.332 INFO:tasks.workunit.client.1.smithi150.stdout:2/860: rmdir d12/d19/db6/da9/d67/d3c/d75 39 2024-03-22T10:43:04.334 INFO:tasks.workunit.client.1.smithi150.stdout:8/802: dread d0/d5d/dd2/f6d [0,4194304] 0 2024-03-22T10:43:04.335 INFO:tasks.workunit.client.1.smithi150.stdout:1/697: mkdir d1/d19/d50/d51/da0/dcf/dd6/de4 0 2024-03-22T10:43:04.336 INFO:tasks.workunit.client.1.smithi150.stdout:4/792: write d0/d32/fa2 [380394,60890] 0 2024-03-22T10:43:04.341 INFO:tasks.workunit.client.1.smithi150.stdout:4/793: stat d0/d1b/d4d/d6c/d61/d71 0 2024-03-22T10:43:04.344 INFO:tasks.workunit.client.1.smithi150.stdout:2/861: creat d12/d19/d1a/f10d x:0 0 0 2024-03-22T10:43:04.350 INFO:tasks.workunit.client.1.smithi150.stdout:5/730: dwrite d0/d8/dc/de/d96/fd9 [0,4194304] 0 2024-03-22T10:43:04.351 INFO:tasks.workunit.client.1.smithi150.stdout:8/803: rename d0/d2/d19/d29/d37/dc6/fd4 to d0/d3c/d71/d7a/d89/db5/d106/f109 0 2024-03-22T10:43:04.351 INFO:tasks.workunit.client.1.smithi150.stdout:4/794: mknod d0/d32/d7c/dc7/c106 0 2024-03-22T10:43:04.352 INFO:tasks.workunit.client.1.smithi150.stdout:1/698: symlink d1/da/d2a/le5 0 2024-03-22T10:43:04.354 INFO:tasks.workunit.client.1.smithi150.stdout:6/724: write db/dd/f5b [1702522,80997] 0 2024-03-22T10:43:04.355 INFO:tasks.workunit.client.1.smithi150.stdout:7/748: dread d2/d5/d7/d38/f4b [0,4194304] 0 2024-03-22T10:43:04.358 INFO:tasks.workunit.client.1.smithi150.stdout:8/804: fdatasync d0/d2/d46/f52 0 2024-03-22T10:43:04.366 INFO:tasks.workunit.client.1.smithi150.stdout:5/731: mknod d0/dae/de0/dce/cf1 0 2024-03-22T10:43:04.366 INFO:tasks.workunit.client.1.smithi150.stdout:6/725: mknod db/dd/d15/d28/d6c/c101 0 2024-03-22T10:43:04.368 INFO:tasks.workunit.client.1.smithi150.stdout:4/795: link d0/d1b/d4d/d9f/fef d0/d2f/d63/d7b/d9c/db1/f107 0 2024-03-22T10:43:04.368 INFO:tasks.workunit.client.1.smithi150.stdout:7/749: mknod d2/d5/d7/d8/d57/d4a/d8d/cfd 0 2024-03-22T10:43:04.372 INFO:tasks.workunit.client.1.smithi150.stdout:2/862: write d12/d19/db6/da9/d67/d3c/f6b [2092280,47292] 0 2024-03-22T10:43:04.376 INFO:tasks.workunit.client.1.smithi150.stdout:2/863: dread - d12/d19/db6/da9/d67/f5a zero size 2024-03-22T10:43:04.378 INFO:tasks.workunit.client.1.smithi150.stdout:1/699: dwrite d1/d19/d50/d6e/fd3 [0,4194304] 0 2024-03-22T10:43:04.387 INFO:tasks.workunit.client.1.smithi150.stdout:4/796: fdatasync d0/d2f/d63/fb4 0 2024-03-22T10:43:04.390 INFO:tasks.workunit.client.1.smithi150.stdout:5/732: dwrite d0/d8/dc/de/d96/db8/fe1 [0,4194304] 0 2024-03-22T10:43:04.391 INFO:tasks.workunit.client.1.smithi150.stdout:4/797: dread - d0/d1b/d4d/d9f/ff1 zero size 2024-03-22T10:43:04.392 INFO:tasks.workunit.client.1.smithi150.stdout:5/733: fsync d0/ddc/fdd 0 2024-03-22T10:43:04.396 INFO:tasks.workunit.client.1.smithi150.stdout:2/864: mkdir d12/d19/db6/da9/d2c/d73/d10e 0 2024-03-22T10:43:04.397 INFO:tasks.workunit.client.1.smithi150.stdout:6/726: mknod db/d52/de6/c102 0 2024-03-22T10:43:04.398 INFO:tasks.workunit.client.1.smithi150.stdout:2/865: stat d12/d19/d1a/c27 0 2024-03-22T10:43:04.400 INFO:tasks.workunit.client.1.smithi150.stdout:1/700: symlink d1/d19/le6 0 2024-03-22T10:43:04.400 INFO:tasks.workunit.client.1.smithi150.stdout:8/805: link d0/d3c/d71/d7a/d89/laa d0/d2/d19/d29/d37/da1/l10a 0 2024-03-22T10:43:04.403 INFO:tasks.workunit.client.1.smithi150.stdout:6/727: stat db/dd/d15/d60/d50/leb 0 2024-03-22T10:43:04.404 INFO:tasks.workunit.client.1.smithi150.stdout:2/866: write d12/d21/fc3 [910899,14310] 0 2024-03-22T10:43:04.413 INFO:tasks.workunit.client.1.smithi150.stdout:5/734: rename d0/d8/dc/de/d14/d4f/d47/f6e to d0/d8/dc/de/d14/d4f/d47/ff2 0 2024-03-22T10:43:04.416 INFO:tasks.workunit.client.1.smithi150.stdout:6/728: mkdir db/dd/d15/d30/ddf/d103 0 2024-03-22T10:43:04.424 INFO:tasks.workunit.client.1.smithi150.stdout:1/701: creat d1/da/d2a/dc0/fe7 x:0 0 0 2024-03-22T10:43:04.425 INFO:tasks.workunit.client.1.smithi150.stdout:5/735: creat d0/d8/dc/de/d14/d2e/d45/ff3 x:0 0 0 2024-03-22T10:43:04.426 INFO:tasks.workunit.client.1.smithi150.stdout:7/750: dread d2/d5/d7/d9/f28 [0,4194304] 0 2024-03-22T10:43:04.440 INFO:tasks.workunit.client.1.smithi150.stdout:5/736: rmdir d0/d8/dc/de/d14/d2e/d46/d4d 39 2024-03-22T10:43:04.440 INFO:tasks.workunit.client.1.smithi150.stdout:6/729: link db/dd/d15/d60/d50/leb db/dd/d15/d60/d50/dc1/l104 0 2024-03-22T10:43:04.442 INFO:tasks.workunit.client.1.smithi150.stdout:4/798: dread d0/d1b/d4d/d6c/d61/d71/fd3 [0,4194304] 0 2024-03-22T10:43:04.442 INFO:tasks.workunit.client.1.smithi150.stdout:7/751: truncate d2/d5/d7/d8/d46/fbc 384806 0 2024-03-22T10:43:04.446 INFO:tasks.workunit.client.1.smithi150.stdout:1/702: dread d1/d19/d1a/d7d/f82 [0,4194304] 0 2024-03-22T10:43:04.452 INFO:tasks.workunit.client.1.smithi150.stdout:2/867: dread d12/d19/d1a/f30 [0,4194304] 0 2024-03-22T10:43:04.453 INFO:tasks.workunit.client.1.smithi150.stdout:4/799: mkdir d0/d1b/d4d/d6c/d61/d71/d108 0 2024-03-22T10:43:04.455 INFO:tasks.workunit.client.1.smithi150.stdout:7/752: creat d2/d5/d7/d8/d46/d93/ffe x:0 0 0 2024-03-22T10:43:04.456 INFO:tasks.workunit.client.1.smithi150.stdout:2/868: chown d12/d19/db6/da9/d2c/d65/l102 456 1 2024-03-22T10:43:04.457 INFO:tasks.workunit.client.1.smithi150.stdout:6/730: link db/d52/lca db/dd/d15/d30/ddf/d103/l105 0 2024-03-22T10:43:04.457 INFO:tasks.workunit.client.1.smithi150.stdout:1/703: dread d1/d19/fbc [0,4194304] 0 2024-03-22T10:43:04.459 INFO:tasks.workunit.client.1.smithi150.stdout:8/806: dwrite d0/d2/d46/f4e [0,4194304] 0 2024-03-22T10:43:04.459 INFO:tasks.workunit.client.1.smithi150.stdout:6/731: readlink db/dd/d15/l3e 0 2024-03-22T10:43:04.461 INFO:tasks.workunit.client.1.smithi150.stdout:2/869: write d12/d19/db6/da9/d67/d3c/fce [1028996,41347] 0 2024-03-22T10:43:04.466 INFO:tasks.workunit.client.1.smithi150.stdout:1/704: unlink d1/da/d2a/f80 0 2024-03-22T10:43:04.470 INFO:tasks.workunit.client.1.smithi150.stdout:8/807: fsync d0/d5d/ffd 0 2024-03-22T10:43:04.475 INFO:tasks.workunit.client.1.smithi150.stdout:2/870: write d12/d19/db6/da9/d67/ff7 [588221,46492] 0 2024-03-22T10:43:04.475 INFO:tasks.workunit.client.1.smithi150.stdout:1/705: fsync d1/da/d2a/d65/d78/fb1 0 2024-03-22T10:43:04.477 INFO:tasks.workunit.client.1.smithi150.stdout:8/808: write d0/d5d/dd2/fae [1724773,46265] 0 2024-03-22T10:43:04.479 INFO:tasks.workunit.client.1.smithi150.stdout:1/706: chown d1/d19/d71 64442 1 2024-03-22T10:43:04.480 INFO:tasks.workunit.client.1.smithi150.stdout:5/737: dwrite d0/d8/dc/de/d14/d2e/d45/f2a [0,4194304] 0 2024-03-22T10:43:04.481 INFO:tasks.workunit.client.1.smithi150.stdout:4/800: dwrite d0/d1b/f23 [0,4194304] 0 2024-03-22T10:43:04.484 INFO:tasks.workunit.client.1.smithi150.stdout:2/871: write d12/d19/db6/da9/d67/d8d/da6/fdb [3361240,48201] 0 2024-03-22T10:43:04.490 INFO:tasks.workunit.client.1.smithi150.stdout:7/753: truncate d2/d5/d7/d8/d57/d4a/f22 4092608 0 2024-03-22T10:43:04.495 INFO:tasks.workunit.client.1.smithi150.stdout:2/872: truncate d12/fbc 322879 0 2024-03-22T10:43:04.497 INFO:tasks.workunit.client.1.smithi150.stdout:1/707: truncate d1/da/d2a/d65/d78/fb1 1344168 0 2024-03-22T10:43:04.498 INFO:tasks.workunit.client.1.smithi150.stdout:4/801: read d0/d1b/f24 [1730330,86120] 0 2024-03-22T10:43:04.498 INFO:tasks.workunit.client.1.smithi150.stdout:5/738: dread d0/f11 [0,4194304] 0 2024-03-22T10:43:04.501 INFO:tasks.workunit.client.1.smithi150.stdout:7/754: creat d2/d5/d7/d8/d57/d4a/d8d/fff x:0 0 0 2024-03-22T10:43:04.508 INFO:tasks.workunit.client.1.smithi150.stdout:8/809: dwrite d0/d2/d19/f47 [0,4194304] 0 2024-03-22T10:43:04.508 INFO:tasks.workunit.client.1.smithi150.stdout:4/802: mknod d0/d47/d72/c109 0 2024-03-22T10:43:04.511 INFO:tasks.workunit.client.1.smithi150.stdout:7/755: rmdir d2/d5/d7/d8/d57 39 2024-03-22T10:43:04.513 INFO:tasks.workunit.client.1.smithi150.stdout:2/873: rename d12/d19/c31 to d12/d19/db6/da9/d61/d98/dc2/c10f 0 2024-03-22T10:43:04.519 INFO:tasks.workunit.client.1.smithi150.stdout:4/803: mkdir d0/d47/d72/de7/d10a 0 2024-03-22T10:43:04.521 INFO:tasks.workunit.client.1.smithi150.stdout:5/739: write d0/d8/dc/f5b [1622480,53201] 0 2024-03-22T10:43:04.524 INFO:tasks.workunit.client.1.smithi150.stdout:7/756: symlink d2/d5/d7/d8/d57/d4a/d8d/l100 0 2024-03-22T10:43:04.540 INFO:tasks.workunit.client.1.smithi150.stdout:5/740: truncate d0/d8/dc/de/d14/d2e/d5a/f86 754265 0 2024-03-22T10:43:04.542 INFO:tasks.workunit.client.1.smithi150.stdout:2/874: mknod d12/d19/db6/da9/d67/d3c/c110 0 2024-03-22T10:43:04.543 INFO:tasks.workunit.client.1.smithi150.stdout:7/757: dread d2/d5/d7/f2b [4194304,4194304] 0 2024-03-22T10:43:04.549 INFO:tasks.workunit.client.1.smithi150.stdout:4/804: dwrite d0/d1b/d4d/d6c/d61/f9e [0,4194304] 0 2024-03-22T10:43:04.550 INFO:tasks.workunit.client.1.smithi150.stdout:5/741: rename d0/d37/db4/lc6 to d0/d8/dc/de/d96/lf4 0 2024-03-22T10:43:04.553 INFO:tasks.workunit.client.1.smithi150.stdout:4/805: symlink d0/d2f/d63/d7b/d9c/db1/dd7/l10b 0 2024-03-22T10:43:04.559 INFO:tasks.workunit.client.1.smithi150.stdout:5/742: creat d0/d8/dc/de/d14/d2e/d46/d9c/dc0/ff5 x:0 0 0 2024-03-22T10:43:04.567 INFO:tasks.workunit.client.1.smithi150.stdout:5/743: chown d0/d37/cd4 8 1 2024-03-22T10:43:04.571 INFO:tasks.workunit.client.1.smithi150.stdout:8/810: dread d0/d2/d7e/f82 [0,4194304] 0 2024-03-22T10:43:04.576 INFO:tasks.workunit.client.1.smithi150.stdout:2/875: dwrite d12/d19/db6/da9/d61/fe9 [0,4194304] 0 2024-03-22T10:43:04.579 INFO:tasks.workunit.client.1.smithi150.stdout:5/744: rename d0/ddc/de8 to d0/df6 0 2024-03-22T10:43:04.580 INFO:tasks.workunit.client.1.smithi150.stdout:7/758: dwrite d2/d5/d7/d8/d46/f66 [4194304,4194304] 0 2024-03-22T10:43:04.581 INFO:tasks.workunit.client.1.smithi150.stdout:8/811: read - d0/d2/d19/d29/d2f/df/fa3 zero size 2024-03-22T10:43:04.589 INFO:tasks.workunit.client.1.smithi150.stdout:4/806: write d0/d47/f96 [306147,12024] 0 2024-03-22T10:43:04.595 INFO:tasks.workunit.client.1.smithi150.stdout:7/759: creat d2/d5/d7/d8/d57/d4a/d8d/f101 x:0 0 0 2024-03-22T10:43:04.597 INFO:tasks.workunit.client.1.smithi150.stdout:5/745: symlink d0/d8/dc/de/d14/d2e/d46/d4d/lf7 0 2024-03-22T10:43:04.598 INFO:tasks.workunit.client.1.smithi150.stdout:4/807: rmdir d0/dce 39 2024-03-22T10:43:04.598 INFO:tasks.workunit.client.1.smithi150.stdout:7/760: write d2/d5/d7/fc1 [1038514,2293] 0 2024-03-22T10:43:04.599 INFO:tasks.workunit.client.1.smithi150.stdout:2/876: link d12/d21/lac d12/d19/db6/d63/dd8/l111 0 2024-03-22T10:43:04.603 INFO:tasks.workunit.client.1.smithi150.stdout:4/808: mkdir d0/d2f/d63/d7b/d10c 0 2024-03-22T10:43:04.605 INFO:tasks.workunit.client.1.smithi150.stdout:4/809: fdatasync d0/de/f81 0 2024-03-22T10:43:04.607 INFO:tasks.workunit.client.1.smithi150.stdout:2/877: mknod d12/d19/db6/da9/d61/c112 0 2024-03-22T10:43:04.609 INFO:tasks.workunit.client.1.smithi150.stdout:2/878: fdatasync d12/d19/db6/d63/dd8/fe8 0 2024-03-22T10:43:04.619 INFO:tasks.workunit.client.1.smithi150.stdout:4/810: rename d0/d1b to d0/d2f/d10d 0 2024-03-22T10:43:04.619 INFO:tasks.workunit.client.1.smithi150.stdout:8/812: write d0/d2/d19/d29/d2f/f3b [1717403,67660] 0 2024-03-22T10:43:04.622 INFO:tasks.workunit.client.1.smithi150.stdout:3/719: sync 2024-03-22T10:43:04.628 INFO:tasks.workunit.client.1.smithi150.stdout:8/813: unlink d0/d5d/f6c 0 2024-03-22T10:43:04.632 INFO:tasks.workunit.client.1.smithi150.stdout:4/811: dread - d0/d2f/d10d/d4d/d9f/fef zero size 2024-03-22T10:43:04.632 INFO:tasks.workunit.client.1.smithi150.stdout:7/761: dwrite d2/d5/d7/d8/d57/d54/d91/dad/fcf [0,4194304] 0 2024-03-22T10:43:04.642 INFO:tasks.workunit.client.1.smithi150.stdout:7/762: chown d2/d5/d7/d8/d46/f66 549378 1 2024-03-22T10:43:04.643 INFO:tasks.workunit.client.1.smithi150.stdout:2/879: dwrite d12/fb7 [0,4194304] 0 2024-03-22T10:43:04.643 INFO:tasks.workunit.client.1.smithi150.stdout:3/720: truncate d5/d22/d2f/d38/d3e/d5b/da6/fab 1671400 0 2024-03-22T10:43:04.649 INFO:tasks.workunit.client.1.smithi150.stdout:4/812: fsync d0/d2f/f52 0 2024-03-22T10:43:04.652 INFO:tasks.workunit.client.1.smithi150.stdout:8/814: dread d0/d2/d19/d29/d2f/f7b [0,4194304] 0 2024-03-22T10:43:04.653 INFO:tasks.workunit.client.1.smithi150.stdout:3/721: dread - d5/d22/d2f/d38/f85 zero size 2024-03-22T10:43:04.659 INFO:tasks.workunit.client.1.smithi150.stdout:3/722: mkdir d5/d22/d2f/d38/d3e/d5b/dee 0 2024-03-22T10:43:04.663 INFO:tasks.workunit.client.1.smithi150.stdout:5/746: dread d0/d8/f9a [0,4194304] 0 2024-03-22T10:43:04.667 INFO:tasks.workunit.client.1.smithi150.stdout:4/813: readlink d0/d2f/d63/d7b/d9c/db1/dd7/l10b 0 2024-03-22T10:43:04.668 INFO:tasks.workunit.client.1.smithi150.stdout:8/815: rename d0/d2/d19/d29/d2f/df/f26 to d0/d3c/d71/d7a/d89/f10b 0 2024-03-22T10:43:04.670 INFO:tasks.workunit.client.1.smithi150.stdout:2/880: getdents d12/d35/d7d 0 2024-03-22T10:43:04.676 INFO:tasks.workunit.client.1.smithi150.stdout:5/747: rename d0/d8/dc/de/d14/d2e/d46/d4d/f6c to d0/dae/ff8 0 2024-03-22T10:43:04.678 INFO:tasks.workunit.client.1.smithi150.stdout:8/816: mknod d0/ddd/dfc/c10c 0 2024-03-22T10:43:04.683 INFO:tasks.workunit.client.1.smithi150.stdout:3/723: rmdir d5/d22/d37/ddd 0 2024-03-22T10:43:04.684 INFO:tasks.workunit.client.1.smithi150.stdout:4/814: mkdir d0/dce/d10e 0 2024-03-22T10:43:04.687 INFO:tasks.workunit.client.1.smithi150.stdout:2/881: getdents d12/d19 0 2024-03-22T10:43:04.692 INFO:tasks.workunit.client.1.smithi150.stdout:7/763: dwrite d2/d5/d7/d9/db/d9f/f8b [0,4194304] 0 2024-03-22T10:43:04.692 INFO:tasks.workunit.client.1.smithi150.stdout:4/815: chown d0/d2f/laf 113638705 1 2024-03-22T10:43:04.693 INFO:tasks.workunit.client.1.smithi150.stdout:3/724: write d5/d22/d37/fd8 [983603,39327] 0 2024-03-22T10:43:04.695 INFO:tasks.workunit.client.1.smithi150.stdout:8/817: truncate d0/d2/d19/d29/d2e/fb7 473052 0 2024-03-22T10:43:04.697 INFO:tasks.workunit.client.1.smithi150.stdout:4/816: chown d0/d2f/d63/c7a 1 1 2024-03-22T10:43:04.703 INFO:tasks.workunit.client.1.smithi150.stdout:8/818: unlink d0/d2/d19/d29/d88/lc4 0 2024-03-22T10:43:04.705 INFO:tasks.workunit.client.1.smithi150.stdout:3/725: dread d5/d22/d2f/d38/fb4 [0,4194304] 0 2024-03-22T10:43:04.710 INFO:tasks.workunit.client.1.smithi150.stdout:2/882: link d12/d19/db6/da9/d67/d8d/da6/ffe d12/d19/db6/df1/f113 0 2024-03-22T10:43:04.713 INFO:tasks.workunit.client.1.smithi150.stdout:7/764: unlink d2/d5/d7/d8/d57/d4a/l1d 0 2024-03-22T10:43:04.716 INFO:tasks.workunit.client.1.smithi150.stdout:7/765: chown d2/d5/d7/d9/db/d6c/d9c 527248 1 2024-03-22T10:43:04.719 INFO:tasks.workunit.client.1.smithi150.stdout:3/726: mkdir d5/d22/d2f/d38/d59/def 0 2024-03-22T10:43:04.719 INFO:tasks.workunit.client.1.smithi150.stdout:9/811: sync 2024-03-22T10:43:04.719 INFO:tasks.workunit.client.1.smithi150.stdout:0/756: sync 2024-03-22T10:43:04.721 INFO:tasks.workunit.client.1.smithi150.stdout:2/883: fsync d12/d19/db6/df1/f108 0 2024-03-22T10:43:04.721 INFO:tasks.workunit.client.1.smithi150.stdout:4/817: dwrite d0/d2f/d10d/d4d/d6c/d61/dd1/fec [0,4194304] 0 2024-03-22T10:43:04.722 INFO:tasks.workunit.client.1.smithi150.stdout:7/766: chown d2/d5/d7/d9/db/c77 97 1 2024-03-22T10:43:04.726 INFO:tasks.workunit.client.1.smithi150.stdout:8/819: truncate d0/d2/d19/d29/d2f/df/d1b/fa0 189295 0 2024-03-22T10:43:04.731 INFO:tasks.workunit.client.1.smithi150.stdout:5/748: write d0/d8/dc/de/d14/d2e/f54 [2694172,31881] 0 2024-03-22T10:43:04.733 INFO:tasks.workunit.client.1.smithi150.stdout:3/727: creat d5/da/d62/ff0 x:0 0 0 2024-03-22T10:43:04.735 INFO:tasks.workunit.client.1.smithi150.stdout:2/884: mkdir d12/d19/db6/da9/d67/d9a/d114 0 2024-03-22T10:43:04.742 INFO:tasks.workunit.client.1.smithi150.stdout:7/767: rename d2/d5/d7/d9/db/d9f/d45/l90 to d2/d5/d7/d8/d57/d4a/d8d/l102 0 2024-03-22T10:43:04.744 INFO:tasks.workunit.client.1.smithi150.stdout:4/818: creat d0/dce/d10e/f10f x:0 0 0 2024-03-22T10:43:04.748 INFO:tasks.workunit.client.1.smithi150.stdout:5/749: mkdir d0/dae/db2/df9 0 2024-03-22T10:43:04.748 INFO:tasks.workunit.client.1.smithi150.stdout:3/728: mknod d5/d22/d2f/d38/d3e/d5b/cf1 0 2024-03-22T10:43:04.750 INFO:tasks.workunit.client.1.smithi150.stdout:2/885: rename d12/d19/db6/d54/fe5 to d12/da1/f115 0 2024-03-22T10:43:04.755 INFO:tasks.workunit.client.1.smithi150.stdout:9/812: write d1/d8/fde [3692832,107170] 0 2024-03-22T10:43:04.755 INFO:tasks.workunit.client.1.smithi150.stdout:2/886: write d12/d19/db6/da9/d61/fe9 [1464603,118911] 0 2024-03-22T10:43:04.756 INFO:tasks.workunit.client.1.smithi150.stdout:5/750: creat d0/ddc/ffa x:0 0 0 2024-03-22T10:43:04.756 INFO:tasks.workunit.client.1.smithi150.stdout:4/819: read d0/de/f70 [455524,40912] 0 2024-03-22T10:43:04.758 INFO:tasks.workunit.client.1.smithi150.stdout:9/813: write d1/d49/f109 [417885,115082] 0 2024-03-22T10:43:04.759 INFO:tasks.workunit.client.1.smithi150.stdout:4/820: readlink d0/l29 0 2024-03-22T10:43:04.761 INFO:tasks.workunit.client.1.smithi150.stdout:5/751: dread - d0/d33/f97 zero size 2024-03-22T10:43:04.768 INFO:tasks.workunit.client.1.smithi150.stdout:2/887: fsync d12/d19/d2d/faa 0 2024-03-22T10:43:04.768 INFO:tasks.workunit.client.1.smithi150.stdout:7/768: dwrite d2/d5/d7/d9/db/d9f/f8b [0,4194304] 0 2024-03-22T10:43:04.773 INFO:tasks.workunit.client.1.smithi150.stdout:4/821: write d0/d2f/d63/d7b/d9c/fed [141776,126388] 0 2024-03-22T10:43:04.774 INFO:tasks.workunit.client.1.smithi150.stdout:9/814: creat d1/d8/de/df/d36/f116 x:0 0 0 2024-03-22T10:43:04.775 INFO:tasks.workunit.client.1.smithi150.stdout:8/820: dwrite d0/d2/d19/d29/d88/f9a [4194304,4194304] 0 2024-03-22T10:43:04.779 INFO:tasks.workunit.client.1.smithi150.stdout:2/888: truncate d12/d19/db6/d54/f5b 1426811 0 2024-03-22T10:43:04.780 INFO:tasks.workunit.client.1.smithi150.stdout:3/729: dwrite d5/d22/d2f/d38/d3e/d5b/da6/fbd [0,4194304] 0 2024-03-22T10:43:04.780 INFO:tasks.workunit.client.1.smithi150.stdout:7/769: write d2/d5/d7/d9/db/d9f/f40 [4733981,118870] 0 2024-03-22T10:43:04.785 INFO:tasks.workunit.client.1.smithi150.stdout:7/770: write d2/d5/d7/fc1 [1009737,83641] 0 2024-03-22T10:43:04.789 INFO:tasks.workunit.client.1.smithi150.stdout:7/771: chown d2/d5/d7/d8/d46/d49 115 1 2024-03-22T10:43:04.797 INFO:tasks.workunit.client.1.smithi150.stdout:2/889: rmdir d12/da1 39 2024-03-22T10:43:04.797 INFO:tasks.workunit.client.1.smithi150.stdout:3/730: mknod d5/da/d78/db7/cf2 0 2024-03-22T10:43:04.799 INFO:tasks.workunit.client.1.smithi150.stdout:4/822: write d0/d2f/d10d/d4d/d6c/d61/d71/fa4 [5000404,118295] 0 2024-03-22T10:43:04.801 INFO:tasks.workunit.client.1.smithi150.stdout:5/752: write d0/d8/dc/de/d14/d2e/f62 [2665366,12176] 0 2024-03-22T10:43:04.806 INFO:tasks.workunit.client.1.smithi150.stdout:4/823: write d0/d32/d7c/f91 [8507355,70159] 0 2024-03-22T10:43:04.811 INFO:tasks.workunit.client.1.smithi150.stdout:3/731: mknod d5/da/d9d/d9f/cf3 0 2024-03-22T10:43:04.819 INFO:tasks.workunit.client.1.smithi150.stdout:3/732: rename d5/d22/d2f/d38/d3e/fb6 to d5/d22/d37/ff4 0 2024-03-22T10:43:04.824 INFO:tasks.workunit.client.1.smithi150.stdout:4/824: write d0/d2f/d63/d7b/d9c/db1/dd7/fd9 [401895,126759] 0 2024-03-22T10:43:04.840 INFO:tasks.workunit.client.1.smithi150.stdout:8/821: dwrite d0/d2/fe [0,4194304] 0 2024-03-22T10:43:04.840 INFO:tasks.workunit.client.1.smithi150.stdout:4/825: mknod d0/d2f/d63/d7b/d9c/db1/dd7/c110 0 2024-03-22T10:43:04.841 INFO:tasks.workunit.client.1.smithi150.stdout:2/890: dwrite d12/d19/db6/da9/d67/f51 [0,4194304] 0 2024-03-22T10:43:04.842 INFO:tasks.workunit.client.1.smithi150.stdout:7/772: dwrite d2/d5/d7/d9/db/d6c/d9c/f9d [0,4194304] 0 2024-03-22T10:43:04.844 INFO:tasks.workunit.client.1.smithi150.stdout:5/753: dread d0/d8/dc/de/d14/d4f/d47/fa0 [0,4194304] 0 2024-03-22T10:43:04.849 INFO:tasks.workunit.client.1.smithi150.stdout:4/826: creat d0/dce/d10e/f111 x:0 0 0 2024-03-22T10:43:04.851 INFO:tasks.workunit.client.1.smithi150.stdout:8/822: chown d0/d2/d19/d29/d2f/df/d1b 497 1 2024-03-22T10:43:04.852 INFO:tasks.workunit.client.1.smithi150.stdout:7/773: dread - d2/d5/d7/d88/f94 zero size 2024-03-22T10:43:04.852 INFO:tasks.workunit.client.1.smithi150.stdout:9/815: dread d1/d8/de/df/da2/fbf [4194304,4194304] 0 2024-03-22T10:43:04.857 INFO:tasks.workunit.client.1.smithi150.stdout:5/754: mkdir d0/d8/dc/de/d14/d4f/d47/dfb 0 2024-03-22T10:43:04.858 INFO:tasks.workunit.client.1.smithi150.stdout:4/827: creat d0/d2f/d63/f112 x:0 0 0 2024-03-22T10:43:04.859 INFO:tasks.workunit.client.1.smithi150.stdout:3/733: dwrite d5/d22/d37/f6c [4194304,4194304] 0 2024-03-22T10:43:04.864 INFO:tasks.workunit.client.1.smithi150.stdout:7/774: write d2/d5/d7/d8/d57/d4a/d8d/ff5 [221790,15618] 0 2024-03-22T10:43:04.865 INFO:tasks.workunit.client.1.smithi150.stdout:8/823: rename d0/d2/d19/d29/d37/da1/l10a to d0/d2/d19/d29/d37/dc2/d107/l10d 0 2024-03-22T10:43:04.868 INFO:tasks.workunit.client.1.smithi150.stdout:5/755: rmdir d0/d33 39 2024-03-22T10:43:04.873 INFO:tasks.workunit.client.1.smithi150.stdout:8/824: truncate d0/d5d/dd9/fe9 869412 0 2024-03-22T10:43:04.875 INFO:tasks.workunit.client.1.smithi150.stdout:5/756: chown d0/d8/dc/d1d/l61 11120 1 2024-03-22T10:43:04.885 INFO:tasks.workunit.client.1.smithi150.stdout:2/891: link d12/d19/db6/da9/d61/d98/dc2/c10f d12/da1/dd6/de0/c116 0 2024-03-22T10:43:04.888 INFO:tasks.workunit.client.1.smithi150.stdout:5/757: fsync d0/d8/dc/f40 0 2024-03-22T10:43:04.889 INFO:tasks.workunit.client.1.smithi150.stdout:3/734: getdents d5/d22/d2f/d38/d59 0 2024-03-22T10:43:04.893 INFO:tasks.workunit.client.1.smithi150.stdout:2/892: write d12/d21/d81/fdf [4832280,116503] 0 2024-03-22T10:43:04.895 INFO:tasks.workunit.client.1.smithi150.stdout:4/828: dwrite d0/d32/fae [4194304,4194304] 0 2024-03-22T10:43:04.901 INFO:tasks.workunit.client.1.smithi150.stdout:6/732: sync 2024-03-22T10:43:04.901 INFO:tasks.workunit.client.1.smithi150.stdout:1/708: sync 2024-03-22T10:43:04.901 INFO:tasks.workunit.client.1.smithi150.stdout:8/825: write d0/d2/f67 [1364303,127810] 0 2024-03-22T10:43:04.902 INFO:tasks.workunit.client.1.smithi150.stdout:9/816: dwrite d1/d8/de/df/d36/d40/d44/fa3 [0,4194304] 0 2024-03-22T10:43:04.913 INFO:tasks.workunit.client.1.smithi150.stdout:2/893: read - d12/d19/db6/da9/d67/d3c/fab zero size 2024-03-22T10:43:04.914 INFO:tasks.workunit.client.1.smithi150.stdout:7/775: dwrite d2/d5/d7/f50 [4194304,4194304] 0 2024-03-22T10:43:04.923 INFO:tasks.workunit.client.1.smithi150.stdout:8/826: symlink d0/d2/d7e/l10e 0 2024-03-22T10:43:04.924 INFO:tasks.workunit.client.1.smithi150.stdout:4/829: chown d0/d2f/d63/d7b/l97 506 1 2024-03-22T10:43:04.924 INFO:tasks.workunit.client.1.smithi150.stdout:1/709: dread d1/d19/d50/d6e/f88 [0,4194304] 0 2024-03-22T10:43:04.925 INFO:tasks.workunit.client.1.smithi150.stdout:7/776: symlink d2/d5/d7/d38/l103 0 2024-03-22T10:43:04.929 INFO:tasks.workunit.client.1.smithi150.stdout:3/735: truncate d5/d22/d2f/f41 7646 0 2024-03-22T10:43:04.933 INFO:tasks.workunit.client.1.smithi150.stdout:8/827: mknod d0/d2/d19/d29/d37/c10f 0 2024-03-22T10:43:04.934 INFO:tasks.workunit.client.1.smithi150.stdout:7/777: write d2/d5/d7/d8/d57/d54/d91/fc6 [1010072,48087] 0 2024-03-22T10:43:04.934 INFO:tasks.workunit.client.1.smithi150.stdout:9/817: getdents d1/d19/d7d/dc1/dca/dd7 0 2024-03-22T10:43:04.935 INFO:tasks.workunit.client.1.smithi150.stdout:5/758: dwrite d0/dae/db2/ff0 [0,4194304] 0 2024-03-22T10:43:04.941 INFO:tasks.workunit.client.1.smithi150.stdout:4/830: creat d0/d2f/d10d/d4d/d6c/f113 x:0 0 0 2024-03-22T10:43:04.941 INFO:tasks.workunit.client.1.smithi150.stdout:1/710: creat d1/d19/d71/fe8 x:0 0 0 2024-03-22T10:43:04.942 INFO:tasks.workunit.client.1.smithi150.stdout:9/818: dread - d1/d19/fea zero size 2024-03-22T10:43:04.942 INFO:tasks.workunit.client.1.smithi150.stdout:2/894: symlink d12/d19/db6/d63/l117 0 2024-03-22T10:43:04.945 INFO:tasks.workunit.client.1.smithi150.stdout:6/733: write db/dd/fa4 [3053594,25609] 0 2024-03-22T10:43:04.947 INFO:tasks.workunit.client.1.smithi150.stdout:7/778: mknod d2/d5/d7/d9/c104 0 2024-03-22T10:43:04.951 INFO:tasks.workunit.client.1.smithi150.stdout:5/759: unlink d0/d8/dc/de/d14/d2e/d45/c1e 0 2024-03-22T10:43:04.953 INFO:tasks.workunit.client.1.smithi150.stdout:1/711: creat d1/da/d2a/d55/db8/fe9 x:0 0 0 2024-03-22T10:43:04.953 INFO:tasks.workunit.client.1.smithi150.stdout:4/831: truncate d0/d32/f105 921374 0 2024-03-22T10:43:04.954 INFO:tasks.workunit.client.1.smithi150.stdout:9/819: stat d1/d8/c68 0 2024-03-22T10:43:04.955 INFO:tasks.workunit.client.1.smithi150.stdout:6/734: write db/d52/d9f/dc6/df1/fa1 [1270781,102220] 0 2024-03-22T10:43:04.962 INFO:tasks.workunit.client.1.smithi150.stdout:0/757: sync 2024-03-22T10:43:04.962 INFO:tasks.workunit.client.1.smithi150.stdout:7/779: mkdir d2/d5/d7/d8/d57/d105 0 2024-03-22T10:43:04.965 INFO:tasks.workunit.client.1.smithi150.stdout:4/832: unlink d0/d2f/d10d/d4d/d6c/f9d 0 2024-03-22T10:43:04.968 INFO:tasks.workunit.client.1.smithi150.stdout:5/760: link d0/dae/de0/fcc d0/dae/de0/ffc 0 2024-03-22T10:43:04.970 INFO:tasks.workunit.client.1.smithi150.stdout:3/736: dwrite d5/da/d9d/fc5 [0,4194304] 0 2024-03-22T10:43:04.971 INFO:tasks.workunit.client.1.smithi150.stdout:9/820: mknod d1/d8/de/dbd/c117 0 2024-03-22T10:43:04.974 INFO:tasks.workunit.client.1.smithi150.stdout:0/758: stat d0/d1/d84/d86/d42/d10/d38/d52/d9b/dac/ldb 0 2024-03-22T10:43:04.975 INFO:tasks.workunit.client.1.smithi150.stdout:5/761: read - d0/d8/dc/de/d14/d2e/d5a/d66/da9/fe5 zero size 2024-03-22T10:43:04.977 INFO:tasks.workunit.client.1.smithi150.stdout:1/712: rename d1/da/d15/d18/lb2 to d1/d19/d50/d6e/lea 0 2024-03-22T10:43:04.979 INFO:tasks.workunit.client.1.smithi150.stdout:4/833: symlink d0/d2f/l114 0 2024-03-22T10:43:04.980 INFO:tasks.workunit.client.1.smithi150.stdout:9/821: creat d1/d8/de/d73/d101/d29/f118 x:0 0 0 2024-03-22T10:43:04.981 INFO:tasks.workunit.client.1.smithi150.stdout:8/828: dwrite d0/d2/d19/d29/d2e/d70/fad [0,4194304] 0 2024-03-22T10:43:04.981 INFO:tasks.workunit.client.1.smithi150.stdout:2/895: link d12/d19/db6/da9/d67/d8d/da6/cd0 d12/d19/db6/da9/d67/d3c/c118 0 2024-03-22T10:43:04.988 INFO:tasks.workunit.client.1.smithi150.stdout:9/822: chown d1/d8/de/df/d24/f53 2213734 1 2024-03-22T10:43:04.989 INFO:tasks.workunit.client.1.smithi150.stdout:6/735: rename db/dd/d15/d60/f53 to db/dd/d15/d28/d6c/d9b/f106 0 2024-03-22T10:43:04.990 INFO:tasks.workunit.client.1.smithi150.stdout:0/759: symlink d0/d1/lf1 0 2024-03-22T10:43:04.992 INFO:tasks.workunit.client.1.smithi150.stdout:6/736: write db/d52/f6f [866063,48345] 0 2024-03-22T10:43:04.996 INFO:tasks.workunit.client.1.smithi150.stdout:7/780: rename d2/d5/d7/d8/d57/d54 to d2/d5/d7/d9/db/d9f/d96/d71/d106 0 2024-03-22T10:43:04.999 INFO:tasks.workunit.client.1.smithi150.stdout:0/760: truncate d0/d1/d84/d86/d11/d66/d35/d5f/fd9 7272288 0 2024-03-22T10:43:05.000 INFO:tasks.workunit.client.1.smithi150.stdout:9/823: creat d1/d8/de/d73/d101/d32/f119 x:0 0 0 2024-03-22T10:43:05.003 INFO:tasks.workunit.client.1.smithi150.stdout:4/834: rename d0/d2f/d63/d7b/d9c/db1/f100 to d0/d47/d72/de7/d10a/f115 0 2024-03-22T10:43:05.006 INFO:tasks.workunit.client.1.smithi150.stdout:5/762: getdents d0/d8/dc/de/d14/d2e 0 2024-03-22T10:43:05.012 INFO:tasks.workunit.client.1.smithi150.stdout:3/737: write d5/d22/d2f/f41 [882515,7879] 0 2024-03-22T10:43:05.013 INFO:tasks.workunit.client.1.smithi150.stdout:3/738: fsync d5/fd 0 2024-03-22T10:43:05.015 INFO:tasks.workunit.client.1.smithi150.stdout:8/829: rename d0/d2/d7e to d0/d2/d19/d29/d2f/df/dff/d110 0 2024-03-22T10:43:05.017 INFO:tasks.workunit.client.1.smithi150.stdout:2/896: dread d12/d19/db6/df1/f108 [0,4194304] 0 2024-03-22T10:43:05.018 INFO:tasks.workunit.client.1.smithi150.stdout:4/835: rmdir d0/d2f/d10d/d4d/d6c/d61/d71 39 2024-03-22T10:43:05.018 INFO:tasks.workunit.client.1.smithi150.stdout:1/713: write d1/da/d15/d18/f3d [5079704,37732] 0 2024-03-22T10:43:05.020 INFO:tasks.workunit.client.1.smithi150.stdout:9/824: mkdir d1/d8/de/df/d11a 0 2024-03-22T10:43:05.022 INFO:tasks.workunit.client.1.smithi150.stdout:5/763: creat d0/d8/dc/de/d14/d2e/d46/d9c/ffd x:0 0 0 2024-03-22T10:43:05.026 INFO:tasks.workunit.client.1.smithi150.stdout:3/739: fdatasync d5/d22/d37/ff4 0 2024-03-22T10:43:05.027 INFO:tasks.workunit.client.1.smithi150.stdout:5/764: write d0/d8/dc/de/d14/d2e/d46/d9c/ffd [182237,49169] 0 2024-03-22T10:43:05.034 INFO:tasks.workunit.client.1.smithi150.stdout:6/737: truncate db/dd/d25/f45 2128525 0 2024-03-22T10:43:05.034 INFO:tasks.workunit.client.1.smithi150.stdout:4/836: unlink d0/d32/c77 0 2024-03-22T10:43:05.035 INFO:tasks.workunit.client.1.smithi150.stdout:2/897: mknod d12/d35/d7d/c119 0 2024-03-22T10:43:05.036 INFO:tasks.workunit.client.1.smithi150.stdout:9/825: creat d1/d8/de/df/d24/d4e/f11b x:0 0 0 2024-03-22T10:43:05.036 INFO:tasks.workunit.client.1.smithi150.stdout:7/781: link d2/d5/d7/c80 d2/d5/d7/d8/d57/d4a/d8d/da9/dcb/c107 0 2024-03-22T10:43:05.036 INFO:tasks.workunit.client.1.smithi150.stdout:0/761: creat d0/d1/d84/d86/d42/d10/d38/ff2 x:0 0 0 2024-03-22T10:43:05.037 INFO:tasks.workunit.client.1.smithi150.stdout:8/830: mknod d0/d2/d46/de3/c111 0 2024-03-22T10:43:05.037 INFO:tasks.workunit.client.1.smithi150.stdout:6/738: stat db/d52/f93 0 2024-03-22T10:43:05.038 INFO:tasks.workunit.client.1.smithi150.stdout:2/898: chown f11 25822550 1 2024-03-22T10:43:05.040 INFO:tasks.workunit.client.1.smithi150.stdout:5/765: truncate d0/d8/dc/de/d14/d2e/d45/dac/fbd 860068 0 2024-03-22T10:43:05.041 INFO:tasks.workunit.client.1.smithi150.stdout:7/782: chown d2/d5/d7/d9/c31 189 1 2024-03-22T10:43:05.043 INFO:tasks.workunit.client.1.smithi150.stdout:3/740: link d5/da/d8d/fbb d5/da/d62/ff5 0 2024-03-22T10:43:05.045 INFO:tasks.workunit.client.1.smithi150.stdout:0/762: dread - d0/d1/d84/d86/d42/fe3 zero size 2024-03-22T10:43:05.046 INFO:tasks.workunit.client.1.smithi150.stdout:1/714: dread d1/d7c/fb5 [0,4194304] 0 2024-03-22T10:43:05.048 INFO:tasks.workunit.client.1.smithi150.stdout:9/826: truncate d1/d8/de/d73/d101/d29/f103 576851 0 2024-03-22T10:43:05.048 INFO:tasks.workunit.client.1.smithi150.stdout:0/763: chown d0/d1/d84/d86/l99 56 1 2024-03-22T10:43:05.048 INFO:tasks.workunit.client.1.smithi150.stdout:8/831: write d0/d5d/dd2/fb4 [1065820,57285] 0 2024-03-22T10:43:05.054 INFO:tasks.workunit.client.1.smithi150.stdout:2/899: creat d12/d19/d2d/dd7/f11a x:0 0 0 2024-03-22T10:43:05.057 INFO:tasks.workunit.client.1.smithi150.stdout:9/827: write d1/d8/de/d73/d101/f50 [616414,33997] 0 2024-03-22T10:43:05.058 INFO:tasks.workunit.client.1.smithi150.stdout:4/837: rename d0/d32 to d0/d2f/d116 0 2024-03-22T10:43:05.061 INFO:tasks.workunit.client.1.smithi150.stdout:9/828: fsync d1/d8/de/d73/f74 0 2024-03-22T10:43:05.062 INFO:tasks.workunit.client.1.smithi150.stdout:0/764: symlink d0/d12/lf3 0 2024-03-22T10:43:05.064 INFO:tasks.workunit.client.1.smithi150.stdout:5/766: dwrite d0/f5 [0,4194304] 0 2024-03-22T10:43:05.066 INFO:tasks.workunit.client.1.smithi150.stdout:8/832: mknod d0/d3c/d71/d7a/d89/db5/d106/c112 0 2024-03-22T10:43:05.066 INFO:tasks.workunit.client.1.smithi150.stdout:1/715: creat d1/d19/d1a/dba/feb x:0 0 0 2024-03-22T10:43:05.067 INFO:tasks.workunit.client.1.smithi150.stdout:9/829: readlink d1/d8/de/df/d36/d40/d41/l56 0 2024-03-22T10:43:05.069 INFO:tasks.workunit.client.1.smithi150.stdout:7/783: write d2/d5/d7/d9/db/d9f/d45/f7d [764286,64436] 0 2024-03-22T10:43:05.071 INFO:tasks.workunit.client.1.smithi150.stdout:3/741: creat d5/d22/ff6 x:0 0 0 2024-03-22T10:43:05.072 INFO:tasks.workunit.client.1.smithi150.stdout:8/833: unlink d0/fb6 0 2024-03-22T10:43:05.074 INFO:tasks.workunit.client.1.smithi150.stdout:0/765: write d0/d1/fbb [658613,78354] 0 2024-03-22T10:43:05.074 INFO:tasks.workunit.client.1.smithi150.stdout:2/900: write d12/d19/db6/da9/d67/f74 [182138,95019] 0 2024-03-22T10:43:05.076 INFO:tasks.workunit.client.1.smithi150.stdout:4/838: truncate d0/d47/faa 3914958 0 2024-03-22T10:43:05.078 INFO:tasks.workunit.client.1.smithi150.stdout:3/742: creat d5/da/d78/d86/ff7 x:0 0 0 2024-03-22T10:43:05.079 INFO:tasks.workunit.client.1.smithi150.stdout:6/739: dread db/dd/d25/fac [0,4194304] 0 2024-03-22T10:43:05.080 INFO:tasks.workunit.client.1.smithi150.stdout:8/834: mkdir d0/d3c/d71/d7a/d89/d113 0 2024-03-22T10:43:05.081 INFO:tasks.workunit.client.1.smithi150.stdout:1/716: symlink d1/d19/dd5/lec 0 2024-03-22T10:43:05.083 INFO:tasks.workunit.client.1.smithi150.stdout:5/767: rename d0/d8/dc/de/d96/fd9 to d0/d8/dc/de/d14/d2e/d5a/ffe 0 2024-03-22T10:43:05.086 INFO:tasks.workunit.client.1.smithi150.stdout:3/743: read d5/da/d9d/fc5 [938074,25864] 0 2024-03-22T10:43:05.088 INFO:tasks.workunit.client.1.smithi150.stdout:5/768: chown d0/d37/db4 372999 1 2024-03-22T10:43:05.089 INFO:tasks.workunit.client.1.smithi150.stdout:2/901: rename d12/d19/db6/da9/d67/d3c/fce to d12/d35/f11b 0 2024-03-22T10:43:05.090 INFO:tasks.workunit.client.1.smithi150.stdout:6/740: read db/dd/d25/f3d [1827832,55157] 0 2024-03-22T10:43:05.091 INFO:tasks.workunit.client.1.smithi150.stdout:1/717: chown d1/da/d15/d18/d4b 8 1 2024-03-22T10:43:05.092 INFO:tasks.workunit.client.1.smithi150.stdout:2/902: stat d12/d19/d2d/d66/c88 0 2024-03-22T10:43:05.094 INFO:tasks.workunit.client.1.smithi150.stdout:5/769: dread - d0/dae/db2/f90 zero size 2024-03-22T10:43:05.094 INFO:tasks.workunit.client.1.smithi150.stdout:4/839: mknod d0/d2f/d10d/d4d/c117 0 2024-03-22T10:43:05.099 INFO:tasks.workunit.client.1.smithi150.stdout:6/741: fdatasync db/d52/d9f/dc6/df1/fa1 0 2024-03-22T10:43:05.103 INFO:tasks.workunit.client.1.smithi150.stdout:4/840: readlink d0/dce/ld2 0 2024-03-22T10:43:05.104 INFO:tasks.workunit.client.1.smithi150.stdout:9/830: dwrite d1/d19/f79 [4194304,4194304] 0 2024-03-22T10:43:05.105 INFO:tasks.workunit.client.1.smithi150.stdout:2/903: readlink d12/d21/d81/lae 0 2024-03-22T10:43:05.106 INFO:tasks.workunit.client.1.smithi150.stdout:5/770: rmdir d0/d33 39 2024-03-22T10:43:05.111 INFO:tasks.workunit.client.1.smithi150.stdout:7/784: dwrite d2/d5/d7/d8/fcc [0,4194304] 0 2024-03-22T10:43:05.111 INFO:tasks.workunit.client.1.smithi150.stdout:6/742: fsync db/d52/d9f/dc6/df1/fbb 0 2024-03-22T10:43:05.113 INFO:tasks.workunit.client.1.smithi150.stdout:1/718: symlink d1/da/d2a/d55/db8/led 0 2024-03-22T10:43:05.116 INFO:tasks.workunit.client.1.smithi150.stdout:1/719: readlink d1/d19/dd5/lec 0 2024-03-22T10:43:05.123 INFO:tasks.workunit.client.1.smithi150.stdout:0/766: dwrite d0/f83 [0,4194304] 0 2024-03-22T10:43:05.123 INFO:tasks.workunit.client.1.smithi150.stdout:4/841: creat d0/d2f/d10d/d4d/d6c/d61/dd1/f118 x:0 0 0 2024-03-22T10:43:05.125 INFO:tasks.workunit.client.1.smithi150.stdout:1/720: write d1/d19/d50/d6e/fb9 [916689,65764] 0 2024-03-22T10:43:05.132 INFO:tasks.workunit.client.1.smithi150.stdout:6/743: chown db/cd3 152417 1 2024-03-22T10:43:05.132 INFO:tasks.workunit.client.1.smithi150.stdout:5/771: symlink d0/d8/dc/de/d14/d2e/d5a/lff 0 2024-03-22T10:43:05.134 INFO:tasks.workunit.client.1.smithi150.stdout:6/744: stat db/dd/d15/d70/ff5 0 2024-03-22T10:43:05.134 INFO:tasks.workunit.client.1.smithi150.stdout:8/835: dread d0/d3c/f4a [0,4194304] 0 2024-03-22T10:43:05.138 INFO:tasks.workunit.client.1.smithi150.stdout:0/767: mknod d0/d1/d84/d86/d11/d66/d35/d5f/cf4 0 2024-03-22T10:43:05.139 INFO:tasks.workunit.client.1.smithi150.stdout:8/836: chown d0/d2/d46/f4e 20574834 1 2024-03-22T10:43:05.142 INFO:tasks.workunit.client.1.smithi150.stdout:7/785: dwrite d2/d5/d7/d9/db/d9f/f40 [0,4194304] 0 2024-03-22T10:43:05.150 INFO:tasks.workunit.client.1.smithi150.stdout:5/772: mkdir d0/d33/d100 0 2024-03-22T10:43:05.151 INFO:tasks.workunit.client.1.smithi150.stdout:2/904: link d12/l29 d12/d19/db6/da9/d97/l11c 0 2024-03-22T10:43:05.158 INFO:tasks.workunit.client.1.smithi150.stdout:9/831: dread d1/d8/de/df/d36/f43 [0,4194304] 0 2024-03-22T10:43:05.162 INFO:tasks.workunit.client.1.smithi150.stdout:4/842: link d0/de/f5b d0/d2f/d63/d7b/f119 0 2024-03-22T10:43:05.162 INFO:tasks.workunit.client.1.smithi150.stdout:1/721: write d1/da/d15/d18/f5f [382714,98997] 0 2024-03-22T10:43:05.166 INFO:tasks.workunit.client.1.smithi150.stdout:3/744: dread d5/d22/d2f/f68 [0,4194304] 0 2024-03-22T10:43:05.168 INFO:tasks.workunit.client.1.smithi150.stdout:2/905: write d12/d19/db6/da9/d97/d78/dbd/fee [839623,102746] 0 2024-03-22T10:43:05.169 INFO:tasks.workunit.client.1.smithi150.stdout:4/843: stat d0/d47/d72 0 2024-03-22T10:43:05.170 INFO:tasks.workunit.client.1.smithi150.stdout:7/786: creat d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/daf/f108 x:0 0 0 2024-03-22T10:43:05.170 INFO:tasks.workunit.client.1.smithi150.stdout:1/722: chown d1/d7c 28890326 1 2024-03-22T10:43:05.175 INFO:tasks.workunit.client.1.smithi150.stdout:8/837: dread d0/d2/d19/d29/d2e/d70/fad [0,4194304] 0 2024-03-22T10:43:05.177 INFO:tasks.workunit.client.1.smithi150.stdout:2/906: creat d12/d35/d109/f11d x:0 0 0 2024-03-22T10:43:05.183 INFO:tasks.workunit.client.1.smithi150.stdout:9/832: read d1/d8/de/df/da2/f91 [1686289,65583] 0 2024-03-22T10:43:05.184 INFO:tasks.workunit.client.1.smithi150.stdout:5/773: dwrite d0/d8/dc/de/d14/d2e/d45/dac/faf [0,4194304] 0 2024-03-22T10:43:05.187 INFO:tasks.workunit.client.1.smithi150.stdout:6/745: dwrite db/dd/d15/d28/f41 [0,4194304] 0 2024-03-22T10:43:05.190 INFO:tasks.workunit.client.1.smithi150.stdout:3/745: rename d5/da/d78/d86/ff7 to d5/d22/d2f/d38/d3e/d72/ff8 0 2024-03-22T10:43:05.194 INFO:tasks.workunit.client.1.smithi150.stdout:9/833: chown d1/d8/de/d73/d101/d32/lf6 143982 1 2024-03-22T10:43:05.196 INFO:tasks.workunit.client.1.smithi150.stdout:2/907: truncate d12/d19/db6/da9/d61/f86 3526088 0 2024-03-22T10:43:05.198 INFO:tasks.workunit.client.1.smithi150.stdout:7/787: unlink d2/d5/d7/d8/d57/c6a 0 2024-03-22T10:43:05.199 INFO:tasks.workunit.client.1.smithi150.stdout:6/746: creat db/dd/d15/d6a/dde/f107 x:0 0 0 2024-03-22T10:43:05.201 INFO:tasks.workunit.client.1.smithi150.stdout:6/747: stat db/d52/dc3 0 2024-03-22T10:43:05.202 INFO:tasks.workunit.client.1.smithi150.stdout:0/768: dwrite d0/d1/d84/d86/d42/d10/f43 [0,4194304] 0 2024-03-22T10:43:05.204 INFO:tasks.workunit.client.1.smithi150.stdout:4/844: dwrite d0/dce/d10e/f111 [0,4194304] 0 2024-03-22T10:43:05.212 INFO:tasks.workunit.client.1.smithi150.stdout:5/774: mkdir d0/d8/d101 0 2024-03-22T10:43:05.217 INFO:tasks.workunit.client.1.smithi150.stdout:8/838: write d0/d2/d19/d29/d2f/f73 [4991422,89996] 0 2024-03-22T10:43:05.219 INFO:tasks.workunit.client.1.smithi150.stdout:7/788: truncate d2/d5/d7/d8/d46/d93/ffe 596714 0 2024-03-22T10:43:05.223 INFO:tasks.workunit.client.1.smithi150.stdout:8/839: stat d0/d5d/dd2/f4b 0 2024-03-22T10:43:05.223 INFO:tasks.workunit.client.1.smithi150.stdout:9/834: write d1/d8/de/df/d36/d40/d41/fac [585683,18465] 0 2024-03-22T10:43:05.226 INFO:tasks.workunit.client.1.smithi150.stdout:5/775: stat d0/d8/dc/de/ld3 0 2024-03-22T10:43:05.228 INFO:tasks.workunit.client.1.smithi150.stdout:0/769: fsync d0/d1/d84/d86/d42/d10/d38/d52/d9b/fb6 0 2024-03-22T10:43:05.233 INFO:tasks.workunit.client.1.smithi150.stdout:9/835: write d1/d8/de/dbd/df8/f10e [957820,74751] 0 2024-03-22T10:43:05.234 INFO:tasks.workunit.client.1.smithi150.stdout:8/840: unlink d0/d2/d19/d29/d2f/df/fa3 0 2024-03-22T10:43:05.237 INFO:tasks.workunit.client.1.smithi150.stdout:3/746: dwrite d5/f31 [4194304,4194304] 0 2024-03-22T10:43:05.241 INFO:tasks.workunit.client.1.smithi150.stdout:5/776: mkdir d0/d8/dc/de/d14/d2e/d45/d102 0 2024-03-22T10:43:05.243 INFO:tasks.workunit.client.1.smithi150.stdout:7/789: getdents d2/d5/d7/d9/db/d6c/da8 0 2024-03-22T10:43:05.244 INFO:tasks.workunit.client.1.smithi150.stdout:0/770: write d0/d1/f39 [5038761,5586] 0 2024-03-22T10:43:05.244 INFO:tasks.workunit.client.1.smithi150.stdout:8/841: unlink d0/d3c/d71/l5e 0 2024-03-22T10:43:05.252 INFO:tasks.workunit.client.1.smithi150.stdout:4/845: dwrite d0/d2f/d116/d7c/dc1/fcc [0,4194304] 0 2024-03-22T10:43:05.257 INFO:tasks.workunit.client.1.smithi150.stdout:3/747: creat d5/da/d62/dba/ff9 x:0 0 0 2024-03-22T10:43:05.257 INFO:tasks.workunit.client.1.smithi150.stdout:8/842: fsync d0/d2/d19/d29/d2f/f108 0 2024-03-22T10:43:05.261 INFO:tasks.workunit.client.1.smithi150.stdout:9/836: creat d1/d8/de/df/d36/d40/d44/db5/f11c x:0 0 0 2024-03-22T10:43:05.264 INFO:tasks.workunit.client.1.smithi150.stdout:6/748: dwrite db/dd/d15/d30/fab [0,4194304] 0 2024-03-22T10:43:05.265 INFO:tasks.workunit.client.1.smithi150.stdout:2/908: dwrite d12/d19/d2d/faa [0,4194304] 0 2024-03-22T10:43:05.266 INFO:tasks.workunit.client.1.smithi150.stdout:7/790: write d2/d5/d7/d9/db/da2/fbe [1160698,38571] 0 2024-03-22T10:43:05.268 INFO:tasks.workunit.client.1.smithi150.stdout:4/846: creat d0/d2f/d10d/d4d/d6c/d61/dd1/f11a x:0 0 0 2024-03-22T10:43:05.268 INFO:tasks.workunit.client.1.smithi150.stdout:0/771: write d0/d1/d84/d86/d42/d10/d38/f4b [1080818,102425] 0 2024-03-22T10:43:05.270 INFO:tasks.workunit.client.1.smithi150.stdout:3/748: readlink d5/d22/dc8/dda/d95/ldb 0 2024-03-22T10:43:05.278 INFO:tasks.workunit.client.1.smithi150.stdout:9/837: chown d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/fff 5408571 1 2024-03-22T10:43:05.283 INFO:tasks.workunit.client.1.smithi150.stdout:3/749: creat d5/da/ffa x:0 0 0 2024-03-22T10:43:05.283 INFO:tasks.workunit.client.1.smithi150.stdout:5/777: dread d0/dae/db2/ff0 [0,4194304] 0 2024-03-22T10:43:05.285 INFO:tasks.workunit.client.1.smithi150.stdout:9/838: symlink d1/d8/de/df/d24/l11d 0 2024-03-22T10:43:05.285 INFO:tasks.workunit.client.1.smithi150.stdout:4/847: creat d0/d2f/d116/d7c/dd4/f11b x:0 0 0 2024-03-22T10:43:05.289 INFO:tasks.workunit.client.1.smithi150.stdout:9/839: fsync d1/d49/f109 0 2024-03-22T10:43:05.289 INFO:tasks.workunit.client.1.smithi150.stdout:4/848: chown d0/d2f/d10d/d68/de0/lf4 93182058 1 2024-03-22T10:43:05.292 INFO:tasks.workunit.client.1.smithi150.stdout:0/772: dread d0/fc7 [0,4194304] 0 2024-03-22T10:43:05.295 INFO:tasks.workunit.client.1.smithi150.stdout:5/778: creat d0/d8/dc/de/d14/d2e/dbc/f103 x:0 0 0 2024-03-22T10:43:05.296 INFO:tasks.workunit.client.1.smithi150.stdout:9/840: chown d1/d8/de/d73/d101/d29/c6f 3907 1 2024-03-22T10:43:05.296 INFO:tasks.workunit.client.1.smithi150.stdout:0/773: truncate d0/d1/d84/d86/f8d 697603 0 2024-03-22T10:43:05.297 INFO:tasks.workunit.client.1.smithi150.stdout:4/849: chown d0/de/ca9 1908023985 1 2024-03-22T10:43:05.298 INFO:tasks.workunit.client.1.smithi150.stdout:4/850: readlink d0/l29 0 2024-03-22T10:43:05.299 INFO:tasks.workunit.client.1.smithi150.stdout:0/774: chown d0/d1/d84/d86/d5a 308730076 1 2024-03-22T10:43:05.300 INFO:tasks.workunit.client.1.smithi150.stdout:5/779: mkdir d0/d37/d104 0 2024-03-22T10:43:05.302 INFO:tasks.workunit.client.1.smithi150.stdout:2/909: write d12/d19/db6/da9/d67/f5a [191878,97894] 0 2024-03-22T10:43:05.306 INFO:tasks.workunit.client.1.smithi150.stdout:7/791: truncate d2/d5/d7/d8/d46/fd2 3902705 0 2024-03-22T10:43:05.307 INFO:tasks.workunit.client.1.smithi150.stdout:0/775: fdatasync d0/d1/d84/d86/d42/d10/d38/d79/f8f 0 2024-03-22T10:43:05.313 INFO:tasks.workunit.client.1.smithi150.stdout:3/750: dwrite d5/d22/d2f/d38/d3e/f55 [0,4194304] 0 2024-03-22T10:43:05.319 INFO:tasks.workunit.client.1.smithi150.stdout:6/749: dwrite db/dd/d15/d28/f58 [4194304,4194304] 0 2024-03-22T10:43:05.325 INFO:tasks.workunit.client.1.smithi150.stdout:5/780: rename d0/d8/dc/de/d96/da5 to d0/d8/dc/de/d14/d2e/d45/dac/d105 0 2024-03-22T10:43:05.326 INFO:tasks.workunit.client.1.smithi150.stdout:3/751: creat d5/d22/d2f/d38/d3e/d5b/da6/ffb x:0 0 0 2024-03-22T10:43:05.328 INFO:tasks.workunit.client.1.smithi150.stdout:7/792: chown d2/d5/d7/d8/d57/d4a/f39 16406465 1 2024-03-22T10:43:05.328 INFO:tasks.workunit.client.1.smithi150.stdout:4/851: dwrite d0/d2f/d10d/d4d/d6c/d61/f48 [0,4194304] 0 2024-03-22T10:43:05.328 INFO:tasks.workunit.client.1.smithi150.stdout:6/750: creat db/d52/f108 x:0 0 0 2024-03-22T10:43:05.331 INFO:tasks.workunit.client.1.smithi150.stdout:0/776: rename d0/d12/cdd to d0/d1/d84/d86/d11/d26/d76/cf5 0 2024-03-22T10:43:05.334 INFO:tasks.workunit.client.1.smithi150.stdout:3/752: symlink d5/da/d78/db7/dec/lfc 0 2024-03-22T10:43:05.335 INFO:tasks.workunit.client.1.smithi150.stdout:7/793: rmdir d2/d5/d7/d9/db/d9f/d96/d71/d106 39 2024-03-22T10:43:05.339 INFO:tasks.workunit.client.1.smithi150.stdout:0/777: mknod d0/d1/d84/d86/d42/d10/d38/d52/d9b/dc2/cf6 0 2024-03-22T10:43:05.340 INFO:tasks.workunit.client.1.smithi150.stdout:5/781: truncate d0/d8/dc/de/d14/d2e/d46/d9c/dc0/ff5 797521 0 2024-03-22T10:43:05.343 INFO:tasks.workunit.client.1.smithi150.stdout:4/852: getdents d0/d2f/d63/d7b/d9c/db1 0 2024-03-22T10:43:05.348 INFO:tasks.workunit.client.1.smithi150.stdout:2/910: dwrite d12/d19/db6/da9/d61/d98/fbb [0,4194304] 0 2024-03-22T10:43:05.350 INFO:tasks.workunit.client.1.smithi150.stdout:5/782: symlink d0/d8/dc/de/d14/d2e/d45/db3/l106 0 2024-03-22T10:43:05.351 INFO:tasks.workunit.client.1.smithi150.stdout:0/778: chown d0/d1/d84/d86/d42/d10/d38/dba/fec 38318 1 2024-03-22T10:43:05.356 INFO:tasks.workunit.client.1.smithi150.stdout:6/751: dwrite db/dd/d15/d6a/dde/f107 [0,4194304] 0 2024-03-22T10:43:05.358 INFO:tasks.workunit.client.1.smithi150.stdout:6/752: dread - db/dd/d15/d30/fd5 zero size 2024-03-22T10:43:05.362 INFO:tasks.workunit.client.1.smithi150.stdout:5/783: symlink d0/d8/l107 0 2024-03-22T10:43:05.378 INFO:tasks.workunit.client.1.smithi150.stdout:2/911: dread d12/da1/f115 [0,4194304] 0 2024-03-22T10:43:05.381 INFO:tasks.workunit.client.1.smithi150.stdout:4/853: dwrite d0/d2f/d63/f101 [0,4194304] 0 2024-03-22T10:43:05.390 INFO:tasks.workunit.client.1.smithi150.stdout:6/753: dwrite db/dd/d15/d60/f67 [0,4194304] 0 2024-03-22T10:43:05.394 INFO:tasks.workunit.client.1.smithi150.stdout:5/784: dwrite d0/d8/dc/de/f82 [0,4194304] 0 2024-03-22T10:43:05.396 INFO:tasks.workunit.client.1.smithi150.stdout:7/794: dread d2/d5/d7/d9/db/d9f/d45/f72 [0,4194304] 0 2024-03-22T10:43:05.401 INFO:tasks.workunit.client.1.smithi150.stdout:0/779: dread d0/f69 [0,4194304] 0 2024-03-22T10:43:05.408 INFO:tasks.workunit.client.1.smithi150.stdout:6/754: creat db/dd/d15/d30/f109 x:0 0 0 2024-03-22T10:43:05.412 INFO:tasks.workunit.client.1.smithi150.stdout:5/785: mknod d0/d8/dc/de/d14/d2e/d45/db3/c108 0 2024-03-22T10:43:05.415 INFO:tasks.workunit.client.1.smithi150.stdout:0/780: fdatasync d0/d1/d84/d86/d11/d66/d35/d5f/fdf 0 2024-03-22T10:43:05.421 INFO:tasks.workunit.client.1.smithi150.stdout:2/912: rename d12/d19/db6/da9/d2c/l71 to d12/d19/db6/da9/d2c/l11e 0 2024-03-22T10:43:05.421 INFO:tasks.workunit.client.1.smithi150.stdout:5/786: symlink d0/d8/dc/de/d14/d2e/d5a/l109 0 2024-03-22T10:43:05.424 INFO:tasks.workunit.client.1.smithi150.stdout:7/795: unlink d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/daf/cc9 0 2024-03-22T10:43:05.427 INFO:tasks.workunit.client.1.smithi150.stdout:7/796: dread - d2/d5/d7/d9/db/d6c/da8/ffa zero size 2024-03-22T10:43:05.427 INFO:tasks.workunit.client.1.smithi150.stdout:2/913: unlink d12/d19/db6/da9/d61/fe9 0 2024-03-22T10:43:05.429 INFO:tasks.workunit.client.1.smithi150.stdout:6/755: link db/d52/lb6 db/l10a 0 2024-03-22T10:43:05.432 INFO:tasks.workunit.client.1.smithi150.stdout:6/756: write db/d52/fc9 [300366,103963] 0 2024-03-22T10:43:05.438 INFO:tasks.workunit.client.1.smithi150.stdout:7/797: symlink d2/d5/d7/d8/d57/d4a/d8d/da9/dcb/l109 0 2024-03-22T10:43:05.441 INFO:tasks.workunit.client.1.smithi150.stdout:6/757: getdents db/dd 0 2024-03-22T10:43:05.443 INFO:tasks.workunit.client.1.smithi150.stdout:7/798: fsync d2/d5/d7/d9/db/d6c/fa1 0 2024-03-22T10:43:05.448 INFO:tasks.workunit.client.1.smithi150.stdout:6/758: symlink db/dd/d15/d6a/l10b 0 2024-03-22T10:43:05.457 INFO:tasks.workunit.client.1.smithi150.stdout:7/799: rename d2/d5/d7/d9/db/l70 to d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/dad/l10a 0 2024-03-22T10:43:05.458 INFO:tasks.workunit.client.1.smithi150.stdout:0/781: write d0/d1/d84/d86/d42/d10/d53/f5e [1418875,17407] 0 2024-03-22T10:43:05.460 INFO:tasks.workunit.client.1.smithi150.stdout:4/854: dwrite d0/d2f/d63/d7b/f119 [0,4194304] 0 2024-03-22T10:43:05.461 INFO:tasks.workunit.client.1.smithi150.stdout:2/914: write d12/d19/db6/f4d [3996267,31231] 0 2024-03-22T10:43:05.463 INFO:tasks.workunit.client.1.smithi150.stdout:0/782: read d0/d1/fbb [694165,65432] 0 2024-03-22T10:43:05.464 INFO:tasks.workunit.client.1.smithi150.stdout:5/787: truncate d0/d8/dc/de/d14/d2e/d45/f78 369937 0 2024-03-22T10:43:05.467 INFO:tasks.workunit.client.1.smithi150.stdout:4/855: creat d0/d47/f11c x:0 0 0 2024-03-22T10:43:05.467 INFO:tasks.workunit.client.1.smithi150.stdout:5/788: readlink d0/d8/dc/de/l3b 0 2024-03-22T10:43:05.474 INFO:tasks.workunit.client.1.smithi150.stdout:5/789: creat d0/d8/dc/de/d96/f10a x:0 0 0 2024-03-22T10:43:05.477 INFO:tasks.workunit.client.1.smithi150.stdout:0/783: getdents d0/d1/d84/d86/da6 0 2024-03-22T10:43:05.480 INFO:tasks.workunit.client.1.smithi150.stdout:6/759: dwrite db/dd/d15/d28/d6c/d9b/ff6 [0,4194304] 0 2024-03-22T10:43:05.486 INFO:tasks.workunit.client.1.smithi150.stdout:5/790: creat d0/d8/dc/de/d96/f10b x:0 0 0 2024-03-22T10:43:05.491 INFO:tasks.workunit.client.1.smithi150.stdout:0/784: symlink d0/d1/d84/d86/da6/lf7 0 2024-03-22T10:43:05.492 INFO:tasks.workunit.client.1.smithi150.stdout:7/800: dwrite d2/d5/d7/d9/db/d6c/da8/ffa [0,4194304] 0 2024-03-22T10:43:05.497 INFO:tasks.workunit.client.1.smithi150.stdout:6/760: dread db/d52/d9f/dc6/df1/f5a [0,4194304] 0 2024-03-22T10:43:05.503 INFO:tasks.workunit.client.1.smithi150.stdout:0/785: fdatasync d0/d1/d84/d86/d42/d10/d38/d52/d9b/fb6 0 2024-03-22T10:43:05.503 INFO:tasks.workunit.client.1.smithi150.stdout:6/761: readlink db/dd/l5e 0 2024-03-22T10:43:05.503 INFO:tasks.workunit.client.1.smithi150.stdout:7/801: read - d2/d5/d7/d9/db/d6c/d9c/fda zero size 2024-03-22T10:43:05.507 INFO:tasks.workunit.client.1.smithi150.stdout:2/915: dwrite d12/d19/d2d/f53 [0,4194304] 0 2024-03-22T10:43:05.511 INFO:tasks.workunit.client.1.smithi150.stdout:0/786: creat d0/d1/d84/ff8 x:0 0 0 2024-03-22T10:43:05.514 INFO:tasks.workunit.client.1.smithi150.stdout:5/791: link d0/d33/d6f/ca6 d0/d8/dc/de/d14/d2e/d45/c10c 0 2024-03-22T10:43:05.522 INFO:tasks.workunit.client.1.smithi150.stdout:7/802: mkdir d2/d5/d7/d8/d10b 0 2024-03-22T10:43:05.522 INFO:tasks.workunit.client.1.smithi150.stdout:2/916: symlink d12/d19/db6/l11f 0 2024-03-22T10:43:05.523 INFO:tasks.workunit.client.1.smithi150.stdout:5/792: symlink d0/d8/dc/de/d14/d2e/d45/db3/l10d 0 2024-03-22T10:43:05.529 INFO:tasks.workunit.client.1.smithi150.stdout:7/803: write d2/d5/d7/d9/f28 [2736974,4924] 0 2024-03-22T10:43:05.530 INFO:tasks.workunit.client.1.smithi150.stdout:6/762: dwrite db/f4a [0,4194304] 0 2024-03-22T10:43:05.531 INFO:tasks.workunit.client.1.smithi150.stdout:2/917: fdatasync d12/d19/db6/da9/d61/d98/fd4 0 2024-03-22T10:43:05.537 INFO:tasks.workunit.client.1.smithi150.stdout:6/763: dread - db/dd/d15/d30/fd5 zero size 2024-03-22T10:43:05.538 INFO:tasks.workunit.client.1.smithi150.stdout:5/793: creat d0/d8/dc/de/d14/d2e/d45/dac/d105/f10e x:0 0 0 2024-03-22T10:43:05.539 INFO:tasks.workunit.client.1.smithi150.stdout:2/918: fdatasync d12/d19/db6/da9/f2a 0 2024-03-22T10:43:05.542 INFO:tasks.workunit.client.1.smithi150.stdout:7/804: write d2/d5/d7/d9/db/d6c/d9c/fc0 [1203843,19580] 0 2024-03-22T10:43:05.547 INFO:tasks.workunit.client.1.smithi150.stdout:0/787: dwrite d0/f85 [0,4194304] 0 2024-03-22T10:43:05.547 INFO:tasks.workunit.client.1.smithi150.stdout:5/794: chown d0/d8/dc/de/d14/d4f/d47 24046229 1 2024-03-22T10:43:05.548 INFO:tasks.workunit.client.1.smithi150.stdout:7/805: chown d2/d5/d7/fc1 1984 1 2024-03-22T10:43:05.548 INFO:tasks.workunit.client.1.smithi150.stdout:2/919: creat d12/d35/f120 x:0 0 0 2024-03-22T10:43:05.555 INFO:tasks.workunit.client.1.smithi150.stdout:7/806: dread - d2/d5/fca zero size 2024-03-22T10:43:05.558 INFO:tasks.workunit.client.1.smithi150.stdout:6/764: rename db/dd/d15/d30/ddf/cea to db/dd/d15/d57/c10c 0 2024-03-22T10:43:05.560 INFO:tasks.workunit.client.1.smithi150.stdout:5/795: getdents d0/dae/db2/df9 0 2024-03-22T10:43:05.560 INFO:tasks.workunit.client.1.smithi150.stdout:7/807: chown d2/d5/d7/d9/db/d9f/d96/fe4 185456 1 2024-03-22T10:43:05.561 INFO:tasks.workunit.client.1.smithi150.stdout:2/920: creat d12/d19/d2d/d66/f121 x:0 0 0 2024-03-22T10:43:05.567 INFO:tasks.workunit.client.1.smithi150.stdout:0/788: symlink d0/d1/d84/d86/d11/d66/d35/d5f/da0/lf9 0 2024-03-22T10:43:05.572 INFO:tasks.workunit.client.1.smithi150.stdout:5/796: creat d0/d8/dc/de/d96/db8/f10f x:0 0 0 2024-03-22T10:43:05.579 INFO:tasks.workunit.client.1.smithi150.stdout:2/921: unlink d12/d19/db6/d63/l117 0 2024-03-22T10:43:05.586 INFO:tasks.workunit.client.1.smithi150.stdout:5/797: rename d0/d8/dc/de/d14/d2e/d46/d4d/cab to d0/d8/dc/de/d14/c110 0 2024-03-22T10:43:05.587 INFO:tasks.workunit.client.1.smithi150.stdout:6/765: dwrite db/dd/d15/d60/d50/dc1/fec [0,4194304] 0 2024-03-22T10:43:05.595 INFO:tasks.workunit.client.1.smithi150.stdout:6/766: rename db/dd/d15/d60/d8b to db/dd/d15/d60/d8b/d10d 22 2024-03-22T10:43:05.607 INFO:tasks.workunit.client.1.smithi150.stdout:7/808: dwrite d2/d5/d7/d9/db/da2/fbe [0,4194304] 0 2024-03-22T10:43:05.609 INFO:tasks.workunit.client.1.smithi150.stdout:6/767: unlink db/dd/d15/d30/ldd 0 2024-03-22T10:43:05.617 INFO:tasks.workunit.client.1.smithi150.stdout:5/798: write d0/f6 [3864980,83911] 0 2024-03-22T10:43:05.619 INFO:tasks.workunit.client.1.smithi150.stdout:6/768: fdatasync db/dd/ffe 0 2024-03-22T10:43:05.620 INFO:tasks.workunit.client.1.smithi150.stdout:7/809: creat d2/d5/d7/d88/f10c x:0 0 0 2024-03-22T10:43:05.621 INFO:tasks.workunit.client.1.smithi150.stdout:2/922: dwrite d12/d35/d109/f11d [0,4194304] 0 2024-03-22T10:43:05.626 INFO:tasks.workunit.client.1.smithi150.stdout:1/723: sync 2024-03-22T10:43:05.626 INFO:tasks.workunit.client.1.smithi150.stdout:8/843: sync 2024-03-22T10:43:05.631 INFO:tasks.workunit.client.1.smithi150.stdout:0/789: dread d0/d1/d84/d86/d11/fc4 [0,4194304] 0 2024-03-22T10:43:05.632 INFO:tasks.workunit.client.1.smithi150.stdout:5/799: mknod d0/d8/dc/de/d14/d2e/d46/d9c/c111 0 2024-03-22T10:43:05.633 INFO:tasks.workunit.client.1.smithi150.stdout:6/769: read db/dd/d15/d28/fa3 [416900,4102] 0 2024-03-22T10:43:05.636 INFO:tasks.workunit.client.1.smithi150.stdout:7/810: mkdir d2/d5/d10d 0 2024-03-22T10:43:05.638 INFO:tasks.workunit.client.1.smithi150.stdout:1/724: creat d1/d19/d50/d51/da0/fee x:0 0 0 2024-03-22T10:43:05.641 INFO:tasks.workunit.client.1.smithi150.stdout:7/811: write d2/d5/d7/d38/fd3 [54176,58173] 0 2024-03-22T10:43:05.641 INFO:tasks.workunit.client.1.smithi150.stdout:8/844: rename d0/d2/d19/d29/fe1 to d0/d2/d19/d29/d88/dec/f114 0 2024-03-22T10:43:05.641 INFO:tasks.workunit.client.1.smithi150.stdout:5/800: rename d0/d37 to d0/d37/d112 22 2024-03-22T10:43:05.645 INFO:tasks.workunit.client.1.smithi150.stdout:2/923: unlink d12/da1/dd6/de0/c116 0 2024-03-22T10:43:05.646 INFO:tasks.workunit.client.1.smithi150.stdout:1/725: symlink d1/d79/lef 0 2024-03-22T10:43:05.646 INFO:tasks.workunit.client.1.smithi150.stdout:0/790: fdatasync d0/d1/d84/d86/d42/d10/d38/d52/f65 0 2024-03-22T10:43:05.648 INFO:tasks.workunit.client.1.smithi150.stdout:2/924: chown d12/da1/dd6/de0 0 1 2024-03-22T10:43:05.649 INFO:tasks.workunit.client.1.smithi150.stdout:6/770: write db/dd/d15/d60/d50/dc1/fba [1356714,42353] 0 2024-03-22T10:43:05.650 INFO:tasks.workunit.client.1.smithi150.stdout:5/801: symlink d0/dae/db2/l113 0 2024-03-22T10:43:05.651 INFO:tasks.workunit.client.1.smithi150.stdout:8/845: mkdir d0/d2/d19/d29/d37/da1/d115 0 2024-03-22T10:43:05.652 INFO:tasks.workunit.client.1.smithi150.stdout:1/726: write d1/d19/d71/fe8 [921443,89568] 0 2024-03-22T10:43:05.652 INFO:tasks.workunit.client.1.smithi150.stdout:6/771: chown db/l9e 440 1 2024-03-22T10:43:05.654 INFO:tasks.workunit.client.1.smithi150.stdout:2/925: write d12/d19/db6/da9/d61/f8c [191939,6442] 0 2024-03-22T10:43:05.656 INFO:tasks.workunit.client.1.smithi150.stdout:0/791: mknod d0/d1/d84/d86/da6/cfa 0 2024-03-22T10:43:05.660 INFO:tasks.workunit.client.1.smithi150.stdout:8/846: rmdir d0/d2/d46/da2 39 2024-03-22T10:43:05.662 INFO:tasks.workunit.client.1.smithi150.stdout:2/926: write d12/d19/db6/d63/fa8 [4030288,77418] 0 2024-03-22T10:43:05.669 INFO:tasks.workunit.client.1.smithi150.stdout:8/847: mknod d0/d2/d19/d29/d2e/d70/c116 0 2024-03-22T10:43:05.669 INFO:tasks.workunit.client.1.smithi150.stdout:5/802: getdents d0/d8/dc/de/d14/d2e/d45/dac/dd7 0 2024-03-22T10:43:05.671 INFO:tasks.workunit.client.1.smithi150.stdout:4/856: sync 2024-03-22T10:43:05.671 INFO:tasks.workunit.client.1.smithi150.stdout:9/841: sync 2024-03-22T10:43:05.671 INFO:tasks.workunit.client.1.smithi150.stdout:3/753: sync 2024-03-22T10:43:05.674 INFO:tasks.workunit.client.1.smithi150.stdout:7/812: dwrite d2/d5/d7/d9/f28 [4194304,4194304] 0 2024-03-22T10:43:05.679 INFO:tasks.workunit.client.1.smithi150.stdout:7/813: readlink d2/d5/l34 0 2024-03-22T10:43:05.679 INFO:tasks.workunit.client.1.smithi150.stdout:4/857: write d0/d2f/d63/f101 [2183849,30284] 0 2024-03-22T10:43:05.681 INFO:tasks.workunit.client.1.smithi150.stdout:9/842: write d1/d19/fea [685805,32008] 0 2024-03-22T10:43:05.681 INFO:tasks.workunit.client.1.smithi150.stdout:3/754: unlink d5/l2c 0 2024-03-22T10:43:05.682 INFO:tasks.workunit.client.1.smithi150.stdout:4/858: write d0/f14 [438996,108487] 0 2024-03-22T10:43:05.686 INFO:tasks.workunit.client.1.smithi150.stdout:5/803: rmdir d0/d8/dc/de/d14/d2e/d45 39 2024-03-22T10:43:05.691 INFO:tasks.workunit.client.1.smithi150.stdout:9/843: mknod d1/d8/de/df/d24/d4e/c11e 0 2024-03-22T10:43:05.691 INFO:tasks.workunit.client.1.smithi150.stdout:2/927: dread f11 [0,4194304] 0 2024-03-22T10:43:05.693 INFO:tasks.workunit.client.1.smithi150.stdout:0/792: dwrite d0/d1/d84/d86/d42/d10/d53/f5e [0,4194304] 0 2024-03-22T10:43:05.700 INFO:tasks.workunit.client.1.smithi150.stdout:1/727: truncate d1/da/d15/d18/d4b/f6a 3089390 0 2024-03-22T10:43:05.700 INFO:tasks.workunit.client.1.smithi150.stdout:6/772: write db/dd/d82/d84/f96 [212632,42046] 0 2024-03-22T10:43:05.703 INFO:tasks.workunit.client.1.smithi150.stdout:5/804: fdatasync d0/d8/dc/f12 0 2024-03-22T10:43:05.712 INFO:tasks.workunit.client.1.smithi150.stdout:9/844: unlink d1/d8/de/df/d24/d4e/fe4 0 2024-03-22T10:43:05.713 INFO:tasks.workunit.client.1.smithi150.stdout:0/793: symlink d0/d1/d84/d86/d11/d26/d76/lfb 0 2024-03-22T10:43:05.713 INFO:tasks.workunit.client.1.smithi150.stdout:8/848: dread d0/f17 [0,4194304] 0 2024-03-22T10:43:05.718 INFO:tasks.workunit.client.1.smithi150.stdout:5/805: rmdir d0/dae 39 2024-03-22T10:43:05.721 INFO:tasks.workunit.client.1.smithi150.stdout:2/928: write d12/d19/f1d [600908,5527] 0 2024-03-22T10:43:05.721 INFO:tasks.workunit.client.1.smithi150.stdout:4/859: dread d0/d2f/d63/d7b/d9c/fa6 [0,4194304] 0 2024-03-22T10:43:05.722 INFO:tasks.workunit.client.1.smithi150.stdout:5/806: write d0/f10 [865372,100597] 0 2024-03-22T10:43:05.724 INFO:tasks.workunit.client.1.smithi150.stdout:9/845: rmdir d1/d8/de/dbd/df8 39 2024-03-22T10:43:05.736 INFO:tasks.workunit.client.1.smithi150.stdout:9/846: write d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/f107 [411857,101509] 0 2024-03-22T10:43:05.740 INFO:tasks.workunit.client.1.smithi150.stdout:7/814: link d2/d5/l6b d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/dad/l10e 0 2024-03-22T10:43:05.744 INFO:tasks.workunit.client.1.smithi150.stdout:4/860: creat d0/d2f/d116/d7c/dc1/f11d x:0 0 0 2024-03-22T10:43:05.746 INFO:tasks.workunit.client.1.smithi150.stdout:2/929: read d12/d19/db6/da9/d61/f86 [786977,85883] 0 2024-03-22T10:43:05.749 INFO:tasks.workunit.client.1.smithi150.stdout:3/755: dread d5/d53/f54 [0,4194304] 0 2024-03-22T10:43:05.750 INFO:tasks.workunit.client.1.smithi150.stdout:5/807: mknod d0/d37/db4/c114 0 2024-03-22T10:43:05.751 INFO:tasks.workunit.client.1.smithi150.stdout:1/728: symlink d1/da/d2a/d55/d97/lf0 0 2024-03-22T10:43:05.753 INFO:tasks.workunit.client.1.smithi150.stdout:6/773: dwrite db/dd/f5b [0,4194304] 0 2024-03-22T10:43:05.754 INFO:tasks.workunit.client.1.smithi150.stdout:8/849: creat d0/d2/f117 x:0 0 0 2024-03-22T10:43:05.755 INFO:tasks.workunit.client.1.smithi150.stdout:2/930: mkdir d12/d19/d2d/dd7/d122 0 2024-03-22T10:43:05.757 INFO:tasks.workunit.client.1.smithi150.stdout:3/756: rmdir d5/d53 39 2024-03-22T10:43:05.759 INFO:tasks.workunit.client.1.smithi150.stdout:0/794: dwrite d0/d1/d84/d86/d42/d10/d38/fc3 [0,4194304] 0 2024-03-22T10:43:05.761 INFO:tasks.workunit.client.1.smithi150.stdout:7/815: write d2/d5/d7/d9/db/d6c/d9c/ff3 [308458,123000] 0 2024-03-22T10:43:05.764 INFO:tasks.workunit.client.1.smithi150.stdout:8/850: fdatasync d0/d2/d19/d29/d88/f105 0 2024-03-22T10:43:05.767 INFO:tasks.workunit.client.1.smithi150.stdout:6/774: unlink db/dd/d15/ca0 0 2024-03-22T10:43:05.771 INFO:tasks.workunit.client.1.smithi150.stdout:7/816: read - d2/d5/d7/d9/db/da2/fea zero size 2024-03-22T10:43:05.773 INFO:tasks.workunit.client.1.smithi150.stdout:4/861: dwrite d0/f8 [0,4194304] 0 2024-03-22T10:43:05.774 INFO:tasks.workunit.client.1.smithi150.stdout:0/795: creat d0/d1/d84/ffc x:0 0 0 2024-03-22T10:43:05.775 INFO:tasks.workunit.client.1.smithi150.stdout:6/775: chown db/dd/d15/d57/f65 191662662 1 2024-03-22T10:43:05.778 INFO:tasks.workunit.client.1.smithi150.stdout:5/808: chown d0/d8/dc/de/d14/d2e/f54 15 1 2024-03-22T10:43:05.784 INFO:tasks.workunit.client.1.smithi150.stdout:6/776: write db/dd/d15/d30/f51 [882099,993] 0 2024-03-22T10:43:05.785 INFO:tasks.workunit.client.1.smithi150.stdout:0/796: creat d0/d1/d84/d86/d11/d66/d37/ffd x:0 0 0 2024-03-22T10:43:05.785 INFO:tasks.workunit.client.1.smithi150.stdout:7/817: fdatasync d2/d5/d7/fd0 0 2024-03-22T10:43:05.789 INFO:tasks.workunit.client.1.smithi150.stdout:5/809: mknod d0/c115 0 2024-03-22T10:43:05.789 INFO:tasks.workunit.client.1.smithi150.stdout:4/862: creat d0/d2f/d10d/d4d/d6c/f11e x:0 0 0 2024-03-22T10:43:05.790 INFO:tasks.workunit.client.1.smithi150.stdout:6/777: chown db/dd/d15/l49 228965 1 2024-03-22T10:43:05.794 INFO:tasks.workunit.client.1.smithi150.stdout:2/931: write d12/f15 [1291213,61840] 0 2024-03-22T10:43:05.796 INFO:tasks.workunit.client.1.smithi150.stdout:3/757: write d5/f46 [421156,19687] 0 2024-03-22T10:43:05.798 INFO:tasks.workunit.client.1.smithi150.stdout:5/810: truncate d0/ddc/fdd 43634 0 2024-03-22T10:43:05.799 INFO:tasks.workunit.client.1.smithi150.stdout:8/851: dwrite d0/d3c/d71/f7f [0,4194304] 0 2024-03-22T10:43:05.802 INFO:tasks.workunit.client.1.smithi150.stdout:7/818: fsync d2/d5/d7/d9/db/d9f/d96/d71/d106/f5f 0 2024-03-22T10:43:05.808 INFO:tasks.workunit.client.1.smithi150.stdout:8/852: creat d0/d3c/d71/f118 x:0 0 0 2024-03-22T10:43:05.810 INFO:tasks.workunit.client.1.smithi150.stdout:5/811: mkdir d0/d8/dc/de/d116 0 2024-03-22T10:43:05.812 INFO:tasks.workunit.client.1.smithi150.stdout:5/812: chown d0/d33/lc8 596264 1 2024-03-22T10:43:05.819 INFO:tasks.workunit.client.1.smithi150.stdout:0/797: write d0/d1/d84/d86/f31 [673523,16588] 0 2024-03-22T10:43:05.820 INFO:tasks.workunit.client.1.smithi150.stdout:6/778: truncate db/dd/fa4 691431 0 2024-03-22T10:43:05.820 INFO:tasks.workunit.client.1.smithi150.stdout:2/932: link d12/d19/db6/da9/d61/la2 d12/d19/db6/da9/l123 0 2024-03-22T10:43:05.820 INFO:tasks.workunit.client.1.smithi150.stdout:5/813: creat d0/d8/dc/de/d14/d4f/d47/f117 x:0 0 0 2024-03-22T10:43:05.828 INFO:tasks.workunit.client.1.smithi150.stdout:2/933: mkdir d12/d21/d124 0 2024-03-22T10:43:05.828 INFO:tasks.workunit.client.1.smithi150.stdout:6/779: read db/d52/d9f/dc6/df1/fd2 [771485,19339] 0 2024-03-22T10:43:05.828 INFO:tasks.workunit.client.1.smithi150.stdout:5/814: creat d0/d8/dc/de/d14/d2e/d46/d9c/f118 x:0 0 0 2024-03-22T10:43:05.830 INFO:tasks.workunit.client.1.smithi150.stdout:0/798: rmdir d0/d1/d84/d86/d42/d10/d38/dba 39 2024-03-22T10:43:05.832 INFO:tasks.workunit.client.1.smithi150.stdout:1/729: dread d1/da/d15/d4f/f60 [0,4194304] 0 2024-03-22T10:43:05.834 INFO:tasks.workunit.client.1.smithi150.stdout:2/934: chown d12/d19/db6/da9/l18 546017 1 2024-03-22T10:43:05.839 INFO:tasks.workunit.client.1.smithi150.stdout:2/935: read d12/d19/db6/da9/d61/f86 [793496,75986] 0 2024-03-22T10:43:05.841 INFO:tasks.workunit.client.1.smithi150.stdout:6/780: creat db/f10e x:0 0 0 2024-03-22T10:43:05.844 INFO:tasks.workunit.client.1.smithi150.stdout:5/815: creat d0/d8/dc/de/f119 x:0 0 0 2024-03-22T10:43:05.845 INFO:tasks.workunit.client.1.smithi150.stdout:5/816: chown d0/d8/dc/de/d96 84841 1 2024-03-22T10:43:05.847 INFO:tasks.workunit.client.1.smithi150.stdout:4/863: dread d0/d2f/d10d/f23 [0,4194304] 0 2024-03-22T10:43:05.848 INFO:tasks.workunit.client.1.smithi150.stdout:5/817: fsync d0/d8/dc/de/d14/fca 0 2024-03-22T10:43:05.849 INFO:tasks.workunit.client.1.smithi150.stdout:0/799: mknod d0/d1/d84/d86/d42/d10/cfe 0 2024-03-22T10:43:05.850 INFO:tasks.workunit.client.1.smithi150.stdout:0/800: readlink d0/d1/l1b 0 2024-03-22T10:43:05.853 INFO:tasks.workunit.client.1.smithi150.stdout:6/781: mkdir db/d52/d9f/d10f 0 2024-03-22T10:43:05.857 INFO:tasks.workunit.client.1.smithi150.stdout:4/864: creat d0/d2f/d63/d7b/d9c/db1/f11f x:0 0 0 2024-03-22T10:43:05.858 INFO:tasks.workunit.client.1.smithi150.stdout:3/758: dread d5/da/f33 [0,4194304] 0 2024-03-22T10:43:05.858 INFO:tasks.workunit.client.1.smithi150.stdout:1/730: rename d1/d19/d50/d51/da0/dcf/dd9/lde to d1/d19/d50/d51/da0/lf1 0 2024-03-22T10:43:05.859 INFO:tasks.workunit.client.1.smithi150.stdout:6/782: creat db/dd/d15/d60/d50/f110 x:0 0 0 2024-03-22T10:43:05.864 INFO:tasks.workunit.client.1.smithi150.stdout:1/731: rename d1/d19/d57/ca1 to d1/da/d2a/d55/cf2 0 2024-03-22T10:43:05.866 INFO:tasks.workunit.client.1.smithi150.stdout:5/818: getdents d0/d8/dc/de/d14/d2e/d46 0 2024-03-22T10:43:05.866 INFO:tasks.workunit.client.1.smithi150.stdout:1/732: fsync d1/da/d15/d4f/fa4 0 2024-03-22T10:43:05.867 INFO:tasks.workunit.client.1.smithi150.stdout:6/783: creat db/dd/d15/d6a/dde/f111 x:0 0 0 2024-03-22T10:43:05.869 INFO:tasks.workunit.client.1.smithi150.stdout:3/759: symlink d5/da/d78/d86/lfd 0 2024-03-22T10:43:05.871 INFO:tasks.workunit.client.1.smithi150.stdout:6/784: chown db/d52/fc9 8 1 2024-03-22T10:43:05.877 INFO:tasks.workunit.client.1.smithi150.stdout:5/819: symlink d0/d8/dc/de/d14/d2e/dbc/l11a 0 2024-03-22T10:43:05.879 INFO:tasks.workunit.client.1.smithi150.stdout:2/936: dread d12/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:43:05.879 INFO:tasks.workunit.client.1.smithi150.stdout:3/760: write d5/d22/d2f/d38/d3e/d72/ff8 [1040153,48535] 0 2024-03-22T10:43:05.882 INFO:tasks.workunit.client.1.smithi150.stdout:5/820: readlink d0/d8/dc/de/d14/d2e/l69 0 2024-03-22T10:43:05.884 INFO:tasks.workunit.client.1.smithi150.stdout:3/761: fdatasync d5/d22/d2f/d38/d59/d88/f8a 0 2024-03-22T10:43:05.885 INFO:tasks.workunit.client.1.smithi150.stdout:4/865: truncate d0/d2f/d10d/f2e 1498539 0 2024-03-22T10:43:05.885 INFO:tasks.workunit.client.1.smithi150.stdout:5/821: chown d0/d37/db4/c114 296985097 1 2024-03-22T10:43:05.886 INFO:tasks.workunit.client.1.smithi150.stdout:2/937: rename d12/d19/d1a to d12/d19/db6/d54/d125 0 2024-03-22T10:43:05.889 INFO:tasks.workunit.client.1.smithi150.stdout:1/733: write d1/f7 [331475,23742] 0 2024-03-22T10:43:05.896 INFO:tasks.workunit.client.1.smithi150.stdout:0/801: dread d0/d1/d84/d86/d11/f7e [0,4194304] 0 2024-03-22T10:43:05.897 INFO:tasks.workunit.client.1.smithi150.stdout:3/762: mknod d5/cfe 0 2024-03-22T10:43:05.904 INFO:tasks.workunit.client.1.smithi150.stdout:3/763: mkdir d5/d22/d2f/d38/d59/dff 0 2024-03-22T10:43:05.904 INFO:tasks.workunit.client.1.smithi150.stdout:6/785: dwrite db/dd/f37 [4194304,4194304] 0 2024-03-22T10:43:05.906 INFO:tasks.workunit.client.1.smithi150.stdout:4/866: link d0/d2f/d10d/d4d/d9f/dc6/ff5 d0/d2f/d10d/d4d/d9f/f120 0 2024-03-22T10:43:05.908 INFO:tasks.workunit.client.1.smithi150.stdout:1/734: mkdir d1/d19/d1a/df3 0 2024-03-22T10:43:05.910 INFO:tasks.workunit.client.1.smithi150.stdout:4/867: fdatasync d0/d2f/d10d/d4d/d6c/d61/dd1/f11a 0 2024-03-22T10:43:05.911 INFO:tasks.workunit.client.1.smithi150.stdout:2/938: chown d12/d19/db6/d63/f6c 1638111851 1 2024-03-22T10:43:05.919 INFO:tasks.workunit.client.1.smithi150.stdout:5/822: dwrite d0/d8/dc/de/d14/d2e/d46/f72 [0,4194304] 0 2024-03-22T10:43:05.922 INFO:tasks.workunit.client.1.smithi150.stdout:2/939: mkdir d12/d19/db6/d63/dd8/d126 0 2024-03-22T10:43:05.928 INFO:tasks.workunit.client.1.smithi150.stdout:5/823: mkdir d0/d8/d101/d11b 0 2024-03-22T10:43:05.930 INFO:tasks.workunit.client.1.smithi150.stdout:6/786: link db/dd/d15/l48 db/dd/d15/d6a/l112 0 2024-03-22T10:43:05.931 INFO:tasks.workunit.client.1.smithi150.stdout:2/940: creat d12/d19/db6/d63/dd8/d126/f127 x:0 0 0 2024-03-22T10:43:05.934 INFO:tasks.workunit.client.1.smithi150.stdout:2/941: stat d12/d19/f1d 0 2024-03-22T10:43:05.936 INFO:tasks.workunit.client.1.smithi150.stdout:5/824: truncate d0/dae/db2/f90 765585 0 2024-03-22T10:43:05.942 INFO:tasks.workunit.client.1.smithi150.stdout:6/787: rename db/d52/d9f/d10f to db/dd/d15/d30/ddf/d113 0 2024-03-22T10:43:05.943 INFO:tasks.workunit.client.1.smithi150.stdout:2/942: stat d12/d19/db6/da9/d67/d3c/c118 0 2024-03-22T10:43:05.945 INFO:tasks.workunit.client.1.smithi150.stdout:1/735: dwrite d1/d19/d50/d6e/f88 [0,4194304] 0 2024-03-22T10:43:05.952 INFO:tasks.workunit.client.1.smithi150.stdout:3/764: dread d5/f31 [0,4194304] 0 2024-03-22T10:43:05.954 INFO:tasks.workunit.client.1.smithi150.stdout:1/736: creat d1/d19/d50/d51/ff4 x:0 0 0 2024-03-22T10:43:05.956 INFO:tasks.workunit.client.1.smithi150.stdout:3/765: write d5/d22/d2f/d9c/fe1 [185530,33575] 0 2024-03-22T10:43:05.962 INFO:tasks.workunit.client.1.smithi150.stdout:5/825: write d0/d8/dc/f40 [98667,17008] 0 2024-03-22T10:43:05.962 INFO:tasks.workunit.client.1.smithi150.stdout:1/737: read d1/d19/d71/f8b [771578,127018] 0 2024-03-22T10:43:05.963 INFO:tasks.workunit.client.1.smithi150.stdout:6/788: link db/dd/d15/d60/fcd db/d52/d9f/dc6/df1/db4/f114 0 2024-03-22T10:43:05.964 INFO:tasks.workunit.client.1.smithi150.stdout:3/766: mkdir d5/d22/d2f/d38/d3e/d72/d100 0 2024-03-22T10:43:05.966 INFO:tasks.workunit.client.1.smithi150.stdout:1/738: chown d1/da/d2a/d65/dbd 1573207 1 2024-03-22T10:43:05.966 INFO:tasks.workunit.client.1.smithi150.stdout:4/868: dread d0/f14 [0,4194304] 0 2024-03-22T10:43:05.967 INFO:tasks.workunit.client.1.smithi150.stdout:3/767: stat d5/da/d62/l97 0 2024-03-22T10:43:05.967 INFO:tasks.workunit.client.1.smithi150.stdout:6/789: symlink db/d52/l115 0 2024-03-22T10:43:05.968 INFO:tasks.workunit.client.1.smithi150.stdout:1/739: dread - d1/da/d2a/fce zero size 2024-03-22T10:43:05.969 INFO:tasks.workunit.client.1.smithi150.stdout:3/768: symlink d5/da/l101 0 2024-03-22T10:43:05.970 INFO:tasks.workunit.client.1.smithi150.stdout:6/790: dread - db/dd/d15/d60/d50/f64 zero size 2024-03-22T10:43:05.971 INFO:tasks.workunit.client.1.smithi150.stdout:5/826: mkdir d0/d8/dc/de/d14/d2e/d45/d102/d11c 0 2024-03-22T10:43:05.971 INFO:tasks.workunit.client.1.smithi150.stdout:3/769: dread - d5/d22/d2f/d38/f85 zero size 2024-03-22T10:43:05.972 INFO:tasks.workunit.client.1.smithi150.stdout:1/740: chown d1/f3a 5 1 2024-03-22T10:43:05.974 INFO:tasks.workunit.client.1.smithi150.stdout:6/791: readlink db/dd/d15/d60/d50/l9d 0 2024-03-22T10:43:05.978 INFO:tasks.workunit.client.1.smithi150.stdout:4/869: getdents d0/d2f/d116/d7c/dc1 0 2024-03-22T10:43:05.978 INFO:tasks.workunit.client.1.smithi150.stdout:6/792: chown db/dd/fd4 164 1 2024-03-22T10:43:05.979 INFO:tasks.workunit.client.1.smithi150.stdout:5/827: rmdir d0/d33/d100 0 2024-03-22T10:43:05.984 INFO:tasks.workunit.client.1.smithi150.stdout:5/828: write d0/d8/dc/de/d14/d2e/d46/f72 [3833169,119713] 0 2024-03-22T10:43:05.988 INFO:tasks.workunit.client.1.smithi150.stdout:3/770: rename d5/da/d78/d86/d81/f9b to d5/da/d9d/d9f/dc1/f102 0 2024-03-22T10:43:06.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:05 smithi115 ceph-mon[29111]: pgmap v21: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 92 MiB/s rd, 225 MiB/s wr, 473 op/s 2024-03-22T10:43:06.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:05 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:06.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:05 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:06.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:05 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:43:06.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:05 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:43:06.037 INFO:tasks.workunit.client.1.smithi150.stdout:9/847: sync 2024-03-22T10:43:06.049 INFO:tasks.workunit.client.1.smithi150.stdout:5/829: write d0/d8/dc/de/d96/db8/fe1 [1305832,104875] 0 2024-03-22T10:43:06.051 INFO:tasks.workunit.client.1.smithi150.stdout:1/741: dwrite d1/d19/d50/d6e/fb9 [0,4194304] 0 2024-03-22T10:43:06.061 INFO:tasks.workunit.client.1.smithi150.stdout:3/771: creat d5/d22/d2f/d38/d59/dff/f103 x:0 0 0 2024-03-22T10:43:06.062 INFO:tasks.workunit.client.1.smithi150.stdout:4/870: dwrite d0/d47/d72/de7/d10a/f115 [0,4194304] 0 2024-03-22T10:43:06.064 INFO:tasks.workunit.client.1.smithi150.stdout:6/793: dwrite db/dd/fd4 [0,4194304] 0 2024-03-22T10:43:06.068 INFO:tasks.workunit.client.1.smithi150.stdout:9/848: getdents d1/d8/d95 0 2024-03-22T10:43:06.070 INFO:tasks.workunit.client.1.smithi150.stdout:1/742: truncate d1/d79/fdd 1572966 0 2024-03-22T10:43:06.074 INFO:tasks.workunit.client.1.smithi150.stdout:6/794: chown db/dd/d15/d28/d6c/d9b/la8 2367 1 2024-03-22T10:43:06.077 INFO:tasks.workunit.client.1.smithi150.stdout:3/772: dread - d5/d22/d2f/d38/d3e/d5b/f9a zero size 2024-03-22T10:43:06.078 INFO:tasks.workunit.client.1.smithi150.stdout:6/795: chown db/dd/d15/d30 62036717 1 2024-03-22T10:43:06.079 INFO:tasks.workunit.client.1.smithi150.stdout:9/849: mkdir d1/d8/de/df/d11a/d11f 0 2024-03-22T10:43:06.082 INFO:tasks.workunit.client.1.smithi150.stdout:6/796: creat db/d52/d9f/dc6/f116 x:0 0 0 2024-03-22T10:43:06.089 INFO:tasks.workunit.client.1.smithi150.stdout:1/743: dread d1/d19/f26 [0,4194304] 0 2024-03-22T10:43:06.089 INFO:tasks.workunit.client.1.smithi150.stdout:3/773: creat d5/d22/d2f/d38/d3e/d72/f104 x:0 0 0 2024-03-22T10:43:06.090 INFO:tasks.workunit.client.1.smithi150.stdout:6/797: rmdir db/dd/d15/d60/d8b 39 2024-03-22T10:43:06.099 INFO:tasks.workunit.client.1.smithi150.stdout:6/798: creat db/d52/d9f/dc6/dd0/f117 x:0 0 0 2024-03-22T10:43:06.109 INFO:tasks.workunit.client.1.smithi150.stdout:3/774: dread d5/f31 [0,4194304] 0 2024-03-22T10:43:06.111 INFO:tasks.workunit.client.1.smithi150.stdout:9/850: dwrite d1/d8/de/df/d36/f116 [0,4194304] 0 2024-03-22T10:43:06.111 INFO:tasks.workunit.client.1.smithi150.stdout:3/775: chown d5/da/d78/ca2 0 1 2024-03-22T10:43:06.116 INFO:tasks.workunit.client.1.smithi150.stdout:9/851: fdatasync d1/d8/de/df/d24/d4e/f93 0 2024-03-22T10:43:06.119 INFO:tasks.workunit.client.1.smithi150.stdout:4/871: dread d0/d2f/d10d/d4d/d6c/d61/f9e [0,4194304] 0 2024-03-22T10:43:06.122 INFO:tasks.workunit.client.1.smithi150.stdout:4/872: write d0/d2f/d116/f105 [891088,30947] 0 2024-03-22T10:43:06.123 INFO:tasks.workunit.client.1.smithi150.stdout:9/852: creat d1/d49/f120 x:0 0 0 2024-03-22T10:43:06.125 INFO:tasks.workunit.client.1.smithi150.stdout:5/830: dread d0/d8/dc/de/d14/d2e/d5a/ffe [0,4194304] 0 2024-03-22T10:43:06.128 INFO:tasks.workunit.client.1.smithi150.stdout:9/853: stat d1/d8/de/df/d36/l4c 0 2024-03-22T10:43:06.132 INFO:tasks.workunit.client.1.smithi150.stdout:3/776: dwrite d5/da/d62/dba/fe3 [0,4194304] 0 2024-03-22T10:43:06.137 INFO:tasks.workunit.client.1.smithi150.stdout:4/873: rename d0/d2f/d116 to d0/dce/d10e/d121 0 2024-03-22T10:43:06.147 INFO:tasks.workunit.client.1.smithi150.stdout:3/777: rmdir d5/da/d9d/ded 0 2024-03-22T10:43:06.147 INFO:tasks.workunit.client.1.smithi150.stdout:4/874: mkdir d0/dce/dfd/d122 0 2024-03-22T10:43:06.147 INFO:tasks.workunit.client.1.smithi150.stdout:5/831: dread d0/d8/dc/d1d/f24 [0,4194304] 0 2024-03-22T10:43:06.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:05 smithi150 ceph-mon[34518]: pgmap v21: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 92 MiB/s rd, 225 MiB/s wr, 473 op/s 2024-03-22T10:43:06.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:05 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:06.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:05 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:06.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:05 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:43:06.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:05 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:43:06.149 INFO:tasks.workunit.client.1.smithi150.stdout:9/854: write d1/d8/de/df/d24/d4e/f93 [4162888,96697] 0 2024-03-22T10:43:06.155 INFO:tasks.workunit.client.1.smithi150.stdout:3/778: mkdir d5/da/d78/d86/de8/d105 0 2024-03-22T10:43:06.156 INFO:tasks.workunit.client.1.smithi150.stdout:1/744: dread d1/da/d2a/f35 [0,4194304] 0 2024-03-22T10:43:06.157 INFO:tasks.workunit.client.1.smithi150.stdout:9/855: write d1/d8/de/df/da2/fbf [9027481,58241] 0 2024-03-22T10:43:06.160 INFO:tasks.workunit.client.1.smithi150.stdout:1/745: chown d1/da/d2a/d55/db8/fe9 223 1 2024-03-22T10:43:06.162 INFO:tasks.workunit.client.1.smithi150.stdout:5/832: write d0/dae/de0/ffc [326801,119521] 0 2024-03-22T10:43:06.167 INFO:tasks.workunit.client.1.smithi150.stdout:6/799: dread db/dd/d15/d57/f65 [0,4194304] 0 2024-03-22T10:43:06.168 INFO:tasks.workunit.client.1.smithi150.stdout:1/746: chown d1/da/d2a/d55/cf2 247720 1 2024-03-22T10:43:06.169 INFO:tasks.workunit.client.1.smithi150.stdout:5/833: fdatasync d0/dae/de0/f7b 0 2024-03-22T10:43:06.172 INFO:tasks.workunit.client.1.smithi150.stdout:9/856: write d1/d49/d98/fcf [939973,101933] 0 2024-03-22T10:43:06.173 INFO:tasks.workunit.client.1.smithi150.stdout:6/800: read db/dd/d25/f3d [267762,122219] 0 2024-03-22T10:43:06.174 INFO:tasks.workunit.client.1.smithi150.stdout:3/779: truncate d5/d53/f75 1448922 0 2024-03-22T10:43:06.175 INFO:tasks.workunit.client.1.smithi150.stdout:4/875: dwrite d0/d2f/f76 [0,4194304] 0 2024-03-22T10:43:06.175 INFO:tasks.workunit.client.1.smithi150.stdout:1/747: getdents d1/d19/d50/d6e/db6/de1 0 2024-03-22T10:43:06.178 INFO:tasks.workunit.client.1.smithi150.stdout:9/857: stat d1/d8/de/d73/d101/d32/f2a 0 2024-03-22T10:43:06.180 INFO:tasks.workunit.client.1.smithi150.stdout:3/780: mkdir d5/da/d78/db7/d106 0 2024-03-22T10:43:06.182 INFO:tasks.workunit.client.1.smithi150.stdout:4/876: rmdir d0/d2f/d10d/d4d/d6c 39 2024-03-22T10:43:06.184 INFO:tasks.workunit.client.1.smithi150.stdout:1/748: fsync d1/d19/d50/d51/f9c 0 2024-03-22T10:43:06.186 INFO:tasks.workunit.client.1.smithi150.stdout:6/801: stat db/dd/d15/d60/d8b/f92 0 2024-03-22T10:43:06.189 INFO:tasks.workunit.client.1.smithi150.stdout:4/877: creat d0/dce/d10e/d121/f123 x:0 0 0 2024-03-22T10:43:06.192 INFO:tasks.workunit.client.1.smithi150.stdout:1/749: unlink d1/d19/d57/f5b 0 2024-03-22T10:43:06.192 INFO:tasks.workunit.client.1.smithi150.stdout:7/819: sync 2024-03-22T10:43:06.193 INFO:tasks.workunit.client.1.smithi150.stdout:8/853: sync 2024-03-22T10:43:06.197 INFO:tasks.workunit.client.1.smithi150.stdout:4/878: read d0/d2f/d10d/fac [127167,129088] 0 2024-03-22T10:43:06.199 INFO:tasks.workunit.client.1.smithi150.stdout:6/802: mknod db/dd/d15/d30/ddf/d103/c118 0 2024-03-22T10:43:06.200 INFO:tasks.workunit.client.1.smithi150.stdout:1/750: write d1/d19/d50/d51/da0/dcf/dd6/faf [797169,120798] 0 2024-03-22T10:43:06.201 INFO:tasks.workunit.client.1.smithi150.stdout:4/879: stat d0/dce/d10e/f111 0 2024-03-22T10:43:06.208 INFO:tasks.workunit.client.1.smithi150.stdout:6/803: write db/dd/d25/f3b [54629,126845] 0 2024-03-22T10:43:06.210 INFO:tasks.workunit.client.1.smithi150.stdout:5/834: dwrite d0/d8/dc/d1d/d89/fc4 [4194304,4194304] 0 2024-03-22T10:43:06.214 INFO:tasks.workunit.client.1.smithi150.stdout:9/858: truncate d1/d8/de/df/d24/f5f 129074 0 2024-03-22T10:43:06.222 INFO:tasks.workunit.client.1.smithi150.stdout:4/880: mknod d0/d2f/d10d/d4d/d6c/d61/dd1/c124 0 2024-03-22T10:43:06.226 INFO:tasks.workunit.client.1.smithi150.stdout:5/835: mknod d0/d8/dc/de/d14/d2e/d45/dac/c11d 0 2024-03-22T10:43:06.231 INFO:tasks.workunit.client.1.smithi150.stdout:9/859: truncate d1/d19/d7d/dc1/fa0 1121027 0 2024-03-22T10:43:06.231 INFO:tasks.workunit.client.1.smithi150.stdout:4/881: creat d0/d2f/d10d/f125 x:0 0 0 2024-03-22T10:43:06.236 INFO:tasks.workunit.client.1.smithi150.stdout:1/751: dwrite d1/d19/d71/f8b [0,4194304] 0 2024-03-22T10:43:06.246 INFO:tasks.workunit.client.1.smithi150.stdout:3/781: dread d5/d22/d2f/d38/d3e/f3f [0,4194304] 0 2024-03-22T10:43:06.246 INFO:tasks.workunit.client.1.smithi150.stdout:6/804: dwrite db/dd/d25/f6d [0,4194304] 0 2024-03-22T10:43:06.252 INFO:tasks.workunit.client.1.smithi150.stdout:4/882: symlink d0/d2f/d10d/d4d/d6c/d61/dd1/l126 0 2024-03-22T10:43:06.255 INFO:tasks.workunit.client.1.smithi150.stdout:9/860: fdatasync d1/d8/de/df/d36/d40/d41/fa9 0 2024-03-22T10:43:06.260 INFO:tasks.workunit.client.1.smithi150.stdout:5/836: rename d0/d8/dc/de/f13 to d0/d8/dc/de/d14/d2e/d46/d9c/f11e 0 2024-03-22T10:43:06.261 INFO:tasks.workunit.client.1.smithi150.stdout:6/805: symlink db/dd/d15/d30/ddf/d103/l119 0 2024-03-22T10:43:06.263 INFO:tasks.workunit.client.1.smithi150.stdout:6/806: dread - db/dd/faa zero size 2024-03-22T10:43:06.265 INFO:tasks.workunit.client.1.smithi150.stdout:4/883: mknod d0/dce/d10e/d121/d7c/dd4/c127 0 2024-03-22T10:43:06.267 INFO:tasks.workunit.client.1.smithi150.stdout:6/807: chown db/dd/d15/d70/c75 11 1 2024-03-22T10:43:06.267 INFO:tasks.workunit.client.1.smithi150.stdout:7/820: dread d2/d5/d7/d8/d57/d4a/f22 [0,4194304] 0 2024-03-22T10:43:06.268 INFO:tasks.workunit.client.1.smithi150.stdout:3/782: unlink d5/da/d9d/d9f/cc6 0 2024-03-22T10:43:06.269 INFO:tasks.workunit.client.1.smithi150.stdout:6/808: chown db/dd/l38 87612 1 2024-03-22T10:43:06.279 INFO:tasks.workunit.client.1.smithi150.stdout:9/861: rename d1/d8/de/df/d24 to d1/d8/de/df/d36/d121 0 2024-03-22T10:43:06.283 INFO:tasks.workunit.client.1.smithi150.stdout:3/783: truncate d5/da/f45 6322641 0 2024-03-22T10:43:06.286 INFO:tasks.workunit.client.1.smithi150.stdout:4/884: rename d0/dce/d10e/d121/d7c/c103 to d0/d2f/d63/d7b/d9c/db1/dd7/c128 0 2024-03-22T10:43:06.288 INFO:tasks.workunit.client.1.smithi150.stdout:3/784: rmdir d5/d53 39 2024-03-22T10:43:06.290 INFO:tasks.workunit.client.1.smithi150.stdout:6/809: write db/dd/f14 [8461707,71413] 0 2024-03-22T10:43:06.297 INFO:tasks.workunit.client.1.smithi150.stdout:3/785: fsync d5/d22/d2f/d38/d3e/d5b/da6/ffb 0 2024-03-22T10:43:06.300 INFO:tasks.workunit.client.1.smithi150.stdout:4/885: chown d0/d2f/d10d/d4d/d9f/fb5 56432237 1 2024-03-22T10:43:06.301 INFO:tasks.workunit.client.1.smithi150.stdout:1/752: dwrite d1/d19/d1a/f49 [0,4194304] 0 2024-03-22T10:43:06.303 INFO:tasks.workunit.client.1.smithi150.stdout:5/837: dread d0/d8/f9a [0,4194304] 0 2024-03-22T10:43:06.303 INFO:tasks.workunit.client.1.smithi150.stdout:9/862: dread d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/fba [0,4194304] 0 2024-03-22T10:43:06.303 INFO:tasks.workunit.client.1.smithi150.stdout:6/810: mkdir db/dd/d15/d30/ddf/d11a 0 2024-03-22T10:43:06.307 INFO:tasks.workunit.client.1.smithi150.stdout:3/786: rmdir d5/d22/d2f/d38/d3e/d5b/da6 39 2024-03-22T10:43:06.307 INFO:tasks.workunit.client.1.smithi150.stdout:7/821: dwrite d2/d5/d7/d9/db/d6c/da8/ffa [0,4194304] 0 2024-03-22T10:43:06.307 INFO:tasks.workunit.client.1.smithi150.stdout:9/863: chown d1/d19/l27 958 1 2024-03-22T10:43:06.311 INFO:tasks.workunit.client.1.smithi150.stdout:4/886: symlink d0/dce/d10e/d121/d7c/d99/l129 0 2024-03-22T10:43:06.311 INFO:tasks.workunit.client.1.smithi150.stdout:5/838: mknod d0/d8/d101/c11f 0 2024-03-22T10:43:06.312 INFO:tasks.workunit.client.1.smithi150.stdout:3/787: fsync d5/d22/d37/fd8 0 2024-03-22T10:43:06.312 INFO:tasks.workunit.client.1.smithi150.stdout:7/822: write d2/d5/d7/fa4 [1024172,79095] 0 2024-03-22T10:43:06.314 INFO:tasks.workunit.client.1.smithi150.stdout:4/887: chown d0/d2f/d63/d7b/d10c 24345941 1 2024-03-22T10:43:06.318 INFO:tasks.workunit.client.1.smithi150.stdout:5/839: read - d0/d8/dc/de/d14/d2e/d45/f87 zero size 2024-03-22T10:43:06.324 INFO:tasks.workunit.client.1.smithi150.stdout:3/788: creat d5/d22/d2f/f107 x:0 0 0 2024-03-22T10:43:06.327 INFO:tasks.workunit.client.1.smithi150.stdout:4/888: truncate d0/d2f/d10d/d4d/d6c/d61/fbb 596515 0 2024-03-22T10:43:06.328 INFO:tasks.workunit.client.1.smithi150.stdout:3/789: stat d5/da/dad/ce2 0 2024-03-22T10:43:06.330 INFO:tasks.workunit.client.1.smithi150.stdout:4/889: chown d0/dce/d10e/d121/d7c/dc1/cc9 99951827 1 2024-03-22T10:43:06.331 INFO:tasks.workunit.client.1.smithi150.stdout:7/823: symlink d2/d5/d7/d9/db/d9f/l10f 0 2024-03-22T10:43:06.334 INFO:tasks.workunit.client.1.smithi150.stdout:5/840: link d0/lba d0/d8/dc/de/d14/d2e/d46/d9c/dc0/l120 0 2024-03-22T10:43:06.334 INFO:tasks.workunit.client.1.smithi150.stdout:4/890: creat d0/d2f/f12a x:0 0 0 2024-03-22T10:43:06.338 INFO:tasks.workunit.client.1.smithi150.stdout:5/841: chown d0/d8/dc/de/d14/fca 2526838 1 2024-03-22T10:43:06.342 INFO:tasks.workunit.client.1.smithi150.stdout:4/891: read d0/dce/d10e/d121/d7c/dc1/fcc [2170650,8905] 0 2024-03-22T10:43:06.344 INFO:tasks.workunit.client.1.smithi150.stdout:3/790: rename d5/d53/l66 to d5/da/d78/d86/l108 0 2024-03-22T10:43:06.346 INFO:tasks.workunit.client.1.smithi150.stdout:5/842: write d0/f10 [1283533,51721] 0 2024-03-22T10:43:06.351 INFO:tasks.workunit.client.1.smithi150.stdout:0/802: sync 2024-03-22T10:43:06.351 INFO:tasks.workunit.client.1.smithi150.stdout:2/943: sync 2024-03-22T10:43:06.351 INFO:tasks.workunit.client.1.smithi150.stdout:1/753: dread d1/d19/d1a/f2b [0,4194304] 0 2024-03-22T10:43:06.352 INFO:tasks.workunit.client.1.smithi150.stdout:7/824: getdents d2/d5/d7/d9/db/d9f/d96 0 2024-03-22T10:43:06.352 INFO:tasks.workunit.client.1.smithi150.stdout:0/803: dread - d0/d1/d84/d86/d42/f72 zero size 2024-03-22T10:43:06.353 INFO:tasks.workunit.client.1.smithi150.stdout:3/791: mkdir d5/da/d109 0 2024-03-22T10:43:06.355 INFO:tasks.workunit.client.1.smithi150.stdout:4/892: creat d0/dce/d10e/d121/d7c/dc1/f12b x:0 0 0 2024-03-22T10:43:06.356 INFO:tasks.workunit.client.1.smithi150.stdout:6/811: dread db/dd/d15/f20 [0,4194304] 0 2024-03-22T10:43:06.357 INFO:tasks.workunit.client.1.smithi150.stdout:3/792: stat d5/da/d78/d86/de8 0 2024-03-22T10:43:06.357 INFO:tasks.workunit.client.1.smithi150.stdout:7/825: fsync d2/d5/d7/d9/db/d9f/d45/ff2 0 2024-03-22T10:43:06.359 INFO:tasks.workunit.client.1.smithi150.stdout:1/754: creat d1/da/d2a/d4e/ff5 x:0 0 0 2024-03-22T10:43:06.360 INFO:tasks.workunit.client.1.smithi150.stdout:6/812: chown db/d52/fb1 1327884881 1 2024-03-22T10:43:06.362 INFO:tasks.workunit.client.1.smithi150.stdout:0/804: mkdir d0/d1/d84/d86/d11/d26/dd2/dff 0 2024-03-22T10:43:06.362 INFO:tasks.workunit.client.1.smithi150.stdout:9/864: dread d1/d8/de/df/f4d [0,4194304] 0 2024-03-22T10:43:06.363 INFO:tasks.workunit.client.1.smithi150.stdout:4/893: creat d0/dce/d10e/d121/d7c/f12c x:0 0 0 2024-03-22T10:43:06.365 INFO:tasks.workunit.client.1.smithi150.stdout:9/865: chown d1/d8/de/df/fd3 1332910 1 2024-03-22T10:43:06.369 INFO:tasks.workunit.client.1.smithi150.stdout:9/866: chown d1/d8/de/dbd/fc6 5970805 1 2024-03-22T10:43:06.372 INFO:tasks.workunit.client.1.smithi150.stdout:4/894: creat d0/d2f/d63/d7b/d10c/f12d x:0 0 0 2024-03-22T10:43:06.373 INFO:tasks.workunit.client.1.smithi150.stdout:0/805: mknod d0/d1/d84/d86/d42/d10/d38/d52/d9b/c100 0 2024-03-22T10:43:06.374 INFO:tasks.workunit.client.1.smithi150.stdout:4/895: read - d0/d2f/d10d/d68/fe5 zero size 2024-03-22T10:43:06.382 INFO:tasks.workunit.client.1.smithi150.stdout:7/826: write d2/d5/d7/d88/f8f [369295,13392] 0 2024-03-22T10:43:06.388 INFO:tasks.workunit.client.1.smithi150.stdout:5/843: dwrite d0/dae/de0/fcc [0,4194304] 0 2024-03-22T10:43:06.395 INFO:tasks.workunit.client.1.smithi150.stdout:6/813: dwrite db/dd/d15/d6a/fd9 [0,4194304] 0 2024-03-22T10:43:06.398 INFO:tasks.workunit.client.1.smithi150.stdout:6/814: stat db/dd/d25/fac 0 2024-03-22T10:43:06.401 INFO:tasks.workunit.client.1.smithi150.stdout:1/755: dwrite d1/d19/f92 [0,4194304] 0 2024-03-22T10:43:06.406 INFO:tasks.workunit.client.1.smithi150.stdout:1/756: chown d1/d19/d50/d51/da0/dcf 0 1 2024-03-22T10:43:06.411 INFO:tasks.workunit.client.1.smithi150.stdout:3/793: rename d5/d22/d2f/d38/d3e/d5b/da6/ffb to d5/d22/d37/f10a 0 2024-03-22T10:43:06.413 INFO:tasks.workunit.client.1.smithi150.stdout:4/896: dwrite d0/d2f/f50 [0,4194304] 0 2024-03-22T10:43:06.413 INFO:tasks.workunit.client.1.smithi150.stdout:7/827: creat d2/d5/d7/d8/d57/d4a/d8d/f110 x:0 0 0 2024-03-22T10:43:06.416 INFO:tasks.workunit.client.1.smithi150.stdout:5/844: mknod d0/d8/dc/de/d14/d68/c121 0 2024-03-22T10:43:06.419 INFO:tasks.workunit.client.1.smithi150.stdout:4/897: dread - d0/dce/d10e/d121/f123 zero size 2024-03-22T10:43:06.420 INFO:tasks.workunit.client.1.smithi150.stdout:9/867: mkdir d1/d19/d7d/dc1/dca/d111/d122 0 2024-03-22T10:43:06.421 INFO:tasks.workunit.client.1.smithi150.stdout:4/898: chown d0/d2f/d10d/d4d/d6c/f11e 267530827 1 2024-03-22T10:43:06.421 INFO:tasks.workunit.client.1.smithi150.stdout:6/815: mkdir db/dd/d15/d30/ddf/d11a/d11b 0 2024-03-22T10:43:06.424 INFO:tasks.workunit.client.1.smithi150.stdout:7/828: fsync d2/d5/d7/d9/db/d6c/d9c/fd1 0 2024-03-22T10:43:06.424 INFO:tasks.workunit.client.1.smithi150.stdout:5/845: truncate d0/d8/f76 286023 0 2024-03-22T10:43:06.424 INFO:tasks.workunit.client.1.smithi150.stdout:3/794: creat d5/da/d109/f10b x:0 0 0 2024-03-22T10:43:06.425 INFO:tasks.workunit.client.1.smithi150.stdout:9/868: rmdir d1/d8/de/df/da2 39 2024-03-22T10:43:06.435 INFO:tasks.workunit.client.1.smithi150.stdout:1/757: dwrite d1/d19/d50/d6e/fd3 [0,4194304] 0 2024-03-22T10:43:06.438 INFO:tasks.workunit.client.1.smithi150.stdout:3/795: write d5/d22/d2f/d38/fb4 [1246002,123472] 0 2024-03-22T10:43:06.442 INFO:tasks.workunit.client.1.smithi150.stdout:5/846: mkdir d0/d8/dc/d1d/d89/d122 0 2024-03-22T10:43:06.447 INFO:tasks.workunit.client.1.smithi150.stdout:7/829: mkdir d2/d5/d7/d8/d57/d4a/d111 0 2024-03-22T10:43:06.449 INFO:tasks.workunit.client.1.smithi150.stdout:3/796: fdatasync d5/da/f57 0 2024-03-22T10:43:06.453 INFO:tasks.workunit.client.1.smithi150.stdout:6/816: dwrite db/dd/d15/d60/f67 [4194304,4194304] 0 2024-03-22T10:43:06.462 INFO:tasks.workunit.client.1.smithi150.stdout:9/869: dwrite d1/d8/fde [0,4194304] 0 2024-03-22T10:43:06.468 INFO:tasks.workunit.client.1.smithi150.stdout:3/797: rename d5/d22/d2f/d9c to d5/d22/d2f/d10c 0 2024-03-22T10:43:06.469 INFO:tasks.workunit.client.1.smithi150.stdout:5/847: write d0/d8/dc/de/d14/d2e/f62 [615382,85238] 0 2024-03-22T10:43:06.473 INFO:tasks.workunit.client.1.smithi150.stdout:7/830: read d2/d5/d7/d8/d46/d49/fae [232741,36816] 0 2024-03-22T10:43:06.480 INFO:tasks.workunit.client.1.smithi150.stdout:6/817: symlink db/d52/d9f/dc6/df1/db4/dfd/l11c 0 2024-03-22T10:43:06.480 INFO:tasks.workunit.client.1.smithi150.stdout:3/798: read d5/d22/d2f/d38/d3e/f6f [3106159,60054] 0 2024-03-22T10:43:06.486 INFO:tasks.workunit.client.1.smithi150.stdout:7/831: rmdir d2/d5/d7/d9/db/d6c 39 2024-03-22T10:43:06.492 INFO:tasks.workunit.client.1.smithi150.stdout:3/799: unlink d5/d22/d2f/f107 0 2024-03-22T10:43:06.497 INFO:tasks.workunit.client.1.smithi150.stdout:3/800: chown d5/d22/d37/l50 926651 1 2024-03-22T10:43:06.498 INFO:tasks.workunit.client.1.smithi150.stdout:9/870: dwrite d1/d8/de/df/d36/d40/d44/db5/f11c [0,4194304] 0 2024-03-22T10:43:06.499 INFO:tasks.workunit.client.1.smithi150.stdout:5/848: rename d0/d8/dc/de/d14/d2e/l69 to d0/d8/dc/de/d14/l123 0 2024-03-22T10:43:06.500 INFO:tasks.workunit.client.1.smithi150.stdout:8/854: sync 2024-03-22T10:43:06.503 INFO:tasks.workunit.client.1.smithi150.stdout:9/871: stat d1/d8/de/d73/d101/d32/c9c 0 2024-03-22T10:43:06.512 INFO:tasks.workunit.client.1.smithi150.stdout:6/818: dread db/dd/d15/d28/d6c/d9b/f106 [0,4194304] 0 2024-03-22T10:43:06.515 INFO:tasks.workunit.client.1.smithi150.stdout:5/849: symlink d0/d33/d6f/l124 0 2024-03-22T10:43:06.518 INFO:tasks.workunit.client.1.smithi150.stdout:3/801: rename d5/da/d78/d86/de8 to d5/d22/d2f/d38/d59/d88/d10d 0 2024-03-22T10:43:06.521 INFO:tasks.workunit.client.1.smithi150.stdout:8/855: unlink d0/d2/d19/d29/d37/fb3 0 2024-03-22T10:43:06.524 INFO:tasks.workunit.client.1.smithi150.stdout:4/899: dwrite d0/d2f/d10d/f24 [0,4194304] 0 2024-03-22T10:43:06.527 INFO:tasks.workunit.client.1.smithi150.stdout:4/900: readlink d0/d47/l79 0 2024-03-22T10:43:06.528 INFO:tasks.workunit.client.1.smithi150.stdout:6/819: unlink db/dd/d15/d6a/f6b 0 2024-03-22T10:43:06.531 INFO:tasks.workunit.client.1.smithi150.stdout:1/758: dwrite d1/d19/d50/d51/f9c [0,4194304] 0 2024-03-22T10:43:06.533 INFO:tasks.workunit.client.1.smithi150.stdout:5/850: unlink d0/d8/dc/d1d/d53/d63/fd6 0 2024-03-22T10:43:06.538 INFO:tasks.workunit.client.1.smithi150.stdout:1/759: truncate d1/da/d15/d4f/fa4 411750 0 2024-03-22T10:43:06.545 INFO:tasks.workunit.client.1.smithi150.stdout:3/802: fdatasync d5/da/f45 0 2024-03-22T10:43:06.550 INFO:tasks.workunit.client.1.smithi150.stdout:6/820: creat db/dd/d15/d60/d50/dc1/f11d x:0 0 0 2024-03-22T10:43:06.551 INFO:tasks.workunit.client.1.smithi150.stdout:7/832: write d2/ffc [1157922,7875] 0 2024-03-22T10:43:06.558 INFO:tasks.workunit.client.1.smithi150.stdout:1/760: rmdir d1/da/d15/d18 39 2024-03-22T10:43:06.559 INFO:tasks.workunit.client.1.smithi150.stdout:6/821: read db/dd/d15/d28/d6c/d9b/ff6 [46740,22497] 0 2024-03-22T10:43:06.559 INFO:tasks.workunit.client.1.smithi150.stdout:9/872: write d1/d8/de/df/d36/d40/d41/fa9 [685791,75057] 0 2024-03-22T10:43:06.562 INFO:tasks.workunit.client.1.smithi150.stdout:5/851: getdents d0/d8/d101/d11b 0 2024-03-22T10:43:06.567 INFO:tasks.workunit.client.1.smithi150.stdout:6/822: chown db/f44 41964 1 2024-03-22T10:43:06.567 INFO:tasks.workunit.client.1.smithi150.stdout:4/901: rmdir d0/dce/dfd/d122 0 2024-03-22T10:43:06.571 INFO:tasks.workunit.client.1.smithi150.stdout:1/761: symlink d1/d19/d50/d6e/lf6 0 2024-03-22T10:43:06.576 INFO:tasks.workunit.client.1.smithi150.stdout:3/803: rename d5/d22/d2f/d38/d3e/ce9 to d5/da/d8d/c10e 0 2024-03-22T10:43:06.577 INFO:tasks.workunit.client.1.smithi150.stdout:6/823: truncate db/dd/d15/d30/f7c 352164 0 2024-03-22T10:43:06.578 INFO:tasks.workunit.client.1.smithi150.stdout:7/833: mknod d2/c112 0 2024-03-22T10:43:06.579 INFO:tasks.workunit.client.1.smithi150.stdout:8/856: dwrite d0/d3c/fce [0,4194304] 0 2024-03-22T10:43:06.587 INFO:tasks.workunit.client.1.smithi150.stdout:5/852: mknod d0/d8/dc/de/d14/d2e/d46/d4d/c125 0 2024-03-22T10:43:06.590 INFO:tasks.workunit.client.1.smithi150.stdout:4/902: rmdir d0/d47/d72/de7/d10a 39 2024-03-22T10:43:06.593 INFO:tasks.workunit.client.1.smithi150.stdout:3/804: truncate d5/d22/d2f/d38/d3e/f3f 2022333 0 2024-03-22T10:43:06.596 INFO:tasks.workunit.client.1.smithi150.stdout:1/762: symlink d1/da/d15/lf7 0 2024-03-22T10:43:06.604 INFO:tasks.workunit.client.1.smithi150.stdout:9/873: dwrite d1/d8/de/df/d36/d40/d44/db5/dda/d104/dc3/fd2 [0,4194304] 0 2024-03-22T10:43:06.607 INFO:tasks.workunit.client.1.smithi150.stdout:6/824: getdents db/d52/d9f/dc6/df1/db4 0 2024-03-22T10:43:06.609 INFO:tasks.workunit.client.1.smithi150.stdout:8/857: getdents d0/d2/d19/d29/d2e/d70 0 2024-03-22T10:43:06.616 INFO:tasks.workunit.client.1.smithi150.stdout:3/805: rmdir d5/da/d62/dba/dd5 39 2024-03-22T10:43:06.622 INFO:tasks.workunit.client.1.smithi150.stdout:7/834: write d2/d5/d7/f2b [1544508,93567] 0 2024-03-22T10:43:06.628 INFO:tasks.workunit.client.1.smithi150.stdout:3/806: mknod d5/d22/d37/c10f 0 2024-03-22T10:43:06.631 INFO:tasks.workunit.client.1.smithi150.stdout:8/858: mknod d0/d2/d19/d29/d37/c119 0 2024-03-22T10:43:06.634 INFO:tasks.workunit.client.1.smithi150.stdout:6/825: dwrite db/dd/d82/d84/fb5 [0,4194304] 0 2024-03-22T10:43:06.638 INFO:tasks.workunit.client.1.smithi150.stdout:4/903: dwrite d0/d2f/d10d/d68/fe5 [0,4194304] 0 2024-03-22T10:43:06.641 INFO:tasks.workunit.client.1.smithi150.stdout:0/806: sync 2024-03-22T10:43:06.641 INFO:tasks.workunit.client.1.smithi150.stdout:2/944: sync 2024-03-22T10:43:06.645 INFO:tasks.workunit.client.1.smithi150.stdout:7/835: write d2/d5/d7/d9/db/d9f/d45/fe3 [1545944,21573] 0 2024-03-22T10:43:06.648 INFO:tasks.workunit.client.1.smithi150.stdout:8/859: fdatasync d0/d2/d19/d29/d37/fd6 0 2024-03-22T10:43:06.655 INFO:tasks.workunit.client.1.smithi150.stdout:5/853: dwrite d0/d8/dc/de/d14/f58 [0,4194304] 0 2024-03-22T10:43:06.657 INFO:tasks.workunit.client.1.smithi150.stdout:1/763: truncate d1/d19/d1a/d7d/f82 3468253 0 2024-03-22T10:43:06.661 INFO:tasks.workunit.client.1.smithi150.stdout:2/945: symlink d12/d19/db6/da9/d97/d78/dbd/l128 0 2024-03-22T10:43:06.661 INFO:tasks.workunit.client.1.smithi150.stdout:6/826: creat db/dd/d15/d30/ddf/d11a/d11b/f11e x:0 0 0 2024-03-22T10:43:06.662 INFO:tasks.workunit.client.1.smithi150.stdout:8/860: symlink d0/d3c/l11a 0 2024-03-22T10:43:06.671 INFO:tasks.workunit.client.1.smithi150.stdout:1/764: creat d1/d19/d50/d51/da0/dcf/dd9/ff8 x:0 0 0 2024-03-22T10:43:06.672 INFO:tasks.workunit.client.1.smithi150.stdout:5/854: write d0/d8/dc/de/d14/d2e/dbc/f103 [486471,122235] 0 2024-03-22T10:43:06.680 INFO:tasks.workunit.client.1.smithi150.stdout:4/904: write d0/d2f/f3e [1004614,26022] 0 2024-03-22T10:43:06.681 INFO:tasks.workunit.client.1.smithi150.stdout:2/946: symlink d12/d19/db6/da9/d2c/d65/l129 0 2024-03-22T10:43:06.681 INFO:tasks.workunit.client.1.smithi150.stdout:1/765: chown d1/d19/d50/d6e/lca 158 1 2024-03-22T10:43:06.686 INFO:tasks.workunit.client.1.smithi150.stdout:8/861: write d0/d2/d19/d29/d37/d7d/dc1/ff4 [878389,77818] 0 2024-03-22T10:43:06.691 INFO:tasks.workunit.client.1.smithi150.stdout:5/855: dread d0/ddc/fdd [0,4194304] 0 2024-03-22T10:43:06.695 INFO:tasks.workunit.client.1.smithi150.stdout:0/807: dwrite d0/d1/d84/d86/d42/f2b [4194304,4194304] 0 2024-03-22T10:43:06.695 INFO:tasks.workunit.client.1.smithi150.stdout:9/874: dwrite d1/d8/de/df/d36/d121/f89 [4194304,4194304] 0 2024-03-22T10:43:06.698 INFO:tasks.workunit.client.1.smithi150.stdout:8/862: dread - d0/d2/f117 zero size 2024-03-22T10:43:06.706 INFO:tasks.workunit.client.1.smithi150.stdout:2/947: write d12/d35/f11b [1719771,20586] 0 2024-03-22T10:43:06.706 INFO:tasks.workunit.client.1.smithi150.stdout:4/905: creat d0/dce/d10e/d121/d7c/dc1/f12e x:0 0 0 2024-03-22T10:43:06.710 INFO:tasks.workunit.client.1.smithi150.stdout:3/807: dwrite d5/d22/d2f/d38/f85 [0,4194304] 0 2024-03-22T10:43:06.717 INFO:tasks.workunit.client.1.smithi150.stdout:0/808: mkdir d0/d12/d101 0 2024-03-22T10:43:06.717 INFO:tasks.workunit.client.1.smithi150.stdout:1/766: creat d1/da/d15/ff9 x:0 0 0 2024-03-22T10:43:06.723 INFO:tasks.workunit.client.1.smithi150.stdout:6/827: truncate db/dd/d25/f6d 357682 0 2024-03-22T10:43:06.729 INFO:tasks.workunit.client.1.smithi150.stdout:5/856: truncate d0/d8/dc/de/d14/d2e/d46/d4d/fb7 135640 0 2024-03-22T10:43:06.730 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:06 smithi150 ceph-mon[34518]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-22T10:43:06.730 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:06 smithi150 ceph-mon[34518]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-22T10:43:06.730 INFO:tasks.workunit.client.1.smithi150.stdout:9/875: symlink d1/d8/de/df/d36/l123 0 2024-03-22T10:43:06.731 INFO:tasks.workunit.client.1.smithi150.stdout:1/767: chown d1/da/l25 1 1 2024-03-22T10:43:06.735 INFO:tasks.workunit.client.1.smithi150.stdout:5/857: write d0/d8/dc/de/d14/d2e/dbc/fc7 [4240037,128828] 0 2024-03-22T10:43:06.735 INFO:tasks.workunit.client.1.smithi150.stdout:8/863: mkdir d0/d2/d19/d29/d2f/df/d1b/dea/d11b 0 2024-03-22T10:43:06.738 INFO:tasks.workunit.client.1.smithi150.stdout:5/858: chown d0/f6 1 1 2024-03-22T10:43:06.740 INFO:tasks.workunit.client.1.smithi150.stdout:5/859: stat d0/d33/la1 0 2024-03-22T10:43:06.740 INFO:tasks.workunit.client.1.smithi150.stdout:2/948: mkdir d12/d19/d12a 0 2024-03-22T10:43:06.743 INFO:tasks.workunit.client.1.smithi150.stdout:3/808: creat d5/d22/dcb/f110 x:0 0 0 2024-03-22T10:43:06.751 INFO:tasks.workunit.client.1.smithi150.stdout:2/949: rmdir d12/d21/ddc 39 2024-03-22T10:43:06.754 INFO:tasks.workunit.client.1.smithi150.stdout:9/876: link d1/d8/de/df/d36/d40/d44/db5/dda/d104/dc3/fd2 d1/d8/de/f124 0 2024-03-22T10:43:06.754 INFO:tasks.workunit.client.1.smithi150.stdout:2/950: stat d12/d35/d7d/c119 0 2024-03-22T10:43:06.758 INFO:tasks.workunit.client.1.smithi150.stdout:8/864: creat d0/d2/d46/f11c x:0 0 0 2024-03-22T10:43:06.766 INFO:tasks.workunit.client.1.smithi150.stdout:0/809: getdents d0/d1/d84/d86/d11/d66/d35/d5f 0 2024-03-22T10:43:06.766 INFO:tasks.workunit.client.1.smithi150.stdout:2/951: rmdir d12/d21/d81 39 2024-03-22T10:43:06.766 INFO:tasks.workunit.client.1.smithi150.stdout:8/865: rmdir d0/d2/d19/d29/d2f/df/dff/d110 39 2024-03-22T10:43:06.770 INFO:tasks.workunit.client.1.smithi150.stdout:5/860: dwrite d0/f3 [4194304,4194304] 0 2024-03-22T10:43:06.771 INFO:tasks.workunit.client.1.smithi150.stdout:1/768: write d1/da/d15/f93 [876695,29400] 0 2024-03-22T10:43:06.781 INFO:tasks.workunit.client.1.smithi150.stdout:6/828: write db/dd/d15/d60/fcd [735781,128126] 0 2024-03-22T10:43:06.781 INFO:tasks.workunit.client.1.smithi150.stdout:8/866: fdatasync d0/d5d/dd2/f49 0 2024-03-22T10:43:06.789 INFO:tasks.workunit.client.1.smithi150.stdout:0/810: fdatasync d0/d1/d84/d86/d11/d66/d35/d92/fad 0 2024-03-22T10:43:06.790 INFO:tasks.workunit.client.1.smithi150.stdout:6/829: rmdir db/dd/d15/d60/d50/dc1 39 2024-03-22T10:43:06.797 INFO:tasks.workunit.client.1.smithi150.stdout:9/877: dwrite d1/d8/fa1 [4194304,4194304] 0 2024-03-22T10:43:06.798 INFO:tasks.workunit.client.1.smithi150.stdout:8/867: truncate d0/f5b 1824758 0 2024-03-22T10:43:06.798 INFO:tasks.workunit.client.1.smithi150.stdout:3/809: dwrite d5/d22/d2f/d38/d3e/d5b/f9a [0,4194304] 0 2024-03-22T10:43:06.800 INFO:tasks.workunit.client.1.smithi150.stdout:0/811: fdatasync d0/d1/d84/d86/d11/d66/d35/fde 0 2024-03-22T10:43:06.800 INFO:tasks.workunit.client.1.smithi150.stdout:5/861: dwrite d0/d8/dc/de/d14/d2e/dbc/f103 [0,4194304] 0 2024-03-22T10:43:06.800 INFO:tasks.workunit.client.1.smithi150.stdout:6/830: creat db/d52/d9f/dc6/f11f x:0 0 0 2024-03-22T10:43:06.803 INFO:tasks.workunit.client.1.smithi150.stdout:6/831: chown db/dd/d15/d57/lf8 212358 1 2024-03-22T10:43:06.805 INFO:tasks.workunit.client.1.smithi150.stdout:1/769: link d1/d19/d71/lb0 d1/d19/d57/lfa 0 2024-03-22T10:43:06.807 INFO:tasks.workunit.client.1.smithi150.stdout:8/868: creat d0/ddd/dfc/f11d x:0 0 0 2024-03-22T10:43:06.809 INFO:tasks.workunit.client.1.smithi150.stdout:3/810: symlink d5/da/d78/d86/d81/l111 0 2024-03-22T10:43:06.810 INFO:tasks.workunit.client.1.smithi150.stdout:3/811: fsync d5/f46 0 2024-03-22T10:43:06.811 INFO:tasks.workunit.client.1.smithi150.stdout:5/862: creat d0/d8/dc/de/d116/f126 x:0 0 0 2024-03-22T10:43:06.818 INFO:tasks.workunit.client.1.smithi150.stdout:9/878: rename d1/d49/f31 to d1/d49/d98/f125 0 2024-03-22T10:43:06.819 INFO:tasks.workunit.client.1.smithi150.stdout:5/863: mknod d0/d8/dc/de/d96/db8/c127 0 2024-03-22T10:43:06.824 INFO:tasks.workunit.client.1.smithi150.stdout:5/864: read - d0/d8/dc/de/d116/f126 zero size 2024-03-22T10:43:06.827 INFO:tasks.workunit.client.1.smithi150.stdout:1/770: dread d1/d19/d1a/f49 [0,4194304] 0 2024-03-22T10:43:06.828 INFO:tasks.workunit.client.1.smithi150.stdout:0/812: dwrite d0/d1/d84/d86/d11/d26/f2e [0,4194304] 0 2024-03-22T10:43:06.831 INFO:tasks.workunit.client.1.smithi150.stdout:5/865: fsync d0/d8/dc/de/d14/d4f/fd1 0 2024-03-22T10:43:06.831 INFO:tasks.workunit.client.1.smithi150.stdout:8/869: dwrite d0/d2/f67 [0,4194304] 0 2024-03-22T10:43:06.832 INFO:tasks.workunit.client.1.smithi150.stdout:1/771: mknod d1/d19/d50/cfb 0 2024-03-22T10:43:06.838 INFO:tasks.workunit.client.1.smithi150.stdout:3/812: dwrite d5/d22/d37/fd7 [0,4194304] 0 2024-03-22T10:43:06.838 INFO:tasks.workunit.client.1.smithi150.stdout:1/772: stat d1/da/d15/d18/d4b/dc8 0 2024-03-22T10:43:06.839 INFO:tasks.workunit.client.1.smithi150.stdout:5/866: write d0/d8/dc/de/d14/d2e/dbc/fc3 [3534577,74600] 0 2024-03-22T10:43:06.839 INFO:tasks.workunit.client.1.smithi150.stdout:8/870: creat d0/d3c/d71/f11e x:0 0 0 2024-03-22T10:43:06.842 INFO:tasks.workunit.client.1.smithi150.stdout:0/813: creat d0/d1/d84/d86/d11/d26/dd2/f102 x:0 0 0 2024-03-22T10:43:06.843 INFO:tasks.workunit.client.1.smithi150.stdout:3/813: write d5/fcf [374152,66515] 0 2024-03-22T10:43:06.847 INFO:tasks.workunit.client.1.smithi150.stdout:8/871: write d0/d2/d19/d29/d2e/d70/fb8 [67816,16292] 0 2024-03-22T10:43:06.852 INFO:tasks.workunit.client.1.smithi150.stdout:5/867: rmdir d0/d8/dc/de/d14/d2e/d45 39 2024-03-22T10:43:06.853 INFO:tasks.workunit.client.1.smithi150.stdout:3/814: symlink d5/da/d9d/d9f/l112 0 2024-03-22T10:43:06.862 INFO:tasks.workunit.client.1.smithi150.stdout:5/868: creat d0/d8/dc/de/f128 x:0 0 0 2024-03-22T10:43:06.865 INFO:tasks.workunit.client.1.smithi150.stdout:1/773: dwrite d1/d19/d50/d6e/f77 [0,4194304] 0 2024-03-22T10:43:06.866 INFO:tasks.workunit.client.1.smithi150.stdout:0/814: dwrite d0/d1/d84/ff8 [0,4194304] 0 2024-03-22T10:43:06.871 INFO:tasks.workunit.client.1.smithi150.stdout:5/869: mknod d0/d8/dc/de/c129 0 2024-03-22T10:43:06.872 INFO:tasks.workunit.client.1.smithi150.stdout:0/815: unlink d0/d1/l7a 0 2024-03-22T10:43:06.874 INFO:tasks.workunit.client.1.smithi150.stdout:1/774: truncate d1/d79/f85 896986 0 2024-03-22T10:43:06.878 INFO:tasks.workunit.client.1.smithi150.stdout:0/816: getdents d0/d1/d84/d86/d42/d10/d38 0 2024-03-22T10:43:06.880 INFO:tasks.workunit.client.1.smithi150.stdout:1/775: creat d1/da/d15/d18/d4b/d8a/ffc x:0 0 0 2024-03-22T10:43:06.882 INFO:tasks.workunit.client.1.smithi150.stdout:0/817: creat d0/d1/d84/d86/da6/f103 x:0 0 0 2024-03-22T10:43:06.885 INFO:tasks.workunit.client.1.smithi150.stdout:1/776: mkdir d1/da/d2a/d65/d78/dfd 0 2024-03-22T10:43:06.887 INFO:tasks.workunit.client.1.smithi150.stdout:1/777: read d1/d19/d50/d6e/fb9 [2528636,33365] 0 2024-03-22T10:43:06.887 INFO:tasks.workunit.client.1.smithi150.stdout:0/818: mkdir d0/d1/d84/d86/d11/d26/dd2/df0/d104 0 2024-03-22T10:43:06.889 INFO:tasks.workunit.client.1.smithi150.stdout:1/778: symlink d1/d79/lfe 0 2024-03-22T10:43:06.891 INFO:tasks.workunit.client.1.smithi150.stdout:1/779: fsync d1/da/d2a/d4e/fd7 0 2024-03-22T10:43:06.895 INFO:tasks.workunit.client.1.smithi150.stdout:1/780: read d1/da/d2a/f7a [1059967,76028] 0 2024-03-22T10:43:06.905 INFO:tasks.workunit.client.1.smithi150.stdout:0/819: dread d0/f89 [0,4194304] 0 2024-03-22T10:43:06.909 INFO:tasks.workunit.client.1.smithi150.stdout:0/820: dread - d0/d1/d84/d86/fed zero size 2024-03-22T10:43:06.909 INFO:tasks.workunit.client.1.smithi150.stdout:0/821: dread - d0/d1/d84/d86/d42/f95 zero size 2024-03-22T10:43:06.911 INFO:tasks.workunit.client.1.smithi150.stdout:0/822: mkdir d0/d1/d84/d86/d105 0 2024-03-22T10:43:06.922 INFO:tasks.workunit.client.1.smithi150.stdout:0/823: creat d0/d1/d84/d86/d11/d66/d35/d92/f106 x:0 0 0 2024-03-22T10:43:06.930 INFO:tasks.workunit.client.1.smithi150.stdout:0/824: mknod d0/d1/d84/d86/d42/d10/d38/d52/d9b/c107 0 2024-03-22T10:43:06.938 INFO:tasks.workunit.client.1.smithi150.stdout:0/825: write d0/d1/d84/d86/d42/f82 [2712705,11642] 0 2024-03-22T10:43:06.952 INFO:tasks.workunit.client.1.smithi150.stdout:5/870: fsync d0/d8/dc/de/d14/d2e/d46/d4d/fb7 0 2024-03-22T10:43:06.968 INFO:tasks.workunit.client.1.smithi150.stdout:0/826: link d0/d1/d84/d86/d42/d10/d38/d52/d9b/dc2/fcb d0/d1/d84/d86/d105/f108 0 2024-03-22T10:43:06.976 INFO:tasks.workunit.client.1.smithi150.stdout:5/871: dwrite d0/d8/dc/de/d96/db8/fe1 [4194304,4194304] 0 2024-03-22T10:43:06.986 INFO:tasks.workunit.client.1.smithi150.stdout:3/815: truncate d5/d53/f71 2664016 0 2024-03-22T10:43:06.989 INFO:tasks.workunit.client.1.smithi150.stdout:9/879: dwrite d1/d8/de/df/f4d [0,4194304] 0 2024-03-22T10:43:06.994 INFO:tasks.workunit.client.1.smithi150.stdout:9/880: stat d1/d8/de/df/d36/d40 0 2024-03-22T10:43:06.997 INFO:tasks.workunit.client.1.smithi150.stdout:3/816: write d5/d22/d2f/d38/d59/dff/f103 [222232,46255] 0 2024-03-22T10:43:06.999 INFO:tasks.workunit.client.1.smithi150.stdout:1/781: write d1/da/d15/f3e [1841316,48825] 0 2024-03-22T10:43:07.002 INFO:tasks.workunit.client.1.smithi150.stdout:3/817: chown d5/d22/d2f/d38/d3e/d72/ff8 191919696 1 2024-03-22T10:43:07.003 INFO:tasks.workunit.client.1.smithi150.stdout:4/906: sync 2024-03-22T10:43:07.003 INFO:tasks.workunit.client.1.smithi150.stdout:7/836: sync 2024-03-22T10:43:07.005 INFO:tasks.workunit.client.1.smithi150.stdout:7/837: write d2/d5/d7/d9/f28 [6294175,128661] 0 2024-03-22T10:43:07.010 INFO:tasks.workunit.client.1.smithi150.stdout:7/838: chown d2/d5/d7/d9/cee 27 1 2024-03-22T10:43:07.010 INFO:tasks.workunit.client.1.smithi150.stdout:5/872: rename d0/d8/dc/de/d14/d2e/f54 to d0/d8/dc/de/d14/d2e/d46/d9c/dc0/f12a 0 2024-03-22T10:43:07.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:06 smithi115 ceph-mon[29111]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-22T10:43:07.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:06 smithi115 ceph-mon[29111]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-22T10:43:07.012 INFO:tasks.workunit.client.1.smithi150.stdout:9/881: symlink d1/d19/d7d/dc1/l126 0 2024-03-22T10:43:07.024 INFO:tasks.workunit.client.1.smithi150.stdout:5/873: mknod d0/df6/c12b 0 2024-03-22T10:43:07.026 INFO:tasks.workunit.client.1.smithi150.stdout:3/818: dwrite d5/d22/d2f/d10c/fe1 [0,4194304] 0 2024-03-22T10:43:07.027 INFO:tasks.workunit.client.1.smithi150.stdout:9/882: rmdir d1/d8/de/df/d36/d40/d44/db5/dda/d104 39 2024-03-22T10:43:07.033 INFO:tasks.workunit.client.1.smithi150.stdout:1/782: creat d1/da/fff x:0 0 0 2024-03-22T10:43:07.034 INFO:tasks.workunit.client.1.smithi150.stdout:5/874: symlink d0/d8/dc/de/d14/l12c 0 2024-03-22T10:43:07.036 INFO:tasks.workunit.client.1.smithi150.stdout:9/883: chown d1/d8/de/df/d36/d121/d4e/de9/lf3 35818542 1 2024-03-22T10:43:07.038 INFO:tasks.workunit.client.1.smithi150.stdout:3/819: symlink d5/d22/d2f/l113 0 2024-03-22T10:43:07.041 INFO:tasks.workunit.client.1.smithi150.stdout:1/783: truncate d1/da/d15/ff9 600332 0 2024-03-22T10:43:07.043 INFO:tasks.workunit.client.1.smithi150.stdout:1/784: stat d1/d19/d1a/dba 0 2024-03-22T10:43:07.045 INFO:tasks.workunit.client.1.smithi150.stdout:3/820: symlink d5/d22/d2f/d38/d59/d88/d10d/l114 0 2024-03-22T10:43:07.049 INFO:tasks.workunit.client.1.smithi150.stdout:5/875: creat d0/d8/dc/de/d14/d2e/d46/d9c/f12d x:0 0 0 2024-03-22T10:43:07.049 INFO:tasks.workunit.client.1.smithi150.stdout:0/827: dwrite d0/d1/d84/d86/f6b [4194304,4194304] 0 2024-03-22T10:43:07.050 INFO:tasks.workunit.client.1.smithi150.stdout:3/821: mknod d5/d22/d2f/c115 0 2024-03-22T10:43:07.053 INFO:tasks.workunit.client.1.smithi150.stdout:1/785: symlink d1/l100 0 2024-03-22T10:43:07.054 INFO:tasks.workunit.client.1.smithi150.stdout:5/876: chown d0/d8/dc/de/d14/d2e/d5a 1861 1 2024-03-22T10:43:07.057 INFO:tasks.workunit.client.1.smithi150.stdout:0/828: symlink d0/d1/d84/d86/d42/d10/d38/l109 0 2024-03-22T10:43:07.063 INFO:tasks.workunit.client.1.smithi150.stdout:0/829: rmdir d0/d1/d84/d86/d11/d66/d35 39 2024-03-22T10:43:07.064 INFO:tasks.workunit.client.1.smithi150.stdout:7/839: dread d2/d5/d7/d8/d57/d4a/f33 [4194304,4194304] 0 2024-03-22T10:43:07.069 INFO:tasks.workunit.client.1.smithi150.stdout:0/830: creat d0/d1/d84/f10a x:0 0 0 2024-03-22T10:43:07.082 INFO:tasks.workunit.client.1.smithi150.stdout:9/884: dwrite d1/d8/de/d73/d101/d29/f118 [0,4194304] 0 2024-03-22T10:43:07.087 INFO:tasks.workunit.client.1.smithi150.stdout:5/877: truncate d0/d8/dc/de/d14/d2e/dbc/fc3 2432449 0 2024-03-22T10:43:07.087 INFO:tasks.workunit.client.1.smithi150.stdout:0/831: unlink d0/d1/d84/d86/d11/d66/d37/ffd 0 2024-03-22T10:43:07.098 INFO:tasks.workunit.client.1.smithi150.stdout:5/878: read d0/d8/dc/de/d14/d2e/d46/d9c/dc0/ff5 [146640,34417] 0 2024-03-22T10:43:07.107 INFO:tasks.workunit.client.1.smithi150.stdout:1/786: dwrite d1/da/d2a/d65/f76 [0,4194304] 0 2024-03-22T10:43:07.114 INFO:tasks.workunit.client.1.smithi150.stdout:5/879: mkdir d0/d8/d101/d11b/d12e 0 2024-03-22T10:43:07.117 INFO:tasks.workunit.client.1.smithi150.stdout:5/880: fdatasync d0/d8/dc/de/d14/d2e/f62 0 2024-03-22T10:43:07.124 INFO:tasks.workunit.client.1.smithi150.stdout:9/885: dread d1/d8/de/df/d36/d121/d4e/f77 [0,4194304] 0 2024-03-22T10:43:07.125 INFO:tasks.workunit.client.1.smithi150.stdout:0/832: dwrite d0/d1/d84/d86/f8d [0,4194304] 0 2024-03-22T10:43:07.131 INFO:tasks.workunit.client.1.smithi150.stdout:0/833: creat d0/d12/f10b x:0 0 0 2024-03-22T10:43:07.136 INFO:tasks.workunit.client.1.smithi150.stdout:5/881: truncate d0/d8/dc/d1d/d53/d63/f9d 861401 0 2024-03-22T10:43:07.137 INFO:tasks.workunit.client.1.smithi150.stdout:7/840: dread d2/d5/d7/d8/d46/fd2 [0,4194304] 0 2024-03-22T10:43:07.142 INFO:tasks.workunit.client.1.smithi150.stdout:7/841: write d2/d5/d7/d38/fd3 [857754,23474] 0 2024-03-22T10:43:07.142 INFO:tasks.workunit.client.1.smithi150.stdout:9/886: chown d1/d8/de/df/d36/d40/d44/db5/dda/d104/f8d 1699 1 2024-03-22T10:43:07.144 INFO:tasks.workunit.client.1.smithi150.stdout:5/882: creat d0/d37/db4/f12f x:0 0 0 2024-03-22T10:43:07.150 INFO:tasks.workunit.client.1.smithi150.stdout:1/787: dwrite d1/da/d2a/faa [0,4194304] 0 2024-03-22T10:43:07.150 INFO:tasks.workunit.client.1.smithi150.stdout:9/887: fdatasync d1/d8/de/df/d36/f7c 0 2024-03-22T10:43:07.150 INFO:tasks.workunit.client.1.smithi150.stdout:5/883: readlink d0/d33/la1 0 2024-03-22T10:43:07.158 INFO:tasks.workunit.client.1.smithi150.stdout:1/788: write d1/d19/d50/d51/da0/dcf/dd6/faf [1310397,102483] 0 2024-03-22T10:43:07.159 INFO:tasks.workunit.client.1.smithi150.stdout:9/888: unlink d1/d8/de/d73/cbe 0 2024-03-22T10:43:07.169 INFO:tasks.workunit.client.1.smithi150.stdout:5/884: unlink d0/d8/dc/de/d14/d2e/d46/d9c/dc0/fcf 0 2024-03-22T10:43:07.171 INFO:tasks.workunit.client.1.smithi150.stdout:0/834: write d0/d1/d84/d86/d42/d10/fa7 [662989,14416] 0 2024-03-22T10:43:07.174 INFO:tasks.workunit.client.1.smithi150.stdout:9/889: dread d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/fba [0,4194304] 0 2024-03-22T10:43:07.174 INFO:tasks.workunit.client.1.smithi150.stdout:5/885: truncate d0/d8/dc/d1d/d53/d63/f9d 1301377 0 2024-03-22T10:43:07.175 INFO:tasks.workunit.client.1.smithi150.stdout:1/789: write d1/d19/d1a/f2b [1248316,86804] 0 2024-03-22T10:43:07.181 INFO:tasks.workunit.client.1.smithi150.stdout:1/790: creat d1/d7c/f101 x:0 0 0 2024-03-22T10:43:07.186 INFO:tasks.workunit.client.1.smithi150.stdout:2/952: sync 2024-03-22T10:43:07.201 INFO:tasks.workunit.client.1.smithi150.stdout:0/835: dwrite d0/d1/d84/d86/d42/d10/d38/fc3 [0,4194304] 0 2024-03-22T10:43:07.208 INFO:tasks.workunit.client.1.smithi150.stdout:1/791: dwrite d1/d19/d50/f90 [0,4194304] 0 2024-03-22T10:43:07.212 INFO:tasks.workunit.client.1.smithi150.stdout:7/842: dread d2/d5/d7/d9/db/d9f/f40 [0,4194304] 0 2024-03-22T10:43:07.224 INFO:tasks.workunit.client.1.smithi150.stdout:0/836: link d0/d1/d84/d86/f31 d0/d1/d84/d86/d11/d26/dd2/df0/f10c 0 2024-03-22T10:43:07.224 INFO:tasks.workunit.client.1.smithi150.stdout:1/792: truncate d1/d79/fdd 1384861 0 2024-03-22T10:43:07.227 INFO:tasks.workunit.client.1.smithi150.stdout:5/886: dwrite d0/d8/dc/de/d14/d2e/d5a/f86 [0,4194304] 0 2024-03-22T10:43:07.228 INFO:tasks.workunit.client.1.smithi150.stdout:7/843: symlink d2/d5/d7/dab/dec/l113 0 2024-03-22T10:43:07.229 INFO:tasks.workunit.client.1.smithi150.stdout:9/890: dwrite d1/d8/de/df/d36/d40/d44/f6e [0,4194304] 0 2024-03-22T10:43:07.234 INFO:tasks.workunit.client.1.smithi150.stdout:1/793: creat d1/d7c/f102 x:0 0 0 2024-03-22T10:43:07.234 INFO:tasks.workunit.client.1.smithi150.stdout:9/891: fsync d1/d8/de/df/d36/f39 0 2024-03-22T10:43:07.237 INFO:tasks.workunit.client.1.smithi150.stdout:5/887: write d0/d8/dc/de/d96/db8/f10f [966045,83157] 0 2024-03-22T10:43:07.239 INFO:tasks.workunit.client.1.smithi150.stdout:7/844: stat d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/daf/f108 0 2024-03-22T10:43:07.242 INFO:tasks.workunit.client.1.smithi150.stdout:1/794: mknod d1/da/d15/c103 0 2024-03-22T10:43:07.246 INFO:tasks.workunit.client.1.smithi150.stdout:5/888: truncate d0/d8/dc/de/f128 342266 0 2024-03-22T10:43:07.246 INFO:tasks.workunit.client.1.smithi150.stdout:1/795: write d1/da/d15/ff9 [472646,11249] 0 2024-03-22T10:43:07.249 INFO:tasks.workunit.client.1.smithi150.stdout:0/837: link d0/d1/d84/d86/d11/d66/d35/fe2 d0/d1/d84/d86/d11/d26/d76/f10d 0 2024-03-22T10:43:07.252 INFO:tasks.workunit.client.1.smithi150.stdout:2/953: dread d12/d19/db6/da9/d67/d8d/da6/fdb [0,4194304] 0 2024-03-22T10:43:07.254 INFO:tasks.workunit.client.1.smithi150.stdout:0/838: dread - d0/d1/d84/d86/da6/fd3 zero size 2024-03-22T10:43:07.256 INFO:tasks.workunit.client.1.smithi150.stdout:7/845: write d2/d5/d7/fd0 [1630866,111034] 0 2024-03-22T10:43:07.256 INFO:tasks.workunit.client.1.smithi150.stdout:1/796: fdatasync d1/d19/d50/d51/da0/dcf/dd6/faf 0 2024-03-22T10:43:07.258 INFO:tasks.workunit.client.1.smithi150.stdout:8/872: sync 2024-03-22T10:43:07.258 INFO:tasks.workunit.client.1.smithi150.stdout:4/907: sync 2024-03-22T10:43:07.258 INFO:tasks.workunit.client.1.smithi150.stdout:6/832: sync 2024-03-22T10:43:07.260 INFO:tasks.workunit.client.1.smithi150.stdout:5/889: rename d0/d8/dc/de/d14/d2e/d46/d9c/dc0/cd8 to d0/dae/de0/dce/c130 0 2024-03-22T10:43:07.260 INFO:tasks.workunit.client.1.smithi150.stdout:7/846: readlink d2/d5/d7/d8/d46/d93/ldd 0 2024-03-22T10:43:07.265 INFO:tasks.workunit.client.1.smithi150.stdout:8/873: stat d0/d2/c28 0 2024-03-22T10:43:07.266 INFO:tasks.workunit.client.1.smithi150.stdout:9/892: dread d1/d8/fde [0,4194304] 0 2024-03-22T10:43:07.270 INFO:tasks.workunit.client.1.smithi150.stdout:1/797: creat d1/d19/d50/d6e/db6/f104 x:0 0 0 2024-03-22T10:43:07.270 INFO:tasks.workunit.client.1.smithi150.stdout:4/908: write d0/dce/d10e/d121/f123 [459214,66787] 0 2024-03-22T10:43:07.270 INFO:tasks.workunit.client.1.smithi150.stdout:6/833: dread - db/d52/d9f/fe1 zero size 2024-03-22T10:43:07.273 INFO:tasks.workunit.client.1.smithi150.stdout:2/954: rename d12/d19/db6/da9/d61/d98/l93 to d12/d19/db6/da9/d67/d9a/d114/l12b 0 2024-03-22T10:43:07.274 INFO:tasks.workunit.client.1.smithi150.stdout:9/893: creat d1/d8/de/f127 x:0 0 0 2024-03-22T10:43:07.275 INFO:tasks.workunit.client.1.smithi150.stdout:1/798: chown d1/da/d2a/d4e/fa2 6796 1 2024-03-22T10:43:07.280 INFO:tasks.workunit.client.1.smithi150.stdout:0/839: write d0/d1/d84/d86/d11/d66/d35/fe2 [74400,95594] 0 2024-03-22T10:43:07.287 INFO:tasks.workunit.client.1.smithi150.stdout:4/909: symlink d0/d47/l12f 0 2024-03-22T10:43:07.294 INFO:tasks.workunit.client.1.smithi150.stdout:8/874: write d0/f5a [1824742,98170] 0 2024-03-22T10:43:07.298 INFO:tasks.workunit.client.1.smithi150.stdout:1/799: chown d1/d19/d50/d51/da0/lf1 3680 1 2024-03-22T10:43:07.298 INFO:tasks.workunit.client.1.smithi150.stdout:0/840: rename d0/d1/d84/d86/d42/f95 to d0/d1/d84/d86/d11/d26/d76/f10e 0 2024-03-22T10:43:07.298 INFO:tasks.workunit.client.1.smithi150.stdout:5/890: link d0/d8/dc/de/d14/d2e/d5a/f86 d0/d37/d104/f131 0 2024-03-22T10:43:07.300 INFO:tasks.workunit.client.1.smithi150.stdout:0/841: readlink d0/d1/d84/d86/d11/d26/l30 0 2024-03-22T10:43:07.301 INFO:tasks.workunit.client.1.smithi150.stdout:7/847: dread d2/d5/d7/d8/d46/f66 [4194304,4194304] 0 2024-03-22T10:43:07.301 INFO:tasks.workunit.client.1.smithi150.stdout:4/910: unlink d0/d2f/d63/d7b/d9c/db1/cbf 0 2024-03-22T10:43:07.308 INFO:tasks.workunit.client.1.smithi150.stdout:2/955: link d12/d21/f4b d12/d19/db6/da9/d2c/f12c 0 2024-03-22T10:43:07.315 INFO:tasks.workunit.client.1.smithi150.stdout:5/891: creat d0/d8/d101/d11b/d12e/f132 x:0 0 0 2024-03-22T10:43:07.318 INFO:tasks.workunit.client.1.smithi150.stdout:0/842: creat d0/d1/d84/d86/d11/d26/dd2/df0/f10f x:0 0 0 2024-03-22T10:43:07.318 INFO:tasks.workunit.client.1.smithi150.stdout:1/800: creat d1/da/d15/d18/f105 x:0 0 0 2024-03-22T10:43:07.322 INFO:tasks.workunit.client.1.smithi150.stdout:6/834: dwrite db/dd/d15/d60/d50/f64 [0,4194304] 0 2024-03-22T10:43:07.322 INFO:tasks.workunit.client.1.smithi150.stdout:5/892: symlink d0/ddc/l133 0 2024-03-22T10:43:07.326 INFO:tasks.workunit.client.1.smithi150.stdout:2/956: read d12/d19/db6/da9/d67/d3c/f6b [660393,9535] 0 2024-03-22T10:43:07.330 INFO:tasks.workunit.client.1.smithi150.stdout:8/875: dwrite d0/ddd/dfc/f11d [0,4194304] 0 2024-03-22T10:43:07.332 INFO:tasks.workunit.client.1.smithi150.stdout:9/894: dread d1/d8/de/df/d36/d40/d44/db5/dda/d104/f84 [0,4194304] 0 2024-03-22T10:43:07.340 INFO:tasks.workunit.client.1.smithi150.stdout:8/876: chown d0/d2/d19/d29/d37/dc2 102 1 2024-03-22T10:43:07.341 INFO:tasks.workunit.client.1.smithi150.stdout:4/911: truncate d0/d2f/d10d/d4d/d6c/d61/fbb 583261 0 2024-03-22T10:43:07.342 INFO:tasks.workunit.client.1.smithi150.stdout:8/877: dread - d0/d2/d46/f11c zero size 2024-03-22T10:43:07.347 INFO:tasks.workunit.client.1.smithi150.stdout:3/822: sync 2024-03-22T10:43:07.353 INFO:tasks.workunit.client.1.smithi150.stdout:4/912: symlink d0/d47/d72/l130 0 2024-03-22T10:43:07.353 INFO:tasks.workunit.client.1.smithi150.stdout:0/843: symlink d0/d1/d84/d86/d42/d10/d38/dba/l110 0 2024-03-22T10:43:07.358 INFO:tasks.workunit.client.1.smithi150.stdout:6/835: link db/dd/d15/d60/d50/c5f db/dd/d15/d30/ddf/d11a/c120 0 2024-03-22T10:43:07.363 INFO:tasks.workunit.client.1.smithi150.stdout:3/823: creat d5/d22/d2f/d38/d3e/f116 x:0 0 0 2024-03-22T10:43:07.365 INFO:tasks.workunit.client.1.smithi150.stdout:7/848: dread d2/d5/d7/d9/db/d6c/d9c/f9d [0,4194304] 0 2024-03-22T10:43:07.371 INFO:tasks.workunit.client.1.smithi150.stdout:5/893: dwrite d0/d8/dc/de/d96/db8/f10f [0,4194304] 0 2024-03-22T10:43:07.376 INFO:tasks.workunit.client.1.smithi150.stdout:1/801: dwrite d1/d19/d50/f5c [0,4194304] 0 2024-03-22T10:43:07.391 INFO:tasks.workunit.client.1.smithi150.stdout:4/913: write d0/d2f/d63/d7b/d9c/db1/f107 [18998,63373] 0 2024-03-22T10:43:07.392 INFO:tasks.workunit.client.1.smithi150.stdout:6/836: mknod db/d52/d9f/dc6/df1/db4/c121 0 2024-03-22T10:43:07.400 INFO:tasks.workunit.client.1.smithi150.stdout:8/878: dread d0/d2/d19/d29/d37/d7d/d9e/fcb [0,4194304] 0 2024-03-22T10:43:07.400 INFO:tasks.workunit.client.1.smithi150.stdout:9/895: dwrite d1/d8/de/d73/d101/d29/fb1 [4194304,4194304] 0 2024-03-22T10:43:07.404 INFO:tasks.workunit.client.1.smithi150.stdout:6/837: dread - db/dd/d15/f7b zero size 2024-03-22T10:43:07.406 INFO:tasks.workunit.client.1.smithi150.stdout:2/957: dread d12/d19/db6/da9/d67/d3c/f92 [0,4194304] 0 2024-03-22T10:43:07.423 INFO:tasks.workunit.client.1.smithi150.stdout:3/824: dwrite d5/d53/f75 [0,4194304] 0 2024-03-22T10:43:07.438 INFO:tasks.workunit.client.1.smithi150.stdout:5/894: rename d0/d37/d104 to d0/d8/dc/de/d14/d4f/d134 0 2024-03-22T10:43:07.439 INFO:tasks.workunit.client.1.smithi150.stdout:6/838: mknod db/dd/d15/d6a/dde/c122 0 2024-03-22T10:43:07.441 INFO:tasks.workunit.client.1.smithi150.stdout:2/958: mkdir d12/d19/db6/d63/dd8/d12d 0 2024-03-22T10:43:07.445 INFO:tasks.workunit.client.1.smithi150.stdout:5/895: fsync d0/d8/dc/de/d14/fb6 0 2024-03-22T10:43:07.447 INFO:tasks.workunit.client.1.smithi150.stdout:9/896: rename d1/d19/dfa to d1/d8/de/dbd/ddc/d128 0 2024-03-22T10:43:07.448 INFO:tasks.workunit.client.1.smithi150.stdout:3/825: symlink d5/d98/l117 0 2024-03-22T10:43:07.449 INFO:tasks.workunit.client.1.smithi150.stdout:7/849: creat d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/f114 x:0 0 0 2024-03-22T10:43:07.451 INFO:tasks.workunit.client.1.smithi150.stdout:3/826: read - d5/da/d62/ff0 zero size 2024-03-22T10:43:07.456 INFO:tasks.workunit.client.1.smithi150.stdout:8/879: rename d0/d2/d19/d29/f9d to d0/d3c/d71/f11f 0 2024-03-22T10:43:07.457 INFO:tasks.workunit.client.1.smithi150.stdout:1/802: getdents d1/da/d2a/d55 0 2024-03-22T10:43:07.460 INFO:tasks.workunit.client.1.smithi150.stdout:3/827: rmdir d5/daf 39 2024-03-22T10:43:07.465 INFO:tasks.workunit.client.1.smithi150.stdout:9/897: dread d1/d8/de/dbd/ddc/ff9 [0,4194304] 0 2024-03-22T10:43:07.465 INFO:tasks.workunit.client.1.smithi150.stdout:6/839: rename db/dd/d82/d84/cd8 to db/d52/d9f/dc6/c123 0 2024-03-22T10:43:07.470 INFO:tasks.workunit.client.1.smithi150.stdout:5/896: link d0/d8/dc/de/d14/d2e/d46/d9c/ffd d0/dae/f135 0 2024-03-22T10:43:07.471 INFO:tasks.workunit.client.1.smithi150.stdout:1/803: creat d1/d19/dd5/f106 x:0 0 0 2024-03-22T10:43:07.472 INFO:tasks.workunit.client.1.smithi150.stdout:3/828: stat d5/d22/c32 0 2024-03-22T10:43:07.472 INFO:tasks.workunit.client.1.smithi150.stdout:7/850: mknod d2/d5/d7/d8/d57/d4a/d111/c115 0 2024-03-22T10:43:07.475 INFO:tasks.workunit.client.1.smithi150.stdout:0/844: dread d0/d1/d84/d86/d42/d10/d38/d52/d68/fb0 [0,4194304] 0 2024-03-22T10:43:07.476 INFO:tasks.workunit.client.1.smithi150.stdout:0/845: readlink d0/d1/lf1 0 2024-03-22T10:43:07.478 INFO:tasks.workunit.client.1.smithi150.stdout:7/851: dread - d2/d5/d7/d9/db/d6c/da8/fba zero size 2024-03-22T10:43:07.480 INFO:tasks.workunit.client.1.smithi150.stdout:1/804: write d1/da/d15/d18/f105 [28129,77609] 0 2024-03-22T10:43:07.486 INFO:tasks.workunit.client.1.smithi150.stdout:9/898: mkdir d1/d8/de/df/d36/d40/d44/db5/dda/d129 0 2024-03-22T10:43:07.488 INFO:tasks.workunit.client.1.smithi150.stdout:5/897: truncate d0/d8/f76 442245 0 2024-03-22T10:43:07.489 INFO:tasks.workunit.client.1.smithi150.stdout:6/840: creat db/dd/d15/d60/d50/f124 x:0 0 0 2024-03-22T10:43:07.492 INFO:tasks.workunit.client.1.smithi150.stdout:6/841: chown db/d52/dc3 2 1 2024-03-22T10:43:07.495 INFO:tasks.workunit.client.1.smithi150.stdout:7/852: rename d2/ffc to d2/d5/d7/d8/d57/d4a/d111/f116 0 2024-03-22T10:43:07.495 INFO:tasks.workunit.client.1.smithi150.stdout:8/880: dread d0/d3c/f44 [4194304,4194304] 0 2024-03-22T10:43:07.497 INFO:tasks.workunit.client.1.smithi150.stdout:6/842: stat db/dd/d82/de9 0 2024-03-22T10:43:07.498 INFO:tasks.workunit.client.1.smithi150.stdout:3/829: dread d5/d22/d2f/d38/d3e/d5b/f9a [0,4194304] 0 2024-03-22T10:43:07.501 INFO:tasks.workunit.client.1.smithi150.stdout:1/805: creat d1/da/d2a/d65/d78/dfd/f107 x:0 0 0 2024-03-22T10:43:07.505 INFO:tasks.workunit.client.1.smithi150.stdout:9/899: creat d1/d8/de/df/d36/d40/d44/db5/dda/d129/f12a x:0 0 0 2024-03-22T10:43:07.506 INFO:tasks.workunit.client.1.smithi150.stdout:5/898: mknod d0/d8/dc/de/d14/d2e/d45/dac/d105/c136 0 2024-03-22T10:43:07.509 INFO:tasks.workunit.client.1.smithi150.stdout:8/881: mknod d0/d2/d19/d29/d2f/df/dff/d110/d96/c120 0 2024-03-22T10:43:07.511 INFO:tasks.workunit.client.1.smithi150.stdout:1/806: mknod d1/c108 0 2024-03-22T10:43:07.513 INFO:tasks.workunit.client.1.smithi150.stdout:7/853: dread d2/d5/d7/d9/db/d6c/d9c/f9d [0,4194304] 0 2024-03-22T10:43:07.515 INFO:tasks.workunit.client.1.smithi150.stdout:5/899: mkdir d0/d33/d6f/d137 0 2024-03-22T10:43:07.515 INFO:tasks.workunit.client.1.smithi150.stdout:9/900: truncate d1/f71 3860746 0 2024-03-22T10:43:07.521 INFO:tasks.workunit.client.1.smithi150.stdout:6/843: link db/dd/d15/d60/f9c db/dd/d15/d6a/f125 0 2024-03-22T10:43:07.522 INFO:tasks.workunit.client.1.smithi150.stdout:3/830: dwrite d5/d22/d2f/d38/d3e/f55 [0,4194304] 0 2024-03-22T10:43:07.524 INFO:tasks.workunit.client.1.smithi150.stdout:1/807: mkdir d1/da/d15/d18/d4b/d109 0 2024-03-22T10:43:07.533 INFO:tasks.workunit.client.1.smithi150.stdout:1/808: write d1/d19/fe3 [364874,15576] 0 2024-03-22T10:43:07.534 INFO:tasks.workunit.client.1.smithi150.stdout:3/831: mkdir d5/d22/d118 0 2024-03-22T10:43:07.536 INFO:tasks.workunit.client.1.smithi150.stdout:6/844: link db/dd/d15/d60/f32 db/d52/d79/f126 0 2024-03-22T10:43:07.538 INFO:tasks.workunit.client.1.smithi150.stdout:2/959: write d12/d35/d7d/fe6 [865205,83244] 0 2024-03-22T10:43:07.542 INFO:tasks.workunit.client.1.smithi150.stdout:8/882: dwrite d0/d2/d19/f78 [0,4194304] 0 2024-03-22T10:43:07.546 INFO:tasks.workunit.client.1.smithi150.stdout:1/809: creat d1/da/d15/d18/d4b/f10a x:0 0 0 2024-03-22T10:43:07.549 INFO:tasks.workunit.client.1.smithi150.stdout:7/854: dwrite d2/d5/d7/d9/db/da2/fea [0,4194304] 0 2024-03-22T10:43:07.549 INFO:tasks.workunit.client.1.smithi150.stdout:8/883: write d0/d2/d46/f11c [282641,24381] 0 2024-03-22T10:43:07.553 INFO:tasks.workunit.client.1.smithi150.stdout:1/810: fdatasync d1/da/d15/f3e 0 2024-03-22T10:43:07.561 INFO:tasks.workunit.client.1.smithi150.stdout:1/811: fdatasync d1/d19/d50/d51/da0/dcf/dd6/faf 0 2024-03-22T10:43:07.579 INFO:tasks.workunit.client.1.smithi150.stdout:6/845: mknod db/dd/d15/d30/ddf/c127 0 2024-03-22T10:43:07.586 INFO:tasks.workunit.client.1.smithi150.stdout:1/812: dwrite d1/d19/d50/d51/ff4 [0,4194304] 0 2024-03-22T10:43:07.623 INFO:tasks.workunit.client.1.smithi150.stdout:0/846: truncate d0/d1/d84/d86/f8d 928043 0 2024-03-22T10:43:07.624 INFO:tasks.workunit.client.1.smithi150.stdout:6/846: creat db/d52/d79/f128 x:0 0 0 2024-03-22T10:43:07.630 INFO:tasks.workunit.client.1.smithi150.stdout:1/813: rename d1/d19/fe3 to d1/d19/d50/d6e/f10b 0 2024-03-22T10:43:07.633 INFO:tasks.workunit.client.1.smithi150.stdout:1/814: write d1/d19/d50/d6e/db6/f104 [468504,65008] 0 2024-03-22T10:43:07.645 INFO:tasks.workunit.client.1.smithi150.stdout:8/884: getdents d0/d3c/d71/d7a/d89/db5/d106 0 2024-03-22T10:43:07.649 INFO:tasks.workunit.client.1.smithi150.stdout:1/815: unlink d1/da/ccc 0 2024-03-22T10:43:07.651 INFO:tasks.workunit.client.1.smithi150.stdout:5/900: truncate d0/d8/dc/de/d14/d2e/dbc/f103 1193217 0 2024-03-22T10:43:07.654 INFO:tasks.workunit.client.1.smithi150.stdout:5/901: read d0/d37/fdf [2418691,27537] 0 2024-03-22T10:43:07.657 INFO:tasks.workunit.client.1.smithi150.stdout:1/816: unlink d1/d19/d71/f8b 0 2024-03-22T10:43:07.659 INFO:tasks.workunit.client.1.smithi150.stdout:9/901: dwrite d1/d8/de/df/d36/d121/f4a [0,4194304] 0 2024-03-22T10:43:07.661 INFO:tasks.workunit.client.1.smithi150.stdout:3/832: write d5/da/f57 [1274783,126742] 0 2024-03-22T10:43:07.663 INFO:tasks.workunit.client.1.smithi150.stdout:5/902: write d0/d37/db4/f12f [659711,83946] 0 2024-03-22T10:43:07.669 INFO:tasks.workunit.client.1.smithi150.stdout:8/885: symlink d0/d2/d46/da2/dd0/l121 0 2024-03-22T10:43:07.670 INFO:tasks.workunit.client.1.smithi150.stdout:5/903: chown d0/d8/dc/de/d14/d2e/d46/d4d/ld5 13396 1 2024-03-22T10:43:07.673 INFO:tasks.workunit.client.1.smithi150.stdout:1/817: fsync d1/da/d15/d18/f5f 0 2024-03-22T10:43:07.674 INFO:tasks.workunit.client.1.smithi150.stdout:9/902: fsync d1/d8/de/d73/d101/d29/f54 0 2024-03-22T10:43:07.677 INFO:tasks.workunit.client.1.smithi150.stdout:9/903: readlink d1/d8/de/d73/d101/led 0 2024-03-22T10:43:07.682 INFO:tasks.workunit.client.1.smithi150.stdout:3/833: creat d5/d22/d2f/d38/d59/d88/d10d/d105/f119 x:0 0 0 2024-03-22T10:43:07.684 INFO:tasks.workunit.client.1.smithi150.stdout:1/818: mkdir d1/d19/d71/d10c 0 2024-03-22T10:43:07.685 INFO:tasks.workunit.client.1.smithi150.stdout:3/834: write d5/d22/d2f/d38/f85 [5050293,111067] 0 2024-03-22T10:43:07.685 INFO:tasks.workunit.client.1.smithi150.stdout:5/904: mknod d0/d8/c138 0 2024-03-22T10:43:07.688 INFO:tasks.workunit.client.1.smithi150.stdout:1/819: readlink d1/da/d2a/d55/db8/led 0 2024-03-22T10:43:07.689 INFO:tasks.workunit.client.1.smithi150.stdout:9/904: symlink d1/d8/de/d73/d101/d32/l12b 0 2024-03-22T10:43:07.692 INFO:tasks.workunit.client.1.smithi150.stdout:8/886: link d0/d2/d19/d29/d88/dec/ffe d0/d2/d19/dc0/dda/f122 0 2024-03-22T10:43:07.697 INFO:tasks.workunit.client.1.smithi150.stdout:9/905: mkdir d1/d19/d7d/dc1/dca/dd7/df0/d12c 0 2024-03-22T10:43:07.698 INFO:tasks.workunit.client.1.smithi150.stdout:3/835: mkdir d5/d22/d2f/d38/d59/def/d11a 0 2024-03-22T10:43:07.700 INFO:tasks.workunit.client.1.smithi150.stdout:1/820: mknod d1/d19/d50/d51/da0/dcf/dd6/de4/c10d 0 2024-03-22T10:43:07.701 INFO:tasks.workunit.client.1.smithi150.stdout:5/905: rename d0/d8/c39 to d0/c139 0 2024-03-22T10:43:07.703 INFO:tasks.workunit.client.1.smithi150.stdout:8/887: creat d0/d3c/d71/d7a/d89/d113/f123 x:0 0 0 2024-03-22T10:43:07.704 INFO:tasks.workunit.client.1.smithi150.stdout:3/836: mknod d5/d22/d2f/d38/d59/d88/c11b 0 2024-03-22T10:43:07.710 INFO:tasks.workunit.client.1.smithi150.stdout:9/906: unlink d1/d8/de/df/da2/fbf 0 2024-03-22T10:43:07.713 INFO:tasks.workunit.client.1.smithi150.stdout:5/906: mkdir d0/d8/dc/de/d14/d2e/d5a/d66/da9/d13a 0 2024-03-22T10:43:07.713 INFO:tasks.workunit.client.1.smithi150.stdout:1/821: symlink d1/da/d15/d18/d4b/l10e 0 2024-03-22T10:43:07.714 INFO:tasks.workunit.client.1.smithi150.stdout:1/822: stat d1/d19/d50 0 2024-03-22T10:43:07.715 INFO:tasks.workunit.client.1.smithi150.stdout:9/907: read d1/d8/de/df/d36/d40/d44/db5/fd9 [365715,89059] 0 2024-03-22T10:43:07.717 INFO:tasks.workunit.client.1.smithi150.stdout:4/914: sync 2024-03-22T10:43:07.734 INFO:tasks.workunit.client.1.smithi150.stdout:8/888: creat d0/d3c/f124 x:0 0 0 2024-03-22T10:43:07.736 INFO:tasks.workunit.client.1.smithi150.stdout:9/908: truncate d1/d8/de/f2c 2147155 0 2024-03-22T10:43:07.738 INFO:tasks.workunit.client.1.smithi150.stdout:4/915: dwrite d0/d2f/f76 [0,4194304] 0 2024-03-22T10:43:07.744 INFO:tasks.workunit.client.1.smithi150.stdout:4/916: truncate d0/d2f/d10d/d4d/fff 741196 0 2024-03-22T10:43:07.755 INFO:tasks.workunit.client.1.smithi150.stdout:9/909: truncate d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/fba 1454599 0 2024-03-22T10:43:07.758 INFO:tasks.workunit.client.1.smithi150.stdout:7/855: dwrite d2/d5/d7/d9/db/d9f/d96/d71/d106/f5f [0,4194304] 0 2024-03-22T10:43:07.764 INFO:tasks.workunit.client.1.smithi150.stdout:4/917: link d0/d2f/d63/d7b/d9c/db1/dd7/lfe d0/dce/d10e/d121/d7c/dc7/df7/l131 0 2024-03-22T10:43:07.770 INFO:tasks.workunit.client.1.smithi150.stdout:4/918: dread - d0/d2f/d63/d7b/d9c/db1/dd7/f104 zero size 2024-03-22T10:43:07.770 INFO:tasks.workunit.client.1.smithi150.stdout:0/847: dwrite d0/d1/d84/d86/d11/d66/d35/d5f/fc0 [0,4194304] 0 2024-03-22T10:43:07.783 INFO:tasks.workunit.client.1.smithi150.stdout:0/848: creat d0/d12/f111 x:0 0 0 2024-03-22T10:43:07.790 INFO:tasks.workunit.client.1.smithi150.stdout:7/856: dwrite d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/fa6 [0,4194304] 0 2024-03-22T10:43:07.799 INFO:tasks.workunit.client.1.smithi150.stdout:0/849: getdents d0/d1/d84/d86/d11/d66/d37 0 2024-03-22T10:43:07.801 INFO:tasks.workunit.client.1.smithi150.stdout:7/857: mkdir d2/d5/d7/d9/db/d9f/d117 0 2024-03-22T10:43:07.809 INFO:tasks.workunit.client.1.smithi150.stdout:7/858: unlink d2/d5/d7/d9/db/d9f/d45/ff2 0 2024-03-22T10:43:07.813 INFO:tasks.workunit.client.1.smithi150.stdout:7/859: dread d2/d5/d7/d8/d46/d93/ffe [0,4194304] 0 2024-03-22T10:43:07.815 INFO:tasks.workunit.client.1.smithi150.stdout:3/837: write d5/d22/d2f/d38/d3e/f4d [1499051,103651] 0 2024-03-22T10:43:07.816 INFO:tasks.workunit.client.1.smithi150.stdout:4/919: mknod d0/d2f/c132 0 2024-03-22T10:43:07.819 INFO:tasks.workunit.client.1.smithi150.stdout:3/838: read - d5/d22/d2f/d38/d3e/f116 zero size 2024-03-22T10:43:07.822 INFO:tasks.workunit.client.1.smithi150.stdout:5/907: rename d0/d8/dc/de/d116 to d0/d8/dc/de/d14/d2e/d5a/d13b 0 2024-03-22T10:43:07.825 INFO:tasks.workunit.client.1.smithi150.stdout:8/889: truncate d0/d3c/fce 4127291 0 2024-03-22T10:43:07.828 INFO:tasks.workunit.client.1.smithi150.stdout:1/823: dwrite d1/d19/f9d [0,4194304] 0 2024-03-22T10:43:07.829 INFO:tasks.workunit.client.1.smithi150.stdout:7/860: creat d2/d5/d7/d9/db/d9f/f118 x:0 0 0 2024-03-22T10:43:07.830 INFO:tasks.workunit.client.1.smithi150.stdout:9/910: rmdir d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7 39 2024-03-22T10:43:07.834 INFO:tasks.workunit.client.1.smithi150.stdout:5/908: creat d0/d8/dc/de/d14/d2e/d5a/d13b/f13c x:0 0 0 2024-03-22T10:43:07.839 INFO:tasks.workunit.client.1.smithi150.stdout:9/911: dread d1/d19/fb4 [0,4194304] 0 2024-03-22T10:43:07.842 INFO:tasks.workunit.client.1.smithi150.stdout:4/920: rename d0/dce/d10e/d121/d7c/dc1/f12b to d0/de/f133 0 2024-03-22T10:43:07.853 INFO:tasks.workunit.client.1.smithi150.stdout:0/850: write d0/d1/d84/d86/fe [2265181,55663] 0 2024-03-22T10:43:07.856 INFO:tasks.workunit.client.1.smithi150.stdout:3/839: rename d5/d22/d2f/d38/c51 to d5/d22/d2f/d38/d3e/d5b/da6/c11c 0 2024-03-22T10:43:07.863 INFO:tasks.workunit.client.1.smithi150.stdout:1/824: dread d1/da/d15/f74 [0,4194304] 0 2024-03-22T10:43:07.865 INFO:tasks.workunit.client.1.smithi150.stdout:9/912: dwrite d1/d8/de/d73/d101/f50 [0,4194304] 0 2024-03-22T10:43:07.866 INFO:tasks.workunit.client.1.smithi150.stdout:1/825: dread - d1/da/fff zero size 2024-03-22T10:43:07.867 INFO:tasks.workunit.client.1.smithi150.stdout:0/851: readlink d0/d1/l41 0 2024-03-22T10:43:07.869 INFO:tasks.workunit.client.1.smithi150.stdout:8/890: getdents d0/d2/d19/d29/d88 0 2024-03-22T10:43:07.870 INFO:tasks.workunit.client.1.smithi150.stdout:4/921: dwrite d0/d2f/d63/d7b/d9c/f102 [0,4194304] 0 2024-03-22T10:43:07.877 INFO:tasks.workunit.client.1.smithi150.stdout:3/840: mknod d5/c11d 0 2024-03-22T10:43:07.877 INFO:tasks.workunit.client.1.smithi150.stdout:5/909: getdents d0/d8/dc/de/d14/d2e/d45/dac 0 2024-03-22T10:43:07.882 INFO:tasks.workunit.client.1.smithi150.stdout:9/913: creat d1/d8/de/df/da2/f12d x:0 0 0 2024-03-22T10:43:07.882 INFO:tasks.workunit.client.1.smithi150.stdout:1/826: mknod d1/d19/d1a/d99/c10f 0 2024-03-22T10:43:07.882 INFO:tasks.workunit.client.1.smithi150.stdout:0/852: mknod d0/d1/d84/d86/d11/d66/d35/d5f/c112 0 2024-03-22T10:43:07.885 INFO:tasks.workunit.client.1.smithi150.stdout:3/841: creat d5/d22/d2f/d38/d59/d88/d10d/d105/f11e x:0 0 0 2024-03-22T10:43:07.888 INFO:tasks.workunit.client.1.smithi150.stdout:8/891: mkdir d0/d2/dbf/d125 0 2024-03-22T10:43:07.892 INFO:tasks.workunit.client.1.smithi150.stdout:2/960: sync 2024-03-22T10:43:07.894 INFO:tasks.workunit.client.1.smithi150.stdout:7/861: truncate d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/fb3 748959 0 2024-03-22T10:43:07.899 INFO:tasks.workunit.client.1.smithi150.stdout:3/842: truncate d5/fd 3853522 0 2024-03-22T10:43:07.901 INFO:tasks.workunit.client.1.smithi150.stdout:7/862: chown d2/d5/d7/d9/db/l3d 1922758 1 2024-03-22T10:43:07.901 INFO:tasks.workunit.client.1.smithi150.stdout:2/961: truncate d12/d19/db6/f72 2924207 0 2024-03-22T10:43:07.903 INFO:tasks.workunit.client.1.smithi150.stdout:9/914: creat d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/f12e x:0 0 0 2024-03-22T10:43:07.907 INFO:tasks.workunit.client.1.smithi150.stdout:8/892: read d0/d2/d19/d29/f63 [1983827,116669] 0 2024-03-22T10:43:07.911 INFO:tasks.workunit.client.1.smithi150.stdout:3/843: unlink d5/d53/f71 0 2024-03-22T10:43:07.915 INFO:tasks.workunit.client.1.smithi150.stdout:3/844: fdatasync d5/d22/d37/fd7 0 2024-03-22T10:43:07.916 INFO:tasks.workunit.client.1.smithi150.stdout:8/893: write d0/d2/d19/d29/d88/f105 [909010,69755] 0 2024-03-22T10:43:07.921 INFO:tasks.workunit.client.1.smithi150.stdout:1/827: dwrite d1/da/d15/d18/f105 [0,4194304] 0 2024-03-22T10:43:07.922 INFO:tasks.workunit.client.1.smithi150.stdout:8/894: write d0/f34 [6602088,4544] 0 2024-03-22T10:43:07.923 INFO:tasks.workunit.client.1.smithi150.stdout:5/910: dread d0/dae/feb [0,4194304] 0 2024-03-22T10:43:07.926 INFO:tasks.workunit.client.1.smithi150.stdout:7/863: creat d2/d5/d7/d9/db/d9f/d96/d71/f119 x:0 0 0 2024-03-22T10:43:07.929 INFO:tasks.workunit.client.1.smithi150.stdout:3/845: write d5/da/f7a [4735568,55082] 0 2024-03-22T10:43:07.931 INFO:tasks.workunit.client.1.smithi150.stdout:5/911: read d0/d8/dc/f5b [1306265,50724] 0 2024-03-22T10:43:07.933 INFO:tasks.workunit.client.1.smithi150.stdout:7/864: chown d2/d5/d7/d8/d57/d4a/d8d/fff 3799671 1 2024-03-22T10:43:07.935 INFO:tasks.workunit.client.1.smithi150.stdout:1/828: link d1/d7c/fb5 d1/da/d2a/d4e/f110 0 2024-03-22T10:43:07.943 INFO:tasks.workunit.client.1.smithi150.stdout:8/895: creat d0/d2/d19/d29/d2f/df/dff/f126 x:0 0 0 2024-03-22T10:43:07.945 INFO:tasks.workunit.client.1.smithi150.stdout:3/846: rename d5/d22/d2f/d38/cc0 to d5/da/d78/db7/d106/c11f 0 2024-03-22T10:43:07.947 INFO:tasks.workunit.client.1.smithi150.stdout:1/829: creat d1/d19/dd5/f111 x:0 0 0 2024-03-22T10:43:07.948 INFO:tasks.workunit.client.1.smithi150.stdout:7/865: creat d2/d5/d7/d88/f11a x:0 0 0 2024-03-22T10:43:07.950 INFO:tasks.workunit.client.1.smithi150.stdout:8/896: symlink d0/d3c/d71/l127 0 2024-03-22T10:43:07.951 INFO:tasks.workunit.client.1.smithi150.stdout:2/962: dwrite d12/f37 [0,4194304] 0 2024-03-22T10:43:07.955 INFO:tasks.workunit.client.1.smithi150.stdout:0/853: dread d0/f85 [0,4194304] 0 2024-03-22T10:43:07.962 INFO:tasks.workunit.client.1.smithi150.stdout:1/830: symlink d1/da/d2a/d65/d78/l112 0 2024-03-22T10:43:07.963 INFO:tasks.workunit.client.1.smithi150.stdout:8/897: mkdir d0/d2/d19/dc0/dda/d128 0 2024-03-22T10:43:07.974 INFO:tasks.workunit.client.1.smithi150.stdout:1/831: rename d1/d79/fdd to d1/da/d15/d18/d4b/d8a/f113 0 2024-03-22T10:43:07.982 INFO:tasks.workunit.client.1.smithi150.stdout:2/963: dwrite d12/d19/db6/d54/d125/f10d [0,4194304] 0 2024-03-22T10:43:07.982 INFO:tasks.workunit.client.1.smithi150.stdout:3/847: write d5/da/d62/ff5 [683393,83692] 0 2024-03-22T10:43:07.982 INFO:tasks.workunit.client.1.smithi150.stdout:0/854: truncate d0/d1/d84/d86/d42/d10/d38/d52/d9b/fb6 497682 0 2024-03-22T10:43:07.984 INFO:tasks.workunit.client.1.smithi150.stdout:7/866: dwrite d2/d5/d7/d9/db/d6c/da8/ffa [0,4194304] 0 2024-03-22T10:43:07.989 INFO:tasks.workunit.client.1.smithi150.stdout:0/855: stat d0/d1/d84/d86/da6/cfa 0 2024-03-22T10:43:07.990 INFO:tasks.workunit.client.1.smithi150.stdout:2/964: readlink d12/d19/db6/d54/lea 0 2024-03-22T10:43:07.994 INFO:tasks.workunit.client.1.smithi150.stdout:1/832: rename d1/da/d2a/d55/db8/cd2 to d1/d19/d50/d6e/db6/de1/c114 0 2024-03-22T10:43:07.999 INFO:tasks.workunit.client.1.smithi150.stdout:0/856: rmdir d0/d1/d84/d86/d42/d10/d38/d52/d9b 39 2024-03-22T10:43:08.004 INFO:tasks.workunit.client.1.smithi150.stdout:1/833: chown d1/da/d2a/d65/d78/fb1 2713 1 2024-03-22T10:43:08.007 INFO:tasks.workunit.client.1.smithi150.stdout:8/898: dwrite d0/d3c/f4a [4194304,4194304] 0 2024-03-22T10:43:08.009 INFO:tasks.workunit.client.1.smithi150.stdout:3/848: rename d5/d22/d2f/d38/d3e/d5b/da6/ceb to d5/d22/d2f/d38/c120 0 2024-03-22T10:43:08.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:07 smithi115 ceph-mon[29111]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:43:08.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:07 smithi115 ceph-mon[29111]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:43:08.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:07 smithi115 ceph-mon[29111]: pgmap v22: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 122 MiB/s rd, 297 MiB/s wr, 649 op/s 2024-03-22T10:43:08.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:07 smithi115 ceph-mon[29111]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:43:08.011 INFO:tasks.workunit.client.1.smithi150.stdout:0/857: write d0/d1/d84/d86/da6/fd3 [709696,57115] 0 2024-03-22T10:43:08.013 INFO:tasks.workunit.client.1.smithi150.stdout:3/849: chown d5/cfe 227479176 1 2024-03-22T10:43:08.013 INFO:tasks.workunit.client.1.smithi150.stdout:1/834: read - d1/d19/d50/d51/da0/dcf/dd9/ff8 zero size 2024-03-22T10:43:08.015 INFO:tasks.workunit.client.1.smithi150.stdout:8/899: chown d0/d2/d19/d29/d2e/d70/c53 12466 1 2024-03-22T10:43:08.017 INFO:tasks.workunit.client.1.smithi150.stdout:0/858: symlink d0/d1/d84/d86/d11/d66/d37/l113 0 2024-03-22T10:43:08.018 INFO:tasks.workunit.client.1.smithi150.stdout:3/850: fsync d5/d22/d37/f10a 0 2024-03-22T10:43:08.019 INFO:tasks.workunit.client.1.smithi150.stdout:7/867: dwrite d2/d5/d7/d8/d57/d4a/d8d/ff5 [0,4194304] 0 2024-03-22T10:43:08.024 INFO:tasks.workunit.client.1.smithi150.stdout:7/868: fsync d2/d5/d7/d88/f8f 0 2024-03-22T10:43:08.027 INFO:tasks.workunit.client.1.smithi150.stdout:3/851: creat d5/d22/dc8/dda/d95/f121 x:0 0 0 2024-03-22T10:43:08.030 INFO:tasks.workunit.client.1.smithi150.stdout:2/965: dwrite d12/d19/db6/d63/fa8 [0,4194304] 0 2024-03-22T10:43:08.031 INFO:tasks.workunit.client.1.smithi150.stdout:8/900: rmdir d0/d2/d19/d29/d37/da1/d115 0 2024-03-22T10:43:08.039 INFO:tasks.workunit.client.1.smithi150.stdout:8/901: chown d0/d2/d19/d29/d37/d7d/d9e/c103 0 1 2024-03-22T10:43:08.044 INFO:tasks.workunit.client.1.smithi150.stdout:2/966: symlink d12/d19/d100/l12e 0 2024-03-22T10:43:08.046 INFO:tasks.workunit.client.1.smithi150.stdout:7/869: creat d2/d5/d7/d8/d57/d105/f11b x:0 0 0 2024-03-22T10:43:08.048 INFO:tasks.workunit.client.1.smithi150.stdout:8/902: mknod d0/d2/d19/d29/d37/d7d/dc1/c129 0 2024-03-22T10:43:08.055 INFO:tasks.workunit.client.1.smithi150.stdout:2/967: fsync d12/d19/f1d 0 2024-03-22T10:43:08.055 INFO:tasks.workunit.client.1.smithi150.stdout:0/859: dwrite d0/d1/d84/d86/d11/d66/d35/d92/fad [0,4194304] 0 2024-03-22T10:43:08.055 INFO:tasks.workunit.client.1.smithi150.stdout:7/870: creat d2/d5/d7/dab/f11c x:0 0 0 2024-03-22T10:43:08.060 INFO:tasks.workunit.client.1.smithi150.stdout:2/968: dread d12/d19/db6/da9/f103 [0,4194304] 0 2024-03-22T10:43:08.063 INFO:tasks.workunit.client.1.smithi150.stdout:3/852: dwrite d5/da/d9d/fa8 [0,4194304] 0 2024-03-22T10:43:08.063 INFO:tasks.workunit.client.1.smithi150.stdout:7/871: mknod d2/d5/d7/d9/c11d 0 2024-03-22T10:43:08.068 INFO:tasks.workunit.client.1.smithi150.stdout:2/969: rmdir d12/d19/db6/df1 39 2024-03-22T10:43:08.072 INFO:tasks.workunit.client.1.smithi150.stdout:7/872: unlink d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/daf/f108 0 2024-03-22T10:43:08.079 INFO:tasks.workunit.client.1.smithi150.stdout:8/903: dwrite d0/ddd/dfc/f11d [4194304,4194304] 0 2024-03-22T10:43:08.086 INFO:tasks.workunit.client.1.smithi150.stdout:3/853: rmdir d5/d22/d2f/d38/d3e/d72/d100 0 2024-03-22T10:43:08.087 INFO:tasks.workunit.client.1.smithi150.stdout:0/860: dwrite d0/f85 [4194304,4194304] 0 2024-03-22T10:43:08.088 INFO:tasks.workunit.client.1.smithi150.stdout:3/854: chown d5/da/d62/l8b 1 1 2024-03-22T10:43:08.109 INFO:tasks.workunit.client.1.smithi150.stdout:0/861: dread d0/d1/d84/d86/d42/d10/fa7 [0,4194304] 0 2024-03-22T10:43:08.109 INFO:tasks.workunit.client.1.smithi150.stdout:7/873: creat d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/f11e x:0 0 0 2024-03-22T10:43:08.109 INFO:tasks.workunit.client.1.smithi150.stdout:2/970: dwrite d12/d35/fb2 [4194304,4194304] 0 2024-03-22T10:43:08.111 INFO:tasks.workunit.client.1.smithi150.stdout:7/874: chown d2/d5 60 1 2024-03-22T10:43:08.119 INFO:tasks.workunit.client.1.smithi150.stdout:0/862: write d0/d1/d84/d86/d42/d10/d38/d52/f65 [2721664,17697] 0 2024-03-22T10:43:08.120 INFO:tasks.workunit.client.1.smithi150.stdout:8/904: dwrite d0/d2/fa8 [0,4194304] 0 2024-03-22T10:43:08.130 INFO:tasks.workunit.client.1.smithi150.stdout:6/847: sync 2024-03-22T10:43:08.136 INFO:tasks.workunit.client.1.smithi150.stdout:3/855: dread d5/da/d62/dba/fe3 [0,4194304] 0 2024-03-22T10:43:08.144 INFO:tasks.workunit.client.1.smithi150.stdout:0/863: dwrite d0/d1/d84/f10a [0,4194304] 0 2024-03-22T10:43:08.146 INFO:tasks.workunit.client.1.smithi150.stdout:7/875: dwrite d2/d5/d7/d9/db/d9f/f8b [0,4194304] 0 2024-03-22T10:43:08.146 INFO:tasks.workunit.client.1.smithi150.stdout:0/864: stat d0/d1/d84/d86/d42/f72 0 2024-03-22T10:43:08.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:07 smithi150 ceph-mon[34518]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:43:08.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:07 smithi150 ceph-mon[34518]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:43:08.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:07 smithi150 ceph-mon[34518]: pgmap v22: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 122 MiB/s rd, 297 MiB/s wr, 649 op/s 2024-03-22T10:43:08.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:07 smithi150 ceph-mon[34518]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:43:08.161 INFO:tasks.workunit.client.1.smithi150.stdout:2/971: dread d12/d19/db6/f4d [0,4194304] 0 2024-03-22T10:43:08.182 INFO:tasks.workunit.client.1.smithi150.stdout:6/848: write db/dd/d25/f76 [187639,62645] 0 2024-03-22T10:43:08.187 INFO:tasks.workunit.client.1.smithi150.stdout:7/876: dread d2/d5/d7/d9/db/d6c/d9c/fd1 [0,4194304] 0 2024-03-22T10:43:08.188 INFO:tasks.workunit.client.1.smithi150.stdout:3/856: unlink d5/l29 0 2024-03-22T10:43:08.189 INFO:tasks.workunit.client.1.smithi150.stdout:6/849: unlink db/dd/l99 0 2024-03-22T10:43:08.193 INFO:tasks.workunit.client.1.smithi150.stdout:0/865: write d0/f7b [8835801,102193] 0 2024-03-22T10:43:08.194 INFO:tasks.workunit.client.1.smithi150.stdout:8/905: rename d0/d2/d19/dc0/dda/d128 to d0/d12a 0 2024-03-22T10:43:08.197 INFO:tasks.workunit.client.1.smithi150.stdout:3/857: unlink d5/d22/d37/ff4 0 2024-03-22T10:43:08.200 INFO:tasks.workunit.client.1.smithi150.stdout:2/972: creat d12/d19/db6/da9/f12f x:0 0 0 2024-03-22T10:43:08.200 INFO:tasks.workunit.client.1.smithi150.stdout:0/866: dread - d0/d1/d84/d86/d42/d10/d53/fdc zero size 2024-03-22T10:43:08.201 INFO:tasks.workunit.client.1.smithi150.stdout:3/858: creat d5/da/f122 x:0 0 0 2024-03-22T10:43:08.202 INFO:tasks.workunit.client.1.smithi150.stdout:7/877: creat d2/d5/d7/d9/db/dfb/f11f x:0 0 0 2024-03-22T10:43:08.204 INFO:tasks.workunit.client.1.smithi150.stdout:2/973: write d12/d19/db6/da9/d67/f4e [3292413,25242] 0 2024-03-22T10:43:08.207 INFO:tasks.workunit.client.1.smithi150.stdout:8/906: rename d0/d2/d46/f4e to d0/d3c/d71/d7a/d89/f12b 0 2024-03-22T10:43:08.209 INFO:tasks.workunit.client.1.smithi150.stdout:0/867: read d0/d1/d84/d86/d11/d26/dd2/df0/f10c [1286436,77953] 0 2024-03-22T10:43:08.213 INFO:tasks.workunit.client.1.smithi150.stdout:6/850: write db/d52/d79/f8e [796224,125441] 0 2024-03-22T10:43:08.216 INFO:tasks.workunit.client.1.smithi150.stdout:3/859: creat d5/d22/d2f/d38/d59/def/f123 x:0 0 0 2024-03-22T10:43:08.217 INFO:tasks.workunit.client.1.smithi150.stdout:7/878: unlink d2/d5/d7/d9/db/d6c/fb6 0 2024-03-22T10:43:08.221 INFO:tasks.workunit.client.1.smithi150.stdout:7/879: stat d2/d5/d7/d9/db/c7a 0 2024-03-22T10:43:08.224 INFO:tasks.workunit.client.1.smithi150.stdout:3/860: symlink d5/d22/d2f/d38/d3e/d5b/da6/l124 0 2024-03-22T10:43:08.227 INFO:tasks.workunit.client.1.smithi150.stdout:2/974: creat d12/d21/f130 x:0 0 0 2024-03-22T10:43:08.230 INFO:tasks.workunit.client.1.smithi150.stdout:0/868: creat d0/d1/d84/d86/d42/d10/d38/d52/f114 x:0 0 0 2024-03-22T10:43:08.236 INFO:tasks.workunit.client.1.smithi150.stdout:0/869: mkdir d0/d1/d84/d86/d11/d26/d76/d115 0 2024-03-22T10:43:08.238 INFO:tasks.workunit.client.1.smithi150.stdout:7/880: creat d2/d5/d7/d9/db/d9f/f120 x:0 0 0 2024-03-22T10:43:08.241 INFO:tasks.workunit.client.1.smithi150.stdout:3/861: dread d5/da/d62/f8e [0,4194304] 0 2024-03-22T10:43:08.243 INFO:tasks.workunit.client.1.smithi150.stdout:0/870: mkdir d0/d1/d84/d86/d11/d66/d35/d5f/d116 0 2024-03-22T10:43:08.245 INFO:tasks.workunit.client.1.smithi150.stdout:8/907: dread d0/d2/d19/d29/d2f/df/d1b/f1c [0,4194304] 0 2024-03-22T10:43:08.249 INFO:tasks.workunit.client.1.smithi150.stdout:3/862: mknod d5/d22/c125 0 2024-03-22T10:43:08.250 INFO:tasks.workunit.client.1.smithi150.stdout:6/851: dwrite db/dd/d15/d60/d50/ffa [0,4194304] 0 2024-03-22T10:43:08.255 INFO:tasks.workunit.client.1.smithi150.stdout:2/975: dwrite d12/d19/db6/d63/dd8/fe8 [0,4194304] 0 2024-03-22T10:43:08.260 INFO:tasks.workunit.client.1.smithi150.stdout:6/852: mkdir db/d52/d9f/dc6/df1/db4/dfd/d129 0 2024-03-22T10:43:08.260 INFO:tasks.workunit.client.1.smithi150.stdout:8/908: truncate d0/d2/d19/d29/d37/f9c 4170379 0 2024-03-22T10:43:08.262 INFO:tasks.workunit.client.1.smithi150.stdout:2/976: creat d12/d19/db6/d54/f131 x:0 0 0 2024-03-22T10:43:08.267 INFO:tasks.workunit.client.1.smithi150.stdout:8/909: mknod d0/d2/d19/d29/d37/c12c 0 2024-03-22T10:43:08.268 INFO:tasks.workunit.client.1.smithi150.stdout:3/863: dread d5/d22/dc8/dda/d95/fea [0,4194304] 0 2024-03-22T10:43:08.273 INFO:tasks.workunit.client.1.smithi150.stdout:0/871: dwrite d0/f6d [0,4194304] 0 2024-03-22T10:43:08.276 INFO:tasks.workunit.client.1.smithi150.stdout:0/872: readlink d0/le4 0 2024-03-22T10:43:08.277 INFO:tasks.workunit.client.1.smithi150.stdout:2/977: rmdir d12/da1/dd6/de0 0 2024-03-22T10:43:08.278 INFO:tasks.workunit.client.1.smithi150.stdout:3/864: symlink d5/da/d78/db7/l126 0 2024-03-22T10:43:08.278 INFO:tasks.workunit.client.1.smithi150.stdout:7/881: dread d2/d5/d7/d8/fcc [0,4194304] 0 2024-03-22T10:43:08.285 INFO:tasks.workunit.client.1.smithi150.stdout:2/978: read - d12/d19/fa4 zero size 2024-03-22T10:43:08.286 INFO:tasks.workunit.client.1.smithi150.stdout:3/865: rmdir d5/d98 39 2024-03-22T10:43:08.287 INFO:tasks.workunit.client.1.smithi150.stdout:2/979: stat d12/d19/db6/d54 0 2024-03-22T10:43:08.288 INFO:tasks.workunit.client.1.smithi150.stdout:7/882: creat d2/d5/d7/d9/f121 x:0 0 0 2024-03-22T10:43:08.295 INFO:tasks.workunit.client.1.smithi150.stdout:2/980: dread - d12/d19/db6/da9/d97/fcb zero size 2024-03-22T10:43:08.295 INFO:tasks.workunit.client.1.smithi150.stdout:3/866: mkdir d5/d22/d2f/d38/d59/d88/d127 0 2024-03-22T10:43:08.297 INFO:tasks.workunit.client.1.smithi150.stdout:2/981: fdatasync d12/da1/ffb 0 2024-03-22T10:43:08.301 INFO:tasks.workunit.client.1.smithi150.stdout:8/910: dwrite d0/ddd/dfc/f11d [4194304,4194304] 0 2024-03-22T10:43:08.306 INFO:tasks.workunit.client.1.smithi150.stdout:2/982: creat d12/d19/db6/da9/d67/d9a/f132 x:0 0 0 2024-03-22T10:43:08.312 INFO:tasks.workunit.client.1.smithi150.stdout:6/853: dread db/dd/d82/d84/fb5 [0,4194304] 0 2024-03-22T10:43:08.315 INFO:tasks.workunit.client.1.smithi150.stdout:8/911: rename d0/d2/d19/dc0 to d0/d3c/d71/d7a/d89/d12d 0 2024-03-22T10:43:08.317 INFO:tasks.workunit.client.1.smithi150.stdout:0/873: dread d0/d1/d84/d86/d11/d66/d35/fde [0,4194304] 0 2024-03-22T10:43:08.319 INFO:tasks.workunit.client.1.smithi150.stdout:7/883: dread d2/d5/d7/d9/db/d9f/d96/fe4 [0,4194304] 0 2024-03-22T10:43:08.320 INFO:tasks.workunit.client.1.smithi150.stdout:6/854: mkdir db/dd/d15/d60/d8b/de7/d12a 0 2024-03-22T10:43:08.325 INFO:tasks.workunit.client.1.smithi150.stdout:2/983: link d12/d19/db6/da9/d97/d78/dbd/fec d12/d21/d81/f133 0 2024-03-22T10:43:08.329 INFO:tasks.workunit.client.1.smithi150.stdout:0/874: chown d0/d1/d84/d86/d11/d26/d76/f10d 75342632 1 2024-03-22T10:43:08.329 INFO:tasks.workunit.client.1.smithi150.stdout:3/867: dwrite d5/f35 [0,4194304] 0 2024-03-22T10:43:08.329 INFO:tasks.workunit.client.1.smithi150.stdout:7/884: write d2/d5/d7/d9/db/da2/fbe [2458591,5742] 0 2024-03-22T10:43:08.342 INFO:tasks.workunit.client.1.smithi150.stdout:7/885: mkdir d2/d5/d7/d8/d57/d4a/d8d/da9/d122 0 2024-03-22T10:43:08.343 INFO:tasks.workunit.client.1.smithi150.stdout:3/868: symlink d5/d22/d2f/d38/d3e/d5b/l128 0 2024-03-22T10:43:08.347 INFO:tasks.workunit.client.1.smithi150.stdout:0/875: creat d0/d1/d84/d86/d11/f117 x:0 0 0 2024-03-22T10:43:08.352 INFO:tasks.workunit.client.1.smithi150.stdout:3/869: read d5/d22/d2f/d38/d3e/d5b/da6/fbd [4080558,88017] 0 2024-03-22T10:43:08.356 INFO:tasks.workunit.client.1.smithi150.stdout:7/886: creat d2/d5/d7/d8/d57/d105/f123 x:0 0 0 2024-03-22T10:43:08.358 INFO:tasks.workunit.client.1.smithi150.stdout:2/984: dwrite d12/d21/f130 [0,4194304] 0 2024-03-22T10:43:08.365 INFO:tasks.workunit.client.1.smithi150.stdout:4/922: sync 2024-03-22T10:43:08.370 INFO:tasks.workunit.client.1.smithi150.stdout:2/985: read d12/f55 [90783,51487] 0 2024-03-22T10:43:08.382 INFO:tasks.workunit.client.1.smithi150.stdout:8/912: dwrite d0/d2/d19/dd1/f101 [0,4194304] 0 2024-03-22T10:43:08.384 INFO:tasks.workunit.client.1.smithi150.stdout:7/887: rename d2/d5/d7/d8/d46/d49 to d2/d124 0 2024-03-22T10:43:08.386 INFO:tasks.workunit.client.1.smithi150.stdout:6/855: dwrite db/dd/d15/d6a/f125 [0,4194304] 0 2024-03-22T10:43:08.520 INFO:tasks.workunit.client.1.smithi150.stdout:4/923: mknod d0/d2f/d10d/d4d/d6c/d61/d71/d108/c134 0 2024-03-22T10:43:08.520 INFO:tasks.workunit.client.1.smithi150.stdout:0/876: getdents d0/d1/d84/d86/d42/d10/d38/d79 0 2024-03-22T10:43:08.523 INFO:tasks.workunit.client.1.smithi150.stdout:3/870: truncate d5/f31 4837233 0 2024-03-22T10:43:08.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.536+0000 7fdb47200700 1 -- 172.21.15.115:0/4104734604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 msgr2=0x7fdb48073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:08.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.536+0000 7fdb47200700 1 --2- 172.21.15.115:0/4104734604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 0x7fdb48073180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009910 tx=0x7fdb38009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:08.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.536+0000 7fdb47200700 1 -- 172.21.15.115:0/4104734604 shutdown_connections 2024-03-22T10:43:08.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.536+0000 7fdb47200700 1 --2- 172.21.15.115:0/4104734604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb48073750 0x7fdb48073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:08.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.536+0000 7fdb47200700 1 --2- 172.21.15.115:0/4104734604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 0x7fdb48073180 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:08.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.536+0000 7fdb47200700 1 -- 172.21.15.115:0/4104734604 >> 172.21.15.115:0/4104734604 conn(0x7fdb480fb560 msgr2=0x7fdb480fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:08.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.538+0000 7fdb47200700 1 -- 172.21.15.115:0/4104734604 shutdown_connections 2024-03-22T10:43:08.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.538+0000 7fdb47200700 1 -- 172.21.15.115:0/4104734604 wait complete. 2024-03-22T10:43:08.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.539+0000 7fdb47200700 1 Processor -- start 2024-03-22T10:43:08.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.539+0000 7fdb47200700 1 -- start start 2024-03-22T10:43:08.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.540+0000 7fdb47200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb48073750 0x7fdb481979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:08.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.540+0000 7fdb47200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 0x7fdb48197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:08.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.540+0000 7fdb47200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48198500 con 0x7fdb48074d20 2024-03-22T10:43:08.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.540+0000 7fdb47200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48198640 con 0x7fdb48073750 2024-03-22T10:43:08.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.540+0000 7fdb45400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 0x7fdb48197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:08.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.540+0000 7fdb45400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 0x7fdb48197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53224/0 (socket says 172.21.15.115:53224) 2024-03-22T10:43:08.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.540+0000 7fdb45400700 1 -- 172.21.15.115:0/4108690825 learned_addr learned my addr 172.21.15.115:0/4108690825 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:08.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.540+0000 7fdb45e00700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb48073750 0x7fdb481979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:08.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.541+0000 7fdb45400700 1 -- 172.21.15.115:0/4108690825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb48073750 msgr2=0x7fdb481979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:08.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.541+0000 7fdb45400700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb48073750 0x7fdb481979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:08.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.541+0000 7fdb45400700 1 -- 172.21.15.115:0/4108690825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb380095f0 con 0x7fdb48074d20 2024-03-22T10:43:08.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.541+0000 7fdb45400700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 0x7fdb48197ee0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d8d0 tx=0x7fdb3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:08.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.541+0000 7fdb3ea00700 1 -- 172.21.15.115:0/4108690825 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb34009940 con 0x7fdb48074d20 2024-03-22T10:43:08.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.541+0000 7fdb3ea00700 1 -- 172.21.15.115:0/4108690825 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb34010460 con 0x7fdb48074d20 2024-03-22T10:43:08.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.542+0000 7fdb3ea00700 1 -- 172.21.15.115:0/4108690825 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3400f5f0 con 0x7fdb48074d20 2024-03-22T10:43:08.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.542+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4819d0f0 con 0x7fdb48074d20 2024-03-22T10:43:08.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.542+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4819d640 con 0x7fdb48074d20 2024-03-22T10:43:08.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.543+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb28005320 con 0x7fdb48074d20 2024-03-22T10:43:08.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.545+0000 7fdb3ea00700 1 -- 172.21.15.115:0/4108690825 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fdb34009aa0 con 0x7fdb48074d20 2024-03-22T10:43:08.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.546+0000 7fdb3ea00700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb2c06dd50 0x7fdb2c070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:08.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.546+0000 7fdb3ea00700 1 -- 172.21.15.115:0/4108690825 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fdb3408d8a0 con 0x7fdb48074d20 2024-03-22T10:43:08.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.547+0000 7fdb45e00700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb2c06dd50 0x7fdb2c070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:08.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.547+0000 7fdb45e00700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb2c06dd50 0x7fdb2c070200 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb38006010 tx=0x7fdb3800b540 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:08.542 INFO:tasks.workunit.client.1.smithi150.stdout:7/888: dread d2/d5/d7/f2b [4194304,4194304] 0 2024-03-22T10:43:08.542 INFO:tasks.workunit.client.1.smithi150.stdout:6/856: dwrite db/dd/d15/d60/d50/f64 [4194304,4194304] 0 2024-03-22T10:43:08.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.552+0000 7fdb3ea00700 1 -- 172.21.15.115:0/4108690825 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fdb3405a940 con 0x7fdb48074d20 2024-03-22T10:43:08.551 INFO:tasks.workunit.client.1.smithi150.stdout:4/924: chown d0/dce/d10e/d121/d7c/dc7/df7/l131 0 1 2024-03-22T10:43:08.555 INFO:tasks.workunit.client.1.smithi150.stdout:0/877: truncate d0/d1/d84/d86/d42/d10/d38/ff2 846919 0 2024-03-22T10:43:08.556 INFO:tasks.workunit.client.1.smithi150.stdout:6/857: rename db/dd/d15/d28/d6c/d9b/cb8 to db/dd/d15/d60/d8b/de7/c12b 0 2024-03-22T10:43:08.557 INFO:tasks.workunit.client.1.smithi150.stdout:3/871: mknod d5/d22/d2f/d38/d3e/d5b/dee/c129 0 2024-03-22T10:43:08.563 INFO:tasks.workunit.client.1.smithi150.stdout:3/872: truncate d5/fcf 609407 0 2024-03-22T10:43:08.568 INFO:tasks.workunit.client.1.smithi150.stdout:6/858: rename db/dd/d15/l42 to db/dd/d15/d57/l12c 0 2024-03-22T10:43:08.572 INFO:tasks.workunit.client.1.smithi150.stdout:3/873: truncate d5/fb 3652700 0 2024-03-22T10:43:08.577 INFO:tasks.workunit.client.1.smithi150.stdout:4/925: dread d0/dce/d10e/d121/f105 [0,4194304] 0 2024-03-22T10:43:08.581 INFO:tasks.workunit.client.1.smithi150.stdout:6/859: symlink db/dd/d15/d28/d6c/d9b/l12d 0 2024-03-22T10:43:08.583 INFO:tasks.workunit.client.1.smithi150.stdout:4/926: read - d0/dce/d10e/d121/f8e zero size 2024-03-22T10:43:08.583 INFO:tasks.workunit.client.1.smithi150.stdout:3/874: symlink d5/d22/d118/l12a 0 2024-03-22T10:43:08.584 INFO:tasks.workunit.client.1.smithi150.stdout:0/878: dwrite d0/d1/d84/d86/da6/f103 [0,4194304] 0 2024-03-22T10:43:08.587 INFO:tasks.workunit.client.1.smithi150.stdout:0/879: stat d0/d1/d84/ff8 0 2024-03-22T10:43:08.588 INFO:tasks.workunit.client.1.smithi150.stdout:3/875: write d5/da/d8d/fbb [392113,6864] 0 2024-03-22T10:43:08.589 INFO:tasks.workunit.client.1.smithi150.stdout:4/927: dread - d0/d2f/d63/d7b/d9c/db1/f11f zero size 2024-03-22T10:43:08.594 INFO:tasks.workunit.client.1.smithi150.stdout:7/889: truncate d2/d5/d7/d9/db/d6c/fd6 1260171 0 2024-03-22T10:43:08.596 INFO:tasks.workunit.client.1.smithi150.stdout:6/860: mknod db/dd/d15/d6a/dde/c12e 0 2024-03-22T10:43:08.599 INFO:tasks.workunit.client.1.smithi150.stdout:6/861: write db/dd/d15/ff7 [9043351,88847] 0 2024-03-22T10:43:08.603 INFO:tasks.workunit.client.1.smithi150.stdout:4/928: rename d0/dce/d10e/d121/d7c/d99/l129 to d0/d2f/d10d/d4d/d6c/d61/dd1/l135 0 2024-03-22T10:43:08.611 INFO:tasks.workunit.client.1.smithi150.stdout:7/890: mkdir d2/d5/d125 0 2024-03-22T10:43:08.617 INFO:tasks.workunit.client.1.smithi150.stdout:0/880: creat d0/d1/d84/d86/d42/d10/d38/f118 x:0 0 0 2024-03-22T10:43:08.618 INFO:tasks.workunit.client.1.smithi150.stdout:9/915: sync 2024-03-22T10:43:08.619 INFO:tasks.workunit.client.1.smithi150.stdout:1/835: sync 2024-03-22T10:43:08.619 INFO:tasks.workunit.client.1.smithi150.stdout:5/912: sync 2024-03-22T10:43:08.620 INFO:tasks.workunit.client.1.smithi150.stdout:3/876: dwrite d5/d53/f54 [0,4194304] 0 2024-03-22T10:43:08.623 INFO:tasks.workunit.client.1.smithi150.stdout:6/862: dread db/dd/d25/f6d [0,4194304] 0 2024-03-22T10:43:08.624 INFO:tasks.workunit.client.1.smithi150.stdout:4/929: dread d0/dce/d10e/d121/f123 [0,4194304] 0 2024-03-22T10:43:08.626 INFO:tasks.workunit.client.1.smithi150.stdout:5/913: fdatasync d0/d8/dc/de/d14/d2e/d45/dac/faf 0 2024-03-22T10:43:08.630 INFO:tasks.workunit.client.1.smithi150.stdout:3/877: write d5/d22/dc8/dda/d95/f121 [843060,45556] 0 2024-03-22T10:43:08.631 INFO:tasks.workunit.client.1.smithi150.stdout:1/836: read d1/f3a [2350720,114794] 0 2024-03-22T10:43:08.636 INFO:tasks.workunit.client.1.smithi150.stdout:5/914: truncate d0/d8/dc/de/d14/d4f/d47/f117 622819 0 2024-03-22T10:43:08.640 INFO:tasks.workunit.client.1.smithi150.stdout:6/863: unlink db/d52/f108 0 2024-03-22T10:43:08.642 INFO:tasks.workunit.client.1.smithi150.stdout:0/881: creat d0/d12/d101/f119 x:0 0 0 2024-03-22T10:43:08.649 INFO:tasks.workunit.client.1.smithi150.stdout:1/837: dread d1/da/d15/f74 [0,4194304] 0 2024-03-22T10:43:08.651 INFO:tasks.workunit.client.1.smithi150.stdout:1/838: readlink d1/da/d15/lf7 0 2024-03-22T10:43:08.655 INFO:tasks.workunit.client.1.smithi150.stdout:6/864: symlink db/d52/de6/l12f 0 2024-03-22T10:43:08.657 INFO:tasks.workunit.client.1.smithi150.stdout:0/882: creat d0/d12/d101/f11a x:0 0 0 2024-03-22T10:43:08.661 INFO:tasks.workunit.client.1.smithi150.stdout:3/878: mkdir d5/d22/d2f/d38/d12b 0 2024-03-22T10:43:08.663 INFO:tasks.workunit.client.1.smithi150.stdout:5/915: mknod d0/d33/d6f/d137/c13d 0 2024-03-22T10:43:08.665 INFO:tasks.workunit.client.1.smithi150.stdout:3/879: chown d5/da/d78/d86/l6a 104039 1 2024-03-22T10:43:08.668 INFO:tasks.workunit.client.1.smithi150.stdout:3/880: dread - d5/da/d62/ff0 zero size 2024-03-22T10:43:08.675 INFO:tasks.workunit.client.1.smithi150.stdout:6/865: symlink db/dd/d25/l130 0 2024-03-22T10:43:08.681 INFO:tasks.workunit.client.1.smithi150.stdout:5/916: symlink d0/d8/dc/de/l13e 0 2024-03-22T10:43:08.682 INFO:tasks.workunit.client.1.smithi150.stdout:4/930: dread d0/d2f/d10d/f74 [0,4194304] 0 2024-03-22T10:43:08.683 INFO:tasks.workunit.client.1.smithi150.stdout:6/866: creat db/d52/d9f/dc6/f131 x:0 0 0 2024-03-22T10:43:08.685 INFO:tasks.workunit.client.1.smithi150.stdout:7/891: dwrite d2/f82 [0,4194304] 0 2024-03-22T10:43:08.688 INFO:tasks.workunit.client.1.smithi150.stdout:6/867: stat db/dd/d15 0 2024-03-22T10:43:08.688 INFO:tasks.workunit.client.1.smithi150.stdout:3/881: read - d5/daf/fd9 zero size 2024-03-22T10:43:08.689 INFO:tasks.workunit.client.1.smithi150.stdout:1/839: truncate d1/da/d15/f74 1416353 0 2024-03-22T10:43:08.690 INFO:tasks.workunit.client.1.smithi150.stdout:5/917: rmdir d0/d8/dc/de/d14/d2e/d46 39 2024-03-22T10:43:08.692 INFO:tasks.workunit.client.1.smithi150.stdout:1/840: chown d1/f6c 30254 1 2024-03-22T10:43:08.692 INFO:tasks.workunit.client.1.smithi150.stdout:3/882: dread - d5/d22/d2f/d38/d3e/d5b/fc7 zero size 2024-03-22T10:43:08.693 INFO:tasks.workunit.client.1.smithi150.stdout:6/868: fdatasync db/dd/d15/d30/fab 0 2024-03-22T10:43:08.698 INFO:tasks.workunit.client.1.smithi150.stdout:5/918: chown d0/d8/dc/de/d96/f10a 22035 1 2024-03-22T10:43:08.699 INFO:tasks.workunit.client.1.smithi150.stdout:1/841: mknod d1/c115 0 2024-03-22T10:43:08.706 INFO:tasks.workunit.client.1.smithi150.stdout:7/892: rename d2/d5/d7/d8/d57 to d2/d5/d7/d8/d57/d4a/d126 22 2024-03-22T10:43:08.712 INFO:tasks.workunit.client.1.smithi150.stdout:4/931: write d0/d2f/d63/d7b/d9c/fa6 [745868,84625] 0 2024-03-22T10:43:08.713 INFO:tasks.workunit.client.1.smithi150.stdout:1/842: fsync d1/d19/d50/d6e/f10b 0 2024-03-22T10:43:08.715 INFO:tasks.workunit.client.1.smithi150.stdout:0/883: dwrite d0/d1/d84/d86/d42/f4a [4194304,4194304] 0 2024-03-22T10:43:08.725 INFO:tasks.workunit.client.1.smithi150.stdout:0/884: rmdir d0/d1/d84/d86/d42/d10/d53 39 2024-03-22T10:43:08.728 INFO:tasks.workunit.client.1.smithi150.stdout:0/885: readlink d0/d12/l18 0 2024-03-22T10:43:08.729 INFO:tasks.workunit.client.1.smithi150.stdout:5/919: dwrite d0/d8/dc/de/f82 [0,4194304] 0 2024-03-22T10:43:08.730 INFO:tasks.workunit.client.1.smithi150.stdout:4/932: mkdir d0/d2f/d63/d7b/d136 0 2024-03-22T10:43:08.730 INFO:tasks.workunit.client.1.smithi150.stdout:0/886: fdatasync d0/d12/f10b 0 2024-03-22T10:43:08.732 INFO:tasks.workunit.client.1.smithi150.stdout:6/869: dwrite db/dd/d15/d6a/d8d/fbd [0,4194304] 0 2024-03-22T10:43:08.738 INFO:tasks.workunit.client.1.smithi150.stdout:1/843: creat d1/da/f116 x:0 0 0 2024-03-22T10:43:08.739 INFO:tasks.workunit.client.1.smithi150.stdout:3/883: getdents d5/d22/d2f/d38/d59/d88/d10d 0 2024-03-22T10:43:08.741 INFO:tasks.workunit.client.1.smithi150.stdout:5/920: write d0/d8/dc/de/d96/db8/fe1 [2648692,121369] 0 2024-03-22T10:43:08.744 INFO:tasks.workunit.client.1.smithi150.stdout:7/893: rename d2/d5/d7/d9/db/d9f/d96/d71/d106/d91/dad to d2/d5/d7/d127 0 2024-03-22T10:43:08.747 INFO:tasks.workunit.client.1.smithi150.stdout:9/916: sync 2024-03-22T10:43:08.747 INFO:tasks.workunit.client.1.smithi150.stdout:2/986: sync 2024-03-22T10:43:08.747 INFO:tasks.workunit.client.1.smithi150.stdout:8/913: sync 2024-03-22T10:43:08.750 INFO:tasks.workunit.client.1.smithi150.stdout:6/870: fsync db/dd/d15/d6a/d8d/fc4 0 2024-03-22T10:43:08.752 INFO:tasks.workunit.client.1.smithi150.stdout:1/844: mknod d1/c117 0 2024-03-22T10:43:08.754 INFO:tasks.workunit.client.1.smithi150.stdout:4/933: dread d0/d2f/d10d/d4d/d6c/d61/fbb [0,4194304] 0 2024-03-22T10:43:08.758 INFO:tasks.workunit.client.1.smithi150.stdout:3/884: mknod d5/d98/c12c 0 2024-03-22T10:43:08.758 INFO:tasks.workunit.client.1.smithi150.stdout:1/845: readlink d1/da/d15/d4f/l83 0 2024-03-22T10:43:08.759 INFO:tasks.workunit.client.1.smithi150.stdout:4/934: rename d0 to d0/d47/d72/d137 22 2024-03-22T10:43:08.767 INFO:tasks.workunit.client.1.smithi150.stdout:2/987: mknod d12/d19/db6/da9/d67/d8d/da6/c134 0 2024-03-22T10:43:08.773 INFO:tasks.workunit.client.1.smithi150.stdout:0/887: dwrite d0/d12/fcd [0,4194304] 0 2024-03-22T10:43:08.773 INFO:tasks.workunit.client.1.smithi150.stdout:3/885: readlink d5/da/d78/d86/l108 0 2024-03-22T10:43:08.780 INFO:tasks.workunit.client.1.smithi150.stdout:1/846: unlink d1/d19/d50/d51/da0/dcf/dd6/de4/c10d 0 2024-03-22T10:43:08.781 INFO:tasks.workunit.client.1.smithi150.stdout:4/935: unlink d0/d2f/d63/d7b/d9c/db1/dd7/c128 0 2024-03-22T10:43:08.783 INFO:tasks.workunit.client.1.smithi150.stdout:2/988: unlink d12/d19/db6/da9/d2c/d65/la3 0 2024-03-22T10:43:08.784 INFO:tasks.workunit.client.1.smithi150.stdout:3/886: rename d5/f25 to d5/d22/d2f/d38/d59/d88/d127/f12d 0 2024-03-22T10:43:08.785 INFO:tasks.workunit.client.1.smithi150.stdout:4/936: chown d0/d2f/d10d/d4d/d6c/f62 25658 1 2024-03-22T10:43:08.789 INFO:tasks.workunit.client.1.smithi150.stdout:9/917: creat d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/f12f x:0 0 0 2024-03-22T10:43:08.789 INFO:tasks.workunit.client.1.smithi150.stdout:1/847: fdatasync d1/f3a 0 2024-03-22T10:43:08.791 INFO:tasks.workunit.client.1.smithi150.stdout:0/888: creat d0/d1/f11b x:0 0 0 2024-03-22T10:43:08.792 INFO:tasks.workunit.client.1.smithi150.stdout:7/894: dwrite d2/d5/d7/d8/d46/fbc [0,4194304] 0 2024-03-22T10:43:08.792 INFO:tasks.workunit.client.1.smithi150.stdout:3/887: chown d5/d22/d2f/d38/d3e/d5b/dee/c129 30089 1 2024-03-22T10:43:08.798 INFO:tasks.workunit.client.1.smithi150.stdout:6/871: dread db/d52/d9f/dc6/df1/ff9 [0,4194304] 0 2024-03-22T10:43:08.801 INFO:tasks.workunit.client.1.smithi150.stdout:2/989: symlink d12/d19/db6/da9/d67/d8d/d107/l135 0 2024-03-22T10:43:08.802 INFO:tasks.workunit.client.1.smithi150.stdout:9/918: creat d1/d8/de/dbd/ddc/f130 x:0 0 0 2024-03-22T10:43:08.808 INFO:tasks.workunit.client.1.smithi150.stdout:5/921: dread d0/f6 [0,4194304] 0 2024-03-22T10:43:08.809 INFO:tasks.workunit.client.1.smithi150.stdout:1/848: symlink d1/da/d2a/d65/d78/l118 0 2024-03-22T10:43:08.809 INFO:tasks.workunit.client.1.smithi150.stdout:8/914: dread d0/d2/f86 [0,4194304] 0 2024-03-22T10:43:08.811 INFO:tasks.workunit.client.1.smithi150.stdout:4/937: dread d0/d2f/d10d/d4d/d9f/fb5 [0,4194304] 0 2024-03-22T10:43:08.813 INFO:tasks.workunit.client.1.smithi150.stdout:3/888: mknod d5/d22/d118/c12e 0 2024-03-22T10:43:08.818 INFO:tasks.workunit.client.1.smithi150.stdout:3/889: dread - d5/da/d62/dba/ff9 zero size 2024-03-22T10:43:08.818 INFO:tasks.workunit.client.1.smithi150.stdout:4/938: dread - d0/d2f/d63/d7b/d9c/db1/dd7/ff2 zero size 2024-03-22T10:43:08.822 INFO:tasks.workunit.client.1.smithi150.stdout:6/872: creat db/d52/d9f/dc6/df1/db4/dfd/f132 x:0 0 0 2024-03-22T10:43:08.824 INFO:tasks.workunit.client.1.smithi150.stdout:1/849: readlink d1/da/lc 0 2024-03-22T10:43:08.827 INFO:tasks.workunit.client.1.smithi150.stdout:1/850: read - d1/da/d2a/dc0/fe7 zero size 2024-03-22T10:43:08.828 INFO:tasks.workunit.client.1.smithi150.stdout:9/919: rename d1/d8/de/d73/d101/l22 to d1/d8/de/d73/d101/d32/l131 0 2024-03-22T10:43:08.832 INFO:tasks.workunit.client.1.smithi150.stdout:8/915: creat d0/d3c/d71/d7a/d89/db5/f12e x:0 0 0 2024-03-22T10:43:08.832 INFO:tasks.workunit.client.1.smithi150.stdout:7/895: symlink d2/d5/d7/d9/db/d9f/d96/d71/l128 0 2024-03-22T10:43:08.832 INFO:tasks.workunit.client.1.smithi150.stdout:9/920: dread - d1/d49/d98/f112 zero size 2024-03-22T10:43:08.834 INFO:tasks.workunit.client.1.smithi150.stdout:5/922: dwrite d0/f10 [0,4194304] 0 2024-03-22T10:43:08.839 INFO:tasks.workunit.client.1.smithi150.stdout:6/873: dread - db/dd/d15/d6a/d8d/f95 zero size 2024-03-22T10:43:08.839 INFO:tasks.workunit.client.1.smithi150.stdout:3/890: rename d5/da/d78/d86/lfd to d5/da/d8d/l12f 0 2024-03-22T10:43:08.846 INFO:tasks.workunit.client.1.smithi150.stdout:5/923: fsync d0/d8/dc/de/d14/d2e/d45/dac/fbd 0 2024-03-22T10:43:08.846 INFO:tasks.workunit.client.1.smithi150.stdout:7/896: creat d2/d5/d7/d38/f129 x:0 0 0 2024-03-22T10:43:08.846 INFO:tasks.workunit.client.1.smithi150.stdout:9/921: chown d1/d8/de/df/d36/d121/d4e/de9/lf3 0 1 2024-03-22T10:43:08.847 INFO:tasks.workunit.client.1.smithi150.stdout:1/851: symlink d1/da/d2a/l119 0 2024-03-22T10:43:08.848 INFO:tasks.workunit.client.1.smithi150.stdout:7/897: readlink d2/d5/d7/d9/db/lb5 0 2024-03-22T10:43:08.851 INFO:tasks.workunit.client.1.smithi150.stdout:2/990: write d12/d19/db6/da9/d67/d9a/fca [6754363,4784] 0 2024-03-22T10:43:08.853 INFO:tasks.workunit.client.1.smithi150.stdout:3/891: symlink d5/d22/dc8/dda/l130 0 2024-03-22T10:43:08.855 INFO:tasks.workunit.client.1.smithi150.stdout:3/892: dread - d5/da/ffa zero size 2024-03-22T10:43:08.856 INFO:tasks.workunit.client.1.smithi150.stdout:1/852: fsync d1/d19/d1a/f49 0 2024-03-22T10:43:08.858 INFO:tasks.workunit.client.1.smithi150.stdout:8/916: link d0/d2/d19/d29/d88/f9a d0/d3c/d71/d7a/f12f 0 2024-03-22T10:43:08.860 INFO:tasks.workunit.client.1.smithi150.stdout:4/939: rename d0/dce/d10e/d121/d7c/d99 to d0/d2f/d10d/d4d/d6c/d61/d138 0 2024-03-22T10:43:08.862 INFO:tasks.workunit.client.1.smithi150.stdout:8/917: chown d0/d3c/d71/d7a/d89/d113 280200535 1 2024-03-22T10:43:08.873 INFO:tasks.workunit.client.1.smithi150.stdout:4/940: read d0/d2f/d10d/f39 [928888,42612] 0 2024-03-22T10:43:08.874 INFO:tasks.workunit.client.1.smithi150.stdout:2/991: symlink d12/d19/l136 0 2024-03-22T10:43:08.879 INFO:tasks.workunit.client.1.smithi150.stdout:8/918: dread d0/d5d/dd9/fe9 [0,4194304] 0 2024-03-22T10:43:08.880 INFO:tasks.workunit.client.1.smithi150.stdout:1/853: mkdir d1/d19/d50/d6e/d11a 0 2024-03-22T10:43:08.884 INFO:tasks.workunit.client.1.smithi150.stdout:7/898: dread d2/d5/d7/d8/d57/d4a/f33 [0,4194304] 0 2024-03-22T10:43:08.890 INFO:tasks.workunit.client.1.smithi150.stdout:6/874: write db/dd/d15/d6a/d8d/fc4 [1037408,58701] 0 2024-03-22T10:43:08.890 INFO:tasks.workunit.client.1.smithi150.stdout:0/889: rename d0/d1/d84/d86/l99 to d0/d1/d84/d86/d42/d10/d38/d52/d9b/dc2/l11c 0 2024-03-22T10:43:08.894 INFO:tasks.workunit.client.1.smithi150.stdout:2/992: truncate f11 1611263 0 2024-03-22T10:43:08.898 INFO:tasks.workunit.client.1.smithi150.stdout:1/854: chown d1/da/d15/d18/ca8 208 1 2024-03-22T10:43:08.898 INFO:tasks.workunit.client.1.smithi150.stdout:3/893: dread d5/d22/d2f/d38/fb4 [0,4194304] 0 2024-03-22T10:43:08.902 INFO:tasks.workunit.client.1.smithi150.stdout:7/899: symlink d2/d5/d7/d88/l12a 0 2024-03-22T10:43:08.902 INFO:tasks.workunit.client.1.smithi150.stdout:4/941: read d0/de/f4b [2632011,125253] 0 2024-03-22T10:43:08.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.912+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb28000bf0 con 0x7fdb2c06dd50 2024-03-22T10:43:08.905 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:08 smithi115 ceph-mon[29111]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:43:08.905 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:08 smithi115 ceph-mon[29111]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:43:08.905 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:08 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:08.905 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:08 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:08.906 INFO:tasks.workunit.client.1.smithi150.stdout:6/875: chown db/dd/d15/c1a 59233 1 2024-03-22T10:43:08.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.914+0000 7fdb3ea00700 1 -- 172.21.15.115:0/4108690825 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fdb28000bf0 con 0x7fdb2c06dd50 2024-03-22T10:43:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.916+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb2c06dd50 msgr2=0x7fdb2c070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.916+0000 7fdb47200700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb2c06dd50 0x7fdb2c070200 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb38006010 tx=0x7fdb3800b540 comp rx=0 tx=0).stop 2024-03-22T10:43:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.916+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 msgr2=0x7fdb48197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.916+0000 7fdb47200700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 0x7fdb48197ee0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d8d0 tx=0x7fdb3400dc90 comp rx=0 tx=0).stop 2024-03-22T10:43:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.917+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 shutdown_connections 2024-03-22T10:43:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.917+0000 7fdb47200700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb2c06dd50 0x7fdb2c070200 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.917+0000 7fdb47200700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb48073750 0x7fdb481979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.917+0000 7fdb47200700 1 --2- 172.21.15.115:0/4108690825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48074d20 0x7fdb48197ee0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:08.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.917+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 >> 172.21.15.115:0/4108690825 conn(0x7fdb480fb560 msgr2=0x7fdb48105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:08.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.917+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 shutdown_connections 2024-03-22T10:43:08.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:08.917+0000 7fdb47200700 1 -- 172.21.15.115:0/4108690825 wait complete. 2024-03-22T10:43:08.912 INFO:tasks.workunit.client.1.smithi150.stdout:0/890: dread - d0/d1/d84/d86/d11/d26/d76/f10e zero size 2024-03-22T10:43:08.916 INFO:tasks.workunit.client.1.smithi150.stdout:9/922: dread d1/d8/d95/fa6 [0,4194304] 0 2024-03-22T10:43:08.916 INFO:tasks.workunit.client.1.smithi150.stdout:5/924: rename d0/d8/dc/de/d14/d4f/d47/ff2 to d0/dae/db2/df9/f13f 0 2024-03-22T10:43:08.917 INFO:tasks.workunit.client.1.smithi150.stdout:3/894: fsync d5/da/f40 0 2024-03-22T10:43:08.921 INFO:tasks.workunit.client.1.smithi150.stdout:7/900: symlink d2/d5/d7/d9/l12b 0 2024-03-22T10:43:08.924 INFO:tasks.workunit.client.1.smithi150.stdout:4/942: mknod d0/d2f/d10d/d4d/d9f/c139 0 2024-03-22T10:43:08.924 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:43:08.929 INFO:tasks.workunit.client.1.smithi150.stdout:0/891: creat d0/d1/d84/d86/da6/dc8/f11d x:0 0 0 2024-03-22T10:43:08.933 INFO:tasks.workunit.client.1.smithi150.stdout:0/892: chown d0/d1/d84/d86/d42 27723 1 2024-03-22T10:43:08.937 INFO:tasks.workunit.client.1.smithi150.stdout:1/855: creat d1/d19/d50/d6e/d11a/f11b x:0 0 0 2024-03-22T10:43:08.937 INFO:tasks.workunit.client.1.smithi150.stdout:4/943: read d0/f8 [1460625,115053] 0 2024-03-22T10:43:08.944 INFO:tasks.workunit.client.1.smithi150.stdout:8/919: dwrite d0/d3c/d71/f59 [4194304,4194304] 0 2024-03-22T10:43:08.944 INFO:tasks.workunit.client.1.smithi150.stdout:6/876: write db/dd/d15/d60/d50/fb0 [999708,127097] 0 2024-03-22T10:43:08.949 INFO:tasks.workunit.client.1.smithi150.stdout:4/944: write d0/d2f/d10d/d4d/d6c/d61/dd1/f118 [657918,88588] 0 2024-03-22T10:43:08.953 INFO:tasks.workunit.client.1.smithi150.stdout:8/920: write d0/d2/f117 [590130,31202] 0 2024-03-22T10:43:08.960 INFO:tasks.workunit.client.1.smithi150.stdout:7/901: symlink d2/d5/d7/d8/d57/l12c 0 2024-03-22T10:43:08.960 INFO:tasks.workunit.client.1.smithi150.stdout:2/993: truncate d12/d19/d2d/fcf 3410297 0 2024-03-22T10:43:08.964 INFO:tasks.workunit.client.1.smithi150.stdout:1/856: read d1/d19/d50/d6e/fb9 [638214,31011] 0 2024-03-22T10:43:08.964 INFO:tasks.workunit.client.1.smithi150.stdout:9/923: link d1/d8/de/d73/d101/d29/lf4 d1/d19/d7d/dc1/dca/dd7/df0/l132 0 2024-03-22T10:43:08.965 INFO:tasks.workunit.client.1.smithi150.stdout:5/925: dwrite d0/d37/fdf [0,4194304] 0 2024-03-22T10:43:08.969 INFO:tasks.workunit.client.1.smithi150.stdout:5/926: chown d0/d8/dc/de/f119 1328871 1 2024-03-22T10:43:08.971 INFO:tasks.workunit.client.1.smithi150.stdout:1/857: write d1/d19/dd5/f111 [980467,49651] 0 2024-03-22T10:43:08.973 INFO:tasks.workunit.client.1.smithi150.stdout:0/893: truncate d0/d1/d84/d86/d42/d10/d38/d52/d68/fb0 3140069 0 2024-03-22T10:43:08.977 INFO:tasks.workunit.client.1.smithi150.stdout:6/877: rename db/dd/d25 to db/d52/d9f/dc6/d133 0 2024-03-22T10:43:08.986 INFO:tasks.workunit.client.1.smithi150.stdout:9/924: truncate d1/d8/de/d73/d101/d29/f9b 3262205 0 2024-03-22T10:43:08.990 INFO:tasks.workunit.client.1.smithi150.stdout:4/945: dwrite d0/dce/d10e/f111 [0,4194304] 0 2024-03-22T10:43:08.993 INFO:tasks.workunit.client.1.smithi150.stdout:5/927: symlink d0/dae/db2/df9/l140 0 2024-03-22T10:43:08.993 INFO:tasks.workunit.client.1.smithi150.stdout:8/921: mknod d0/d2/c130 0 2024-03-22T10:43:08.995 INFO:tasks.workunit.client.1.smithi150.stdout:0/894: dread - d0/d1/d84/d86/d42/d10/d38/d52/d68/fe1 zero size 2024-03-22T10:43:09.004 INFO:tasks.workunit.client.1.smithi150.stdout:2/994: rename d12/d19/db6/da9/d67/d3c/c52 to d12/d19/db6/da9/d67/d8d/c137 0 2024-03-22T10:43:09.005 INFO:tasks.workunit.client.1.smithi150.stdout:3/895: dread d5/d22/d2f/d38/d59/d88/d127/f12d [4194304,4194304] 0 2024-03-22T10:43:09.008 INFO:tasks.workunit.client.1.smithi150.stdout:7/902: write d2/d5/d7/d8/d57/d4a/f63 [4313650,123605] 0 2024-03-22T10:43:09.009 INFO:tasks.workunit.client.1.smithi150.stdout:6/878: mkdir db/dd/d15/d28/d6c/d134 0 2024-03-22T10:43:09.013 INFO:tasks.workunit.client.1.smithi150.stdout:5/928: mknod d0/df6/c141 0 2024-03-22T10:43:09.016 INFO:tasks.workunit.client.1.smithi150.stdout:0/895: symlink d0/d12/d101/l11e 0 2024-03-22T10:43:09.022 INFO:tasks.workunit.client.1.smithi150.stdout:8/922: creat d0/d2/d19/d29/d37/dc2/d107/f131 x:0 0 0 2024-03-22T10:43:09.024 INFO:tasks.workunit.client.1.smithi150.stdout:9/925: mkdir d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/d133 0 2024-03-22T10:43:09.028 INFO:tasks.workunit.client.1.smithi150.stdout:8/923: chown d0/d3c/d71/d7a/d89/db5/lca 21607 1 2024-03-22T10:43:09.032 INFO:tasks.workunit.client.1.smithi150.stdout:1/858: rename d1/d19/d50/d6e/d11a to d1/da/d2a/d65/d78/dfd/d11c 0 2024-03-22T10:43:09.042 INFO:tasks.workunit.client.1.smithi150.stdout:6/879: write db/dd/d15/d60/d50/dc1/fba [5065304,12277] 0 2024-03-22T10:43:09.044 INFO:tasks.workunit.client.1.smithi150.stdout:4/946: dread d0/d2f/d10d/d4d/d6c/d61/d138/fab [0,4194304] 0 2024-03-22T10:43:09.044 INFO:tasks.workunit.client.1.smithi150.stdout:7/903: write d2/d5/d7/d9/db/d9f/d96/fe4 [2756584,81926] 0 2024-03-22T10:43:09.054 INFO:tasks.workunit.client.1.smithi150.stdout:1/859: unlink d1/da/d2a/d65/d78/dfd/f107 0 2024-03-22T10:43:09.057 INFO:tasks.workunit.client.1.smithi150.stdout:1/860: write d1/da/d2a/d65/d78/fb7 [79531,85005] 0 2024-03-22T10:43:09.061 INFO:tasks.workunit.client.1.smithi150.stdout:6/880: creat db/d52/f135 x:0 0 0 2024-03-22T10:43:09.061 INFO:tasks.workunit.client.1.smithi150.stdout:0/896: fdatasync d0/f56 0 2024-03-22T10:43:09.063 INFO:tasks.workunit.client.1.smithi150.stdout:4/947: fdatasync d0/de/f4b 0 2024-03-22T10:43:09.063 INFO:tasks.workunit.client.1.smithi150.stdout:9/926: dread d1/d8/de/df/d36/d121/d4e/f93 [0,4194304] 0 2024-03-22T10:43:09.080 INFO:tasks.workunit.client.1.smithi150.stdout:8/924: link d0/d2/d19/d29/d2e/caf d0/d2/d19/d29/d37/d7d/d9e/c132 0 2024-03-22T10:43:09.080 INFO:tasks.workunit.client.1.smithi150.stdout:3/896: dread d5/d22/dc8/dda/f99 [0,4194304] 0 2024-03-22T10:43:09.083 INFO:tasks.workunit.client.1.smithi150.stdout:4/948: rename d0/d2f/d63/d7b/d9c/db1/dd7/feb to d0/dce/d10e/d121/d7c/f13a 0 2024-03-22T10:43:09.090 INFO:tasks.workunit.client.1.smithi150.stdout:1/861: dwrite d1/d19/d71/fe8 [0,4194304] 0 2024-03-22T10:43:09.091 INFO:tasks.workunit.client.1.smithi150.stdout:7/904: getdents d2/d5/d7/d8/d57/d4a/d111 0 2024-03-22T10:43:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.100+0000 7f0ac2000700 1 -- 172.21.15.115:0/3402183700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc071910 msgr2=0x7f0abc071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.100+0000 7f0ac2000700 1 --2- 172.21.15.115:0/3402183700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc071910 0x7f0abc071d20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009910 tx=0x7f0aac009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.100+0000 7f0ac2000700 1 -- 172.21.15.115:0/3402183700 shutdown_connections 2024-03-22T10:43:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.100+0000 7f0ac2000700 1 --2- 172.21.15.115:0/3402183700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0abc0722f0 0x7f0abc10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.100+0000 7f0ac2000700 1 --2- 172.21.15.115:0/3402183700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc071910 0x7f0abc071d20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.100+0000 7f0ac2000700 1 -- 172.21.15.115:0/3402183700 >> 172.21.15.115:0/3402183700 conn(0x7f0abc06d050 msgr2=0x7f0abc06f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:09.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.101+0000 7f0ac2000700 1 -- 172.21.15.115:0/3402183700 shutdown_connections 2024-03-22T10:43:09.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.101+0000 7f0ac2000700 1 -- 172.21.15.115:0/3402183700 wait complete. 2024-03-22T10:43:09.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.102+0000 7f0ac2000700 1 Processor -- start 2024-03-22T10:43:09.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.102+0000 7f0ac2000700 1 -- start start 2024-03-22T10:43:09.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.103+0000 7f0ac2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0abc071910 0x7f0abc1a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:09.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.103+0000 7f0ac2000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc0722f0 0x7f0abc1a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:09.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.103+0000 7f0ac2000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc1a52e0 con 0x7f0abc0722f0 2024-03-22T10:43:09.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.103+0000 7f0ac2000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc1a5420 con 0x7f0abc071910 2024-03-22T10:43:09.098 INFO:tasks.workunit.client.1.smithi150.stdout:0/897: dwrite d0/d1/d84/f10a [0,4194304] 0 2024-03-22T10:43:09.098 INFO:tasks.workunit.client.1.smithi150.stdout:6/881: dwrite db/d52/d9f/dc6/f116 [0,4194304] 0 2024-03-22T10:43:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.103+0000 7f0abb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0abc071910 0x7f0abc1a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.103+0000 7f0abb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0abc071910 0x7f0abc1a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:50026/0 (socket says 172.21.15.115:50026) 2024-03-22T10:43:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.103+0000 7f0abb400700 1 -- 172.21.15.115:0/3303633658 learned_addr learned my addr 172.21.15.115:0/3303633658 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.104+0000 7f0abaa00700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc0722f0 0x7f0abc1a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.104+0000 7f0abb400700 1 -- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc0722f0 msgr2=0x7f0abc1a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.104+0000 7f0abb400700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc0722f0 0x7f0abc1a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.104+0000 7f0abb400700 1 -- 172.21.15.115:0/3303633658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac0095f0 con 0x7f0abc071910 2024-03-22T10:43:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.104+0000 7f0abb400700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0abc071910 0x7f0abc1a4780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00bd70 tx=0x7f0aac00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.108+0000 7f0ab3e00700 1 -- 172.21.15.115:0/3303633658 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aac01d070 con 0x7f0abc071910 2024-03-22T10:43:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.108+0000 7f0ab3e00700 1 -- 172.21.15.115:0/3303633658 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0aac00f460 con 0x7f0abc071910 2024-03-22T10:43:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.108+0000 7f0ab3e00700 1 -- 172.21.15.115:0/3303633658 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aac021620 con 0x7f0abc071910 2024-03-22T10:43:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.108+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc1a9e70 con 0x7f0abc071910 2024-03-22T10:43:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.108+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc1aa2e0 con 0x7f0abc071910 2024-03-22T10:43:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.110+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc19ea40 con 0x7f0abc071910 2024-03-22T10:43:09.104 INFO:tasks.workunit.client.1.smithi150.stdout:3/897: truncate d5/d22/d2f/d38/d59/d88/d10d/d105/f11e 621056 0 2024-03-22T10:43:09.104 INFO:tasks.workunit.client.1.smithi150.stdout:4/949: chown d0/d2f/d10d/d4d/d6c/d61/c45 53634 1 2024-03-22T10:43:09.106 INFO:tasks.workunit.client.1.smithi150.stdout:1/862: fsync d1/da/d15/d18/d4b/f53 0 2024-03-22T10:43:09.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.118+0000 7f0ab3e00700 1 -- 172.21.15.115:0/3303633658 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f0aac00fa40 con 0x7f0abc071910 2024-03-22T10:43:09.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.120+0000 7f0ab3e00700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f0aa006de60 0x7f0aa0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:09.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.120+0000 7f0ab3e00700 1 -- 172.21.15.115:0/3303633658 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f0aac08f140 con 0x7f0abc071910 2024-03-22T10:43:09.113 INFO:tasks.workunit.client.1.smithi150.stdout:8/925: getdents d0/ddd/dfc 0 2024-03-22T10:43:09.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.121+0000 7f0abaa00700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f0aa006de60 0x7f0aa0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:09.114 INFO:tasks.workunit.client.1.smithi150.stdout:0/898: chown d0/d1/d84/d86/d42/c2d 84983756 1 2024-03-22T10:43:09.115 INFO:tasks.workunit.client.1.smithi150.stdout:7/905: symlink d2/d5/d125/l12d 0 2024-03-22T10:43:09.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.123+0000 7f0ab3e00700 1 -- 172.21.15.115:0/3303633658 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f0aac05c450 con 0x7f0abc071910 2024-03-22T10:43:09.116 INFO:tasks.workunit.client.1.smithi150.stdout:9/927: dread d1/d8/de/d73/d101/d29/fb1 [0,4194304] 0 2024-03-22T10:43:09.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.124+0000 7f0abaa00700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f0aa006de60 0x7f0aa0070310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8005950 tx=0x7f0aa80058e0 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:09.117 INFO:tasks.workunit.client.1.smithi150.stdout:0/899: read - d0/d1/d84/d86/da6/dc8/f11d zero size 2024-03-22T10:43:09.118 INFO:tasks.workunit.client.1.smithi150.stdout:3/898: chown d5/da/d62/dba/dd5/fe0 21025 1 2024-03-22T10:43:09.121 INFO:tasks.workunit.client.1.smithi150.stdout:7/906: fdatasync d2/d5/d7/d9/db/d9f/d45/fe3 0 2024-03-22T10:43:09.124 INFO:tasks.workunit.client.1.smithi150.stdout:3/899: write d5/d22/d2f/d38/d3e/d5b/fc7 [713273,45202] 0 2024-03-22T10:43:09.131 INFO:tasks.workunit.client.1.smithi150.stdout:4/950: rename d0/d2f/d63/d7b/d9c/db1/dd7/ff2 to d0/dce/d10e/d121/d7c/dd4/f13b 0 2024-03-22T10:43:09.134 INFO:tasks.workunit.client.1.smithi150.stdout:8/926: symlink d0/d2/d19/d29/d2e/l133 0 2024-03-22T10:43:09.134 INFO:tasks.workunit.client.1.smithi150.stdout:7/907: fdatasync d2/d5/d7/d8/d46/f66 0 2024-03-22T10:43:09.135 INFO:tasks.workunit.client.1.smithi150.stdout:0/900: mknod d0/d1/d84/d86/d11/d26/dd2/dff/c11f 0 2024-03-22T10:43:09.136 INFO:tasks.workunit.client.1.smithi150.stdout:3/900: write d5/d22/d2f/d38/d3e/d5b/f7f [562779,106055] 0 2024-03-22T10:43:09.139 INFO:tasks.workunit.client.1.smithi150.stdout:4/951: creat d0/d2f/d63/d7b/f13c x:0 0 0 2024-03-22T10:43:09.140 INFO:tasks.workunit.client.1.smithi150.stdout:8/927: chown d0/d2/d46/da2/lac 1525 1 2024-03-22T10:43:09.146 INFO:tasks.workunit.client.1.smithi150.stdout:3/901: write d5/d22/d2f/f68 [1110469,114164] 0 2024-03-22T10:43:09.146 INFO:tasks.workunit.client.1.smithi150.stdout:0/901: getdents d0/d1/d84/d86/d11/d66/d35/d92 0 2024-03-22T10:43:09.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:08 smithi150 ceph-mon[34518]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:43:09.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:08 smithi150 ceph-mon[34518]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:43:09.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:08 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:09.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:08 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:09.149 INFO:tasks.workunit.client.1.smithi150.stdout:4/952: creat d0/d47/d72/de7/d10a/f13d x:0 0 0 2024-03-22T10:43:09.154 INFO:tasks.workunit.client.1.smithi150.stdout:7/908: rename d2/d5/d7/d8/d57/d4a/d8d/da9/dcb/c107 to d2/d5/d7/d8/d57/d4a/d8d/da9/c12e 0 2024-03-22T10:43:09.154 INFO:tasks.workunit.client.1.smithi150.stdout:1/863: dread d1/d19/d50/d51/f6b [0,4194304] 0 2024-03-22T10:43:09.154 INFO:tasks.workunit.client.1.smithi150.stdout:0/902: write d0/d1/d84/d86/d42/d10/d38/ff2 [1445721,15751] 0 2024-03-22T10:43:09.159 INFO:tasks.workunit.client.1.smithi150.stdout:1/864: symlink d1/da/d2a/d55/db8/l11d 0 2024-03-22T10:43:09.160 INFO:tasks.workunit.client.1.smithi150.stdout:8/928: write d0/d2/d19/d29/f63 [4611722,105955] 0 2024-03-22T10:43:09.162 INFO:tasks.workunit.client.1.smithi150.stdout:0/903: write d0/d1/d84/d86/d42/d10/d38/ff2 [1292871,46050] 0 2024-03-22T10:43:09.165 INFO:tasks.workunit.client.1.smithi150.stdout:7/909: getdents d2/d5/d7/d9/db/d6c/d9c 0 2024-03-22T10:43:09.166 INFO:tasks.workunit.client.1.smithi150.stdout:0/904: mknod d0/d12/d101/c120 0 2024-03-22T10:43:09.167 INFO:tasks.workunit.client.1.smithi150.stdout:1/865: mkdir d1/da/d2a/d65/d78/dfd/d11c/d11e 0 2024-03-22T10:43:09.168 INFO:tasks.workunit.client.1.smithi150.stdout:9/928: dread d1/d8/de/df/d36/d121/d4e/f77 [0,4194304] 0 2024-03-22T10:43:09.170 INFO:tasks.workunit.client.1.smithi150.stdout:7/910: rmdir d2/d5/d7/d38 39 2024-03-22T10:43:09.174 INFO:tasks.workunit.client.1.smithi150.stdout:9/929: mkdir d1/d19/d7d/dc1/d134 0 2024-03-22T10:43:09.177 INFO:tasks.workunit.client.1.smithi150.stdout:7/911: symlink d2/d5/d125/l12f 0 2024-03-22T10:43:09.180 INFO:tasks.workunit.client.1.smithi150.stdout:3/902: truncate d5/d22/d2f/d38/d3e/f4d 1356973 0 2024-03-22T10:43:09.180 INFO:tasks.workunit.client.1.smithi150.stdout:7/912: chown d2/d5/d7/d9/db/d9f/d45/f72 247344118 1 2024-03-22T10:43:09.182 INFO:tasks.workunit.client.1.smithi150.stdout:1/866: creat d1/da/d15/f11f x:0 0 0 2024-03-22T10:43:09.184 INFO:tasks.workunit.client.1.smithi150.stdout:4/953: dread d0/dce/d10e/d121/fae [4194304,4194304] 0 2024-03-22T10:43:09.186 INFO:tasks.workunit.client.1.smithi150.stdout:3/903: dread d5/fcf [0,4194304] 0 2024-03-22T10:43:09.189 INFO:tasks.workunit.client.1.smithi150.stdout:0/905: truncate d0/f7b 8922935 0 2024-03-22T10:43:09.192 INFO:tasks.workunit.client.1.smithi150.stdout:1/867: dread d1/da/d15/ff9 [0,4194304] 0 2024-03-22T10:43:09.193 INFO:tasks.workunit.client.1.smithi150.stdout:4/954: creat d0/d2f/d63/d7b/d9c/db1/f13e x:0 0 0 2024-03-22T10:43:09.194 INFO:tasks.workunit.client.1.smithi150.stdout:1/868: chown d1/d19/d1a/d7d 215571732 1 2024-03-22T10:43:09.194 INFO:tasks.workunit.client.1.smithi150.stdout:3/904: mkdir d5/d22/d37/d131 0 2024-03-22T10:43:09.195 INFO:tasks.workunit.client.1.smithi150.stdout:7/913: link d2/d5/l87 d2/d5/d7/d127/l130 0 2024-03-22T10:43:09.195 INFO:tasks.workunit.client.1.smithi150.stdout:1/869: chown d1/da/d15/d18/d4b/d8a 8 1 2024-03-22T10:43:09.196 INFO:tasks.workunit.client.1.smithi150.stdout:0/906: creat d0/d1/d84/f121 x:0 0 0 2024-03-22T10:43:09.197 INFO:tasks.workunit.client.1.smithi150.stdout:9/930: dread d1/d8/de/df/d36/d40/d44/db5/dda/d104/f84 [0,4194304] 0 2024-03-22T10:43:09.198 INFO:tasks.workunit.client.1.smithi150.stdout:3/905: write d5/d22/d37/fd8 [138261,70917] 0 2024-03-22T10:43:09.199 INFO:tasks.workunit.client.1.smithi150.stdout:1/870: creat d1/d19/d50/f120 x:0 0 0 2024-03-22T10:43:09.202 INFO:tasks.workunit.client.1.smithi150.stdout:7/914: fdatasync d2/d5/d7/d9/f53 0 2024-03-22T10:43:09.202 INFO:tasks.workunit.client.1.smithi150.stdout:0/907: rmdir d0/d1/d84/d86/d42/d10/d38/d52/d9b/dc2 39 2024-03-22T10:43:09.203 INFO:tasks.workunit.client.1.smithi150.stdout:9/931: chown d1/d8/de/df/d36/f5c 543867 1 2024-03-22T10:43:09.208 INFO:tasks.workunit.client.1.smithi150.stdout:7/915: truncate d2/d5/d7/d88/f10c 997430 0 2024-03-22T10:43:09.208 INFO:tasks.workunit.client.1.smithi150.stdout:4/955: link d0/d2f/d10d/f74 d0/d2f/d10d/d4d/d6c/d61/dd1/f13f 0 2024-03-22T10:43:09.213 INFO:tasks.workunit.client.1.smithi150.stdout:0/908: symlink d0/d1/d84/l122 0 2024-03-22T10:43:09.223 INFO:tasks.workunit.client.1.smithi150.stdout:7/916: rename d2/d5/f7c to d2/d5/d7/d38/f131 0 2024-03-22T10:43:09.225 INFO:tasks.workunit.client.1.smithi150.stdout:9/932: link d1/d8/de/df/d36/fcb d1/d19/d7d/dc1/dca/dd7/df0/d12c/f135 0 2024-03-22T10:43:09.228 INFO:tasks.workunit.client.1.smithi150.stdout:9/933: chown d1/d8/de/df/d36/d121 241 1 2024-03-22T10:43:09.229 INFO:tasks.workunit.client.1.smithi150.stdout:0/909: getdents d0/d1/d84/d86/d5a 0 2024-03-22T10:43:09.234 INFO:tasks.workunit.client.1.smithi150.stdout:7/917: mknod d2/d5/d7/d8/d46/c132 0 2024-03-22T10:43:09.242 INFO:tasks.workunit.client.1.smithi150.stdout:3/906: dread d5/d22/d2f/d38/d3e/f7d [0,4194304] 0 2024-03-22T10:43:09.242 INFO:tasks.workunit.client.1.smithi150.stdout:4/956: dwrite d0/d2f/d10d/d4d/d6c/d61/dd1/f11a [0,4194304] 0 2024-03-22T10:43:09.243 INFO:tasks.workunit.client.1.smithi150.stdout:1/871: dread d1/d19/d50/d6e/f88 [0,4194304] 0 2024-03-22T10:43:09.246 INFO:tasks.workunit.client.1.smithi150.stdout:4/957: mkdir d0/d47/d140 0 2024-03-22T10:43:09.251 INFO:tasks.workunit.client.1.smithi150.stdout:4/958: write d0/d2f/d10d/d4d/d6c/d61/d71/fa4 [3594930,89422] 0 2024-03-22T10:43:09.261 INFO:tasks.workunit.client.1.smithi150.stdout:4/959: link d0/d47/f11c d0/d2f/d63/d7b/d9c/db1/f141 0 2024-03-22T10:43:09.262 INFO:tasks.workunit.client.1.smithi150.stdout:1/872: dread d1/d19/d50/d6e/db6/f104 [0,4194304] 0 2024-03-22T10:43:09.264 INFO:tasks.workunit.client.1.smithi150.stdout:7/918: dwrite d2/d5/d7/d8/d57/d4a/d111/f116 [0,4194304] 0 2024-03-22T10:43:09.266 INFO:tasks.workunit.client.1.smithi150.stdout:4/960: mknod d0/d2f/d63/d7b/d9c/db1/dd7/c142 0 2024-03-22T10:43:09.267 INFO:tasks.workunit.client.1.smithi150.stdout:7/919: chown d2/cc 90420 1 2024-03-22T10:43:09.276 INFO:tasks.workunit.client.1.smithi150.stdout:7/920: creat d2/d124/f133 x:0 0 0 2024-03-22T10:43:09.276 INFO:tasks.workunit.client.1.smithi150.stdout:1/873: rename d1/d19/d50/d51/da0/dcf/dd6 to d1/da/d15/d18/d121 0 2024-03-22T10:43:09.279 INFO:tasks.workunit.client.1.smithi150.stdout:1/874: chown d1/d19/d50 20822103 1 2024-03-22T10:43:09.281 INFO:tasks.workunit.client.1.smithi150.stdout:0/910: dread d0/d1/f39 [0,4194304] 0 2024-03-22T10:43:09.282 INFO:tasks.workunit.client.1.smithi150.stdout:3/907: dread d5/f10 [0,4194304] 0 2024-03-22T10:43:09.283 INFO:tasks.workunit.client.1.smithi150.stdout:4/961: rename d0/f8 to d0/d2f/d10d/d4d/d6c/d61/d98/f143 0 2024-03-22T10:43:09.289 INFO:tasks.workunit.client.1.smithi150.stdout:1/875: read d1/d19/d50/d6e/fc6 [113559,127946] 0 2024-03-22T10:43:09.290 INFO:tasks.workunit.client.1.smithi150.stdout:3/908: mknod d5/d22/d2f/d38/d3e/d5b/dee/c132 0 2024-03-22T10:43:09.291 INFO:tasks.workunit.client.1.smithi150.stdout:7/921: rmdir d2/d5/d7/d8/d57/d4a/d8d/da9/d122 0 2024-03-22T10:43:09.293 INFO:tasks.workunit.client.1.smithi150.stdout:7/922: dread - d2/d124/f133 zero size 2024-03-22T10:43:09.295 INFO:tasks.workunit.client.1.smithi150.stdout:4/962: creat d0/de/f144 x:0 0 0 2024-03-22T10:43:09.300 INFO:tasks.workunit.client.1.smithi150.stdout:7/923: mknod d2/d5/d7/d8/d10b/c134 0 2024-03-22T10:43:09.304 INFO:tasks.workunit.client.1.smithi150.stdout:0/911: dread d0/d1/d84/d86/d11/fc4 [0,4194304] 0 2024-03-22T10:43:09.315 INFO:tasks.workunit.client.1.smithi150.stdout:4/963: dwrite d0/d2f/d10d/d4d/d6c/d61/dd1/fec [0,4194304] 0 2024-03-22T10:43:09.320 INFO:tasks.workunit.client.1.smithi150.stdout:7/924: rmdir d2/d5/d7/d9/db/d9f/d117 0 2024-03-22T10:43:09.320 INFO:tasks.workunit.client.1.smithi150.stdout:0/912: dread - d0/d1/d84/d86/d42/d10/d38/d52/d9b/dac/feb zero size 2024-03-22T10:43:09.332 INFO:tasks.workunit.client.1.smithi150.stdout:0/913: mknod d0/d12/dd6/c123 0 2024-03-22T10:43:09.332 INFO:tasks.workunit.client.1.smithi150.stdout:1/876: dread d1/f73 [0,4194304] 0 2024-03-22T10:43:09.332 INFO:tasks.workunit.client.1.smithi150.stdout:0/914: getdents d0 0 2024-03-22T10:43:09.344 INFO:tasks.workunit.client.1.smithi150.stdout:4/964: dwrite d0/d2f/d10d/d4d/d9f/fef [0,4194304] 0 2024-03-22T10:43:09.346 INFO:tasks.workunit.client.1.smithi150.stdout:7/925: dwrite d2/d5/d7/d9/db/da2/fbe [0,4194304] 0 2024-03-22T10:43:09.353 INFO:tasks.workunit.client.1.smithi150.stdout:1/877: dwrite d1/d19/d1a/f49 [4194304,4194304] 0 2024-03-22T10:43:09.353 INFO:tasks.workunit.client.1.smithi150.stdout:4/965: creat d0/dce/d10e/d121/d7c/dc7/f145 x:0 0 0 2024-03-22T10:43:09.354 INFO:tasks.workunit.client.1.smithi150.stdout:4/966: readlink d0/d2f/d10d/l36 0 2024-03-22T10:43:09.358 INFO:tasks.workunit.client.1.smithi150.stdout:1/878: truncate d1/da/f67 355338 0 2024-03-22T10:43:09.358 INFO:tasks.workunit.client.1.smithi150.stdout:4/967: read d0/d2f/d10d/d4d/d6c/d61/d138/fab [1395901,53884] 0 2024-03-22T10:43:09.362 INFO:tasks.workunit.client.1.smithi150.stdout:1/879: unlink d1/lbb 0 2024-03-22T10:43:09.366 INFO:tasks.workunit.client.1.smithi150.stdout:1/880: rmdir d1/d19/d1a 39 2024-03-22T10:43:09.372 INFO:tasks.workunit.client.1.smithi150.stdout:1/881: symlink d1/da/d2a/d65/d78/l122 0 2024-03-22T10:43:09.373 INFO:tasks.workunit.client.1.smithi150.stdout:4/968: dread d0/d2f/d10d/fac [0,4194304] 0 2024-03-22T10:43:09.377 INFO:tasks.workunit.client.1.smithi150.stdout:1/882: rename d1/da/d15/d4f/dc9 to d1/d19/d50/d51/d123 0 2024-03-22T10:43:09.377 INFO:tasks.workunit.client.1.smithi150.stdout:4/969: creat d0/d47/d72/de7/d10a/f146 x:0 0 0 2024-03-22T10:43:09.378 INFO:tasks.workunit.client.1.smithi150.stdout:1/883: fsync d1/d19/d50/d51/da0/fee 0 2024-03-22T10:43:09.386 INFO:tasks.workunit.client.1.smithi150.stdout:4/970: rename d0/dce/d10e/d121/l3f to d0/d2f/d63/d7b/d9c/db1/dd7/l147 0 2024-03-22T10:43:09.387 INFO:tasks.workunit.client.1.smithi150.stdout:1/884: getdents d1/da/d2a/d55 0 2024-03-22T10:43:09.389 INFO:tasks.workunit.client.1.smithi150.stdout:0/915: dread d0/d1/d84/d86/d42/d10/d53/f57 [4194304,4194304] 0 2024-03-22T10:43:09.391 INFO:tasks.workunit.client.1.smithi150.stdout:1/885: dread d1/d19/fbc [0,4194304] 0 2024-03-22T10:43:09.392 INFO:tasks.workunit.client.1.smithi150.stdout:4/971: link d0/d47/d72/db6/cb7 d0/dce/d10e/d121/c148 0 2024-03-22T10:43:09.397 INFO:tasks.workunit.client.1.smithi150.stdout:0/916: rename d0/d1/d84/d86/d11/d66/d35/d5f/d116 to d0/d1/d84/d86/d5a/d124 0 2024-03-22T10:43:09.400 INFO:tasks.workunit.client.1.smithi150.stdout:0/917: creat d0/d1/d84/d86/d11/d26/f125 x:0 0 0 2024-03-22T10:43:09.403 INFO:tasks.workunit.client.1.smithi150.stdout:0/918: dread - d0/d1/d84/d86/d11/d66/d35/d92/f106 zero size 2024-03-22T10:43:09.456 INFO:tasks.workunit.client.1.smithi150.stdout:2/995: sync 2024-03-22T10:43:09.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.491+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0abc061190 con 0x7f0aa006de60 2024-03-22T10:43:09.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.493+0000 7f0ab3e00700 1 -- 172.21.15.115:0/3303633658 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f0abc061190 con 0x7f0aa006de60 2024-03-22T10:43:09.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f0aa006de60 msgr2=0x7f0aa0070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:09.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f0aa006de60 0x7f0aa0070310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8005950 tx=0x7f0aa80058e0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0abc071910 msgr2=0x7f0abc1a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:09.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0abc071910 0x7f0abc1a4780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00bd70 tx=0x7f0aac00be50 comp rx=0 tx=0).stop 2024-03-22T10:43:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 shutdown_connections 2024-03-22T10:43:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f0aa006de60 0x7f0aa0070310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0abc071910 0x7f0abc1a4780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 --2- 172.21.15.115:0/3303633658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc0722f0 0x7f0abc1a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 >> 172.21.15.115:0/3303633658 conn(0x7f0abc06d050 msgr2=0x7f0abc10b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 shutdown_connections 2024-03-22T10:43:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.496+0000 7f0ac2000700 1 -- 172.21.15.115:0/3303633658 wait complete. 2024-03-22T10:43:09.493 INFO:tasks.workunit.client.1.smithi150.stdout:4/972: dwrite d0/d2f/d10d/d4d/d6c/f113 [0,4194304] 0 2024-03-22T10:43:09.494 INFO:tasks.workunit.client.1.smithi150.stdout:1/886: dread d1/da/d2a/f35 [0,4194304] 0 2024-03-22T10:43:09.495 INFO:tasks.workunit.client.1.smithi150.stdout:0/919: truncate d0/d1/d84/d86/d11/d26/dd2/df0/f10c 965772 0 2024-03-22T10:43:09.506 INFO:tasks.workunit.client.1.smithi150.stdout:1/887: mknod d1/d19/d1a/df3/c124 0 2024-03-22T10:43:09.509 INFO:tasks.workunit.client.1.smithi150.stdout:1/888: fsync d1/d19/d1a/f2b 0 2024-03-22T10:43:09.513 INFO:tasks.workunit.client.1.smithi150.stdout:2/996: dwrite d12/d19/d2d/faa [0,4194304] 0 2024-03-22T10:43:09.514 INFO:tasks.workunit.client.1.smithi150.stdout:1/889: fdatasync d1/f73 0 2024-03-22T10:43:09.519 INFO:tasks.workunit.client.1.smithi150.stdout:4/973: dwrite d0/d2f/d10d/d4d/d6c/d61/f48 [0,4194304] 0 2024-03-22T10:43:09.519 INFO:tasks.workunit.client.1.smithi150.stdout:1/890: chown d1/d19/d50/d51/l9f 2863863 1 2024-03-22T10:43:09.531 INFO:tasks.workunit.client.1.smithi150.stdout:0/920: dwrite d0/d1/d84/d86/d42/d10/d38/fc3 [0,4194304] 0 2024-03-22T10:43:09.532 INFO:tasks.workunit.client.1.smithi150.stdout:2/997: creat d12/d19/db6/da9/d61/d98/f138 x:0 0 0 2024-03-22T10:43:09.535 INFO:tasks.workunit.client.1.smithi150.stdout:4/974: chown d0/d47/l12f 8118080 1 2024-03-22T10:43:09.546 INFO:tasks.workunit.client.1.smithi150.stdout:0/921: dread d0/fc7 [0,4194304] 0 2024-03-22T10:43:09.547 INFO:tasks.workunit.client.1.smithi150.stdout:0/922: chown d0/d12 11188676 1 2024-03-22T10:43:09.555 INFO:tasks.workunit.client.1.smithi150.stdout:1/891: mkdir d1/da/d2a/d65/dbd/d125 0 2024-03-22T10:43:09.556 INFO:tasks.workunit.client.1.smithi150.stdout:4/975: link d0/d2f/d63/f112 d0/d2f/d10d/d68/f149 0 2024-03-22T10:43:09.560 INFO:tasks.workunit.client.1.smithi150.stdout:4/976: read - d0/dce/d10e/d121/d7c/f82 zero size 2024-03-22T10:43:09.564 INFO:tasks.workunit.client.1.smithi150.stdout:4/977: rename d0/dce/d10e/d121/fae to d0/d47/d140/f14a 0 2024-03-22T10:43:09.574 INFO:tasks.workunit.client.1.smithi150.stdout:2/998: dread d12/d19/db6/da9/d67/f51 [0,4194304] 0 2024-03-22T10:43:09.575 INFO:tasks.workunit.client.1.smithi150.stdout:1/892: dwrite d1/f7 [0,4194304] 0 2024-03-22T10:43:09.578 INFO:tasks.workunit.client.1.smithi150.stdout:4/978: rename d0/d2f/d10d/c20 to d0/d47/d72/de7/d10a/c14b 0 2024-03-22T10:43:09.580 INFO:tasks.workunit.client.1.smithi150.stdout:4/979: chown d0/d2f/d63 225309760 1 2024-03-22T10:43:09.581 INFO:tasks.workunit.client.1.smithi150.stdout:1/893: mknod d1/da/d15/d18/c126 0 2024-03-22T10:43:09.588 INFO:tasks.workunit.client.1.smithi150.stdout:1/894: rmdir d1/da/d15/d18/d4b 39 2024-03-22T10:43:09.591 INFO:tasks.workunit.client.1.smithi150.stdout:4/980: creat d0/d2f/d63/d7b/f14c x:0 0 0 2024-03-22T10:43:09.593 INFO:tasks.workunit.client.1.smithi150.stdout:1/895: creat d1/da/d2a/d55/d97/f127 x:0 0 0 2024-03-22T10:43:09.599 INFO:tasks.workunit.client.1.smithi150.stdout:1/896: write d1/d19/d1a/dba/feb [658885,75706] 0 2024-03-22T10:43:09.601 INFO:tasks.workunit.client.1.smithi150.stdout:2/999: dwrite d12/d19/db6/da9/d67/d3c/f6b [0,4194304] 0 2024-03-22T10:43:09.608 INFO:tasks.workunit.client.1.smithi150.stdout:1/897: mkdir d1/da/d2a/d55/d97/d128 0 2024-03-22T10:43:09.616 INFO:tasks.workunit.client.1.smithi150.stdout:0/923: dwrite d0/d1/d84/d86/d42/f77 [0,4194304] 0 2024-03-22T10:43:09.630 INFO:tasks.workunit.client.1.smithi150.stdout:0/924: write d0/d1/d84/d86/d42/fe3 [226968,92759] 0 2024-03-22T10:43:09.634 INFO:tasks.workunit.client.1.smithi150.stdout:4/981: dwrite d0/dce/d10e/d121/fba [0,4194304] 0 2024-03-22T10:43:09.637 INFO:tasks.workunit.client.1.smithi150.stdout:4/982: chown d0/d2f/d63/d7b/d9c/db1/f11f 144 1 2024-03-22T10:43:09.639 INFO:tasks.workunit.client.1.smithi150.stdout:1/898: dwrite d1/f73 [0,4194304] 0 2024-03-22T10:43:09.640 INFO:tasks.workunit.client.1.smithi150.stdout:0/925: truncate d0/d1/d84/d86/d42/d10/f43 4816784 0 2024-03-22T10:43:09.641 INFO:tasks.workunit.client.1.smithi150.stdout:8/929: sync 2024-03-22T10:43:09.642 INFO:tasks.workunit.client.1.smithi150.stdout:9/934: sync 2024-03-22T10:43:09.642 INFO:tasks.workunit.client.1.smithi150.stdout:6/882: sync 2024-03-22T10:43:09.646 INFO:tasks.workunit.client.1.smithi150.stdout:5/929: sync 2024-03-22T10:43:09.648 INFO:tasks.workunit.client.1.smithi150.stdout:0/926: chown d0/d1/d84/d86/d42/d10/d38/f4c 4733980 1 2024-03-22T10:43:09.655 INFO:tasks.workunit.client.1.smithi150.stdout:6/883: truncate db/dd/ffe 3190634 0 2024-03-22T10:43:09.658 INFO:tasks.workunit.client.1.smithi150.stdout:6/884: chown db/d52/d79/ff3 26 1 2024-03-22T10:43:09.658 INFO:tasks.workunit.client.1.smithi150.stdout:9/935: rename d1/d8/de/df/d36/d40/d44/db5/fd9 to d1/d8/de/dbd/df8/f136 0 2024-03-22T10:43:09.659 INFO:tasks.workunit.client.1.smithi150.stdout:6/885: chown db/d52/f93 66 1 2024-03-22T10:43:09.660 INFO:tasks.workunit.client.1.smithi150.stdout:5/930: creat d0/d8/dc/de/d14/d2e/d45/d102/d11c/f142 x:0 0 0 2024-03-22T10:43:09.662 INFO:tasks.workunit.client.1.smithi150.stdout:9/936: rmdir d1/d8/de/df/da2 39 2024-03-22T10:43:09.668 INFO:tasks.workunit.client.1.smithi150.stdout:9/937: chown d1/d8/de/df/d36/fcb 11900 1 2024-03-22T10:43:09.674 INFO:tasks.workunit.client.1.smithi150.stdout:5/931: getdents d0/d8/dc/de/d14/d2e/d46/d9c/dc0 0 2024-03-22T10:43:09.677 INFO:tasks.workunit.client.1.smithi150.stdout:5/932: fsync d0/d8/dc/de/d14/d2e/d45/f87 0 2024-03-22T10:43:09.680 INFO:tasks.workunit.client.1.smithi150.stdout:9/938: rename d1/d8/de/df/d36/d121/d4e/f77 to d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/f137 0 2024-03-22T10:43:09.680 INFO:tasks.workunit.client.1.smithi150.stdout:4/983: write d0/dce/d10e/d121/f41 [792083,94395] 0 2024-03-22T10:43:09.683 INFO:tasks.workunit.client.1.smithi150.stdout:5/933: truncate d0/f22 1687555 0 2024-03-22T10:43:09.686 INFO:tasks.workunit.client.1.smithi150.stdout:0/927: truncate d0/f85 2553473 0 2024-03-22T10:43:09.691 INFO:tasks.workunit.client.1.smithi150.stdout:0/928: readlink d0/d1/d84/d86/d42/d10/d38/dba/l110 0 2024-03-22T10:43:09.696 INFO:tasks.workunit.client.1.smithi150.stdout:9/939: creat d1/d8/de/df/d36/d121/d4e/de9/f138 x:0 0 0 2024-03-22T10:43:09.696 INFO:tasks.workunit.client.1.smithi150.stdout:5/934: rmdir d0/d8/dc/de/d14/d2e/dbc 39 2024-03-22T10:43:09.699 INFO:tasks.workunit.client.1.smithi150.stdout:8/930: dwrite d0/d2/d19/d29/d88/f9a [0,4194304] 0 2024-03-22T10:43:09.700 INFO:tasks.workunit.client.1.smithi150.stdout:5/935: stat d0/d8/f23 0 2024-03-22T10:43:09.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.715+0000 7f334c800700 1 -- 172.21.15.115:0/4140584477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440fe9d0 msgr2=0x7f33440fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:09.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.715+0000 7f334c800700 1 --2- 172.21.15.115:0/4140584477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440fe9d0 0x7f33440fede0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3338009910 tx=0x7f3338009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:09.709 INFO:tasks.workunit.client.1.smithi150.stdout:9/940: stat d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7 0 2024-03-22T10:43:09.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.716+0000 7f334c800700 1 -- 172.21.15.115:0/4140584477 shutdown_connections 2024-03-22T10:43:09.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.716+0000 7f334c800700 1 --2- 172.21.15.115:0/4140584477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33440ff320 0x7f33440ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.716+0000 7f334c800700 1 --2- 172.21.15.115:0/4140584477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440fe9d0 0x7f33440fede0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.716+0000 7f334c800700 1 -- 172.21.15.115:0/4140584477 >> 172.21.15.115:0/4140584477 conn(0x7f33440fa5e0 msgr2=0x7f33440fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:09.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.717+0000 7f334c800700 1 -- 172.21.15.115:0/4140584477 shutdown_connections 2024-03-22T10:43:09.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.717+0000 7f334c800700 1 -- 172.21.15.115:0/4140584477 wait complete. 2024-03-22T10:43:09.711 INFO:tasks.workunit.client.1.smithi150.stdout:6/886: dwrite db/dd/f33 [4194304,4194304] 0 2024-03-22T10:43:09.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.719+0000 7f334c800700 1 Processor -- start 2024-03-22T10:43:09.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.719+0000 7f334c800700 1 -- start start 2024-03-22T10:43:09.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.719+0000 7f334c800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33440fe9d0 0x7f3344197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:09.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.719+0000 7f334c800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440ff320 0x7f33441980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:09.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.719+0000 7f334c800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3344198710 con 0x7f33440ff320 2024-03-22T10:43:09.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.719+0000 7f334c800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3344198850 con 0x7f33440fe9d0 2024-03-22T10:43:09.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.720+0000 7f3349400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440ff320 0x7f33441980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:09.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.720+0000 7f3349400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440ff320 0x7f33441980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53254/0 (socket says 172.21.15.115:53254) 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.720+0000 7f3349400700 1 -- 172.21.15.115:0/1042084304 learned_addr learned my addr 172.21.15.115:0/1042084304 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.720+0000 7f3349e00700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33440fe9d0 0x7f3344197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.720+0000 7f3349400700 1 -- 172.21.15.115:0/1042084304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33440fe9d0 msgr2=0x7f3344197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.720+0000 7f3349400700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33440fe9d0 0x7f3344197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.720+0000 7f3349400700 1 -- 172.21.15.115:0/1042084304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33380095f0 con 0x7f33440ff320 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.721+0000 7f3349400700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440ff320 0x7f33441980f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f333000b4b0 tx=0x7f333000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.721+0000 7f333ea00700 1 -- 172.21.15.115:0/1042084304 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3330011800 con 0x7f33440ff320 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.721+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3344100c40 con 0x7f33440ff320 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.721+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3344101210 con 0x7f33440ff320 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.722+0000 7f333ea00700 1 -- 172.21.15.115:0/1042084304 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3330011e10 con 0x7f33440ff320 2024-03-22T10:43:09.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.722+0000 7f333ea00700 1 -- 172.21.15.115:0/1042084304 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333000f2e0 con 0x7f33440ff320 2024-03-22T10:43:09.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.724+0000 7f333ea00700 1 -- 172.21.15.115:0/1042084304 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f333000f440 con 0x7f33440ff320 2024-03-22T10:43:09.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.725+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3324005320 con 0x7f33440ff320 2024-03-22T10:43:09.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.726+0000 7f333ea00700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f332c06dda0 0x7f332c070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:09.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.726+0000 7f333ea00700 1 -- 172.21.15.115:0/1042084304 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f333009f600 con 0x7f33440ff320 2024-03-22T10:43:09.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.728+0000 7f3349e00700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f332c06dda0 0x7f332c070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:09.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.728+0000 7f3349e00700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f332c06dda0 0x7f332c070250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3338006010 tx=0x7f333800b560 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:09.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:09.733+0000 7f333ea00700 1 -- 172.21.15.115:0/1042084304 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f333006c9a0 con 0x7f33440ff320 2024-03-22T10:43:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:09 smithi115 ceph-mon[29111]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:43:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:09 smithi115 ceph-mon[29111]: pgmap v23: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 73 MiB/s rd, 183 MiB/s wr, 405 op/s 2024-03-22T10:43:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:09 smithi115 ceph-mon[29111]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:09 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:09 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:09 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:09.736 INFO:tasks.workunit.client.1.smithi150.stdout:9/941: mkdir d1/d19/d7d/dc1/dca/dd7/df0/d139 0 2024-03-22T10:43:09.736 INFO:tasks.workunit.client.1.smithi150.stdout:8/931: mknod d0/d2/d19/d29/d37/d7d/c134 0 2024-03-22T10:43:09.736 INFO:tasks.workunit.client.1.smithi150.stdout:4/984: write d0/d2f/d10d/d4d/d6c/d61/d98/f143 [44922,78007] 0 2024-03-22T10:43:09.738 INFO:tasks.workunit.client.1.smithi150.stdout:4/985: fsync d0/dce/d10e/d121/d7c/f91 0 2024-03-22T10:43:09.747 INFO:tasks.workunit.client.1.smithi150.stdout:5/936: dwrite d0/d8/f34 [0,4194304] 0 2024-03-22T10:43:09.753 INFO:tasks.workunit.client.1.smithi150.stdout:8/932: mkdir d0/d2/d19/d29/d2e/d135 0 2024-03-22T10:43:09.756 INFO:tasks.workunit.client.1.smithi150.stdout:9/942: rename d1/d8/fa1 to d1/d8/de/dbd/ddc/f13a 0 2024-03-22T10:43:09.756 INFO:tasks.workunit.client.1.smithi150.stdout:5/937: mknod d0/dae/c143 0 2024-03-22T10:43:09.759 INFO:tasks.workunit.client.1.smithi150.stdout:5/938: read d0/d8/f34 [2756112,58425] 0 2024-03-22T10:43:09.763 INFO:tasks.workunit.client.1.smithi150.stdout:6/887: link db/dd/d15/d28/l68 db/d52/de6/l136 0 2024-03-22T10:43:09.768 INFO:tasks.workunit.client.1.smithi150.stdout:0/929: truncate d0/d1/d84/d86/d42/d10/d38/fc3 2535692 0 2024-03-22T10:43:09.770 INFO:tasks.workunit.client.1.smithi150.stdout:4/986: mknod d0/d2f/d63/d7b/c14d 0 2024-03-22T10:43:09.774 INFO:tasks.workunit.client.1.smithi150.stdout:9/943: mknod d1/d8/de/c13b 0 2024-03-22T10:43:09.780 INFO:tasks.workunit.client.1.smithi150.stdout:5/939: creat d0/d8/dc/de/d14/d4f/d134/f144 x:0 0 0 2024-03-22T10:43:09.783 INFO:tasks.workunit.client.1.smithi150.stdout:8/933: dwrite d0/d5d/dd2/fb4 [0,4194304] 0 2024-03-22T10:43:09.786 INFO:tasks.workunit.client.1.smithi150.stdout:6/888: rename db/dd/d15/d30/f4b to db/d52/de6/f137 0 2024-03-22T10:43:09.787 INFO:tasks.workunit.client.1.smithi150.stdout:6/889: chown db/f10e 11 1 2024-03-22T10:43:09.789 INFO:tasks.workunit.client.1.smithi150.stdout:5/940: creat d0/dae/de0/f145 x:0 0 0 2024-03-22T10:43:09.791 INFO:tasks.workunit.client.1.smithi150.stdout:6/890: write db/f10e [867083,20620] 0 2024-03-22T10:43:09.801 INFO:tasks.workunit.client.1.smithi150.stdout:5/941: mknod d0/d8/dc/d1d/d53/c146 0 2024-03-22T10:43:09.801 INFO:tasks.workunit.client.1.smithi150.stdout:0/930: dwrite d0/d12/d101/f119 [0,4194304] 0 2024-03-22T10:43:09.803 INFO:tasks.workunit.client.1.smithi150.stdout:5/942: stat d0/f38 0 2024-03-22T10:43:09.807 INFO:tasks.workunit.client.1.smithi150.stdout:9/944: dwrite d1/d8/de/df/d36/d121/d4e/f11b [0,4194304] 0 2024-03-22T10:43:09.810 INFO:tasks.workunit.client.1.smithi150.stdout:0/931: dread - d0/d1/d84/d86/d42/d10/d38/d52/d9b/dac/feb zero size 2024-03-22T10:43:09.811 INFO:tasks.workunit.client.1.smithi150.stdout:5/943: write d0/d8/dc/de/f119 [711158,48911] 0 2024-03-22T10:43:09.814 INFO:tasks.workunit.client.1.smithi150.stdout:4/987: dread d0/d2f/f76 [0,4194304] 0 2024-03-22T10:43:09.816 INFO:tasks.workunit.client.1.smithi150.stdout:9/945: chown d1/d8/de/df/d36/d40/d41/cf1 678 1 2024-03-22T10:43:09.823 INFO:tasks.workunit.client.1.smithi150.stdout:5/944: rename d0/d8/dc/de/d14/d4f/d134/f144 to d0/d8/dc/de/d14/d2e/d45/dac/dec/f147 0 2024-03-22T10:43:09.826 INFO:tasks.workunit.client.1.smithi150.stdout:9/946: creat d1/d19/d7d/dc1/dca/dd7/df0/d12c/f13c x:0 0 0 2024-03-22T10:43:09.828 INFO:tasks.workunit.client.1.smithi150.stdout:4/988: rename d0/d47/ca7 to d0/d2f/d10d/c14e 0 2024-03-22T10:43:09.830 INFO:tasks.workunit.client.1.smithi150.stdout:5/945: truncate d0/d8/dc/de/d14/d2e/d45/f87 168124 0 2024-03-22T10:43:09.830 INFO:tasks.workunit.client.1.smithi150.stdout:0/932: creat d0/d1/d84/d86/d42/d10/d38/d52/d9b/f126 x:0 0 0 2024-03-22T10:43:09.834 INFO:tasks.workunit.client.1.smithi150.stdout:6/891: truncate db/dd/d15/d60/d50/dc1/fba 3194780 0 2024-03-22T10:43:09.834 INFO:tasks.workunit.client.1.smithi150.stdout:0/933: chown d0/l94 0 1 2024-03-22T10:43:09.836 INFO:tasks.workunit.client.1.smithi150.stdout:3/909: sync 2024-03-22T10:43:09.838 INFO:tasks.workunit.client.1.smithi150.stdout:7/926: sync 2024-03-22T10:43:09.848 INFO:tasks.workunit.client.1.smithi150.stdout:4/989: write d0/d2f/fdb [502647,126648] 0 2024-03-22T10:43:09.849 INFO:tasks.workunit.client.1.smithi150.stdout:7/927: dread - d2/d5/d7/d8/d57/d4a/d8d/f101 zero size 2024-03-22T10:43:09.860 INFO:tasks.workunit.client.1.smithi150.stdout:9/947: dwrite d1/d8/de/dbd/ddc/ff9 [0,4194304] 0 2024-03-22T10:43:09.860 INFO:tasks.workunit.client.1.smithi150.stdout:0/934: dwrite d0/f83 [4194304,4194304] 0 2024-03-22T10:43:09.864 INFO:tasks.workunit.client.1.smithi150.stdout:7/928: write d2/d5/d7/d8/d57/d4a/d111/f116 [2130875,84780] 0 2024-03-22T10:43:09.864 INFO:tasks.workunit.client.1.smithi150.stdout:6/892: dwrite db/dd/fd4 [0,4194304] 0 2024-03-22T10:43:09.868 INFO:tasks.workunit.client.1.smithi150.stdout:0/935: readlink d0/d1/l4 0 2024-03-22T10:43:09.869 INFO:tasks.workunit.client.1.smithi150.stdout:7/929: write d2/d5/d7/d38/f4b [834503,91079] 0 2024-03-22T10:43:09.873 INFO:tasks.workunit.client.1.smithi150.stdout:0/936: truncate d0/d12/d101/f11a 235033 0 2024-03-22T10:43:09.875 INFO:tasks.workunit.client.1.smithi150.stdout:7/930: creat d2/d5/d7/d9/f135 x:0 0 0 2024-03-22T10:43:09.884 INFO:tasks.workunit.client.1.smithi150.stdout:4/990: dwrite d0/dce/d10e/d121/d7c/dc7/f145 [0,4194304] 0 2024-03-22T10:43:09.894 INFO:tasks.workunit.client.1.smithi150.stdout:0/937: truncate d0/f89 1149397 0 2024-03-22T10:43:09.895 INFO:tasks.workunit.client.1.smithi150.stdout:4/991: mknod d0/d2f/d10d/d4d/d9f/c14f 0 2024-03-22T10:43:09.901 INFO:tasks.workunit.client.1.smithi150.stdout:4/992: creat d0/d2f/f150 x:0 0 0 2024-03-22T10:43:09.902 INFO:tasks.workunit.client.1.smithi150.stdout:3/910: dread d5/da/d9d/fc5 [0,4194304] 0 2024-03-22T10:43:09.902 INFO:tasks.workunit.client.1.smithi150.stdout:7/931: creat d2/d5/d7/d8/d57/f136 x:0 0 0 2024-03-22T10:43:09.905 INFO:tasks.workunit.client.1.smithi150.stdout:3/911: write d5/d22/d37/fd8 [1253812,3367] 0 2024-03-22T10:43:09.907 INFO:tasks.workunit.client.1.smithi150.stdout:6/893: write db/d52/d9f/dc6/df1/fbc [1455621,31308] 0 2024-03-22T10:43:09.911 INFO:tasks.workunit.client.1.smithi150.stdout:4/993: symlink d0/d2f/d10d/d4d/l151 0 2024-03-22T10:43:09.918 INFO:tasks.workunit.client.1.smithi150.stdout:0/938: truncate d0/d1/d84/d86/d42/d10/d38/fc3 2052484 0 2024-03-22T10:43:09.918 INFO:tasks.workunit.client.1.smithi150.stdout:6/894: truncate db/d52/d9f/dc6/d133/fac 1645236 0 2024-03-22T10:43:09.920 INFO:tasks.workunit.client.1.smithi150.stdout:0/939: mknod d0/d1/d84/d86/d11/d26/d76/d115/c127 0 2024-03-22T10:43:09.921 INFO:tasks.workunit.client.1.smithi150.stdout:3/912: dread d5/da/d9d/fc5 [0,4194304] 0 2024-03-22T10:43:09.925 INFO:tasks.workunit.client.1.smithi150.stdout:6/895: symlink db/dd/d15/d60/d50/dc1/l138 0 2024-03-22T10:43:09.929 INFO:tasks.workunit.client.1.smithi150.stdout:3/913: mkdir d5/d22/d2f/d38/d59/dff/d133 0 2024-03-22T10:43:09.932 INFO:tasks.workunit.client.1.smithi150.stdout:3/914: chown d5/da/l4c 484 1 2024-03-22T10:43:09.940 INFO:tasks.workunit.client.1.smithi150.stdout:3/915: fdatasync d5/da/d62/dba/fe3 0 2024-03-22T10:43:09.941 INFO:tasks.workunit.client.1.smithi150.stdout:4/994: dwrite d0/d2f/d10d/d68/f149 [0,4194304] 0 2024-03-22T10:43:09.946 INFO:tasks.workunit.client.1.smithi150.stdout:3/916: symlink d5/d22/d2f/d38/d59/dff/d133/l134 0 2024-03-22T10:43:09.948 INFO:tasks.workunit.client.1.smithi150.stdout:6/896: write db/dd/fa4 [229679,69479] 0 2024-03-22T10:43:09.953 INFO:tasks.workunit.client.1.smithi150.stdout:0/940: dwrite d0/d1/d84/d86/d42/d10/fb7 [0,4194304] 0 2024-03-22T10:43:09.965 INFO:tasks.workunit.client.1.smithi150.stdout:0/941: dread - d0/d1/d84/d86/d11/d26/dd2/f102 zero size 2024-03-22T10:43:09.966 INFO:tasks.workunit.client.1.smithi150.stdout:4/995: truncate d0/d2f/f50 1379911 0 2024-03-22T10:43:09.966 INFO:tasks.workunit.client.1.smithi150.stdout:6/897: rename db/dd/d15/d6a/dde to db/dd/d15/d60/d8b/d139 0 2024-03-22T10:43:09.972 INFO:tasks.workunit.client.1.smithi150.stdout:6/898: mkdir db/d52/d9f/dc6/df1/db4/dfd/d129/d13a 0 2024-03-22T10:43:09.975 INFO:tasks.workunit.client.1.smithi150.stdout:3/917: dwrite d5/d22/d2f/d38/d59/d88/fbe [0,4194304] 0 2024-03-22T10:43:09.983 INFO:tasks.workunit.client.1.smithi150.stdout:3/918: read d5/d53/fa0 [2546163,103745] 0 2024-03-22T10:43:09.984 INFO:tasks.workunit.client.1.smithi150.stdout:3/919: chown d5/d22/d2f/d38/d59 715600 1 2024-03-22T10:43:09.985 INFO:tasks.workunit.client.1.smithi150.stdout:4/996: truncate d0/dce/d10e/d121/f105 685111 0 2024-03-22T10:43:09.986 INFO:tasks.workunit.client.1.smithi150.stdout:3/920: fdatasync d5/d53/f75 0 2024-03-22T10:43:09.989 INFO:tasks.workunit.client.1.smithi150.stdout:4/997: rmdir d0/d2f/d10d/d68 39 2024-03-22T10:43:09.993 INFO:tasks.workunit.client.1.smithi150.stdout:4/998: mknod d0/d47/d72/de7/d10a/c152 0 2024-03-22T10:43:09.996 INFO:tasks.workunit.client.1.smithi150.stdout:4/999: chown d0/cb8 172 1 2024-03-22T10:43:09.999 INFO:tasks.workunit.client.1.smithi150.stdout:6/899: dwrite db/dd/f1b [0,4194304] 0 2024-03-22T10:43:10.002 INFO:tasks.workunit.client.1.smithi150.stdout:6/900: dread - db/d52/d9f/dc6/dd0/f117 zero size 2024-03-22T10:43:10.007 INFO:tasks.workunit.client.1.smithi150.stdout:6/901: creat db/dd/d82/d84/dc8/f13b x:0 0 0 2024-03-22T10:43:10.019 INFO:tasks.workunit.client.1.smithi150.stdout:3/921: dread d5/d53/fc2 [0,4194304] 0 2024-03-22T10:43:10.026 INFO:tasks.workunit.client.1.smithi150.stdout:3/922: write d5/d22/dc8/dda/f99 [4021080,126709] 0 2024-03-22T10:43:10.027 INFO:tasks.workunit.client.1.smithi150.stdout:1/899: sync 2024-03-22T10:43:10.028 INFO:tasks.workunit.client.1.smithi150.stdout:3/923: creat d5/da/f135 x:0 0 0 2024-03-22T10:43:10.028 INFO:tasks.workunit.client.1.smithi150.stdout:6/902: dread db/d52/d9f/dc6/d133/f45 [0,4194304] 0 2024-03-22T10:43:10.030 INFO:tasks.workunit.client.1.smithi150.stdout:6/903: readlink db/dd/d15/d70/le5 0 2024-03-22T10:43:10.031 INFO:tasks.workunit.client.1.smithi150.stdout:3/924: dread d5/d22/d2f/d38/d59/d88/f8a [0,4194304] 0 2024-03-22T10:43:10.032 INFO:tasks.workunit.client.1.smithi150.stdout:3/925: chown d5/da/c23 24 1 2024-03-22T10:43:10.036 INFO:tasks.workunit.client.1.smithi150.stdout:1/900: rename d1/da to d1/d19/d50/d51/da0/dc4/d129 0 2024-03-22T10:43:10.038 INFO:tasks.workunit.client.1.smithi150.stdout:6/904: creat db/dd/d15/d30/ddf/f13c x:0 0 0 2024-03-22T10:43:10.038 INFO:tasks.workunit.client.1.smithi150.stdout:3/926: dread d5/da/d8d/fbb [0,4194304] 0 2024-03-22T10:43:10.040 INFO:tasks.workunit.client.1.smithi150.stdout:1/901: creat d1/d19/d50/d51/d123/f12a x:0 0 0 2024-03-22T10:43:10.042 INFO:tasks.workunit.client.1.smithi150.stdout:3/927: creat d5/d22/d2f/d38/d59/d88/d10d/f136 x:0 0 0 2024-03-22T10:43:10.045 INFO:tasks.workunit.client.1.smithi150.stdout:6/905: link db/d52/d9f/dc6/df1/db4/fd6 db/dd/d15/d30/ddf/d113/f13d 0 2024-03-22T10:43:10.046 INFO:tasks.workunit.client.1.smithi150.stdout:1/902: fdatasync d1/d19/d50/d51/da0/dc4/d129/d15/d18/f8c 0 2024-03-22T10:43:10.047 INFO:tasks.workunit.client.1.smithi150.stdout:6/906: fdatasync db/dd/d15/d30/fab 0 2024-03-22T10:43:10.049 INFO:tasks.workunit.client.1.smithi150.stdout:3/928: creat d5/da/d62/dba/f137 x:0 0 0 2024-03-22T10:43:10.050 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:09 smithi150 ceph-mon[34518]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:43:10.050 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:09 smithi150 ceph-mon[34518]: pgmap v23: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 73 MiB/s rd, 183 MiB/s wr, 405 op/s 2024-03-22T10:43:10.050 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:09 smithi150 ceph-mon[34518]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:10.050 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:09 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:10.050 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:09 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:10.050 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:09 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:10.052 INFO:tasks.workunit.client.1.smithi150.stdout:1/903: creat d1/d19/d50/dc7/f12b x:0 0 0 2024-03-22T10:43:10.053 INFO:tasks.workunit.client.1.smithi150.stdout:3/929: fdatasync d5/d22/d37/fe6 0 2024-03-22T10:43:10.054 INFO:tasks.workunit.client.1.smithi150.stdout:6/907: mkdir db/dd/d15/d28/d6c/d134/d13e 0 2024-03-22T10:43:10.059 INFO:tasks.workunit.client.1.smithi150.stdout:6/908: creat db/dd/d15/d6a/d8d/f13f x:0 0 0 2024-03-22T10:43:10.061 INFO:tasks.workunit.client.1.smithi150.stdout:3/930: mkdir d5/d22/d2f/d38/d138 0 2024-03-22T10:43:10.065 INFO:tasks.workunit.client.1.smithi150.stdout:6/909: rename db/dd/d82/led to db/dd/d15/d60/d50/dc1/l140 0 2024-03-22T10:43:10.066 INFO:tasks.workunit.client.1.smithi150.stdout:3/931: creat d5/d22/d2f/d38/d59/d88/f139 x:0 0 0 2024-03-22T10:43:10.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.081+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3324000bf0 con 0x7f332c06dda0 2024-03-22T10:43:10.084 INFO:tasks.workunit.client.1.smithi150.stdout:6/910: dwrite db/d52/d9f/dc6/d133/f45 [0,4194304] 0 2024-03-22T10:43:10.084 INFO:tasks.workunit.client.1.smithi150.stdout:3/932: dwrite d5/da/d109/f10b [0,4194304] 0 2024-03-22T10:43:10.084 INFO:tasks.workunit.client.1.smithi150.stdout:1/904: dread d1/d19/d50/d6e/fd3 [0,4194304] 0 2024-03-22T10:43:10.087 INFO:tasks.workunit.client.1.smithi150.stdout:3/933: mknod d5/d22/d37/c13a 0 2024-03-22T10:43:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.097+0000 7f333ea00700 1 -- 172.21.15.115:0/1042084304 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f3324000bf0 con 0x7f332c06dda0 2024-03-22T10:43:10.090 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:43:10.090 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (9m) 12s ago 11m 14.2M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:43:10.090 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (11m) 12s ago 11m 7822k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (10m) 35s ago 10m 7684k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (11m) 12s ago 11m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (10m) 35s ago 10m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (9m) 12s ago 11m 73.8M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (4m) 12s ago 4m 114M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (4m) 12s ago 4m 13.9M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (4m) 35s ago 4m 13.9M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (4m) 35s ago 4m 125M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (60s) 12s ago 13m 476M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (119s) 35s ago 10m 101M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 02c6d1a3b66d 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (13m) 12s ago 13m 53.0M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (10m) 35s ago 10m 42.3M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (11m) 12s ago 11m 7843k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (10m) 35s ago 10m 8048k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:43:10.091 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (9m) 12s ago 9m 115M 1217M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:43:10.092 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (8m) 12s ago 8m 114M 1217M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:43:10.092 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (7m) 12s ago 7m 101M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:43:10.092 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (7m) 35s ago 7m 108M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:43:10.092 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (6m) 35s ago 6m 104M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:43:10.092 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (6m) 35s ago 6m 90.0M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:43:10.092 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (70s) 12s ago 10m 48.5M - 2.43.0 a07b618ecd1d 43302dacd900 2024-03-22T10:43:10.092 INFO:tasks.workunit.client.1.smithi150.stdout:3/934: fdatasync d5/d22/d2f/d38/fb4 0 2024-03-22T10:43:10.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.101+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f332c06dda0 msgr2=0x7f332c070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:10.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.101+0000 7f334c800700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f332c06dda0 0x7f332c070250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3338006010 tx=0x7f333800b560 comp rx=0 tx=0).stop 2024-03-22T10:43:10.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.101+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440ff320 msgr2=0x7f33441980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:10.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.101+0000 7f334c800700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440ff320 0x7f33441980f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f333000b4b0 tx=0x7f333000b7c0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.102+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 shutdown_connections 2024-03-22T10:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.102+0000 7f334c800700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f332c06dda0 0x7f332c070250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.102+0000 7f334c800700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33440fe9d0 0x7f3344197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.102+0000 7f334c800700 1 --2- 172.21.15.115:0/1042084304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33440ff320 0x7f33441980f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.102+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 >> 172.21.15.115:0/1042084304 conn(0x7f33440fa5e0 msgr2=0x7f3344107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.102+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 shutdown_connections 2024-03-22T10:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.102+0000 7f334c800700 1 -- 172.21.15.115:0/1042084304 wait complete. 2024-03-22T10:43:10.099 INFO:tasks.workunit.client.1.smithi150.stdout:3/935: link d5/l6 d5/da/d78/d86/d81/l13b 0 2024-03-22T10:43:10.103 INFO:tasks.workunit.client.1.smithi150.stdout:3/936: mkdir d5/d13c 0 2024-03-22T10:43:10.106 INFO:tasks.workunit.client.1.smithi150.stdout:1/905: dwrite d1/d19/d50/d51/da0/dc4/d129/d15/f11f [0,4194304] 0 2024-03-22T10:43:10.109 INFO:tasks.workunit.client.1.smithi150.stdout:3/937: unlink d5/da/c11 0 2024-03-22T10:43:10.109 INFO:tasks.workunit.client.1.smithi150.stdout:6/911: dwrite db/d52/d9f/fc0 [0,4194304] 0 2024-03-22T10:43:10.112 INFO:tasks.workunit.client.1.smithi150.stdout:1/906: creat d1/d19/dd5/f12c x:0 0 0 2024-03-22T10:43:10.112 INFO:tasks.workunit.client.1.smithi150.stdout:3/938: chown d5/d53/l80 12 1 2024-03-22T10:43:10.114 INFO:tasks.workunit.client.1.smithi150.stdout:6/912: readlink db/l9e 0 2024-03-22T10:43:10.121 INFO:tasks.workunit.client.1.smithi150.stdout:1/907: rename d1/d19/d50/d51/da0/dc4/d129/d15/d4f to d1/d19/d50/d51/d123/d12d 0 2024-03-22T10:43:10.122 INFO:tasks.workunit.client.1.smithi150.stdout:6/913: unlink db/d52/d9f/dc6/d133/f45 0 2024-03-22T10:43:10.125 INFO:tasks.workunit.client.1.smithi150.stdout:6/914: fsync db/dd/d15/d6a/f125 0 2024-03-22T10:43:10.131 INFO:tasks.workunit.client.1.smithi150.stdout:3/939: dread d5/da/f40 [0,4194304] 0 2024-03-22T10:43:10.132 INFO:tasks.workunit.client.1.smithi150.stdout:1/908: rename d1/d19/d50/d51/da0/dc4/d129/d2a/d65/d78/l112 to d1/d19/d50/d51/da0/dcf/l12e 0 2024-03-22T10:43:10.137 INFO:tasks.workunit.client.1.smithi150.stdout:1/909: creat d1/d19/d71/f12f x:0 0 0 2024-03-22T10:43:10.138 INFO:tasks.workunit.client.1.smithi150.stdout:8/934: sync 2024-03-22T10:43:10.139 INFO:tasks.workunit.client.1.smithi150.stdout:3/940: creat d5/d22/d2f/d38/d12b/f13d x:0 0 0 2024-03-22T10:43:10.144 INFO:tasks.workunit.client.1.smithi150.stdout:6/915: dwrite db/dd/d15/d6a/fd9 [4194304,4194304] 0 2024-03-22T10:43:10.145 INFO:tasks.workunit.client.1.smithi150.stdout:1/910: creat d1/d19/d50/d51/da0/dc4/d129/d15/d18/f130 x:0 0 0 2024-03-22T10:43:10.150 INFO:tasks.workunit.client.1.smithi150.stdout:1/911: creat d1/d79/f131 x:0 0 0 2024-03-22T10:43:10.153 INFO:tasks.workunit.client.1.smithi150.stdout:6/916: rename db/dd/d15/d60/d8b/d139/c12e to db/dd/d15/d28/d6c/c141 0 2024-03-22T10:43:10.154 INFO:tasks.workunit.client.1.smithi150.stdout:3/941: write d5/d22/d2f/d38/d3e/d5b/da6/fbd [3232327,103867] 0 2024-03-22T10:43:10.159 INFO:tasks.workunit.client.1.smithi150.stdout:3/942: chown d5/d22/d2f/d38/d3e/d5b/lae 40137164 1 2024-03-22T10:43:10.160 INFO:tasks.workunit.client.1.smithi150.stdout:6/917: creat db/dd/d15/d28/d6c/d9b/f142 x:0 0 0 2024-03-22T10:43:10.166 INFO:tasks.workunit.client.1.smithi150.stdout:6/918: symlink db/dd/d15/d6a/d8d/l143 0 2024-03-22T10:43:10.167 INFO:tasks.workunit.client.1.smithi150.stdout:6/919: chown db/dd/d15 31042 1 2024-03-22T10:43:10.167 INFO:tasks.workunit.client.1.smithi150.stdout:3/943: rmdir d5/d22/dc8/dda/d95/daa 0 2024-03-22T10:43:10.168 INFO:tasks.workunit.client.1.smithi150.stdout:5/946: sync 2024-03-22T10:43:10.168 INFO:tasks.workunit.client.1.smithi150.stdout:0/942: sync 2024-03-22T10:43:10.169 INFO:tasks.workunit.client.1.smithi150.stdout:7/932: sync 2024-03-22T10:43:10.169 INFO:tasks.workunit.client.1.smithi150.stdout:9/948: sync 2024-03-22T10:43:10.172 INFO:tasks.workunit.client.1.smithi150.stdout:9/949: chown d1/d8/de/d73/d101/d29 20490433 1 2024-03-22T10:43:10.178 INFO:tasks.workunit.client.1.smithi150.stdout:6/920: mknod db/dd/d15/d30/ddf/d11a/d11b/c144 0 2024-03-22T10:43:10.179 INFO:tasks.workunit.client.1.smithi150.stdout:7/933: dread d2/d5/d7/d8/d46/d93/ffe [0,4194304] 0 2024-03-22T10:43:10.179 INFO:tasks.workunit.client.1.smithi150.stdout:5/947: creat d0/d8/f148 x:0 0 0 2024-03-22T10:43:10.180 INFO:tasks.workunit.client.1.smithi150.stdout:9/950: symlink d1/d8/de/df/d36/d121/d4e/de9/l13d 0 2024-03-22T10:43:10.180 INFO:tasks.workunit.client.1.smithi150.stdout:6/921: rmdir db/dd/d15/d28/d6c/d134 39 2024-03-22T10:43:10.181 INFO:tasks.workunit.client.1.smithi150.stdout:8/935: dread d0/d2/d19/d29/d37/f9c [0,4194304] 0 2024-03-22T10:43:10.188 INFO:tasks.workunit.client.1.smithi150.stdout:6/922: getdents db/dd/d82/de9 0 2024-03-22T10:43:10.188 INFO:tasks.workunit.client.1.smithi150.stdout:9/951: read - d1/d19/d7d/dc1/dca/dd7/df0/d12c/f135 zero size 2024-03-22T10:43:10.189 INFO:tasks.workunit.client.1.smithi150.stdout:8/936: read d0/d2/f5f [634341,125367] 0 2024-03-22T10:43:10.190 INFO:tasks.workunit.client.1.smithi150.stdout:1/912: dread d1/d19/d50/d51/f9c [0,4194304] 0 2024-03-22T10:43:10.191 INFO:tasks.workunit.client.1.smithi150.stdout:6/923: chown db/dd/d15/d60/d8b/d139 2877709 1 2024-03-22T10:43:10.194 INFO:tasks.workunit.client.1.smithi150.stdout:5/948: write d0/dae/db2/f6b [1822368,32136] 0 2024-03-22T10:43:10.195 INFO:tasks.workunit.client.1.smithi150.stdout:3/944: dread d5/d22/d2f/d38/d3e/f3f [0,4194304] 0 2024-03-22T10:43:10.195 INFO:tasks.workunit.client.1.smithi150.stdout:8/937: chown d0/d3c/d71/d7a/d89/f12b 493464 1 2024-03-22T10:43:10.195 INFO:tasks.workunit.client.1.smithi150.stdout:6/924: creat db/d52/d9f/dc6/f145 x:0 0 0 2024-03-22T10:43:10.200 INFO:tasks.workunit.client.1.smithi150.stdout:9/952: mkdir d1/d8/de/df/d36/d121/d4e/de9/d13e 0 2024-03-22T10:43:10.200 INFO:tasks.workunit.client.1.smithi150.stdout:6/925: fsync db/dd/d15/d28/f41 0 2024-03-22T10:43:10.201 INFO:tasks.workunit.client.1.smithi150.stdout:5/949: stat d0/d8/dc/de/f119 0 2024-03-22T10:43:10.202 INFO:tasks.workunit.client.1.smithi150.stdout:3/945: mkdir d5/d22/d2f/d38/d59/dff/d133/d13e 0 2024-03-22T10:43:10.205 INFO:tasks.workunit.client.1.smithi150.stdout:8/938: mknod d0/d2/d19/d29/d2f/df/c136 0 2024-03-22T10:43:10.208 INFO:tasks.workunit.client.1.smithi150.stdout:9/953: creat d1/d8/de/dbd/f13f x:0 0 0 2024-03-22T10:43:10.210 INFO:tasks.workunit.client.1.smithi150.stdout:6/926: mknod db/dd/d15/d28/d6c/d9b/c146 0 2024-03-22T10:43:10.211 INFO:tasks.workunit.client.1.smithi150.stdout:3/946: chown d5/d22/d2f/f68 30818 1 2024-03-22T10:43:10.211 INFO:tasks.workunit.client.1.smithi150.stdout:5/950: dread - d0/d8/dc/de/d14/d2e/d45/dac/dec/f147 zero size 2024-03-22T10:43:10.212 INFO:tasks.workunit.client.1.smithi150.stdout:0/943: dread d0/d1/d84/d86/d42/f4a [0,4194304] 0 2024-03-22T10:43:10.217 INFO:tasks.workunit.client.1.smithi150.stdout:7/934: dwrite d2/d5/d7/d8/d57/f3c [0,4194304] 0 2024-03-22T10:43:10.218 INFO:tasks.workunit.client.1.smithi150.stdout:6/927: dread - db/d52/d9f/dc6/f145 zero size 2024-03-22T10:43:10.223 INFO:tasks.workunit.client.1.smithi150.stdout:6/928: write db/dd/d15/d60/fcd [1508223,128659] 0 2024-03-22T10:43:10.225 INFO:tasks.workunit.client.1.smithi150.stdout:1/913: dwrite d1/d19/d50/d51/da0/dc4/d129/d2a/d65/d78/dfd/d11c/f11b [0,4194304] 0 2024-03-22T10:43:10.225 INFO:tasks.workunit.client.1.smithi150.stdout:3/947: mkdir d5/da/d109/d13f 0 2024-03-22T10:43:10.228 INFO:tasks.workunit.client.1.smithi150.stdout:0/944: rmdir d0/d1/d84/d86/d11/d66/d35/d5f/da0 39 2024-03-22T10:43:10.229 INFO:tasks.workunit.client.1.smithi150.stdout:7/935: mknod d2/d5/d7/d8/d57/d105/c137 0 2024-03-22T10:43:10.233 INFO:tasks.workunit.client.1.smithi150.stdout:6/929: creat db/dd/d15/d30/ddf/d103/f147 x:0 0 0 2024-03-22T10:43:10.233 INFO:tasks.workunit.client.1.smithi150.stdout:5/951: mkdir d0/d8/dc/de/d14/d2e/d45/dac/dd7/d149 0 2024-03-22T10:43:10.236 INFO:tasks.workunit.client.1.smithi150.stdout:3/948: write d5/d22/d2f/d38/d3e/d5b/f9a [4547920,81192] 0 2024-03-22T10:43:10.236 INFO:tasks.workunit.client.1.smithi150.stdout:8/939: creat d0/d2/d19/d29/d2f/df/dff/f137 x:0 0 0 2024-03-22T10:43:10.241 INFO:tasks.workunit.client.1.smithi150.stdout:0/945: unlink d0/d1/d84/d86/d11/d26/c50 0 2024-03-22T10:43:10.241 INFO:tasks.workunit.client.1.smithi150.stdout:6/930: rmdir db/d52 39 2024-03-22T10:43:10.246 INFO:tasks.workunit.client.1.smithi150.stdout:5/952: rename d0/dae/ff8 to d0/d8/dc/de/d14/d2e/d45/dac/dec/f14a 0 2024-03-22T10:43:10.246 INFO:tasks.workunit.client.1.smithi150.stdout:8/940: mkdir d0/d2/d19/d29/d2f/df/dff/d138 0 2024-03-22T10:43:10.252 INFO:tasks.workunit.client.1.smithi150.stdout:9/954: dread d1/d19/d7d/dc1/dca/dd7/ff5 [0,4194304] 0 2024-03-22T10:43:10.252 INFO:tasks.workunit.client.1.smithi150.stdout:1/914: link d1/d19/d50/d51/da0/dc4/d129/d15/d18/c126 d1/d19/d50/d51/da0/dc4/d129/d15/d18/d121/de4/c132 0 2024-03-22T10:43:10.255 INFO:tasks.workunit.client.1.smithi150.stdout:0/946: creat d0/d1/d84/d86/d11/d66/d35/d5f/da0/f128 x:0 0 0 2024-03-22T10:43:10.256 INFO:tasks.workunit.client.1.smithi150.stdout:6/931: symlink db/dd/d15/d60/d50/l148 0 2024-03-22T10:43:10.261 INFO:tasks.workunit.client.1.smithi150.stdout:7/936: dwrite d2/d5/d7/d9/db/d9f/d96/d71/f119 [0,4194304] 0 2024-03-22T10:43:10.271 INFO:tasks.workunit.client.1.smithi150.stdout:8/941: link d0/d2/fe d0/d2/d46/da2/f139 0 2024-03-22T10:43:10.279 INFO:tasks.workunit.client.1.smithi150.stdout:8/942: symlink d0/d3c/d71/l13a 0 2024-03-22T10:43:10.285 INFO:tasks.workunit.client.1.smithi150.stdout:3/949: dread d5/d22/d2f/f30 [0,4194304] 0 2024-03-22T10:43:10.286 INFO:tasks.workunit.client.1.smithi150.stdout:9/955: truncate d1/d8/de/df/d36/d121/fe3 3147936 0 2024-03-22T10:43:10.287 INFO:tasks.workunit.client.1.smithi150.stdout:1/915: dwrite d1/d19/d50/d51/da0/dc4/d129/d15/d18/f105 [0,4194304] 0 2024-03-22T10:43:10.287 INFO:tasks.workunit.client.1.smithi150.stdout:0/947: rename d0/d1/d84/d86/d11/fd5 to d0/d1/d84/d86/d42/d10/d38/d52/d9b/f129 0 2024-03-22T10:43:10.289 INFO:tasks.workunit.client.1.smithi150.stdout:3/950: read - d5/da/f135 zero size 2024-03-22T10:43:10.292 INFO:tasks.workunit.client.1.smithi150.stdout:7/937: dwrite d2/d5/d7/f50 [0,4194304] 0 2024-03-22T10:43:10.292 INFO:tasks.workunit.client.1.smithi150.stdout:5/953: dwrite d0/d8/dc/de/d14/d2e/d46/f72 [0,4194304] 0 2024-03-22T10:43:10.293 INFO:tasks.workunit.client.1.smithi150.stdout:8/943: mknod d0/d2/d19/d29/d2f/df/dff/d138/c13b 0 2024-03-22T10:43:10.295 INFO:tasks.workunit.client.1.smithi150.stdout:5/954: readlink d0/d8/dc/l4b 0 2024-03-22T10:43:10.297 INFO:tasks.workunit.client.1.smithi150.stdout:3/951: fdatasync d5/d22/d2f/d38/d3e/d72/ff8 0 2024-03-22T10:43:10.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.305+0000 7fefe5200700 1 -- 172.21.15.115:0/2561491880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe00737b0 msgr2=0x7fefe0073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:10.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.305+0000 7fefe5200700 1 --2- 172.21.15.115:0/2561491880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe00737b0 0x7fefe0073c20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009940 tx=0x7fefd0009c50 comp rx=0 tx=0).stop 2024-03-22T10:43:10.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.305+0000 7fefe5200700 1 -- 172.21.15.115:0/2561491880 shutdown_connections 2024-03-22T10:43:10.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.305+0000 7fefe5200700 1 --2- 172.21.15.115:0/2561491880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe00737b0 0x7fefe0073c20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.305+0000 7fefe5200700 1 --2- 172.21.15.115:0/2561491880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe0074d80 0x7fefe00731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.305+0000 7fefe5200700 1 -- 172.21.15.115:0/2561491880 >> 172.21.15.115:0/2561491880 conn(0x7fefe00fb6e0 msgr2=0x7fefe00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.306+0000 7fefe5200700 1 -- 172.21.15.115:0/2561491880 shutdown_connections 2024-03-22T10:43:10.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.306+0000 7fefe5200700 1 -- 172.21.15.115:0/2561491880 wait complete. 2024-03-22T10:43:10.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.307+0000 7fefe5200700 1 Processor -- start 2024-03-22T10:43:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.308+0000 7fefe5200700 1 -- start start 2024-03-22T10:43:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.308+0000 7fefe5200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe00737b0 0x7fefe0103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.308+0000 7fefe5200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe0074d80 0x7fefe01039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.308+0000 7fefe5200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00fff70 con 0x7fefe0074d80 2024-03-22T10:43:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.308+0000 7fefe5200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01000e0 con 0x7fefe00737b0 2024-03-22T10:43:10.302 INFO:tasks.workunit.client.1.smithi150.stdout:0/948: symlink d0/d1/d84/d86/l12a 0 2024-03-22T10:43:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.309+0000 7fefdd400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe0074d80 0x7fefe01039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:10.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.309+0000 7fefdd400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe0074d80 0x7fefe01039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53270/0 (socket says 172.21.15.115:53270) 2024-03-22T10:43:10.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.309+0000 7fefdd400700 1 -- 172.21.15.115:0/2047156564 learned_addr learned my addr 172.21.15.115:0/2047156564 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:10.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.310+0000 7fefdde00700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe00737b0 0x7fefe0103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:10.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.310+0000 7fefdd400700 1 -- 172.21.15.115:0/2047156564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe00737b0 msgr2=0x7fefe0103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:10.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.310+0000 7fefdd400700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe00737b0 0x7fefe0103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.310+0000 7fefdd400700 1 -- 172.21.15.115:0/2047156564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd00095f0 con 0x7fefe0074d80 2024-03-22T10:43:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.311+0000 7fefdd400700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe0074d80 0x7fefe01039c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fefd00055e0 tx=0x7fefd0005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.312+0000 7fefd6a00700 1 -- 172.21.15.115:0/2047156564 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefd001d070 con 0x7fefe0074d80 2024-03-22T10:43:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.312+0000 7fefd6a00700 1 -- 172.21.15.115:0/2047156564 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fefd000f460 con 0x7fefe0074d80 2024-03-22T10:43:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.312+0000 7fefd6a00700 1 -- 172.21.15.115:0/2047156564 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefd0021640 con 0x7fefe0074d80 2024-03-22T10:43:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.312+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe0100360 con 0x7fefe0074d80 2024-03-22T10:43:10.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.312+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe01008d0 con 0x7fefe0074d80 2024-03-22T10:43:10.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.314+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe010af30 con 0x7fefe0074d80 2024-03-22T10:43:10.308 INFO:tasks.workunit.client.1.smithi150.stdout:8/944: mkdir d0/d3c/d13c 0 2024-03-22T10:43:10.308 INFO:tasks.workunit.client.1.smithi150.stdout:6/932: dread db/dd/d15/d60/d50/dc1/fec [0,4194304] 0 2024-03-22T10:43:10.308 INFO:tasks.workunit.client.1.smithi150.stdout:9/956: creat d1/d8/de/df/d36/d40/d44/db5/dda/d104/f140 x:0 0 0 2024-03-22T10:43:10.308 INFO:tasks.workunit.client.1.smithi150.stdout:3/952: symlink d5/da/d78/l140 0 2024-03-22T10:43:10.310 INFO:tasks.workunit.client.1.smithi150.stdout:7/938: rename d2/d5/d7/d9/db/c77 to d2/d5/d7/d8/d46/c138 0 2024-03-22T10:43:10.311 INFO:tasks.workunit.client.1.smithi150.stdout:5/955: symlink d0/d8/dc/de/d14/d2e/d5a/d66/l14b 0 2024-03-22T10:43:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.317+0000 7fefd6a00700 1 -- 172.21.15.115:0/2047156564 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fefd000f5d0 con 0x7fefe0074d80 2024-03-22T10:43:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.318+0000 7fefd6a00700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fefc406ddf0 0x7fefc40702a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.318+0000 7fefd6a00700 1 -- 172.21.15.115:0/2047156564 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fefd008f1c0 con 0x7fefe0074d80 2024-03-22T10:43:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.319+0000 7fefdde00700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fefc406ddf0 0x7fefc40702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:10.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.320+0000 7fefdde00700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fefc406ddf0 0x7fefc40702a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fefcc006fd0 tx=0x7fefcc009190 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:10.314 INFO:tasks.workunit.client.1.smithi150.stdout:3/953: mknod d5/d22/d2f/d38/d59/d88/d127/c141 0 2024-03-22T10:43:10.317 INFO:tasks.workunit.client.1.smithi150.stdout:8/945: truncate d0/d2/d46/ff1 1104980 0 2024-03-22T10:43:10.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.326+0000 7fefd6a00700 1 -- 172.21.15.115:0/2047156564 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fefd005c4d0 con 0x7fefe0074d80 2024-03-22T10:43:10.320 INFO:tasks.workunit.client.1.smithi150.stdout:9/957: read - d1/d8/de/df/d36/d40/d44/db5/fdb zero size 2024-03-22T10:43:10.321 INFO:tasks.workunit.client.1.smithi150.stdout:7/939: mknod d2/d5/d7/d127/c139 0 2024-03-22T10:43:10.322 INFO:tasks.workunit.client.1.smithi150.stdout:5/956: symlink d0/d8/dc/de/d14/l14c 0 2024-03-22T10:43:10.323 INFO:tasks.workunit.client.1.smithi150.stdout:3/954: write d5/da/d62/dba/f137 [884347,114853] 0 2024-03-22T10:43:10.325 INFO:tasks.workunit.client.1.smithi150.stdout:7/940: fdatasync d2/d5/d7/fa4 0 2024-03-22T10:43:10.325 INFO:tasks.workunit.client.1.smithi150.stdout:6/933: mknod db/dd/d15/d28/d6c/d134/d13e/c149 0 2024-03-22T10:43:10.328 INFO:tasks.workunit.client.1.smithi150.stdout:8/946: mkdir d0/d2/d46/de3/d13d 0 2024-03-22T10:43:10.329 INFO:tasks.workunit.client.1.smithi150.stdout:1/916: dwrite d1/d19/d50/d51/da0/dc4/d129/d15/d18/d121/faf [0,4194304] 0 2024-03-22T10:43:10.334 INFO:tasks.workunit.client.1.smithi150.stdout:3/955: unlink d5/d53/l96 0 2024-03-22T10:43:10.339 INFO:tasks.workunit.client.1.smithi150.stdout:0/949: dread d0/d1/d84/d86/d42/d10/d38/f4c [4194304,4194304] 0 2024-03-22T10:43:10.342 INFO:tasks.workunit.client.1.smithi150.stdout:6/934: unlink db/dd/d15/d30/ddf/lf2 0 2024-03-22T10:43:10.345 INFO:tasks.workunit.client.1.smithi150.stdout:8/947: symlink d0/d2/d19/d29/d37/dc6/l13e 0 2024-03-22T10:43:10.349 INFO:tasks.workunit.client.1.smithi150.stdout:9/958: dwrite d1/d49/f109 [0,4194304] 0 2024-03-22T10:43:10.354 INFO:tasks.workunit.client.1.smithi150.stdout:7/941: rename d2/d5/d7/d9/db/d9f to d2/d5/d7/d9/db/dfb/d13a 0 2024-03-22T10:43:10.363 INFO:tasks.workunit.client.1.smithi150.stdout:1/917: mknod d1/d19/d1a/df3/c133 0 2024-03-22T10:43:10.366 INFO:tasks.workunit.client.1.smithi150.stdout:6/935: mknod db/dd/d15/d60/d8b/de7/c14a 0 2024-03-22T10:43:10.368 INFO:tasks.workunit.client.1.smithi150.stdout:8/948: chown d0/d3c/fce 72374 1 2024-03-22T10:43:10.374 INFO:tasks.workunit.client.1.smithi150.stdout:1/918: fsync d1/d19/f26 0 2024-03-22T10:43:10.382 INFO:tasks.workunit.client.1.smithi150.stdout:8/949: creat d0/d2/d19/d29/d37/d7d/d9e/f13f x:0 0 0 2024-03-22T10:43:10.388 INFO:tasks.workunit.client.1.smithi150.stdout:3/956: truncate d5/d22/d2f/d38/fb4 80861 0 2024-03-22T10:43:10.388 INFO:tasks.workunit.client.1.smithi150.stdout:9/959: link d1/d8/de/d73/d101/d29/c6f d1/d19/d7d/dc1/c141 0 2024-03-22T10:43:10.391 INFO:tasks.workunit.client.1.smithi150.stdout:5/957: dwrite d0/d8/dc/de/d14/d2e/d45/dac/dec/f14a [4194304,4194304] 0 2024-03-22T10:43:10.392 INFO:tasks.workunit.client.1.smithi150.stdout:7/942: getdents d2/d5/d7/d9/db/dfb/d13a/d45 0 2024-03-22T10:43:10.396 INFO:tasks.workunit.client.1.smithi150.stdout:1/919: dread d1/d79/f85 [0,4194304] 0 2024-03-22T10:43:10.400 INFO:tasks.workunit.client.1.smithi150.stdout:6/936: write db/d52/de6/f137 [530483,52646] 0 2024-03-22T10:43:10.402 INFO:tasks.workunit.client.1.smithi150.stdout:7/943: write d2/d5/d7/d9/db/dfb/d13a/f40 [1736443,41867] 0 2024-03-22T10:43:10.407 INFO:tasks.workunit.client.1.smithi150.stdout:6/937: write db/dd/d15/d30/ddf/f13c [494775,18182] 0 2024-03-22T10:43:10.410 INFO:tasks.workunit.client.1.smithi150.stdout:5/958: rename d0/d8/dc/d1d/d53 to d0/d8/dc/de/d14/d4f/d47/d14d 0 2024-03-22T10:43:10.412 INFO:tasks.workunit.client.1.smithi150.stdout:7/944: chown d2/d5/d7/d9/db/dfb/d13a/d96/d71/d106/d91/daf/cce 1485209680 1 2024-03-22T10:43:10.415 INFO:tasks.workunit.client.1.smithi150.stdout:5/959: mkdir d0/dae/db2/d14e 0 2024-03-22T10:43:10.419 INFO:tasks.workunit.client.1.smithi150.stdout:9/960: getdents d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7 0 2024-03-22T10:43:10.422 INFO:tasks.workunit.client.1.smithi150.stdout:7/945: symlink d2/l13b 0 2024-03-22T10:43:10.424 INFO:tasks.workunit.client.1.smithi150.stdout:3/957: dwrite d5/d22/d2f/d38/d3e/d72/ff8 [0,4194304] 0 2024-03-22T10:43:10.426 INFO:tasks.workunit.client.1.smithi150.stdout:7/946: stat d2/d5/d7/c14 0 2024-03-22T10:43:10.426 INFO:tasks.workunit.client.1.smithi150.stdout:5/960: creat d0/d8/dc/de/d14/d4f/d134/f14f x:0 0 0 2024-03-22T10:43:10.432 INFO:tasks.workunit.client.1.smithi150.stdout:1/920: dwrite d1/d19/d50/dc7/f12b [0,4194304] 0 2024-03-22T10:43:10.437 INFO:tasks.workunit.client.1.smithi150.stdout:8/950: dread d0/d2/d19/d29/d2f/df/d1b/f30 [0,4194304] 0 2024-03-22T10:43:10.437 INFO:tasks.workunit.client.1.smithi150.stdout:7/947: symlink d2/d5/d7/d8/d57/d4a/d111/l13c 0 2024-03-22T10:43:10.440 INFO:tasks.workunit.client.1.smithi150.stdout:3/958: symlink d5/d22/dc8/dda/d95/l142 0 2024-03-22T10:43:10.446 INFO:tasks.workunit.client.1.smithi150.stdout:6/938: dwrite db/dd/d15/d60/d50/f110 [0,4194304] 0 2024-03-22T10:43:10.447 INFO:tasks.workunit.client.1.smithi150.stdout:8/951: fsync d0/d2/d19/d29/d37/f9c 0 2024-03-22T10:43:10.451 INFO:tasks.workunit.client.1.smithi150.stdout:5/961: link d0/c139 d0/d8/dc/de/d14/d2e/d45/dac/d105/c150 0 2024-03-22T10:43:10.454 INFO:tasks.workunit.client.1.smithi150.stdout:1/921: rename d1/d19/le6 to d1/d19/d50/d51/da0/dc4/d129/d2a/l134 0 2024-03-22T10:43:10.456 INFO:tasks.workunit.client.1.smithi150.stdout:6/939: dread - db/dd/d82/f8c zero size 2024-03-22T10:43:10.459 INFO:tasks.workunit.client.1.smithi150.stdout:8/952: write d0/d2/d19/d29/d2e/d70/fad [576379,117757] 0 2024-03-22T10:43:10.471 INFO:tasks.workunit.client.1.smithi150.stdout:3/959: rename d5/da/l4c to d5/d22/d2f/d38/d12b/l143 0 2024-03-22T10:43:10.472 INFO:tasks.workunit.client.1.smithi150.stdout:1/922: dread d1/d19/d50/d6e/f10b [0,4194304] 0 2024-03-22T10:43:10.474 INFO:tasks.workunit.client.1.smithi150.stdout:5/962: creat d0/d8/dc/de/d14/d2e/d45/d102/d11c/f151 x:0 0 0 2024-03-22T10:43:10.475 INFO:tasks.workunit.client.1.smithi150.stdout:1/923: chown d1/d19/d50/d51/f9c 0 1 2024-03-22T10:43:10.476 INFO:tasks.workunit.client.1.smithi150.stdout:5/963: chown d0/d8/l7c 3442769 1 2024-03-22T10:43:10.477 INFO:tasks.workunit.client.1.smithi150.stdout:9/961: dread d1/d8/de/df/f5b [0,4194304] 0 2024-03-22T10:43:10.480 INFO:tasks.workunit.client.1.smithi150.stdout:3/960: write d5/d53/fa0 [553193,14208] 0 2024-03-22T10:43:10.483 INFO:tasks.workunit.client.1.smithi150.stdout:7/948: truncate d2/d5/d7/d9/db/d6c/d9c/fc0 802184 0 2024-03-22T10:43:10.486 INFO:tasks.workunit.client.1.smithi150.stdout:5/964: symlink d0/d8/dc/d1d/l152 0 2024-03-22T10:43:10.487 INFO:tasks.workunit.client.1.smithi150.stdout:1/924: mknod d1/d19/d50/d51/da0/dc4/d129/d2a/d4e/c135 0 2024-03-22T10:43:10.488 INFO:tasks.workunit.client.1.smithi150.stdout:9/962: write d1/d8/de/df/d36/d121/f89 [3733589,13559] 0 2024-03-22T10:43:10.488 INFO:tasks.workunit.client.1.smithi150.stdout:3/961: write d5/da/f57 [1664852,51806] 0 2024-03-22T10:43:10.491 INFO:tasks.workunit.client.1.smithi150.stdout:6/940: dwrite db/dd/d15/d30/ddf/d103/f147 [0,4194304] 0 2024-03-22T10:43:10.491 INFO:tasks.workunit.client.1.smithi150.stdout:7/949: chown d2/d5/d7/d9/db/dfb/d13a/c89 13817456 1 2024-03-22T10:43:10.501 INFO:tasks.workunit.client.1.smithi150.stdout:1/925: creat d1/d19/d50/d51/d123/f136 x:0 0 0 2024-03-22T10:43:10.502 INFO:tasks.workunit.client.1.smithi150.stdout:8/953: dwrite d0/d5d/dd2/fb4 [4194304,4194304] 0 2024-03-22T10:43:10.502 INFO:tasks.workunit.client.1.smithi150.stdout:6/941: creat db/dd/d15/d6a/d8d/f14b x:0 0 0 2024-03-22T10:43:10.509 INFO:tasks.workunit.client.1.smithi150.stdout:5/965: rename d0/d8/dc/de/d14/d68 to d0/d8/dc/de/d14/d2e/d45/dac/dec/d153 0 2024-03-22T10:43:10.509 INFO:tasks.workunit.client.1.smithi150.stdout:7/950: symlink d2/d5/d7/d9/db/l13d 0 2024-03-22T10:43:10.513 INFO:tasks.workunit.client.1.smithi150.stdout:9/963: rmdir d1/d8/de/d73/d101/d32/dd1 0 2024-03-22T10:43:10.515 INFO:tasks.workunit.client.1.smithi150.stdout:1/926: symlink d1/d19/d50/dc7/l137 0 2024-03-22T10:43:10.515 INFO:tasks.workunit.client.1.smithi150.stdout:0/950: sync 2024-03-22T10:43:10.517 INFO:tasks.workunit.client.1.smithi150.stdout:8/954: creat d0/d3c/d71/d7a/f140 x:0 0 0 2024-03-22T10:43:10.517 INFO:tasks.workunit.client.1.smithi150.stdout:3/962: truncate d5/da/f7a 4618292 0 2024-03-22T10:43:10.519 INFO:tasks.workunit.client.1.smithi150.stdout:3/963: write d5/d53/f54 [2892852,93675] 0 2024-03-22T10:43:10.521 INFO:tasks.workunit.client.1.smithi150.stdout:8/955: truncate d0/d2/d19/d29/d88/dec/ffe 212475 0 2024-03-22T10:43:10.522 INFO:tasks.workunit.client.1.smithi150.stdout:5/966: creat d0/d33/d6f/d137/f154 x:0 0 0 2024-03-22T10:43:10.523 INFO:tasks.workunit.client.1.smithi150.stdout:9/964: mkdir d1/d19/d7d/dc1/dca/dd7/df0/d12c/d142 0 2024-03-22T10:43:10.530 INFO:tasks.workunit.client.1.smithi150.stdout:3/964: mkdir d5/da/d78/db7/dec/d144 0 2024-03-22T10:43:10.537 INFO:tasks.workunit.client.1.smithi150.stdout:8/956: unlink d0/ddd/dfc/c102 0 2024-03-22T10:43:10.540 INFO:tasks.workunit.client.1.smithi150.stdout:9/965: rmdir d1/d8/de/df 39 2024-03-22T10:43:10.540 INFO:tasks.workunit.client.1.smithi150.stdout:6/942: getdents db/dd/d15/d30/ddf 0 2024-03-22T10:43:10.543 INFO:tasks.workunit.client.1.smithi150.stdout:7/951: dwrite d2/d5/d7/d9/db/dfb/d13a/d96/d71/d106/d91/f114 [0,4194304] 0 2024-03-22T10:43:10.554 INFO:tasks.workunit.client.1.smithi150.stdout:5/967: dwrite d0/dae/de0/ffc [0,4194304] 0 2024-03-22T10:43:10.563 INFO:tasks.workunit.client.1.smithi150.stdout:0/951: dwrite d0/d1/d84/d86/d42/d10/d53/f5e [0,4194304] 0 2024-03-22T10:43:10.564 INFO:tasks.workunit.client.1.smithi150.stdout:8/957: dwrite d0/d2/d19/d29/d2e/f3a [0,4194304] 0 2024-03-22T10:43:10.564 INFO:tasks.workunit.client.1.smithi150.stdout:5/968: chown d0/dae/c143 153106 1 2024-03-22T10:43:10.572 INFO:tasks.workunit.client.1.smithi150.stdout:1/927: dread d1/d19/d50/d6e/f77 [0,4194304] 0 2024-03-22T10:43:10.574 INFO:tasks.workunit.client.1.smithi150.stdout:0/952: write d0/d1/d84/d86/d42/d10/d38/d79/fbe [991848,117002] 0 2024-03-22T10:43:10.576 INFO:tasks.workunit.client.1.smithi150.stdout:1/928: fsync d1/d19/d50/d51/da0/dc4/d129/d2a/d4e/fd7 0 2024-03-22T10:43:10.592 INFO:tasks.workunit.client.1.smithi150.stdout:6/943: mkdir db/d52/d9f/dc6/df1/db4/dfd/d129/d14c 0 2024-03-22T10:43:10.593 INFO:tasks.workunit.client.1.smithi150.stdout:0/953: mkdir d0/d1/d84/d86/d11/d26/dd2/df0/d12b 0 2024-03-22T10:43:10.598 INFO:tasks.workunit.client.1.smithi150.stdout:5/969: rename d0/f22 to d0/d8/d101/d11b/d12e/f155 0 2024-03-22T10:43:10.604 INFO:tasks.workunit.client.1.smithi150.stdout:3/965: link d5/d22/d2f/d38/d3e/d5b/f76 d5/d22/d118/f145 0 2024-03-22T10:43:10.607 INFO:tasks.workunit.client.1.smithi150.stdout:8/958: rename d0/d2/d19/d29/d2f/df/f45 to d0/d3c/d71/f141 0 2024-03-22T10:43:10.608 INFO:tasks.workunit.client.1.smithi150.stdout:7/952: getdents d2/d5/d7/dab/dec 0 2024-03-22T10:43:10.608 INFO:tasks.workunit.client.1.smithi150.stdout:6/944: dread db/dd/d82/d84/f87 [0,4194304] 0 2024-03-22T10:43:10.613 INFO:tasks.workunit.client.1.smithi150.stdout:3/966: creat d5/d22/d2f/d38/d3e/d5b/dee/f146 x:0 0 0 2024-03-22T10:43:10.614 INFO:tasks.workunit.client.1.smithi150.stdout:9/966: link d1/d49/cab d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/d133/c143 0 2024-03-22T10:43:10.615 INFO:tasks.workunit.client.1.smithi150.stdout:6/945: unlink db/dd/c12 0 2024-03-22T10:43:10.615 INFO:tasks.workunit.client.1.smithi150.stdout:0/954: dread d0/d12/d101/f119 [0,4194304] 0 2024-03-22T10:43:10.618 INFO:tasks.workunit.client.1.smithi150.stdout:8/959: symlink d0/d2/d19/d29/d2e/l142 0 2024-03-22T10:43:10.629 INFO:tasks.workunit.client.1.smithi150.stdout:8/960: write d0/d2/d19/d29/d2f/df/d1b/f1d [952471,84065] 0 2024-03-22T10:43:10.639 INFO:tasks.workunit.client.1.smithi150.stdout:7/953: creat d2/d5/d7/d9/db/dfb/d13a/d96/d71/d106/d91/f13e x:0 0 0 2024-03-22T10:43:10.642 INFO:tasks.workunit.client.1.smithi150.stdout:0/955: dwrite d0/d1/d84/d86/d42/d10/f43 [4194304,4194304] 0 2024-03-22T10:43:10.642 INFO:tasks.workunit.client.1.smithi150.stdout:8/961: rename d0/d2/d19/d29/d88 to d0/d3c/d71/d7a/d89/d143 0 2024-03-22T10:43:10.649 INFO:tasks.workunit.client.1.smithi150.stdout:6/946: getdents db/dd/d15/d30/ddf/d11a 0 2024-03-22T10:43:10.656 INFO:tasks.workunit.client.1.smithi150.stdout:8/962: link d0/d2/d19/d29/d2e/f3a d0/d2/d19/d29/d37/da1/f144 0 2024-03-22T10:43:10.659 INFO:tasks.workunit.client.1.smithi150.stdout:0/956: truncate d0/d1/d84/d86/d42/d10/d38/fc3 412522 0 2024-03-22T10:43:10.662 INFO:tasks.workunit.client.1.smithi150.stdout:6/947: creat db/d52/d9f/dc6/df1/db4/dfd/d129/d13a/f14d x:0 0 0 2024-03-22T10:43:10.662 INFO:tasks.workunit.client.1.smithi150.stdout:7/954: dread d2/d5/d7/d9/db/dfb/d13a/d96/f8c [0,4194304] 0 2024-03-22T10:43:10.677 INFO:tasks.workunit.client.1.smithi150.stdout:6/948: creat db/d52/d9f/dc6/f14e x:0 0 0 2024-03-22T10:43:10.684 INFO:tasks.workunit.client.1.smithi150.stdout:0/957: creat d0/d1/d84/d86/d42/d10/d38/d52/d68/f12c x:0 0 0 2024-03-22T10:43:10.691 INFO:tasks.workunit.client.1.smithi150.stdout:0/958: mkdir d0/d1/d84/d86/da6/d12d 0 2024-03-22T10:43:10.697 INFO:tasks.workunit.client.1.smithi150.stdout:8/963: truncate d0/d2/fa8 1531362 0 2024-03-22T10:43:10.699 INFO:tasks.workunit.client.1.smithi150.stdout:7/955: write d2/d5/d7/d9/db/d6c/d9c/fc0 [113969,61519] 0 2024-03-22T10:43:10.701 INFO:tasks.workunit.client.1.smithi150.stdout:6/949: dwrite db/dd/d15/d60/f9c [4194304,4194304] 0 2024-03-22T10:43:10.701 INFO:tasks.workunit.client.1.smithi150.stdout:0/959: mkdir d0/d1/d84/d86/d42/d10/d38/d52/d9b/db4/d12e 0 2024-03-22T10:43:10.703 INFO:tasks.workunit.client.1.smithi150.stdout:8/964: rmdir d0/d2/d46/da2/dd0 39 2024-03-22T10:43:10.706 INFO:tasks.workunit.client.1.smithi150.stdout:6/950: mknod db/dd/d82/c14f 0 2024-03-22T10:43:10.706 INFO:tasks.workunit.client.1.smithi150.stdout:0/960: chown d0/d1/d84/d86/d42/d10/d38/d79 365 1 2024-03-22T10:43:10.707 INFO:tasks.workunit.client.1.smithi150.stdout:6/951: fdatasync db/dd/fa4 0 2024-03-22T10:43:10.710 INFO:tasks.workunit.client.1.smithi150.stdout:8/965: rmdir d0/d3c/d71/d7a/d89/d12d/dda 39 2024-03-22T10:43:10.711 INFO:tasks.workunit.client.1.smithi150.stdout:7/956: symlink d2/d5/d7/d9/db/dfb/d13a/d96/dbf/l13f 0 2024-03-22T10:43:10.712 INFO:tasks.workunit.client.1.smithi150.stdout:6/952: readlink db/l3f 0 2024-03-22T10:43:10.714 INFO:tasks.workunit.client.1.smithi150.stdout:7/957: fdatasync d2/d5/d7/d9/f135 0 2024-03-22T10:43:10.715 INFO:tasks.workunit.client.1.smithi150.stdout:0/961: creat d0/f12f x:0 0 0 2024-03-22T10:43:10.718 INFO:tasks.workunit.client.1.smithi150.stdout:8/966: mkdir d0/d2/d19/d145 0 2024-03-22T10:43:10.718 INFO:tasks.workunit.client.1.smithi150.stdout:6/953: truncate db/dd/d15/d60/d50/f124 421058 0 2024-03-22T10:43:10.721 INFO:tasks.workunit.client.1.smithi150.stdout:7/958: creat d2/d5/d7/d8/d46/f140 x:0 0 0 2024-03-22T10:43:10.723 INFO:tasks.workunit.client.1.smithi150.stdout:6/954: write db/d52/d9f/dc6/df1/db4/dfd/d129/d13a/f14d [1034371,4813] 0 2024-03-22T10:43:10.725 INFO:tasks.workunit.client.1.smithi150.stdout:6/955: fdatasync db/d52/d9f/dc6/f14e 0 2024-03-22T10:43:10.733 INFO:tasks.workunit.client.1.smithi150.stdout:6/956: mkdir db/d52/d9f/dc6/df1/db4/dfd/d129/d14c/d150 0 2024-03-22T10:43:10.737 INFO:tasks.workunit.client.1.smithi150.stdout:0/962: mknod d0/d1/d84/d86/d42/d10/d38/d52/d9b/db4/c130 0 2024-03-22T10:43:10.740 INFO:tasks.workunit.client.1.smithi150.stdout:6/957: symlink db/d52/d9f/dc6/d133/l151 0 2024-03-22T10:43:10.741 INFO:tasks.workunit.client.1.smithi150.stdout:8/967: dwrite d0/d3c/f44 [0,4194304] 0 2024-03-22T10:43:10.741 INFO:tasks.workunit.client.1.smithi150.stdout:7/959: dwrite d2/d5/d7/d9/db/dfb/d13a/d45/fe3 [0,4194304] 0 2024-03-22T10:43:10.744 INFO:tasks.workunit.client.1.smithi150.stdout:0/963: write d0/d1/d84/d86/d42/d10/d38/d79/fbe [373166,111698] 0 2024-03-22T10:43:10.753 INFO:tasks.workunit.client.1.smithi150.stdout:6/958: unlink db/dd/d15/d60/f86 0 2024-03-22T10:43:10.754 INFO:tasks.workunit.client.1.smithi150.stdout:0/964: creat d0/d1/d84/d86/d5a/f131 x:0 0 0 2024-03-22T10:43:10.754 INFO:tasks.workunit.client.1.smithi150.stdout:0/965: chown d0/d1 151314 1 2024-03-22T10:43:10.755 INFO:tasks.workunit.client.1.smithi150.stdout:7/960: fsync d2/d5/d7/d9/db/d6c/d9c/fd1 0 2024-03-22T10:43:10.757 INFO:tasks.workunit.client.1.smithi150.stdout:8/968: creat d0/d2/d46/de3/d13d/f146 x:0 0 0 2024-03-22T10:43:10.759 INFO:tasks.workunit.client.1.smithi150.stdout:8/969: stat d0/d2/d19/f78 0 2024-03-22T10:43:10.760 INFO:tasks.workunit.client.1.smithi150.stdout:7/961: unlink d2/d5/d7/d8/d57/d4a/d8d/cb8 0 2024-03-22T10:43:10.761 INFO:tasks.workunit.client.1.smithi150.stdout:0/966: creat d0/d1/d84/d86/d11/d26/d76/d115/f132 x:0 0 0 2024-03-22T10:43:10.764 INFO:tasks.workunit.client.1.smithi150.stdout:7/962: rmdir d2/d5/d7/d8/d57/d4a 39 2024-03-22T10:43:10.767 INFO:tasks.workunit.client.1.smithi150.stdout:8/970: symlink d0/d2/d19/d29/d37/dc2/d107/l147 0 2024-03-22T10:43:10.772 INFO:tasks.workunit.client.1.smithi150.stdout:8/971: symlink d0/d2/d46/l148 0 2024-03-22T10:43:10.774 INFO:tasks.workunit.client.1.smithi150.stdout:6/959: dwrite db/dd/d15/d28/d6c/d9b/f142 [0,4194304] 0 2024-03-22T10:43:10.781 INFO:tasks.workunit.client.1.smithi150.stdout:6/960: rename db/dd/d15/d60/d50/l88 to db/dd/d15/d30/ddf/d113/l152 0 2024-03-22T10:43:10.784 INFO:tasks.workunit.client.1.smithi150.stdout:8/972: getdents d0/d2/d19/d29/d37/d7d/d9e 0 2024-03-22T10:43:10.784 INFO:tasks.workunit.client.1.smithi150.stdout:6/961: dread - db/d52/d9f/fe1 zero size 2024-03-22T10:43:10.786 INFO:tasks.workunit.client.1.smithi150.stdout:7/963: write d2/d5/d7/d9/db/f76 [3436814,24565] 0 2024-03-22T10:43:10.786 INFO:tasks.workunit.client.1.smithi150.stdout:0/967: write d0/d1/d84/d86/d42/d10/d38/d52/d68/fb0 [966882,50330] 0 2024-03-22T10:43:10.790 INFO:tasks.workunit.client.1.smithi150.stdout:7/964: chown d2/d5/d7/d8/d57/f3c 444969726 1 2024-03-22T10:43:10.790 INFO:tasks.workunit.client.1.smithi150.stdout:0/968: fdatasync d0/d1/d84/d86/d11/f117 0 2024-03-22T10:43:10.794 INFO:tasks.workunit.client.1.smithi150.stdout:6/962: stat db/dd/d15/d30/ddf/d113/f13d 0 2024-03-22T10:43:10.795 INFO:tasks.workunit.client.1.smithi150.stdout:0/969: mkdir d0/d1/d84/d86/d5a/d133 0 2024-03-22T10:43:10.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.803+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fefe01a24e0 con 0x7fefe0074d80 2024-03-22T10:43:10.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.804+0000 7fefd6a00700 1 -- 172.21.15.115:0/2047156564 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fefd005c060 con 0x7fefe0074d80 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:43:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:43:10.798 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:43:10.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:43:10.798 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:43:10.798 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-03-22T10:43:10.798 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:43:10.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:43:10.798 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:43:10.799 INFO:tasks.workunit.client.1.smithi150.stdout:6/963: truncate db/d52/fb1 330737 0 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.807+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fefc406ddf0 msgr2=0x7fefc40702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.807+0000 7fefe5200700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fefc406ddf0 0x7fefc40702a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fefcc006fd0 tx=0x7fefcc009190 comp rx=0 tx=0).stop 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.807+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe0074d80 msgr2=0x7fefe01039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.807+0000 7fefe5200700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe0074d80 0x7fefe01039c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fefd00055e0 tx=0x7fefd0005610 comp rx=0 tx=0).stop 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.807+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 shutdown_connections 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.808+0000 7fefe5200700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fefc406ddf0 0x7fefc40702a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.808+0000 7fefe5200700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefe00737b0 0x7fefe0103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.808+0000 7fefe5200700 1 --2- 172.21.15.115:0/2047156564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefe0074d80 0x7fefe01039c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.808+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 >> 172.21.15.115:0/2047156564 conn(0x7fefe00fb6e0 msgr2=0x7fefe0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.808+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 shutdown_connections 2024-03-22T10:43:10.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.808+0000 7fefe5200700 1 -- 172.21.15.115:0/2047156564 wait complete. 2024-03-22T10:43:10.801 INFO:tasks.workunit.client.1.smithi150.stdout:0/970: chown d0/d1/d84/d86/d11/d66/d35/d5f/ca4 1325199 1 2024-03-22T10:43:10.801 INFO:tasks.workunit.client.1.smithi150.stdout:7/965: write d2/d5/d7/d8/d57/d4a/f33 [857886,28882] 0 2024-03-22T10:43:10.803 INFO:tasks.workunit.client.1.smithi150.stdout:6/964: chown db/d52/d9f/dc6/d133/l151 10680506 1 2024-03-22T10:43:10.805 INFO:tasks.workunit.client.1.smithi150.stdout:7/966: readlink d2/d5/d7/d9/db/dfb/d13a/d96/l97 0 2024-03-22T10:43:10.806 INFO:tasks.workunit.client.1.smithi150.stdout:0/971: unlink d0/d1/d84/fcf 0 2024-03-22T10:43:10.807 INFO:tasks.workunit.client.1.smithi150.stdout:6/965: rmdir db/dd/d15/d60 39 2024-03-22T10:43:10.809 INFO:tasks.workunit.client.1.smithi150.stdout:8/973: dwrite d0/d3c/d71/d7a/d89/d143/f105 [0,4194304] 0 2024-03-22T10:43:10.810 INFO:tasks.workunit.client.1.smithi150.stdout:0/972: chown d0/d1/d84/d86/da6/lf7 399 1 2024-03-22T10:43:10.812 INFO:tasks.workunit.client.1.smithi150.stdout:6/966: write db/dd/d15/d30/f7c [490997,53829] 0 2024-03-22T10:43:10.817 INFO:tasks.workunit.client.1.smithi150.stdout:0/973: write d0/d1/d84/d86/d42/d10/d38/d52/fbd [1520222,59693] 0 2024-03-22T10:43:10.828 INFO:tasks.workunit.client.1.smithi150.stdout:7/967: dread d2/d5/d7/fc1 [0,4194304] 0 2024-03-22T10:43:10.830 INFO:tasks.workunit.client.1.smithi150.stdout:8/974: write d0/d5d/ffd [503980,45888] 0 2024-03-22T10:43:10.830 INFO:tasks.workunit.client.1.smithi150.stdout:6/967: write db/d52/fc9 [1165806,96552] 0 2024-03-22T10:43:10.831 INFO:tasks.workunit.client.1.smithi150.stdout:7/968: readlink d2/d5/l87 0 2024-03-22T10:43:10.834 INFO:tasks.workunit.client.1.smithi150.stdout:6/968: stat db/d52/d9f/dc6/c123 0 2024-03-22T10:43:10.840 INFO:tasks.workunit.client.1.smithi150.stdout:7/969: rename d2/d5/d7/d9/db/dfb/d13a/d96/d71/d106/f5f to d2/d5/d125/f141 0 2024-03-22T10:43:10.845 INFO:tasks.workunit.client.1.smithi150.stdout:7/970: fsync d2/d124/fae 0 2024-03-22T10:43:10.853 INFO:tasks.workunit.client.1.smithi150.stdout:0/974: dread d0/d1/d84/ff8 [0,4194304] 0 2024-03-22T10:43:10.853 INFO:tasks.workunit.client.1.smithi150.stdout:7/971: write d2/d5/d7/d9/db/dfb/d13a/d96/d71/d106/d91/f114 [2138610,122427] 0 2024-03-22T10:43:10.854 INFO:tasks.workunit.client.1.smithi150.stdout:8/975: dwrite d0/d3c/d71/d7a/d89/d143/dec/f114 [0,4194304] 0 2024-03-22T10:43:10.855 INFO:tasks.workunit.client.1.smithi150.stdout:6/969: dwrite db/dd/d15/d70/ff5 [0,4194304] 0 2024-03-22T10:43:10.857 INFO:tasks.workunit.client.1.smithi150.stdout:8/976: chown d0/f1 3831871 1 2024-03-22T10:43:10.869 INFO:tasks.workunit.client.1.smithi150.stdout:7/972: creat d2/d5/d7/dab/dec/f142 x:0 0 0 2024-03-22T10:43:10.872 INFO:tasks.workunit.client.1.smithi150.stdout:6/970: symlink db/d52/d9f/dc6/dd0/l153 0 2024-03-22T10:43:10.878 INFO:tasks.workunit.client.1.smithi150.stdout:6/971: creat db/d52/d79/f154 x:0 0 0 2024-03-22T10:43:10.881 INFO:tasks.workunit.client.1.smithi150.stdout:7/973: truncate d2/d5/d7/d9/db/dfb/d13a/d96/d71/d106/d91/f13e 389420 0 2024-03-22T10:43:10.881 INFO:tasks.workunit.client.1.smithi150.stdout:8/977: dwrite d0/d5d/dd2/fb4 [4194304,4194304] 0 2024-03-22T10:43:10.885 INFO:tasks.workunit.client.1.smithi150.stdout:6/972: mkdir db/dd/d15/d28/d6c/d9b/d155 0 2024-03-22T10:43:10.889 INFO:tasks.workunit.client.1.smithi150.stdout:8/978: chown d0/d3c/d71/d7a/d89/d143/cf0 24 1 2024-03-22T10:43:10.889 INFO:tasks.workunit.client.1.smithi150.stdout:0/975: dread d0/d12/fcd [0,4194304] 0 2024-03-22T10:43:10.889 INFO:tasks.workunit.client.1.smithi150.stdout:6/973: dread - db/dd/d15/d60/d8b/f92 zero size 2024-03-22T10:43:10.894 INFO:tasks.workunit.client.1.smithi150.stdout:8/979: symlink d0/d3c/d71/d7a/d89/d12d/dda/l149 0 2024-03-22T10:43:10.896 INFO:tasks.workunit.client.1.smithi150.stdout:6/974: rename db/dd/d15/d60/c8f to db/dd/d15/d28/c156 0 2024-03-22T10:43:10.898 INFO:tasks.workunit.client.1.smithi150.stdout:0/976: creat d0/d1/d84/d86/d11/d26/dd2/df0/d12b/f134 x:0 0 0 2024-03-22T10:43:10.900 INFO:tasks.workunit.client.1.smithi150.stdout:7/974: truncate d2/d5/d7/d9/db/d6c/fa1 396923 0 2024-03-22T10:43:10.900 INFO:tasks.workunit.client.1.smithi150.stdout:8/980: creat d0/d3c/d71/d7a/d89/d113/f14a x:0 0 0 2024-03-22T10:43:10.904 INFO:tasks.workunit.client.1.smithi150.stdout:7/975: unlink d2/d5/d7/d38/f131 0 2024-03-22T10:43:10.907 INFO:tasks.workunit.client.1.smithi150.stdout:0/977: creat d0/d1/d84/d86/d42/d10/d38/d52/d9b/f135 x:0 0 0 2024-03-22T10:43:10.908 INFO:tasks.workunit.client.1.smithi150.stdout:8/981: rename d0/d2/d19/d29/d2e/c93 to d0/d2/d19/dd1/c14b 0 2024-03-22T10:43:10.910 INFO:tasks.workunit.client.1.smithi150.stdout:0/978: chown d0/f56 444 1 2024-03-22T10:43:10.911 INFO:tasks.workunit.client.1.smithi150.stdout:7/976: truncate d2/d5/d7/d88/fdb 482041 0 2024-03-22T10:43:10.912 INFO:tasks.workunit.client.1.smithi150.stdout:0/979: fdatasync d0/d1/d84/d86/d5a/f131 0 2024-03-22T10:43:10.913 INFO:tasks.workunit.client.1.smithi150.stdout:6/975: dwrite db/dd/d15/d60/f9c [4194304,4194304] 0 2024-03-22T10:43:10.920 INFO:tasks.workunit.client.1.smithi150.stdout:6/976: mknod db/d52/d9f/dc6/df1/db4/dfd/c157 0 2024-03-22T10:43:10.922 INFO:tasks.workunit.client.1.smithi150.stdout:7/977: mkdir d2/d5/d7/d8/d57/d4a/d143 0 2024-03-22T10:43:10.923 INFO:tasks.workunit.client.1.smithi150.stdout:6/977: dread - db/d52/d79/f128 zero size 2024-03-22T10:43:10.931 INFO:tasks.workunit.client.1.smithi150.stdout:7/978: mkdir d2/d5/d7/d8/d57/d4a/d111/d144 0 2024-03-22T10:43:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.991+0000 7fba4f200700 1 -- 172.21.15.115:0/939450601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 msgr2=0x7fba501003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.991+0000 7fba4f200700 1 --2- 172.21.15.115:0/939450601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 0x7fba501003a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fba40009910 tx=0x7fba40009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.991+0000 7fba4f200700 1 -- 172.21.15.115:0/939450601 shutdown_connections 2024-03-22T10:43:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.991+0000 7fba4f200700 1 --2- 172.21.15.115:0/939450601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba50101190 0x7fba501015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.991+0000 7fba4f200700 1 --2- 172.21.15.115:0/939450601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 0x7fba501003a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.991+0000 7fba4f200700 1 -- 172.21.15.115:0/939450601 >> 172.21.15.115:0/939450601 conn(0x7fba500fb500 msgr2=0x7fba500fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:10.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.992+0000 7fba4f200700 1 -- 172.21.15.115:0/939450601 shutdown_connections 2024-03-22T10:43:10.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.992+0000 7fba4f200700 1 -- 172.21.15.115:0/939450601 wait complete. 2024-03-22T10:43:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:10 smithi115 ceph-mon[29111]: Reconfiguring prometheus.smithi115 (dependencies changed)... 2024-03-22T10:43:10.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:10 smithi115 ceph-mon[29111]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:10.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:10 smithi115 ceph-mon[29111]: Reconfiguring daemon prometheus.smithi115 on smithi115 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.993+0000 7fba4f200700 1 Processor -- start 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.993+0000 7fba4f200700 1 -- start start 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4f200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 0x7fba50074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4f200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba50101190 0x7fba50073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4f200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba50199420 con 0x7fba500fff90 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4f200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba50199560 con 0x7fba50101190 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4de00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 0x7fba50074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4de00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 0x7fba50074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53286/0 (socket says 172.21.15.115:53286) 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4de00700 1 -- 172.21.15.115:0/1119463194 learned_addr learned my addr 172.21.15.115:0/1119463194 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4d400700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba50101190 0x7fba50073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:10.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4de00700 1 -- 172.21.15.115:0/1119463194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba50101190 msgr2=0x7fba50073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:10.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4de00700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba50101190 0x7fba50073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:10.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.994+0000 7fba4de00700 1 -- 172.21.15.115:0/1119463194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba400095f0 con 0x7fba500fff90 2024-03-22T10:43:10.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.995+0000 7fba4de00700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 0x7fba50074af0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fba4000f690 tx=0x7fba4000f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:10.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.995+0000 7fba46a00700 1 -- 172.21.15.115:0/1119463194 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba4001d070 con 0x7fba500fff90 2024-03-22T10:43:10.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.995+0000 7fba46a00700 1 -- 172.21.15.115:0/1119463194 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fba40005dc0 con 0x7fba500fff90 2024-03-22T10:43:10.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.995+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba50073680 con 0x7fba500fff90 2024-03-22T10:43:10.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.995+0000 7fba46a00700 1 -- 172.21.15.115:0/1119463194 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba40017630 con 0x7fba500fff90 2024-03-22T10:43:10.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.995+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba50073b40 con 0x7fba500fff90 2024-03-22T10:43:10.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.997+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba30005320 con 0x7fba500fff90 2024-03-22T10:43:10.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.998+0000 7fba46a00700 1 -- 172.21.15.115:0/1119463194 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fba4000fb20 con 0x7fba500fff90 2024-03-22T10:43:10.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:10.999+0000 7fba46a00700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fba3406dd50 0x7fba34070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:10.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.000+0000 7fba46a00700 1 -- 172.21.15.115:0/1119463194 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fba4008ef00 con 0x7fba500fff90 2024-03-22T10:43:10.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.000+0000 7fba4d400700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fba3406dd50 0x7fba34070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:10.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.001+0000 7fba4d400700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fba3406dd50 0x7fba34070200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009c60 tx=0x7fba3c009260 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:10.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.004+0000 7fba46a00700 1 -- 172.21.15.115:0/1119463194 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fba4005c110 con 0x7fba500fff90 2024-03-22T10:43:11.008 INFO:tasks.workunit.client.1.smithi150.stdout:7/979: readlink d2/d5/d7/d9/db/da2/le6 0 2024-03-22T10:43:11.008 INFO:tasks.workunit.client.1.smithi150.stdout:8/982: dwrite d0/d2/f12 [0,4194304] 0 2024-03-22T10:43:11.012 INFO:tasks.workunit.client.1.smithi150.stdout:7/980: mknod d2/d5/d7/d9/db/da2/c145 0 2024-03-22T10:43:11.012 INFO:tasks.workunit.client.1.smithi150.stdout:8/983: chown d0/d3c/d71/d7a/d89/d12d/dda/f122 66231569 1 2024-03-22T10:43:11.019 INFO:tasks.workunit.client.1.smithi150.stdout:8/984: mkdir d0/d2/d46/de3/d14c 0 2024-03-22T10:43:11.024 INFO:tasks.workunit.client.1.smithi150.stdout:8/985: creat d0/d3c/d13c/f14d x:0 0 0 2024-03-22T10:43:11.032 INFO:tasks.workunit.client.1.smithi150.stdout:6/978: dread db/dd/d15/d60/d50/f64 [4194304,4194304] 0 2024-03-22T10:43:11.035 INFO:tasks.workunit.client.1.smithi150.stdout:8/986: rename d0/d2/d19/dd1/f101 to d0/d2/d19/d29/d37/d7d/f14e 0 2024-03-22T10:43:11.038 INFO:tasks.workunit.client.1.smithi150.stdout:6/979: mkdir db/dd/d15/d28/d6c/d158 0 2024-03-22T10:43:11.046 INFO:tasks.workunit.client.1.smithi150.stdout:6/980: mknod db/dd/d15/d28/d6c/c159 0 2024-03-22T10:43:11.046 INFO:tasks.workunit.client.1.smithi150.stdout:7/981: dwrite d2/d5/d7/d8/d57/f5b [0,4194304] 0 2024-03-22T10:43:11.048 INFO:tasks.workunit.client.1.smithi150.stdout:6/981: chown db/d52/f6f 27684468 1 2024-03-22T10:43:11.050 INFO:tasks.workunit.client.1.smithi150.stdout:7/982: mknod d2/d5/d7/d8/d57/d4a/d8d/da9/c146 0 2024-03-22T10:43:11.051 INFO:tasks.workunit.client.1.smithi150.stdout:6/982: creat db/dd/d15/d70/f15a x:0 0 0 2024-03-22T10:43:11.055 INFO:tasks.workunit.client.1.smithi150.stdout:8/987: dwrite d0/d3c/fed [0,4194304] 0 2024-03-22T10:43:11.055 INFO:tasks.workunit.client.1.smithi150.stdout:7/983: mknod d2/d5/d7/d9/db/d6c/c147 0 2024-03-22T10:43:11.056 INFO:tasks.workunit.client.1.smithi150.stdout:6/983: symlink db/dd/d15/d6a/l15b 0 2024-03-22T10:43:11.062 INFO:tasks.workunit.client.1.smithi150.stdout:6/984: symlink db/dd/d82/d84/dc8/l15c 0 2024-03-22T10:43:11.067 INFO:tasks.workunit.client.1.smithi150.stdout:6/985: mknod db/dd/d15/d70/c15d 0 2024-03-22T10:43:11.089 INFO:tasks.workunit.client.1.smithi150.stdout:8/988: dread d0/d2/f5f [0,4194304] 0 2024-03-22T10:43:11.093 INFO:tasks.workunit.client.1.smithi150.stdout:6/986: dwrite db/d52/d9f/dc6/df1/ff9 [0,4194304] 0 2024-03-22T10:43:11.097 INFO:tasks.workunit.client.1.smithi150.stdout:7/984: dread d2/d5/d7/f50 [4194304,4194304] 0 2024-03-22T10:43:11.097 INFO:tasks.workunit.client.1.smithi150.stdout:6/987: creat db/d52/d9f/f15e x:0 0 0 2024-03-22T10:43:11.097 INFO:tasks.workunit.client.1.smithi150.stdout:8/989: truncate d0/d2/d19/d29/d2f/ff9 421390 0 2024-03-22T10:43:11.099 INFO:tasks.workunit.client.1.smithi150.stdout:7/985: truncate d2/fcd 802016 0 2024-03-22T10:43:11.100 INFO:tasks.workunit.client.1.smithi150.stdout:6/988: symlink db/dd/d15/d60/d8b/de7/l15f 0 2024-03-22T10:43:11.103 INFO:tasks.workunit.client.1.smithi150.stdout:8/990: unlink d0/d2/d19/d29/d2f/df/dff/d110/d96/f99 0 2024-03-22T10:43:11.107 INFO:tasks.workunit.client.1.smithi150.stdout:8/991: dread d0/d2/d19/d29/d2f/df/d1b/fa0 [0,4194304] 0 2024-03-22T10:43:11.109 INFO:tasks.workunit.client.1.smithi150.stdout:6/989: rename db/dd/d15/d60/d50/ffa to db/d52/d9f/dc6/df1/db4/dfd/d129/d13a/f160 0 2024-03-22T10:43:11.110 INFO:tasks.workunit.client.1.smithi150.stdout:8/992: fdatasync d0/f1 0 2024-03-22T10:43:11.113 INFO:tasks.workunit.client.1.smithi150.stdout:6/990: creat db/dd/f161 x:0 0 0 2024-03-22T10:43:11.113 INFO:tasks.workunit.client.1.smithi150.stdout:8/993: mknod d0/d3c/d13c/c14f 0 2024-03-22T10:43:11.114 INFO:tasks.workunit.client.1.smithi150.stdout:6/991: chown db/dd/l11 4484 1 2024-03-22T10:43:11.116 INFO:tasks.workunit.client.1.smithi150.stdout:3/967: sync 2024-03-22T10:43:11.116 INFO:tasks.workunit.client.1.smithi150.stdout:1/929: sync 2024-03-22T10:43:11.116 INFO:tasks.workunit.client.1.smithi150.stdout:9/967: sync 2024-03-22T10:43:11.117 INFO:tasks.workunit.client.1.smithi150.stdout:5/970: sync 2024-03-22T10:43:11.117 INFO:tasks.workunit.client.1.smithi150.stdout:6/992: write db/d52/d9f/fc0 [4593026,107997] 0 2024-03-22T10:43:11.118 INFO:tasks.workunit.client.1.smithi150.stdout:8/994: write d0/d2/d19/d29/d37/d7d/dc1/ff4 [1060990,77838] 0 2024-03-22T10:43:11.123 INFO:tasks.workunit.client.1.smithi150.stdout:3/968: creat d5/d22/dc8/dda/d95/f147 x:0 0 0 2024-03-22T10:43:11.123 INFO:tasks.workunit.client.1.smithi150.stdout:1/930: read d1/d19/d50/d51/d123/d12d/f60 [666891,104915] 0 2024-03-22T10:43:11.124 INFO:tasks.workunit.client.1.smithi150.stdout:8/995: readlink d0/d2/d46/da2/lb2 0 2024-03-22T10:43:11.125 INFO:tasks.workunit.client.1.smithi150.stdout:1/931: chown d1/c108 3 1 2024-03-22T10:43:11.128 INFO:tasks.workunit.client.1.smithi150.stdout:5/971: creat d0/d8/dc/de/d14/f156 x:0 0 0 2024-03-22T10:43:11.129 INFO:tasks.workunit.client.1.smithi150.stdout:6/993: unlink db/d52/d9f/dc6/df1/fbb 0 2024-03-22T10:43:11.133 INFO:tasks.workunit.client.1.smithi150.stdout:1/932: truncate d1/d19/d1a/f1e 645515 0 2024-03-22T10:43:11.135 INFO:tasks.workunit.client.1.smithi150.stdout:5/972: creat d0/d8/dc/de/d14/d2e/d45/dac/f157 x:0 0 0 2024-03-22T10:43:11.137 INFO:tasks.workunit.client.1.smithi150.stdout:8/996: dread d0/d5d/dd9/fe9 [0,4194304] 0 2024-03-22T10:43:11.139 INFO:tasks.workunit.client.1.smithi150.stdout:8/997: write d0/d2/f117 [33237,109545] 0 2024-03-22T10:43:11.143 INFO:tasks.workunit.client.1.smithi150.stdout:8/998: readlink d0/d2/d19/d29/d2f/df/dff/d110/lc3 0 2024-03-22T10:43:11.145 INFO:tasks.workunit.client.1.smithi150.stdout:9/968: getdents d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99 0 2024-03-22T10:43:11.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:10 smithi150 ceph-mon[34518]: Reconfiguring prometheus.smithi115 (dependencies changed)... 2024-03-22T10:43:11.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:10 smithi150 ceph-mon[34518]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:11.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:10 smithi150 ceph-mon[34518]: Reconfiguring daemon prometheus.smithi115 on smithi115 2024-03-22T10:43:11.150 INFO:tasks.workunit.client.1.smithi150.stdout:9/969: chown d1/d8/de/cae 690 1 2024-03-22T10:43:11.151 INFO:tasks.workunit.client.1.smithi150.stdout:5/973: write d0/f38 [688022,106991] 0 2024-03-22T10:43:11.152 INFO:tasks.workunit.client.1.smithi150.stdout:6/994: link db/dd/d15/d60/d8b/de7/c14a db/dd/d15/d30/ddf/d113/c162 0 2024-03-22T10:43:11.156 INFO:tasks.workunit.client.1.smithi150.stdout:5/974: unlink d0/d37/l60 0 2024-03-22T10:43:11.156 INFO:tasks.workunit.client.1.smithi150.stdout:3/969: dwrite d5/da/d9d/fc5 [0,4194304] 0 2024-03-22T10:43:11.159 INFO:tasks.workunit.client.1.smithi150.stdout:1/933: dwrite d1/f3a [0,4194304] 0 2024-03-22T10:43:11.162 INFO:tasks.workunit.client.1.smithi150.stdout:5/975: stat d0/d8/dc/de/d14/d2e/d45/db3/ce2 0 2024-03-22T10:43:11.164 INFO:tasks.workunit.client.1.smithi150.stdout:1/934: readlink d1/d19/d50/d51/da0/dc4/d129/d2a/l119 0 2024-03-22T10:43:11.166 INFO:tasks.workunit.client.1.smithi150.stdout:6/995: chown db/dd/d15/d28/d6c/c141 268 1 2024-03-22T10:43:11.166 INFO:tasks.workunit.client.1.smithi150.stdout:3/970: write d5/d22/d2f/d38/d3e/d5b/da6/fbd [450629,101321] 0 2024-03-22T10:43:11.166 INFO:tasks.workunit.client.1.smithi150.stdout:8/999: link d0/d2/d19/d29/d37/l84 d0/d2/d19/d29/d2f/df/d1b/l150 0 2024-03-22T10:43:11.174 INFO:tasks.workunit.client.1.smithi150.stdout:1/935: truncate d1/d19/d50/d51/da0/dc4/d129/d2a/fce 247940 0 2024-03-22T10:43:11.174 INFO:tasks.workunit.client.1.smithi150.stdout:6/996: read db/d52/f6f [3648816,51692] 0 2024-03-22T10:43:11.179 INFO:tasks.workunit.client.1.smithi150.stdout:6/997: mknod db/dd/d15/d28/d6c/d134/d13e/c163 0 2024-03-22T10:43:11.182 INFO:tasks.workunit.client.1.smithi150.stdout:9/970: dwrite d1/d8/de/df/f11 [0,4194304] 0 2024-03-22T10:43:11.193 INFO:tasks.workunit.client.1.smithi150.stdout:6/998: rmdir db/dd/d15/d28/d6c/d158 0 2024-03-22T10:43:11.194 INFO:tasks.workunit.client.1.smithi150.stdout:9/971: chown d1/d8/de/df/d36/d40/d44/db5/f113 21 1 2024-03-22T10:43:11.194 INFO:tasks.workunit.client.1.smithi150.stdout:3/971: dwrite d5/f35 [0,4194304] 0 2024-03-22T10:43:11.196 INFO:tasks.workunit.client.1.smithi150.stdout:5/976: dwrite d0/d8/dc/de/d14/d2e/d5a/ffe [0,4194304] 0 2024-03-22T10:43:11.201 INFO:tasks.workunit.client.1.smithi150.stdout:6/999: creat db/d52/d9f/dc6/df1/f164 x:0 0 0 2024-03-22T10:43:11.204 INFO:tasks.workunit.client.1.smithi150.stdout:3/972: truncate d5/f4f 103852 0 2024-03-22T10:43:11.205 INFO:tasks.workunit.client.1.smithi150.stdout:1/936: dwrite d1/fa6 [0,4194304] 0 2024-03-22T10:43:11.205 INFO:tasks.workunit.client.1.smithi150.stdout:9/972: symlink d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/l144 0 2024-03-22T10:43:11.210 INFO:tasks.workunit.client.1.smithi150.stdout:9/973: fsync d1/d8/de/dbd/f13f 0 2024-03-22T10:43:11.210 INFO:tasks.workunit.client.1.smithi150.stdout:5/977: dread d0/d37/db4/f12f [0,4194304] 0 2024-03-22T10:43:11.216 INFO:tasks.workunit.client.1.smithi150.stdout:9/974: write d1/d8/de/df/d36/d121/d4e/f93 [2821632,105458] 0 2024-03-22T10:43:11.220 INFO:tasks.workunit.client.1.smithi150.stdout:3/973: link d5/d22/c32 d5/d22/dcb/c148 0 2024-03-22T10:43:11.222 INFO:tasks.workunit.client.1.smithi150.stdout:5/978: link d0/dae/db2/c4e d0/d8/dc/de/d14/d2e/d46/d9c/dc0/c158 0 2024-03-22T10:43:11.224 INFO:tasks.workunit.client.1.smithi150.stdout:1/937: getdents d1/d19/d50/d51/da0/dc4/d129/d15/d18/d121/de4 0 2024-03-22T10:43:11.233 INFO:tasks.workunit.client.1.smithi150.stdout:1/938: creat d1/d19/d50/d51/da0/dc4/d129/d15/d18/d4b/dc8/f138 x:0 0 0 2024-03-22T10:43:11.234 INFO:tasks.workunit.client.1.smithi150.stdout:0/980: sync 2024-03-22T10:43:11.234 INFO:tasks.workunit.client.1.smithi150.stdout:1/939: fsync d1/d19/d1a/f49 0 2024-03-22T10:43:11.237 INFO:tasks.workunit.client.1.smithi150.stdout:7/986: sync 2024-03-22T10:43:11.237 INFO:tasks.workunit.client.1.smithi150.stdout:0/981: symlink d0/d1/d84/l136 0 2024-03-22T10:43:11.243 INFO:tasks.workunit.client.1.smithi150.stdout:3/974: write d5/da/f63 [2177310,73869] 0 2024-03-22T10:43:11.245 INFO:tasks.workunit.client.1.smithi150.stdout:7/987: mknod d2/d5/d7/d8/d57/d4a/d8d/da9/c148 0 2024-03-22T10:43:11.246 INFO:tasks.workunit.client.1.smithi150.stdout:5/979: truncate d0/d8/dc/de/f82 2135889 0 2024-03-22T10:43:11.246 INFO:tasks.workunit.client.1.smithi150.stdout:9/975: unlink d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7/f107 0 2024-03-22T10:43:11.248 INFO:tasks.workunit.client.1.smithi150.stdout:0/982: dread d0/d1/f39 [4194304,4194304] 0 2024-03-22T10:43:11.255 INFO:tasks.workunit.client.1.smithi150.stdout:0/983: symlink d0/d1/d84/d86/d42/d10/d38/d52/d9b/dc2/l137 0 2024-03-22T10:43:11.257 INFO:tasks.workunit.client.1.smithi150.stdout:1/940: dwrite d1/d19/d50/f5c [0,4194304] 0 2024-03-22T10:43:11.257 INFO:tasks.workunit.client.1.smithi150.stdout:5/980: fsync d0/f11 0 2024-03-22T10:43:11.261 INFO:tasks.workunit.client.1.smithi150.stdout:1/941: write d1/d19/d50/d51/f6b [4383283,25241] 0 2024-03-22T10:43:11.264 INFO:tasks.workunit.client.1.smithi150.stdout:0/984: symlink d0/d1/d84/d86/d11/d26/d76/l138 0 2024-03-22T10:43:11.265 INFO:tasks.workunit.client.1.smithi150.stdout:5/981: mkdir d0/d8/dc/d159 0 2024-03-22T10:43:11.272 INFO:tasks.workunit.client.1.smithi150.stdout:3/975: dwrite d5/d22/d2f/d38/d3e/d5b/fd4 [0,4194304] 0 2024-03-22T10:43:11.276 INFO:tasks.workunit.client.1.smithi150.stdout:0/985: rename d0/d1/d84/d86/d42/d10/d53/f5e to d0/d1/d84/d86/d11/d66/d37/f139 0 2024-03-22T10:43:11.285 INFO:tasks.workunit.client.1.smithi150.stdout:3/976: symlink d5/d22/d2f/d38/d138/l149 0 2024-03-22T10:43:11.291 INFO:tasks.workunit.client.1.smithi150.stdout:5/982: link d0/d8/dc/d1d/d89/fc4 d0/d8/dc/de/d14/d2e/f15a 0 2024-03-22T10:43:11.292 INFO:tasks.workunit.client.1.smithi150.stdout:3/977: chown d5/d53/fb3 118030887 1 2024-03-22T10:43:11.294 INFO:tasks.workunit.client.1.smithi150.stdout:3/978: stat d5/d22/d2f/d38/d3e/d5b/ccc 0 2024-03-22T10:43:11.298 INFO:tasks.workunit.client.1.smithi150.stdout:5/983: read d0/d8/dc/de/d14/d4f/f3d [68486,52144] 0 2024-03-22T10:43:11.301 INFO:tasks.workunit.client.1.smithi150.stdout:3/979: truncate d5/d22/d2f/d38/d59/d88/f8a 1377708 0 2024-03-22T10:43:11.301 INFO:tasks.workunit.client.1.smithi150.stdout:5/984: chown d0/d8/dc/f40 377 1 2024-03-22T10:43:11.305 INFO:tasks.workunit.client.1.smithi150.stdout:3/980: mkdir d5/da/d109/d14a 0 2024-03-22T10:43:11.306 INFO:tasks.workunit.client.1.smithi150.stdout:0/986: dwrite d0/d1/d84/d86/d11/d26/dd2/df0/f10c [0,4194304] 0 2024-03-22T10:43:11.310 INFO:tasks.workunit.client.1.smithi150.stdout:1/942: dread d1/d19/d50/d51/ff4 [0,4194304] 0 2024-03-22T10:43:11.318 INFO:tasks.workunit.client.1.smithi150.stdout:3/981: write d5/da/f33 [1499721,24829] 0 2024-03-22T10:43:11.319 INFO:tasks.workunit.client.1.smithi150.stdout:1/943: readlink d1/d19/d50/d51/da0/dc4/d129/l25 0 2024-03-22T10:43:11.324 INFO:tasks.workunit.client.1.smithi150.stdout:3/982: mkdir d5/da/d8d/d14b 0 2024-03-22T10:43:11.326 INFO:tasks.workunit.client.1.smithi150.stdout:5/985: dwrite d0/d8/dc/de/f128 [0,4194304] 0 2024-03-22T10:43:11.329 INFO:tasks.workunit.client.1.smithi150.stdout:1/944: write d1/d19/d50/d51/da0/dc4/d129/f116 [169958,29873] 0 2024-03-22T10:43:11.329 INFO:tasks.workunit.client.1.smithi150.stdout:5/986: chown d0/d8/d101 554173 1 2024-03-22T10:43:11.332 INFO:tasks.workunit.client.1.smithi150.stdout:3/983: write d5/d22/d2f/d38/d59/d88/d10d/f136 [989599,115469] 0 2024-03-22T10:43:11.335 INFO:tasks.workunit.client.1.smithi150.stdout:5/987: write d0/dae/feb [5018267,129023] 0 2024-03-22T10:43:11.335 INFO:tasks.workunit.client.1.smithi150.stdout:0/987: dwrite d0/d1/d84/d86/d11/d26/f125 [0,4194304] 0 2024-03-22T10:43:11.338 INFO:tasks.workunit.client.1.smithi150.stdout:3/984: chown d5/da/d8d/l12f 1543 1 2024-03-22T10:43:11.341 INFO:tasks.workunit.client.1.smithi150.stdout:0/988: chown d0/d12/dd6 1468857685 1 2024-03-22T10:43:11.341 INFO:tasks.workunit.client.1.smithi150.stdout:5/988: rename d0/d33/d6f/l88 to d0/d33/l15b 0 2024-03-22T10:43:11.343 INFO:tasks.workunit.client.1.smithi150.stdout:3/985: rmdir d5/da/d78/d86 39 2024-03-22T10:43:11.348 INFO:tasks.workunit.client.1.smithi150.stdout:5/989: creat d0/d8/d101/d11b/d12e/f15c x:0 0 0 2024-03-22T10:43:11.353 INFO:tasks.workunit.client.1.smithi150.stdout:0/989: rename d0/d1/d84/d86/d42/f4a to d0/f13a 0 2024-03-22T10:43:11.357 INFO:tasks.workunit.client.1.smithi150.stdout:5/990: unlink d0/d8/dc/de/d14/d2e/d5a/cea 0 2024-03-22T10:43:11.359 INFO:tasks.workunit.client.1.smithi150.stdout:1/945: dwrite d1/d19/d50/f120 [0,4194304] 0 2024-03-22T10:43:11.366 INFO:tasks.workunit.client.1.smithi150.stdout:3/986: dwrite d5/d22/d2f/d38/d59/d88/d10d/f136 [0,4194304] 0 2024-03-22T10:43:11.367 INFO:tasks.workunit.client.1.smithi150.stdout:5/991: mkdir d0/d8/dc/de/d14/d2e/d5a/d66/da9/d15d 0 2024-03-22T10:43:11.368 INFO:tasks.workunit.client.1.smithi150.stdout:3/987: fsync d5/da/d62/dba/ff9 0 2024-03-22T10:43:11.370 INFO:tasks.workunit.client.1.smithi150.stdout:0/990: read d0/d12/fd8 [248900,94902] 0 2024-03-22T10:43:11.373 INFO:tasks.workunit.client.1.smithi150.stdout:3/988: chown d5/d22/d2f/d38/d3e/f55 87397 1 2024-03-22T10:43:11.377 INFO:tasks.workunit.client.1.smithi150.stdout:3/989: chown d5/d22/d2f/d38/d59/d88/d10d/f136 53660 1 2024-03-22T10:43:11.380 INFO:tasks.workunit.client.1.smithi150.stdout:3/990: chown d5/d22/d37/fe6 12652 1 2024-03-22T10:43:11.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.388+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fba30005cc0 con 0x7fba500fff90 2024-03-22T10:43:11.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.389+0000 7fba46a00700 1 -- 172.21.15.115:0/1119463194 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1856 (secure 0 0 0) 0x7fba4005bca0 con 0x7fba500fff90 2024-03-22T10:43:11.381 INFO:teuthology.orchestra.run.smithi115.stdout:e13 2024-03-22T10:43:11.381 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 13 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:41:04.070466+0000 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:43:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:43:11.383 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:11.384 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:11.385 INFO:tasks.workunit.client.1.smithi150.stdout:5/992: link d0/d8/dc/de/d14/d2e/d46/d4d/fb7 d0/d8/dc/de/d14/d2e/d46/d9c/dc0/f15e 0 2024-03-22T10:43:11.385 INFO:tasks.workunit.client.1.smithi150.stdout:0/991: mkdir d0/d1/d84/d86/d11/d66/d13b 0 2024-03-22T10:43:11.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fba3406dd50 msgr2=0x7fba34070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:11.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fba3406dd50 0x7fba34070200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009c60 tx=0x7fba3c009260 comp rx=0 tx=0).stop 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 msgr2=0x7fba50074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 0x7fba50074af0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fba4000f690 tx=0x7fba4000f6c0 comp rx=0 tx=0).stop 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 shutdown_connections 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fba3406dd50 0x7fba34070200 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba500fff90 0x7fba50074af0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 --2- 172.21.15.115:0/1119463194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba50101190 0x7fba50073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 >> 172.21.15.115:0/1119463194 conn(0x7fba500fb500 msgr2=0x7fba501043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 shutdown_connections 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.393+0000 7fba4f200700 1 -- 172.21.15.115:0/1119463194 wait complete. 2024-03-22T10:43:11.386 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 13 2024-03-22T10:43:11.387 INFO:tasks.workunit.client.1.smithi150.stdout:5/993: mkdir d0/d8/dc/de/d14/d4f/d134/d15f 0 2024-03-22T10:43:11.390 INFO:tasks.workunit.client.1.smithi150.stdout:3/991: link d5/d22/d2f/d38/d3e/d5b/da6/l124 d5/da/d8d/d14b/l14c 0 2024-03-22T10:43:11.391 INFO:tasks.workunit.client.1.smithi150.stdout:1/946: dwrite d1/d19/d50/d51/da0/dc4/d129/d15/d18/d4b/d8a/f113 [0,4194304] 0 2024-03-22T10:43:11.394 INFO:tasks.workunit.client.1.smithi150.stdout:5/994: dread - d0/d8/dc/de/d96/f10a zero size 2024-03-22T10:43:11.397 INFO:tasks.workunit.client.1.smithi150.stdout:3/992: rmdir d5/da/d9d/d9f/dc1 39 2024-03-22T10:43:11.399 INFO:tasks.workunit.client.1.smithi150.stdout:5/995: creat d0/ddc/f160 x:0 0 0 2024-03-22T10:43:11.401 INFO:tasks.workunit.client.1.smithi150.stdout:1/947: mkdir d1/d19/d50/d51/da0/dc4/d129/d2a/dc0/d139 0 2024-03-22T10:43:11.404 INFO:tasks.workunit.client.1.smithi150.stdout:1/948: chown d1/d7c/fdf 38885 1 2024-03-22T10:43:11.407 INFO:tasks.workunit.client.1.smithi150.stdout:5/996: rename d0/d8/dc/de/d14/d2e/d45/d102/d11c to d0/d8/dc/de/d14/d2e/d45/d102/d11c/d161 22 2024-03-22T10:43:11.407 INFO:tasks.workunit.client.1.smithi150.stdout:3/993: truncate d5/f4f 491009 0 2024-03-22T10:43:11.407 INFO:tasks.workunit.client.1.smithi150.stdout:1/949: fsync d1/d19/d50/d51/d123/f12a 0 2024-03-22T10:43:11.408 INFO:tasks.workunit.client.1.smithi150.stdout:0/992: dwrite d0/d1/d84/d86/d42/d10/d38/d52/f65 [4194304,4194304] 0 2024-03-22T10:43:11.416 INFO:tasks.workunit.client.1.smithi150.stdout:5/997: mknod d0/d8/dc/de/d14/d4f/d134/c162 0 2024-03-22T10:43:11.428 INFO:tasks.workunit.client.1.smithi150.stdout:1/950: dread d1/d19/d50/d51/da0/dc4/d129/d2a/d65/d78/fb1 [0,4194304] 0 2024-03-22T10:43:11.428 INFO:tasks.workunit.client.1.smithi150.stdout:0/993: dwrite d0/d1/d84/d86/f6b [4194304,4194304] 0 2024-03-22T10:43:11.430 INFO:tasks.workunit.client.1.smithi150.stdout:1/951: chown d1/d79/f131 3 1 2024-03-22T10:43:11.431 INFO:tasks.workunit.client.1.smithi150.stdout:3/994: dwrite d5/d22/d2f/d38/f85 [0,4194304] 0 2024-03-22T10:43:11.432 INFO:tasks.workunit.client.1.smithi150.stdout:0/994: truncate d0/d1/d84/d86/da6/fd3 1728171 0 2024-03-22T10:43:11.435 INFO:tasks.workunit.client.1.smithi150.stdout:3/995: symlink d5/da/d62/dba/dd5/l14d 0 2024-03-22T10:43:11.437 INFO:tasks.workunit.client.1.smithi150.stdout:1/952: chown d1/d19/d50/d51/da0/dc4/d129/f16 1 1 2024-03-22T10:43:11.439 INFO:tasks.workunit.client.1.smithi150.stdout:0/995: rmdir d0/d1/d84 39 2024-03-22T10:43:11.444 INFO:tasks.workunit.client.1.smithi150.stdout:3/996: creat d5/d22/d2f/d38/d59/dff/d133/d13e/f14e x:0 0 0 2024-03-22T10:43:11.445 INFO:tasks.workunit.client.1.smithi150.stdout:1/953: mknod d1/d19/d71/d10c/c13a 0 2024-03-22T10:43:11.447 INFO:tasks.workunit.client.1.smithi150.stdout:0/996: dread - d0/d1/d84/d86/d105/f108 zero size 2024-03-22T10:43:11.448 INFO:tasks.workunit.client.1.smithi150.stdout:3/997: creat d5/d22/d2f/d10c/f14f x:0 0 0 2024-03-22T10:43:11.449 INFO:tasks.workunit.client.1.smithi150.stdout:1/954: creat d1/d19/d1a/d7d/f13b x:0 0 0 2024-03-22T10:43:11.453 INFO:tasks.workunit.client.1.smithi150.stdout:3/998: mkdir d5/da/d78/db7/d106/d150 0 2024-03-22T10:43:11.455 INFO:tasks.workunit.client.1.smithi150.stdout:1/955: chown d1/d19/d50/d51/da0/dc4/d129/d15/ff9 988199 1 2024-03-22T10:43:11.457 INFO:tasks.workunit.client.1.smithi150.stdout:3/999: mkdir d5/d22/d2f/d38/d3e/d5b/d151 0 2024-03-22T10:43:11.461 INFO:tasks.workunit.client.1.smithi150.stdout:1/956: mknod d1/d19/d50/d51/da0/dc4/d129/d15/d18/d121/c13c 0 2024-03-22T10:43:11.465 INFO:tasks.workunit.client.1.smithi150.stdout:7/988: sync 2024-03-22T10:43:11.465 INFO:tasks.workunit.client.1.smithi150.stdout:9/976: sync 2024-03-22T10:43:11.468 INFO:tasks.workunit.client.1.smithi150.stdout:7/989: readlink d2/d5/d7/d9/db/dfb/d13a/d96/lac 0 2024-03-22T10:43:11.476 INFO:tasks.workunit.client.1.smithi150.stdout:0/997: dwrite d0/d1/d84/d86/da6/fd0 [0,4194304] 0 2024-03-22T10:43:11.478 INFO:tasks.workunit.client.1.smithi150.stdout:5/998: dread d0/d8/dc/de/d14/d4f/d134/f131 [0,4194304] 0 2024-03-22T10:43:11.478 INFO:tasks.workunit.client.1.smithi150.stdout:1/957: write d1/d19/f9e [998737,103056] 0 2024-03-22T10:43:11.479 INFO:tasks.workunit.client.1.smithi150.stdout:7/990: rename d2/d5/d7/d9/ce0 to d2/d5/d7/d9/db/dfb/d13a/d96/d71/c149 0 2024-03-22T10:43:11.481 INFO:tasks.workunit.client.1.smithi150.stdout:5/999: dread - d0/d8/dc/de/d14/d2e/d5a/d13b/f126 zero size 2024-03-22T10:43:11.481 INFO:tasks.workunit.client.1.smithi150.stdout:9/977: write d1/d8/f82 [930477,30419] 0 2024-03-22T10:43:11.483 INFO:tasks.workunit.client.1.smithi150.stdout:0/998: write d0/f6d [8205026,106106] 0 2024-03-22T10:43:11.484 INFO:tasks.workunit.client.1.smithi150.stdout:7/991: write d2/d5/d7/d9/db/da2/fbe [329153,100007] 0 2024-03-22T10:43:11.487 INFO:tasks.workunit.client.1.smithi150.stdout:1/958: unlink d1/d19/d50/d51/da0/dc4/d129/d15/d18/f8c 0 2024-03-22T10:43:11.492 INFO:tasks.workunit.client.1.smithi150.stdout:1/959: readlink d1/d19/d50/d6e/l7e 0 2024-03-22T10:43:11.496 INFO:tasks.workunit.client.1.smithi150.stdout:7/992: link d2/d5/d7/d9/db/dfb/d13a/d96/d71/f119 d2/d5/d7/d8/d57/d4a/df6/f14a 0 2024-03-22T10:43:11.502 INFO:tasks.workunit.client.1.smithi150.stdout:9/978: link d1/d8/de/df/d36/d40/d44/db5/dda/d104/dc3/fe0 d1/d8/de/dbd/ddc/f145 0 2024-03-22T10:43:11.504 INFO:tasks.workunit.client.1.smithi150.stdout:7/993: mkdir d2/d5/d7/d8/d57/d4a/df6/d14b 0 2024-03-22T10:43:11.508 INFO:tasks.workunit.client.1.smithi150.stdout:9/979: mkdir d1/d8/de/df/d36/d121/d4e/de9/d13e/d146 0 2024-03-22T10:43:11.508 INFO:tasks.workunit.client.1.smithi150.stdout:1/960: write d1/d19/d50/d51/da0/dc4/d129/d2a/d65/d78/fb1 [358751,43886] 0 2024-03-22T10:43:11.511 INFO:tasks.workunit.client.1.smithi150.stdout:7/994: rename d2/d5/l87 to d2/d5/d7/d8/d46/d93/l14c 0 2024-03-22T10:43:11.514 INFO:tasks.workunit.client.1.smithi150.stdout:1/961: mknod d1/d19/d50/d6e/c13d 0 2024-03-22T10:43:11.516 INFO:tasks.workunit.client.1.smithi150.stdout:0/999: dwrite d0/d1/d84/d86/d11/d26/f9f [0,4194304] 0 2024-03-22T10:43:11.518 INFO:tasks.workunit.client.1.smithi150.stdout:9/980: mknod d1/d19/d7d/dc1/dca/dd7/df0/c147 0 2024-03-22T10:43:11.522 INFO:tasks.workunit.client.1.smithi150.stdout:1/962: fsync d1/d19/d50/d51/da0/dc4/d129/d15/d18/d4b/fc1 0 2024-03-22T10:43:11.528 INFO:tasks.workunit.client.1.smithi150.stdout:1/963: rmdir d1/d19/d1a 39 2024-03-22T10:43:11.531 INFO:tasks.workunit.client.1.smithi150.stdout:1/964: write d1/d19/d50/f120 [2110165,14795] 0 2024-03-22T10:43:11.535 INFO:tasks.workunit.client.1.smithi150.stdout:7/995: link d2/d5/d7/d8/d57/d4a/d8d/da9/c12e d2/d5/d7/d38/c14d 0 2024-03-22T10:43:11.535 INFO:tasks.workunit.client.1.smithi150.stdout:1/965: write d1/d19/d50/d51/f6b [2391287,88236] 0 2024-03-22T10:43:11.543 INFO:tasks.workunit.client.1.smithi150.stdout:9/981: dwrite d1/d8/de/df/f4d [0,4194304] 0 2024-03-22T10:43:11.543 INFO:tasks.workunit.client.1.smithi150.stdout:7/996: chown d2/d5/d7/d9/db/dfb 1993749405 1 2024-03-22T10:43:11.559 INFO:tasks.workunit.client.1.smithi150.stdout:1/966: dread d1/d19/f9e [0,4194304] 0 2024-03-22T10:43:11.572 INFO:tasks.workunit.client.1.smithi150.stdout:9/982: dread d1/d8/de/df/d36/f39 [0,4194304] 0 2024-03-22T10:43:11.574 INFO:tasks.workunit.client.1.smithi150.stdout:9/983: write d1/d8/de/f127 [333417,75439] 0 2024-03-22T10:43:11.575 INFO:tasks.workunit.client.1.smithi150.stdout:7/997: rmdir d2/d5/d7/d8/d57/d4a/d111 39 2024-03-22T10:43:11.575 INFO:tasks.workunit.client.1.smithi150.stdout:9/984: chown d1/c7 31 1 2024-03-22T10:43:11.577 INFO:tasks.workunit.client.1.smithi150.stdout:1/967: symlink d1/d19/d50/d51/da0/dc4/d129/d15/l13e 0 2024-03-22T10:43:11.579 INFO:tasks.workunit.client.1.smithi150.stdout:7/998: symlink d2/d5/d7/d8/d57/d4a/l14e 0 2024-03-22T10:43:11.580 INFO:tasks.workunit.client.1.smithi150.stdout:9/985: mkdir d1/d19/d7d/dc1/dca/d111/d122/d148 0 2024-03-22T10:43:11.583 INFO:tasks.workunit.client.1.smithi150.stdout:1/968: creat d1/d19/d50/d51/da0/dc4/d129/d2a/dc0/d139/f13f x:0 0 0 2024-03-22T10:43:11.583 INFO:tasks.workunit.client.1.smithi150.stdout:7/999: mknod d2/d5/d125/c14f 0 2024-03-22T10:43:11.595 INFO:tasks.workunit.client.1.smithi150.stdout:1/969: dread d1/d19/d50/d51/da0/dc4/d129/d15/ff9 [0,4194304] 0 2024-03-22T10:43:11.605 INFO:tasks.workunit.client.1.smithi150.stdout:1/970: creat d1/d19/d50/d51/da0/dc4/d129/d15/d18/d4b/dc8/f140 x:0 0 0 2024-03-22T10:43:11.610 INFO:tasks.workunit.client.1.smithi150.stdout:1/971: creat d1/d19/d50/d51/da0/f141 x:0 0 0 2024-03-22T10:43:11.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.621+0000 7fa886c00700 1 -- 172.21.15.115:0/3690637071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8800ff050 msgr2=0x7fa8800ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:11.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.621+0000 7fa886c00700 1 --2- 172.21.15.115:0/3690637071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8800ff050 0x7fa8800ff460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa870009910 tx=0x7fa870009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:11.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.621+0000 7fa886c00700 1 -- 172.21.15.115:0/3690637071 shutdown_connections 2024-03-22T10:43:11.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.621+0000 7fa886c00700 1 --2- 172.21.15.115:0/3690637071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa880100250 0x7fa8801006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:11.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.621+0000 7fa886c00700 1 --2- 172.21.15.115:0/3690637071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8800ff050 0x7fa8800ff460 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa870009910 tx=0x7fa870009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:11.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.621+0000 7fa886c00700 1 -- 172.21.15.115:0/3690637071 >> 172.21.15.115:0/3690637071 conn(0x7fa8800fa5e0 msgr2=0x7fa8800fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:11.615 INFO:tasks.workunit.client.1.smithi150.stdout:1/972: write d1/d19/d50/d51/da0/dc4/d129/d15/d18/d4b/d8a/ffc [590140,7858] 0 2024-03-22T10:43:11.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.622+0000 7fa886c00700 1 -- 172.21.15.115:0/3690637071 shutdown_connections 2024-03-22T10:43:11.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.622+0000 7fa886c00700 1 -- 172.21.15.115:0/3690637071 wait complete. 2024-03-22T10:43:11.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.623+0000 7fa886c00700 1 Processor -- start 2024-03-22T10:43:11.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.624+0000 7fa886c00700 1 -- start start 2024-03-22T10:43:11.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.624+0000 7fa886c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa880100250 0x7fa880102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:11.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.624+0000 7fa886c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa880101630 0x7fa880101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:11.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.624+0000 7fa886c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880199840 con 0x7fa880100250 2024-03-22T10:43:11.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.624+0000 7fa886c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8801999b0 con 0x7fa880101630 2024-03-22T10:43:11.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.625+0000 7fa87f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa880101630 0x7fa880101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:11.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.625+0000 7fa87f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa880101630 0x7fa880101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:50098/0 (socket says 172.21.15.115:50098) 2024-03-22T10:43:11.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.625+0000 7fa87f400700 1 -- 172.21.15.115:0/1297631576 learned_addr learned my addr 172.21.15.115:0/1297631576 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:11.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.625+0000 7fa87fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa880100250 0x7fa880102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:11.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.627+0000 7fa87fe00700 1 -- 172.21.15.115:0/1297631576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa880101630 msgr2=0x7fa880101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:11.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.627+0000 7fa87fe00700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa880101630 0x7fa880101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:11.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.627+0000 7fa87fe00700 1 -- 172.21.15.115:0/1297631576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700095f0 con 0x7fa880100250 2024-03-22T10:43:11.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.628+0000 7fa87fe00700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa880100250 0x7fa880102fe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa8800feba0 tx=0x7fa87000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:11.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.629+0000 7fa87cc00700 1 -- 172.21.15.115:0/1297631576 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa87001d070 con 0x7fa880100250 2024-03-22T10:43:11.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.630+0000 7fa87cc00700 1 -- 172.21.15.115:0/1297631576 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa870005dc0 con 0x7fa880100250 2024-03-22T10:43:11.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.630+0000 7fa87cc00700 1 -- 172.21.15.115:0/1297631576 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa870017400 con 0x7fa880100250 2024-03-22T10:43:11.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.630+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa880101fe0 con 0x7fa880100250 2024-03-22T10:43:11.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.630+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8801024a0 con 0x7fa880100250 2024-03-22T10:43:11.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.632+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa880107ba0 con 0x7fa880100250 2024-03-22T10:43:11.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.634+0000 7fa87cc00700 1 -- 172.21.15.115:0/1297631576 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fa87000fd00 con 0x7fa880100250 2024-03-22T10:43:11.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.636+0000 7fa87cc00700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fa86406dd90 0x7fa864070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:11.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.637+0000 7fa87cc00700 1 -- 172.21.15.115:0/1297631576 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fa87008e580 con 0x7fa880100250 2024-03-22T10:43:11.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.637+0000 7fa87f400700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fa86406dd90 0x7fa864070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.642+0000 7fa87f400700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fa86406dd90 0x7fa864070240 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009cb0 tx=0x7fa86c009260 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:11.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.650+0000 7fa87cc00700 1 -- 172.21.15.115:0/1297631576 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fa87005b8d0 con 0x7fa880100250 2024-03-22T10:43:11.654 INFO:tasks.workunit.client.1.smithi150.stdout:1/973: dread d1/d19/d50/d51/da0/dc4/d129/d15/f11f [0,4194304] 0 2024-03-22T10:43:11.661 INFO:tasks.workunit.client.1.smithi150.stdout:1/974: symlink d1/d19/d50/d51/da0/dc4/d129/d2a/d65/d78/dfd/d11c/d11e/l142 0 2024-03-22T10:43:11.669 INFO:tasks.workunit.client.1.smithi150.stdout:1/975: rename d1/d19/d1a/la5 to d1/d19/l143 0 2024-03-22T10:43:11.674 INFO:tasks.workunit.client.1.smithi150.stdout:1/976: fsync d1/d19/d50/d51/da0/dc4/d129/d2a/d4e/ff5 0 2024-03-22T10:43:11.679 INFO:tasks.workunit.client.1.smithi150.stdout:1/977: chown d1/d19/d50/d51/da0/dc4/d129/d2a/d55/cf2 878134 1 2024-03-22T10:43:11.682 INFO:tasks.workunit.client.1.smithi150.stdout:1/978: write d1/d19/d50/d51/da0/dc4/d129/d15/ff9 [1411184,75756] 0 2024-03-22T10:43:11.687 INFO:tasks.workunit.client.1.smithi150.stdout:1/979: symlink d1/d19/d50/d51/da0/dc4/d129/d2a/d65/dbd/d125/l144 0 2024-03-22T10:43:11.691 INFO:tasks.workunit.client.1.smithi150.stdout:1/980: unlink d1/d19/d50/d51/da0/dc4/d129/l6d 0 2024-03-22T10:43:11.702 INFO:tasks.workunit.client.1.smithi150.stdout:1/981: dwrite d1/d19/d50/d51/ff4 [0,4194304] 0 2024-03-22T10:43:11.709 INFO:tasks.workunit.client.1.smithi150.stdout:1/982: chown d1/d19/d50/d51/da0/dc4/d129/d15/d18/l34 60471 1 2024-03-22T10:43:11.714 INFO:tasks.workunit.client.1.smithi150.stdout:1/983: creat d1/d19/d50/d51/da0/dc4/d129/d2a/d65/d78/dfd/f145 x:0 0 0 2024-03-22T10:43:11.868 INFO:tasks.workunit.client.1.smithi150.stdout:9/986: sync 2024-03-22T10:43:11.876 INFO:tasks.workunit.client.1.smithi150.stdout:9/987: fsync d1/d8/de/d73/d101/d32/f2a 0 2024-03-22T10:43:11.883 INFO:tasks.workunit.client.1.smithi150.stdout:9/988: read d1/d8/de/d73/d101/d29/f118 [3011315,39119] 0 2024-03-22T10:43:11.896 INFO:tasks.workunit.client.1.smithi150.stdout:9/989: getdents d1/d8/de/df/d36/d40/d44/db5/dda/d104/d99/da7 0 2024-03-22T10:43:11.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:11 smithi150 ceph-mon[34518]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:11.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:11 smithi150 ceph-mon[34518]: pgmap v24: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 104 MiB/s rd, 243 MiB/s wr, 572 op/s 2024-03-22T10:43:11.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:11 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2047156564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:43:11.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:11 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/1119463194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:43:11.901 INFO:tasks.workunit.client.1.smithi150.stdout:9/990: fdatasync d1/d49/d98/fcf 0 2024-03-22T10:43:11.908 INFO:tasks.workunit.client.1.smithi150.stdout:1/984: sync 2024-03-22T10:43:11.910 INFO:tasks.workunit.client.1.smithi150.stdout:9/991: mkdir d1/d8/de/df/d36/d121/d4e/de9/d13e/d149 0 2024-03-22T10:43:11.915 INFO:tasks.workunit.client.1.smithi150.stdout:1/985: mknod d1/d19/d1a/d7d/c146 0 2024-03-22T10:43:11.920 INFO:tasks.workunit.client.1.smithi150.stdout:9/992: creat d1/d19/d7d/dc1/dca/dd7/f14a x:0 0 0 2024-03-22T10:43:11.922 INFO:tasks.workunit.client.1.smithi150.stdout:1/986: write d1/d19/d50/d51/d123/f136 [990027,101961] 0 2024-03-22T10:43:11.932 INFO:tasks.workunit.client.1.smithi150.stdout:1/987: truncate d1/d79/f85 882178 0 2024-03-22T10:43:11.941 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:11 smithi115 ceph-mon[29111]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:11.941 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:11 smithi115 ceph-mon[29111]: pgmap v24: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 104 MiB/s rd, 243 MiB/s wr, 572 op/s 2024-03-22T10:43:11.941 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:11 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2047156564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:43:11.942 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:11 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/1119463194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:43:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.949+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8801024a0 con 0x7fa86406dd90 2024-03-22T10:43:11.943 INFO:tasks.workunit.client.1.smithi150.stdout:1/988: creat d1/d19/d50/d51/da0/dc4/d129/d2a/d55/db8/f147 x:0 0 0 2024-03-22T10:43:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.952+0000 7fa87cc00700 1 -- 172.21.15.115:0/1297631576 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fa8801024a0 con 0x7fa86406dd90 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr" 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "2/23 daemons upgraded", 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "", 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:43:11.945 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:43:11.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.955+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fa86406dd90 msgr2=0x7fa864070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:11.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.955+0000 7fa886c00700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fa86406dd90 0x7fa864070240 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009cb0 tx=0x7fa86c009260 comp rx=0 tx=0).stop 2024-03-22T10:43:11.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.955+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa880100250 msgr2=0x7fa880102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:11.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.955+0000 7fa886c00700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa880100250 0x7fa880102fe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa8800feba0 tx=0x7fa87000f740 comp rx=0 tx=0).stop 2024-03-22T10:43:11.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.955+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 shutdown_connections 2024-03-22T10:43:11.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.955+0000 7fa886c00700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fa86406dd90 0x7fa864070240 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:11.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.955+0000 7fa886c00700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa880100250 0x7fa880102fe0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:11.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.955+0000 7fa886c00700 1 --2- 172.21.15.115:0/1297631576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa880101630 0x7fa880101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:11.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.956+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 >> 172.21.15.115:0/1297631576 conn(0x7fa8800fa5e0 msgr2=0x7fa8800683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:11.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.956+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 shutdown_connections 2024-03-22T10:43:11.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:11.956+0000 7fa886c00700 1 -- 172.21.15.115:0/1297631576 wait complete. 2024-03-22T10:43:11.962 INFO:tasks.workunit.client.1.smithi150.stdout:9/993: dwrite d1/d8/d95/fc7 [0,4194304] 0 2024-03-22T10:43:11.964 INFO:tasks.workunit.client.1.smithi150.stdout:1/989: write d1/d19/d50/d51/da0/dc4/d129/d2a/d4e/fc2 [1018146,55704] 0 2024-03-22T10:43:11.969 INFO:tasks.workunit.client.1.smithi150.stdout:1/990: chown d1/d19/d50/d51/da0/dc4/d129/d15 461832 1 2024-03-22T10:43:11.973 INFO:tasks.workunit.client.1.smithi150.stdout:9/994: mknod d1/d8/de/d73/c14b 0 2024-03-22T10:43:11.976 INFO:tasks.workunit.client.1.smithi150.stdout:1/991: fdatasync d1/d19/d50/d51/d123/d12d/f60 0 2024-03-22T10:43:11.982 INFO:tasks.workunit.client.1.smithi150.stdout:9/995: truncate d1/d8/de/df/d36/d121/f89 8782900 0 2024-03-22T10:43:11.996 INFO:tasks.workunit.client.1.smithi150.stdout:9/996: write d1/d8/de/d73/d101/d32/f119 [1031002,102086] 0 2024-03-22T10:43:12.003 INFO:tasks.workunit.client.1.smithi150.stdout:9/997: read d1/d8/f65 [4500469,71389] 0 2024-03-22T10:43:12.010 INFO:tasks.workunit.client.1.smithi150.stdout:9/998: dread d1/d8/de/d73/d101/d29/f103 [0,4194304] 0 2024-03-22T10:43:12.021 INFO:tasks.workunit.client.1.smithi150.stdout:1/992: dwrite d1/d79/f85 [0,4194304] 0 2024-03-22T10:43:12.028 INFO:tasks.workunit.client.1.smithi150.stdout:1/993: readlink d1/d19/d50/d51/da0/dc4/d129/d2a/d65/d78/dfd/d11c/d11e/l142 0 2024-03-22T10:43:12.030 INFO:tasks.workunit.client.1.smithi150.stdout:1/994: dread - d1/d19/dd5/f12c zero size 2024-03-22T10:43:12.035 INFO:tasks.workunit.client.1.smithi150.stdout:9/999: dwrite d1/d8/de/f127 [0,4194304] 0 2024-03-22T10:43:12.037 INFO:tasks.workunit.client.1.smithi150.stdout:1/995: chown d1/d19/d50/d51/da0/dc4/d129/c48 48294 1 2024-03-22T10:43:12.042 INFO:tasks.workunit.client.1.smithi150.stdout:1/996: mknod d1/d19/d50/d51/da0/dcf/c148 0 2024-03-22T10:43:12.045 INFO:tasks.workunit.client.1.smithi150.stdout:1/997: symlink d1/d19/d50/d51/da0/dc4/d129/d15/d18/d4b/dc8/l149 0 2024-03-22T10:43:12.052 INFO:tasks.workunit.client.1.smithi150.stdout:1/998: read d1/d19/d50/d51/da0/dc4/d129/d2a/d4e/f110 [2341197,46642] 0 2024-03-22T10:43:12.053 INFO:tasks.workunit.client.1.smithi150.stdout:1/999: read - d1/d7c/f101 zero size 2024-03-22T10:43:12.061 INFO:tasks.workunit.client.1.smithi150.stderr:+ rm -rf -- ./tmp.zi0gx6aUjm 2024-03-22T10:43:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.125+0000 7fc594c00700 1 -- 172.21.15.115:0/3254454085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5900737b0 msgr2=0x7fc590073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.125+0000 7fc594c00700 1 --2- 172.21.15.115:0/3254454085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5900737b0 0x7fc590073c20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009940 tx=0x7fc57c009c50 comp rx=0 tx=0).stop 2024-03-22T10:43:12.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.125+0000 7fc594c00700 1 -- 172.21.15.115:0/3254454085 shutdown_connections 2024-03-22T10:43:12.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.125+0000 7fc594c00700 1 --2- 172.21.15.115:0/3254454085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5900737b0 0x7fc590073c20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:12.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.125+0000 7fc594c00700 1 --2- 172.21.15.115:0/3254454085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc590074d80 0x7fc5900731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:12.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.125+0000 7fc594c00700 1 -- 172.21.15.115:0/3254454085 >> 172.21.15.115:0/3254454085 conn(0x7fc5900fb6e0 msgr2=0x7fc5900fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:12.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.125+0000 7fc594c00700 1 -- 172.21.15.115:0/3254454085 shutdown_connections 2024-03-22T10:43:12.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.125+0000 7fc594c00700 1 -- 172.21.15.115:0/3254454085 wait complete. 2024-03-22T10:43:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.126+0000 7fc594c00700 1 Processor -- start 2024-03-22T10:43:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.126+0000 7fc594c00700 1 -- start start 2024-03-22T10:43:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.126+0000 7fc594c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5900737b0 0x7fc590197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.127+0000 7fc594c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc590074d80 0x7fc5901980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.127+0000 7fc594c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5901986c0 con 0x7fc590074d80 2024-03-22T10:43:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.127+0000 7fc594c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc590198800 con 0x7fc5900737b0 2024-03-22T10:43:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.127+0000 7fc58de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5900737b0 0x7fc590197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.127+0000 7fc58d400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc590074d80 0x7fc5901980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.127+0000 7fc58de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5900737b0 0x7fc590197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:46974/0 (socket says 172.21.15.115:46974) 2024-03-22T10:43:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.127+0000 7fc58d400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc590074d80 0x7fc5901980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44248/0 (socket says 172.21.15.115:44248) 2024-03-22T10:43:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.127+0000 7fc58de00700 1 -- 172.21.15.115:0/2477570170 learned_addr learned my addr 172.21.15.115:0/2477570170 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.128+0000 7fc58de00700 1 -- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc590074d80 msgr2=0x7fc5901980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:12.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.128+0000 7fc58de00700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc590074d80 0x7fc5901980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:12.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.128+0000 7fc58de00700 1 -- 172.21.15.115:0/2477570170 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c0095f0 con 0x7fc5900737b0 2024-03-22T10:43:12.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.128+0000 7fc58d400700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc590074d80 0x7fc5901980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T10:43:12.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.129+0000 7fc58de00700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5900737b0 0x7fc590197b60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc58000d8d0 tx=0x7fc58000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:12.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.129+0000 7fc586a00700 1 -- 172.21.15.115:0/2477570170 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5800098a0 con 0x7fc5900737b0 2024-03-22T10:43:12.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.129+0000 7fc586a00700 1 -- 172.21.15.115:0/2477570170 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc580010460 con 0x7fc5900737b0 2024-03-22T10:43:12.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.129+0000 7fc586a00700 1 -- 172.21.15.115:0/2477570170 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc58000f5f0 con 0x7fc5900737b0 2024-03-22T10:43:12.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.129+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59019d2b0 con 0x7fc5900737b0 2024-03-22T10:43:12.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.130+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59019d880 con 0x7fc5900737b0 2024-03-22T10:43:12.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.131+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59010ae90 con 0x7fc5900737b0 2024-03-22T10:43:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.133+0000 7fc586a00700 1 -- 172.21.15.115:0/2477570170 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fc580009a00 con 0x7fc5900737b0 2024-03-22T10:43:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.133+0000 7fc586a00700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fc57406dd50 0x7fc574070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.134+0000 7fc586a00700 1 -- 172.21.15.115:0/2477570170 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fc58008d8d0 con 0x7fc5900737b0 2024-03-22T10:43:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.134+0000 7fc58d400700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fc57406dd50 0x7fc574070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.135+0000 7fc58d400700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fc57406dd50 0x7fc574070200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009910 tx=0x7fc57c005ab0 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:12.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.143+0000 7fc586a00700 1 -- 172.21.15.115:0/2477570170 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fc58005ab60 con 0x7fc5900737b0 2024-03-22T10:43:12.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.510+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc59019df10 con 0x7fc5900737b0 2024-03-22T10:43:12.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.511+0000 7fc586a00700 1 -- 172.21.15.115:0/2477570170 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc5800160e0 con 0x7fc5900737b0 2024-03-22T10:43:12.503 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:43:12.503 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:43:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:43:12.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fc57406dd50 msgr2=0x7fc574070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:12.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fc57406dd50 0x7fc574070200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009910 tx=0x7fc57c005ab0 comp rx=0 tx=0).stop 2024-03-22T10:43:12.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5900737b0 msgr2=0x7fc590197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:12.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5900737b0 0x7fc590197b60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc58000d8d0 tx=0x7fc58000dbe0 comp rx=0 tx=0).stop 2024-03-22T10:43:12.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 shutdown_connections 2024-03-22T10:43:12.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fc57406dd50 0x7fc574070200 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:12.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5900737b0 0x7fc590197b60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:12.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 --2- 172.21.15.115:0/2477570170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc590074d80 0x7fc5901980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:12.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 >> 172.21.15.115:0/2477570170 conn(0x7fc5900fb6e0 msgr2=0x7fc590105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:12.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 shutdown_connections 2024-03-22T10:43:12.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:12.514+0000 7fc594c00700 1 -- 172.21.15.115:0/2477570170 wait complete. 2024-03-22T10:43:12.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:12 smithi115 ceph-mon[29111]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:12.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:12 smithi115 ceph-mon[29111]: from='client.? 172.21.15.115:0/2477570170' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:43:13.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:12 smithi150 ceph-mon[34518]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:13.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:12 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2477570170' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:43:13.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:13 smithi115 ceph-mon[29111]: pgmap v25: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 61 MiB/s rd, 132 MiB/s wr, 343 op/s 2024-03-22T10:43:13.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:13 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:43:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:13 smithi150 ceph-mon[34518]: pgmap v25: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 61 MiB/s rd, 132 MiB/s wr, 343 op/s 2024-03-22T10:43:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:13 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:43:16.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:15 smithi115 ceph-mon[29111]: pgmap v26: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 61 MiB/s rd, 132 MiB/s wr, 343 op/s 2024-03-22T10:43:16.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:15 smithi150 ceph-mon[34518]: pgmap v26: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 61 MiB/s rd, 132 MiB/s wr, 343 op/s 2024-03-22T10:43:18.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:17 smithi115 ceph-mon[29111]: pgmap v27: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 70 MiB/s rd, 160 MiB/s wr, 454 op/s 2024-03-22T10:43:18.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:17 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:18.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:17 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:18.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:17 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:43:18.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:17 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:43:18.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:17 smithi150 ceph-mon[34518]: pgmap v27: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 70 MiB/s rd, 160 MiB/s wr, 454 op/s 2024-03-22T10:43:18.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:17 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:18.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:17 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:18.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:17 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:43:18.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:17 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:43:19.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:18 smithi115 ceph-mon[29111]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:43:19.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:18 smithi150 ceph-mon[34518]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:43:20.021 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:19 smithi150 ceph-mon[34518]: pgmap v28: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 40 MiB/s rd, 89 MiB/s wr, 278 op/s 2024-03-22T10:43:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:19 smithi115 ceph-mon[29111]: pgmap v28: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 40 MiB/s rd, 89 MiB/s wr, 278 op/s 2024-03-22T10:43:20.554 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-22T10:43:20.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-22T10:43:21.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:20 smithi115 ceph-mon[29111]: Upgrade: Updating mgr.smithi150.fmoumv 2024-03-22T10:43:21.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:20 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:43:21.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:20 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:43:21.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:20 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:43:21.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:20 smithi115 ceph-mon[29111]: Deploying daemon mgr.smithi150.fmoumv on smithi150 2024-03-22T10:43:21.120 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:20 smithi150 ceph-mon[34518]: Upgrade: Updating mgr.smithi150.fmoumv 2024-03-22T10:43:21.120 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:20 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:43:21.120 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:20 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:43:21.120 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:20 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:43:21.120 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:20 smithi150 ceph-mon[34518]: Deploying daemon mgr.smithi150.fmoumv on smithi150 2024-03-22T10:43:21.994 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:21 smithi150 ceph-mon[34518]: pgmap v29: 65 pgs: 65 active+clean; 2.1 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 90 MiB/s wr, 348 op/s 2024-03-22T10:43:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:21 smithi115 ceph-mon[29111]: pgmap v29: 65 pgs: 65 active+clean; 2.1 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 90 MiB/s wr, 348 op/s 2024-03-22T10:43:24.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:23 smithi150 ceph-mon[34518]: pgmap v30: 65 pgs: 65 active+clean; 2.1 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 8.8 MiB/s rd, 30 MiB/s wr, 181 op/s 2024-03-22T10:43:24.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:23 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:43:24.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:23 smithi115 ceph-mon[29111]: pgmap v30: 65 pgs: 65 active+clean; 2.1 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 8.8 MiB/s rd, 30 MiB/s wr, 181 op/s 2024-03-22T10:43:24.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:23 smithi115 ceph-mon[29111]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:43:26.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:25 smithi150 ceph-mon[34518]: pgmap v31: 65 pgs: 65 active+clean; 2.1 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 8.8 MiB/s rd, 30 MiB/s wr, 181 op/s 2024-03-22T10:43:26.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:25 smithi115 ceph-mon[29111]: pgmap v31: 65 pgs: 65 active+clean; 2.1 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 8.8 MiB/s rd, 30 MiB/s wr, 181 op/s 2024-03-22T10:43:27.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:27 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:27.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:27 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:27.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:27 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:43:27.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:27 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:27.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:27 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:27.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:27 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:43:28.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:28 smithi115 ceph-mon[29111]: pgmap v32: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 8.8 MiB/s rd, 30 MiB/s wr, 254 op/s 2024-03-22T10:43:28.337 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:28 smithi150 ceph-mon[34518]: pgmap v32: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 8.8 MiB/s rd, 30 MiB/s wr, 254 op/s 2024-03-22T10:43:29.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:29 smithi150 ceph-mon[34518]: pgmap v33: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-03-22T10:43:29.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:29 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:43:29.509 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:29 smithi115 ceph-mon[29111]: pgmap v33: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-03-22T10:43:29.509 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:29 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:43:32.066 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:31 smithi150 ceph-mon[34518]: pgmap v34: 65 pgs: 65 active+clean; 1.1 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 31 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-03-22T10:43:32.067 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:31 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:32.067 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:31 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:32.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:31 smithi115 ceph-mon[29111]: pgmap v34: 65 pgs: 65 active+clean; 1.1 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 31 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-03-22T10:43:32.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:31 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:32.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:31 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:34.100 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:33 smithi150 ceph-mon[34518]: pgmap v35: 65 pgs: 65 active+clean; 1.1 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-03-22T10:43:34.100 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:33 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:34.100 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:33 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:34.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:33 smithi115 ceph-mon[29111]: pgmap v35: 65 pgs: 65 active+clean; 1.1 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-03-22T10:43:34.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:33 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:34.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:33 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:34.614 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-22T10:43:34.614 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-22T10:43:35.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:35 smithi150 ceph-mon[34518]: pgmap v36: 65 pgs: 65 active+clean; 1.1 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-03-22T10:43:36.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:35 smithi115 ceph-mon[29111]: pgmap v36: 65 pgs: 65 active+clean; 1.1 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-03-22T10:43:37.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:37 smithi150 ceph-mon[34518]: pgmap v37: 65 pgs: 65 active+clean; 592 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-03-22T10:43:37.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:37 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:37.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:37 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:37.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:37 smithi150 ceph-mon[34518]: Standby manager daemon smithi150.fmoumv restarted 2024-03-22T10:43:37.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:37 smithi150 ceph-mon[34518]: Standby manager daemon smithi150.fmoumv started 2024-03-22T10:43:37.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:37 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/1435977634' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/crt"}]: dispatch 2024-03-22T10:43:37.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:37 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/1435977634' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:43:37.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:37 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/1435977634' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/key"}]: dispatch 2024-03-22T10:43:37.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:37 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/1435977634' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:43:38.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:37 smithi115 ceph-mon[29111]: pgmap v37: 65 pgs: 65 active+clean; 592 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-03-22T10:43:38.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:37 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:38.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:37 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:38.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:37 smithi115 ceph-mon[29111]: Standby manager daemon smithi150.fmoumv restarted 2024-03-22T10:43:38.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:37 smithi115 ceph-mon[29111]: Standby manager daemon smithi150.fmoumv started 2024-03-22T10:43:38.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:37 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/1435977634' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/crt"}]: dispatch 2024-03-22T10:43:38.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:37 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/1435977634' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:43:38.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:37 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/1435977634' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/key"}]: dispatch 2024-03-22T10:43:38.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:37 smithi115 ceph-mon[29111]: from='mgr.? 172.21.15.150:0/1435977634' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:43:39.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: mgrmap e31: smithi115.iqzwsz(active, since 69s), standbys: smithi150.fmoumv 2024-03-22T10:43:39.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi115.iqzwsz"}]: dispatch 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi115.iqzwsz"}]': finished 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi150.fmoumv"}]: dispatch 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi150.fmoumv"}]': finished 2024-03-22T10:43:39.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:38 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T10:43:39.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: mgrmap e31: smithi115.iqzwsz(active, since 69s), standbys: smithi150.fmoumv 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi115.iqzwsz"}]: dispatch 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi115.iqzwsz"}]': finished 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi150.fmoumv"}]: dispatch 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi150.fmoumv"}]': finished 2024-03-22T10:43:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:38 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T10:43:40.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:39 smithi115 ceph-mon[29111]: Upgrade: Setting container_image for all mgr 2024-03-22T10:43:40.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:39 smithi115 ceph-mon[29111]: pgmap v38: 65 pgs: 65 active+clean; 592 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-03-22T10:43:40.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:39 smithi150 ceph-mon[34518]: Upgrade: Setting container_image for all mgr 2024-03-22T10:43:40.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:39 smithi150 ceph-mon[34518]: pgmap v38: 65 pgs: 65 active+clean; 592 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-03-22T10:43:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:41 smithi150 ceph-mon[34518]: pgmap v39: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 2.1 MiB/s wr, 150 op/s 2024-03-22T10:43:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:41 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:41 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:43:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:41 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:43:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:41 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:43:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:41 smithi115 ceph-mon[29111]: pgmap v39: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 2.1 MiB/s wr, 150 op/s 2024-03-22T10:43:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:41 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:43:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:41 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:43:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:41 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:43:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:41 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:43:42.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.688+0000 7f7ce1400700 1 -- 172.21.15.115:0/2705431092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 msgr2=0x7f7cdc073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:42.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.688+0000 7f7ce1400700 1 --2- 172.21.15.115:0/2705431092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 0x7f7cdc073c20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009940 tx=0x7f7cc8009c50 comp rx=0 tx=0).stop 2024-03-22T10:43:42.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.689+0000 7f7ce1400700 1 -- 172.21.15.115:0/2705431092 shutdown_connections 2024-03-22T10:43:42.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.689+0000 7f7ce1400700 1 --2- 172.21.15.115:0/2705431092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 0x7f7cdc073c20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:42.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.689+0000 7f7ce1400700 1 --2- 172.21.15.115:0/2705431092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cdc074d80 0x7f7cdc0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:42.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.689+0000 7f7ce1400700 1 -- 172.21.15.115:0/2705431092 >> 172.21.15.115:0/2705431092 conn(0x7f7cdc0fb6e0 msgr2=0x7f7cdc0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:42.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.689+0000 7f7ce1400700 1 -- 172.21.15.115:0/2705431092 shutdown_connections 2024-03-22T10:43:42.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.689+0000 7f7ce1400700 1 -- 172.21.15.115:0/2705431092 wait complete. 2024-03-22T10:43:42.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.690+0000 7f7ce1400700 1 Processor -- start 2024-03-22T10:43:42.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.690+0000 7f7ce1400700 1 -- start start 2024-03-22T10:43:42.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.690+0000 7f7ce1400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 0x7f7cdc197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:42.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.690+0000 7f7ce1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cdc074d80 0x7f7cdc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:42.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.690+0000 7f7ce1400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc1987a0 con 0x7f7cdc0737b0 2024-03-22T10:43:42.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.690+0000 7f7ce1400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc198910 con 0x7f7cdc074d80 2024-03-22T10:43:42.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.691+0000 7f7cda800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 0x7f7cdc197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:42.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.691+0000 7f7cda800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 0x7f7cdc197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33934/0 (socket says 172.21.15.115:33934) 2024-03-22T10:43:42.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.691+0000 7f7cda800700 1 -- 172.21.15.115:0/3463978632 learned_addr learned my addr 172.21.15.115:0/3463978632 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.691+0000 7f7cd9e00700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cdc074d80 0x7f7cdc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7cda800700 1 -- 172.21.15.115:0/3463978632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cdc074d80 msgr2=0x7f7cdc1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7cda800700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cdc074d80 0x7f7cdc1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7cda800700 1 -- 172.21.15.115:0/3463978632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc80095f0 con 0x7f7cdc0737b0 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7cd9e00700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cdc074d80 0x7f7cdc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7cda800700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 0x7f7cdc197c60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00b4b0 tx=0x7f7ccc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7cd3400700 1 -- 172.21.15.115:0/3463978632 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ccc011800 con 0x7f7cdc0737b0 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7cd3400700 1 -- 172.21.15.115:0/3463978632 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7ccc011e10 con 0x7f7cdc0737b0 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7cd3400700 1 -- 172.21.15.115:0/3463978632 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ccc00f2e0 con 0x7f7cdc0737b0 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc19d3a0 con 0x7f7cdc0737b0 2024-03-22T10:43:42.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.692+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc19d920 con 0x7f7cdc0737b0 2024-03-22T10:43:42.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.694+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc10af10 con 0x7f7cdc0737b0 2024-03-22T10:43:42.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.695+0000 7f7cd3400700 1 -- 172.21.15.115:0/3463978632 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f7ccc010380 con 0x7f7cdc0737b0 2024-03-22T10:43:42.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.696+0000 7f7cd3400700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f7cc006dda0 0x7f7cc0070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:42.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.696+0000 7f7cd3400700 1 -- 172.21.15.115:0/3463978632 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f7ccc08d630 con 0x7f7cdc0737b0 2024-03-22T10:43:42.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.696+0000 7f7cd9e00700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f7cc006dda0 0x7f7cc0070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:42.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.697+0000 7f7cd9e00700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f7cc006dda0 0x7f7cc0070250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8006010 tx=0x7f7cc8005ab0 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:42.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:42.703+0000 7f7cd3400700 1 -- 172.21.15.115:0/3463978632 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f7ccc056e20 con 0x7f7cdc0737b0 2024-03-22T10:43:43.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.136+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cdc107650 con 0x7f7cc006dda0 2024-03-22T10:43:43.128 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:42 smithi115 ceph-mon[29111]: Upgrade: Updating mon.smithi115 2024-03-22T10:43:43.128 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:42 smithi115 ceph-mon[29111]: Deploying daemon mon.smithi115 on smithi115 2024-03-22T10:43:43.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.138+0000 7f7cd3400700 1 -- 172.21.15.115:0/3463978632 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f7cdc107650 con 0x7f7cc006dda0 2024-03-22T10:43:43.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f7cc006dda0 msgr2=0x7f7cc0070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:43.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f7cc006dda0 0x7f7cc0070250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8006010 tx=0x7f7cc8005ab0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 msgr2=0x7f7cdc197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 0x7f7cdc197c60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00b4b0 tx=0x7f7ccc00b870 comp rx=0 tx=0).stop 2024-03-22T10:43:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 shutdown_connections 2024-03-22T10:43:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f7cc006dda0 0x7f7cc0070250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cdc0737b0 0x7f7cdc197c60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 --2- 172.21.15.115:0/3463978632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cdc074d80 0x7f7cdc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 >> 172.21.15.115:0/3463978632 conn(0x7f7cdc0fb6e0 msgr2=0x7f7cdc105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.141+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 shutdown_connections 2024-03-22T10:43:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.142+0000 7f7ce1400700 1 -- 172.21.15.115:0/3463978632 wait complete. 2024-03-22T10:43:43.150 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:43:43.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.312+0000 7f0941e00700 1 -- 172.21.15.115:0/2634172851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 msgr2=0x7f093c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:43.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.312+0000 7f0941e00700 1 --2- 172.21.15.115:0/2634172851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 0x7f093c0ff790 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0928009910 tx=0x7f0928009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:43.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.312+0000 7f0941e00700 1 -- 172.21.15.115:0/2634172851 shutdown_connections 2024-03-22T10:43:43.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.312+0000 7f0941e00700 1 --2- 172.21.15.115:0/2634172851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 0x7f093c0ff790 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.312+0000 7f0941e00700 1 --2- 172.21.15.115:0/2634172851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0fe9d0 0x7f093c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.312+0000 7f0941e00700 1 -- 172.21.15.115:0/2634172851 >> 172.21.15.115:0/2634172851 conn(0x7f093c0fa5e0 msgr2=0x7f093c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:43.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.313+0000 7f0941e00700 1 -- 172.21.15.115:0/2634172851 shutdown_connections 2024-03-22T10:43:43.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.313+0000 7f0941e00700 1 -- 172.21.15.115:0/2634172851 wait complete. 2024-03-22T10:43:43.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.314+0000 7f0941e00700 1 Processor -- start 2024-03-22T10:43:43.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.314+0000 7f0941e00700 1 -- start start 2024-03-22T10:43:43.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.314+0000 7f0941e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0fe9d0 0x7f093c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.314+0000 7f0941e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 0x7f093c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.314+0000 7f0941e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c198690 con 0x7f093c0ff320 2024-03-22T10:43:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.314+0000 7f0941e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c1987d0 con 0x7f093c0fe9d0 2024-03-22T10:43:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.315+0000 7f093a000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 0x7f093c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:43.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.315+0000 7f093a000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 0x7f093c198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33952/0 (socket says 172.21.15.115:33952) 2024-03-22T10:43:43.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.315+0000 7f093a000700 1 -- 172.21.15.115:0/994680972 learned_addr learned my addr 172.21.15.115:0/994680972 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:43.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.315+0000 7f093aa00700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0fe9d0 0x7f093c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:43.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.315+0000 7f093a000700 1 -- 172.21.15.115:0/994680972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0fe9d0 msgr2=0x7f093c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:43.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.315+0000 7f093a000700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0fe9d0 0x7f093c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.315+0000 7f093a000700 1 -- 172.21.15.115:0/994680972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09280095f0 con 0x7f093c0ff320 2024-03-22T10:43:43.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.316+0000 7f093a000700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 0x7f093c198070 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0928005b40 tx=0x7f092800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:43.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.316+0000 7f0940a00700 1 -- 172.21.15.115:0/994680972 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f092801d070 con 0x7f093c0ff320 2024-03-22T10:43:43.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.316+0000 7f0940a00700 1 -- 172.21.15.115:0/994680972 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f092800f460 con 0x7f093c0ff320 2024-03-22T10:43:43.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.316+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c19d220 con 0x7f093c0ff320 2024-03-22T10:43:43.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.316+0000 7f0940a00700 1 -- 172.21.15.115:0/994680972 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0928021640 con 0x7f093c0ff320 2024-03-22T10:43:43.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.316+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c19d710 con 0x7f093c0ff320 2024-03-22T10:43:43.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.318+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c10ae80 con 0x7f093c0ff320 2024-03-22T10:43:43.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.319+0000 7f0940a00700 1 -- 172.21.15.115:0/994680972 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f092800fa90 con 0x7f093c0ff320 2024-03-22T10:43:43.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.320+0000 7f0940a00700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f092006dd50 0x7f0920070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:43.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.320+0000 7f0940a00700 1 -- 172.21.15.115:0/994680972 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f092808ed40 con 0x7f093c0ff320 2024-03-22T10:43:43.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.320+0000 7f093aa00700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f092006dd50 0x7f0920070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:43.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.321+0000 7f093aa00700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f092006dd50 0x7f0920070200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f092c005950 tx=0x7f092c0058e0 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.328+0000 7f0940a00700 1 -- 172.21.15.115:0/994680972 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f092805c070 con 0x7f093c0ff320 2024-03-22T10:43:43.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:42 smithi150 ceph-mon[34518]: Upgrade: Updating mon.smithi115 2024-03-22T10:43:43.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:42 smithi150 ceph-mon[34518]: Deploying daemon mon.smithi115 on smithi115 2024-03-22T10:43:43.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.683+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f093c108720 con 0x7f092006dd50 2024-03-22T10:43:43.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.685+0000 7f0940a00700 1 -- 172.21.15.115:0/994680972 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f093c108720 con 0x7f092006dd50 2024-03-22T10:43:43.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.688+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f092006dd50 msgr2=0x7f0920070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:43.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.688+0000 7f0941e00700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f092006dd50 0x7f0920070200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f092c005950 tx=0x7f092c0058e0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.688+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 msgr2=0x7f093c198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:43.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.688+0000 7f0941e00700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 0x7f093c198070 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0928005b40 tx=0x7f092800be20 comp rx=0 tx=0).stop 2024-03-22T10:43:43.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.689+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 shutdown_connections 2024-03-22T10:43:43.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.689+0000 7f0941e00700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f092006dd50 0x7f0920070200 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.689+0000 7f0941e00700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0fe9d0 0x7f093c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.689+0000 7f0941e00700 1 --2- 172.21.15.115:0/994680972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f093c0ff320 0x7f093c198070 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.689+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 >> 172.21.15.115:0/994680972 conn(0x7f093c0fa5e0 msgr2=0x7f093c107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:43.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.689+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 shutdown_connections 2024-03-22T10:43:43.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.689+0000 7f0941e00700 1 -- 172.21.15.115:0/994680972 wait complete. 2024-03-22T10:43:43.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.884+0000 7efeed000700 1 -- 172.21.15.115:0/3529852550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8072640 msgr2=0x7efee8107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:43.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.884+0000 7efeed000700 1 --2- 172.21.15.115:0/3529852550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8072640 0x7efee8107d50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efed4009910 tx=0x7efed4009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:43.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.884+0000 7efeed000700 1 -- 172.21.15.115:0/3529852550 shutdown_connections 2024-03-22T10:43:43.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.884+0000 7efeed000700 1 --2- 172.21.15.115:0/3529852550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8072640 0x7efee8107d50 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efed4009910 tx=0x7efed4009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:43.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.884+0000 7efeed000700 1 --2- 172.21.15.115:0/3529852550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee8071cf0 0x7efee8072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.884+0000 7efeed000700 1 -- 172.21.15.115:0/3529852550 >> 172.21.15.115:0/3529852550 conn(0x7efee806d050 msgr2=0x7efee806f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:43.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.885+0000 7efeed000700 1 -- 172.21.15.115:0/3529852550 shutdown_connections 2024-03-22T10:43:43.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.885+0000 7efeed000700 1 -- 172.21.15.115:0/3529852550 wait complete. 2024-03-22T10:43:43.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.886+0000 7efeed000700 1 Processor -- start 2024-03-22T10:43:43.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.887+0000 7efeed000700 1 -- start start 2024-03-22T10:43:43.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.887+0000 7efeed000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8071cf0 0x7efee81a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:43.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.887+0000 7efeed000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee81a0c20 0x7efee81a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:43.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.887+0000 7efeed000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee81a1120 con 0x7efee8071cf0 2024-03-22T10:43:43.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.887+0000 7efeed000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee81a1290 con 0x7efee81a0c20 2024-03-22T10:43:43.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.888+0000 7efee5e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8071cf0 0x7efee81a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:43.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.888+0000 7efee5e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8071cf0 0x7efee81a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33976/0 (socket says 172.21.15.115:33976) 2024-03-22T10:43:43.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.888+0000 7efee5e00700 1 -- 172.21.15.115:0/68829194 learned_addr learned my addr 172.21.15.115:0/68829194 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:43.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.888+0000 7efee5400700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee81a0c20 0x7efee81a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:43.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.889+0000 7efee5e00700 1 -- 172.21.15.115:0/68829194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee81a0c20 msgr2=0x7efee81a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:43.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.889+0000 7efee5e00700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee81a0c20 0x7efee81a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:43.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.889+0000 7efee5e00700 1 -- 172.21.15.115:0/68829194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40095f0 con 0x7efee8071cf0 2024-03-22T10:43:43.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.889+0000 7efee5e00700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8071cf0 0x7efee81a06e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efed800b4b0 tx=0x7efed800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:43.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.890+0000 7efedea00700 1 -- 172.21.15.115:0/68829194 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efed8011800 con 0x7efee8071cf0 2024-03-22T10:43:43.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.890+0000 7efedea00700 1 -- 172.21.15.115:0/68829194 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efed8011e10 con 0x7efee8071cf0 2024-03-22T10:43:43.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.890+0000 7efedea00700 1 -- 172.21.15.115:0/68829194 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efed800f2e0 con 0x7efee8071cf0 2024-03-22T10:43:43.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.890+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee81a6230 con 0x7efee8071cf0 2024-03-22T10:43:43.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.890+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee81a6750 con 0x7efee8071cf0 2024-03-22T10:43:43.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.892+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec8005320 con 0x7efee8071cf0 2024-03-22T10:43:43.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.893+0000 7efedea00700 1 -- 172.21.15.115:0/68829194 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7efed8011960 con 0x7efee8071cf0 2024-03-22T10:43:43.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.895+0000 7efedea00700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7efecc06dd60 0x7efecc070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:43.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.895+0000 7efedea00700 1 -- 172.21.15.115:0/68829194 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7efed808d400 con 0x7efee8071cf0 2024-03-22T10:43:43.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.895+0000 7efee5400700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7efecc06dd60 0x7efecc070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:43.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.896+0000 7efee5400700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7efecc06dd60 0x7efecc070210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4009fd0 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:43.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:43.905+0000 7efedea00700 1 -- 172.21.15.115:0/68829194 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7efed805a7a0 con 0x7efee8071cf0 2024-03-22T10:43:44.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:43 smithi115 ceph-mon[29111]: pgmap v40: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-03-22T10:43:44.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:43 smithi115 ceph-mon[29111]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:43:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.300+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efec8000bf0 con 0x7efecc06dd60 2024-03-22T10:43:44.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.319+0000 7efedea00700 1 -- 172.21.15.115:0/68829194 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7efec8000bf0 con 0x7efecc06dd60 2024-03-22T10:43:44.311 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (10m) 7s ago 12m 14.2M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (12m) 7s ago 12m 7894k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (11m) 13s ago 11m 7755k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (12m) 7s ago 12m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (10m) 13s ago 10m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (9m) 7s ago 11m 74.1M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (5m) 7s ago 5m 175M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (5m) 7s ago 5m 14.3M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (4m) 13s ago 4m 14.3M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (5m) 13s ago 5m 157M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (94s) 7s ago 13m 493M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (19s) 13s ago 10m 64.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (14m) 7s ago 14m 54.2M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:43:44.312 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (10m) 13s ago 10m 43.9M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:43:44.313 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (12m) 7s ago 12m 7990k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:43:44.313 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (10m) 13s ago 10m 7834k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:43:44.313 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (9m) 7s ago 9m 180M 1217M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:43:44.313 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (9m) 7s ago 9m 190M 1217M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:43:44.313 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (8m) 7s ago 8m 153M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:43:44.313 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (8m) 13s ago 8m 206M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:43:44.313 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (7m) 13s ago 7m 191M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:43:44.313 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (6m) 13s ago 6m 202M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:43:44.313 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (27s) 7s ago 11m 52.4M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:43:44.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7efecc06dd60 msgr2=0x7efecc070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:44.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7efecc06dd60 0x7efecc070210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4009fd0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8071cf0 msgr2=0x7efee81a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:44.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8071cf0 0x7efee81a06e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efed800b4b0 tx=0x7efed800b7c0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 shutdown_connections 2024-03-22T10:43:44.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7efecc06dd60 0x7efecc070210 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4009fd0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee8071cf0 0x7efee81a06e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 --2- 172.21.15.115:0/68829194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee81a0c20 0x7efee81a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 >> 172.21.15.115:0/68829194 conn(0x7efee806d050 msgr2=0x7efee810ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:44.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.322+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 shutdown_connections 2024-03-22T10:43:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.323+0000 7efeed000700 1 -- 172.21.15.115:0/68829194 wait complete. 2024-03-22T10:43:44.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:43 smithi150 ceph-mon[34518]: pgmap v40: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-03-22T10:43:44.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:43 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:43:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.512+0000 7f4a97c00700 1 -- 172.21.15.115:0/1520318247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 msgr2=0x7f4a90073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.512+0000 7f4a97c00700 1 --2- 172.21.15.115:0/1520318247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 0x7f4a90073c20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009910 tx=0x7f4a7c009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.513+0000 7f4a97c00700 1 -- 172.21.15.115:0/1520318247 shutdown_connections 2024-03-22T10:43:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.513+0000 7f4a97c00700 1 --2- 172.21.15.115:0/1520318247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 0x7f4a90073c20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.513+0000 7f4a97c00700 1 --2- 172.21.15.115:0/1520318247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 0x7f4a900731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.513+0000 7f4a97c00700 1 -- 172.21.15.115:0/1520318247 >> 172.21.15.115:0/1520318247 conn(0x7f4a900fb6e0 msgr2=0x7f4a900fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:44.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.513+0000 7f4a97c00700 1 -- 172.21.15.115:0/1520318247 shutdown_connections 2024-03-22T10:43:44.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.513+0000 7f4a97c00700 1 -- 172.21.15.115:0/1520318247 wait complete. 2024-03-22T10:43:44.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.514+0000 7f4a97c00700 1 Processor -- start 2024-03-22T10:43:44.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.514+0000 7f4a97c00700 1 -- start start 2024-03-22T10:43:44.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.515+0000 7f4a97c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 0x7f4a90071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.515+0000 7f4a97c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 0x7f4a900721b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.515+0000 7f4a97c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900727f0 con 0x7f4a900737b0 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.515+0000 7f4a97c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90072960 con 0x7f4a90074d80 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.515+0000 7f4a95200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 0x7f4a90071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.515+0000 7f4a8fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 0x7f4a900721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.515+0000 7f4a95200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 0x7f4a90071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33986/0 (socket says 172.21.15.115:33986) 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.515+0000 7f4a95200700 1 -- 172.21.15.115:0/2903854168 learned_addr learned my addr 172.21.15.115:0/2903854168 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.516+0000 7f4a95200700 1 -- 172.21.15.115:0/2903854168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 msgr2=0x7f4a900721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.516+0000 7f4a95200700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 0x7f4a900721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.516+0000 7f4a95200700 1 -- 172.21.15.115:0/2903854168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c0095f0 con 0x7f4a900737b0 2024-03-22T10:43:44.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.517+0000 7f4a95200700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 0x7f4a90071c70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c8a0 tx=0x7f4a8400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:44.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.517+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a840078e0 con 0x7f4a900737b0 2024-03-22T10:43:44.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.517+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a90198b90 con 0x7f4a900737b0 2024-03-22T10:43:44.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.517+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a901990b0 con 0x7f4a900737b0 2024-03-22T10:43:44.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.519+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4a8400f450 con 0x7f4a900737b0 2024-03-22T10:43:44.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.519+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9010f290 con 0x7f4a900737b0 2024-03-22T10:43:44.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.519+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a8400e5c0 con 0x7f4a900737b0 2024-03-22T10:43:44.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.521+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f4a8400e7e0 con 0x7f4a900737b0 2024-03-22T10:43:44.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.523+0000 7f4a8d600700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f4a7806dda0 0x7f4a78070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:44.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.523+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f4a8408d6c0 con 0x7f4a900737b0 2024-03-22T10:43:44.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.524+0000 7f4a8fe00700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f4a7806dda0 0x7f4a78070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:44.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.526+0000 7f4a8fe00700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f4a7806dda0 0x7f4a78070250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b5c0 tx=0x7f4a7c005fb0 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:44.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.529+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f4a84056e20 con 0x7f4a900737b0 2024-03-22T10:43:44.752 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:44 smithi115 systemd[1]: Stopping Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:43:44.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a95200700 1 -- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 msgr2=0x7f4a90071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-22T10:43:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a95200700 1 -- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 msgr2=0x7f4a90071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T10:43:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a95200700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 0x7f4a90071c70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c8a0 tx=0x7f4a8400cbb0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T10:43:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a95200700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 0x7f4a90071c70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c8a0 tx=0x7f4a8400cbb0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 msgr2=0x7f4a90071c70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a8d600700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a900737b0 0x7f4a90071c70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a8d600700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 0x7f4a78077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a8d600700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a78078110 0x7f4a7807a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7807afb0 con 0x7f4a78078110 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7807b0f0 con 0x7f4a90074d80 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a95c00700 1 -- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a78078110 msgr2=0x7f4a7807a590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:3300/0 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.973+0000 7f4a95c00700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a78078110 0x7f4a7807a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.974+0000 7f4a95200700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 0x7f4a78077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.975+0000 7f4a95200700 1 -- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a78078110 msgr2=0x7f4a7807a590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:43:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.975+0000 7f4a95200700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a78078110 0x7f4a7807a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.975+0000 7f4a95200700 1 -- 172.21.15.115:0/2903854168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=3+}) v3 -- 0x7f4a7807b280 con 0x7f4a90074d80 2024-03-22T10:43:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.975+0000 7f4a95200700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 0x7f4a78077960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a8408b7e0 tx=0x7f4a8400a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.976+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a84015e00 con 0x7f4a90074d80 2024-03-22T10:43:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.976+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4a84058e80 con 0x7f4a90074d80 2024-03-22T10:43:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:44.978+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f4a8408ba60 con 0x7f4a90074d80 2024-03-22T10:43:44.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.005+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4a901997b0 con 0x7f4a90074d80 2024-03-22T10:43:44.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.006+0000 7f4a8d600700 1 -- 172.21.15.115:0/2903854168 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f4a8406aed0 con 0x7f4a90074d80 2024-03-22T10:43:44.998 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:43:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:43:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:43:44.999 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:43:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.009+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f4a7806dda0 msgr2=0x7f4a78070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.009+0000 7f4a97c00700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f4a7806dda0 0x7f4a78070250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b5c0 tx=0x7f4a7c005fb0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.010+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 msgr2=0x7f4a78077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.010+0000 7f4a97c00700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 0x7f4a78077960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a8408b7e0 tx=0x7f4a8400a400 comp rx=0 tx=0).stop 2024-03-22T10:43:45.002 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:44 smithi115 ceph-mon[29111]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:45.002 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:44 smithi115 ceph-mon[29111]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:45.003 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:44 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[29107]: 2024-03-22T10:43:44.970+0000 7fc4b5400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi115 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T10:43:45.003 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:44 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[29107]: 2024-03-22T10:43:44.970+0000 7fc4b5400700 -1 mon.smithi115@0(leader) e2 *** Got Signal Terminated *** 2024-03-22T10:43:45.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.010+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 shutdown_connections 2024-03-22T10:43:45.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.010+0000 7f4a97c00700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a78078110 0x7f4a7807a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.010+0000 7f4a97c00700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a90074d80 0x7f4a78077960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.010+0000 7f4a97c00700 1 --2- 172.21.15.115:0/2903854168 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f4a7806dda0 0x7f4a78070250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.010+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 >> 172.21.15.115:0/2903854168 conn(0x7f4a900fb6e0 msgr2=0x7f4a901019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:45.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.010+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 shutdown_connections 2024-03-22T10:43:45.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.011+0000 7f4a97c00700 1 -- 172.21.15.115:0/2903854168 wait complete. 2024-03-22T10:43:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.203+0000 7f2de6200700 1 -- 172.21.15.115:0/2811744530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0100c40 msgr2=0x7f2de0103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.203+0000 7f2de6200700 1 --2- 172.21.15.115:0/2811744530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0100c40 0x7f2de0103020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009910 tx=0x7f2dcc009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.203+0000 7f2de6200700 1 -- 172.21.15.115:0/2811744530 shutdown_connections 2024-03-22T10:43:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.203+0000 7f2de6200700 1 --2- 172.21.15.115:0/2811744530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2de0103560 0x7f2de0105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.203+0000 7f2de6200700 1 --2- 172.21.15.115:0/2811744530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0100c40 0x7f2de0103020 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.203+0000 7f2de6200700 1 -- 172.21.15.115:0/2811744530 >> 172.21.15.115:0/2811744530 conn(0x7f2de00fa5e0 msgr2=0x7f2de00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.204+0000 7f2de6200700 1 -- 172.21.15.115:0/2811744530 shutdown_connections 2024-03-22T10:43:45.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.204+0000 7f2de6200700 1 -- 172.21.15.115:0/2811744530 wait complete. 2024-03-22T10:43:45.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.205+0000 7f2de6200700 1 Processor -- start 2024-03-22T10:43:45.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.205+0000 7f2de6200700 1 -- start start 2024-03-22T10:43:45.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.205+0000 7f2de6200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2de0100c40 0x7f2de0195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:45.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.205+0000 7f2de6200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0103560 0x7f2de0195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:45.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.205+0000 7f2ddf400700 1 -- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2de0100c40 msgr2=0x7f2de0195a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:3300/0 2024-03-22T10:43:45.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.205+0000 7f2de6200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de0196580 con 0x7f2de0100c40 2024-03-22T10:43:45.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.205+0000 7f2ddf400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2de0100c40 0x7f2de0195a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:43:45.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.205+0000 7f2de6200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de01966c0 con 0x7f2de0103560 2024-03-22T10:43:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.206+0000 7f2ddea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0103560 0x7f2de0195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.206+0000 7f2ddea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0103560 0x7f2de0195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:45526/0 (socket says 172.21.15.115:45526) 2024-03-22T10:43:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.206+0000 7f2ddea00700 1 -- 172.21.15.115:0/20533447 learned_addr learned my addr 172.21.15.115:0/20533447 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.206+0000 7f2ddea00700 1 -- 172.21.15.115:0/20533447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2de0100c40 msgr2=0x7f2de0195a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:43:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.206+0000 7f2ddea00700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2de0100c40 0x7f2de0195a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.206+0000 7f2ddea00700 1 -- 172.21.15.115:0/20533447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0095f0 con 0x7f2de0103560 2024-03-22T10:43:45.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.207+0000 7f2ddea00700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0103560 0x7f2de0195f60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800b9b0 tx=0x7f2dd800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:45.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.207+0000 7f2dd7e00700 1 -- 172.21.15.115:0/20533447 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dd800c980 con 0x7f2de0103560 2024-03-22T10:43:45.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.207+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de019b120 con 0x7f2de0103560 2024-03-22T10:43:45.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.207+0000 7f2dd7e00700 1 -- 172.21.15.115:0/20533447 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2dd8014440 con 0x7f2de0103560 2024-03-22T10:43:45.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.207+0000 7f2dd7e00700 1 -- 172.21.15.115:0/20533447 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dd8012530 con 0x7f2de0103560 2024-03-22T10:43:45.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.207+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de019b6f0 con 0x7f2de0103560 2024-03-22T10:43:45.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.209+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de018fc10 con 0x7f2de0103560 2024-03-22T10:43:45.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.210+0000 7f2dd7e00700 1 -- 172.21.15.115:0/20533447 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f2dd80145b0 con 0x7f2de0103560 2024-03-22T10:43:45.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.211+0000 7f2dd7e00700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2dc406dd00 0x7f2dc40701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:45.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.211+0000 7f2dd7e00700 1 -- 172.21.15.115:0/20533447 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f2dd808d3b0 con 0x7f2de0103560 2024-03-22T10:43:45.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.212+0000 7f2ddf400700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2dc406dd00 0x7f2dc40701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:45.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.212+0000 7f2ddf400700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2dc406dd00 0x7f2dc40701b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc005090 tx=0x7f2dcc009fd0 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:45.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.219+0000 7f2dd7e00700 1 -- 172.21.15.115:0/20533447 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f2dd805a880 con 0x7f2de0103560 2024-03-22T10:43:45.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:45 smithi115 podman[78440]: 2024-03-22 10:43:45.159762628 +0000 UTC m=+0.369521516 container died d5ceb65ee6ecd123f66507043c8e07dfa6c7f5dec86dbd2cfc6631c2f6bf7981 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.build-date=20231212, io.buildah.version=1.29.1) 2024-03-22T10:43:45.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:44 smithi150 ceph-mon[34518]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:45.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:44 smithi150 ceph-mon[34518]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:45.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.571+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2de002cde0 con 0x7f2de0103560 2024-03-22T10:43:45.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.572+0000 7f2dd7e00700 1 -- 172.21.15.115:0/20533447 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f2dd8019770 con 0x7f2de0103560 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout:e14 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 14 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:43:45.565 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:43:22.786262+0000 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:43:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:45.567 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.576+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2dc406dd00 msgr2=0x7f2dc40701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.576+0000 7f2de6200700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2dc406dd00 0x7f2dc40701b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc005090 tx=0x7f2dcc009fd0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.576+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0103560 msgr2=0x7f2de0195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.576+0000 7f2de6200700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0103560 0x7f2de0195f60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800b9b0 tx=0x7f2dd800bd70 comp rx=0 tx=0).stop 2024-03-22T10:43:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.576+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 shutdown_connections 2024-03-22T10:43:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.576+0000 7f2de6200700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f2dc406dd00 0x7f2dc40701b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.576+0000 7f2de6200700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2de0100c40 0x7f2de0195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.576+0000 7f2de6200700 1 --2- 172.21.15.115:0/20533447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de0103560 0x7f2de0195f60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.576+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 >> 172.21.15.115:0/20533447 conn(0x7f2de00fa5e0 msgr2=0x7f2de00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:45.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.577+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 shutdown_connections 2024-03-22T10:43:45.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.577+0000 7f2de6200700 1 -- 172.21.15.115:0/20533447 wait complete. 2024-03-22T10:43:45.570 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 14 2024-03-22T10:43:45.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.759+0000 7fdb5f800700 1 -- 172.21.15.115:0/3196534093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58101350 msgr2=0x7fdb581017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:45.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.759+0000 7fdb5f800700 1 --2- 172.21.15.115:0/3196534093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58101350 0x7fdb581017a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009910 tx=0x7fdb44009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:45.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.759+0000 7fdb5f800700 1 -- 172.21.15.115:0/3196534093 shutdown_connections 2024-03-22T10:43:45.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.759+0000 7fdb5f800700 1 --2- 172.21.15.115:0/3196534093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58101350 0x7fdb581017a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.759+0000 7fdb5f800700 1 --2- 172.21.15.115:0/3196534093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb58100150 0x7fdb58100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.759+0000 7fdb5f800700 1 -- 172.21.15.115:0/3196534093 >> 172.21.15.115:0/3196534093 conn(0x7fdb580fb6e0 msgr2=0x7fdb580fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:45.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.760+0000 7fdb5f800700 1 -- 172.21.15.115:0/3196534093 shutdown_connections 2024-03-22T10:43:45.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.760+0000 7fdb5f800700 1 -- 172.21.15.115:0/3196534093 wait complete. 2024-03-22T10:43:45.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.761+0000 7fdb5f800700 1 Processor -- start 2024-03-22T10:43:45.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.761+0000 7fdb5f800700 1 -- start start 2024-03-22T10:43:45.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.762+0000 7fdb5f800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58100150 0x7fdb58197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:45.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.762+0000 7fdb5f800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb58101350 0x7fdb58198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:45.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.762+0000 7fdb5f800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb58198790 con 0x7fdb58101350 2024-03-22T10:43:45.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.762+0000 7fdb5f800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5819d1a0 con 0x7fdb58100150 2024-03-22T10:43:45.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.762+0000 7fdb57e00700 1 -- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb58101350 msgr2=0x7fdb58198170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:3300/0 2024-03-22T10:43:45.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.762+0000 7fdb57e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb58101350 0x7fdb58198170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:43:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.762+0000 7fdb5ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58100150 0x7fdb58197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.762+0000 7fdb5ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58100150 0x7fdb58197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:45542/0 (socket says 172.21.15.115:45542) 2024-03-22T10:43:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.762+0000 7fdb5ce00700 1 -- 172.21.15.115:0/1583689013 learned_addr learned my addr 172.21.15.115:0/1583689013 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.763+0000 7fdb5ce00700 1 -- 172.21.15.115:0/1583689013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb58101350 msgr2=0x7fdb58198170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:43:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.763+0000 7fdb5ce00700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb58101350 0x7fdb58198170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.763+0000 7fdb5ce00700 1 -- 172.21.15.115:0/1583689013 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440095f0 con 0x7fdb58100150 2024-03-22T10:43:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.764+0000 7fdb5ce00700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58100150 0x7fdb58197c30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00b4d0 tx=0x7fdb4c00b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.764+0000 7fdb55600700 1 -- 172.21.15.115:0/1583689013 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb4c011800 con 0x7fdb58100150 2024-03-22T10:43:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.764+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5819d3a0 con 0x7fdb58100150 2024-03-22T10:43:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.764+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5819d940 con 0x7fdb58100150 2024-03-22T10:43:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.765+0000 7fdb55600700 1 -- 172.21.15.115:0/1583689013 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb4c011e10 con 0x7fdb58100150 2024-03-22T10:43:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.765+0000 7fdb55600700 1 -- 172.21.15.115:0/1583689013 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb4c00f2e0 con 0x7fdb58100150 2024-03-22T10:43:45.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.766+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb38005320 con 0x7fdb58100150 2024-03-22T10:43:45.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.769+0000 7fdb55600700 1 -- 172.21.15.115:0/1583689013 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fdb4c011960 con 0x7fdb58100150 2024-03-22T10:43:45.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.770+0000 7fdb55600700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb4006dd00 0x7fdb400701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:45.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.770+0000 7fdb55600700 1 -- 172.21.15.115:0/1583689013 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fdb4c08d7f0 con 0x7fdb58100150 2024-03-22T10:43:45.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.770+0000 7fdb57e00700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb4006dd00 0x7fdb400701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:45.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.771+0000 7fdb57e00700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb4006dd00 0x7fdb400701b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005090 tx=0x7fdb44009fd0 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:45.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:45.784+0000 7fdb55600700 1 -- 172.21.15.115:0/1583689013 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7fdb4c0570e0 con 0x7fdb58100150 2024-03-22T10:43:46.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.111+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 --> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb38000bf0 con 0x7fdb4006dd00 2024-03-22T10:43:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.113+0000 7fdb55600700 1 -- 172.21.15.115:0/1583689013 <== mgr.14710 v2:172.21.15.115:6800/246266260 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fdb38000bf0 con 0x7fdb4006dd00 2024-03-22T10:43:46.109 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:43:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:43:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:43:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:43:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:43:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr" 2024-03-22T10:43:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:43:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "2/23 daemons upgraded", 2024-03-22T10:43:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading mon daemons", 2024-03-22T10:43:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:43:46.110 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:43:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.118+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb4006dd00 msgr2=0x7fdb400701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.118+0000 7fdb5f800700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb4006dd00 0x7fdb400701b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005090 tx=0x7fdb44009fd0 comp rx=0 tx=0).stop 2024-03-22T10:43:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.118+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58100150 msgr2=0x7fdb58197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.118+0000 7fdb5f800700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58100150 0x7fdb58197c30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00b4d0 tx=0x7fdb4c00b890 comp rx=0 tx=0).stop 2024-03-22T10:43:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.118+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 shutdown_connections 2024-03-22T10:43:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.118+0000 7fdb5f800700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7fdb4006dd00 0x7fdb400701b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.118+0000 7fdb5f800700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb58100150 0x7fdb58197c30 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00b4d0 tx=0x7fdb4c00b890 comp rx=0 tx=0).stop 2024-03-22T10:43:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.118+0000 7fdb5f800700 1 --2- 172.21.15.115:0/1583689013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb58101350 0x7fdb58198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.118+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 >> 172.21.15.115:0/1583689013 conn(0x7fdb580fb6e0 msgr2=0x7fdb58102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.119+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 shutdown_connections 2024-03-22T10:43:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.119+0000 7fdb5f800700 1 -- 172.21.15.115:0/1583689013 wait complete. 2024-03-22T10:43:46.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.276+0000 7f1f40c00700 1 -- 172.21.15.115:0/3553989781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 msgr2=0x7f1f3c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.276+0000 7f1f40c00700 1 --2- 172.21.15.115:0/3553989781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 0x7f1f3c100560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009910 tx=0x7f1f2c009c20 comp rx=0 tx=0).stop 2024-03-22T10:43:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.276+0000 7f1f40c00700 1 -- 172.21.15.115:0/3553989781 shutdown_connections 2024-03-22T10:43:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.276+0000 7f1f40c00700 1 --2- 172.21.15.115:0/3553989781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f3c101350 0x7f1f3c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.276+0000 7f1f40c00700 1 --2- 172.21.15.115:0/3553989781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 0x7f1f3c100560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.276+0000 7f1f40c00700 1 -- 172.21.15.115:0/3553989781 >> 172.21.15.115:0/3553989781 conn(0x7f1f3c0fb6e0 msgr2=0x7f1f3c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.276+0000 7f1f40c00700 1 -- 172.21.15.115:0/3553989781 shutdown_connections 2024-03-22T10:43:46.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.276+0000 7f1f40c00700 1 -- 172.21.15.115:0/3553989781 wait complete. 2024-03-22T10:43:46.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.278+0000 7f1f40c00700 1 Processor -- start 2024-03-22T10:43:46.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.278+0000 7f1f40c00700 1 -- start start 2024-03-22T10:43:46.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.278+0000 7f1f40c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 0x7f1f3c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:46.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.278+0000 7f1f40c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f3c101350 0x7f1f3c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:46.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.278+0000 7f1f40c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c1987c0 con 0x7f1f3c101350 2024-03-22T10:43:46.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.278+0000 7f1f40c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c19d1d0 con 0x7f1f3c100150 2024-03-22T10:43:46.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.278+0000 7f1f39400700 1 -- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f3c101350 msgr2=0x7f1f3c1981a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:3300/0 2024-03-22T10:43:46.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.278+0000 7f1f39400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f3c101350 0x7f1f3c1981a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T10:43:46.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.279+0000 7f1f39e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 0x7f1f3c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:46.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.279+0000 7f1f39e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 0x7f1f3c197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:45548/0 (socket says 172.21.15.115:45548) 2024-03-22T10:43:46.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.279+0000 7f1f39e00700 1 -- 172.21.15.115:0/2756391075 learned_addr learned my addr 172.21.15.115:0/2756391075 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:43:46.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.279+0000 7f1f39e00700 1 -- 172.21.15.115:0/2756391075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f3c101350 msgr2=0x7f1f3c1981a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T10:43:46.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.279+0000 7f1f39e00700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f3c101350 0x7f1f3c1981a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:46.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.279+0000 7f1f39e00700 1 -- 172.21.15.115:0/2756391075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0095f0 con 0x7f1f3c100150 2024-03-22T10:43:46.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.280+0000 7f1f39e00700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 0x7f1f3c197c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c006010 tx=0x7f1f2c00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:46.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.280+0000 7f1f32a00700 1 -- 172.21.15.115:0/2756391075 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2c01d070 con 0x7f1f3c100150 2024-03-22T10:43:46.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.280+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c19d370 con 0x7f1f3c100150 2024-03-22T10:43:46.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.280+0000 7f1f32a00700 1 -- 172.21.15.115:0/2756391075 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1f2c00f460 con 0x7f1f3c100150 2024-03-22T10:43:46.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.280+0000 7f1f32a00700 1 -- 172.21.15.115:0/2756391075 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2c021620 con 0x7f1f3c100150 2024-03-22T10:43:46.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.280+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c19d8b0 con 0x7f1f3c100150 2024-03-22T10:43:46.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.282+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c105430 con 0x7f1f3c100150 2024-03-22T10:43:46.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.283+0000 7f1f32a00700 1 -- 172.21.15.115:0/2756391075 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f1f2c00f5d0 con 0x7f1f3c100150 2024-03-22T10:43:46.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.284+0000 7f1f32a00700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f1f2006dca0 0x7f1f20070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:43:46.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.285+0000 7f1f32a00700 1 -- 172.21.15.115:0/2756391075 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f1f2c08ef00 con 0x7f1f3c100150 2024-03-22T10:43:46.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.285+0000 7f1f39400700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f1f2006dca0 0x7f1f20070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:43:46.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.286+0000 7f1f39400700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f1f2006dca0 0x7f1f20070150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1f28000f20 tx=0x7f1f28006e30 comp rx=0 tx=0).ready entity=mgr.14710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:43:46.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.290+0000 7f1f32a00700 1 -- 172.21.15.115:0/2756391075 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185430 (secure 0 0 0) 0x7f1f2c058680 con 0x7f1f3c100150 2024-03-22T10:43:46.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.710+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1f3c02ce90 con 0x7f1f3c100150 2024-03-22T10:43:46.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.711+0000 7f1f32a00700 1 -- 172.21.15.115:0/2756391075 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1f2c05bd40 con 0x7f1f3c100150 2024-03-22T10:43:46.704 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:43:46.704 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:43:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f1f2006dca0 msgr2=0x7f1f20070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f1f2006dca0 0x7f1f20070150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1f28000f20 tx=0x7f1f28006e30 comp rx=0 tx=0).stop 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 msgr2=0x7f1f3c197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 0x7f1f3c197c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c006010 tx=0x7f1f2c00be20 comp rx=0 tx=0).stop 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 shutdown_connections 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.115:6800/246266260,v1:172.21.15.115:6801/246266260] conn(0x7f1f2006dca0 0x7f1f20070150 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3c100150 0x7f1f3c197c60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 --2- 172.21.15.115:0/2756391075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f3c101350 0x7f1f3c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 >> 172.21.15.115:0/2756391075 conn(0x7f1f3c0fb6e0 msgr2=0x7f1f3c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.714+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 shutdown_connections 2024-03-22T10:43:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:43:46.715+0000 7f1f40c00700 1 -- 172.21.15.115:0/2756391075 wait complete. 2024-03-22T10:43:47.224 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:46 smithi115 podman[78440]: 2024-03-22 10:43:46.889519644 +0000 UTC m=+2.099278520 container cleanup d5ceb65ee6ecd123f66507043c8e07dfa6c7f5dec86dbd2cfc6631c2f6bf7981 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, io.buildah.version=1.29.1, ceph=True, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-03-22T10:43:47.224 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:46 smithi115 bash[78440]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115 2024-03-22T10:43:47.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:47 smithi115 podman[78465]: 2024-03-22 10:43:47.233521605 +0000 UTC m=+2.060747869 container remove d5ceb65ee6ecd123f66507043c8e07dfa6c7f5dec86dbd2cfc6631c2f6bf7981 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-03-22T10:43:47.979 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:47 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115.service: Deactivated successfully. 2024-03-22T10:43:47.979 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:47 smithi115 systemd[1]: Stopped Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:43:47.979 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:47 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115.service: Consumed 26.174s CPU time. 2024-03-22T10:43:48.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:47 smithi115 systemd[1]: Starting Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:43:48.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:48 smithi115 podman[78655]: 2024-03-22 10:43:48.317962456 +0000 UTC m=+0.036484305 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:43:48.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:48 smithi115 podman[78655]: 2024-03-22 10:43:48.434875443 +0000 UTC m=+0.153397278 container create 52dfd9dbb5ad704e5db064ea29c5964479b4acfab8b08bc77ab35825254bc95b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-22T10:43:49.906 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 podman[78655]: 2024-03-22 10:43:49.530831486 +0000 UTC m=+1.249353367 container init 52dfd9dbb5ad704e5db064ea29c5964479b4acfab8b08bc77ab35825254bc95b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 podman[78655]: 2024-03-22 10:43:49.54302447 +0000 UTC m=+1.261546314 container start 52dfd9dbb5ad704e5db064ea29c5964479b4acfab8b08bc77ab35825254bc95b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: pidfile_write: ignore empty --pid-file 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: load: jerasure load: lrc 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: RocksDB version: 7.9.2 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Git sha 0 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: DB SUMMARY 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: DB Session ID: AZH07ZIKQBV01DDB7UJD 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: CURRENT file: CURRENT 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1503 Bytes 2024-03-22T10:43:49.907 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi115/store.db dir, Total Num: 1, files: 000032.sst 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi115/store.db: 000030.log size: 3104852 ; 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.error_if_exists: 0 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.create_if_missing: 0 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.paranoid_checks: 1 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.env: 0x558857c39b20 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.info_log: 0x55885a463840 2024-03-22T10:43:49.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.statistics: (nil) 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.use_fsync: 0 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_log_file_size: 0 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.allow_fallocate: 1 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T10:43:49.909 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.use_direct_reads: 0 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.db_log_dir: 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.wal_dir: 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T10:43:49.910 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.write_buffer_manager: 0x55885a4361e0 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.unordered_write: 0 2024-03-22T10:43:49.911 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.row_cache: None 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.wal_filter: None 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.two_write_queues: 0 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.wal_compression: 0 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.atomic_flush: 0 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T10:43:49.912 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.log_readahead_size: 0 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_background_jobs: 2 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_background_compactions: -1 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_subcompactions: 1 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T10:43:49.913 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_open_files: -1 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_background_flushes: -1 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Compression algorithms supported: 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: kZSTD supported: 0 2024-03-22T10:43:49.914 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: kXpressCompression supported: 0 2024-03-22T10:43:49.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T10:43:49.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: kZlibCompression supported: 1 2024-03-22T10:43:49.915 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: kSnappyCompression supported: 1 2024-03-22T10:43:49.916 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: kLZ4Compression supported: 1 2024-03-22T10:43:49.916 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: kBZip2Compression supported: 0 2024-03-22T10:43:49.916 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T10:43:49.916 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T10:43:49.916 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi115/store.db/MANIFEST-000015 2024-03-22T10:43:49.916 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T10:43:49.916 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.merge_operator: 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_filter: None 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55885a463940) 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: cache_index_and_filter_blocks: 1 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T10:43:49.917 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: pin_top_level_index_and_filter: 1 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: index_type: 0 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: data_block_index_type: 0 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: index_shortening: 1 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: checksum: 4 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: no_block_cache: 0 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache: 0x55885a469090 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache_name: BinnedLRUCache 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache_options: 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: capacity : 536870912 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: num_shard_bits : 4 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: strict_capacity_limit : 0 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: high_pri_pool_ratio: 0.000 2024-03-22T10:43:49.918 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache_compressed: (nil) 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: persistent_cache: (nil) 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_size: 4096 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_size_deviation: 10 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_restart_interval: 16 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: index_block_restart_interval: 1 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: metadata_block_size: 4096 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: partition_filters: 0 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: use_delta_encoding: 1 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: filter_policy: bloomfilter 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: whole_key_filtering: 1 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: verify_compression: 0 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: read_amp_bytes_per_bit: 0 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: format_version: 5 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: enable_index_compression: 1 2024-03-22T10:43:49.919 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_align: 0 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: max_auto_readahead_size: 262144 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: prepopulate_block_cache: 0 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: initial_auto_readahead_size: 8192 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression: NoCompression 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.num_levels: 7 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T10:43:49.920 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T10:43:49.921 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T10:43:49.922 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T10:43:49.923 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.table_properties_collectors: 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.inplace_update_support: 0 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T10:43:49.924 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.bloom_locality: 0 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.max_successive_merges: 0 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.ttl: 2592000 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.enable_blob_files: false 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.min_blob_size: 0 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T10:43:49.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi115/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12469, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0be229ed-63fe-4e11-89f3-2fbdbcb326ae 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711104229604098, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711104229619114, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 2830862, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12470, "largest_seqno": 14437, "table_properties": {"data_size": 2822686, "index_size": 4787, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2373, "raw_key_size": 19692, "raw_average_key_size": 20, "raw_value_size": 2804708, "raw_average_value_size": 2986, "num_data_blocks": 218, "num_entries": 939, "num_filter_entries": 939, "num_deletions": 253, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711104229, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0be229ed-63fe-4e11-89f3-2fbdbcb326ae", "db_session_id": "AZH07ZIKQBV01DDB7UJD", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711104229619392, "job": 1, "event": "recovery_finished"} 2024-03-22T10:43:49.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi115/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55885a55a000 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: DB pointer 0x55885a546000 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** DB Stats ** 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T10:43:49.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** Compaction Stats [default] ** 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: L0 1/0 2.70 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 452.8 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-22T10:43:49.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: L6 1/0 4.92 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-22T10:43:49.929 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Sum 2/0 7.62 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 452.8 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-22T10:43:49.929 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 452.8 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-22T10:43:49.929 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:43:49.929 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** Compaction Stats [default] ** 2024-03-22T10:43:49.929 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T10:43:49.929 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T10:43:49.929 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 452.8 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-22T10:43:49.929 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative compaction: 0.00 GB write, 121.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval compaction: 0.00 GB write, 121.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T10:43:49.930 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Block cache BinnedLRUCache@0x55885a469090#2 capacity: 512.00 MB usage: 7.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.53 KB,0.000482798%) IndexBlock(1,5.08 KB,0.000968575%) Misc(1,0.00 KB,0%) 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: starting mon.smithi115 rank 0 at public addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] at bind addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi115 fsid fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: mon.smithi115@-1(???) e2 preinit fsid fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: mon.smithi115@-1(???).mds e14 new map 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: mon.smithi115@-1(???).mds e14 print_map 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: e14 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: legacy client fscid: 1 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:43:49.931 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Filesystem 'cephfs' (1) 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: fs_name cephfs 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: epoch 14 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: modified 2024-03-22T10:43:22.786262+0000 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: tableserver 0 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: root 0 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: session_timeout 60 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: session_autoclose 300 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: max_file_size 1099511627776 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: max_xattr_size 65536 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: required_client_features {} 2024-03-22T10:43:49.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: last_failure 0 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: last_failure_osd_epoch 0 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: max_mds 2 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: in 0,1 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: up {0=14454,1=24241} 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: failed 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: damaged 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: stopped 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: data_pools [3] 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: metadata_pool 2 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: inline_data enabled 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: balancer 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: bal_rank_mask -1 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: standby_count_wanted 1 2024-03-22T10:43:49.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: qdb_cluster leader: 0 members: 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: [mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: [mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Standby daemons: 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: [mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: [mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: mon.smithi115@-1(???).osd e47 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: mon.smithi115@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: mon.smithi115@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T10:43:49.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: mon.smithi115@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T10:43:49.935 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 ceph-mon[78672]: mon.smithi115@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-03-22T10:43:49.935 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 bash[78655]: 52dfd9dbb5ad704e5db064ea29c5964479b4acfab8b08bc77ab35825254bc95b 2024-03-22T10:43:49.935 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:49 smithi115 systemd[1]: Started Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:43:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: pgmap v41: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-03-22T10:43:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:43:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: from='client.14804 172.21.15.115:0/2903854168' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/20533447' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: from='client.24525 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: pgmap v42: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2756391075' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: pgmap v43: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: pgmap v44: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: mon.smithi115 calling monitor election 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: mon.smithi115 is new leader, mons smithi115,smithi150 in quorum (ranks 0,1) 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: monmap e2: 2 mons at {smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],smithi150=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: mgrmap e31: smithi115.iqzwsz(active, since 84s), standbys: smithi150.fmoumv 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:43:53.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:43:53.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:43:53.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: from='mgr.14710 ' entity='' 2024-03-22T10:43:53.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:43:52 smithi115 ceph-mon[78672]: mgrmap e32: smithi115.iqzwsz(active, since 84s), standbys: smithi150.fmoumv 2024-03-22T10:43:53.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: pgmap v41: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: from='mgr.14710 172.21.15.115:0/2299509731' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: from='client.14804 172.21.15.115:0/2903854168' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/20533447' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: from='client.24525 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: pgmap v42: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/2756391075' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: pgmap v43: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: pgmap v44: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: mon.smithi115 calling monitor election 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: mon.smithi115 is new leader, mons smithi115,smithi150 in quorum (ranks 0,1) 2024-03-22T10:43:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: monmap e2: 2 mons at {smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],smithi150=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T10:43:53.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:43:53.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T10:43:53.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: mgrmap e31: smithi115.iqzwsz(active, since 84s), standbys: smithi150.fmoumv 2024-03-22T10:43:53.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:43:53.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:43:53.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:43:53.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: from='mgr.14710 ' entity='' 2024-03-22T10:43:53.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:43:52 smithi150 ceph-mon[34518]: mgrmap e32: smithi115.iqzwsz(active, since 84s), standbys: smithi150.fmoumv 2024-03-22T10:44:04.361 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: Active manager daemon smithi115.iqzwsz restarted 2024-03-22T10:44:04.362 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: Activating manager daemon smithi115.iqzwsz 2024-03-22T10:44:04.362 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T10:44:04.362 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: mgrmap e33: smithi115.iqzwsz(active, starting, since 0.00996435s), standbys: smithi150.fmoumv 2024-03-22T10:44:04.362 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:44:04.362 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:44:04.362 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:44:04.363 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:44:04.363 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:44:04.363 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:44:04.363 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:44:04.363 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi150.fmoumv", "id": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:44:04.363 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:44:04.363 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: Standby manager daemon smithi150.fmoumv restarted 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: Standby manager daemon smithi150.fmoumv started 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/2235141175' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/crt"}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/2235141175' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: Manager daemon smithi115.iqzwsz is now available 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/2235141175' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/key"}]: dispatch 2024-03-22T10:44:04.364 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:04 smithi150 ceph-mon[34518]: from='mgr.? 172.21.15.150:0/2235141175' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:44:04.371 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: Active manager daemon smithi115.iqzwsz restarted 2024-03-22T10:44:04.371 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: Activating manager daemon smithi115.iqzwsz 2024-03-22T10:44:04.371 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T10:44:04.371 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: mgrmap e33: smithi115.iqzwsz(active, starting, since 0.00996435s), standbys: smithi150.fmoumv 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.iqzwsz", "id": "smithi115.iqzwsz"}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr metadata", "who": "smithi150.fmoumv", "id": "smithi150.fmoumv"}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:44:04.372 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T10:44:04.373 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T10:44:04.373 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T10:44:04.373 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: Standby manager daemon smithi150.fmoumv restarted 2024-03-22T10:44:04.373 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: Standby manager daemon smithi150.fmoumv started 2024-03-22T10:44:04.373 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.? 172.21.15.150:0/2235141175' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/crt"}]: dispatch 2024-03-22T10:44:04.373 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.? 172.21.15.150:0/2235141175' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T10:44:04.373 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: Manager daemon smithi115.iqzwsz is now available 2024-03-22T10:44:04.373 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.? 172.21.15.150:0/2235141175' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi150.fmoumv/key"}]: dispatch 2024-03-22T10:44:04.373 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:04 smithi115 ceph-mon[78672]: from='mgr.? 172.21.15.150:0/2235141175' entity='mgr.smithi150.fmoumv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T10:44:05.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:05 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:44:05.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:05 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:44:05.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:05 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:44:05.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:05 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:44:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:05 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/trash_purge_schedule"}]: dispatch 2024-03-22T10:44:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:05 smithi150 ceph-mon[34518]: [22/Mar/2024:10:44:04] ENGINE Bus STARTING 2024-03-22T10:44:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:05 smithi150 ceph-mon[34518]: mgrmap e34: smithi115.iqzwsz(active, since 1.01542s), standbys: smithi150.fmoumv 2024-03-22T10:44:05.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:44:05.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:44:05.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:44:05.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/mirror_snapshot_schedule"}]: dispatch 2024-03-22T10:44:05.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.iqzwsz/trash_purge_schedule"}]: dispatch 2024-03-22T10:44:05.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:05 smithi115 ceph-mon[78672]: [22/Mar/2024:10:44:04] ENGINE Bus STARTING 2024-03-22T10:44:05.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:05 smithi115 ceph-mon[78672]: mgrmap e34: smithi115.iqzwsz(active, since 1.01542s), standbys: smithi150.fmoumv 2024-03-22T10:44:06.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:06 smithi150 ceph-mon[34518]: [22/Mar/2024:10:44:04] ENGINE Serving on https://172.21.15.115:7150 2024-03-22T10:44:06.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:06 smithi150 ceph-mon[34518]: [22/Mar/2024:10:44:05] ENGINE Serving on http://172.21.15.115:8765 2024-03-22T10:44:06.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:06 smithi150 ceph-mon[34518]: [22/Mar/2024:10:44:05] ENGINE Bus STARTED 2024-03-22T10:44:06.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:06 smithi150 ceph-mon[34518]: pgmap v3: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:44:06.399 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:06 smithi115 ceph-mon[78672]: [22/Mar/2024:10:44:04] ENGINE Serving on https://172.21.15.115:7150 2024-03-22T10:44:06.399 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:06 smithi115 ceph-mon[78672]: [22/Mar/2024:10:44:05] ENGINE Serving on http://172.21.15.115:8765 2024-03-22T10:44:06.399 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:06 smithi115 ceph-mon[78672]: [22/Mar/2024:10:44:05] ENGINE Bus STARTED 2024-03-22T10:44:06.399 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:06 smithi115 ceph-mon[78672]: pgmap v3: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:44:07.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:07 smithi150 ceph-mon[34518]: pgmap v4: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:44:07.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:07 smithi150 ceph-mon[34518]: mgrmap e35: smithi115.iqzwsz(active, since 2s), standbys: smithi150.fmoumv 2024-03-22T10:44:07.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:07 smithi115 ceph-mon[78672]: pgmap v4: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:44:07.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:07 smithi115 ceph-mon[78672]: mgrmap e35: smithi115.iqzwsz(active, since 2s), standbys: smithi150.fmoumv 2024-03-22T10:44:09.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:09 smithi150 ceph-mon[34518]: pgmap v5: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:44:09.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:09 smithi150 ceph-mon[34518]: mgrmap e36: smithi115.iqzwsz(active, since 4s), standbys: smithi150.fmoumv 2024-03-22T10:44:09.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:09 smithi115 ceph-mon[78672]: pgmap v5: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:44:09.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:09 smithi115 ceph-mon[78672]: mgrmap e36: smithi115.iqzwsz(active, since 4s), standbys: smithi150.fmoumv 2024-03-22T10:44:10.597 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:10 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:10.598 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:10 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:10.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:10 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:10.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:10 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:11.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:11 smithi115 ceph-mon[78672]: pgmap v6: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:44:11.762 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:11 smithi150 ceph-mon[34518]: pgmap v6: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:44:13.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:12 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:13.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:12 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:13.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:12 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:13.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:12 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:14.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:13 smithi150 ceph-mon[34518]: pgmap v7: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-22T10:44:14.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:13 smithi115 ceph-mon[78672]: pgmap v7: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-22T10:44:16.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:15 smithi150 ceph-mon[34518]: pgmap v8: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-03-22T10:44:16.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:15 smithi115 ceph-mon[78672]: pgmap v8: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.886+0000 7f9ed0000700 1 -- 172.21.15.115:0/2583406355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 msgr2=0x7f9ec81006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.886+0000 7f9ed0000700 1 --2- 172.21.15.115:0/2583406355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 0x7f9ec81006a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009910 tx=0x7f9eb4009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.887+0000 7f9ed0000700 1 -- 172.21.15.115:0/2583406355 shutdown_connections 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.887+0000 7f9ed0000700 1 --2- 172.21.15.115:0/2583406355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 0x7f9ec81006a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.887+0000 7f9ed0000700 1 --2- 172.21.15.115:0/2583406355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ec80ff050 0x7f9ec80ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.887+0000 7f9ed0000700 1 -- 172.21.15.115:0/2583406355 >> 172.21.15.115:0/2583406355 conn(0x7f9ec80fa5e0 msgr2=0x7f9ec80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.887+0000 7f9ed0000700 1 -- 172.21.15.115:0/2583406355 shutdown_connections 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.887+0000 7f9ed0000700 1 -- 172.21.15.115:0/2583406355 wait complete. 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.888+0000 7f9ed0000700 1 Processor -- start 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.888+0000 7f9ed0000700 1 -- start start 2024-03-22T10:44:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.889+0000 7f9ed0000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ec80ff050 0x7f9ec8195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.889+0000 7f9ed0000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 0x7f9ec8195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.889+0000 7f9ed0000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8196440 con 0x7f9ec8100250 2024-03-22T10:44:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.889+0000 7f9ed0000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8196580 con 0x7f9ec80ff050 2024-03-22T10:44:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.889+0000 7f9eccc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 0x7f9ec8195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.889+0000 7f9eccc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 0x7f9ec8195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35626/0 (socket says 172.21.15.115:35626) 2024-03-22T10:44:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.889+0000 7f9eccc00700 1 -- 172.21.15.115:0/3643934994 learned_addr learned my addr 172.21.15.115:0/3643934994 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.889+0000 7f9ecd600700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ec80ff050 0x7f9ec8195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.890+0000 7f9eccc00700 1 -- 172.21.15.115:0/3643934994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ec80ff050 msgr2=0x7f9ec8195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.890+0000 7f9eccc00700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ec80ff050 0x7f9ec8195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.890+0000 7f9eccc00700 1 -- 172.21.15.115:0/3643934994 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb40095f0 con 0x7f9ec8100250 2024-03-22T10:44:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.890+0000 7f9eccc00700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 0x7f9ec8195eb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4005b40 tx=0x7f9eb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.890+0000 7f9ec6000700 1 -- 172.21.15.115:0/3643934994 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb401d070 con 0x7f9ec8100250 2024-03-22T10:44:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.890+0000 7f9ec6000700 1 -- 172.21.15.115:0/3643934994 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9eb4022470 con 0x7f9ec8100250 2024-03-22T10:44:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.890+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec819afe0 con 0x7f9ec8100250 2024-03-22T10:44:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.890+0000 7f9ec6000700 1 -- 172.21.15.115:0/3643934994 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb400f690 con 0x7f9ec8100250 2024-03-22T10:44:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.890+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec819b4a0 con 0x7f9ec8100250 2024-03-22T10:44:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.892+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec8069030 con 0x7f9ec8100250 2024-03-22T10:44:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.892+0000 7f9ec6000700 1 -- 172.21.15.115:0/3643934994 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f9eb40225e0 con 0x7f9ec8100250 2024-03-22T10:44:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.893+0000 7f9ec6000700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9eb006e0d0 0x7f9eb0070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.894+0000 7f9ec6000700 1 -- 172.21.15.115:0/3643934994 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9eb408f830 con 0x7f9ec8100250 2024-03-22T10:44:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.894+0000 7f9ecd600700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9eb006e0d0 0x7f9eb0070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.894+0000 7f9ecd600700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9eb006e0d0 0x7f9eb0070580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005fd0 tx=0x7f9ebc005e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:16.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:16.899+0000 7f9ec6000700 1 -- 172.21.15.115:0/3643934994 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9eb405c690 con 0x7f9ec8100250 2024-03-22T10:44:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.145+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ec8104ba0 con 0x7f9eb006e0d0 2024-03-22T10:44:17.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.147+0000 7f9ec6000700 1 -- 172.21.15.115:0/3643934994 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f9ec8104ba0 con 0x7f9eb006e0d0 2024-03-22T10:44:17.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.150+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9eb006e0d0 msgr2=0x7f9eb0070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:17.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.150+0000 7f9ed0000700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9eb006e0d0 0x7f9eb0070580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005fd0 tx=0x7f9ebc005e50 comp rx=0 tx=0).stop 2024-03-22T10:44:17.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.150+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 msgr2=0x7f9ec8195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:17.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.150+0000 7f9ed0000700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 0x7f9ec8195eb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4005b40 tx=0x7f9eb400be20 comp rx=0 tx=0).stop 2024-03-22T10:44:17.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.151+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 shutdown_connections 2024-03-22T10:44:17.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.151+0000 7f9ed0000700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9eb006e0d0 0x7f9eb0070580 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.151+0000 7f9ed0000700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ec80ff050 0x7f9ec8195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.151+0000 7f9ed0000700 1 --2- 172.21.15.115:0/3643934994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ec8100250 0x7f9ec8195eb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.151+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 >> 172.21.15.115:0/3643934994 conn(0x7f9ec80fa5e0 msgr2=0x7f9ec8103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:17.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.151+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 shutdown_connections 2024-03-22T10:44:17.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.151+0000 7f9ed0000700 1 -- 172.21.15.115:0/3643934994 wait complete. 2024-03-22T10:44:17.157 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:44:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.340+0000 7f1de5800700 1 -- 172.21.15.115:0/1705271337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1de00ff050 msgr2=0x7f1de00ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.340+0000 7f1de5800700 1 --2- 172.21.15.115:0/1705271337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1de00ff050 0x7f1de00ff460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009960 tx=0x7f1dd0009c70 comp rx=0 tx=0).stop 2024-03-22T10:44:17.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.340+0000 7f1de5800700 1 -- 172.21.15.115:0/1705271337 shutdown_connections 2024-03-22T10:44:17.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.340+0000 7f1de5800700 1 --2- 172.21.15.115:0/1705271337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0100250 0x7f1de01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.340+0000 7f1de5800700 1 --2- 172.21.15.115:0/1705271337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1de00ff050 0x7f1de00ff460 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.340+0000 7f1de5800700 1 -- 172.21.15.115:0/1705271337 >> 172.21.15.115:0/1705271337 conn(0x7f1de00fa5e0 msgr2=0x7f1de00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:17.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.340+0000 7f1de5800700 1 -- 172.21.15.115:0/1705271337 shutdown_connections 2024-03-22T10:44:17.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.341+0000 7f1de5800700 1 -- 172.21.15.115:0/1705271337 wait complete. 2024-03-22T10:44:17.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.342+0000 7f1de5800700 1 Processor -- start 2024-03-22T10:44:17.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.342+0000 7f1de5800700 1 -- start start 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.342+0000 7f1de5800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1de00ff050 0x7f1de010eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.342+0000 7f1de5800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0100250 0x7f1de010f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.342+0000 7f1de5800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0114150 con 0x7f1de00ff050 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.342+0000 7f1de5800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de010f540 con 0x7f1de0100250 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.343+0000 7f1dde000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0100250 0x7f1de010f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.343+0000 7f1dde000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0100250 0x7f1de010f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.115:49614/0 (socket says 172.21.15.115:49614) 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.343+0000 7f1dde000700 1 -- 172.21.15.115:0/3103898944 learned_addr learned my addr 172.21.15.115:0/3103898944 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.343+0000 7f1ddea00700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1de00ff050 0x7f1de010eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.344+0000 7f1dde000700 1 -- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1de00ff050 msgr2=0x7f1de010eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:17.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.344+0000 7f1dde000700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1de00ff050 0x7f1de010eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.344+0000 7f1dde000700 1 -- 172.21.15.115:0/3103898944 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd00095f0 con 0x7f1de0100250 2024-03-22T10:44:17.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.344+0000 7f1ddea00700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1de00ff050 0x7f1de010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:44:17.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.345+0000 7f1dde000700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0100250 0x7f1de010f000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d8d0 tx=0x7f1dcc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:17.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.345+0000 7f1dd7400700 1 -- 172.21.15.115:0/3103898944 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dcc009940 con 0x7f1de0100250 2024-03-22T10:44:17.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.345+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de01a9fe0 con 0x7f1de0100250 2024-03-22T10:44:17.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.345+0000 7f1dd7400700 1 -- 172.21.15.115:0/3103898944 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1dcc010460 con 0x7f1de0100250 2024-03-22T10:44:17.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.345+0000 7f1dd7400700 1 -- 172.21.15.115:0/3103898944 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dcc00f5f0 con 0x7f1de0100250 2024-03-22T10:44:17.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.345+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de01aa4d0 con 0x7f1de0100250 2024-03-22T10:44:17.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.347+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de0106a80 con 0x7f1de0100250 2024-03-22T10:44:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.348+0000 7f1dd7400700 1 -- 172.21.15.115:0/3103898944 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 36) v1 ==== 91937+0+0 (secure 0 0 0) 0x7f1dcc0105d0 con 0x7f1de0100250 2024-03-22T10:44:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.349+0000 7f1dd7400700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1dc406e120 0x7f1dc40705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.349+0000 7f1dd7400700 1 -- 172.21.15.115:0/3103898944 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1dcc08df00 con 0x7f1de0100250 2024-03-22T10:44:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.350+0000 7f1ddea00700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1dc406e120 0x7f1dc40705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.351+0000 7f1ddea00700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1dc406e120 0x7f1dc40705d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0006010 tx=0x7f1dd00058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.364+0000 7f1dd7400700 1 -- 172.21.15.115:0/3103898944 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1dcc05acc0 con 0x7f1de0100250 2024-03-22T10:44:17.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.785+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1de0104ba0 con 0x7f1dc406e120 2024-03-22T10:44:17.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.787+0000 7f1dd7400700 1 -- 172.21.15.115:0/3103898944 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f1de0104ba0 con 0x7f1dc406e120 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.789+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1dc406e120 msgr2=0x7f1dc40705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.789+0000 7f1de5800700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1dc406e120 0x7f1dc40705d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0006010 tx=0x7f1dd00058e0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.789+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0100250 msgr2=0x7f1de010f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.789+0000 7f1de5800700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0100250 0x7f1de010f000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d8d0 tx=0x7f1dcc00dc90 comp rx=0 tx=0).stop 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.790+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 shutdown_connections 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.790+0000 7f1de5800700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1dc406e120 0x7f1dc40705d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.790+0000 7f1de5800700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1de00ff050 0x7f1de010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.790+0000 7f1de5800700 1 --2- 172.21.15.115:0/3103898944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1de0100250 0x7f1de010f000 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.790+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 >> 172.21.15.115:0/3103898944 conn(0x7f1de00fa5e0 msgr2=0x7f1de0103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.790+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 shutdown_connections 2024-03-22T10:44:17.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.790+0000 7f1de5800700 1 -- 172.21.15.115:0/3103898944 wait complete. 2024-03-22T10:44:17.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.954+0000 7f31ef000700 1 -- 172.21.15.115:0/3156869534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81035b0 msgr2=0x7f31e8103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:17.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.954+0000 7f31ef000700 1 --2- 172.21.15.115:0/3156869534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81035b0 0x7f31e8103a00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009910 tx=0x7f31d4009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:17.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.954+0000 7f31ef000700 1 -- 172.21.15.115:0/3156869534 shutdown_connections 2024-03-22T10:44:17.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.954+0000 7f31ef000700 1 --2- 172.21.15.115:0/3156869534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81035b0 0x7f31e8103a00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.954+0000 7f31ef000700 1 --2- 172.21.15.115:0/3156869534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e81023b0 0x7f31e81027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.954+0000 7f31ef000700 1 -- 172.21.15.115:0/3156869534 >> 172.21.15.115:0/3156869534 conn(0x7f31e80fd960 msgr2=0x7f31e80ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:17.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.954+0000 7f31ef000700 1 -- 172.21.15.115:0/3156869534 shutdown_connections 2024-03-22T10:44:17.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.954+0000 7f31ef000700 1 -- 172.21.15.115:0/3156869534 wait complete. 2024-03-22T10:44:17.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.955+0000 7f31ef000700 1 Processor -- start 2024-03-22T10:44:17.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.955+0000 7f31ef000700 1 -- start start 2024-03-22T10:44:17.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.956+0000 7f31ef000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81023b0 0x7f31e8071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:17.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.956+0000 7f31ef000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e81035b0 0x7f31e8072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:17.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.956+0000 7f31ef000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8072850 con 0x7f31e81023b0 2024-03-22T10:44:17.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.956+0000 7f31ef000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8072990 con 0x7f31e81035b0 2024-03-22T10:44:17.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.956+0000 7f31e7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81023b0 0x7f31e8071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:17.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.956+0000 7f31e7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81023b0 0x7f31e8071cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35658/0 (socket says 172.21.15.115:35658) 2024-03-22T10:44:17.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.956+0000 7f31e7e00700 1 -- 172.21.15.115:0/2376437983 learned_addr learned my addr 172.21.15.115:0/2376437983 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.956+0000 7f31e7400700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e81035b0 0x7f31e8072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.957+0000 7f31e7e00700 1 -- 172.21.15.115:0/2376437983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e81035b0 msgr2=0x7f31e8072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.957+0000 7f31e7e00700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e81035b0 0x7f31e8072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.957+0000 7f31e7e00700 1 -- 172.21.15.115:0/2376437983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d40095f0 con 0x7f31e81023b0 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.957+0000 7f31e7e00700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81023b0 0x7f31e8071cf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f31d800d900 tx=0x7f31d800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.957+0000 7f31e4c00700 1 -- 172.21.15.115:0/2376437983 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31d8009900 con 0x7f31e81023b0 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.957+0000 7f31e4c00700 1 -- 172.21.15.115:0/2376437983 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f31d8010460 con 0x7f31e81023b0 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.957+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e81a5b30 con 0x7f31e81023b0 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.957+0000 7f31e4c00700 1 -- 172.21.15.115:0/2376437983 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31d800f5b0 con 0x7f31e81023b0 2024-03-22T10:44:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.957+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e81a6080 con 0x7f31e81023b0 2024-03-22T10:44:17.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.959+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e810af20 con 0x7f31e81023b0 2024-03-22T10:44:17.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.960+0000 7f31e4c00700 1 -- 172.21.15.115:0/2376437983 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f31d80105d0 con 0x7f31e81023b0 2024-03-22T10:44:17.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.961+0000 7f31e4c00700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f31cc06e0e0 0x7f31cc070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:17.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.962+0000 7f31e4c00700 1 -- 172.21.15.115:0/2376437983 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f31d808dc80 con 0x7f31e81023b0 2024-03-22T10:44:17.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.962+0000 7f31e7400700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f31cc06e0e0 0x7f31cc070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:17.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.963+0000 7f31e7400700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f31cc06e0e0 0x7f31cc070590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f31d400b5c0 tx=0x7f31d4005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:17.967+0000 7f31e4c00700 1 -- 172.21.15.115:0/2376437983 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f31d805aac0 con 0x7f31e81023b0 2024-03-22T10:44:18.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:17 smithi150 ceph-mon[34518]: pgmap v9: 65 pgs: 65 active+clean; 322 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-03-22T10:44:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.242+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f31e8107f00 con 0x7f31cc06e0e0 2024-03-22T10:44:18.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:17 smithi115 ceph-mon[78672]: pgmap v9: 65 pgs: 65 active+clean; 322 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-03-22T10:44:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.259+0000 7f31e4c00700 1 -- 172.21.15.115:0/2376437983 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f31e8107f00 con 0x7f31cc06e0e0 2024-03-22T10:44:18.253 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:44:18.253 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (10m) 41s ago 12m 14.2M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (13m) 41s ago 12m 7894k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (11m) 8s ago 11m 7810k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (12m) 41s ago 12m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (11m) 8s ago 11m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (10m) 41s ago 12m 74.1M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (5m) 41s ago 5m 175M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (5m) 41s ago 5m 14.3M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (5m) 8s ago 5m 14.7M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (5m) 8s ago 5m 160M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (2m) 41s ago 14m 493M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (52s) 8s ago 11m 447M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (14m) 41s ago 14m 54.2M 2048M 18.2.0 dc2bc1663786 d5ceb65ee6ec 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (11m) 8s ago 11m 53.2M 2048M 18.2.0 dc2bc1663786 7bc06c4be590 2024-03-22T10:44:18.254 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (12m) 41s ago 12m 7990k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:44:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (11m) 8s ago 11m 8212k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:44:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (10m) 41s ago 10m 180M 1217M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:44:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (9m) 41s ago 9m 190M 1217M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:44:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (9m) 41s ago 9m 153M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:44:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (8m) 8s ago 8m 214M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:44:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (8m) 8s ago 8m 197M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:44:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (7m) 8s ago 7m 182M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:44:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (61s) 41s ago 12m 52.4M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:44:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.266+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f31cc06e0e0 msgr2=0x7f31cc070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.266+0000 7f31ef000700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f31cc06e0e0 0x7f31cc070590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f31d400b5c0 tx=0x7f31d4005ee0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.266+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81023b0 msgr2=0x7f31e8071cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.266+0000 7f31ef000700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81023b0 0x7f31e8071cf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f31d800d900 tx=0x7f31d800dc10 comp rx=0 tx=0).stop 2024-03-22T10:44:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.266+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 shutdown_connections 2024-03-22T10:44:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.266+0000 7f31ef000700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f31cc06e0e0 0x7f31cc070590 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f31d400b5c0 tx=0x7f31d4005ee0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.266+0000 7f31ef000700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31e81023b0 0x7f31e8071cf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.266+0000 7f31ef000700 1 --2- 172.21.15.115:0/2376437983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e81035b0 0x7f31e8072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.266+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 >> 172.21.15.115:0/2376437983 conn(0x7f31e80fd960 msgr2=0x7f31e81067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.267+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 shutdown_connections 2024-03-22T10:44:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.267+0000 7f31ef000700 1 -- 172.21.15.115:0/2376437983 wait complete. 2024-03-22T10:44:18.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.433+0000 7f42fde00700 1 -- 172.21.15.115:0/4207867268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f81035c0 msgr2=0x7f42f81059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:18.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.433+0000 7f42fde00700 1 --2- 172.21.15.115:0/4207867268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f81035c0 0x7f42f81059a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009910 tx=0x7f42e4009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:18.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.433+0000 7f42fde00700 1 -- 172.21.15.115:0/4207867268 shutdown_connections 2024-03-22T10:44:18.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.433+0000 7f42fde00700 1 --2- 172.21.15.115:0/4207867268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f81035c0 0x7f42f81059a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.433+0000 7f42fde00700 1 --2- 172.21.15.115:0/4207867268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42f8100ca0 0x7f42f8103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.433+0000 7f42fde00700 1 -- 172.21.15.115:0/4207867268 >> 172.21.15.115:0/4207867268 conn(0x7f42f80fa5e0 msgr2=0x7f42f80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:18.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.433+0000 7f42fde00700 1 -- 172.21.15.115:0/4207867268 shutdown_connections 2024-03-22T10:44:18.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.434+0000 7f42fde00700 1 -- 172.21.15.115:0/4207867268 wait complete. 2024-03-22T10:44:18.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.434+0000 7f42fde00700 1 Processor -- start 2024-03-22T10:44:18.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.435+0000 7f42fde00700 1 -- start start 2024-03-22T10:44:18.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.435+0000 7f42fde00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f8100ca0 0x7f42f8100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:18.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.435+0000 7f42fde00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42f81035c0 0x7f42f80fec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:18.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.435+0000 7f42fde00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80ff190 con 0x7f42f8100ca0 2024-03-22T10:44:18.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.435+0000 7f42fde00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80ff2d0 con 0x7f42f81035c0 2024-03-22T10:44:18.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.435+0000 7f42f6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f8100ca0 0x7f42f8100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:18.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.435+0000 7f42f6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f8100ca0 0x7f42f8100600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35682/0 (socket says 172.21.15.115:35682) 2024-03-22T10:44:18.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.435+0000 7f42f6a00700 1 -- 172.21.15.115:0/367542206 learned_addr learned my addr 172.21.15.115:0/367542206 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.436+0000 7f42f6a00700 1 -- 172.21.15.115:0/367542206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42f81035c0 msgr2=0x7f42f80fec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.436+0000 7f42f6a00700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42f81035c0 0x7f42f80fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.436+0000 7f42f6a00700 1 -- 172.21.15.115:0/367542206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e40095f0 con 0x7f42f8100ca0 2024-03-22T10:44:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.436+0000 7f42f6a00700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f8100ca0 0x7f42f8100600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42e800b4b0 tx=0x7f42e800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.436+0000 7f42fca00700 1 -- 172.21.15.115:0/367542206 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42e8011800 con 0x7f42f8100ca0 2024-03-22T10:44:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.436+0000 7f42fca00700 1 -- 172.21.15.115:0/367542206 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f42e8011e10 con 0x7f42f8100ca0 2024-03-22T10:44:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.436+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f80ff5b0 con 0x7f42f8100ca0 2024-03-22T10:44:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.436+0000 7f42fca00700 1 -- 172.21.15.115:0/367542206 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42e800f2e0 con 0x7f42f8100ca0 2024-03-22T10:44:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.437+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f819da00 con 0x7f42f8100ca0 2024-03-22T10:44:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.438+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f818fbf0 con 0x7f42f8100ca0 2024-03-22T10:44:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.438+0000 7f42fca00700 1 -- 172.21.15.115:0/367542206 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f42e8010380 con 0x7f42f8100ca0 2024-03-22T10:44:18.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.439+0000 7f42fca00700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f42dc06e0d0 0x7f42dc070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:18.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.439+0000 7f42fca00700 1 -- 172.21.15.115:0/367542206 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f42e808da30 con 0x7f42f8100ca0 2024-03-22T10:44:18.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.440+0000 7f42f6000700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f42dc06e0d0 0x7f42dc070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:18.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.440+0000 7f42f6000700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f42dc06e0d0 0x7f42dc070580 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42e400b5c0 tx=0x7f42e4005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:18.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.444+0000 7f42fca00700 1 -- 172.21.15.115:0/367542206 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f42e805a9a0 con 0x7f42f8100ca0 2024-03-22T10:44:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.856+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f42f802ce70 con 0x7f42f8100ca0 2024-03-22T10:44:18.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.857+0000 7f42fca00700 1 -- 172.21.15.115:0/367542206 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+779 (secure 0 0 0) 0x7f42e805a530 con 0x7f42f8100ca0 2024-03-22T10:44:18.848 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:44:18.848 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:44:18.848 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 11, 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 3 2024-03-22T10:44:18.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:44:18.850 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:44:18.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f42dc06e0d0 msgr2=0x7f42dc070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:18.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f42dc06e0d0 0x7f42dc070580 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42e400b5c0 tx=0x7f42e4005fd0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f8100ca0 msgr2=0x7f42f8100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:18.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f8100ca0 0x7f42f8100600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42e800b4b0 tx=0x7f42e800b870 comp rx=0 tx=0).stop 2024-03-22T10:44:18.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 shutdown_connections 2024-03-22T10:44:18.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f42dc06e0d0 0x7f42dc070580 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42f8100ca0 0x7f42f8100600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 --2- 172.21.15.115:0/367542206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42f81035c0 0x7f42f80fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 >> 172.21.15.115:0/367542206 conn(0x7f42f80fa5e0 msgr2=0x7f42f80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.860+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 shutdown_connections 2024-03-22T10:44:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:18.861+0000 7f42fde00700 1 -- 172.21.15.115:0/367542206 wait complete. 2024-03-22T10:44:19.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.038+0000 7fe008400700 1 -- 172.21.15.115:0/463670284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 msgr2=0x7fe0001017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:19.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.038+0000 7fe008400700 1 --2- 172.21.15.115:0/463670284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 0x7fe0001017a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009910 tx=0x7fdfec009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:19.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.039+0000 7fe008400700 1 -- 172.21.15.115:0/463670284 shutdown_connections 2024-03-22T10:44:19.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.039+0000 7fe008400700 1 --2- 172.21.15.115:0/463670284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 0x7fe0001017a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.039+0000 7fe008400700 1 --2- 172.21.15.115:0/463670284 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000100150 0x7fe000100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.039+0000 7fe008400700 1 -- 172.21.15.115:0/463670284 >> 172.21.15.115:0/463670284 conn(0x7fe0000fb6e0 msgr2=0x7fe0000fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:19.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.039+0000 7fe008400700 1 -- 172.21.15.115:0/463670284 shutdown_connections 2024-03-22T10:44:19.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.039+0000 7fe008400700 1 -- 172.21.15.115:0/463670284 wait complete. 2024-03-22T10:44:19.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.040+0000 7fe008400700 1 Processor -- start 2024-03-22T10:44:19.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.040+0000 7fe008400700 1 -- start start 2024-03-22T10:44:19.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.040+0000 7fe008400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000100150 0x7fe000195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:19.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.040+0000 7fe008400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 0x7fe000195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.040+0000 7fe008400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0001964c0 con 0x7fe000101350 2024-03-22T10:44:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.040+0000 7fe008400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000196600 con 0x7fe000100150 2024-03-22T10:44:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.041+0000 7fe005000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 0x7fe000195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.041+0000 7fe005000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 0x7fe000195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35700/0 (socket says 172.21.15.115:35700) 2024-03-22T10:44:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.041+0000 7fe005000700 1 -- 172.21.15.115:0/3770139588 learned_addr learned my addr 172.21.15.115:0/3770139588 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.041+0000 7fe005a00700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000100150 0x7fe000195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.041+0000 7fe005000700 1 -- 172.21.15.115:0/3770139588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000100150 msgr2=0x7fe000195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.041+0000 7fe005000700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000100150 0x7fe000195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.041+0000 7fe005000700 1 -- 172.21.15.115:0/3770139588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0095f0 con 0x7fe000101350 2024-03-22T10:44:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.042+0000 7fe005000700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 0x7fe000195ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00b5c0 tx=0x7fdfec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.042+0000 7fdffe000700 1 -- 172.21.15.115:0/3770139588 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfec01d070 con 0x7fe000101350 2024-03-22T10:44:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.042+0000 7fdffe000700 1 -- 172.21.15.115:0/3770139588 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdfec00f460 con 0x7fe000101350 2024-03-22T10:44:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.042+0000 7fdffe000700 1 -- 172.21.15.115:0/3770139588 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfec021640 con 0x7fe000101350 2024-03-22T10:44:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.042+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00019b050 con 0x7fe000101350 2024-03-22T10:44:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.042+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00019b540 con 0x7fe000101350 2024-03-22T10:44:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.045+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe000108ca0 con 0x7fe000101350 2024-03-22T10:44:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.045+0000 7fdffe000700 1 -- 172.21.15.115:0/3770139588 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fdfec00f5d0 con 0x7fe000101350 2024-03-22T10:44:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.047+0000 7fdffe000700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdfe806e0d0 0x7fdfe8070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.047+0000 7fdffe000700 1 -- 172.21.15.115:0/3770139588 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdfec08f370 con 0x7fe000101350 2024-03-22T10:44:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.048+0000 7fe005a00700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdfe806e0d0 0x7fdfe8070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.049+0000 7fe005a00700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdfe806e0d0 0x7fdfe8070580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdff4005950 tx=0x7fdff40058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.054+0000 7fdffe000700 1 -- 172.21.15.115:0/3770139588 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdfec05c130 con 0x7fe000101350 2024-03-22T10:44:19.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:18 smithi150 ceph-mon[34518]: from='client.34124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:19.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:18 smithi150 ceph-mon[34518]: from='client.24561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:18 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:18 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:18 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/367542206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:44:19.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:18 smithi115 ceph-mon[78672]: from='client.34124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:19.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:18 smithi115 ceph-mon[78672]: from='client.24561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:19.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:18 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:19.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:18 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:19.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:18 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/367542206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:44:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.406+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe000108e60 con 0x7fe000101350 2024-03-22T10:44:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.406+0000 7fdffe000700 1 -- 172.21.15.115:0/3770139588 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1926 (secure 0 0 0) 0x7fdfec05bcc0 con 0x7fe000101350 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout:e14 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 14 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:44:19.399 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:43:22.786262+0000 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:44:19.400 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:44:19.401 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 0 members: 2024-03-22T10:44:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:44:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:44:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:44:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:44:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdfe806e0d0 msgr2=0x7fdfe8070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdfe806e0d0 0x7fdfe8070580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdff4005950 tx=0x7fdff40058e0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 msgr2=0x7fe000195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 0x7fe000195ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00b5c0 tx=0x7fdfec00be20 comp rx=0 tx=0).stop 2024-03-22T10:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 shutdown_connections 2024-03-22T10:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdfe806e0d0 0x7fdfe8070580 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000100150 0x7fe000195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 --2- 172.21.15.115:0/3770139588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe000101350 0x7fe000195ea0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 >> 172.21.15.115:0/3770139588 conn(0x7fe0000fb6e0 msgr2=0x7fe000104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 shutdown_connections 2024-03-22T10:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.410+0000 7fe008400700 1 -- 172.21.15.115:0/3770139588 wait complete. 2024-03-22T10:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 14 2024-03-22T10:44:19.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.609+0000 7fdf26e00700 1 -- 172.21.15.115:0/1947257183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20100150 msgr2=0x7fdf20100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:19.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.609+0000 7fdf26e00700 1 --2- 172.21.15.115:0/1947257183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20100150 0x7fdf20100560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009870 tx=0x7fdf10009b80 comp rx=0 tx=0).stop 2024-03-22T10:44:19.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.610+0000 7fdf26e00700 1 -- 172.21.15.115:0/1947257183 shutdown_connections 2024-03-22T10:44:19.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.610+0000 7fdf26e00700 1 --2- 172.21.15.115:0/1947257183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20101350 0x7fdf201017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.610+0000 7fdf26e00700 1 --2- 172.21.15.115:0/1947257183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20100150 0x7fdf20100560 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.610+0000 7fdf26e00700 1 -- 172.21.15.115:0/1947257183 >> 172.21.15.115:0/1947257183 conn(0x7fdf200fb6e0 msgr2=0x7fdf200fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:19.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.610+0000 7fdf26e00700 1 -- 172.21.15.115:0/1947257183 shutdown_connections 2024-03-22T10:44:19.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.610+0000 7fdf26e00700 1 -- 172.21.15.115:0/1947257183 wait complete. 2024-03-22T10:44:19.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.611+0000 7fdf26e00700 1 Processor -- start 2024-03-22T10:44:19.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.612+0000 7fdf26e00700 1 -- start start 2024-03-22T10:44:19.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.612+0000 7fdf26e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100150 0x7fdf201959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:19.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.612+0000 7fdf26e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20101350 0x7fdf20195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:19.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.612+0000 7fdf26e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf20196500 con 0x7fdf20101350 2024-03-22T10:44:19.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.612+0000 7fdf26e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf20196640 con 0x7fdf20100150 2024-03-22T10:44:19.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.612+0000 7fdf1f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20101350 0x7fdf20195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:19.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.612+0000 7fdf1f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20101350 0x7fdf20195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35726/0 (socket says 172.21.15.115:35726) 2024-03-22T10:44:19.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.612+0000 7fdf1f400700 1 -- 172.21.15.115:0/3390518639 learned_addr learned my addr 172.21.15.115:0/3390518639 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:19.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.613+0000 7fdf1fe00700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100150 0x7fdf201959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:19.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.613+0000 7fdf1f400700 1 -- 172.21.15.115:0/3390518639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100150 msgr2=0x7fdf201959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:19.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.613+0000 7fdf1f400700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100150 0x7fdf201959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.613+0000 7fdf1f400700 1 -- 172.21.15.115:0/3390518639 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10009520 con 0x7fdf20101350 2024-03-22T10:44:19.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.613+0000 7fdf1f400700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20101350 0x7fdf20195ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00e940 tx=0x7fdf0c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:19.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.614+0000 7fdf1cc00700 1 -- 172.21.15.115:0/3390518639 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0c00eee0 con 0x7fdf20101350 2024-03-22T10:44:19.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.614+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2019b0f0 con 0x7fdf20101350 2024-03-22T10:44:19.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.614+0000 7fdf1cc00700 1 -- 172.21.15.115:0/3390518639 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf0c004ce0 con 0x7fdf20101350 2024-03-22T10:44:19.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.614+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2019b6c0 con 0x7fdf20101350 2024-03-22T10:44:19.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.615+0000 7fdf1cc00700 1 -- 172.21.15.115:0/3390518639 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0c005650 con 0x7fdf20101350 2024-03-22T10:44:19.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.616+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf20108ca0 con 0x7fdf20101350 2024-03-22T10:44:19.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.617+0000 7fdf1cc00700 1 -- 172.21.15.115:0/3390518639 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fdf0c004750 con 0x7fdf20101350 2024-03-22T10:44:19.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.618+0000 7fdf1cc00700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdf0406e0d0 0x7fdf04070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.619+0000 7fdf1cc00700 1 -- 172.21.15.115:0/3390518639 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdf0c014070 con 0x7fdf20101350 2024-03-22T10:44:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.619+0000 7fdf1fe00700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdf0406e0d0 0x7fdf04070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.621+0000 7fdf1fe00700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdf0406e0d0 0x7fdf04070580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000b6f0 tx=0x7fdf10019040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.630+0000 7fdf1cc00700 1 -- 172.21.15.115:0/3390518639 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf0c057d70 con 0x7fdf20101350 2024-03-22T10:44:19.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.941+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf20105ca0 con 0x7fdf0406e0d0 2024-03-22T10:44:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.944+0000 7fdf1cc00700 1 -- 172.21.15.115:0/3390518639 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fdf20105ca0 con 0x7fdf0406e0d0 2024-03-22T10:44:19.935 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:44:19.935 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:44:19.935 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:44:19.935 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:44:19.935 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:44:19.936 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr" 2024-03-22T10:44:19.936 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:44:19.936 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "3/23 daemons upgraded", 2024-03-22T10:44:19.936 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "", 2024-03-22T10:44:19.936 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:44:19.936 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:44:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.946+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdf0406e0d0 msgr2=0x7fdf04070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.946+0000 7fdf26e00700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdf0406e0d0 0x7fdf04070580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000b6f0 tx=0x7fdf10019040 comp rx=0 tx=0).stop 2024-03-22T10:44:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.947+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20101350 msgr2=0x7fdf20195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.947+0000 7fdf26e00700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20101350 0x7fdf20195ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00e940 tx=0x7fdf0c00ed00 comp rx=0 tx=0).stop 2024-03-22T10:44:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.947+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 shutdown_connections 2024-03-22T10:44:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.947+0000 7fdf26e00700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdf0406e0d0 0x7fdf04070580 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.947+0000 7fdf26e00700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20100150 0x7fdf201959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.947+0000 7fdf26e00700 1 --2- 172.21.15.115:0/3390518639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf20101350 0x7fdf20195ee0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.947+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 >> 172.21.15.115:0/3390518639 conn(0x7fdf200fb6e0 msgr2=0x7fdf20104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.947+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 shutdown_connections 2024-03-22T10:44:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:19.947+0000 7fdf26e00700 1 -- 172.21.15.115:0/3390518639 wait complete. 2024-03-22T10:44:20.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.110+0000 7f5966400700 1 -- 172.21.15.115:0/1718548406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960100170 msgr2=0x7f5960100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:20.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.110+0000 7f5966400700 1 --2- 172.21.15.115:0/1718548406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960100170 0x7f5960100580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5950009910 tx=0x7f5950009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:20.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.110+0000 7f5966400700 1 -- 172.21.15.115:0/1718548406 shutdown_connections 2024-03-22T10:44:20.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.110+0000 7f5966400700 1 --2- 172.21.15.115:0/1718548406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5960101370 0x7f59601017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:20.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.110+0000 7f5966400700 1 --2- 172.21.15.115:0/1718548406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960100170 0x7f5960100580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:20.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.110+0000 7f5966400700 1 -- 172.21.15.115:0/1718548406 >> 172.21.15.115:0/1718548406 conn(0x7f59600fb6e0 msgr2=0x7f59600fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:20.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.110+0000 7f5966400700 1 -- 172.21.15.115:0/1718548406 shutdown_connections 2024-03-22T10:44:20.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.110+0000 7f5966400700 1 -- 172.21.15.115:0/1718548406 wait complete. 2024-03-22T10:44:20.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.111+0000 7f5966400700 1 Processor -- start 2024-03-22T10:44:20.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.112+0000 7f5966400700 1 -- start start 2024-03-22T10:44:20.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.112+0000 7f5966400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5960100170 0x7f5960197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.112+0000 7f5966400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960101370 0x7f5960198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.112+0000 7f5966400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5960198720 con 0x7f5960101370 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.112+0000 7f5966400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5960198860 con 0x7f5960100170 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.112+0000 7f595ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960101370 0x7f5960198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.112+0000 7f595ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960101370 0x7f5960198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35742/0 (socket says 172.21.15.115:35742) 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.112+0000 7f595ea00700 1 -- 172.21.15.115:0/3786587849 learned_addr learned my addr 172.21.15.115:0/3786587849 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.112+0000 7f595f400700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5960100170 0x7f5960197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.113+0000 7f595ea00700 1 -- 172.21.15.115:0/3786587849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5960100170 msgr2=0x7f5960197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.113+0000 7f595ea00700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5960100170 0x7f5960197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:20.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.113+0000 7f595ea00700 1 -- 172.21.15.115:0/3786587849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500095f0 con 0x7f5960101370 2024-03-22T10:44:20.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.113+0000 7f595ea00700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960101370 0x7f5960198100 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f594c00d8d0 tx=0x7f594c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:20.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.113+0000 7f5957e00700 1 -- 172.21.15.115:0/3786587849 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594c009940 con 0x7f5960101370 2024-03-22T10:44:20.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.114+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596019d310 con 0x7f5960101370 2024-03-22T10:44:20.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.114+0000 7f5957e00700 1 -- 172.21.15.115:0/3786587849 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f594c010460 con 0x7f5960101370 2024-03-22T10:44:20.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.114+0000 7f5957e00700 1 -- 172.21.15.115:0/3786587849 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594c00f5f0 con 0x7f5960101370 2024-03-22T10:44:20.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.114+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596019d860 con 0x7f5960101370 2024-03-22T10:44:20.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.115+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5960105450 con 0x7f5960101370 2024-03-22T10:44:20.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.116+0000 7f5957e00700 1 -- 172.21.15.115:0/3786587849 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f594c0105d0 con 0x7f5960101370 2024-03-22T10:44:20.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.118+0000 7f5957e00700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f594406e0d0 0x7f5944070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:20.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.118+0000 7f5957e00700 1 -- 172.21.15.115:0/3786587849 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f594c08df40 con 0x7f5960101370 2024-03-22T10:44:20.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.118+0000 7f595f400700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f594406e0d0 0x7f5944070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:20.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.119+0000 7f595f400700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f594406e0d0 0x7f5944070580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5950006010 tx=0x7f595000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:20.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.124+0000 7f5957e00700 1 -- 172.21.15.115:0/3786587849 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f594c05aa70 con 0x7f5960101370 2024-03-22T10:44:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:19 smithi150 ceph-mon[34518]: pgmap v10: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-03-22T10:44:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:19 smithi150 ceph-mon[34518]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:19 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:44:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:19 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3770139588' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:44:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:19 smithi115 ceph-mon[78672]: pgmap v10: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-03-22T10:44:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:19 smithi115 ceph-mon[78672]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:44:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:19 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3770139588' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:44:20.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.627+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f596002ce10 con 0x7f5960101370 2024-03-22T10:44:20.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.628+0000 7f5957e00700 1 -- 172.21.15.115:0/3786587849 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f594c05a890 con 0x7f5960101370 2024-03-22T10:44:20.619 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:44:20.619 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:44:20.619 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:44:20.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f594406e0d0 msgr2=0x7f5944070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:20.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f594406e0d0 0x7f5944070580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5950006010 tx=0x7f595000b540 comp rx=0 tx=0).stop 2024-03-22T10:44:20.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960101370 msgr2=0x7f5960198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:20.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960101370 0x7f5960198100 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f594c00d8d0 tx=0x7f594c00dc90 comp rx=0 tx=0).stop 2024-03-22T10:44:20.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 shutdown_connections 2024-03-22T10:44:20.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f594406e0d0 0x7f5944070580 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:20.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5960100170 0x7f5960197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:20.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 --2- 172.21.15.115:0/3786587849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5960101370 0x7f5960198100 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:20.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 >> 172.21.15.115:0/3786587849 conn(0x7f59600fb6e0 msgr2=0x7f5960102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:20.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 shutdown_connections 2024-03-22T10:44:20.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:20.631+0000 7f5966400700 1 -- 172.21.15.115:0/3786587849 wait complete. 2024-03-22T10:44:21.885 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:21 smithi115 ceph-mon[78672]: from='client.34144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:21.885 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:21 smithi115 ceph-mon[78672]: pgmap v11: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-03-22T10:44:21.885 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:21.885 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:21.885 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:21 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3786587849' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:44:21.885 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:21.885 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:21.886 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:21.886 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:44:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:21 smithi150 ceph-mon[34518]: from='client.34144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:21 smithi150 ceph-mon[34518]: pgmap v11: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-03-22T10:44:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:21 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:21 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:21 smithi150 ceph-mon[34518]: from='client.? 172.21.15.115:0/3786587849' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:44:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:21 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:21 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:21 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:21 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:44:22.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:22 smithi150 ceph-mon[34518]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-22T10:44:22.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:22 smithi150 ceph-mon[34518]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-22T10:44:22.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:22 smithi150 ceph-mon[34518]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:44:22.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:22 smithi150 ceph-mon[34518]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:44:23.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:22 smithi115 ceph-mon[78672]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-22T10:44:23.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:22 smithi115 ceph-mon[78672]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-22T10:44:23.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:22 smithi115 ceph-mon[78672]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:44:23.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:22 smithi115 ceph-mon[78672]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T10:44:23.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:23 smithi115 ceph-mon[78672]: pgmap v12: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61 KiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-03-22T10:44:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:23 smithi115 ceph-mon[78672]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:44:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:23 smithi115 ceph-mon[78672]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:44:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:23.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:23 smithi150 ceph-mon[34518]: pgmap v12: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61 KiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-03-22T10:44:23.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:23 smithi150 ceph-mon[34518]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:44:23.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:23 smithi150 ceph-mon[34518]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-22T10:44:23.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:23 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:23.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:23 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:24.986 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:24 smithi150 ceph-mon[34518]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:44:24.986 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:24 smithi150 ceph-mon[34518]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:44:24.986 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:24 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:24.986 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:24 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:24.987 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:24 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:24.987 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:24 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:44:24.987 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:24 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:44:24.987 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:24 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T10:44:25.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:24 smithi115 ceph-mon[78672]: Updating smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:44:25.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:24 smithi115 ceph-mon[78672]: Updating smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T10:44:25.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:24 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:25.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:24 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:25.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:24 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:25.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:24 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:44:25.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:24 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:44:25.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:24 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T10:44:25.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:25 smithi150 ceph-mon[34518]: pgmap v13: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-03-22T10:44:26.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:25 smithi115 ceph-mon[78672]: pgmap v13: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-03-22T10:44:28.031 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:27 smithi150 ceph-mon[34518]: pgmap v14: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-03-22T10:44:28.031 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:27 smithi150 ceph-mon[34518]: Upgrade: Updating mon.smithi150 2024-03-22T10:44:28.031 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:27 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:28.031 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:27 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:44:28.031 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:27 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:44:28.031 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:27 smithi150 ceph-mon[34518]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:28.031 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:27 smithi150 ceph-mon[34518]: Deploying daemon mon.smithi150 on smithi150 2024-03-22T10:44:28.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:27 smithi115 ceph-mon[78672]: pgmap v14: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-03-22T10:44:28.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:27 smithi115 ceph-mon[78672]: Upgrade: Updating mon.smithi150 2024-03-22T10:44:28.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:28.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:44:28.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:44:28.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:28.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:27 smithi115 ceph-mon[78672]: Deploying daemon mon.smithi150 on smithi150 2024-03-22T10:44:29.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:29 smithi150 systemd[1]: Stopping Ceph mon.smithi150 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:44:29.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:29 smithi150 ceph-mon[34518]: pgmap v15: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-03-22T10:44:30.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:29 smithi115 ceph-mon[78672]: pgmap v15: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-03-22T10:44:30.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:30 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150[34510]: 2024-03-22T10:44:30.086+0000 7f0da2800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi150 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T10:44:30.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:30 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150[34510]: 2024-03-22T10:44:30.086+0000 7f0da2800700 -1 mon.smithi150@1(peon) e2 *** Got Signal Terminated *** 2024-03-22T10:44:30.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:30 smithi150 podman[65364]: 2024-03-22 10:44:30.116980037 +0000 UTC m=+0.183526054 container died 7bc06c4be590c4e93bf67170ce6e56e264e9f4c9c025975baac9d77ce96d07d4 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, RELEASE=HEAD, ceph=True, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux ) 2024-03-22T10:44:33.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:33 smithi150 podman[65364]: 2024-03-22 10:44:33.604559891 +0000 UTC m=+3.671105908 container cleanup 7bc06c4be590c4e93bf67170ce6e56e264e9f4c9c025975baac9d77ce96d07d4 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-22T10:44:33.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:33 smithi150 bash[65364]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150 2024-03-22T10:44:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:33 smithi150 podman[65378]: 2024-03-22 10:44:33.922877127 +0000 UTC m=+3.795470138 container remove 7bc06c4be590c4e93bf67170ce6e56e264e9f4c9c025975baac9d77ce96d07d4 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1) 2024-03-22T10:44:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:34 smithi150 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi150.service: Deactivated successfully. 2024-03-22T10:44:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:34 smithi150 systemd[1]: Stopped Ceph mon.smithi150 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:44:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:34 smithi150 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi150.service: Consumed 15.634s CPU time. 2024-03-22T10:44:34.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:34 smithi150 systemd[1]: Starting Ceph mon.smithi150 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:44:35.330 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:34 smithi150 podman[65496]: 2024-03-22 10:44:34.88572337 +0000 UTC m=+0.043174465 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:44:35.330 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 podman[65496]: 2024-03-22 10:44:35.024416509 +0000 UTC m=+0.181867610 container create f35dc3320e8598c8a38f7bf12a64810fc7412d12a4483d231c6270c98da2cba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-22T10:44:35.330 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 podman[65496]: 2024-03-22 10:44:35.338389283 +0000 UTC m=+0.495840442 container init f35dc3320e8598c8a38f7bf12a64810fc7412d12a4483d231c6270c98da2cba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:44:35.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 podman[65496]: 2024-03-22 10:44:35.347672485 +0000 UTC m=+0.505123613 container start f35dc3320e8598c8a38f7bf12a64810fc7412d12a4483d231c6270c98da2cba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-22T10:44:35.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T10:44:35.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-22T10:44:35.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: pidfile_write: ignore empty --pid-file 2024-03-22T10:44:35.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: load: jerasure load: lrc 2024-03-22T10:44:35.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: RocksDB version: 7.9.2 2024-03-22T10:44:35.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Git sha 0 2024-03-22T10:44:35.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-22T10:44:35.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: DB SUMMARY 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: DB Session ID: 2FGT6GN1X305SUMAFCEV 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: CURRENT file: CURRENT 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1397 Bytes 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi150/store.db dir, Total Num: 1, files: 000027.sst 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi150/store.db: 000025.log size: 7377265 ; 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.error_if_exists: 0 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.create_if_missing: 0 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.paranoid_checks: 1 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T10:44:35.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.env: 0x561dec9bfb20 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.info_log: 0x561deeacd840 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.statistics: (nil) 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.use_fsync: 0 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_log_file_size: 0 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T10:44:35.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.allow_fallocate: 1 2024-03-22T10:44:35.653 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T10:44:35.653 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T10:44:35.653 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.use_direct_reads: 0 2024-03-22T10:44:35.653 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T10:44:35.653 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T10:44:35.653 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.db_log_dir: 2024-03-22T10:44:35.653 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.wal_dir: 2024-03-22T10:44:35.653 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T10:44:35.653 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.write_buffer_manager: 0x561deeaa01e0 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T10:44:35.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.unordered_write: 0 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.row_cache: None 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.wal_filter: None 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.two_write_queues: 0 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.wal_compression: 0 2024-03-22T10:44:35.655 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.atomic_flush: 0 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.log_readahead_size: 0 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_background_jobs: 2 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_background_compactions: -1 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_subcompactions: 1 2024-03-22T10:44:35.656 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_open_files: -1 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_background_flushes: -1 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Compression algorithms supported: 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T10:44:35.657 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: kZSTD supported: 0 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: kXpressCompression supported: 0 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: kZlibCompression supported: 1 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: kSnappyCompression supported: 1 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: kLZ4Compression supported: 1 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: kBZip2Compression supported: 0 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi150/store.db/MANIFEST-000010 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.merge_operator: 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_filter: None 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T10:44:35.658 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561deeacd940) 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: cache_index_and_filter_blocks: 1 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: pin_top_level_index_and_filter: 1 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: index_type: 0 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: data_block_index_type: 0 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: index_shortening: 1 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: checksum: 4 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: no_block_cache: 0 2024-03-22T10:44:35.659 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: block_cache: 0x561deead3090 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: block_cache_name: BinnedLRUCache 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: block_cache_options: 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: capacity : 536870912 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: num_shard_bits : 4 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: strict_capacity_limit : 0 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: high_pri_pool_ratio: 0.000 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: block_cache_compressed: (nil) 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: persistent_cache: (nil) 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: block_size: 4096 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: block_size_deviation: 10 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: block_restart_interval: 16 2024-03-22T10:44:35.660 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: index_block_restart_interval: 1 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: metadata_block_size: 4096 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: partition_filters: 0 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: use_delta_encoding: 1 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: filter_policy: bloomfilter 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: whole_key_filtering: 1 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: verify_compression: 0 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: read_amp_bytes_per_bit: 0 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: format_version: 5 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: enable_index_compression: 1 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: block_align: 0 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: max_auto_readahead_size: 262144 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: prepopulate_block_cache: 0 2024-03-22T10:44:35.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: initial_auto_readahead_size: 8192 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression: NoCompression 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.num_levels: 7 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T10:44:35.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T10:44:35.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T10:44:35.664 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T10:44:35.665 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.table_properties_collectors: 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.inplace_update_support: 0 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.bloom_locality: 0 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.max_successive_merges: 0 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T10:44:35.666 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.ttl: 2592000 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.enable_blob_files: false 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.min_blob_size: 0 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi150/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12413, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-22T10:44:35.667 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7bcb8e28-9d4e-49f8-84dc-d65f06543014 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711104275404398, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711104275435851, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4434149, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12418, "largest_seqno": 15173, "table_properties": {"data_size": 4423574, "index_size": 6546, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3013, "raw_key_size": 25271, "raw_average_key_size": 21, "raw_value_size": 4400479, "raw_average_value_size": 3722, "num_data_blocks": 297, "num_entries": 1182, "num_filter_entries": 1182, "num_deletions": 253, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711104275, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7bcb8e28-9d4e-49f8-84dc-d65f06543014", "db_session_id": "2FGT6GN1X305SUMAFCEV", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711104275436138, "job": 1, "event": "recovery_finished"} 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi150/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561deebc4000 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: DB pointer 0x561deebb0000 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: ** DB Stats ** 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T10:44:35.668 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: ** Compaction Stats [default] ** 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: L0 1/0 4.23 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 422.0 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: L6 1/0 4.92 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Sum 2/0 9.15 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 422.0 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 422.0 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: 2024-03-22T10:44:35.669 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: ** Compaction Stats [default] ** 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 422.0 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Cumulative compaction: 0.00 GB write, 109.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T10:44:35.670 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Interval compaction: 0.00 GB write, 109.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Block cache BinnedLRUCache@0x561deead3090#2 capacity: 512.00 MB usage: 9.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,3.03 KB,0.000578165%) IndexBlock(1,6.58 KB,0.00125468%) Misc(1,0.00 KB,0%) 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: starting mon.smithi150 rank 1 at public addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] at bind addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi150 fsid fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: mon.smithi150@-1(???) e2 preinit fsid fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: mon.smithi150@-1(???).mds e14 new map 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: mon.smithi150@-1(???).mds e14 print_map 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: e14 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: legacy client fscid: 1 2024-03-22T10:44:35.671 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Filesystem 'cephfs' (1) 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: fs_name cephfs 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: epoch 14 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: modified 2024-03-22T10:43:22.786262+0000 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: tableserver 0 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: root 0 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: session_timeout 60 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: session_autoclose 300 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: max_file_size 1099511627776 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: max_xattr_size 65536 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: required_client_features {} 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: last_failure 0 2024-03-22T10:44:35.672 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: last_failure_osd_epoch 0 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: max_mds 2 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: in 0,1 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: up {0=14454,1=24241} 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: failed 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: damaged 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: stopped 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: data_pools [3] 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: metadata_pool 2 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: inline_data enabled 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: balancer 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: bal_rank_mask -1 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: standby_count_wanted 1 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: qdb_cluster leader: 0 members: 2024-03-22T10:44:35.673 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: [mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: [mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: Standby daemons: 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: [mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout: [mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: mon.smithi150@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: mon.smithi150@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: mon.smithi150@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: mon.smithi150@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 bash[65496]: f35dc3320e8598c8a38f7bf12a64810fc7412d12a4483d231c6270c98da2cba1 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 ceph-mon[65513]: mon.smithi150@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-03-22T10:44:35.674 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:35 smithi150 systemd[1]: Started Ceph mon.smithi150 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:44:36.929 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-22T10:44:36.930 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-22T10:44:38.077 DEBUG:teuthology.parallel:result is None 2024-03-22T10:44:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:44:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: mon.smithi150 calling monitor election 2024-03-22T10:44:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: mon.smithi115 calling monitor election 2024-03-22T10:44:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:44:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: mon.smithi115 is new leader, mons smithi115,smithi150 in quorum (ranks 0,1) 2024-03-22T10:44:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: monmap e3: 2 mons at {smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],smithi150=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T10:44:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:44:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T10:44:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: mgrmap e36: smithi115.iqzwsz(active, since 32s), standbys: smithi150.fmoumv 2024-03-22T10:44:38.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:44:38.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:44:38.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:44:38.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:38.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:38.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:37 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:44:38.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2024-03-22T10:44:38.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: mon.smithi150 calling monitor election 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: mon.smithi115 calling monitor election 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mon metadata", "id": "smithi150"}]: dispatch 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: mon.smithi115 is new leader, mons smithi115,smithi150 in quorum (ranks 0,1) 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: monmap e3: 2 mons at {smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],smithi150=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: mgrmap e36: smithi115.iqzwsz(active, since 32s), standbys: smithi150.fmoumv 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:37 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:44:39.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:38 smithi115 ceph-mon[78672]: pgmap v20: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-03-22T10:44:39.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:38 smithi150 ceph-mon[65513]: pgmap v20: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-03-22T10:44:41.729 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:41 smithi150 ceph-mon[65513]: pgmap v21: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-03-22T10:44:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:41 smithi115 ceph-mon[78672]: pgmap v21: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-03-22T10:44:42.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:42 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:42.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:42 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:43.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:42 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:43.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:42 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:43.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:43 smithi150 ceph-mon[65513]: pgmap v22: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.5 MiB/s wr, 238 op/s 2024-03-22T10:44:44.043 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:43 smithi115 ceph-mon[78672]: pgmap v22: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.5 MiB/s wr, 238 op/s 2024-03-22T10:44:45.323 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:45.323 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:45.323 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:45 smithi150 ceph-mon[65513]: pgmap v23: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 941 KiB/s wr, 151 op/s 2024-03-22T10:44:45.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:45.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:45.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:45 smithi115 ceph-mon[78672]: pgmap v23: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 941 KiB/s wr, 151 op/s 2024-03-22T10:44:47.263 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:47 smithi150 ceph-mon[65513]: pgmap v24: 65 pgs: 65 active+clean; 323 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-03-22T10:44:47.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:47 smithi115 ceph-mon[78672]: pgmap v24: 65 pgs: 65 active+clean; 323 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-03-22T10:44:49.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:49 smithi115 ceph-mon[78672]: pgmap v25: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 700 KiB/s wr, 144 op/s 2024-03-22T10:44:49.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:49.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:49.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:49.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:44:49.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:49.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:49 smithi150 ceph-mon[65513]: pgmap v25: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 700 KiB/s wr, 144 op/s 2024-03-22T10:44:49.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:49.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:49.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:49.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:44:49.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:50.408 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:50 smithi115 ceph-mon[78672]: Reconfiguring mon.smithi115 (monmap changed)... 2024-03-22T10:44:50.408 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:50 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:44:50.408 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:50 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:44:50.408 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:50 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:50.408 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:50 smithi115 ceph-mon[78672]: Reconfiguring daemon mon.smithi115 on smithi115 2024-03-22T10:44:50.408 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:50 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:44:50.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:50 smithi150 ceph-mon[65513]: Reconfiguring mon.smithi115 (monmap changed)... 2024-03-22T10:44:50.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:50 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:44:50.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:50 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:44:50.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:50 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:50.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:50 smithi150 ceph-mon[65513]: Reconfiguring daemon mon.smithi115 on smithi115 2024-03-22T10:44:50.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:50 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:44:50.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.802+0000 7f6ab2400700 1 -- 172.21.15.115:0/586664212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1035a0 msgr2=0x7f6aac1039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:50.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.802+0000 7f6ab2400700 1 --2- 172.21.15.115:0/586664212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1035a0 0x7f6aac1039f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009910 tx=0x7f6a98009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.802+0000 7f6ab2400700 1 -- 172.21.15.115:0/586664212 shutdown_connections 2024-03-22T10:44:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.802+0000 7f6ab2400700 1 --2- 172.21.15.115:0/586664212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1035a0 0x7f6aac1039f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.802+0000 7f6ab2400700 1 --2- 172.21.15.115:0/586664212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aac1023a0 0x7f6aac1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.802+0000 7f6ab2400700 1 -- 172.21.15.115:0/586664212 >> 172.21.15.115:0/586664212 conn(0x7f6aac0fd990 msgr2=0x7f6aac0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.802+0000 7f6ab2400700 1 -- 172.21.15.115:0/586664212 shutdown_connections 2024-03-22T10:44:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.802+0000 7f6ab2400700 1 -- 172.21.15.115:0/586664212 wait complete. 2024-03-22T10:44:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.803+0000 7f6ab2400700 1 Processor -- start 2024-03-22T10:44:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.803+0000 7f6ab2400700 1 -- start start 2024-03-22T10:44:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.803+0000 7f6ab2400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1023a0 0x7f6aac197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.803+0000 7f6ab2400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aac1035a0 0x7f6aac1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.803+0000 7f6ab2400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac198670 con 0x7f6aac1023a0 2024-03-22T10:44:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.803+0000 7f6ab2400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1987b0 con 0x7f6aac1035a0 2024-03-22T10:44:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.804+0000 7f6aab400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1023a0 0x7f6aac197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.804+0000 7f6aab400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1023a0 0x7f6aac197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43716/0 (socket says 172.21.15.115:43716) 2024-03-22T10:44:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.804+0000 7f6aab400700 1 -- 172.21.15.115:0/404059371 learned_addr learned my addr 172.21.15.115:0/404059371 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.804+0000 7f6aab400700 1 -- 172.21.15.115:0/404059371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aac1035a0 msgr2=0x7f6aac1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.804+0000 7f6aab400700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aac1035a0 0x7f6aac1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.804+0000 7f6aab400700 1 -- 172.21.15.115:0/404059371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a980095f0 con 0x7f6aac1023a0 2024-03-22T10:44:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.804+0000 7f6aab400700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1023a0 0x7f6aac197ba0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c910 tx=0x7f6a9c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.805+0000 7f6aa3e00700 1 -- 172.21.15.115:0/404059371 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9c0041f0 con 0x7f6aac1023a0 2024-03-22T10:44:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.805+0000 7f6aa3e00700 1 -- 172.21.15.115:0/404059371 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a9c007db0 con 0x7f6aac1023a0 2024-03-22T10:44:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.805+0000 7f6aa3e00700 1 -- 172.21.15.115:0/404059371 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9c003d60 con 0x7f6aac1023a0 2024-03-22T10:44:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.805+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac19d270 con 0x7f6aac1023a0 2024-03-22T10:44:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.805+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac19d790 con 0x7f6aac1023a0 2024-03-22T10:44:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.806+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac10aef0 con 0x7f6aac1023a0 2024-03-22T10:44:50.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.807+0000 7f6aa3e00700 1 -- 172.21.15.115:0/404059371 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f6a9c007900 con 0x7f6aac1023a0 2024-03-22T10:44:50.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.808+0000 7f6aa3e00700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6a9006e0d0 0x7f6a90070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:50.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.809+0000 7f6aa3e00700 1 -- 172.21.15.115:0/404059371 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6a9c08dcc0 con 0x7f6aac1023a0 2024-03-22T10:44:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.809+0000 7f6aaaa00700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6a9006e0d0 0x7f6a90070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.809+0000 7f6aaaa00700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6a9006e0d0 0x7f6a90070580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b5c0 tx=0x7f6a98005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:50.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:50.814+0000 7f6aa3e00700 1 -- 172.21.15.115:0/404059371 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a9c093020 con 0x7f6aac1023a0 2024-03-22T10:44:51.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.099+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6aac107ef0 con 0x7f6a9006e0d0 2024-03-22T10:44:51.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.103+0000 7f6aa3e00700 1 -- 172.21.15.115:0/404059371 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f6aac107ef0 con 0x7f6a9006e0d0 2024-03-22T10:44:51.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6a9006e0d0 msgr2=0x7f6a90070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6a9006e0d0 0x7f6a90070580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b5c0 tx=0x7f6a98005fd0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1023a0 msgr2=0x7f6aac197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1023a0 0x7f6aac197ba0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c910 tx=0x7f6a9c00ccd0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 shutdown_connections 2024-03-22T10:44:51.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6a9006e0d0 0x7f6a90070580 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac1023a0 0x7f6aac197ba0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 --2- 172.21.15.115:0/404059371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aac1035a0 0x7f6aac1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 >> 172.21.15.115:0/404059371 conn(0x7f6aac0fd990 msgr2=0x7f6aac1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:51.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 shutdown_connections 2024-03-22T10:44:51.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.106+0000 7f6ab2400700 1 -- 172.21.15.115:0/404059371 wait complete. 2024-03-22T10:44:51.114 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:44:51.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.263+0000 7f2f0f200700 1 -- 172.21.15.115:0/853335863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101021c0 msgr2=0x7f2f101025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.263+0000 7f2f0f200700 1 --2- 172.21.15.115:0/853335863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101021c0 0x7f2f101025d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009910 tx=0x7f2f00009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:51.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.263+0000 7f2f0f200700 1 -- 172.21.15.115:0/853335863 shutdown_connections 2024-03-22T10:44:51.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.263+0000 7f2f0f200700 1 --2- 172.21.15.115:0/853335863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f101033c0 0x7f2f10103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.263+0000 7f2f0f200700 1 --2- 172.21.15.115:0/853335863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101021c0 0x7f2f101025d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.263+0000 7f2f0f200700 1 -- 172.21.15.115:0/853335863 >> 172.21.15.115:0/853335863 conn(0x7f2f100fd8d0 msgr2=0x7f2f100ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:51.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.263+0000 7f2f0f200700 1 -- 172.21.15.115:0/853335863 shutdown_connections 2024-03-22T10:44:51.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.263+0000 7f2f0f200700 1 -- 172.21.15.115:0/853335863 wait complete. 2024-03-22T10:44:51.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.264+0000 7f2f0f200700 1 Processor -- start 2024-03-22T10:44:51.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.264+0000 7f2f0f200700 1 -- start start 2024-03-22T10:44:51.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.265+0000 7f2f0f200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f101021c0 0x7f2f101979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:51.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.265+0000 7f2f0f200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101033c0 0x7f2f10197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:51.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.265+0000 7f2f0f200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10198520 con 0x7f2f101033c0 2024-03-22T10:44:51.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.265+0000 7f2f0f200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10198660 con 0x7f2f101021c0 2024-03-22T10:44:51.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.265+0000 7f2f0d400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101033c0 0x7f2f10197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:51.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.265+0000 7f2f0d400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101033c0 0x7f2f10197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43738/0 (socket says 172.21.15.115:43738) 2024-03-22T10:44:51.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.265+0000 7f2f0d400700 1 -- 172.21.15.115:0/2402252278 learned_addr learned my addr 172.21.15.115:0/2402252278 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.266+0000 7f2f0de00700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f101021c0 0x7f2f101979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.266+0000 7f2f0d400700 1 -- 172.21.15.115:0/2402252278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f101021c0 msgr2=0x7f2f101979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.266+0000 7f2f0d400700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f101021c0 0x7f2f101979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.266+0000 7f2f0d400700 1 -- 172.21.15.115:0/2402252278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f000095f0 con 0x7f2f101033c0 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.266+0000 7f2f0d400700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101033c0 0x7f2f10197f00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00d8d0 tx=0x7f2efc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.266+0000 7f2f06a00700 1 -- 172.21.15.115:0/2402252278 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2efc009940 con 0x7f2f101033c0 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.266+0000 7f2f06a00700 1 -- 172.21.15.115:0/2402252278 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2efc010460 con 0x7f2f101033c0 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.266+0000 7f2f06a00700 1 -- 172.21.15.115:0/2402252278 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2efc00f5f0 con 0x7f2f101033c0 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.266+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1019d110 con 0x7f2f101033c0 2024-03-22T10:44:51.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.267+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1019d660 con 0x7f2f101033c0 2024-03-22T10:44:51.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.268+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1010ad10 con 0x7f2f101033c0 2024-03-22T10:44:51.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.269+0000 7f2f06a00700 1 -- 172.21.15.115:0/2402252278 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f2efc0105d0 con 0x7f2f101033c0 2024-03-22T10:44:51.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.270+0000 7f2f06a00700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2ef406e0d0 0x7f2ef4070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:51.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.270+0000 7f2f06a00700 1 -- 172.21.15.115:0/2402252278 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2efc08df00 con 0x7f2f101033c0 2024-03-22T10:44:51.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.270+0000 7f2f0de00700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2ef406e0d0 0x7f2ef4070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:51.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.271+0000 7f2f0de00700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2ef406e0d0 0x7f2ef4070580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2f00006010 tx=0x7f2f0000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:51.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.276+0000 7f2f06a00700 1 -- 172.21.15.115:0/2402252278 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2efc05acc0 con 0x7f2f101033c0 2024-03-22T10:44:51.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:51 smithi115 ceph-mon[78672]: pgmap v26: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 698 KiB/s wr, 150 op/s 2024-03-22T10:44:51.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.525+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f10107d10 con 0x7f2ef406e0d0 2024-03-22T10:44:51.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.527+0000 7f2f06a00700 1 -- 172.21.15.115:0/2402252278 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f2f10107d10 con 0x7f2ef406e0d0 2024-03-22T10:44:51.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.530+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2ef406e0d0 msgr2=0x7f2ef4070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.530+0000 7f2f0f200700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2ef406e0d0 0x7f2ef4070580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2f00006010 tx=0x7f2f0000b540 comp rx=0 tx=0).stop 2024-03-22T10:44:51.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.530+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101033c0 msgr2=0x7f2f10197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.530+0000 7f2f0f200700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101033c0 0x7f2f10197f00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00d8d0 tx=0x7f2efc00dc90 comp rx=0 tx=0).stop 2024-03-22T10:44:51.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.531+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 shutdown_connections 2024-03-22T10:44:51.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.531+0000 7f2f0f200700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2ef406e0d0 0x7f2ef4070580 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.531+0000 7f2f0f200700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f101021c0 0x7f2f101979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.531+0000 7f2f0f200700 1 --2- 172.21.15.115:0/2402252278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f101033c0 0x7f2f10197f00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.531+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 >> 172.21.15.115:0/2402252278 conn(0x7f2f100fd8d0 msgr2=0x7f2f101065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:51.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.531+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 shutdown_connections 2024-03-22T10:44:51.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.531+0000 7f2f0f200700 1 -- 172.21.15.115:0/2402252278 wait complete. 2024-03-22T10:44:51.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:51 smithi150 ceph-mon[65513]: pgmap v26: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 698 KiB/s wr, 150 op/s 2024-03-22T10:44:51.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.683+0000 7ff0d6e00700 1 -- 172.21.15.115:0/198596766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0100150 msgr2=0x7ff0d0100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.683+0000 7ff0d6e00700 1 --2- 172.21.15.115:0/198596766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0100150 0x7ff0d0100560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009910 tx=0x7ff0c0009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:51.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.683+0000 7ff0d6e00700 1 -- 172.21.15.115:0/198596766 shutdown_connections 2024-03-22T10:44:51.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.683+0000 7ff0d6e00700 1 --2- 172.21.15.115:0/198596766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d0101350 0x7ff0d01017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.683+0000 7ff0d6e00700 1 --2- 172.21.15.115:0/198596766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0100150 0x7ff0d0100560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.683+0000 7ff0d6e00700 1 -- 172.21.15.115:0/198596766 >> 172.21.15.115:0/198596766 conn(0x7ff0d00fb6e0 msgr2=0x7ff0d00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:51.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.684+0000 7ff0d6e00700 1 -- 172.21.15.115:0/198596766 shutdown_connections 2024-03-22T10:44:51.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.684+0000 7ff0d6e00700 1 -- 172.21.15.115:0/198596766 wait complete. 2024-03-22T10:44:51.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.685+0000 7ff0d6e00700 1 Processor -- start 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.685+0000 7ff0d6e00700 1 -- start start 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.685+0000 7ff0d6e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d0100150 0x7ff0d0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.685+0000 7ff0d6e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0101350 0x7ff0d01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.685+0000 7ff0d6e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d01986e0 con 0x7ff0d0101350 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.685+0000 7ff0d6e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0198820 con 0x7ff0d0100150 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.685+0000 7ff0cf400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0101350 0x7ff0d01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.685+0000 7ff0cf400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0101350 0x7ff0d01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43750/0 (socket says 172.21.15.115:43750) 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.685+0000 7ff0cf400700 1 -- 172.21.15.115:0/1250469016 learned_addr learned my addr 172.21.15.115:0/1250469016 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.686+0000 7ff0cfe00700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d0100150 0x7ff0d0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.686+0000 7ff0cf400700 1 -- 172.21.15.115:0/1250469016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d0100150 msgr2=0x7ff0d0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.686+0000 7ff0cf400700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d0100150 0x7ff0d0197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.686+0000 7ff0cf400700 1 -- 172.21.15.115:0/1250469016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00095f0 con 0x7ff0d0101350 2024-03-22T10:44:51.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.686+0000 7ff0cf400700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0101350 0x7ff0d01980c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00c8a0 tx=0x7ff0bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:51.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.687+0000 7ff0ccc00700 1 -- 172.21.15.115:0/1250469016 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0bc00cea0 con 0x7ff0d0101350 2024-03-22T10:44:51.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.687+0000 7ff0ccc00700 1 -- 172.21.15.115:0/1250469016 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff0bc004830 con 0x7ff0d0101350 2024-03-22T10:44:51.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.687+0000 7ff0ccc00700 1 -- 172.21.15.115:0/1250469016 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0bc0055b0 con 0x7ff0d0101350 2024-03-22T10:44:51.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.687+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d019d2d0 con 0x7ff0d0101350 2024-03-22T10:44:51.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.687+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d019d820 con 0x7ff0d0101350 2024-03-22T10:44:51.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.688+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d0105430 con 0x7ff0d0101350 2024-03-22T10:44:51.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.689+0000 7ff0ccc00700 1 -- 172.21.15.115:0/1250469016 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7ff0bc007ad0 con 0x7ff0d0101350 2024-03-22T10:44:51.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.690+0000 7ff0ccc00700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff0b406e0d0 0x7ff0b4070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:51.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.690+0000 7ff0ccc00700 1 -- 172.21.15.115:0/1250469016 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff0bc08dc60 con 0x7ff0d0101350 2024-03-22T10:44:51.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.690+0000 7ff0cfe00700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff0b406e0d0 0x7ff0b4070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:51.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.691+0000 7ff0cfe00700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff0b406e0d0 0x7ff0b4070580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0006010 tx=0x7ff0c000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.699+0000 7ff0ccc00700 1 -- 172.21.15.115:0/1250469016 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff0bc05aa20 con 0x7ff0d0101350 2024-03-22T10:44:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.936+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0d0061190 con 0x7ff0b406e0d0 2024-03-22T10:44:51.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.954+0000 7ff0ccc00700 1 -- 172.21.15.115:0/1250469016 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff0d0061190 con 0x7ff0b406e0d0 2024-03-22T10:44:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:44:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (11m) 33s ago 13m 13.8M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:44:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (13m) 33s ago 13m 7948k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:44:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (12m) 10s ago 12m 7847k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:44:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (13m) 33s ago 13m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:44:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (12m) 10s ago 12m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:44:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (11m) 33s ago 12m 74.3M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:44:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (6m) 33s ago 6m 156M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:44:51.947 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (6m) 33s ago 6m 15.2M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (6m) 10s ago 6m 15.0M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (6m) 10s ago 6m 141M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (2m) 33s ago 14m 479M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (86s) 10s ago 11m 448M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (63s) 33s ago 15m 31.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (17s) 10s ago 11m 22.9M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (13m) 33s ago 13m 7726k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (12m) 10s ago 11m 8124k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (10m) 33s ago 10m 186M 1217M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (10m) 33s ago 10m 192M 1217M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (9m) 33s ago 9m 155M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:44:51.948 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (9m) 10s ago 9m 234M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:44:51.949 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (8m) 10s ago 8m 218M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:44:51.949 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (7m) 10s ago 7m 193M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:44:51.949 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (95s) 33s ago 12m 57.7M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:44:51.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff0b406e0d0 msgr2=0x7ff0b4070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff0b406e0d0 0x7ff0b4070580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0006010 tx=0x7ff0c000b540 comp rx=0 tx=0).stop 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0101350 msgr2=0x7ff0d01980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0101350 0x7ff0d01980c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00c8a0 tx=0x7ff0bc00cc60 comp rx=0 tx=0).stop 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 shutdown_connections 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff0b406e0d0 0x7ff0b4070580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d0100150 0x7ff0d0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 --2- 172.21.15.115:0/1250469016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d0101350 0x7ff0d01980c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 >> 172.21.15.115:0/1250469016 conn(0x7ff0d00fb6e0 msgr2=0x7ff0d0102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 shutdown_connections 2024-03-22T10:44:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:51.957+0000 7ff0d6e00700 1 -- 172.21.15.115:0/1250469016 wait complete. 2024-03-22T10:44:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.107+0000 7f668f600700 1 -- 172.21.15.115:0/3159493552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 msgr2=0x7f6688105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.107+0000 7f668f600700 1 --2- 172.21.15.115:0/3159493552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 0x7f6688105900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6674009910 tx=0x7f6674009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.108+0000 7f668f600700 1 -- 172.21.15.115:0/3159493552 shutdown_connections 2024-03-22T10:44:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.108+0000 7f668f600700 1 --2- 172.21.15.115:0/3159493552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 0x7f6688105900 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.108+0000 7f668f600700 1 --2- 172.21.15.115:0/3159493552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6688100c00 0x7f6688102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.108+0000 7f668f600700 1 -- 172.21.15.115:0/3159493552 >> 172.21.15.115:0/3159493552 conn(0x7f66880fa600 msgr2=0x7f66880fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.108+0000 7f668f600700 1 -- 172.21.15.115:0/3159493552 shutdown_connections 2024-03-22T10:44:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.108+0000 7f668f600700 1 -- 172.21.15.115:0/3159493552 wait complete. 2024-03-22T10:44:52.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.109+0000 7f668f600700 1 Processor -- start 2024-03-22T10:44:52.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.109+0000 7f668f600700 1 -- start start 2024-03-22T10:44:52.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.109+0000 7f668f600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6688100c00 0x7f6688195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:52.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.109+0000 7f668f600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 0x7f6688195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:52.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.109+0000 7f668f600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66881964d0 con 0x7f6688103520 2024-03-22T10:44:52.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.109+0000 7f668f600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6688196610 con 0x7f6688100c00 2024-03-22T10:44:52.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.110+0000 7f6687e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 0x7f6688195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:52.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.110+0000 7f6687e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 0x7f6688195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49622/0 (socket says 172.21.15.115:49622) 2024-03-22T10:44:52.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.110+0000 7f6687e00700 1 -- 172.21.15.115:0/3001763606 learned_addr learned my addr 172.21.15.115:0/3001763606 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:52.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.110+0000 7f668cc00700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6688100c00 0x7f6688195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:52.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.110+0000 7f6687e00700 1 -- 172.21.15.115:0/3001763606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6688100c00 msgr2=0x7f6688195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:52.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.110+0000 7f6687e00700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6688100c00 0x7f6688195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.110+0000 7f6687e00700 1 -- 172.21.15.115:0/3001763606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66740095f0 con 0x7f6688103520 2024-03-22T10:44:52.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.110+0000 7f668cc00700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6688100c00 0x7f6688195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:44:52.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.111+0000 7f6687e00700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 0x7f6688195eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f667400b5c0 tx=0x7f667400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:52.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.111+0000 7f6685600700 1 -- 172.21.15.115:0/3001763606 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f667401d070 con 0x7f6688103520 2024-03-22T10:44:52.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.111+0000 7f6685600700 1 -- 172.21.15.115:0/3001763606 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f667400f460 con 0x7f6688103520 2024-03-22T10:44:52.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.111+0000 7f6685600700 1 -- 172.21.15.115:0/3001763606 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6674021640 con 0x7f6688103520 2024-03-22T10:44:52.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.111+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668819b060 con 0x7f6688103520 2024-03-22T10:44:52.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.111+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668819b550 con 0x7f6688103520 2024-03-22T10:44:52.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.112+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668818fb90 con 0x7f6688103520 2024-03-22T10:44:52.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.113+0000 7f6685600700 1 -- 172.21.15.115:0/3001763606 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f667400f5d0 con 0x7f6688103520 2024-03-22T10:44:52.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.115+0000 7f6685600700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f667006e0d0 0x7f6670070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:52.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.115+0000 7f6685600700 1 -- 172.21.15.115:0/3001763606 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f667408f2d0 con 0x7f6688103520 2024-03-22T10:44:52.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.115+0000 7f668cc00700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f667006e0d0 0x7f6670070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:52.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.116+0000 7f668cc00700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f667006e0d0 0x7f6670070580 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f667c005950 tx=0x7f667c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:52.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.120+0000 7f6685600700 1 -- 172.21.15.115:0/3001763606 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f667405c130 con 0x7f6688103520 2024-03-22T10:44:52.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.458+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f668802ce10 con 0x7f6688103520 2024-03-22T10:44:52.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.462+0000 7f6685600700 1 -- 172.21.15.115:0/3001763606 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f667405bcc0 con 0x7f6688103520 2024-03-22T10:44:52.454 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:44:52.454 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:44:52.454 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:44:52.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:44:52.454 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:44:52.454 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:44:52.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:44:52.455 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:44:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.464+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f667006e0d0 msgr2=0x7f6670070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.464+0000 7f668f600700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f667006e0d0 0x7f6670070580 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f667c005950 tx=0x7f667c0058e0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.464+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 msgr2=0x7f6688195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.464+0000 7f668f600700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 0x7f6688195eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f667400b5c0 tx=0x7f667400be20 comp rx=0 tx=0).stop 2024-03-22T10:44:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.465+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 shutdown_connections 2024-03-22T10:44:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.465+0000 7f668f600700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f667006e0d0 0x7f6670070580 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.465+0000 7f668f600700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6688100c00 0x7f6688195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.465+0000 7f668f600700 1 --2- 172.21.15.115:0/3001763606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6688103520 0x7f6688195eb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.465+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 >> 172.21.15.115:0/3001763606 conn(0x7f66880fa600 msgr2=0x7f66880fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.465+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 shutdown_connections 2024-03-22T10:44:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.465+0000 7f668f600700 1 -- 172.21.15.115:0/3001763606 wait complete. 2024-03-22T10:44:52.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:52 smithi115 ceph-mon[78672]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:52.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:52.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:52.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:52 smithi115 ceph-mon[78672]: Reconfiguring mgr.smithi115.iqzwsz (monmap changed)... 2024-03-22T10:44:52.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.iqzwsz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:44:52.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:44:52.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:52.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:52 smithi115 ceph-mon[78672]: Reconfiguring daemon mgr.smithi115.iqzwsz on smithi115 2024-03-22T10:44:52.599 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:52 smithi115 ceph-mon[78672]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:52.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.618+0000 7f00eca00700 1 -- 172.21.15.115:0/2641409004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 msgr2=0x7f00e8102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:52.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.618+0000 7f00eca00700 1 --2- 172.21.15.115:0/2641409004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 0x7f00e8102790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009920 tx=0x7f00d8009c30 comp rx=0 tx=0).stop 2024-03-22T10:44:52.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.618+0000 7f00eca00700 1 -- 172.21.15.115:0/2641409004 shutdown_connections 2024-03-22T10:44:52.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.618+0000 7f00eca00700 1 --2- 172.21.15.115:0/2641409004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e8103580 0x7f00e81039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.618+0000 7f00eca00700 1 --2- 172.21.15.115:0/2641409004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 0x7f00e8102790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.618+0000 7f00eca00700 1 -- 172.21.15.115:0/2641409004 >> 172.21.15.115:0/2641409004 conn(0x7f00e80fd930 msgr2=0x7f00e80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:52.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.618+0000 7f00eca00700 1 -- 172.21.15.115:0/2641409004 shutdown_connections 2024-03-22T10:44:52.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.618+0000 7f00eca00700 1 -- 172.21.15.115:0/2641409004 wait complete. 2024-03-22T10:44:52.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.619+0000 7f00eca00700 1 Processor -- start 2024-03-22T10:44:52.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.619+0000 7f00eca00700 1 -- start start 2024-03-22T10:44:52.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.619+0000 7f00eca00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 0x7f00e8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:52.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.619+0000 7f00eca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e8103580 0x7f00e81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:52.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.619+0000 7f00eca00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8198700 con 0x7f00e8102380 2024-03-22T10:44:52.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.619+0000 7f00eca00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8198840 con 0x7f00e8103580 2024-03-22T10:44:52.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.620+0000 7f00e5e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 0x7f00e8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:52.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.620+0000 7f00e5e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 0x7f00e8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49640/0 (socket says 172.21.15.115:49640) 2024-03-22T10:44:52.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.620+0000 7f00e5e00700 1 -- 172.21.15.115:0/3159891704 learned_addr learned my addr 172.21.15.115:0/3159891704 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:52.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.620+0000 7f00e5400700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e8103580 0x7f00e81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:52.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.620+0000 7f00e5e00700 1 -- 172.21.15.115:0/3159891704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e8103580 msgr2=0x7f00e81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:52.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.620+0000 7f00e5e00700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e8103580 0x7f00e81980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.620+0000 7f00e5e00700 1 -- 172.21.15.115:0/3159891704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8009580 con 0x7f00e8102380 2024-03-22T10:44:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.621+0000 7f00e5e00700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 0x7f00e8197ba0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f00d8004df0 tx=0x7f00d8004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.621+0000 7f00dea00700 1 -- 172.21.15.115:0/3159891704 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d801d070 con 0x7f00e8102380 2024-03-22T10:44:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.621+0000 7f00dea00700 1 -- 172.21.15.115:0/3159891704 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f00d8022470 con 0x7f00e8102380 2024-03-22T10:44:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.621+0000 7f00dea00700 1 -- 172.21.15.115:0/3159891704 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d800f690 con 0x7f00e8102380 2024-03-22T10:44:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.621+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e819d290 con 0x7f00e8102380 2024-03-22T10:44:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.621+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e819d800 con 0x7f00e8102380 2024-03-22T10:44:52.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.622+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e810aed0 con 0x7f00e8102380 2024-03-22T10:44:52.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.624+0000 7f00dea00700 1 -- 172.21.15.115:0/3159891704 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f00d8022ac0 con 0x7f00e8102380 2024-03-22T10:44:52.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.624+0000 7f00dea00700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f00cc06e0e0 0x7f00cc070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:52.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.625+0000 7f00dea00700 1 -- 172.21.15.115:0/3159891704 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f00d808f340 con 0x7f00e8102380 2024-03-22T10:44:52.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.625+0000 7f00e5400700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f00cc06e0e0 0x7f00cc070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:52.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.625+0000 7f00e5400700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f00cc06e0e0 0x7f00cc070590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f00d4005950 tx=0x7f00d40058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:52.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.629+0000 7f00dea00700 1 -- 172.21.15.115:0/3159891704 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f00d805c1b0 con 0x7f00e8102380 2024-03-22T10:44:52.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:52 smithi150 ceph-mon[65513]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:52.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:52.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:52.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:52 smithi150 ceph-mon[65513]: Reconfiguring mgr.smithi115.iqzwsz (monmap changed)... 2024-03-22T10:44:52.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.iqzwsz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:44:52.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:44:52.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:52.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:52 smithi150 ceph-mon[65513]: Reconfiguring daemon mgr.smithi115.iqzwsz on smithi115 2024-03-22T10:44:52.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:52 smithi150 ceph-mon[65513]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:52.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.971+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f00e819e3c0 con 0x7f00e8102380 2024-03-22T10:44:52.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.972+0000 7f00dea00700 1 -- 172.21.15.115:0/3159891704 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1926 (secure 0 0 0) 0x7f00d805bd40 con 0x7f00e8102380 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:e14 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 14 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:43:22.786262+0000 2024-03-22T10:44:52.965 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:44:52.966 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 0 members: 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:44:52.967 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:44:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:44:52.968 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:52.968 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:44:52.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.975+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f00cc06e0e0 msgr2=0x7f00cc070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:52.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.975+0000 7f00eca00700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f00cc06e0e0 0x7f00cc070590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f00d4005950 tx=0x7f00d40058e0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.975+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 msgr2=0x7f00e8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:52.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.975+0000 7f00eca00700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 0x7f00e8197ba0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f00d8004df0 tx=0x7f00d8004ed0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.976+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 shutdown_connections 2024-03-22T10:44:52.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.976+0000 7f00eca00700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f00cc06e0e0 0x7f00cc070590 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.976+0000 7f00eca00700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e8102380 0x7f00e8197ba0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.976+0000 7f00eca00700 1 --2- 172.21.15.115:0/3159891704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e8103580 0x7f00e81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:52.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.976+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 >> 172.21.15.115:0/3159891704 conn(0x7f00e80fd930 msgr2=0x7f00e81067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:52.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.976+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 shutdown_connections 2024-03-22T10:44:52.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:52.976+0000 7f00eca00700 1 -- 172.21.15.115:0/3159891704 wait complete. 2024-03-22T10:44:52.970 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 14 2024-03-22T10:44:53.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.126+0000 7fc782e00700 1 -- 172.21.15.115:0/2873418385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c074d20 msgr2=0x7fc77c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:53.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.126+0000 7fc782e00700 1 --2- 172.21.15.115:0/2873418385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c074d20 0x7fc77c073180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc770009910 tx=0x7fc770009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:53.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.126+0000 7fc782e00700 1 -- 172.21.15.115:0/2873418385 shutdown_connections 2024-03-22T10:44:53.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.126+0000 7fc782e00700 1 --2- 172.21.15.115:0/2873418385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc77c073750 0x7fc77c073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.126+0000 7fc782e00700 1 --2- 172.21.15.115:0/2873418385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c074d20 0x7fc77c073180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.126+0000 7fc782e00700 1 -- 172.21.15.115:0/2873418385 >> 172.21.15.115:0/2873418385 conn(0x7fc77c0fb570 msgr2=0x7fc77c0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:53.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.127+0000 7fc782e00700 1 -- 172.21.15.115:0/2873418385 shutdown_connections 2024-03-22T10:44:53.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.127+0000 7fc782e00700 1 -- 172.21.15.115:0/2873418385 wait complete. 2024-03-22T10:44:53.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.128+0000 7fc782e00700 1 Processor -- start 2024-03-22T10:44:53.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.128+0000 7fc782e00700 1 -- start start 2024-03-22T10:44:53.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.128+0000 7fc782e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c073750 0x7fc77c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:53.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.128+0000 7fc782e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc77c074d20 0x7fc77c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:53.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.128+0000 7fc782e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c198590 con 0x7fc77c073750 2024-03-22T10:44:53.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.128+0000 7fc782e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1986d0 con 0x7fc77c074d20 2024-03-22T10:44:53.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.129+0000 7fc781a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c073750 0x7fc77c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:53.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.129+0000 7fc781a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c073750 0x7fc77c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49658/0 (socket says 172.21.15.115:49658) 2024-03-22T10:44:53.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.129+0000 7fc781a00700 1 -- 172.21.15.115:0/2287982988 learned_addr learned my addr 172.21.15.115:0/2287982988 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:53.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.129+0000 7fc781a00700 1 -- 172.21.15.115:0/2287982988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc77c074d20 msgr2=0x7fc77c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:53.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.129+0000 7fc781a00700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc77c074d20 0x7fc77c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.129+0000 7fc781a00700 1 -- 172.21.15.115:0/2287982988 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7700095f0 con 0x7fc77c073750 2024-03-22T10:44:53.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.129+0000 7fc781a00700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c073750 0x7fc77c197a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc770005b40 tx=0x7fc77000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:53.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.130+0000 7fc77a000700 1 -- 172.21.15.115:0/2287982988 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc77001d070 con 0x7fc77c073750 2024-03-22T10:44:53.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.130+0000 7fc77a000700 1 -- 172.21.15.115:0/2287982988 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc770022470 con 0x7fc77c073750 2024-03-22T10:44:53.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.130+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c19d120 con 0x7fc77c073750 2024-03-22T10:44:53.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.130+0000 7fc77a000700 1 -- 172.21.15.115:0/2287982988 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc77000f690 con 0x7fc77c073750 2024-03-22T10:44:53.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.130+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c19d690 con 0x7fc77c073750 2024-03-22T10:44:53.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.131+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c10ad80 con 0x7fc77c073750 2024-03-22T10:44:53.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.132+0000 7fc77a000700 1 -- 172.21.15.115:0/2287982988 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fc7700225e0 con 0x7fc77c073750 2024-03-22T10:44:53.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.133+0000 7fc77a000700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76406e0d0 0x7fc764070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:53.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.133+0000 7fc77a000700 1 -- 172.21.15.115:0/2287982988 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc77008f770 con 0x7fc77c073750 2024-03-22T10:44:53.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.133+0000 7fc781000700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76406e0d0 0x7fc764070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:53.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.134+0000 7fc781000700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76406e0d0 0x7fc764070580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc7680097b0 tx=0x7fc768006d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:53.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.138+0000 7fc77a000700 1 -- 172.21.15.115:0/2287982988 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc77005c5b0 con 0x7fc77c073750 2024-03-22T10:44:53.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.399+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc77c107520 con 0x7fc76406e0d0 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.404+0000 7fc77a000700 1 -- 172.21.15.115:0/2287982988 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fc77c107520 con 0x7fc76406e0d0 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "mon" 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:44:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "4/23 daemons upgraded", 2024-03-22T10:44:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading mon daemons", 2024-03-22T10:44:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:44:53.397 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:44:53.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76406e0d0 msgr2=0x7fc764070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:53.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76406e0d0 0x7fc764070580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc7680097b0 tx=0x7fc768006d20 comp rx=0 tx=0).stop 2024-03-22T10:44:53.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c073750 msgr2=0x7fc77c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:53.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c073750 0x7fc77c197a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc770005b40 tx=0x7fc77000be20 comp rx=0 tx=0).stop 2024-03-22T10:44:53.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 shutdown_connections 2024-03-22T10:44:53.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76406e0d0 0x7fc764070580 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c073750 0x7fc77c197a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 --2- 172.21.15.115:0/2287982988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc77c074d20 0x7fc77c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 >> 172.21.15.115:0/2287982988 conn(0x7fc77c0fb570 msgr2=0x7fc77c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:53.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.407+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 shutdown_connections 2024-03-22T10:44:53.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.408+0000 7fc782e00700 1 -- 172.21.15.115:0/2287982988 wait complete. 2024-03-22T10:44:53.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.577+0000 7f7829c00700 1 -- 172.21.15.115:0/1659670709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 msgr2=0x7f7824105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:53.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.577+0000 7f7829c00700 1 --2- 172.21.15.115:0/1659670709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 0x7f7824105940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7810009910 tx=0x7f7810009c20 comp rx=0 tx=0).stop 2024-03-22T10:44:53.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.577+0000 7f7829c00700 1 -- 172.21.15.115:0/1659670709 shutdown_connections 2024-03-22T10:44:53.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.577+0000 7f7829c00700 1 --2- 172.21.15.115:0/1659670709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 0x7f7824105940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.577+0000 7f7829c00700 1 --2- 172.21.15.115:0/1659670709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7824100c40 0x7f7824103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.577+0000 7f7829c00700 1 -- 172.21.15.115:0/1659670709 >> 172.21.15.115:0/1659670709 conn(0x7f78240fa600 msgr2=0x7f78240fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:53.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.577+0000 7f7829c00700 1 -- 172.21.15.115:0/1659670709 shutdown_connections 2024-03-22T10:44:53.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.577+0000 7f7829c00700 1 -- 172.21.15.115:0/1659670709 wait complete. 2024-03-22T10:44:53.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.578+0000 7f7829c00700 1 Processor -- start 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.579+0000 7f7829c00700 1 -- start start 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.579+0000 7f7829c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7824100c40 0x7f7824197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.579+0000 7f7829c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 0x7f7824198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.579+0000 7f7829c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78241986b0 con 0x7f7824103560 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.579+0000 7f7829c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78241987f0 con 0x7f7824100c40 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.579+0000 7f7822000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 0x7f7824198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.579+0000 7f7822000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 0x7f7824198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49680/0 (socket says 172.21.15.115:49680) 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.579+0000 7f7822000700 1 -- 172.21.15.115:0/120256747 learned_addr learned my addr 172.21.15.115:0/120256747 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.580+0000 7f7822a00700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7824100c40 0x7f7824197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.580+0000 7f7822000700 1 -- 172.21.15.115:0/120256747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7824100c40 msgr2=0x7f7824197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:53.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.580+0000 7f7822000700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7824100c40 0x7f7824197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.580+0000 7f7822000700 1 -- 172.21.15.115:0/120256747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100095f0 con 0x7f7824103560 2024-03-22T10:44:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.580+0000 7f7822000700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 0x7f7824198090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7810005b40 tx=0x7f781000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.580+0000 7f781b400700 1 -- 172.21.15.115:0/120256747 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f781001d070 con 0x7f7824103560 2024-03-22T10:44:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.580+0000 7f781b400700 1 -- 172.21.15.115:0/120256747 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7810022470 con 0x7f7824103560 2024-03-22T10:44:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.580+0000 7f781b400700 1 -- 172.21.15.115:0/120256747 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f781000f690 con 0x7f7824103560 2024-03-22T10:44:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.580+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782419d240 con 0x7f7824103560 2024-03-22T10:44:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.581+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782419d730 con 0x7f7824103560 2024-03-22T10:44:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.582+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78240fc1c0 con 0x7f7824103560 2024-03-22T10:44:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.583+0000 7f781b400700 1 -- 172.21.15.115:0/120256747 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f78100225e0 con 0x7f7824103560 2024-03-22T10:44:53.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.584+0000 7f781b400700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f780806e0d0 0x7f7808070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:44:53.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.584+0000 7f781b400700 1 -- 172.21.15.115:0/120256747 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f781008f770 con 0x7f7824103560 2024-03-22T10:44:53.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.584+0000 7f7822a00700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f780806e0d0 0x7f7808070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:44:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.585+0000 7f7822a00700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f780806e0d0 0x7f7808070580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7814005950 tx=0x7f7814005cf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:44:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.589+0000 7f781b400700 1 -- 172.21.15.115:0/120256747 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f781005c5b0 con 0x7f7824103560 2024-03-22T10:44:53.639 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-22T10:44:53.640 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-22T10:44:53.724 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:53 smithi150 ceph-mon[65513]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:53.724 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:53 smithi150 ceph-mon[65513]: pgmap v27: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 912 KiB/s wr, 165 op/s 2024-03-22T10:44:53.724 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:53 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3001763606' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:44:53.724 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:53 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3159891704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:44:53.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:53 smithi115 ceph-mon[78672]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:53.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:53 smithi115 ceph-mon[78672]: pgmap v27: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 912 KiB/s wr, 165 op/s 2024-03-22T10:44:53.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:53 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3001763606' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:44:53.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:53 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3159891704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:44:53.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.913+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f782402ce40 con 0x7f7824103560 2024-03-22T10:44:53.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.913+0000 7f781b400700 1 -- 172.21.15.115:0/120256747 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f78100270e0 con 0x7f7824103560 2024-03-22T10:44:53.906 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:44:53.906 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:44:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:44:53.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f780806e0d0 msgr2=0x7f7808070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f780806e0d0 0x7f7808070580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7814005950 tx=0x7f7814005cf0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 msgr2=0x7f7824198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 0x7f7824198090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7810005b40 tx=0x7f781000be20 comp rx=0 tx=0).stop 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 shutdown_connections 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f780806e0d0 0x7f7808070580 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7824100c40 0x7f7824197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 --2- 172.21.15.115:0/120256747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7824103560 0x7f7824198090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 >> 172.21.15.115:0/120256747 conn(0x7f78240fa600 msgr2=0x7f78240ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 shutdown_connections 2024-03-22T10:44:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:44:53.916+0000 7f7829c00700 1 -- 172.21.15.115:0/120256747 wait complete. 2024-03-22T10:44:54.675 DEBUG:teuthology.parallel:result is None 2024-03-22T10:44:54.675 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: from='client.34172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: Reconfiguring ceph-exporter.smithi115 (monmap changed)... 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: Unable to update caps for client.ceph-exporter.smithi115 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi115"}]: dispatch 2024-03-22T10:44:54.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:54.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: Reconfiguring daemon ceph-exporter.smithi115 on smithi115 2024-03-22T10:44:54.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:54 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/120256747' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:44:54.776 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T10:44:54.777 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: from='client.34172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: Reconfiguring ceph-exporter.smithi115 (monmap changed)... 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: Unable to update caps for client.ceph-exporter.smithi115 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi115"}]: dispatch 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:54.861 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: Reconfiguring daemon ceph-exporter.smithi115 on smithi115 2024-03-22T10:44:54.862 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:54 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/120256747' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:44:54.935 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-22T10:44:54.935 DEBUG:teuthology.parallel:result is None 2024-03-22T10:44:55.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:55 smithi150 ceph-mon[65513]: pgmap v28: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 515 KiB/s wr, 98 op/s 2024-03-22T10:44:56.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:55 smithi115 ceph-mon[78672]: pgmap v28: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 515 KiB/s wr, 98 op/s 2024-03-22T10:44:56.866 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:56 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:56.866 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:56 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:56.866 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:56 smithi115 ceph-mon[78672]: Reconfiguring crash.smithi115 (monmap changed)... 2024-03-22T10:44:56.866 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:56 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:44:56.866 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:56 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:56.866 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:56 smithi115 ceph-mon[78672]: Reconfiguring daemon crash.smithi115 on smithi115 2024-03-22T10:44:57.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:56 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:57.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:56 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:57.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:56 smithi150 ceph-mon[65513]: Reconfiguring crash.smithi115 (monmap changed)... 2024-03-22T10:44:57.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:56 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:44:57.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:56 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:57.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:56 smithi150 ceph-mon[65513]: Reconfiguring daemon crash.smithi115 on smithi115 2024-03-22T10:44:57.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:57 smithi115 ceph-mon[78672]: pgmap v29: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 16 KiB/s rd, 559 KiB/s wr, 105 op/s 2024-03-22T10:44:58.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:57 smithi150 ceph-mon[65513]: pgmap v29: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 16 KiB/s rd, 559 KiB/s wr, 105 op/s 2024-03-22T10:44:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:58 smithi115 ceph-mon[78672]: Reconfiguring osd.0 (monmap changed)... 2024-03-22T10:44:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T10:44:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:58 smithi115 ceph-mon[78672]: Reconfiguring daemon osd.0 on smithi115 2024-03-22T10:44:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:44:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:58 smithi150 ceph-mon[65513]: Reconfiguring osd.0 (monmap changed)... 2024-03-22T10:44:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T10:44:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:44:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:58 smithi150 ceph-mon[65513]: Reconfiguring daemon osd.0 on smithi115 2024-03-22T10:45:00.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:44:59 smithi150 ceph-mon[65513]: pgmap v30: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 8.6 KiB/s rd, 280 KiB/s wr, 55 op/s 2024-03-22T10:45:00.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:44:59 smithi115 ceph-mon[78672]: pgmap v30: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 8.6 KiB/s rd, 280 KiB/s wr, 55 op/s 2024-03-22T10:45:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:01 smithi115 ceph-mon[78672]: pgmap v31: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 8.6 KiB/s rd, 280 KiB/s wr, 50 op/s 2024-03-22T10:45:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:01 smithi115 ceph-mon[78672]: Reconfiguring osd.1 (monmap changed)... 2024-03-22T10:45:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T10:45:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:01.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:01 smithi115 ceph-mon[78672]: Reconfiguring daemon osd.1 on smithi115 2024-03-22T10:45:01.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:01 smithi150 ceph-mon[65513]: pgmap v31: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 8.6 KiB/s rd, 280 KiB/s wr, 50 op/s 2024-03-22T10:45:01.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:01.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:01.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:01 smithi150 ceph-mon[65513]: Reconfiguring osd.1 (monmap changed)... 2024-03-22T10:45:01.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T10:45:01.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:01.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:01 smithi150 ceph-mon[65513]: Reconfiguring daemon osd.1 on smithi115 2024-03-22T10:45:03.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:03 smithi115 ceph-mon[78672]: pgmap v32: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 9.7 KiB/s rd, 284 KiB/s wr, 39 op/s 2024-03-22T10:45:03.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:03 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:03.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:03 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:03.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:03 smithi115 ceph-mon[78672]: Reconfiguring osd.2 (monmap changed)... 2024-03-22T10:45:03.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:03 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T10:45:03.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:03 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:03.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:03 smithi115 ceph-mon[78672]: Reconfiguring daemon osd.2 on smithi115 2024-03-22T10:45:03.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:03 smithi150 ceph-mon[65513]: pgmap v32: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 9.7 KiB/s rd, 284 KiB/s wr, 39 op/s 2024-03-22T10:45:03.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:03 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:03.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:03 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:03.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:03 smithi150 ceph-mon[65513]: Reconfiguring osd.2 (monmap changed)... 2024-03-22T10:45:03.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:03 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T10:45:03.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:03 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:03.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:03 smithi150 ceph-mon[65513]: Reconfiguring daemon osd.2 on smithi115 2024-03-22T10:45:04.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:45:04.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:45:05.821 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:05 smithi115 ceph-mon[78672]: pgmap v33: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 70 KiB/s wr, 13 op/s 2024-03-22T10:45:05.821 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:05.821 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:05.821 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:05 smithi115 ceph-mon[78672]: Reconfiguring mds.cephfs.smithi115.ggxfzi (monmap changed)... 2024-03-22T10:45:05.821 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.ggxfzi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:45:05.821 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:05.821 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:05 smithi115 ceph-mon[78672]: Reconfiguring daemon mds.cephfs.smithi115.ggxfzi on smithi115 2024-03-22T10:45:05.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:05 smithi150 ceph-mon[65513]: pgmap v33: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 70 KiB/s wr, 13 op/s 2024-03-22T10:45:05.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:05.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:05.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:05 smithi150 ceph-mon[65513]: Reconfiguring mds.cephfs.smithi115.ggxfzi (monmap changed)... 2024-03-22T10:45:05.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.ggxfzi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:45:05.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:05.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:05 smithi150 ceph-mon[65513]: Reconfiguring daemon mds.cephfs.smithi115.ggxfzi on smithi115 2024-03-22T10:45:08.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:07 smithi115 ceph-mon[78672]: pgmap v34: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 70 KiB/s wr, 13 op/s 2024-03-22T10:45:08.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:07 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:08.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:07 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:08.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:07 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.wjotmn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:45:08.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:07 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:08.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:07 smithi150 ceph-mon[65513]: pgmap v34: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 70 KiB/s wr, 13 op/s 2024-03-22T10:45:08.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:07 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:08.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:07 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:08.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:07 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.wjotmn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:45:08.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:07 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:09.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:08 smithi150 ceph-mon[65513]: Reconfiguring mds.cephfs.smithi115.wjotmn (monmap changed)... 2024-03-22T10:45:09.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:08 smithi150 ceph-mon[65513]: Reconfiguring daemon mds.cephfs.smithi115.wjotmn on smithi115 2024-03-22T10:45:09.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:08 smithi115 ceph-mon[78672]: Reconfiguring mds.cephfs.smithi115.wjotmn (monmap changed)... 2024-03-22T10:45:09.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:08 smithi115 ceph-mon[78672]: Reconfiguring daemon mds.cephfs.smithi115.wjotmn on smithi115 2024-03-22T10:45:10.145 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:09 smithi150 ceph-mon[65513]: pgmap v35: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 26 KiB/s wr, 6 op/s 2024-03-22T10:45:10.145 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:09 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:10.145 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:09 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:10.145 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:09 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:45:10.145 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:09 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:45:10.146 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:09 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-22T10:45:10.146 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:09 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi150"}]: dispatch 2024-03-22T10:45:10.146 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:09 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:10.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:09 smithi115 ceph-mon[78672]: pgmap v35: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 26 KiB/s wr, 6 op/s 2024-03-22T10:45:10.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:09 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:10.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:09 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:10.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:09 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:45:10.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:09 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:45:10.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:09 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-22T10:45:10.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:09 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi150"}]: dispatch 2024-03-22T10:45:10.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:09 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:11.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:10 smithi150 ceph-mon[65513]: Reconfiguring ceph-exporter.smithi150 (monmap changed)... 2024-03-22T10:45:11.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:10 smithi150 ceph-mon[65513]: Unable to update caps for client.ceph-exporter.smithi150 2024-03-22T10:45:11.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:10 smithi150 ceph-mon[65513]: Reconfiguring daemon ceph-exporter.smithi150 on smithi150 2024-03-22T10:45:11.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:10 smithi115 ceph-mon[78672]: Reconfiguring ceph-exporter.smithi150 (monmap changed)... 2024-03-22T10:45:11.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:10 smithi115 ceph-mon[78672]: Unable to update caps for client.ceph-exporter.smithi150 2024-03-22T10:45:11.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:10 smithi115 ceph-mon[78672]: Reconfiguring daemon ceph-exporter.smithi150 on smithi150 2024-03-22T10:45:12.098 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:11 smithi150 ceph-mon[65513]: pgmap v36: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2024-03-22T10:45:12.098 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:12.098 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:12.098 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi150", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:45:12.098 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:12.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:11 smithi115 ceph-mon[78672]: pgmap v36: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2024-03-22T10:45:12.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:12.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:12.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi150", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:45:12.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:13.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:12 smithi150 ceph-mon[65513]: Reconfiguring crash.smithi150 (monmap changed)... 2024-03-22T10:45:13.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:12 smithi150 ceph-mon[65513]: Reconfiguring daemon crash.smithi150 on smithi150 2024-03-22T10:45:13.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:12 smithi115 ceph-mon[78672]: Reconfiguring crash.smithi150 (monmap changed)... 2024-03-22T10:45:13.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:12 smithi115 ceph-mon[78672]: Reconfiguring daemon crash.smithi150 on smithi150 2024-03-22T10:45:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:13 smithi150 ceph-mon[65513]: pgmap v37: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 4.2 KiB/s wr, 2 op/s 2024-03-22T10:45:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:45:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:45:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:14.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:13 smithi115 ceph-mon[78672]: pgmap v37: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 4.2 KiB/s wr, 2 op/s 2024-03-22T10:45:14.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:14.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:14.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi150.fmoumv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T10:45:14.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T10:45:14.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:15.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:14 smithi115 ceph-mon[78672]: Reconfiguring mgr.smithi150.fmoumv (monmap changed)... 2024-03-22T10:45:15.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:14 smithi115 ceph-mon[78672]: Reconfiguring daemon mgr.smithi150.fmoumv on smithi150 2024-03-22T10:45:15.279 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:14 smithi150 ceph-mon[65513]: Reconfiguring mgr.smithi150.fmoumv (monmap changed)... 2024-03-22T10:45:15.279 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:14 smithi150 ceph-mon[65513]: Reconfiguring daemon mgr.smithi150.fmoumv on smithi150 2024-03-22T10:45:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:15 smithi115 ceph-mon[78672]: pgmap v38: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:15 smithi115 ceph-mon[78672]: Reconfiguring mon.smithi150 (monmap changed)... 2024-03-22T10:45:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:45:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:45:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:15 smithi115 ceph-mon[78672]: Reconfiguring daemon mon.smithi150 on smithi150 2024-03-22T10:45:16.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:15 smithi150 ceph-mon[65513]: pgmap v38: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:16.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:16.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:16.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:15 smithi150 ceph-mon[65513]: Reconfiguring mon.smithi150 (monmap changed)... 2024-03-22T10:45:16.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T10:45:16.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T10:45:16.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:16.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:15 smithi150 ceph-mon[65513]: Reconfiguring daemon mon.smithi150 on smithi150 2024-03-22T10:45:17.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:16 smithi115 ceph-mon[78672]: pgmap v39: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:17.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:16 smithi150 ceph-mon[65513]: pgmap v39: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:19.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:18 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:18 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:18 smithi150 ceph-mon[65513]: Reconfiguring osd.3 (monmap changed)... 2024-03-22T10:45:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:18 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T10:45:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:18 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:18 smithi150 ceph-mon[65513]: Reconfiguring daemon osd.3 on smithi150 2024-03-22T10:45:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:18 smithi150 ceph-mon[65513]: pgmap v40: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:19.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:18 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:19.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:18 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:19.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:18 smithi115 ceph-mon[78672]: Reconfiguring osd.3 (monmap changed)... 2024-03-22T10:45:19.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:18 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T10:45:19.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:18 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:19.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:18 smithi115 ceph-mon[78672]: Reconfiguring daemon osd.3 on smithi150 2024-03-22T10:45:19.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:18 smithi115 ceph-mon[78672]: pgmap v40: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:45:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:45:21.419 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:21 smithi150 ceph-mon[65513]: pgmap v41: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:21.419 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:21.420 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:21.420 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:21 smithi150 ceph-mon[65513]: Reconfiguring osd.4 (monmap changed)... 2024-03-22T10:45:21.420 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T10:45:21.420 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:21.420 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:21 smithi150 ceph-mon[65513]: Reconfiguring daemon osd.4 on smithi150 2024-03-22T10:45:21.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:21 smithi115 ceph-mon[78672]: pgmap v41: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:21.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:21.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:21.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:21 smithi115 ceph-mon[78672]: Reconfiguring osd.4 (monmap changed)... 2024-03-22T10:45:21.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T10:45:21.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:21.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:21 smithi115 ceph-mon[78672]: Reconfiguring daemon osd.4 on smithi150 2024-03-22T10:45:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:23 smithi115 ceph-mon[78672]: pgmap v42: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:23 smithi115 ceph-mon[78672]: Reconfiguring osd.5 (monmap changed)... 2024-03-22T10:45:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T10:45:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:23.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:23 smithi115 ceph-mon[78672]: Reconfiguring daemon osd.5 on smithi150 2024-03-22T10:45:23.896 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:23 smithi150 ceph-mon[65513]: pgmap v42: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:23.896 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:23 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:23.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:23 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:23.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:23 smithi150 ceph-mon[65513]: Reconfiguring osd.5 (monmap changed)... 2024-03-22T10:45:23.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:23 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T10:45:23.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:23 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:23.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:23 smithi150 ceph-mon[65513]: Reconfiguring daemon osd.5 on smithi150 2024-03-22T10:45:24.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.089+0000 7f3f69800700 1 -- 172.21.15.115:0/254455844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 msgr2=0x7f3f64107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.089+0000 7f3f69800700 1 --2- 172.21.15.115:0/254455844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 0x7f3f64107df0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009910 tx=0x7f3f50009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:24.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.089+0000 7f3f69800700 1 -- 172.21.15.115:0/254455844 shutdown_connections 2024-03-22T10:45:24.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.089+0000 7f3f69800700 1 --2- 172.21.15.115:0/254455844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 0x7f3f64107df0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.089+0000 7f3f69800700 1 --2- 172.21.15.115:0/254455844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f641067a0 0x7f3f64106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.089+0000 7f3f69800700 1 -- 172.21.15.115:0/254455844 >> 172.21.15.115:0/254455844 conn(0x7f3f64075910 msgr2=0x7f3f64077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:24.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.089+0000 7f3f69800700 1 -- 172.21.15.115:0/254455844 shutdown_connections 2024-03-22T10:45:24.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.089+0000 7f3f69800700 1 -- 172.21.15.115:0/254455844 wait complete. 2024-03-22T10:45:24.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.090+0000 7f3f69800700 1 Processor -- start 2024-03-22T10:45:24.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.090+0000 7f3f69800700 1 -- start start 2024-03-22T10:45:24.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.091+0000 7f3f69800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f641067a0 0x7f3f6419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:24.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.091+0000 7f3f69800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 0x7f3f6419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:24.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.091+0000 7f3f69800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6419cb00 con 0x7f3f641079a0 2024-03-22T10:45:24.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.091+0000 7f3f69800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6419cc40 con 0x7f3f641067a0 2024-03-22T10:45:24.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.091+0000 7f3f62000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 0x7f3f6419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:24.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.091+0000 7f3f62000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 0x7f3f6419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60260/0 (socket says 172.21.15.115:60260) 2024-03-22T10:45:24.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.091+0000 7f3f62000700 1 -- 172.21.15.115:0/1028254835 learned_addr learned my addr 172.21.15.115:0/1028254835 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:24.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.091+0000 7f3f62a00700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f641067a0 0x7f3f6419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:24.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f62000700 1 -- 172.21.15.115:0/1028254835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f641067a0 msgr2=0x7f3f6419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f62000700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f641067a0 0x7f3f6419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f62000700 1 -- 172.21.15.115:0/1028254835 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f500095f0 con 0x7f3f641079a0 2024-03-22T10:45:24.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f62a00700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f641067a0 0x7f3f6419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:45:24.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f62000700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 0x7f3f6419c4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000bd70 tx=0x7f3f5000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:24.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f5b400700 1 -- 172.21.15.115:0/1028254835 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5001d070 con 0x7f3f641079a0 2024-03-22T10:45:24.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f5b400700 1 -- 172.21.15.115:0/1028254835 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f50022470 con 0x7f3f641079a0 2024-03-22T10:45:24.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f641a1690 con 0x7f3f641079a0 2024-03-22T10:45:24.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f5b400700 1 -- 172.21.15.115:0/1028254835 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5000f670 con 0x7f3f641079a0 2024-03-22T10:45:24.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.092+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f641a1b80 con 0x7f3f641079a0 2024-03-22T10:45:24.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.094+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6410f2f0 con 0x7f3f641079a0 2024-03-22T10:45:24.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.095+0000 7f3f5b400700 1 -- 172.21.15.115:0/1028254835 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f3f50022a50 con 0x7f3f641079a0 2024-03-22T10:45:24.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.096+0000 7f3f5b400700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f3f4806e0e0 0x7f3f48070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:24.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.096+0000 7f3f5b400700 1 -- 172.21.15.115:0/1028254835 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3f5008f580 con 0x7f3f641079a0 2024-03-22T10:45:24.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.096+0000 7f3f62a00700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f3f4806e0e0 0x7f3f48070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:24.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.097+0000 7f3f62a00700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f3f4806e0e0 0x7f3f48070590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f54005fd0 tx=0x7f3f54005f00 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:24.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.101+0000 7f3f5b400700 1 -- 172.21.15.115:0/1028254835 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f5005c530 con 0x7f3f641079a0 2024-03-22T10:45:24.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.347+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f6410c2f0 con 0x7f3f4806e0e0 2024-03-22T10:45:24.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.349+0000 7f3f5b400700 1 -- 172.21.15.115:0/1028254835 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f3f6410c2f0 con 0x7f3f4806e0e0 2024-03-22T10:45:24.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.352+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f3f4806e0e0 msgr2=0x7f3f48070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.352+0000 7f3f69800700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f3f4806e0e0 0x7f3f48070590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f54005fd0 tx=0x7f3f54005f00 comp rx=0 tx=0).stop 2024-03-22T10:45:24.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.352+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 msgr2=0x7f3f6419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.352+0000 7f3f69800700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 0x7f3f6419c4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000bd70 tx=0x7f3f5000be50 comp rx=0 tx=0).stop 2024-03-22T10:45:24.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.352+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 shutdown_connections 2024-03-22T10:45:24.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.352+0000 7f3f69800700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f3f4806e0e0 0x7f3f48070590 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.352+0000 7f3f69800700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f641067a0 0x7f3f6419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.352+0000 7f3f69800700 1 --2- 172.21.15.115:0/1028254835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f641079a0 0x7f3f6419c4e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.352+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 >> 172.21.15.115:0/1028254835 conn(0x7f3f64075910 msgr2=0x7f3f6410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:24.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.353+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 shutdown_connections 2024-03-22T10:45:24.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.353+0000 7f3f69800700 1 -- 172.21.15.115:0/1028254835 wait complete. 2024-03-22T10:45:24.359 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:45:24.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.508+0000 7fd56b800700 1 -- 172.21.15.115:0/3910645593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 msgr2=0x7fd5641039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.508+0000 7fd56b800700 1 --2- 172.21.15.115:0/3910645593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 0x7fd5641039f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd550009910 tx=0x7fd550009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:24.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.508+0000 7fd56b800700 1 -- 172.21.15.115:0/3910645593 shutdown_connections 2024-03-22T10:45:24.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.508+0000 7fd56b800700 1 --2- 172.21.15.115:0/3910645593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 0x7fd5641039f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.508+0000 7fd56b800700 1 --2- 172.21.15.115:0/3910645593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641023a0 0x7fd5641027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.508+0000 7fd56b800700 1 -- 172.21.15.115:0/3910645593 >> 172.21.15.115:0/3910645593 conn(0x7fd5640fd990 msgr2=0x7fd5640ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:24.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.508+0000 7fd56b800700 1 -- 172.21.15.115:0/3910645593 shutdown_connections 2024-03-22T10:45:24.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.508+0000 7fd56b800700 1 -- 172.21.15.115:0/3910645593 wait complete. 2024-03-22T10:45:24.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.509+0000 7fd56b800700 1 Processor -- start 2024-03-22T10:45:24.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.510+0000 7fd56b800700 1 -- start start 2024-03-22T10:45:24.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.510+0000 7fd56b800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641023a0 0x7fd564197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:24.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.510+0000 7fd56b800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 0x7fd5641980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:24.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.510+0000 7fd56b800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564198700 con 0x7fd5641035a0 2024-03-22T10:45:24.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.510+0000 7fd56b800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564198840 con 0x7fd5641023a0 2024-03-22T10:45:24.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.510+0000 7fd563e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 0x7fd5641980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:24.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.510+0000 7fd563e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 0x7fd5641980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60284/0 (socket says 172.21.15.115:60284) 2024-03-22T10:45:24.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.510+0000 7fd563e00700 1 -- 172.21.15.115:0/428780806 learned_addr learned my addr 172.21.15.115:0/428780806 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:24.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.511+0000 7fd568e00700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641023a0 0x7fd564197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:24.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.511+0000 7fd563e00700 1 -- 172.21.15.115:0/428780806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641023a0 msgr2=0x7fd564197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.511+0000 7fd563e00700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641023a0 0x7fd564197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.511+0000 7fd563e00700 1 -- 172.21.15.115:0/428780806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5500095f0 con 0x7fd5641035a0 2024-03-22T10:45:24.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.511+0000 7fd563e00700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 0x7fd5641980e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd550005b40 tx=0x7fd55000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:24.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.511+0000 7fd561600700 1 -- 172.21.15.115:0/428780806 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd55001d070 con 0x7fd5641035a0 2024-03-22T10:45:24.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.511+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56419d290 con 0x7fd5641035a0 2024-03-22T10:45:24.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.511+0000 7fd561600700 1 -- 172.21.15.115:0/428780806 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd55000f460 con 0x7fd5641035a0 2024-03-22T10:45:24.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.511+0000 7fd561600700 1 -- 172.21.15.115:0/428780806 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd550021640 con 0x7fd5641035a0 2024-03-22T10:45:24.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.512+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56419d780 con 0x7fd5641035a0 2024-03-22T10:45:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.513+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56410aef0 con 0x7fd5641035a0 2024-03-22T10:45:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.514+0000 7fd561600700 1 -- 172.21.15.115:0/428780806 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fd55000fa90 con 0x7fd5641035a0 2024-03-22T10:45:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.515+0000 7fd561600700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd54c06e0d0 0x7fd54c070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.515+0000 7fd561600700 1 -- 172.21.15.115:0/428780806 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd55008f3d0 con 0x7fd5641035a0 2024-03-22T10:45:24.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.515+0000 7fd568e00700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd54c06e0d0 0x7fd54c070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:24.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.516+0000 7fd568e00700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd54c06e0d0 0x7fd54c070580 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd558005950 tx=0x7fd5580058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.520+0000 7fd561600700 1 -- 172.21.15.115:0/428780806 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd55005c110 con 0x7fd5641035a0 2024-03-22T10:45:24.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.762+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd56419de80 con 0x7fd54c06e0d0 2024-03-22T10:45:24.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.764+0000 7fd561600700 1 -- 172.21.15.115:0/428780806 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fd56419de80 con 0x7fd54c06e0d0 2024-03-22T10:45:24.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd54c06e0d0 msgr2=0x7fd54c070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd54c06e0d0 0x7fd54c070580 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd558005950 tx=0x7fd5580058e0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 msgr2=0x7fd5641980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 0x7fd5641980e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd550005b40 tx=0x7fd55000be20 comp rx=0 tx=0).stop 2024-03-22T10:45:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 shutdown_connections 2024-03-22T10:45:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd54c06e0d0 0x7fd54c070580 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641023a0 0x7fd564197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 --2- 172.21.15.115:0/428780806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5641035a0 0x7fd5641980e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 >> 172.21.15.115:0/428780806 conn(0x7fd5640fd990 msgr2=0x7fd5641067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 shutdown_connections 2024-03-22T10:45:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.767+0000 7fd56b800700 1 -- 172.21.15.115:0/428780806 wait complete. 2024-03-22T10:45:24.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.924+0000 7fe8a0600700 1 -- 172.21.15.115:0/3434161443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 msgr2=0x7fe8980731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.924+0000 7fe8a0600700 1 --2- 172.21.15.115:0/3434161443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 0x7fe8980731e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009940 tx=0x7fe88c009c50 comp rx=0 tx=0).stop 2024-03-22T10:45:24.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.924+0000 7fe8a0600700 1 -- 172.21.15.115:0/3434161443 shutdown_connections 2024-03-22T10:45:24.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.924+0000 7fe8a0600700 1 --2- 172.21.15.115:0/3434161443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8980737b0 0x7fe898073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.924+0000 7fe8a0600700 1 --2- 172.21.15.115:0/3434161443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 0x7fe8980731e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.924+0000 7fe8a0600700 1 -- 172.21.15.115:0/3434161443 >> 172.21.15.115:0/3434161443 conn(0x7fe8980fb6e0 msgr2=0x7fe8980fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:24.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.924+0000 7fe8a0600700 1 -- 172.21.15.115:0/3434161443 shutdown_connections 2024-03-22T10:45:24.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.924+0000 7fe8a0600700 1 -- 172.21.15.115:0/3434161443 wait complete. 2024-03-22T10:45:24.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.925+0000 7fe8a0600700 1 Processor -- start 2024-03-22T10:45:24.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.925+0000 7fe8a0600700 1 -- start start 2024-03-22T10:45:24.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.926+0000 7fe8a0600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8980737b0 0x7fe898103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:24.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.926+0000 7fe8a0600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 0x7fe8981039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:24.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.926+0000 7fe8a0600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980fff70 con 0x7fe898074d80 2024-03-22T10:45:24.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.926+0000 7fe8a0600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981000e0 con 0x7fe8980737b0 2024-03-22T10:45:24.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.926+0000 7fe89d200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 0x7fe8981039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:24.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.926+0000 7fe89d200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 0x7fe8981039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60304/0 (socket says 172.21.15.115:60304) 2024-03-22T10:45:24.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.926+0000 7fe89d200700 1 -- 172.21.15.115:0/2831538752 learned_addr learned my addr 172.21.15.115:0/2831538752 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:24.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.927+0000 7fe89dc00700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8980737b0 0x7fe898103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:24.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.927+0000 7fe89d200700 1 -- 172.21.15.115:0/2831538752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8980737b0 msgr2=0x7fe898103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:24.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.927+0000 7fe89d200700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8980737b0 0x7fe898103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:24.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.927+0000 7fe89d200700 1 -- 172.21.15.115:0/2831538752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c0095f0 con 0x7fe898074d80 2024-03-22T10:45:24.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.927+0000 7fe89d200700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 0x7fe8981039c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe88400d8d0 tx=0x7fe88400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:24.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.927+0000 7fe896000700 1 -- 172.21.15.115:0/2831538752 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe884009940 con 0x7fe898074d80 2024-03-22T10:45:24.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.928+0000 7fe896000700 1 -- 172.21.15.115:0/2831538752 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe884010460 con 0x7fe898074d80 2024-03-22T10:45:24.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.928+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8981003c0 con 0x7fe898074d80 2024-03-22T10:45:24.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.928+0000 7fe896000700 1 -- 172.21.15.115:0/2831538752 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe88400f5f0 con 0x7fe898074d80 2024-03-22T10:45:24.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.928+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe898100990 con 0x7fe898074d80 2024-03-22T10:45:24.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.929+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89810ae90 con 0x7fe898074d80 2024-03-22T10:45:24.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.930+0000 7fe896000700 1 -- 172.21.15.115:0/2831538752 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fe8840105d0 con 0x7fe898074d80 2024-03-22T10:45:24.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.931+0000 7fe896000700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe88006e0d0 0x7fe880070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:24.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.931+0000 7fe896000700 1 -- 172.21.15.115:0/2831538752 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe88408df00 con 0x7fe898074d80 2024-03-22T10:45:24.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.931+0000 7fe89dc00700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe88006e0d0 0x7fe880070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:24.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.932+0000 7fe89dc00700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe88006e0d0 0x7fe880070580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe88c006010 tx=0x7fe88c005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:24.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:24.936+0000 7fe896000700 1 -- 172.21.15.115:0/2831538752 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe88405acc0 con 0x7fe898074d80 2024-03-22T10:45:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.182+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe898107650 con 0x7fe88006e0d0 2024-03-22T10:45:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.199+0000 7fe896000700 1 -- 172.21.15.115:0/2831538752 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fe898107650 con 0x7fe88006e0d0 2024-03-22T10:45:25.190 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:45:25.191 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (11m) 66s ago 13m 13.8M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:45:25.191 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (14m) 66s ago 14m 7948k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:45:25.191 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (12m) 43s ago 12m 7847k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:45:25.191 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (14m) 66s ago 14m 6656k - 18.2.0 dc2bc1663786 9e7672efeafa 2024-03-22T10:45:25.191 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (12m) 43s ago 12m 6665k - 18.2.0 dc2bc1663786 9b767883509a 2024-03-22T10:45:25.191 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (11m) 66s ago 13m 74.3M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:45:25.191 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (6m) 66s ago 6m 156M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:45:25.191 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (6m) 66s ago 6m 15.2M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:45:25.192 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (6m) 43s ago 6m 15.0M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:45:25.192 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (6m) 43s ago 6m 141M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:45:25.192 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (3m) 66s ago 15m 479M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:45:25.192 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (119s) 43s ago 12m 448M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:45:25.192 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (96s) 66s ago 15m 31.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:45:25.192 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (50s) 43s ago 12m 22.9M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:45:25.192 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (13m) 66s ago 13m 7726k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:45:25.192 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (12m) 43s ago 12m 8124k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:45:25.192 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (11m) 66s ago 11m 186M 1217M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:45:25.193 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (10m) 66s ago 10m 192M 1217M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:45:25.193 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (10m) 66s ago 10m 155M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:45:25.193 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (9m) 43s ago 9m 234M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:45:25.193 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (9m) 43s ago 9m 218M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:45:25.193 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (8m) 43s ago 8m 193M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:45:25.193 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (2m) 66s ago 13m 57.7M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:45:25.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe88006e0d0 msgr2=0x7fe880070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe88006e0d0 0x7fe880070580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe88c006010 tx=0x7fe88c005a90 comp rx=0 tx=0).stop 2024-03-22T10:45:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 msgr2=0x7fe8981039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 0x7fe8981039c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe88400d8d0 tx=0x7fe88400dc90 comp rx=0 tx=0).stop 2024-03-22T10:45:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 shutdown_connections 2024-03-22T10:45:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe88006e0d0 0x7fe880070580 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8980737b0 0x7fe898103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 --2- 172.21.15.115:0/2831538752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe898074d80 0x7fe8981039c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 >> 172.21.15.115:0/2831538752 conn(0x7fe8980fb6e0 msgr2=0x7fe898105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:25.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 shutdown_connections 2024-03-22T10:45:25.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.202+0000 7fe8a0600700 1 -- 172.21.15.115:0/2831538752 wait complete. 2024-03-22T10:45:25.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.357+0000 7fe22fe00700 1 -- 172.21.15.115:0/103759169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300ff160 msgr2=0x7fe2300ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:25.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.357+0000 7fe22fe00700 1 --2- 172.21.15.115:0/103759169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300ff160 0x7fe2300ff5d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009910 tx=0x7fe21c009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.357+0000 7fe22fe00700 1 -- 172.21.15.115:0/103759169 shutdown_connections 2024-03-22T10:45:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.357+0000 7fe22fe00700 1 --2- 172.21.15.115:0/103759169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300ff160 0x7fe2300ff5d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.357+0000 7fe22fe00700 1 --2- 172.21.15.115:0/103759169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2300fe810 0x7fe2300fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.357+0000 7fe22fe00700 1 -- 172.21.15.115:0/103759169 >> 172.21.15.115:0/103759169 conn(0x7fe2300fa440 msgr2=0x7fe2300fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.357+0000 7fe22fe00700 1 -- 172.21.15.115:0/103759169 shutdown_connections 2024-03-22T10:45:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.357+0000 7fe22fe00700 1 -- 172.21.15.115:0/103759169 wait complete. 2024-03-22T10:45:25.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.358+0000 7fe22fe00700 1 Processor -- start 2024-03-22T10:45:25.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.358+0000 7fe22fe00700 1 -- start start 2024-03-22T10:45:25.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.358+0000 7fe22fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300fe810 0x7fe230197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:25.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2300ff160 0x7fe230197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:25.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22fe00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230198420 con 0x7fe2300fe810 2024-03-22T10:45:25.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22fe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230198560 con 0x7fe2300ff160 2024-03-22T10:45:25.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300fe810 0x7fe230197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:25.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300fe810 0x7fe230197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60310/0 (socket says 172.21.15.115:60310) 2024-03-22T10:45:25.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22ea00700 1 -- 172.21.15.115:0/1122030154 learned_addr learned my addr 172.21.15.115:0/1122030154 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:25.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22e000700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2300ff160 0x7fe230197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22ea00700 1 -- 172.21.15.115:0/1122030154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2300ff160 msgr2=0x7fe230197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22ea00700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2300ff160 0x7fe230197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.359+0000 7fe22ea00700 1 -- 172.21.15.115:0/1122030154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c0095f0 con 0x7fe2300fe810 2024-03-22T10:45:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.360+0000 7fe22ea00700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300fe810 0x7fe230197950 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe22000c930 tx=0x7fe22000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.360+0000 7fe227400700 1 -- 172.21.15.115:0/1122030154 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe220007ad0 con 0x7fe2300fe810 2024-03-22T10:45:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.360+0000 7fe227400700 1 -- 172.21.15.115:0/1122030154 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe220007c30 con 0x7fe2300fe810 2024-03-22T10:45:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.360+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23019cfc0 con 0x7fe2300fe810 2024-03-22T10:45:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.360+0000 7fe227400700 1 -- 172.21.15.115:0/1122030154 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe220018630 con 0x7fe2300fe810 2024-03-22T10:45:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.360+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23019d4e0 con 0x7fe2300fe810 2024-03-22T10:45:25.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.361+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23010acc0 con 0x7fe2300fe810 2024-03-22T10:45:25.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.362+0000 7fe227400700 1 -- 172.21.15.115:0/1122030154 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fe22000f450 con 0x7fe2300fe810 2024-03-22T10:45:25.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.363+0000 7fe227400700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe21406e0d0 0x7fe214070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:25.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.363+0000 7fe227400700 1 -- 172.21.15.115:0/1122030154 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe22008e250 con 0x7fe2300fe810 2024-03-22T10:45:25.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.363+0000 7fe22e000700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe21406e0d0 0x7fe214070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:25.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.364+0000 7fe22e000700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe21406e0d0 0x7fe214070580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe21c006010 tx=0x7fe21c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:25.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.368+0000 7fe227400700 1 -- 172.21.15.115:0/1122030154 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe22005af90 con 0x7fe2300fe810 2024-03-22T10:45:25.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.683+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe23019db50 con 0x7fe2300fe810 2024-03-22T10:45:25.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.684+0000 7fe227400700 1 -- 172.21.15.115:0/1122030154 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7fe22005ab20 con 0x7fe2300fe810 2024-03-22T10:45:25.678 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:45:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:45:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:45:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:45:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:45:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:45:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:45:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:45:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:45:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:45:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:45:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:45:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:45:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:45:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-03-22T10:45:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:45:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:45:25.681 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:45:25.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.687+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe21406e0d0 msgr2=0x7fe214070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:25.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.687+0000 7fe22fe00700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe21406e0d0 0x7fe214070580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe21c006010 tx=0x7fe21c00b540 comp rx=0 tx=0).stop 2024-03-22T10:45:25.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.687+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300fe810 msgr2=0x7fe230197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:25.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.687+0000 7fe22fe00700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300fe810 0x7fe230197950 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe22000c930 tx=0x7fe22000ccf0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.687+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 shutdown_connections 2024-03-22T10:45:25.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.687+0000 7fe22fe00700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe21406e0d0 0x7fe214070580 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.687+0000 7fe22fe00700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2300fe810 0x7fe230197950 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.687+0000 7fe22fe00700 1 --2- 172.21.15.115:0/1122030154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2300ff160 0x7fe230197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.687+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 >> 172.21.15.115:0/1122030154 conn(0x7fe2300fa440 msgr2=0x7fe230106e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:25.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.688+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 shutdown_connections 2024-03-22T10:45:25.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.688+0000 7fe22fe00700 1 -- 172.21.15.115:0/1122030154 wait complete. 2024-03-22T10:45:25.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.831+0000 7ff553200700 1 -- 172.21.15.115:0/1163639399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 msgr2=0x7ff5541003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:25.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.831+0000 7ff553200700 1 --2- 172.21.15.115:0/1163639399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 0x7ff5541003d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff544009910 tx=0x7ff544009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:25.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.831+0000 7ff553200700 1 -- 172.21.15.115:0/1163639399 shutdown_connections 2024-03-22T10:45:25.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.831+0000 7ff553200700 1 --2- 172.21.15.115:0/1163639399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5541011c0 0x7ff554101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.831+0000 7ff553200700 1 --2- 172.21.15.115:0/1163639399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 0x7ff5541003d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.832+0000 7ff553200700 1 -- 172.21.15.115:0/1163639399 >> 172.21.15.115:0/1163639399 conn(0x7ff5540fb570 msgr2=0x7ff5540fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.832+0000 7ff553200700 1 -- 172.21.15.115:0/1163639399 shutdown_connections 2024-03-22T10:45:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.832+0000 7ff553200700 1 -- 172.21.15.115:0/1163639399 wait complete. 2024-03-22T10:45:25.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff553200700 1 Processor -- start 2024-03-22T10:45:25.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff553200700 1 -- start start 2024-03-22T10:45:25.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff553200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 0x7ff554074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff553200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5541011c0 0x7ff554073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff553200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554073680 con 0x7ff5540fffc0 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff553200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540737c0 con 0x7ff5541011c0 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff551e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 0x7ff554074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff551e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 0x7ff554074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60318/0 (socket says 172.21.15.115:60318) 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff551e00700 1 -- 172.21.15.115:0/380879081 learned_addr learned my addr 172.21.15.115:0/380879081 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.833+0000 7ff54be00700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5541011c0 0x7ff554073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.834+0000 7ff551e00700 1 -- 172.21.15.115:0/380879081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5541011c0 msgr2=0x7ff554073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.834+0000 7ff551e00700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5541011c0 0x7ff554073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.834+0000 7ff551e00700 1 -- 172.21.15.115:0/380879081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5440095f0 con 0x7ff5540fffc0 2024-03-22T10:45:25.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.834+0000 7ff551e00700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 0x7ff554074af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff54400bd70 tx=0x7ff54400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:25.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.834+0000 7ff54aa00700 1 -- 172.21.15.115:0/380879081 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff54401d070 con 0x7ff5540fffc0 2024-03-22T10:45:25.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.834+0000 7ff54aa00700 1 -- 172.21.15.115:0/380879081 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff544005dc0 con 0x7ff5540fffc0 2024-03-22T10:45:25.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.834+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff554073a40 con 0x7ff5540fffc0 2024-03-22T10:45:25.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.834+0000 7ff54aa00700 1 -- 172.21.15.115:0/380879081 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff544021410 con 0x7ff5540fffc0 2024-03-22T10:45:25.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.834+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff554073f30 con 0x7ff5540fffc0 2024-03-22T10:45:25.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.836+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff554108b10 con 0x7ff5540fffc0 2024-03-22T10:45:25.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.837+0000 7ff54aa00700 1 -- 172.21.15.115:0/380879081 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7ff54400f7f0 con 0x7ff5540fffc0 2024-03-22T10:45:25.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.837+0000 7ff54aa00700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff53806e0d0 0x7ff538070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:25.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.837+0000 7ff54aa00700 1 -- 172.21.15.115:0/380879081 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff54408f360 con 0x7ff5540fffc0 2024-03-22T10:45:25.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.838+0000 7ff54be00700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff53806e0d0 0x7ff538070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:25.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.838+0000 7ff54be00700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff53806e0d0 0x7ff538070580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005950 tx=0x7ff53c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:25.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:25.843+0000 7ff54aa00700 1 -- 172.21.15.115:0/380879081 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff54405c0a0 con 0x7ff5540fffc0 2024-03-22T10:45:25.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:25 smithi150 ceph-mon[65513]: pgmap v43: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:25.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:25 smithi150 ceph-mon[65513]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:25.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:25 smithi150 ceph-mon[65513]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:25.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:25.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:25.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.zvyrzc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:45:25.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:26.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:25 smithi115 ceph-mon[78672]: pgmap v43: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:26.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:25 smithi115 ceph-mon[78672]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:26.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:25 smithi115 ceph-mon[78672]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:26.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:26.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:26.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.zvyrzc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:45:26.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:26.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.107+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff554108d00 con 0x7ff5540fffc0 2024-03-22T10:45:26.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.108+0000 7ff54aa00700 1 -- 172.21.15.115:0/380879081 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1926 (secure 0 0 0) 0x7ff54405bc30 con 0x7ff5540fffc0 2024-03-22T10:45:26.099 INFO:teuthology.orchestra.run.smithi115.stdout:e14 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 14 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:43:22.786262+0000 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:45:26.100 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:45:26.101 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 0 members: 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:45:26.102 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:45:26.103 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:45:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.110+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff53806e0d0 msgr2=0x7ff538070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.110+0000 7ff553200700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff53806e0d0 0x7ff538070580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005950 tx=0x7ff53c00b410 comp rx=0 tx=0).stop 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.110+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 msgr2=0x7ff554074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.110+0000 7ff553200700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 0x7ff554074af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff54400bd70 tx=0x7ff54400bda0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.111+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 shutdown_connections 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.111+0000 7ff553200700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff53806e0d0 0x7ff538070580 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.111+0000 7ff553200700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5540fffc0 0x7ff554074af0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.111+0000 7ff553200700 1 --2- 172.21.15.115:0/380879081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5541011c0 0x7ff554073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.111+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 >> 172.21.15.115:0/380879081 conn(0x7ff5540fb570 msgr2=0x7ff5541043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.111+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 shutdown_connections 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.111+0000 7ff553200700 1 -- 172.21.15.115:0/380879081 wait complete. 2024-03-22T10:45:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 14 2024-03-22T10:45:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.256+0000 7f392a400700 1 -- 172.21.15.115:0/2943046572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241079a0 msgr2=0x7f3924107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.256+0000 7f392a400700 1 --2- 172.21.15.115:0/2943046572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241079a0 0x7f3924107df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3910009910 tx=0x7f3910009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.256+0000 7f392a400700 1 -- 172.21.15.115:0/2943046572 shutdown_connections 2024-03-22T10:45:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.256+0000 7f392a400700 1 --2- 172.21.15.115:0/2943046572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241079a0 0x7f3924107df0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.256+0000 7f392a400700 1 --2- 172.21.15.115:0/2943046572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39241067a0 0x7f3924106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.256+0000 7f392a400700 1 -- 172.21.15.115:0/2943046572 >> 172.21.15.115:0/2943046572 conn(0x7f3924075910 msgr2=0x7f3924077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.257+0000 7f392a400700 1 -- 172.21.15.115:0/2943046572 shutdown_connections 2024-03-22T10:45:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.257+0000 7f392a400700 1 -- 172.21.15.115:0/2943046572 wait complete. 2024-03-22T10:45:26.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.258+0000 7f392a400700 1 Processor -- start 2024-03-22T10:45:26.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.258+0000 7f392a400700 1 -- start start 2024-03-22T10:45:26.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.258+0000 7f392a400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241067a0 0x7f392419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:26.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.258+0000 7f392a400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39241079a0 0x7f392419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:26.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.258+0000 7f392a400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392419cb70 con 0x7f39241067a0 2024-03-22T10:45:26.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.258+0000 7f392a400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392419ccb0 con 0x7f39241079a0 2024-03-22T10:45:26.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.258+0000 7f3923400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241067a0 0x7f392419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:26.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.258+0000 7f3923400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241067a0 0x7f392419c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60344/0 (socket says 172.21.15.115:60344) 2024-03-22T10:45:26.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.258+0000 7f3923400700 1 -- 172.21.15.115:0/2585106863 learned_addr learned my addr 172.21.15.115:0/2585106863 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f3922a00700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39241079a0 0x7f392419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f3923400700 1 -- 172.21.15.115:0/2585106863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39241079a0 msgr2=0x7f392419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f3923400700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39241079a0 0x7f392419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f3923400700 1 -- 172.21.15.115:0/2585106863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39100095f0 con 0x7f39241067a0 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f3923400700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241067a0 0x7f392419c010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f391400b480 tx=0x7f391400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f391be00700 1 -- 172.21.15.115:0/2585106863 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39140118c0 con 0x7f39241067a0 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f391be00700 1 -- 172.21.15.115:0/2585106863 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3914010380 con 0x7f39241067a0 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f391be00700 1 -- 172.21.15.115:0/2585106863 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f391400f4f0 con 0x7f39241067a0 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39241a1760 con 0x7f39241067a0 2024-03-22T10:45:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.259+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39241a1d30 con 0x7f39241067a0 2024-03-22T10:45:26.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.261+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392410f390 con 0x7f39241067a0 2024-03-22T10:45:26.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.262+0000 7f391be00700 1 -- 172.21.15.115:0/2585106863 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f39140104f0 con 0x7f39241067a0 2024-03-22T10:45:26.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.262+0000 7f391be00700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f390806e0d0 0x7f3908070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:26.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.263+0000 7f391be00700 1 -- 172.21.15.115:0/2585106863 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f391408daf0 con 0x7f39241067a0 2024-03-22T10:45:26.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.263+0000 7f3922a00700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f390806e0d0 0x7f3908070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:26.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.263+0000 7f3922a00700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f390806e0d0 0x7f3908070580 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3910006010 tx=0x7f391000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:26.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.267+0000 7f391be00700 1 -- 172.21.15.115:0/2585106863 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f391405a9c0 con 0x7f39241067a0 2024-03-22T10:45:26.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.500+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f392410c2f0 con 0x7f390806e0d0 2024-03-22T10:45:26.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.502+0000 7f391be00700 1 -- 172.21.15.115:0/2585106863 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f392410c2f0 con 0x7f390806e0d0 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: "mon" 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "4/23 daemons upgraded", 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading mon daemons", 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:45:26.494 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.505+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f390806e0d0 msgr2=0x7f3908070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.505+0000 7f392a400700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f390806e0d0 0x7f3908070580 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3910006010 tx=0x7f391000b540 comp rx=0 tx=0).stop 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.505+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241067a0 msgr2=0x7f392419c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.505+0000 7f392a400700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241067a0 0x7f392419c010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f391400b480 tx=0x7f391400b840 comp rx=0 tx=0).stop 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.505+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 shutdown_connections 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.505+0000 7f392a400700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f390806e0d0 0x7f3908070580 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.505+0000 7f392a400700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39241067a0 0x7f392419c010 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.505+0000 7f392a400700 1 --2- 172.21.15.115:0/2585106863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39241079a0 0x7f392419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.505+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 >> 172.21.15.115:0/2585106863 conn(0x7f3924075910 msgr2=0x7f392410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:26.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.506+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 shutdown_connections 2024-03-22T10:45:26.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.506+0000 7f392a400700 1 -- 172.21.15.115:0/2585106863 wait complete. 2024-03-22T10:45:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.651+0000 7f032b800700 1 -- 172.21.15.115:0/1106451403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241023a0 msgr2=0x7f03241027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.651+0000 7f032b800700 1 --2- 172.21.15.115:0/1106451403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241023a0 0x7f03241027b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0318009910 tx=0x7f0318009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.652+0000 7f032b800700 1 -- 172.21.15.115:0/1106451403 shutdown_connections 2024-03-22T10:45:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.652+0000 7f032b800700 1 --2- 172.21.15.115:0/1106451403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03241035a0 0x7f03241039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.652+0000 7f032b800700 1 --2- 172.21.15.115:0/1106451403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241023a0 0x7f03241027b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.652+0000 7f032b800700 1 -- 172.21.15.115:0/1106451403 >> 172.21.15.115:0/1106451403 conn(0x7f03240fd990 msgr2=0x7f03240ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.652+0000 7f032b800700 1 -- 172.21.15.115:0/1106451403 shutdown_connections 2024-03-22T10:45:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.652+0000 7f032b800700 1 -- 172.21.15.115:0/1106451403 wait complete. 2024-03-22T10:45:26.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.653+0000 7f032b800700 1 Processor -- start 2024-03-22T10:45:26.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.653+0000 7f032b800700 1 -- start start 2024-03-22T10:45:26.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.653+0000 7f032b800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03241023a0 0x7f0324197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:26.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.653+0000 7f032b800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241035a0 0x7f0324198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:26.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.654+0000 7f032b800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03241987a0 con 0x7f03241035a0 2024-03-22T10:45:26.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.654+0000 7f032b800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032419d1b0 con 0x7f03241023a0 2024-03-22T10:45:26.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.654+0000 7f0323e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241035a0 0x7f0324198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:26.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.654+0000 7f0323e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241035a0 0x7f0324198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60372/0 (socket says 172.21.15.115:60372) 2024-03-22T10:45:26.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.654+0000 7f0323e00700 1 -- 172.21.15.115:0/3669678076 learned_addr learned my addr 172.21.15.115:0/3669678076 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.654+0000 7f0328e00700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03241023a0 0x7f0324197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.654+0000 7f0323e00700 1 -- 172.21.15.115:0/3669678076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03241023a0 msgr2=0x7f0324197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.654+0000 7f0323e00700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03241023a0 0x7f0324197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.654+0000 7f0323e00700 1 -- 172.21.15.115:0/3669678076 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180095f0 con 0x7f03241035a0 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.655+0000 7f0323e00700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241035a0 0x7f0324198180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f031000b4b0 tx=0x7f031000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.655+0000 7f0321600700 1 -- 172.21.15.115:0/3669678076 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0310011800 con 0x7f03241035a0 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.655+0000 7f0321600700 1 -- 172.21.15.115:0/3669678076 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0310011e10 con 0x7f03241035a0 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.655+0000 7f0321600700 1 -- 172.21.15.115:0/3669678076 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f031000f2e0 con 0x7f03241035a0 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.655+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032419d3b0 con 0x7f03241035a0 2024-03-22T10:45:26.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.655+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032419d950 con 0x7f03241035a0 2024-03-22T10:45:26.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.656+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032410af70 con 0x7f03241035a0 2024-03-22T10:45:26.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.658+0000 7f0321600700 1 -- 172.21.15.115:0/3669678076 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f0310010380 con 0x7f03241035a0 2024-03-22T10:45:26.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.659+0000 7f0321600700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f030c06e0d0 0x7f030c070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:26.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.659+0000 7f0321600700 1 -- 172.21.15.115:0/3669678076 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f031008da40 con 0x7f03241035a0 2024-03-22T10:45:26.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.659+0000 7f0328e00700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f030c06e0d0 0x7f030c070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:26.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.660+0000 7f0328e00700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f030c06e0d0 0x7f030c070580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0318005090 tx=0x7f031800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:26.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.664+0000 7f0321600700 1 -- 172.21.15.115:0/3669678076 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0310056df0 con 0x7f03241035a0 2024-03-22T10:45:26.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:26 smithi115 ceph-mon[78672]: Reconfiguring mds.cephfs.smithi150.zvyrzc (monmap changed)... 2024-03-22T10:45:26.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:26 smithi115 ceph-mon[78672]: Reconfiguring daemon mds.cephfs.smithi150.zvyrzc on smithi150 2024-03-22T10:45:26.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:26 smithi115 ceph-mon[78672]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:26.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:26 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1122030154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:26.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:26 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/380879081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:45:26.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:26 smithi150 ceph-mon[65513]: Reconfiguring mds.cephfs.smithi150.zvyrzc (monmap changed)... 2024-03-22T10:45:26.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:26 smithi150 ceph-mon[65513]: Reconfiguring daemon mds.cephfs.smithi150.zvyrzc on smithi150 2024-03-22T10:45:26.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:26 smithi150 ceph-mon[65513]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:26.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:26 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1122030154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:26.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:26 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/380879081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:45:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.954+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f032419e010 con 0x7f03241035a0 2024-03-22T10:45:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.955+0000 7f0321600700 1 -- 172.21.15.115:0/3669678076 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f03100150e0 con 0x7f03241035a0 2024-03-22T10:45:26.946 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:45:26.946 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:45:26.946 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:45:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f030c06e0d0 msgr2=0x7f030c070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f030c06e0d0 0x7f030c070580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0318005090 tx=0x7f031800b540 comp rx=0 tx=0).stop 2024-03-22T10:45:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241035a0 msgr2=0x7f0324198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241035a0 0x7f0324198180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f031000b4b0 tx=0x7f031000b870 comp rx=0 tx=0).stop 2024-03-22T10:45:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 shutdown_connections 2024-03-22T10:45:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f030c06e0d0 0x7f030c070580 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03241023a0 0x7f0324197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 --2- 172.21.15.115:0/3669678076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03241035a0 0x7f0324198180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 >> 172.21.15.115:0/3669678076 conn(0x7f03240fd990 msgr2=0x7f03241067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 shutdown_connections 2024-03-22T10:45:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:26.958+0000 7f032b800700 1 -- 172.21.15.115:0/3669678076 wait complete. 2024-03-22T10:45:28.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:27 smithi115 ceph-mon[78672]: pgmap v44: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:28.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:27 smithi115 ceph-mon[78672]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:28.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:27 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3669678076' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:45:28.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:28.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:28.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.slzwwb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:45:28.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:28.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:27 smithi150 ceph-mon[65513]: pgmap v44: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:28.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:27 smithi150 ceph-mon[65513]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:28.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:27 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3669678076' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:45:28.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:28.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:28.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.slzwwb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:45:28.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:28.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:28 smithi150 ceph-mon[65513]: Reconfiguring mds.cephfs.smithi150.slzwwb (monmap changed)... 2024-03-22T10:45:28.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:28 smithi150 ceph-mon[65513]: Reconfiguring daemon mds.cephfs.smithi150.slzwwb on smithi150 2024-03-22T10:45:29.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:28 smithi115 ceph-mon[78672]: Reconfiguring mds.cephfs.smithi150.slzwwb (monmap changed)... 2024-03-22T10:45:29.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:28 smithi115 ceph-mon[78672]: Reconfiguring daemon mds.cephfs.smithi150.slzwwb on smithi150 2024-03-22T10:45:30.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: pgmap v45: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi115"}]: dispatch 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi115"}]': finished 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi150"}]: dispatch 2024-03-22T10:45:30.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi150"}]': finished 2024-03-22T10:45:30.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: pgmap v45: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:30.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:45:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi115"}]: dispatch 2024-03-22T10:45:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi115"}]': finished 2024-03-22T10:45:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi150"}]: dispatch 2024-03-22T10:45:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi150"}]': finished 2024-03-22T10:45:30.681 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:30 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all mon 2024-03-22T10:45:31.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:30 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all mon 2024-03-22T10:45:32.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:31 smithi115 ceph-mon[78672]: pgmap v46: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:32.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:31 smithi150 ceph-mon[65513]: pgmap v46: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:32.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:32 smithi115 ceph-mon[78672]: Upgrade: Updating crash.smithi115 (1/2) 2024-03-22T10:45:32.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:32 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:32.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:32 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:45:32.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:32 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:32.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:32 smithi115 ceph-mon[78672]: Deploying daemon crash.smithi115 on smithi115 2024-03-22T10:45:33.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:32 smithi150 ceph-mon[65513]: Upgrade: Updating crash.smithi115 (1/2) 2024-03-22T10:45:33.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:32 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:33.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:32 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:45:33.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:32 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:33.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:32 smithi150 ceph-mon[65513]: Deploying daemon crash.smithi115 on smithi115 2024-03-22T10:45:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:33 smithi115 ceph-mon[78672]: pgmap v47: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:34.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:33 smithi150 ceph-mon[65513]: pgmap v47: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:35.206 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:45:35.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:45:36.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:35 smithi115 ceph-mon[78672]: pgmap v48: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:36.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:35 smithi150 ceph-mon[65513]: pgmap v48: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:37.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:36 smithi115 ceph-mon[78672]: pgmap v49: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T10:45:37.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:36 smithi150 ceph-mon[65513]: pgmap v49: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T10:45:38.203 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:37 smithi150 ceph-mon[65513]: mgrmap e37: smithi115.iqzwsz(active, since 92s), standbys: smithi150.fmoumv 2024-03-22T10:45:38.203 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:37 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:38.203 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:37 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:38.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:37 smithi115 ceph-mon[78672]: mgrmap e37: smithi115.iqzwsz(active, since 92s), standbys: smithi150.fmoumv 2024-03-22T10:45:38.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:37 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:38.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:37 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:39.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:38 smithi150 ceph-mon[65513]: pgmap v50: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:39.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:38 smithi115 ceph-mon[78672]: pgmap v50: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:40.551 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:40 smithi150 ceph-mon[65513]: Upgrade: Updating crash.smithi150 (2/2) 2024-03-22T10:45:40.551 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:40.551 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi150", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:45:40.551 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:40.551 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:40 smithi150 ceph-mon[65513]: Deploying daemon crash.smithi150 on smithi150 2024-03-22T10:45:40.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:40 smithi115 ceph-mon[78672]: Upgrade: Updating crash.smithi150 (2/2) 2024-03-22T10:45:40.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:40.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi150", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T10:45:40.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:40.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:40 smithi115 ceph-mon[78672]: Deploying daemon crash.smithi150 on smithi150 2024-03-22T10:45:41.811 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:41 smithi150 ceph-mon[65513]: pgmap v51: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:42.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:41 smithi115 ceph-mon[78672]: pgmap v51: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:43.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:43 smithi115 ceph-mon[78672]: pgmap v52: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:43.888 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:43 smithi150 ceph-mon[65513]: pgmap v52: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:46.109 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:45 smithi115 ceph-mon[78672]: pgmap v53: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:46.109 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:46.109 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:46.109 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:45:46.117 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:45 smithi150 ceph-mon[65513]: pgmap v53: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:46.117 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:46.117 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:46.117 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:45:48.051 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:47 smithi115 ceph-mon[78672]: pgmap v54: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:48.062 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:47 smithi150 ceph-mon[65513]: pgmap v54: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T10:45:49.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:49 smithi150 ceph-mon[65513]: pgmap v55: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T10:45:49.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:49.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:45:49.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:50.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:49 smithi115 ceph-mon[78672]: pgmap v55: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T10:45:50.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:50.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:45:50.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:51.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:51 smithi150 ceph-mon[65513]: pgmap v56: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:51.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:51.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:51.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:51.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:52.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:51 smithi115 ceph-mon[78672]: pgmap v56: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:52.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:52.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:52.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:52.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:54.082 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:53 smithi150 ceph-mon[65513]: pgmap v57: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:54.083 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:53 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:54.083 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:53 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:53 smithi115 ceph-mon[78672]: pgmap v57: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:53 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:53 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:56.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:55 smithi115 ceph-mon[78672]: pgmap v58: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:56.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:55 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:56.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:55 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:56.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:55 smithi150 ceph-mon[65513]: pgmap v58: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:56.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:55 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:56.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:55 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.118+0000 7f5721a00700 1 -- 172.21.15.115:0/3539787126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 msgr2=0x7f571c1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.118+0000 7f5721a00700 1 --2- 172.21.15.115:0/3539787126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 0x7f571c1017a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5704009910 tx=0x7f5704009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.118+0000 7f5721a00700 1 -- 172.21.15.115:0/3539787126 shutdown_connections 2024-03-22T10:45:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.118+0000 7f5721a00700 1 --2- 172.21.15.115:0/3539787126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 0x7f571c1017a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.118+0000 7f5721a00700 1 --2- 172.21.15.115:0/3539787126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f571c100150 0x7f571c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.118+0000 7f5721a00700 1 -- 172.21.15.115:0/3539787126 >> 172.21.15.115:0/3539787126 conn(0x7f571c0fb6e0 msgr2=0x7f571c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.118+0000 7f5721a00700 1 -- 172.21.15.115:0/3539787126 shutdown_connections 2024-03-22T10:45:57.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.118+0000 7f5721a00700 1 -- 172.21.15.115:0/3539787126 wait complete. 2024-03-22T10:45:57.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.120+0000 7f5721a00700 1 Processor -- start 2024-03-22T10:45:57.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.120+0000 7f5721a00700 1 -- start start 2024-03-22T10:45:57.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.120+0000 7f5721a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f571c100150 0x7f571c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:57.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.120+0000 7f5721a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 0x7f571c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:57.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.120+0000 7f5721a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c198640 con 0x7f571c101350 2024-03-22T10:45:57.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.120+0000 7f5721a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c198780 con 0x7f571c100150 2024-03-22T10:45:57.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.121+0000 7f5713e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 0x7f571c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:57.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.121+0000 7f5713e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 0x7f571c1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48732/0 (socket says 172.21.15.115:48732) 2024-03-22T10:45:57.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.121+0000 7f5713e00700 1 -- 172.21.15.115:0/1762309681 learned_addr learned my addr 172.21.15.115:0/1762309681 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:57.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.121+0000 7f571aa00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f571c100150 0x7f571c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:57.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.121+0000 7f5713e00700 1 -- 172.21.15.115:0/1762309681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f571c100150 msgr2=0x7f571c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.121+0000 7f5713e00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f571c100150 0x7f571c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.121+0000 7f5713e00700 1 -- 172.21.15.115:0/1762309681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040095f0 con 0x7f571c101350 2024-03-22T10:45:57.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.122+0000 7f5713e00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 0x7f571c1980b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5704005b40 tx=0x7f570400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:57.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.122+0000 7f5713400700 1 -- 172.21.15.115:0/1762309681 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f570401d070 con 0x7f571c101350 2024-03-22T10:45:57.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.122+0000 7f5713400700 1 -- 172.21.15.115:0/1762309681 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5704022470 con 0x7f571c101350 2024-03-22T10:45:57.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.122+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c19d1e0 con 0x7f571c101350 2024-03-22T10:45:57.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.122+0000 7f5713400700 1 -- 172.21.15.115:0/1762309681 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f570400f690 con 0x7f571c101350 2024-03-22T10:45:57.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.122+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c19d6a0 con 0x7f571c101350 2024-03-22T10:45:57.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.124+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f8005320 con 0x7f571c101350 2024-03-22T10:45:57.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.124+0000 7f5713400700 1 -- 172.21.15.115:0/1762309681 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f570400f880 con 0x7f571c101350 2024-03-22T10:45:57.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.125+0000 7f5713400700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f570006dfa0 0x7f5700070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:57.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.125+0000 7f5713400700 1 -- 172.21.15.115:0/1762309681 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f570408f5d0 con 0x7f571c101350 2024-03-22T10:45:57.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.126+0000 7f571aa00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f570006dfa0 0x7f5700070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:57.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.126+0000 7f571aa00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f570006dfa0 0x7f5700070450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f570c005fd0 tx=0x7f570c005e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:57.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.131+0000 7f5713400700 1 -- 172.21.15.115:0/1762309681 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f570405c5b0 con 0x7f571c101350 2024-03-22T10:45:57.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.380+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56f8000bf0 con 0x7f570006dfa0 2024-03-22T10:45:57.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.382+0000 7f5713400700 1 -- 172.21.15.115:0/1762309681 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f56f8000bf0 con 0x7f570006dfa0 2024-03-22T10:45:57.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.385+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f570006dfa0 msgr2=0x7f5700070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.385+0000 7f5721a00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f570006dfa0 0x7f5700070450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f570c005fd0 tx=0x7f570c005e50 comp rx=0 tx=0).stop 2024-03-22T10:45:57.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.385+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 msgr2=0x7f571c1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.385+0000 7f5721a00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 0x7f571c1980b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5704005b40 tx=0x7f570400be20 comp rx=0 tx=0).stop 2024-03-22T10:45:57.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.386+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 shutdown_connections 2024-03-22T10:45:57.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.386+0000 7f5721a00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f570006dfa0 0x7f5700070450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.386+0000 7f5721a00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f571c100150 0x7f571c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.386+0000 7f5721a00700 1 --2- 172.21.15.115:0/1762309681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f571c101350 0x7f571c1980b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.386+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 >> 172.21.15.115:0/1762309681 conn(0x7f571c0fb6e0 msgr2=0x7f571c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:57.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.386+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 shutdown_connections 2024-03-22T10:45:57.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.386+0000 7f5721a00700 1 -- 172.21.15.115:0/1762309681 wait complete. 2024-03-22T10:45:57.394 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:45:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.538+0000 7f7e5fc00700 1 -- 172.21.15.115:0/2980347222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 msgr2=0x7f7e58100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.538+0000 7f7e5fc00700 1 --2- 172.21.15.115:0/2980347222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 0x7f7e58100560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c009910 tx=0x7f7e4c009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:57.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.538+0000 7f7e5fc00700 1 -- 172.21.15.115:0/2980347222 shutdown_connections 2024-03-22T10:45:57.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.538+0000 7f7e5fc00700 1 --2- 172.21.15.115:0/2980347222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e58101350 0x7f7e581017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.538+0000 7f7e5fc00700 1 --2- 172.21.15.115:0/2980347222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 0x7f7e58100560 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.538+0000 7f7e5fc00700 1 -- 172.21.15.115:0/2980347222 >> 172.21.15.115:0/2980347222 conn(0x7f7e580fb6e0 msgr2=0x7f7e580fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:57.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.538+0000 7f7e5fc00700 1 -- 172.21.15.115:0/2980347222 shutdown_connections 2024-03-22T10:45:57.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.538+0000 7f7e5fc00700 1 -- 172.21.15.115:0/2980347222 wait complete. 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.539+0000 7f7e5fc00700 1 Processor -- start 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.540+0000 7f7e5fc00700 1 -- start start 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.540+0000 7f7e5fc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 0x7f7e58074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.540+0000 7f7e5fc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e58101350 0x7f7e58073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.540+0000 7f7e5fc00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e58073680 con 0x7f7e58100150 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.540+0000 7f7e5fc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580737c0 con 0x7f7e58101350 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.540+0000 7f7e5d200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 0x7f7e58074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.540+0000 7f7e5d200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 0x7f7e58074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48748/0 (socket says 172.21.15.115:48748) 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.540+0000 7f7e5d200700 1 -- 172.21.15.115:0/856309733 learned_addr learned my addr 172.21.15.115:0/856309733 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.541+0000 7f7e5d200700 1 -- 172.21.15.115:0/856309733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e58101350 msgr2=0x7f7e58073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.541+0000 7f7e5d200700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e58101350 0x7f7e58073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.541+0000 7f7e5d200700 1 -- 172.21.15.115:0/856309733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c0095f0 con 0x7f7e58100150 2024-03-22T10:45:57.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.541+0000 7f7e5d200700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 0x7f7e58074af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00bda0 tx=0x7f7e4c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:57.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.541+0000 7f7e55600700 1 -- 172.21.15.115:0/856309733 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e4c01d070 con 0x7f7e58100150 2024-03-22T10:45:57.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.541+0000 7f7e55600700 1 -- 172.21.15.115:0/856309733 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7e4c022470 con 0x7f7e58100150 2024-03-22T10:45:57.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.541+0000 7f7e55600700 1 -- 172.21.15.115:0/856309733 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e4c00f650 con 0x7f7e58100150 2024-03-22T10:45:57.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.541+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e58073a40 con 0x7f7e58100150 2024-03-22T10:45:57.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.541+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e58073f30 con 0x7f7e58100150 2024-03-22T10:45:57.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.542+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e58108ca0 con 0x7f7e58100150 2024-03-22T10:45:57.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.544+0000 7f7e55600700 1 -- 172.21.15.115:0/856309733 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f7e4c022a80 con 0x7f7e58100150 2024-03-22T10:45:57.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.544+0000 7f7e55600700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7e4006dfa0 0x7f7e40070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:57.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.544+0000 7f7e55600700 1 -- 172.21.15.115:0/856309733 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7e4c08f2d0 con 0x7f7e58100150 2024-03-22T10:45:57.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.545+0000 7f7e57e00700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7e4006dfa0 0x7f7e40070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:57.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.545+0000 7f7e57e00700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7e4006dfa0 0x7f7e40070450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400a990 tx=0x7f7e44005c80 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:57.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.549+0000 7f7e55600700 1 -- 172.21.15.115:0/856309733 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e4c05c480 con 0x7f7e58100150 2024-03-22T10:45:57.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.800+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e58105ca0 con 0x7f7e4006dfa0 2024-03-22T10:45:57.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.803+0000 7f7e55600700 1 -- 172.21.15.115:0/856309733 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f7e58105ca0 con 0x7f7e4006dfa0 2024-03-22T10:45:57.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.805+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7e4006dfa0 msgr2=0x7f7e40070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7e4006dfa0 0x7f7e40070450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400a990 tx=0x7f7e44005c80 comp rx=0 tx=0).stop 2024-03-22T10:45:57.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 msgr2=0x7f7e58074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 0x7f7e58074af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00bda0 tx=0x7f7e4c00be80 comp rx=0 tx=0).stop 2024-03-22T10:45:57.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 shutdown_connections 2024-03-22T10:45:57.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7e4006dfa0 0x7f7e40070450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e58100150 0x7f7e58074af0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 --2- 172.21.15.115:0/856309733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e58101350 0x7f7e58073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 >> 172.21.15.115:0/856309733 conn(0x7f7e580fb6e0 msgr2=0x7f7e58104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:57.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 shutdown_connections 2024-03-22T10:45:57.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.806+0000 7f7e5fc00700 1 -- 172.21.15.115:0/856309733 wait complete. 2024-03-22T10:45:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.963+0000 7f7add400700 1 -- 172.21.15.115:0/3755271475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 msgr2=0x7f7ad8102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.963+0000 7f7add400700 1 --2- 172.21.15.115:0/3755271475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 0x7f7ad8102fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009910 tx=0x7f7ac8009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.963+0000 7f7add400700 1 -- 172.21.15.115:0/3755271475 shutdown_connections 2024-03-22T10:45:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.963+0000 7f7add400700 1 --2- 172.21.15.115:0/3755271475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ad8103500 0x7f7ad81058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.963+0000 7f7add400700 1 --2- 172.21.15.115:0/3755271475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 0x7f7ad8102fc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.963+0000 7f7add400700 1 -- 172.21.15.115:0/3755271475 >> 172.21.15.115:0/3755271475 conn(0x7f7ad80fa5e0 msgr2=0x7f7ad80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.963+0000 7f7add400700 1 -- 172.21.15.115:0/3755271475 shutdown_connections 2024-03-22T10:45:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.963+0000 7f7add400700 1 -- 172.21.15.115:0/3755271475 wait complete. 2024-03-22T10:45:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.964+0000 7f7add400700 1 Processor -- start 2024-03-22T10:45:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.964+0000 7f7add400700 1 -- start start 2024-03-22T10:45:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.965+0000 7f7add400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 0x7f7ad8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.965+0000 7f7add400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ad8103500 0x7f7ad8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.965+0000 7f7add400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad8198660 con 0x7f7ad8100be0 2024-03-22T10:45:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.965+0000 7f7add400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad81987a0 con 0x7f7ad8103500 2024-03-22T10:45:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.965+0000 7f7ad6800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 0x7f7ad8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.965+0000 7f7ad6800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 0x7f7ad8197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48754/0 (socket says 172.21.15.115:48754) 2024-03-22T10:45:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.965+0000 7f7ad6800700 1 -- 172.21.15.115:0/3811292518 learned_addr learned my addr 172.21.15.115:0/3811292518 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.965+0000 7f7ad5e00700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ad8103500 0x7f7ad8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.966+0000 7f7ad6800700 1 -- 172.21.15.115:0/3811292518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ad8103500 msgr2=0x7f7ad8198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.966+0000 7f7ad6800700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ad8103500 0x7f7ad8198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.966+0000 7f7ad6800700 1 -- 172.21.15.115:0/3811292518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80095f0 con 0x7f7ad8100be0 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.966+0000 7f7ad6800700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 0x7f7ad8197b00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8005b40 tx=0x7f7ac800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.966+0000 7f7acf400700 1 -- 172.21.15.115:0/3811292518 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ac801d070 con 0x7f7ad8100be0 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.966+0000 7f7acf400700 1 -- 172.21.15.115:0/3811292518 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7ac8022470 con 0x7f7ad8100be0 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.966+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad819d1f0 con 0x7f7ad8100be0 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.966+0000 7f7acf400700 1 -- 172.21.15.115:0/3811292518 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ac800f690 con 0x7f7ad8100be0 2024-03-22T10:45:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.967+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad819d6e0 con 0x7f7ad8100be0 2024-03-22T10:45:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.968+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad80fc1c0 con 0x7f7ad8100be0 2024-03-22T10:45:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.969+0000 7f7acf400700 1 -- 172.21.15.115:0/3811292518 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f7ac80225e0 con 0x7f7ad8100be0 2024-03-22T10:45:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.970+0000 7f7acf400700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7abc06dfa0 0x7f7abc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.970+0000 7f7acf400700 1 -- 172.21.15.115:0/3811292518 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7ac808f650 con 0x7f7ad8100be0 2024-03-22T10:45:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.970+0000 7f7ad5e00700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7abc06dfa0 0x7f7abc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.971+0000 7f7ad5e00700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7abc06dfa0 0x7f7abc070450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4005fd0 tx=0x7f7ac400a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:57.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:57.975+0000 7f7acf400700 1 -- 172.21.15.115:0/3811292518 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ac805c630 con 0x7f7ad8100be0 2024-03-22T10:45:58.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:57 smithi150 ceph-mon[65513]: pgmap v59: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:58.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.214+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ad8061190 con 0x7f7abc06dfa0 2024-03-22T10:45:58.205 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:57 smithi115 ceph-mon[78672]: pgmap v59: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:45:58.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.235+0000 7f7acf400700 1 -- 172.21.15.115:0/3811292518 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7ad8061190 con 0x7f7abc06dfa0 2024-03-22T10:45:58.226 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:45:58.226 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (12m) 6s ago 14m 14.0M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:45:58.226 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (14m) 6s ago 14m 8078k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:45:58.226 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (13m) 8s ago 13m 7952k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:45:58.226 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (21s) 6s ago 14m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:45:58.226 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (14s) 8s ago 13m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:45:58.226 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (12m) 6s ago 14m 74.4M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:45:58.226 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (7m) 6s ago 7m 127M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (7m) 6s ago 7m 16.2M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (7m) 8s ago 7m 15.5M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (7m) 8s ago 7m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (3m) 6s ago 16m 502M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (2m) 8s ago 13m 449M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (2m) 6s ago 16m 42.7M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (83s) 8s ago 12m 33.6M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (14m) 6s ago 14m 7982k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (13m) 8s ago 13m 7721k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (11m) 6s ago 11m 202M 1217M 18.2.0 dc2bc1663786 3dff5085579f 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (11m) 6s ago 11m 208M 1217M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (10m) 6s ago 10m 165M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (10m) 8s ago 10m 235M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (9m) 8s ago 9m 225M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (9m) 8s ago 9m 195M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:45:58.227 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (2m) 6s ago 13m 67.7M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.237+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7abc06dfa0 msgr2=0x7f7abc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.237+0000 7f7add400700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7abc06dfa0 0x7f7abc070450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4005fd0 tx=0x7f7ac400a560 comp rx=0 tx=0).stop 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.237+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 msgr2=0x7f7ad8197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.237+0000 7f7add400700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 0x7f7ad8197b00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8005b40 tx=0x7f7ac800be20 comp rx=0 tx=0).stop 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.238+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 shutdown_connections 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.238+0000 7f7add400700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7abc06dfa0 0x7f7abc070450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.238+0000 7f7add400700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ad8100be0 0x7f7ad8197b00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.238+0000 7f7add400700 1 --2- 172.21.15.115:0/3811292518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ad8103500 0x7f7ad8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.238+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 >> 172.21.15.115:0/3811292518 conn(0x7f7ad80fa5e0 msgr2=0x7f7ad80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.238+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 shutdown_connections 2024-03-22T10:45:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.238+0000 7f7add400700 1 -- 172.21.15.115:0/3811292518 wait complete. 2024-03-22T10:45:58.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.389+0000 7f5032800700 1 -- 172.21.15.115:0/580126641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c103580 msgr2=0x7f502c1039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:58.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.389+0000 7f5032800700 1 --2- 172.21.15.115:0/580126641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c103580 0x7f502c1039d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5014009910 tx=0x7f5014009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:58.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.389+0000 7f5032800700 1 -- 172.21.15.115:0/580126641 shutdown_connections 2024-03-22T10:45:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.389+0000 7f5032800700 1 --2- 172.21.15.115:0/580126641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c103580 0x7f502c1039d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.389+0000 7f5032800700 1 --2- 172.21.15.115:0/580126641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c102380 0x7f502c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.389+0000 7f5032800700 1 -- 172.21.15.115:0/580126641 >> 172.21.15.115:0/580126641 conn(0x7f502c0fd930 msgr2=0x7f502c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.389+0000 7f5032800700 1 -- 172.21.15.115:0/580126641 shutdown_connections 2024-03-22T10:45:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.389+0000 7f5032800700 1 -- 172.21.15.115:0/580126641 wait complete. 2024-03-22T10:45:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.391+0000 7f5032800700 1 Processor -- start 2024-03-22T10:45:58.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.391+0000 7f5032800700 1 -- start start 2024-03-22T10:45:58.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.391+0000 7f5032800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c102380 0x7f502c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.391+0000 7f5032800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c103580 0x7f502c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.391+0000 7f5032800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1987c0 con 0x7f502c102380 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.391+0000 7f5032800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c19d1d0 con 0x7f502c103580 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.392+0000 7f502b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c102380 0x7f502c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.392+0000 7f502b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c102380 0x7f502c197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48770/0 (socket says 172.21.15.115:48770) 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.392+0000 7f502b400700 1 -- 172.21.15.115:0/3270659139 learned_addr learned my addr 172.21.15.115:0/3270659139 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.392+0000 7f5023e00700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c103580 0x7f502c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.392+0000 7f502b400700 1 -- 172.21.15.115:0/3270659139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c103580 msgr2=0x7f502c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.392+0000 7f502b400700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c103580 0x7f502c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.392+0000 7f502b400700 1 -- 172.21.15.115:0/3270659139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140095f0 con 0x7f502c102380 2024-03-22T10:45:58.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.393+0000 7f502b400700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c102380 0x7f502c197c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f501c00b480 tx=0x7f501c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:58.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.393+0000 7f5028c00700 1 -- 172.21.15.115:0/3270659139 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f501c0118c0 con 0x7f502c102380 2024-03-22T10:45:58.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.393+0000 7f5028c00700 1 -- 172.21.15.115:0/3270659139 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f501c010380 con 0x7f502c102380 2024-03-22T10:45:58.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.393+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c19d3d0 con 0x7f502c102380 2024-03-22T10:45:58.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.393+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c19d970 con 0x7f502c102380 2024-03-22T10:45:58.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.393+0000 7f5028c00700 1 -- 172.21.15.115:0/3270659139 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f501c00f4f0 con 0x7f502c102380 2024-03-22T10:45:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.394+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c10af70 con 0x7f502c102380 2024-03-22T10:45:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.396+0000 7f5028c00700 1 -- 172.21.15.115:0/3270659139 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f501c0104f0 con 0x7f502c102380 2024-03-22T10:45:58.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.397+0000 7f5028c00700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f501006dfa0 0x7f5010070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:58.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.397+0000 7f5028c00700 1 -- 172.21.15.115:0/3270659139 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f501c08d9e0 con 0x7f502c102380 2024-03-22T10:45:58.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.398+0000 7f5023e00700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f501006dfa0 0x7f5010070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:58.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.398+0000 7f5023e00700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f501006dfa0 0x7f5010070450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5014006010 tx=0x7f501400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:58.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.404+0000 7f5028c00700 1 -- 172.21.15.115:0/3270659139 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f501c056e10 con 0x7f502c102380 2024-03-22T10:45:58.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.730+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f502c10b160 con 0x7f502c102380 2024-03-22T10:45:58.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.731+0000 7f5028c00700 1 -- 172.21.15.115:0/3270659139 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f501c05a4d0 con 0x7f502c102380 2024-03-22T10:45:58.722 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:45:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:45:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:45:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:45:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:45:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:45:58.723 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:45:58.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f501006dfa0 msgr2=0x7f5010070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:58.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f501006dfa0 0x7f5010070450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5014006010 tx=0x7f501400b540 comp rx=0 tx=0).stop 2024-03-22T10:45:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c102380 msgr2=0x7f502c197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c102380 0x7f502c197c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f501c00b480 tx=0x7f501c00b840 comp rx=0 tx=0).stop 2024-03-22T10:45:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 shutdown_connections 2024-03-22T10:45:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f501006dfa0 0x7f5010070450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f502c102380 0x7f502c197c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 --2- 172.21.15.115:0/3270659139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c103580 0x7f502c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 >> 172.21.15.115:0/3270659139 conn(0x7f502c0fd930 msgr2=0x7f502c1067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 shutdown_connections 2024-03-22T10:45:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.734+0000 7f5032800700 1 -- 172.21.15.115:0/3270659139 wait complete. 2024-03-22T10:45:58.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.887+0000 7fe277a00700 1 -- 172.21.15.115:0/1638284602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 msgr2=0x7fe270102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:58.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.887+0000 7fe277a00700 1 --2- 172.21.15.115:0/1638284602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 0x7fe270102fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe264009910 tx=0x7fe264009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:58.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.887+0000 7fe277a00700 1 -- 172.21.15.115:0/1638284602 shutdown_connections 2024-03-22T10:45:58.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.887+0000 7fe277a00700 1 --2- 172.21.15.115:0/1638284602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe270103500 0x7fe2701058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.887+0000 7fe277a00700 1 --2- 172.21.15.115:0/1638284602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 0x7fe270102fc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.887+0000 7fe277a00700 1 -- 172.21.15.115:0/1638284602 >> 172.21.15.115:0/1638284602 conn(0x7fe2700fa5e0 msgr2=0x7fe2700fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:58.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.887+0000 7fe277a00700 1 -- 172.21.15.115:0/1638284602 shutdown_connections 2024-03-22T10:45:58.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.887+0000 7fe277a00700 1 -- 172.21.15.115:0/1638284602 wait complete. 2024-03-22T10:45:58.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.888+0000 7fe277a00700 1 Processor -- start 2024-03-22T10:45:58.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.889+0000 7fe277a00700 1 -- start start 2024-03-22T10:45:58.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.889+0000 7fe277a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 0x7fe270197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:58.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.889+0000 7fe277a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe270103500 0x7fe270198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:58.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.889+0000 7fe277a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe270198660 con 0x7fe270100be0 2024-03-22T10:45:58.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.889+0000 7fe277a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2701987a0 con 0x7fe270103500 2024-03-22T10:45:58.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.889+0000 7fe275000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 0x7fe270197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:58.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.889+0000 7fe275000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 0x7fe270197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48786/0 (socket says 172.21.15.115:48786) 2024-03-22T10:45:58.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.889+0000 7fe275000700 1 -- 172.21.15.115:0/2737087776 learned_addr learned my addr 172.21.15.115:0/2737087776 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:58.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.890+0000 7fe26fe00700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe270103500 0x7fe270198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:58.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.890+0000 7fe275000700 1 -- 172.21.15.115:0/2737087776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe270103500 msgr2=0x7fe270198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:58.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.890+0000 7fe275000700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe270103500 0x7fe270198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:58.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.890+0000 7fe275000700 1 -- 172.21.15.115:0/2737087776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640095f0 con 0x7fe270100be0 2024-03-22T10:45:58.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.890+0000 7fe275000700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 0x7fe270197b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe26400b5c0 tx=0x7fe26400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:58.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.890+0000 7fe26d600700 1 -- 172.21.15.115:0/2737087776 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe26401d070 con 0x7fe270100be0 2024-03-22T10:45:58.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.890+0000 7fe26d600700 1 -- 172.21.15.115:0/2737087776 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe26400f460 con 0x7fe270100be0 2024-03-22T10:45:58.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.890+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27019d1f0 con 0x7fe270100be0 2024-03-22T10:45:58.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.891+0000 7fe26d600700 1 -- 172.21.15.115:0/2737087776 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe264021640 con 0x7fe270100be0 2024-03-22T10:45:58.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.891+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27019d6e0 con 0x7fe270100be0 2024-03-22T10:45:58.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.892+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2700fc1c0 con 0x7fe270100be0 2024-03-22T10:45:58.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.893+0000 7fe26d600700 1 -- 172.21.15.115:0/2737087776 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe26400f5d0 con 0x7fe270100be0 2024-03-22T10:45:58.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.893+0000 7fe26d600700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe25806dfa0 0x7fe258070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:58.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.894+0000 7fe26d600700 1 -- 172.21.15.115:0/2737087776 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe26408f150 con 0x7fe270100be0 2024-03-22T10:45:58.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.894+0000 7fe26fe00700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe25806dfa0 0x7fe258070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:58.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.895+0000 7fe26fe00700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe25806dfa0 0x7fe258070450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe25c005950 tx=0x7fe25c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:58.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:58.899+0000 7fe26d600700 1 -- 172.21.15.115:0/2737087776 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe26405c150 con 0x7fe270100be0 2024-03-22T10:45:59.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi115"}]: dispatch 2024-03-22T10:45:59.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi115"}]': finished 2024-03-22T10:45:59.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi150"}]: dispatch 2024-03-22T10:45:59.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi150"}]': finished 2024-03-22T10:45:59.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T10:45:59.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:58 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3270659139' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:45:59.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi115"}]: dispatch 2024-03-22T10:45:59.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi115"}]': finished 2024-03-22T10:45:59.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi150"}]: dispatch 2024-03-22T10:45:59.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi150"}]': finished 2024-03-22T10:45:59.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T10:45:59.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:58 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3270659139' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:45:59.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.182+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe270061960 con 0x7fe270100be0 2024-03-22T10:45:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.182+0000 7fe26d600700 1 -- 172.21.15.115:0/2737087776 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1926 (secure 0 0 0) 0x7fe26405bce0 con 0x7fe270100be0 2024-03-22T10:45:59.174 INFO:teuthology.orchestra.run.smithi115.stdout:e14 2024-03-22T10:45:59.174 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:45:59.174 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:45:59.174 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:45:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:45:59.174 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:45:59.174 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:45:59.174 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 14 2024-03-22T10:45:59.174 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:43:22.786262+0000 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:45:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 0 members: 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:45:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:45:59.177 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:45:59.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.185+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe25806dfa0 msgr2=0x7fe258070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:59.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.185+0000 7fe277a00700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe25806dfa0 0x7fe258070450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe25c005950 tx=0x7fe25c0058e0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.185+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 msgr2=0x7fe270197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:59.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.185+0000 7fe277a00700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 0x7fe270197b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe26400b5c0 tx=0x7fe26400be20 comp rx=0 tx=0).stop 2024-03-22T10:45:59.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.185+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 shutdown_connections 2024-03-22T10:45:59.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.185+0000 7fe277a00700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe25806dfa0 0x7fe258070450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.186+0000 7fe277a00700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe270100be0 0x7fe270197b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.186+0000 7fe277a00700 1 --2- 172.21.15.115:0/2737087776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe270103500 0x7fe270198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.186+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 >> 172.21.15.115:0/2737087776 conn(0x7fe2700fa5e0 msgr2=0x7fe2700ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:59.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.186+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 shutdown_connections 2024-03-22T10:45:59.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.186+0000 7fe277a00700 1 -- 172.21.15.115:0/2737087776 wait complete. 2024-03-22T10:45:59.178 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 14 2024-03-22T10:45:59.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.346+0000 7f653f400700 1 -- 172.21.15.115:0/3647688519 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65280054b0 con 0x7f65400ff320 2024-03-22T10:45:59.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.347+0000 7f6547000700 1 -- 172.21.15.115:0/3647688519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 msgr2=0x7f65400ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:59.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.347+0000 7f6547000700 1 --2- 172.21.15.115:0/3647688519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 0x7f65400ff790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6528009910 tx=0x7f6528009c20 comp rx=0 tx=0).stop 2024-03-22T10:45:59.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.347+0000 7f6547000700 1 -- 172.21.15.115:0/3647688519 shutdown_connections 2024-03-22T10:45:59.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.347+0000 7f6547000700 1 --2- 172.21.15.115:0/3647688519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 0x7f65400ff790 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.347+0000 7f6547000700 1 --2- 172.21.15.115:0/3647688519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65400fe9d0 0x7f65400fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.347+0000 7f6547000700 1 -- 172.21.15.115:0/3647688519 >> 172.21.15.115:0/3647688519 conn(0x7f65400fa5e0 msgr2=0x7f65400fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:59.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.347+0000 7f6547000700 1 -- 172.21.15.115:0/3647688519 shutdown_connections 2024-03-22T10:45:59.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.347+0000 7f6547000700 1 -- 172.21.15.115:0/3647688519 wait complete. 2024-03-22T10:45:59.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.348+0000 7f6547000700 1 Processor -- start 2024-03-22T10:45:59.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.348+0000 7f6547000700 1 -- start start 2024-03-22T10:45:59.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.348+0000 7f6547000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65400fe9d0 0x7f6540197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:59.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f6547000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 0x7f6540198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:59.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f6547000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6540198600 con 0x7f65400ff320 2024-03-22T10:45:59.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f6547000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6540198740 con 0x7f65400fe9d0 2024-03-22T10:45:59.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f6537400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 0x7f6540198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:59.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f6537400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 0x7f6540198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48804/0 (socket says 172.21.15.115:48804) 2024-03-22T10:45:59.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f6537400700 1 -- 172.21.15.115:0/2860531433 learned_addr learned my addr 172.21.15.115:0/2860531433 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:59.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f653fe00700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65400fe9d0 0x7f6540197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:59.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f6537400700 1 -- 172.21.15.115:0/2860531433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65400fe9d0 msgr2=0x7f6540197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:59.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f6537400700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65400fe9d0 0x7f6540197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.349+0000 7f6537400700 1 -- 172.21.15.115:0/2860531433 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65280095f0 con 0x7f65400ff320 2024-03-22T10:45:59.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.350+0000 7f6537400700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 0x7f6540198070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6528006010 tx=0x7f652800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:59.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.350+0000 7f653d600700 1 -- 172.21.15.115:0/2860531433 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f652801d070 con 0x7f65400ff320 2024-03-22T10:45:59.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.350+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654019d1a0 con 0x7f65400ff320 2024-03-22T10:45:59.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.350+0000 7f653d600700 1 -- 172.21.15.115:0/2860531433 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f652800f460 con 0x7f65400ff320 2024-03-22T10:45:59.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.350+0000 7f653d600700 1 -- 172.21.15.115:0/2860531433 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6528021640 con 0x7f65400ff320 2024-03-22T10:45:59.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.350+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654019d660 con 0x7f65400ff320 2024-03-22T10:45:59.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.351+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654010ae80 con 0x7f65400ff320 2024-03-22T10:45:59.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.352+0000 7f653d600700 1 -- 172.21.15.115:0/2860531433 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f652800fab0 con 0x7f65400ff320 2024-03-22T10:45:59.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.353+0000 7f653d600700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f652406dfa0 0x7f6524070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:59.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.353+0000 7f653d600700 1 -- 172.21.15.115:0/2860531433 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f652808f220 con 0x7f65400ff320 2024-03-22T10:45:59.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.353+0000 7f653fe00700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f652406dfa0 0x7f6524070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:59.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.354+0000 7f653fe00700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f652406dfa0 0x7f6524070450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f653000e270 tx=0x7f6530009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:59.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.358+0000 7f653d600700 1 -- 172.21.15.115:0/2860531433 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f652805c200 con 0x7f65400ff320 2024-03-22T10:45:59.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.622+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6540108720 con 0x7f652406dfa0 2024-03-22T10:45:59.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.624+0000 7f653d600700 1 -- 172.21.15.115:0/2860531433 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6540108720 con 0x7f652406dfa0 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "6/23 daemons upgraded", 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T10:45:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:45:59.616 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.626+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f652406dfa0 msgr2=0x7f6524070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.626+0000 7f6547000700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f652406dfa0 0x7f6524070450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f653000e270 tx=0x7f6530009040 comp rx=0 tx=0).stop 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.626+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 msgr2=0x7f6540198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.626+0000 7f6547000700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 0x7f6540198070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6528006010 tx=0x7f652800be20 comp rx=0 tx=0).stop 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.627+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 shutdown_connections 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.627+0000 7f6547000700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f652406dfa0 0x7f6524070450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.627+0000 7f6547000700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65400fe9d0 0x7f6540197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.627+0000 7f6547000700 1 --2- 172.21.15.115:0/2860531433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65400ff320 0x7f6540198070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.627+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 >> 172.21.15.115:0/2860531433 conn(0x7f65400fa5e0 msgr2=0x7f6540107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:59.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.627+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 shutdown_connections 2024-03-22T10:45:59.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.627+0000 7f6547000700 1 -- 172.21.15.115:0/2860531433 wait complete. 2024-03-22T10:45:59.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.788+0000 7f34a0c00700 1 -- 172.21.15.115:0/525418845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c0737b0 msgr2=0x7f349c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:59.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.788+0000 7f34a0c00700 1 --2- 172.21.15.115:0/525418845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c0737b0 0x7f349c073c20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3484009940 tx=0x7f3484009c50 comp rx=0 tx=0).stop 2024-03-22T10:45:59.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.788+0000 7f34a0c00700 1 -- 172.21.15.115:0/525418845 shutdown_connections 2024-03-22T10:45:59.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.788+0000 7f34a0c00700 1 --2- 172.21.15.115:0/525418845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c0737b0 0x7f349c073c20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.788+0000 7f34a0c00700 1 --2- 172.21.15.115:0/525418845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c074d80 0x7f349c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.788+0000 7f34a0c00700 1 -- 172.21.15.115:0/525418845 >> 172.21.15.115:0/525418845 conn(0x7f349c0fb6e0 msgr2=0x7f349c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:45:59.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.789+0000 7f34a0c00700 1 -- 172.21.15.115:0/525418845 shutdown_connections 2024-03-22T10:45:59.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.789+0000 7f34a0c00700 1 -- 172.21.15.115:0/525418845 wait complete. 2024-03-22T10:45:59.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.790+0000 7f34a0c00700 1 Processor -- start 2024-03-22T10:45:59.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.790+0000 7f34a0c00700 1 -- start start 2024-03-22T10:45:59.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.790+0000 7f34a0c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c0737b0 0x7f349c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:59.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.790+0000 7f34a0c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c074d80 0x7f349c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:59.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.790+0000 7f34a0c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c1986c0 con 0x7f349c074d80 2024-03-22T10:45:59.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.790+0000 7f34a0c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c198800 con 0x7f349c0737b0 2024-03-22T10:45:59.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.790+0000 7f3491400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c074d80 0x7f349c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:59.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.790+0000 7f3491400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c074d80 0x7f349c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48820/0 (socket says 172.21.15.115:48820) 2024-03-22T10:45:59.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.790+0000 7f3491400700 1 -- 172.21.15.115:0/2378368217 learned_addr learned my addr 172.21.15.115:0/2378368217 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.791+0000 7f3499e00700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c0737b0 0x7f349c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.791+0000 7f3491400700 1 -- 172.21.15.115:0/2378368217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c0737b0 msgr2=0x7f349c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.791+0000 7f3491400700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c0737b0 0x7f349c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.791+0000 7f3491400700 1 -- 172.21.15.115:0/2378368217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840095f0 con 0x7f349c074d80 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.791+0000 7f3499e00700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c0737b0 0x7f349c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.791+0000 7f3491400700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c074d80 0x7f349c1980a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34840055f0 tx=0x7f34840056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.791+0000 7f3493400700 1 -- 172.21.15.115:0/2378368217 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f348401d070 con 0x7f349c074d80 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.791+0000 7f3493400700 1 -- 172.21.15.115:0/2378368217 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3484003ba0 con 0x7f349c074d80 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.791+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349c19d250 con 0x7f349c074d80 2024-03-22T10:45:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.792+0000 7f3493400700 1 -- 172.21.15.115:0/2378368217 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3484021640 con 0x7f349c074d80 2024-03-22T10:45:59.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.792+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349c19d7c0 con 0x7f349c074d80 2024-03-22T10:45:59.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.793+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349c10ae90 con 0x7f349c074d80 2024-03-22T10:45:59.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.794+0000 7f3493400700 1 -- 172.21.15.115:0/2378368217 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f348400fa10 con 0x7f349c074d80 2024-03-22T10:45:59.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.794+0000 7f3493400700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f348006dfa0 0x7f3480070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:45:59.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.795+0000 7f3493400700 1 -- 172.21.15.115:0/2378368217 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f348408f200 con 0x7f349c074d80 2024-03-22T10:45:59.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.795+0000 7f3499e00700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f348006dfa0 0x7f3480070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:45:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.795+0000 7f3499e00700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f348006dfa0 0x7f3480070450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f348c0060b0 tx=0x7f348c0076a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:45:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:45:59.800+0000 7f3493400700 1 -- 172.21.15.115:0/2378368217 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f348405c160 con 0x7f349c074d80 2024-03-22T10:46:00.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:59 smithi115 ceph-mon[78672]: pgmap v60: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:00.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:59 smithi115 ceph-mon[78672]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:00.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:59 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all crash 2024-03-22T10:46:00.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:59 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T10:46:00.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:59 smithi115 ceph-mon[78672]: Upgrade: osd.0 is safe to restart 2024-03-22T10:46:00.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:45:59 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2737087776' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:46:00.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.101+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f349c19dec0 con 0x7f349c074d80 2024-03-22T10:46:00.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.102+0000 7f3493400700 1 -- 172.21.15.115:0/2378368217 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f348405bcf0 con 0x7f349c074d80 2024-03-22T10:46:00.093 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:46:00.093 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:46:00.093 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:46:00.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.105+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f348006dfa0 msgr2=0x7f3480070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:00.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.105+0000 7f34a0c00700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f348006dfa0 0x7f3480070450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f348c0060b0 tx=0x7f348c0076a0 comp rx=0 tx=0).stop 2024-03-22T10:46:00.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.105+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c074d80 msgr2=0x7f349c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:00.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.105+0000 7f34a0c00700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c074d80 0x7f349c1980a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34840055f0 tx=0x7f34840056d0 comp rx=0 tx=0).stop 2024-03-22T10:46:00.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.105+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 shutdown_connections 2024-03-22T10:46:00.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.105+0000 7f34a0c00700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f348006dfa0 0x7f3480070450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:00.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.105+0000 7f34a0c00700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c0737b0 0x7f349c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:00.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.105+0000 7f34a0c00700 1 --2- 172.21.15.115:0/2378368217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f349c074d80 0x7f349c1980a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:00.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.105+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 >> 172.21.15.115:0/2378368217 conn(0x7f349c0fb6e0 msgr2=0x7f349c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:00.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.106+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 shutdown_connections 2024-03-22T10:46:00.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:00.106+0000 7f34a0c00700 1 -- 172.21.15.115:0/2378368217 wait complete. 2024-03-22T10:46:00.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:59 smithi150 ceph-mon[65513]: pgmap v60: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:00.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:59 smithi150 ceph-mon[65513]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:00.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:59 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all crash 2024-03-22T10:46:00.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:59 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T10:46:00.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:59 smithi150 ceph-mon[65513]: Upgrade: osd.0 is safe to restart 2024-03-22T10:46:00.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:45:59 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2737087776' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:46:01.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:00 smithi150 ceph-mon[65513]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:01.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:00 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2378368217' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:46:01.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:00 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:01.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:00 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T10:46:01.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:00 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:46:01.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:00 smithi115 ceph-mon[78672]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:01.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:00 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2378368217' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:46:01.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:00 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:01.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:00 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T10:46:01.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:00 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:46:02.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:01 smithi115 ceph-mon[78672]: pgmap v61: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:02.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:01 smithi115 ceph-mon[78672]: Upgrade: Updating osd.0 2024-03-22T10:46:02.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:01 smithi115 ceph-mon[78672]: Deploying daemon osd.0 on smithi115 2024-03-22T10:46:02.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:01 smithi150 ceph-mon[65513]: pgmap v61: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:02.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:01 smithi150 ceph-mon[65513]: Upgrade: Updating osd.0 2024-03-22T10:46:02.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:01 smithi150 ceph-mon[65513]: Deploying daemon osd.0 on smithi115 2024-03-22T10:46:04.067 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:03 smithi115 ceph-mon[78672]: pgmap v62: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:04.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:03 smithi150 ceph-mon[65513]: pgmap v62: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:05.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:46:05.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:46:05.510 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:05 smithi115 systemd[1]: Stopping Ceph osd.0 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:46:05.871 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:05 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[43926]: 2024-03-22T10:46:05.613+0000 7f7a1e400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T10:46:05.871 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:05 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[43926]: 2024-03-22T10:46:05.613+0000 7f7a1e400700 -1 osd.0 48 *** Got signal Terminated *** 2024-03-22T10:46:05.871 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:05 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[43926]: 2024-03-22T10:46:05.613+0000 7f7a1e400700 -1 osd.0 48 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T10:46:06.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:05 smithi150 ceph-mon[65513]: pgmap v63: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:06.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:05 smithi150 ceph-mon[65513]: osd.0 marked itself down and dead 2024-03-22T10:46:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:05 smithi115 ceph-mon[78672]: pgmap v63: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:05 smithi115 ceph-mon[78672]: osd.0 marked itself down and dead 2024-03-22T10:46:06.629 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:06 smithi115 podman[84871]: 2024-03-22 10:46:06.353427511 +0000 UTC m=+0.857505235 container died 3dff5085579f678b4e14a2becfa7dceb73827608af5ebab8bf6611f0fffc166e (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=HEAD, ceph=True, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-22T10:46:06.889 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:06 smithi115 podman[84871]: 2024-03-22 10:46:06.757126719 +0000 UTC m=+1.261204447 container cleanup 3dff5085579f678b4e14a2becfa7dceb73827608af5ebab8bf6611f0fffc166e (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, RELEASE=HEAD, org.label-schema.build-date=20231212) 2024-03-22T10:46:06.889 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:06 smithi115 bash[84871]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0 2024-03-22T10:46:07.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:06 smithi150 ceph-mon[65513]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:46:07.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:06 smithi150 ceph-mon[65513]: osdmap e49: 6 total, 5 up, 6 in 2024-03-22T10:46:07.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:06 smithi150 ceph-mon[65513]: osdmap e50: 6 total, 5 up, 6 in 2024-03-22T10:46:07.261 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:07 smithi115 podman[84890]: 2024-03-22 10:46:07.000741382 +0000 UTC m=+0.631644197 container remove 3dff5085579f678b4e14a2becfa7dceb73827608af5ebab8bf6611f0fffc166e (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T10:46:07.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:06 smithi115 ceph-mon[78672]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:46:07.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:06 smithi115 ceph-mon[78672]: osdmap e49: 6 total, 5 up, 6 in 2024-03-22T10:46:07.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:06 smithi115 ceph-mon[78672]: osdmap e50: 6 total, 5 up, 6 in 2024-03-22T10:46:08.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:07 smithi150 ceph-mon[65513]: pgmap v65: 65 pgs: 9 stale+active+clean, 56 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:08.157 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:07 smithi115 ceph-mon[78672]: pgmap v65: 65 pgs: 9 stale+active+clean, 56 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:08.157 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:07 smithi115 podman[84977]: 2024-03-22 10:46:07.747326098 +0000 UTC m=+0.031970251 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:46:08.157 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:07 smithi115 podman[84977]: 2024-03-22 10:46:07.859731834 +0000 UTC m=+0.144376023 container create 198f8e81426af7ffebff1e802d587110a04ba1e78690431da9db6b4339a03771 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-22T10:46:08.511 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:08 smithi115 podman[84977]: 2024-03-22 10:46:08.165620268 +0000 UTC m=+0.450264511 container init 198f8e81426af7ffebff1e802d587110a04ba1e78690431da9db6b4339a03771 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-22T10:46:08.511 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:08 smithi115 podman[84977]: 2024-03-22 10:46:08.175157235 +0000 UTC m=+0.459801451 container start 198f8e81426af7ffebff1e802d587110a04ba1e78690431da9db6b4339a03771 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=) 2024-03-22T10:46:08.511 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:08 smithi115 podman[84977]: 2024-03-22 10:46:08.262376543 +0000 UTC m=+0.547020726 container attach 198f8e81426af7ffebff1e802d587110a04ba1e78690431da9db6b4339a03771 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-22T10:46:08.947 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:08 smithi115 podman[84977]: 2024-03-22 10:46:08.610910243 +0000 UTC m=+0.895554459 container died 198f8e81426af7ffebff1e802d587110a04ba1e78690431da9db6b4339a03771 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-22T10:46:09.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:08 smithi115 ceph-mon[78672]: pgmap v67: 65 pgs: 6 peering, 51 active+clean, 8 stale+active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:09.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:08 smithi115 ceph-mon[78672]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T10:46:09.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:08 smithi150 ceph-mon[65513]: pgmap v67: 65 pgs: 6 peering, 51 active+clean, 8 stale+active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:09.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:08 smithi150 ceph-mon[65513]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T10:46:09.675 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:09 smithi115 podman[85003]: 2024-03-22 10:46:09.295423921 +0000 UTC m=+0.665559908 container remove 198f8e81426af7ffebff1e802d587110a04ba1e78690431da9db6b4339a03771 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5) 2024-03-22T10:46:09.675 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:09 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-22T10:46:09.675 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:09 smithi115 systemd[1]: Stopped Ceph osd.0 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:46:09.675 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:09 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.0.service: Consumed 1min 30.343s CPU time. 2024-03-22T10:46:10.011 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:09 smithi115 systemd[1]: Starting Ceph osd.0 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:46:10.378 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:10 smithi115 ceph-mon[78672]: Health check failed: Degraded data redundancy: 56/369 objects degraded (15.176%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T10:46:10.379 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:10 smithi115 podman[85103]: 2024-03-22 10:46:09.961348323 +0000 UTC m=+0.032039403 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:46:10.379 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:10 smithi115 podman[85103]: 2024-03-22 10:46:10.09240922 +0000 UTC m=+0.163100337 container create ec0dc4cc56fd1e26cb7872aca7c3280e8ce52533dafe2731ecff15343a06fa9a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-22T10:46:10.379 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:10 smithi115 podman[85103]: 2024-03-22 10:46:10.386992481 +0000 UTC m=+0.457683598 container init ec0dc4cc56fd1e26cb7872aca7c3280e8ce52533dafe2731ecff15343a06fa9a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-03-22T10:46:10.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:10 smithi150 ceph-mon[65513]: Health check failed: Degraded data redundancy: 56/369 objects degraded (15.176%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T10:46:10.761 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:10 smithi115 podman[85103]: 2024-03-22 10:46:10.401314344 +0000 UTC m=+0.472005468 container start ec0dc4cc56fd1e26cb7872aca7c3280e8ce52533dafe2731ecff15343a06fa9a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-22T10:46:10.761 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:10 smithi115 podman[85103]: 2024-03-22 10:46:10.611798014 +0000 UTC m=+0.682489114 container attach ec0dc4cc56fd1e26cb7872aca7c3280e8ce52533dafe2731ecff15343a06fa9a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-22T10:46:11.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:11 smithi115 ceph-mon[78672]: pgmap v68: 65 pgs: 16 active+undersized+degraded, 6 peering, 12 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:46:11.511 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate[85117]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T10:46:11.511 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate[85117]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 bash[85103]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 bash[85103]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 bash[85103]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 bash[85103]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 bash[85103]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 bash[85103]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 bash[85103]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate[85117]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate[85117]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate[85117]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate[85117]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate[85117]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-22T10:46:11.512 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 podman[85103]: 2024-03-22 10:46:11.303079548 +0000 UTC m=+1.373770652 container died ec0dc4cc56fd1e26cb7872aca7c3280e8ce52533dafe2731ecff15343a06fa9a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-22T10:46:11.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:11 smithi150 ceph-mon[65513]: pgmap v68: 65 pgs: 16 active+undersized+degraded, 6 peering, 12 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:46:12.260 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:11 smithi115 podman[85257]: 2024-03-22 10:46:11.957464069 +0000 UTC m=+0.636300749 container remove ec0dc4cc56fd1e26cb7872aca7c3280e8ce52533dafe2731ecff15343a06fa9a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-activate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-22T10:46:12.569 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:12 smithi115 ceph-mon[78672]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T10:46:12.570 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:12 smithi115 podman[85322]: 2024-03-22 10:46:12.207887842 +0000 UTC m=+0.031564611 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:46:12.570 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:12 smithi115 podman[85322]: 2024-03-22 10:46:12.341897657 +0000 UTC m=+0.165574497 container create f8691567d116ed892bef35b3e110448b5d80597ec13e98fbde7cc2a541306e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-22T10:46:12.570 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:12 smithi115 podman[85322]: 2024-03-22 10:46:12.578707043 +0000 UTC m=+0.402383844 container init f8691567d116ed892bef35b3e110448b5d80597ec13e98fbde7cc2a541306e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-22T10:46:12.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:12 smithi150 ceph-mon[65513]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T10:46:13.010 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:12 smithi115 podman[85322]: 2024-03-22 10:46:12.590054601 +0000 UTC m=+0.413731365 container start f8691567d116ed892bef35b3e110448b5d80597ec13e98fbde7cc2a541306e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-03-22T10:46:13.010 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:12 smithi115 ceph-osd[85341]: -- 172.21.15.115:0/117955605 <== mon.0 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55908ef95680 con 0x55908ef5ec00 2024-03-22T10:46:13.010 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:12 smithi115 bash[85322]: f8691567d116ed892bef35b3e110448b5d80597ec13e98fbde7cc2a541306e59 2024-03-22T10:46:13.010 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:12 smithi115 systemd[1]: Started Ceph osd.0 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:46:13.497 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:13 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[85337]: 2024-03-22T10:46:13.157+0000 7f86660567c0 -1 Falling back to public interface 2024-03-22T10:46:14.110 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:13 smithi115 ceph-mon[78672]: pgmap v69: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:14.111 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:14.111 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:14.111 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:46:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:13 smithi150 ceph-mon[65513]: pgmap v69: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:46:16.076 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:15 smithi115 ceph-mon[78672]: pgmap v70: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:16.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:15 smithi150 ceph-mon[65513]: pgmap v70: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:18.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:17 smithi150 ceph-mon[65513]: pgmap v71: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:18.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:17 smithi115 ceph-mon[78672]: pgmap v71: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:18.261 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:18 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[85337]: 2024-03-22T10:46:18.000+0000 7f86660567c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-22T10:46:18.743 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:18 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[85337]: 2024-03-22T10:46:18.344+0000 7f86660567c0 -1 osd.0 48 log_to_monitors true 2024-03-22T10:46:19.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:18 smithi115 ceph-mon[78672]: from='osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T10:46:19.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:18 smithi150 ceph-mon[65513]: from='osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T10:46:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:19 smithi150 ceph-mon[65513]: pgmap v72: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:19 smithi150 ceph-mon[65513]: from='osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T10:46:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:19 smithi150 ceph-mon[65513]: osdmap e51: 6 total, 5 up, 6 in 2024-03-22T10:46:20.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:19 smithi150 ceph-mon[65513]: from='osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:46:20.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:20.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:46:20.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:19 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 72/369 objects degraded (19.512%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T10:46:20.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:20.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:20.260 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[85337]: 2024-03-22T10:46:19.941+0000 7f8650400700 -1 osd.0 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T10:46:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-mon[78672]: pgmap v72: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-mon[78672]: from='osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T10:46:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-mon[78672]: osdmap e51: 6 total, 5 up, 6 in 2024-03-22T10:46:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-mon[78672]: from='osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:46:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:46:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 72/369 objects degraded (19.512%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T10:46:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:21.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:20 smithi115 ceph-mon[78672]: from='osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745]' entity='osd.0' 2024-03-22T10:46:21.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:20 smithi115 ceph-mon[78672]: pgmap v74: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:21.329 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:20 smithi150 ceph-mon[65513]: from='osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745]' entity='osd.0' 2024-03-22T10:46:21.329 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:20 smithi150 ceph-mon[65513]: pgmap v74: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:46:22.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:21 smithi115 ceph-mon[78672]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:46:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:21 smithi115 ceph-mon[78672]: osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745] boot 2024-03-22T10:46:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:21 smithi115 ceph-mon[78672]: osdmap e52: 6 total, 6 up, 6 in 2024-03-22T10:46:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:46:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:21 smithi115 ceph-mon[78672]: osdmap e53: 6 total, 6 up, 6 in 2024-03-22T10:46:22.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:21 smithi150 ceph-mon[65513]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:46:22.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:21 smithi150 ceph-mon[65513]: osd.0 [v2:172.21.15.115:6802/2453947745,v1:172.21.15.115:6803/2453947745] boot 2024-03-22T10:46:22.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:21 smithi150 ceph-mon[65513]: osdmap e52: 6 total, 6 up, 6 in 2024-03-22T10:46:22.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T10:46:22.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:22.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:22.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:21 smithi150 ceph-mon[65513]: osdmap e53: 6 total, 6 up, 6 in 2024-03-22T10:46:23.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:22 smithi115 ceph-mon[78672]: pgmap v77: 65 pgs: 5 peering, 16 active+undersized+degraded, 13 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:46:23.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:22 smithi150 ceph-mon[65513]: pgmap v77: 65 pgs: 5 peering, 16 active+undersized+degraded, 13 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:46:25.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:25 smithi115 ceph-mon[78672]: pgmap v78: 65 pgs: 5 peering, 13 active+undersized+degraded, 7 active+undersized, 40 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43/369 objects degraded (11.653%) 2024-03-22T10:46:25.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:25 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 43/369 objects degraded (11.653%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T10:46:25.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:25 smithi150 ceph-mon[65513]: pgmap v78: 65 pgs: 5 peering, 13 active+undersized+degraded, 7 active+undersized, 40 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43/369 objects degraded (11.653%) 2024-03-22T10:46:25.549 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:25 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 43/369 objects degraded (11.653%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T10:46:26.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:26 smithi115 ceph-mon[78672]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/369 objects degraded (11.653%), 13 pgs degraded) 2024-03-22T10:46:26.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:26.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:26.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:46:26.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:46:26.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:26.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:26 smithi150 ceph-mon[65513]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/369 objects degraded (11.653%), 13 pgs degraded) 2024-03-22T10:46:26.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:26.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:26.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:46:26.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:46:26.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:27.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:27 smithi115 ceph-mon[78672]: pgmap v79: 65 pgs: 5 peering, 60 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:27.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:46:27.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:46:27.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:46:27.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:46:27.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T10:46:27.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:27 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T10:46:27.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:27 smithi115 ceph-mon[78672]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-22T10:46:27.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:27 smithi150 ceph-mon[65513]: pgmap v79: 65 pgs: 5 peering, 60 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:27.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:46:27.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:46:27.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:46:27.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:46:27.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T10:46:27.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:27 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T10:46:27.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:27 smithi150 ceph-mon[65513]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-22T10:46:29.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:29 smithi115 ceph-mon[78672]: pgmap v80: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:29.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:29 smithi150 ceph-mon[65513]: pgmap v80: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:30.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.275+0000 7f3651e00700 1 -- 172.21.15.115:0/676435795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1035a0 msgr2=0x7f364c1039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:30.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.275+0000 7f3651e00700 1 --2- 172.21.15.115:0/676435795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1035a0 0x7f364c1039f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3634009910 tx=0x7f3634009c20 comp rx=0 tx=0).stop 2024-03-22T10:46:30.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.275+0000 7f3651e00700 1 -- 172.21.15.115:0/676435795 shutdown_connections 2024-03-22T10:46:30.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.275+0000 7f3651e00700 1 --2- 172.21.15.115:0/676435795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1035a0 0x7f364c1039f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.275+0000 7f3651e00700 1 --2- 172.21.15.115:0/676435795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364c1023a0 0x7f364c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.275+0000 7f3651e00700 1 -- 172.21.15.115:0/676435795 >> 172.21.15.115:0/676435795 conn(0x7f364c0fd990 msgr2=0x7f364c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:30.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.275+0000 7f3651e00700 1 -- 172.21.15.115:0/676435795 shutdown_connections 2024-03-22T10:46:30.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.275+0000 7f3651e00700 1 -- 172.21.15.115:0/676435795 wait complete. 2024-03-22T10:46:30.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.276+0000 7f3651e00700 1 Processor -- start 2024-03-22T10:46:30.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.277+0000 7f3651e00700 1 -- start start 2024-03-22T10:46:30.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.277+0000 7f3651e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1023a0 0x7f364c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:30.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.277+0000 7f3651e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364c1035a0 0x7f364c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:30.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.277+0000 7f3651e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c198690 con 0x7f364c1023a0 2024-03-22T10:46:30.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.277+0000 7f3651e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c1987d0 con 0x7f364c1035a0 2024-03-22T10:46:30.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.277+0000 7f364aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1023a0 0x7f364c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:30.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.277+0000 7f364aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1023a0 0x7f364c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51910/0 (socket says 172.21.15.115:51910) 2024-03-22T10:46:30.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.277+0000 7f364aa00700 1 -- 172.21.15.115:0/529900048 learned_addr learned my addr 172.21.15.115:0/529900048 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f3642000700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364c1035a0 0x7f364c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f364aa00700 1 -- 172.21.15.115:0/529900048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364c1035a0 msgr2=0x7f364c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f364aa00700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364c1035a0 0x7f364c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f364aa00700 1 -- 172.21.15.115:0/529900048 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340095f0 con 0x7f364c1023a0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f364aa00700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1023a0 0x7f364c197b30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f363c00ada0 tx=0x7f363c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f3650a00700 1 -- 172.21.15.115:0/529900048 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f363c00f070 con 0x7f364c1023a0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f3650a00700 1 -- 172.21.15.115:0/529900048 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f363c007be0 con 0x7f364c1023a0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c19d280 con 0x7f364c1023a0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f3650a00700 1 -- 172.21.15.115:0/529900048 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f363c0083f0 con 0x7f364c1023a0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.278+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c19d7d0 con 0x7f364c1023a0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.280+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c10aef0 con 0x7f364c1023a0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.280+0000 7f3650a00700 1 -- 172.21.15.115:0/529900048 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f363c00cad0 con 0x7f364c1023a0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.281+0000 7f3650a00700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f363006dfa0 0x7f3630070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.281+0000 7f3650a00700 1 -- 172.21.15.115:0/529900048 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f363c08d650 con 0x7f364c1023a0 2024-03-22T10:46:30.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.282+0000 7f3642000700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f363006dfa0 0x7f3630070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.282+0000 7f3642000700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f363006dfa0 0x7f3630070450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3634006010 tx=0x7f363400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.287+0000 7f3650a00700 1 -- 172.21.15.115:0/529900048 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f363c05a6d0 con 0x7f364c1023a0 2024-03-22T10:46:30.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.535+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f364c107ef0 con 0x7f363006dfa0 2024-03-22T10:46:30.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.539+0000 7f3650a00700 1 -- 172.21.15.115:0/529900048 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f364c107ef0 con 0x7f363006dfa0 2024-03-22T10:46:30.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f363006dfa0 msgr2=0x7f3630070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:30.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f363006dfa0 0x7f3630070450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3634006010 tx=0x7f363400b540 comp rx=0 tx=0).stop 2024-03-22T10:46:30.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1023a0 msgr2=0x7f364c197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:30.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1023a0 0x7f364c197b30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f363c00ada0 tx=0x7f363c00c330 comp rx=0 tx=0).stop 2024-03-22T10:46:30.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 shutdown_connections 2024-03-22T10:46:30.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f363006dfa0 0x7f3630070450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c1023a0 0x7f364c197b30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 --2- 172.21.15.115:0/529900048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364c1035a0 0x7f364c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 >> 172.21.15.115:0/529900048 conn(0x7f364c0fd990 msgr2=0x7f364c1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:30.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.542+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 shutdown_connections 2024-03-22T10:46:30.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.543+0000 7f3651e00700 1 -- 172.21.15.115:0/529900048 wait complete. 2024-03-22T10:46:30.551 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:46:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.701+0000 7f30b6200700 1 -- 172.21.15.115:0/464510447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 msgr2=0x7f30b01017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.702+0000 7f30b6200700 1 --2- 172.21.15.115:0/464510447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 0x7f30b01017a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3098009910 tx=0x7f3098009c20 comp rx=0 tx=0).stop 2024-03-22T10:46:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.702+0000 7f30b6200700 1 -- 172.21.15.115:0/464510447 shutdown_connections 2024-03-22T10:46:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.702+0000 7f30b6200700 1 --2- 172.21.15.115:0/464510447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 0x7f30b01017a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.702+0000 7f30b6200700 1 --2- 172.21.15.115:0/464510447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30b0100150 0x7f30b0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.702+0000 7f30b6200700 1 -- 172.21.15.115:0/464510447 >> 172.21.15.115:0/464510447 conn(0x7f30b00fb6e0 msgr2=0x7f30b00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.702+0000 7f30b6200700 1 -- 172.21.15.115:0/464510447 shutdown_connections 2024-03-22T10:46:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.702+0000 7f30b6200700 1 -- 172.21.15.115:0/464510447 wait complete. 2024-03-22T10:46:30.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.703+0000 7f30b6200700 1 Processor -- start 2024-03-22T10:46:30.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.703+0000 7f30b6200700 1 -- start start 2024-03-22T10:46:30.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.703+0000 7f30b6200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30b0100150 0x7f30b0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:30.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.703+0000 7f30b6200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 0x7f30b0073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:30.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.703+0000 7f30b6200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b0073680 con 0x7f30b0101350 2024-03-22T10:46:30.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.703+0000 7f30b6200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00737c0 con 0x7f30b0100150 2024-03-22T10:46:30.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.704+0000 7f30a7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 0x7f30b0073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:30.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.704+0000 7f30a7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 0x7f30b0073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51936/0 (socket says 172.21.15.115:51936) 2024-03-22T10:46:30.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.704+0000 7f30a7e00700 1 -- 172.21.15.115:0/933644240 learned_addr learned my addr 172.21.15.115:0/933644240 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:46:30.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.704+0000 7f30a7e00700 1 -- 172.21.15.115:0/933644240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30b0100150 msgr2=0x7f30b0074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:30.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.704+0000 7f30af400700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30b0100150 0x7f30b0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:30.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.704+0000 7f30a7e00700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30b0100150 0x7f30b0074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.704+0000 7f30a7e00700 1 -- 172.21.15.115:0/933644240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980095f0 con 0x7f30b0101350 2024-03-22T10:46:30.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.704+0000 7f30a7e00700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 0x7f30b0073140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f309800bd70 tx=0x7f309800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:30.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.705+0000 7f30acc00700 1 -- 172.21.15.115:0/933644240 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f309801d070 con 0x7f30b0101350 2024-03-22T10:46:30.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.705+0000 7f30acc00700 1 -- 172.21.15.115:0/933644240 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3098022470 con 0x7f30b0101350 2024-03-22T10:46:30.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.705+0000 7f30acc00700 1 -- 172.21.15.115:0/933644240 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f309800f670 con 0x7f30b0101350 2024-03-22T10:46:30.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.705+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b0073a40 con 0x7f30b0101350 2024-03-22T10:46:30.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.705+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b0073f30 con 0x7f30b0101350 2024-03-22T10:46:30.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.706+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b0108ca0 con 0x7f30b0101350 2024-03-22T10:46:30.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.707+0000 7f30acc00700 1 -- 172.21.15.115:0/933644240 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f3098022a50 con 0x7f30b0101350 2024-03-22T10:46:30.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.708+0000 7f30acc00700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f309406dfa0 0x7f3094070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:30.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.708+0000 7f30acc00700 1 -- 172.21.15.115:0/933644240 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f309808f2d0 con 0x7f30b0101350 2024-03-22T10:46:30.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.708+0000 7f30af400700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f309406dfa0 0x7f3094070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:30.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.709+0000 7f30af400700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f309406dfa0 0x7f3094070450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30a0005950 tx=0x7f30a00058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:30.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.713+0000 7f30acc00700 1 -- 172.21.15.115:0/933644240 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f309805c360 con 0x7f30b0101350 2024-03-22T10:46:30.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.961+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30b0105ca0 con 0x7f309406dfa0 2024-03-22T10:46:30.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.963+0000 7f30acc00700 1 -- 172.21.15.115:0/933644240 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f30b0105ca0 con 0x7f309406dfa0 2024-03-22T10:46:30.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.966+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f309406dfa0 msgr2=0x7f3094070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.966+0000 7f30b6200700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f309406dfa0 0x7f3094070450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30a0005950 tx=0x7f30a00058e0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.966+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 msgr2=0x7f30b0073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.966+0000 7f30b6200700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 0x7f30b0073140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f309800bd70 tx=0x7f309800be50 comp rx=0 tx=0).stop 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.966+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 shutdown_connections 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.966+0000 7f30b6200700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f309406dfa0 0x7f3094070450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.966+0000 7f30b6200700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30b0100150 0x7f30b0074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.966+0000 7f30b6200700 1 --2- 172.21.15.115:0/933644240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b0101350 0x7f30b0073140 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.967+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 >> 172.21.15.115:0/933644240 conn(0x7f30b00fb6e0 msgr2=0x7f30b0104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.967+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 shutdown_connections 2024-03-22T10:46:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:30.967+0000 7f30b6200700 1 -- 172.21.15.115:0/933644240 wait complete. 2024-03-22T10:46:31.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.114+0000 7f8226000700 1 -- 172.21.15.115:0/417634295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201079a0 msgr2=0x7f8220107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.114+0000 7f8226000700 1 --2- 172.21.15.115:0/417634295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201079a0 0x7f8220107df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8208009910 tx=0x7f8208009c20 comp rx=0 tx=0).stop 2024-03-22T10:46:31.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.114+0000 7f8226000700 1 -- 172.21.15.115:0/417634295 shutdown_connections 2024-03-22T10:46:31.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.114+0000 7f8226000700 1 --2- 172.21.15.115:0/417634295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201079a0 0x7f8220107df0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.114+0000 7f8226000700 1 --2- 172.21.15.115:0/417634295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82201067a0 0x7f8220106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.114+0000 7f8226000700 1 -- 172.21.15.115:0/417634295 >> 172.21.15.115:0/417634295 conn(0x7f8220075910 msgr2=0x7f8220077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:31.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.114+0000 7f8226000700 1 -- 172.21.15.115:0/417634295 shutdown_connections 2024-03-22T10:46:31.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.114+0000 7f8226000700 1 -- 172.21.15.115:0/417634295 wait complete. 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.115+0000 7f8226000700 1 Processor -- start 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.116+0000 7f8226000700 1 -- start start 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.116+0000 7f8226000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201067a0 0x7f822019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.116+0000 7f8226000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82201079a0 0x7f822019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.116+0000 7f8226000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822019cb00 con 0x7f82201067a0 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.116+0000 7f8226000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822019cc40 con 0x7f82201079a0 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.116+0000 7f821f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201067a0 0x7f822019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.116+0000 7f821f400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201067a0 0x7f822019bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51950/0 (socket says 172.21.15.115:51950) 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.116+0000 7f821f400700 1 -- 172.21.15.115:0/2695401100 learned_addr learned my addr 172.21.15.115:0/2695401100 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f821f400700 1 -- 172.21.15.115:0/2695401100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82201079a0 msgr2=0x7f822019c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f821ea00700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82201079a0 0x7f822019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:31.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f821f400700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82201079a0 0x7f822019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f821f400700 1 -- 172.21.15.115:0/2695401100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080095f0 con 0x7f82201067a0 2024-03-22T10:46:31.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f821f400700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201067a0 0x7f822019bfa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f821000c8f0 tx=0x7f821000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:31.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f8217e00700 1 -- 172.21.15.115:0/2695401100 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8210007a70 con 0x7f82201067a0 2024-03-22T10:46:31.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f8217e00700 1 -- 172.21.15.115:0/2695401100 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8210007bd0 con 0x7f82201067a0 2024-03-22T10:46:31.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82201a16f0 con 0x7f82201067a0 2024-03-22T10:46:31.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f8217e00700 1 -- 172.21.15.115:0/2695401100 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8210018630 con 0x7f82201067a0 2024-03-22T10:46:31.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.117+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82201a1c40 con 0x7f82201067a0 2024-03-22T10:46:31.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.118+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822010f2f0 con 0x7f82201067a0 2024-03-22T10:46:31.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.119+0000 7f8217e00700 1 -- 172.21.15.115:0/2695401100 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f821000f450 con 0x7f82201067a0 2024-03-22T10:46:31.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.120+0000 7f8217e00700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f820406dfa0 0x7f8204070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:31.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.120+0000 7f8217e00700 1 -- 172.21.15.115:0/2695401100 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f821008dd30 con 0x7f82201067a0 2024-03-22T10:46:31.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.121+0000 7f821ea00700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f820406dfa0 0x7f8204070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:31.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.121+0000 7f821ea00700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f820406dfa0 0x7f8204070450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f820800b5c0 tx=0x7f8208005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:31.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.125+0000 7f8217e00700 1 -- 172.21.15.115:0/2695401100 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f821005adc0 con 0x7f82201067a0 2024-03-22T10:46:31.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.353+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f822010c2f0 con 0x7f820406dfa0 2024-03-22T10:46:31.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.365+0000 7f8217e00700 1 -- 172.21.15.115:0/2695401100 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f822010c2f0 con 0x7f820406dfa0 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (12m) 11s ago 14m 14.0M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (15m) 11s ago 15m 8112k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (13m) 42s ago 13m 7952k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (54s) 11s ago 15m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (47s) 42s ago 13m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (12m) 11s ago 14m 74.5M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (7m) 11s ago 7m 127M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (7m) 11s ago 7m 16.4M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:46:31.358 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (7m) 42s ago 7m 15.5M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (7m) 42s ago 7m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (4m) 11s ago 16m 504M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (3m) 42s ago 13m 449M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (2m) 11s ago 16m 44.1M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (116s) 42s ago 13m 33.6M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (15m) 11s ago 15m 8044k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (13m) 42s ago 13m 7721k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (19s) 11s ago 12m 11.1M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (11m) 11s ago 11m 208M 1217M 18.2.0 dc2bc1663786 7ad362092636 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (11m) 11s ago 11m 166M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (10m) 42s ago 10m 235M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (10m) 42s ago 10m 225M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:46:31.359 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (9m) 42s ago 9m 195M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:46:31.360 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (3m) 11s ago 14m 67.8M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:46:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f820406dfa0 msgr2=0x7f8204070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f820406dfa0 0x7f8204070450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f820800b5c0 tx=0x7f8208005960 comp rx=0 tx=0).stop 2024-03-22T10:46:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201067a0 msgr2=0x7f822019bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201067a0 0x7f822019bfa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f821000c8f0 tx=0x7f821000ccb0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 shutdown_connections 2024-03-22T10:46:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f820406dfa0 0x7f8204070450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82201067a0 0x7f822019bfa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 --2- 172.21.15.115:0/2695401100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82201079a0 0x7f822019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 >> 172.21.15.115:0/2695401100 conn(0x7f8220075910 msgr2=0x7f822010abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 shutdown_connections 2024-03-22T10:46:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.369+0000 7f8226000700 1 -- 172.21.15.115:0/2695401100 wait complete. 2024-03-22T10:46:31.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:31 smithi115 ceph-mon[78672]: pgmap v81: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:31.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:31 smithi115 ceph-mon[78672]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:31.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.520+0000 7f71af000700 1 -- 172.21.15.115:0/3362649863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 msgr2=0x7f71a81039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.520+0000 7f71af000700 1 --2- 172.21.15.115:0/3362649863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 0x7f71a81039f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7194009910 tx=0x7f7194009c20 comp rx=0 tx=0).stop 2024-03-22T10:46:31.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.520+0000 7f71af000700 1 -- 172.21.15.115:0/3362649863 shutdown_connections 2024-03-22T10:46:31.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.520+0000 7f71af000700 1 --2- 172.21.15.115:0/3362649863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 0x7f71a81039f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.520+0000 7f71af000700 1 --2- 172.21.15.115:0/3362649863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81023a0 0x7f71a81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.520+0000 7f71af000700 1 -- 172.21.15.115:0/3362649863 >> 172.21.15.115:0/3362649863 conn(0x7f71a80fd990 msgr2=0x7f71a80ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:31.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.520+0000 7f71af000700 1 -- 172.21.15.115:0/3362649863 shutdown_connections 2024-03-22T10:46:31.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.520+0000 7f71af000700 1 -- 172.21.15.115:0/3362649863 wait complete. 2024-03-22T10:46:31.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.521+0000 7f71af000700 1 Processor -- start 2024-03-22T10:46:31.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.521+0000 7f71af000700 1 -- start start 2024-03-22T10:46:31.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.522+0000 7f71af000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81023a0 0x7f71a8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:31.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.522+0000 7f71af000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 0x7f71a81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:31.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.522+0000 7f71af000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a8198700 con 0x7f71a81035a0 2024-03-22T10:46:31.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.522+0000 7f71af000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a8198840 con 0x7f71a81023a0 2024-03-22T10:46:31.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.522+0000 7f71a7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 0x7f71a81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:31.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.522+0000 7f71a7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 0x7f71a81980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51968/0 (socket says 172.21.15.115:51968) 2024-03-22T10:46:31.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.522+0000 7f71a7400700 1 -- 172.21.15.115:0/153575553 learned_addr learned my addr 172.21.15.115:0/153575553 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:46:31.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.522+0000 7f71a7e00700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81023a0 0x7f71a8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:31.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.522+0000 7f71a7400700 1 -- 172.21.15.115:0/153575553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81023a0 msgr2=0x7f71a8197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.523+0000 7f71a7400700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81023a0 0x7f71a8197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.523+0000 7f71a7400700 1 -- 172.21.15.115:0/153575553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71940095f0 con 0x7f71a81035a0 2024-03-22T10:46:31.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.523+0000 7f71a7400700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 0x7f71a81980e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7194005b40 tx=0x7f719400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:31.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.523+0000 7f71a4c00700 1 -- 172.21.15.115:0/153575553 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f719401d070 con 0x7f71a81035a0 2024-03-22T10:46:31.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.523+0000 7f71a4c00700 1 -- 172.21.15.115:0/153575553 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f719400f460 con 0x7f71a81035a0 2024-03-22T10:46:31.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.523+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a819d290 con 0x7f71a81035a0 2024-03-22T10:46:31.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.523+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a819d780 con 0x7f71a81035a0 2024-03-22T10:46:31.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.524+0000 7f71a4c00700 1 -- 172.21.15.115:0/153575553 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7194021640 con 0x7f71a81035a0 2024-03-22T10:46:31.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.524+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7188005320 con 0x7f71a81035a0 2024-03-22T10:46:31.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.525+0000 7f71a4c00700 1 -- 172.21.15.115:0/153575553 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f7194003a60 con 0x7f71a81035a0 2024-03-22T10:46:31.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.526+0000 7f71a4c00700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718c06dfa0 0x7f718c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:31.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.526+0000 7f71a4c00700 1 -- 172.21.15.115:0/153575553 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f719408efe0 con 0x7f71a81035a0 2024-03-22T10:46:31.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.527+0000 7f71a7e00700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718c06dfa0 0x7f718c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:31.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.527+0000 7f71a7e00700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718c06dfa0 0x7f718c070450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7198005950 tx=0x7f71980058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:31.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.531+0000 7f71a4c00700 1 -- 172.21.15.115:0/153575553 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f719405bfe0 con 0x7f71a81035a0 2024-03-22T10:46:31.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:31 smithi150 ceph-mon[65513]: pgmap v81: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:31.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:31 smithi150 ceph-mon[65513]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.828+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7188006200 con 0x7f71a81035a0 2024-03-22T10:46:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.829+0000 7f71a4c00700 1 -- 172.21.15.115:0/153575553 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f719405bb70 con 0x7f71a81035a0 2024-03-22T10:46:31.821 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:46:31.821 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:46:31.821 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 9, 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 5 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:46:31.822 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:46:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.832+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718c06dfa0 msgr2=0x7f718c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.832+0000 7f71af000700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718c06dfa0 0x7f718c070450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7198005950 tx=0x7f71980058e0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.832+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 msgr2=0x7f71a81980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.832+0000 7f71af000700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 0x7f71a81980e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7194005b40 tx=0x7f719400be20 comp rx=0 tx=0).stop 2024-03-22T10:46:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.832+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 shutdown_connections 2024-03-22T10:46:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.832+0000 7f71af000700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718c06dfa0 0x7f718c070450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.832+0000 7f71af000700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81023a0 0x7f71a8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.832+0000 7f71af000700 1 --2- 172.21.15.115:0/153575553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71a81035a0 0x7f71a81980e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.832+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 >> 172.21.15.115:0/153575553 conn(0x7f71a80fd990 msgr2=0x7f71a81067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:31.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.833+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 shutdown_connections 2024-03-22T10:46:31.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.833+0000 7f71af000700 1 -- 172.21.15.115:0/153575553 wait complete. 2024-03-22T10:46:31.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.976+0000 7fafcee00700 1 -- 172.21.15.115:0/3281965226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 msgr2=0x7fafc8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.976+0000 7fafcee00700 1 --2- 172.21.15.115:0/3281965226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 0x7fafc8106bb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fafb800b0a0 tx=0x7fafb800b3b0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.976+0000 7fafcee00700 1 -- 172.21.15.115:0/3281965226 shutdown_connections 2024-03-22T10:46:31.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.976+0000 7fafcee00700 1 --2- 172.21.15.115:0/3281965226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc81079a0 0x7fafc8107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.976+0000 7fafcee00700 1 --2- 172.21.15.115:0/3281965226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 0x7fafc8106bb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.976+0000 7fafcee00700 1 -- 172.21.15.115:0/3281965226 >> 172.21.15.115:0/3281965226 conn(0x7fafc8075910 msgr2=0x7fafc8077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:31.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.977+0000 7fafcee00700 1 -- 172.21.15.115:0/3281965226 shutdown_connections 2024-03-22T10:46:31.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.977+0000 7fafcee00700 1 -- 172.21.15.115:0/3281965226 wait complete. 2024-03-22T10:46:31.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.978+0000 7fafcee00700 1 Processor -- start 2024-03-22T10:46:31.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.978+0000 7fafcee00700 1 -- start start 2024-03-22T10:46:31.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.978+0000 7fafcee00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 0x7fafc819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:31.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.978+0000 7fafcee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc81079a0 0x7fafc819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:31.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.978+0000 7fafcee00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc819cb50 con 0x7fafc81067a0 2024-03-22T10:46:31.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.978+0000 7fafcee00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc819cc90 con 0x7fafc81079a0 2024-03-22T10:46:31.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.979+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 0x7fafc819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:31.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.979+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 0x7fafc819bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33388/0 (socket says 172.21.15.115:33388) 2024-03-22T10:46:31.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.979+0000 7fafc7e00700 1 -- 172.21.15.115:0/1445881706 learned_addr learned my addr 172.21.15.115:0/1445881706 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:46:31.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.979+0000 7fafc7e00700 1 -- 172.21.15.115:0/1445881706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc81079a0 msgr2=0x7fafc819c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:31.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.979+0000 7fafc7e00700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc81079a0 0x7fafc819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:31.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.979+0000 7fafc7e00700 1 -- 172.21.15.115:0/1445881706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8009d00 con 0x7fafc81067a0 2024-03-22T10:46:31.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.979+0000 7fafc7e00700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 0x7fafc819bff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006070 tx=0x7fafb80097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:31.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.980+0000 7fafc5600700 1 -- 172.21.15.115:0/1445881706 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafb800e040 con 0x7fafc81067a0 2024-03-22T10:46:31.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.980+0000 7fafc5600700 1 -- 172.21.15.115:0/1445881706 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fafb80044f0 con 0x7fafc81067a0 2024-03-22T10:46:31.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.980+0000 7fafc5600700 1 -- 172.21.15.115:0/1445881706 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafb801daf0 con 0x7fafc81067a0 2024-03-22T10:46:31.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.980+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc81a16e0 con 0x7fafc81067a0 2024-03-22T10:46:31.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.980+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc81a1c50 con 0x7fafc81067a0 2024-03-22T10:46:31.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.981+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc810f2f0 con 0x7fafc81067a0 2024-03-22T10:46:31.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.982+0000 7fafc5600700 1 -- 172.21.15.115:0/1445881706 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fafb8004040 con 0x7fafc81067a0 2024-03-22T10:46:31.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.983+0000 7fafc5600700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fafac06dfa0 0x7fafac070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:31.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.983+0000 7fafc5600700 1 -- 172.21.15.115:0/1445881706 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fafb808f700 con 0x7fafc81067a0 2024-03-22T10:46:31.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.983+0000 7fafbf400700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fafac06dfa0 0x7fafac070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:31.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.984+0000 7fafbf400700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fafac06dfa0 0x7fafac070450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fafb0005950 tx=0x7fafb00058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:31.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:31.987+0000 7fafc5600700 1 -- 172.21.15.115:0/1445881706 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fafb805c780 con 0x7fafc81067a0 2024-03-22T10:46:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.242+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fafc81a22e0 con 0x7fafc81067a0 2024-03-22T10:46:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.243+0000 7fafc5600700 1 -- 172.21.15.115:0/1445881706 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1926 (secure 0 0 0) 0x7fafb805c310 con 0x7fafc81067a0 2024-03-22T10:46:32.236 INFO:teuthology.orchestra.run.smithi115.stdout:e14 2024-03-22T10:46:32.236 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:46:32.236 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 14 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:43:22.786262+0000 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:46:32.237 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:46:32.238 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 0 members: 2024-03-22T10:46:32.239 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:46:32.239 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:46:32.239 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:46:32.239 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:46:32.239 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:46:32.239 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:46:32.239 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:46:32.239 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fafac06dfa0 msgr2=0x7fafac070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fafac06dfa0 0x7fafac070450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fafb0005950 tx=0x7fafb00058e0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 msgr2=0x7fafc819bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 0x7fafc819bff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006070 tx=0x7fafb80097b0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 shutdown_connections 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fafac06dfa0 0x7fafac070450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81067a0 0x7fafc819bff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 --2- 172.21.15.115:0/1445881706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc81079a0 0x7fafc819c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 >> 172.21.15.115:0/1445881706 conn(0x7fafc8075910 msgr2=0x7fafc810abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 shutdown_connections 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.247+0000 7fafcee00700 1 -- 172.21.15.115:0/1445881706 wait complete. 2024-03-22T10:46:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 14 2024-03-22T10:46:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.401+0000 7ff71ee00700 1 -- 172.21.15.115:0/3713090686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181079a0 msgr2=0x7ff718107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.401+0000 7ff71ee00700 1 --2- 172.21.15.115:0/3713090686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181079a0 0x7ff718107df0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff700009910 tx=0x7ff700009c20 comp rx=0 tx=0).stop 2024-03-22T10:46:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.401+0000 7ff71ee00700 1 -- 172.21.15.115:0/3713090686 shutdown_connections 2024-03-22T10:46:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.401+0000 7ff71ee00700 1 --2- 172.21.15.115:0/3713090686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181079a0 0x7ff718107df0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.401+0000 7ff71ee00700 1 --2- 172.21.15.115:0/3713090686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181067a0 0x7ff718106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.401+0000 7ff71ee00700 1 -- 172.21.15.115:0/3713090686 >> 172.21.15.115:0/3713090686 conn(0x7ff718075910 msgr2=0x7ff718077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.401+0000 7ff71ee00700 1 -- 172.21.15.115:0/3713090686 shutdown_connections 2024-03-22T10:46:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.401+0000 7ff71ee00700 1 -- 172.21.15.115:0/3713090686 wait complete. 2024-03-22T10:46:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.402+0000 7ff71ee00700 1 Processor -- start 2024-03-22T10:46:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.402+0000 7ff71ee00700 1 -- start start 2024-03-22T10:46:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff71ee00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181067a0 0x7ff71819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff71ee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181079a0 0x7ff71819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff71ee00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71819cb00 con 0x7ff7181067a0 2024-03-22T10:46:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff71ee00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71819cc40 con 0x7ff7181079a0 2024-03-22T10:46:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff717e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181067a0 0x7ff71819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff717e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181067a0 0x7ff71819bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33400/0 (socket says 172.21.15.115:33400) 2024-03-22T10:46:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff717e00700 1 -- 172.21.15.115:0/192338029 learned_addr learned my addr 172.21.15.115:0/192338029 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:46:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff70f400700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181079a0 0x7ff71819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff717e00700 1 -- 172.21.15.115:0/192338029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181079a0 msgr2=0x7ff71819c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff717e00700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181079a0 0x7ff71819c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff717e00700 1 -- 172.21.15.115:0/192338029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000095f0 con 0x7ff7181067a0 2024-03-22T10:46:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.403+0000 7ff70f400700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181079a0 0x7ff71819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:46:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.404+0000 7ff717e00700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181067a0 0x7ff71819bfa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff70800d900 tx=0x7ff70800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.404+0000 7ff715600700 1 -- 172.21.15.115:0/192338029 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7080041f0 con 0x7ff7181067a0 2024-03-22T10:46:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.404+0000 7ff715600700 1 -- 172.21.15.115:0/192338029 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff708004350 con 0x7ff7181067a0 2024-03-22T10:46:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.404+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7181a16f0 con 0x7ff7181067a0 2024-03-22T10:46:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.404+0000 7ff715600700 1 -- 172.21.15.115:0/192338029 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff708003da0 con 0x7ff7181067a0 2024-03-22T10:46:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.404+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7181a1c40 con 0x7ff7181067a0 2024-03-22T10:46:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.405+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71810f2f0 con 0x7ff7181067a0 2024-03-22T10:46:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.406+0000 7ff715600700 1 -- 172.21.15.115:0/192338029 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff708009730 con 0x7ff7181067a0 2024-03-22T10:46:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.407+0000 7ff715600700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff6fc06dfa0 0x7ff6fc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.407+0000 7ff715600700 1 -- 172.21.15.115:0/192338029 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff70808ded0 con 0x7ff7181067a0 2024-03-22T10:46:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.407+0000 7ff70f400700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff6fc06dfa0 0x7ff6fc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.408+0000 7ff70f400700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff6fc06dfa0 0x7ff6fc070450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff70000b5c0 tx=0x7ff700005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.412+0000 7ff715600700 1 -- 172.21.15.115:0/192338029 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff70805ae30 con 0x7ff7181067a0 2024-03-22T10:46:32.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:32 smithi115 ceph-mon[78672]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:32.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:32 smithi115 ceph-mon[78672]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:32.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:32 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/153575553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:46:32.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:32 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1445881706' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:46:32.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.651+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff71810c2f0 con 0x7ff6fc06dfa0 2024-03-22T10:46:32.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.654+0000 7ff715600700 1 -- 172.21.15.115:0/192338029 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff71810c2f0 con 0x7ff6fc06dfa0 2024-03-22T10:46:32.646 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:46:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:46:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:46:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:46:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:46:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:46:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:46:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:46:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:46:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "7/23 daemons upgraded", 2024-03-22T10:46:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T10:46:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:46:32.647 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:46:32.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:32 smithi150 ceph-mon[65513]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:32.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:32 smithi150 ceph-mon[65513]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:32.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:32 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/153575553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:46:32.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:32 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1445881706' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:46:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.656+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff6fc06dfa0 msgr2=0x7ff6fc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff6fc06dfa0 0x7ff6fc070450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff70000b5c0 tx=0x7ff700005960 comp rx=0 tx=0).stop 2024-03-22T10:46:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181067a0 msgr2=0x7ff71819bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181067a0 0x7ff71819bfa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff70800d900 tx=0x7ff70800dcc0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 shutdown_connections 2024-03-22T10:46:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff6fc06dfa0 0x7ff6fc070450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7181067a0 0x7ff71819bfa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 --2- 172.21.15.115:0/192338029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181079a0 0x7ff71819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 >> 172.21.15.115:0/192338029 conn(0x7ff718075910 msgr2=0x7ff71810abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:32.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 shutdown_connections 2024-03-22T10:46:32.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.657+0000 7ff71ee00700 1 -- 172.21.15.115:0/192338029 wait complete. 2024-03-22T10:46:32.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.807+0000 7fd0e2200700 1 -- 172.21.15.115:0/816194047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 msgr2=0x7fd0dc1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:32.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.807+0000 7fd0e2200700 1 --2- 172.21.15.115:0/816194047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 0x7fd0dc1017a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009910 tx=0x7fd0c8009c20 comp rx=0 tx=0).stop 2024-03-22T10:46:32.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.807+0000 7fd0e2200700 1 -- 172.21.15.115:0/816194047 shutdown_connections 2024-03-22T10:46:32.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.807+0000 7fd0e2200700 1 --2- 172.21.15.115:0/816194047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 0x7fd0dc1017a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.807+0000 7fd0e2200700 1 --2- 172.21.15.115:0/816194047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0dc100150 0x7fd0dc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.807+0000 7fd0e2200700 1 -- 172.21.15.115:0/816194047 >> 172.21.15.115:0/816194047 conn(0x7fd0dc0fb6e0 msgr2=0x7fd0dc0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:32.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.807+0000 7fd0e2200700 1 -- 172.21.15.115:0/816194047 shutdown_connections 2024-03-22T10:46:32.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.807+0000 7fd0e2200700 1 -- 172.21.15.115:0/816194047 wait complete. 2024-03-22T10:46:32.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.808+0000 7fd0e2200700 1 Processor -- start 2024-03-22T10:46:32.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.808+0000 7fd0e2200700 1 -- start start 2024-03-22T10:46:32.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.808+0000 7fd0e2200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0dc100150 0x7fd0dc197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:32.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.808+0000 7fd0e2200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 0x7fd0dc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:32.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.808+0000 7fd0e2200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc1986d0 con 0x7fd0dc101350 2024-03-22T10:46:32.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.808+0000 7fd0e2200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc198810 con 0x7fd0dc100150 2024-03-22T10:46:32.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.809+0000 7fd0daa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 0x7fd0dc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:32.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.809+0000 7fd0daa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 0x7fd0dc1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33422/0 (socket says 172.21.15.115:33422) 2024-03-22T10:46:32.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.809+0000 7fd0daa00700 1 -- 172.21.15.115:0/2793983338 learned_addr learned my addr 172.21.15.115:0/2793983338 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:46:32.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.809+0000 7fd0db400700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0dc100150 0x7fd0dc197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:32.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.809+0000 7fd0daa00700 1 -- 172.21.15.115:0/2793983338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0dc100150 msgr2=0x7fd0dc197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:32.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.809+0000 7fd0daa00700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0dc100150 0x7fd0dc197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:32.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.809+0000 7fd0daa00700 1 -- 172.21.15.115:0/2793983338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c80095f0 con 0x7fd0dc101350 2024-03-22T10:46:32.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.809+0000 7fd0daa00700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 0x7fd0dc1980b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800bd70 tx=0x7fd0c800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:32.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.810+0000 7fd0d3e00700 1 -- 172.21.15.115:0/2793983338 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c801d070 con 0x7fd0dc101350 2024-03-22T10:46:32.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.810+0000 7fd0d3e00700 1 -- 172.21.15.115:0/2793983338 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd0c8022470 con 0x7fd0dc101350 2024-03-22T10:46:32.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.810+0000 7fd0d3e00700 1 -- 172.21.15.115:0/2793983338 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c800f670 con 0x7fd0dc101350 2024-03-22T10:46:32.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.810+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc19d260 con 0x7fd0dc101350 2024-03-22T10:46:32.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.810+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc19d750 con 0x7fd0dc101350 2024-03-22T10:46:32.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.811+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc105430 con 0x7fd0dc101350 2024-03-22T10:46:32.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.812+0000 7fd0d3e00700 1 -- 172.21.15.115:0/2793983338 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fd0c8022a50 con 0x7fd0dc101350 2024-03-22T10:46:32.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.813+0000 7fd0d3e00700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd0c006dfa0 0x7fd0c0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:46:32.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.813+0000 7fd0d3e00700 1 -- 172.21.15.115:0/2793983338 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd0c808f2f0 con 0x7fd0dc101350 2024-03-22T10:46:32.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.813+0000 7fd0db400700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd0c006dfa0 0x7fd0c0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:46:32.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.813+0000 7fd0db400700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd0c006dfa0 0x7fd0c0070450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005950 tx=0x7fd0cc0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:46:32.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:32.817+0000 7fd0d3e00700 1 -- 172.21.15.115:0/2793983338 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd0c805c4a0 con 0x7fd0dc101350 2024-03-22T10:46:33.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.110+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd0dc061960 con 0x7fd0dc101350 2024-03-22T10:46:33.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.110+0000 7fd0d3e00700 1 -- 172.21.15.115:0/2793983338 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd0c805c030 con 0x7fd0dc101350 2024-03-22T10:46:33.102 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:46:33.102 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:46:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:46:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.113+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd0c006dfa0 msgr2=0x7fd0c0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.113+0000 7fd0e2200700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd0c006dfa0 0x7fd0c0070450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005950 tx=0x7fd0cc0058e0 comp rx=0 tx=0).stop 2024-03-22T10:46:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.113+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 msgr2=0x7fd0dc1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:46:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.113+0000 7fd0e2200700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 0x7fd0dc1980b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800bd70 tx=0x7fd0c800be50 comp rx=0 tx=0).stop 2024-03-22T10:46:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.114+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 shutdown_connections 2024-03-22T10:46:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.114+0000 7fd0e2200700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd0c006dfa0 0x7fd0c0070450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.114+0000 7fd0e2200700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0dc100150 0x7fd0dc197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.114+0000 7fd0e2200700 1 --2- 172.21.15.115:0/2793983338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0dc101350 0x7fd0dc1980b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:46:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.114+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 >> 172.21.15.115:0/2793983338 conn(0x7fd0dc0fb6e0 msgr2=0x7fd0dc102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:46:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.114+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 shutdown_connections 2024-03-22T10:46:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:46:33.114+0000 7fd0e2200700 1 -- 172.21.15.115:0/2793983338 wait complete. 2024-03-22T10:46:33.501 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:33 smithi115 ceph-mon[78672]: pgmap v82: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:33.501 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:33 smithi115 ceph-mon[78672]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:33.501 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:33 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2793983338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:46:33.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:33 smithi150 ceph-mon[65513]: pgmap v82: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:33.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:33 smithi150 ceph-mon[65513]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:46:33.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:33 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2793983338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:46:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:35 smithi115 ceph-mon[78672]: pgmap v83: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:35.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:35.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:46:35.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:35 smithi150 ceph-mon[65513]: pgmap v83: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:46:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:37.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:37 smithi115 ceph-mon[78672]: pgmap v84: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:37.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:37 smithi150 ceph-mon[65513]: pgmap v84: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:39.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:39 smithi115 ceph-mon[78672]: pgmap v85: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:39.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:39 smithi150 ceph-mon[65513]: pgmap v85: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:41.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:41 smithi150 ceph-mon[65513]: pgmap v86: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:41 smithi115 ceph-mon[78672]: pgmap v86: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:42.584 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:42 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T10:46:42.584 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:42 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T10:46:42.584 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:42 smithi115 ceph-mon[78672]: Upgrade: osd.1 is safe to restart 2024-03-22T10:46:42.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:42 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T10:46:42.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:42 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T10:46:42.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:42 smithi150 ceph-mon[65513]: Upgrade: osd.1 is safe to restart 2024-03-22T10:46:43.619 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:43 smithi115 ceph-mon[78672]: pgmap v87: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:43.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:43 smithi150 ceph-mon[65513]: pgmap v87: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:44.873 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:44 smithi115 ceph-mon[78672]: Upgrade: Updating osd.1 2024-03-22T10:46:44.873 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:44 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:44.873 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:44 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T10:46:44.873 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:44 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:46:44.874 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:44 smithi115 ceph-mon[78672]: Deploying daemon osd.1 on smithi115 2024-03-22T10:46:45.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:44 smithi150 ceph-mon[65513]: Upgrade: Updating osd.1 2024-03-22T10:46:45.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:44 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:45.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:44 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T10:46:45.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:44 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:46:45.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:44 smithi150 ceph-mon[65513]: Deploying daemon osd.1 on smithi115 2024-03-22T10:46:46.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:45 smithi115 ceph-mon[78672]: pgmap v88: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:46.112 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:45 smithi150 ceph-mon[65513]: pgmap v88: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:48.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:47 smithi115 ceph-mon[78672]: pgmap v89: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:48.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:47 smithi150 ceph-mon[65513]: pgmap v89: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:48.760 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:48 smithi115 systemd[1]: Stopping Ceph osd.1 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:46:49.262 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:48 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[49501]: 2024-03-22T10:46:48.895+0000 7f44a8a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T10:46:49.262 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:48 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[49501]: 2024-03-22T10:46:48.895+0000 7f44a8a00700 -1 osd.1 53 *** Got signal Terminated *** 2024-03-22T10:46:49.262 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:48 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[49501]: 2024-03-22T10:46:48.895+0000 7f44a8a00700 -1 osd.1 53 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T10:46:50.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:49 smithi115 ceph-mon[78672]: pgmap v90: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:50.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:49 smithi115 ceph-mon[78672]: osd.1 marked itself down and dead 2024-03-22T10:46:50.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:46:50.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:49 smithi150 ceph-mon[65513]: pgmap v90: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:50.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:49 smithi150 ceph-mon[65513]: osd.1 marked itself down and dead 2024-03-22T10:46:50.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:46:50.586 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:50 smithi115 podman[88201]: 2024-03-22 10:46:50.324295466 +0000 UTC m=+1.547300474 container died 7ad3620926360a1b64219ac4087a2c247841711bd2ce5a59bb2630c33a78e1d9 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, ceph=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-22T10:46:50.939 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:50 smithi115 ceph-mon[78672]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:46:50.940 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:50 smithi115 ceph-mon[78672]: osdmap e54: 6 total, 5 up, 6 in 2024-03-22T10:46:50.940 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:50 smithi115 podman[88201]: 2024-03-22 10:46:50.730002593 +0000 UTC m=+1.953007594 container cleanup 7ad3620926360a1b64219ac4087a2c247841711bd2ce5a59bb2630c33a78e1d9 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD) 2024-03-22T10:46:50.940 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:50 smithi115 bash[88201]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1 2024-03-22T10:46:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:50 smithi150 ceph-mon[65513]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:46:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:50 smithi150 ceph-mon[65513]: osdmap e54: 6 total, 5 up, 6 in 2024-03-22T10:46:51.260 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:50 smithi115 podman[88221]: 2024-03-22 10:46:50.947921455 +0000 UTC m=+0.610873309 container remove 7ad3620926360a1b64219ac4087a2c247841711bd2ce5a59bb2630c33a78e1d9 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0) 2024-03-22T10:46:52.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:51 smithi115 ceph-mon[78672]: pgmap v92: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:52.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:51 smithi115 ceph-mon[78672]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-22T10:46:52.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:51 smithi115 ceph-mon[78672]: osdmap e55: 6 total, 5 up, 6 in 2024-03-22T10:46:52.011 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:51 smithi115 podman[88306]: 2024-03-22 10:46:51.603685405 +0000 UTC m=+0.032252762 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:46:52.011 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:51 smithi115 podman[88306]: 2024-03-22 10:46:51.707033757 +0000 UTC m=+0.135601127 container create 8041c033766887d1c0541e3e367effaad63a1109f6f9fbb8f232af1b268f2c28 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:46:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:51 smithi150 ceph-mon[65513]: pgmap v92: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:51 smithi150 ceph-mon[65513]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-22T10:46:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:51 smithi150 ceph-mon[65513]: osdmap e55: 6 total, 5 up, 6 in 2024-03-22T10:46:52.511 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:52 smithi115 podman[88306]: 2024-03-22 10:46:52.122843269 +0000 UTC m=+0.551410670 container init 8041c033766887d1c0541e3e367effaad63a1109f6f9fbb8f232af1b268f2c28 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-22T10:46:52.511 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:52 smithi115 podman[88306]: 2024-03-22 10:46:52.13489067 +0000 UTC m=+0.563458032 container start 8041c033766887d1c0541e3e367effaad63a1109f6f9fbb8f232af1b268f2c28 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-03-22T10:46:52.511 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:52 smithi115 podman[88306]: 2024-03-22 10:46:52.235074058 +0000 UTC m=+0.663641491 container attach 8041c033766887d1c0541e3e367effaad63a1109f6f9fbb8f232af1b268f2c28 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-22T10:46:52.921 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:52 smithi115 podman[88332]: 2024-03-22 10:46:52.624102124 +0000 UTC m=+0.035319315 container died 8041c033766887d1c0541e3e367effaad63a1109f6f9fbb8f232af1b268f2c28 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-22T10:46:53.717 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:53 smithi115 podman[88332]: 2024-03-22 10:46:53.291143718 +0000 UTC m=+0.702360944 container remove 8041c033766887d1c0541e3e367effaad63a1109f6f9fbb8f232af1b268f2c28 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-22T10:46:53.717 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:53 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-22T10:46:53.717 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:53 smithi115 systemd[1]: Stopped Ceph osd.1 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:46:53.717 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:53 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.1.service: Consumed 1min 37.669s CPU time. 2024-03-22T10:46:53.717 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:53 smithi115 systemd[1]: Starting Ceph osd.1 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:46:54.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:53 smithi115 ceph-mon[78672]: pgmap v94: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:54.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:53 smithi150 ceph-mon[65513]: pgmap v94: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:46:54.410 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:54 smithi115 podman[88436]: 2024-03-22 10:46:53.977960647 +0000 UTC m=+0.034982884 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:46:54.411 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:54 smithi115 podman[88436]: 2024-03-22 10:46:54.106092286 +0000 UTC m=+0.163114539 container create fd3f182cba879c4b0506c73fe899b9f76cbbe30c20506ee3ac4a36a3b108b22f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-22T10:46:54.760 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:54 smithi115 podman[88436]: 2024-03-22 10:46:54.418062041 +0000 UTC m=+0.475084337 container init fd3f182cba879c4b0506c73fe899b9f76cbbe30c20506ee3ac4a36a3b108b22f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-03-22T10:46:54.761 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:54 smithi115 podman[88436]: 2024-03-22 10:46:54.430901805 +0000 UTC m=+0.487924030 container start fd3f182cba879c4b0506c73fe899b9f76cbbe30c20506ee3ac4a36a3b108b22f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T10:46:54.761 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:54 smithi115 podman[88436]: 2024-03-22 10:46:54.533997088 +0000 UTC m=+0.591019347 container attach fd3f182cba879c4b0506c73fe899b9f76cbbe30c20506ee3ac4a36a3b108b22f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T10:46:55.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:54 smithi150 ceph-mon[65513]: Health check failed: Degraded data redundancy: 8/369 objects degraded (2.168%), 2 pgs degraded (PG_DEGRADED) 2024-03-22T10:46:55.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:54 smithi115 ceph-mon[78672]: Health check failed: Degraded data redundancy: 8/369 objects degraded (2.168%), 2 pgs degraded (PG_DEGRADED) 2024-03-22T10:46:55.686 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate[88451]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate[88451]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 bash[88436]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 bash[88436]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 bash[88436]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 bash[88436]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 bash[88436]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 bash[88436]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 bash[88436]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate[88451]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate[88451]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate[88451]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-22T10:46:55.687 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate[88451]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T10:46:55.688 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate[88451]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-22T10:46:55.688 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:55 smithi115 podman[88436]: 2024-03-22 10:46:55.354367124 +0000 UTC m=+1.411389414 container died fd3f182cba879c4b0506c73fe899b9f76cbbe30c20506ee3ac4a36a3b108b22f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-22T10:46:56.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:55 smithi115 ceph-mon[78672]: pgmap v95: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 8/369 objects degraded (2.168%) 2024-03-22T10:46:56.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:55 smithi150 ceph-mon[65513]: pgmap v95: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 8/369 objects degraded (2.168%) 2024-03-22T10:46:56.450 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:56 smithi115 podman[88576]: 2024-03-22 10:46:56.171300755 +0000 UTC m=+0.798711343 container remove fd3f182cba879c4b0506c73fe899b9f76cbbe30c20506ee3ac4a36a3b108b22f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-22T10:46:56.760 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:56 smithi115 podman[88633]: 2024-03-22 10:46:56.423301127 +0000 UTC m=+0.036686335 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:46:56.761 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:56 smithi115 podman[88633]: 2024-03-22 10:46:56.555878477 +0000 UTC m=+0.169263722 container create 4e693719f2d78ed18e0f004cd0fab32a479c9fdc3e3eed7c5da10b8b423007d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-22T10:46:57.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:56 smithi150 ceph-mon[65513]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-22T10:46:57.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:56 smithi115 ceph-mon[78672]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-22T10:46:57.260 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:56 smithi115 podman[88633]: 2024-03-22 10:46:56.792510548 +0000 UTC m=+0.405895800 container init 4e693719f2d78ed18e0f004cd0fab32a479c9fdc3e3eed7c5da10b8b423007d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-22T10:46:57.260 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:56 smithi115 podman[88633]: 2024-03-22 10:46:56.803053291 +0000 UTC m=+0.416438542 container start 4e693719f2d78ed18e0f004cd0fab32a479c9fdc3e3eed7c5da10b8b423007d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-22T10:46:57.260 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:56 smithi115 ceph-osd[88652]: -- 172.21.15.115:0/585110804 <== mon.0 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562bdcdbd680 con 0x562bdcd87000 2024-03-22T10:46:57.260 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:56 smithi115 bash[88633]: 4e693719f2d78ed18e0f004cd0fab32a479c9fdc3e3eed7c5da10b8b423007d3 2024-03-22T10:46:57.261 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:56 smithi115 systemd[1]: Started Ceph osd.1 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:46:57.760 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:46:57 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[88648]: 2024-03-22T10:46:57.374+0000 7f951f08a7c0 -1 Falling back to public interface 2024-03-22T10:46:58.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:57 smithi150 ceph-mon[65513]: pgmap v96: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:46:58.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:57 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:58.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:57 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:58.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:57 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:46:58.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:57 smithi115 ceph-mon[78672]: pgmap v96: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:46:58.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:57 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:58.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:57 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:46:58.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:57 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:47:00.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:59 smithi150 ceph-mon[65513]: pgmap v97: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:47:00.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:46:59 smithi150 ceph-mon[65513]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:47:00.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:59 smithi115 ceph-mon[78672]: pgmap v97: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:47:00.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:46:59 smithi115 ceph-mon[78672]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:47:02.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:01 smithi150 ceph-mon[65513]: pgmap v98: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:47:02.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:01 smithi115 ceph-mon[78672]: pgmap v98: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:47:02.760 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:47:02 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[88648]: 2024-03-22T10:47:02.557+0000 7f951f08a7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-22T10:47:03.248 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:47:03 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[88648]: 2024-03-22T10:47:03.006+0000 7f951f08a7c0 -1 osd.1 53 log_to_monitors true 2024-03-22T10:47:03.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.275+0000 7f80d2e00700 1 -- 172.21.15.115:0/3259876972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1035a0 msgr2=0x7f80cc1039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:03.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.275+0000 7f80d2e00700 1 --2- 172.21.15.115:0/3259876972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1035a0 0x7f80cc1039f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009910 tx=0x7f80b8009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:03.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.275+0000 7f80d2e00700 1 -- 172.21.15.115:0/3259876972 shutdown_connections 2024-03-22T10:47:03.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.275+0000 7f80d2e00700 1 --2- 172.21.15.115:0/3259876972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1035a0 0x7f80cc1039f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.275+0000 7f80d2e00700 1 --2- 172.21.15.115:0/3259876972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc1023a0 0x7f80cc1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.275+0000 7f80d2e00700 1 -- 172.21.15.115:0/3259876972 >> 172.21.15.115:0/3259876972 conn(0x7f80cc0fd990 msgr2=0x7f80cc0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:03.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.275+0000 7f80d2e00700 1 -- 172.21.15.115:0/3259876972 shutdown_connections 2024-03-22T10:47:03.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.275+0000 7f80d2e00700 1 -- 172.21.15.115:0/3259876972 wait complete. 2024-03-22T10:47:03.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.276+0000 7f80d2e00700 1 Processor -- start 2024-03-22T10:47:03.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.276+0000 7f80d2e00700 1 -- start start 2024-03-22T10:47:03.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80d2e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1023a0 0x7f80cc197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:03.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80d2e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc1035a0 0x7f80cc1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80d2e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc198700 con 0x7f80cc1023a0 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80d2e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc198840 con 0x7f80cc1035a0 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80cbe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1023a0 0x7f80cc197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80cbe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1023a0 0x7f80cc197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34460/0 (socket says 172.21.15.115:34460) 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80cbe00700 1 -- 172.21.15.115:0/880789308 learned_addr learned my addr 172.21.15.115:0/880789308 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80cb400700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc1035a0 0x7f80cc1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80cbe00700 1 -- 172.21.15.115:0/880789308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc1035a0 msgr2=0x7f80cc1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.277+0000 7f80cbe00700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc1035a0 0x7f80cc1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.278+0000 7f80cbe00700 1 -- 172.21.15.115:0/880789308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80095f0 con 0x7f80cc1023a0 2024-03-22T10:47:03.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.278+0000 7f80cbe00700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1023a0 0x7f80cc197ba0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00d900 tx=0x7f80bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.278+0000 7f80c8c00700 1 -- 172.21.15.115:0/880789308 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80bc0041f0 con 0x7f80cc1023a0 2024-03-22T10:47:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.278+0000 7f80c8c00700 1 -- 172.21.15.115:0/880789308 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f80bc004350 con 0x7f80cc1023a0 2024-03-22T10:47:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.278+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc19d2f0 con 0x7f80cc1023a0 2024-03-22T10:47:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.278+0000 7f80c8c00700 1 -- 172.21.15.115:0/880789308 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80bc003da0 con 0x7f80cc1023a0 2024-03-22T10:47:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.278+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc19d840 con 0x7f80cc1023a0 2024-03-22T10:47:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.279+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc10aef0 con 0x7f80cc1023a0 2024-03-22T10:47:03.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.280+0000 7f80c8c00700 1 -- 172.21.15.115:0/880789308 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f80bc009730 con 0x7f80cc1023a0 2024-03-22T10:47:03.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.281+0000 7f80c8c00700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f80b006dfa0 0x7f80b0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:03.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.281+0000 7f80c8c00700 1 -- 172.21.15.115:0/880789308 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f80bc08ded0 con 0x7f80cc1023a0 2024-03-22T10:47:03.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.282+0000 7f80cb400700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f80b006dfa0 0x7f80b0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:03.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.282+0000 7f80cb400700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f80b006dfa0 0x7f80b0070450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005b40 tx=0x7f80b8005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:03.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.286+0000 7f80c8c00700 1 -- 172.21.15.115:0/880789308 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f80bc05ae30 con 0x7f80cc1023a0 2024-03-22T10:47:03.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.530+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80cc107ef0 con 0x7f80b006dfa0 2024-03-22T10:47:03.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.532+0000 7f80c8c00700 1 -- 172.21.15.115:0/880789308 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f80cc107ef0 con 0x7f80b006dfa0 2024-03-22T10:47:03.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f80b006dfa0 msgr2=0x7f80b0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:03.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f80b006dfa0 0x7f80b0070450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005b40 tx=0x7f80b8005a90 comp rx=0 tx=0).stop 2024-03-22T10:47:03.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1023a0 msgr2=0x7f80cc197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:03.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1023a0 0x7f80cc197ba0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00d900 tx=0x7f80bc00dcc0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 shutdown_connections 2024-03-22T10:47:03.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f80b006dfa0 0x7f80b0070450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80cc1023a0 0x7f80cc197ba0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 --2- 172.21.15.115:0/880789308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc1035a0 0x7f80cc1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 >> 172.21.15.115:0/880789308 conn(0x7f80cc0fd990 msgr2=0x7f80cc1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:03.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 shutdown_connections 2024-03-22T10:47:03.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.535+0000 7f80d2e00700 1 -- 172.21.15.115:0/880789308 wait complete. 2024-03-22T10:47:03.540 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:47:03.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.697+0000 7f7020c00700 1 -- 172.21.15.115:0/2385739282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 msgr2=0x7f701c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:03.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.697+0000 7f7020c00700 1 --2- 172.21.15.115:0/2385739282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 0x7f701c100560 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f700c009910 tx=0x7f700c009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:03.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.697+0000 7f7020c00700 1 -- 172.21.15.115:0/2385739282 shutdown_connections 2024-03-22T10:47:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.697+0000 7f7020c00700 1 --2- 172.21.15.115:0/2385739282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f701c101350 0x7f701c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.697+0000 7f7020c00700 1 --2- 172.21.15.115:0/2385739282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 0x7f701c100560 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.697+0000 7f7020c00700 1 -- 172.21.15.115:0/2385739282 >> 172.21.15.115:0/2385739282 conn(0x7f701c0fb6e0 msgr2=0x7f701c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.697+0000 7f7020c00700 1 -- 172.21.15.115:0/2385739282 shutdown_connections 2024-03-22T10:47:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.697+0000 7f7020c00700 1 -- 172.21.15.115:0/2385739282 wait complete. 2024-03-22T10:47:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.699+0000 7f7020c00700 1 Processor -- start 2024-03-22T10:47:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.699+0000 7f7020c00700 1 -- start start 2024-03-22T10:47:03.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.699+0000 7f7020c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 0x7f701c1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:03.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.699+0000 7f7020c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f701c101350 0x7f701c195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:03.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.699+0000 7f7020c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c196500 con 0x7f701c100150 2024-03-22T10:47:03.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.699+0000 7f7020c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c196640 con 0x7f701c101350 2024-03-22T10:47:03.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.700+0000 7f7019e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 0x7f701c1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:03.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.700+0000 7f7019e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 0x7f701c1959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34476/0 (socket says 172.21.15.115:34476) 2024-03-22T10:47:03.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.700+0000 7f7019e00700 1 -- 172.21.15.115:0/1201989506 learned_addr learned my addr 172.21.15.115:0/1201989506 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:03.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.700+0000 7f7019400700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f701c101350 0x7f701c195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:03.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.700+0000 7f7019e00700 1 -- 172.21.15.115:0/1201989506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f701c101350 msgr2=0x7f701c195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:03.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.700+0000 7f7019e00700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f701c101350 0x7f701c195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.700+0000 7f7019e00700 1 -- 172.21.15.115:0/1201989506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0095f0 con 0x7f701c100150 2024-03-22T10:47:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.701+0000 7f7019e00700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 0x7f701c1959a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f700c00bd70 tx=0x7f700c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.701+0000 7f7012a00700 1 -- 172.21.15.115:0/1201989506 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f700c01d070 con 0x7f701c100150 2024-03-22T10:47:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.701+0000 7f7012a00700 1 -- 172.21.15.115:0/1201989506 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f700c00f460 con 0x7f701c100150 2024-03-22T10:47:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.701+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c19b090 con 0x7f701c100150 2024-03-22T10:47:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.701+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c19b580 con 0x7f701c100150 2024-03-22T10:47:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.701+0000 7f7012a00700 1 -- 172.21.15.115:0/1201989506 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f700c021620 con 0x7f701c100150 2024-03-22T10:47:03.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.702+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c108ca0 con 0x7f701c100150 2024-03-22T10:47:03.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.704+0000 7f7012a00700 1 -- 172.21.15.115:0/1201989506 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f700c00fa40 con 0x7f701c100150 2024-03-22T10:47:03.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.705+0000 7f7012a00700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f700006dfa0 0x7f7000070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:03.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.705+0000 7f7012a00700 1 -- 172.21.15.115:0/1201989506 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f700c08f360 con 0x7f701c100150 2024-03-22T10:47:03.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.705+0000 7f7019400700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f700006dfa0 0x7f7000070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:03.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.706+0000 7f7019400700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f700006dfa0 0x7f7000070450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7008005950 tx=0x7f70080058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:03.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:03.709+0000 7f7012a00700 1 -- 172.21.15.115:0/1201989506 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f700c05c480 con 0x7f701c100150 2024-03-22T10:47:03.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.001+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f701c105ca0 con 0x7f700006dfa0 2024-03-22T10:47:03.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.003+0000 7f7012a00700 1 -- 172.21.15.115:0/1201989506 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f701c105ca0 con 0x7f700006dfa0 2024-03-22T10:47:03.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.005+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f700006dfa0 msgr2=0x7f7000070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:03.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.005+0000 7f7020c00700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f700006dfa0 0x7f7000070450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7008005950 tx=0x7f70080058e0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.005+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 msgr2=0x7f701c1959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:03.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.005+0000 7f7020c00700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 0x7f701c1959a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f700c00bd70 tx=0x7f700c00be50 comp rx=0 tx=0).stop 2024-03-22T10:47:03.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.006+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 shutdown_connections 2024-03-22T10:47:03.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.006+0000 7f7020c00700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f700006dfa0 0x7f7000070450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.006+0000 7f7020c00700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100150 0x7f701c1959a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.006+0000 7f7020c00700 1 --2- 172.21.15.115:0/1201989506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f701c101350 0x7f701c195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:03.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.006+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 >> 172.21.15.115:0/1201989506 conn(0x7f701c0fb6e0 msgr2=0x7f701c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:03.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.006+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 shutdown_connections 2024-03-22T10:47:03.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.006+0000 7f7020c00700 1 -- 172.21.15.115:0/1201989506 wait complete. 2024-03-22T10:47:04.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.157+0000 7ff2da000700 1 -- 172.21.15.115:0/2253722511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40fea50 msgr2=0x7ff2d40fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:04.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.157+0000 7ff2da000700 1 --2- 172.21.15.115:0/2253722511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40fea50 0x7ff2d40fee60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009910 tx=0x7ff2c4009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:04.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.157+0000 7ff2da000700 1 -- 172.21.15.115:0/2253722511 shutdown_connections 2024-03-22T10:47:04.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.157+0000 7ff2da000700 1 --2- 172.21.15.115:0/2253722511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2d40ff3a0 0x7ff2d40ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.157+0000 7ff2da000700 1 --2- 172.21.15.115:0/2253722511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40fea50 0x7ff2d40fee60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.157+0000 7ff2da000700 1 -- 172.21.15.115:0/2253722511 >> 172.21.15.115:0/2253722511 conn(0x7ff2d40fa600 msgr2=0x7ff2d40fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:04.149 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:03 smithi115 ceph-mon[78672]: pgmap v99: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:47:04.149 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:03 smithi115 ceph-mon[78672]: from='osd.1 [v2:172.21.15.115:6810/3813430676,v1:172.21.15.115:6811/3813430676]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T10:47:04.149 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 10:47:03 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[88648]: 2024-03-22T10:47:03.918+0000 7f950e400700 -1 osd.1 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T10:47:04.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.158+0000 7ff2da000700 1 -- 172.21.15.115:0/2253722511 shutdown_connections 2024-03-22T10:47:04.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.158+0000 7ff2da000700 1 -- 172.21.15.115:0/2253722511 wait complete. 2024-03-22T10:47:04.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:03 smithi150 ceph-mon[65513]: pgmap v99: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:47:04.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:03 smithi150 ceph-mon[65513]: from='osd.1 [v2:172.21.15.115:6810/3813430676,v1:172.21.15.115:6811/3813430676]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T10:47:04.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.159+0000 7ff2da000700 1 Processor -- start 2024-03-22T10:47:04.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.159+0000 7ff2da000700 1 -- start start 2024-03-22T10:47:04.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.160+0000 7ff2da000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2d40fea50 0x7ff2d4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:04.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.160+0000 7ff2da000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40ff3a0 0x7ff2d41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:04.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.160+0000 7ff2da000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4198630 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.160+0000 7ff2da000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4198770 con 0x7ff2d40fea50 2024-03-22T10:47:04.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.160+0000 7ff2d2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40ff3a0 0x7ff2d41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:04.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.160+0000 7ff2d2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40ff3a0 0x7ff2d41980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34504/0 (socket says 172.21.15.115:34504) 2024-03-22T10:47:04.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.160+0000 7ff2d2a00700 1 -- 172.21.15.115:0/1087273324 learned_addr learned my addr 172.21.15.115:0/1087273324 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:04.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.161+0000 7ff2d3400700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2d40fea50 0x7ff2d4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:04.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.161+0000 7ff2d2a00700 1 -- 172.21.15.115:0/1087273324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2d40fea50 msgr2=0x7ff2d4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:04.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.161+0000 7ff2d2a00700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2d40fea50 0x7ff2d4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.161+0000 7ff2d2a00700 1 -- 172.21.15.115:0/1087273324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c40095f0 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.161+0000 7ff2d2a00700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40ff3a0 0x7ff2d41980a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000c930 tx=0x7ff2c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:04.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.162+0000 7ff2cbe00700 1 -- 172.21.15.115:0/1087273324 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2c0007ad0 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.162+0000 7ff2cbe00700 1 -- 172.21.15.115:0/1087273324 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff2c0007c30 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.162+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d419d230 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.162+0000 7ff2cbe00700 1 -- 172.21.15.115:0/1087273324 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2c0018630 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.162+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d419d750 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.163+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d410aea0 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.164+0000 7ff2cbe00700 1 -- 172.21.15.115:0/1087273324 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff2c000f450 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.165+0000 7ff2cbe00700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff2b806dfa0 0x7ff2b8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:04.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.165+0000 7ff2cbe00700 1 -- 172.21.15.115:0/1087273324 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff2c008e0a0 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.165+0000 7ff2d3400700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff2b806dfa0 0x7ff2b8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:04.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.166+0000 7ff2d3400700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff2b806dfa0 0x7ff2b8070450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4006010 tx=0x7ff2c400b580 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:04.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.170+0000 7ff2cbe00700 1 -- 172.21.15.115:0/1087273324 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff2c005b1b0 con 0x7ff2d40ff3a0 2024-03-22T10:47:04.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.415+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2d41087a0 con 0x7ff2b806dfa0 2024-03-22T10:47:04.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.435+0000 7ff2cbe00700 1 -- 172.21.15.115:0/1087273324 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff2d41087a0 con 0x7ff2b806dfa0 2024-03-22T10:47:04.426 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:47:04.426 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (13m) 0s ago 15m 14.1M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:47:04.426 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (15m) 0s ago 15m 8414k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (14m) 75s ago 14m 7952k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (87s) 0s ago 15m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (80s) 75s ago 14m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (13m) 0s ago 15m 73.5M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (8m) 0s ago 8m 127M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (8m) 0s ago 8m 16.7M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (8m) 75s ago 8m 15.5M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (8m) 75s ago 8m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (4m) 0s ago 17m 505M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (3m) 75s ago 14m 449M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (3m) 0s ago 17m 45.0M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (2m) 75s ago 14m 33.6M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (15m) 0s ago 15m 8232k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (14m) 75s ago 14m 7721k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:47:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (52s) 0s ago 13m 82.6M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:47:04.428 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (7s) 0s ago 12m 11.1M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:47:04.428 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (11m) 0s ago 11m 166M 1217M 18.2.0 dc2bc1663786 52d9f92f53a8 2024-03-22T10:47:04.428 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (11m) 75s ago 11m 235M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:47:04.428 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (10m) 75s ago 10m 225M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:47:04.428 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (10m) 75s ago 10m 195M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:47:04.428 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (3m) 0s ago 14m 70.4M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:47:04.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff2b806dfa0 msgr2=0x7ff2b8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:04.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff2b806dfa0 0x7ff2b8070450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4006010 tx=0x7ff2c400b580 comp rx=0 tx=0).stop 2024-03-22T10:47:04.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40ff3a0 msgr2=0x7ff2d41980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:04.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40ff3a0 0x7ff2d41980a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000c930 tx=0x7ff2c000ccf0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 shutdown_connections 2024-03-22T10:47:04.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff2b806dfa0 0x7ff2b8070450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2d40fea50 0x7ff2d4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 --2- 172.21.15.115:0/1087273324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2d40ff3a0 0x7ff2d41980a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 >> 172.21.15.115:0/1087273324 conn(0x7ff2d40fa600 msgr2=0x7ff2d4107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:04.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 shutdown_connections 2024-03-22T10:47:04.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.438+0000 7ff2da000700 1 -- 172.21.15.115:0/1087273324 wait complete. 2024-03-22T10:47:04.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.590+0000 7fb888000700 1 -- 172.21.15.115:0/1436587766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880075700 msgr2=0x7fb880075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:04.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.590+0000 7fb888000700 1 --2- 172.21.15.115:0/1436587766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880075700 0x7fb880075b10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb874009910 tx=0x7fb874009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:04.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.591+0000 7fb888000700 1 -- 172.21.15.115:0/1436587766 shutdown_connections 2024-03-22T10:47:04.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.591+0000 7fb888000700 1 --2- 172.21.15.115:0/1436587766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb880076950 0x7fb880076dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.591+0000 7fb888000700 1 --2- 172.21.15.115:0/1436587766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880075700 0x7fb880075b10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.591+0000 7fb888000700 1 -- 172.21.15.115:0/1436587766 >> 172.21.15.115:0/1436587766 conn(0x7fb8800fd930 msgr2=0x7fb8800ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:04.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.591+0000 7fb888000700 1 -- 172.21.15.115:0/1436587766 shutdown_connections 2024-03-22T10:47:04.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.591+0000 7fb888000700 1 -- 172.21.15.115:0/1436587766 wait complete. 2024-03-22T10:47:04.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.592+0000 7fb888000700 1 Processor -- start 2024-03-22T10:47:04.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.592+0000 7fb888000700 1 -- start start 2024-03-22T10:47:04.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.593+0000 7fb888000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb880075700 0x7fb88019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:04.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.593+0000 7fb888000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880076950 0x7fb88019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:04.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.593+0000 7fb888000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88019caf0 con 0x7fb880076950 2024-03-22T10:47:04.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.593+0000 7fb888000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88019cc30 con 0x7fb880075700 2024-03-22T10:47:04.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.593+0000 7fb884c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880076950 0x7fb88019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:04.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.593+0000 7fb884c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880076950 0x7fb88019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34522/0 (socket says 172.21.15.115:34522) 2024-03-22T10:47:04.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.593+0000 7fb884c00700 1 -- 172.21.15.115:0/3711074261 learned_addr learned my addr 172.21.15.115:0/3711074261 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:04.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.593+0000 7fb885600700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb880075700 0x7fb88019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:04.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.593+0000 7fb884c00700 1 -- 172.21.15.115:0/3711074261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb880075700 msgr2=0x7fb88019bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:04.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.594+0000 7fb884c00700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb880075700 0x7fb88019bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.594+0000 7fb884c00700 1 -- 172.21.15.115:0/3711074261 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740095f0 con 0x7fb880076950 2024-03-22T10:47:04.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.594+0000 7fb885600700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb880075700 0x7fb88019bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:47:04.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.594+0000 7fb884c00700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880076950 0x7fb88019c4d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00d8d0 tx=0x7fb86c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:04.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.594+0000 7fb87e000700 1 -- 172.21.15.115:0/3711074261 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86c009940 con 0x7fb880076950 2024-03-22T10:47:04.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.594+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8801a16e0 con 0x7fb880076950 2024-03-22T10:47:04.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.594+0000 7fb87e000700 1 -- 172.21.15.115:0/3711074261 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb86c010460 con 0x7fb880076950 2024-03-22T10:47:04.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.594+0000 7fb87e000700 1 -- 172.21.15.115:0/3711074261 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86c00f5f0 con 0x7fb880076950 2024-03-22T10:47:04.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.594+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8801a1c30 con 0x7fb880076950 2024-03-22T10:47:04.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.596+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88010f2d0 con 0x7fb880076950 2024-03-22T10:47:04.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.597+0000 7fb87e000700 1 -- 172.21.15.115:0/3711074261 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb86c0105d0 con 0x7fb880076950 2024-03-22T10:47:04.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.598+0000 7fb87e000700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb86806dfa0 0x7fb868070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:04.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.598+0000 7fb87e000700 1 -- 172.21.15.115:0/3711074261 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb86c08dbe0 con 0x7fb880076950 2024-03-22T10:47:04.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.598+0000 7fb885600700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb86806dfa0 0x7fb868070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:04.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.599+0000 7fb885600700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb86806dfa0 0x7fb868070450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb874006010 tx=0x7fb87400b580 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:04.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.603+0000 7fb87e000700 1 -- 172.21.15.115:0/3711074261 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb86c05ac60 con 0x7fb880076950 2024-03-22T10:47:04.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.950+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb8801a22c0 con 0x7fb880076950 2024-03-22T10:47:04.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.951+0000 7fb87e000700 1 -- 172.21.15.115:0/3711074261 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fb86c05a7f0 con 0x7fb880076950 2024-03-22T10:47:04.941 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:47:04.941 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:47:04.941 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:47:04.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 8, 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:47:04.942 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:47:04.943 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:47:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb86806dfa0 msgr2=0x7fb868070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb86806dfa0 0x7fb868070450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb874006010 tx=0x7fb87400b580 comp rx=0 tx=0).stop 2024-03-22T10:47:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880076950 msgr2=0x7fb88019c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880076950 0x7fb88019c4d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00d8d0 tx=0x7fb86c00dc90 comp rx=0 tx=0).stop 2024-03-22T10:47:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 shutdown_connections 2024-03-22T10:47:04.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb86806dfa0 0x7fb868070450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb880075700 0x7fb88019bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 --2- 172.21.15.115:0/3711074261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb880076950 0x7fb88019c4d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:04.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 >> 172.21.15.115:0/3711074261 conn(0x7fb8800fd930 msgr2=0x7fb88010c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:04.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 shutdown_connections 2024-03-22T10:47:04.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:04.954+0000 7fb888000700 1 -- 172.21.15.115:0/3711074261 wait complete. 2024-03-22T10:47:05.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.110+0000 7f9a17400700 1 -- 172.21.15.115:0/2047904568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10073750 msgr2=0x7f9a10073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:05.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.110+0000 7f9a17400700 1 --2- 172.21.15.115:0/2047904568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10073750 0x7f9a10073bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009910 tx=0x7f99fc009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:05.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.110+0000 7f9a17400700 1 -- 172.21.15.115:0/2047904568 shutdown_connections 2024-03-22T10:47:05.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.110+0000 7f9a17400700 1 --2- 172.21.15.115:0/2047904568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10073750 0x7f9a10073bc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.110+0000 7f9a17400700 1 --2- 172.21.15.115:0/2047904568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a10074d20 0x7f9a10073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.110+0000 7f9a17400700 1 -- 172.21.15.115:0/2047904568 >> 172.21.15.115:0/2047904568 conn(0x7f9a100fb6e0 msgr2=0x7f9a100fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:05.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.110+0000 7f9a17400700 1 -- 172.21.15.115:0/2047904568 shutdown_connections 2024-03-22T10:47:05.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.110+0000 7f9a17400700 1 -- 172.21.15.115:0/2047904568 wait complete. 2024-03-22T10:47:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.111+0000 7f9a17400700 1 Processor -- start 2024-03-22T10:47:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.111+0000 7f9a17400700 1 -- start start 2024-03-22T10:47:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.112+0000 7f9a17400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a10073750 0x7f9a10197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.112+0000 7f9a17400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10074d20 0x7f9a10198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.112+0000 7f9a17400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a101986b0 con 0x7f9a10074d20 2024-03-22T10:47:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.112+0000 7f9a17400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a101987f0 con 0x7f9a10073750 2024-03-22T10:47:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.112+0000 7f9a0fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10074d20 0x7f9a10198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.112+0000 7f9a0fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10074d20 0x7f9a10198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34536/0 (socket says 172.21.15.115:34536) 2024-03-22T10:47:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.112+0000 7f9a0fe00700 1 -- 172.21.15.115:0/799437546 learned_addr learned my addr 172.21.15.115:0/799437546 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.112+0000 7f9a14a00700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a10073750 0x7f9a10197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.113+0000 7f9a0fe00700 1 -- 172.21.15.115:0/799437546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a10073750 msgr2=0x7f9a10197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.113+0000 7f9a0fe00700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a10073750 0x7f9a10197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.113+0000 7f9a0fe00700 1 -- 172.21.15.115:0/799437546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0095f0 con 0x7f9a10074d20 2024-03-22T10:47:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.113+0000 7f9a0fe00700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10074d20 0x7f9a10198090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00bd70 tx=0x7f99fc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.113+0000 7f9a0d600700 1 -- 172.21.15.115:0/799437546 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99fc01d070 con 0x7f9a10074d20 2024-03-22T10:47:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.113+0000 7f9a0d600700 1 -- 172.21.15.115:0/799437546 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f99fc005dc0 con 0x7f9a10074d20 2024-03-22T10:47:05.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.113+0000 7f9a0d600700 1 -- 172.21.15.115:0/799437546 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99fc021410 con 0x7f9a10074d20 2024-03-22T10:47:05.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.113+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1019d240 con 0x7f9a10074d20 2024-03-22T10:47:05.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.114+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1019d730 con 0x7f9a10074d20 2024-03-22T10:47:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.115+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1010ae90 con 0x7f9a10074d20 2024-03-22T10:47:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.116+0000 7f9a0d600700 1 -- 172.21.15.115:0/799437546 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f99fc00f7d0 con 0x7f9a10074d20 2024-03-22T10:47:05.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.117+0000 7f9a0d600700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f99f806dfa0 0x7f99f8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:05.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.117+0000 7f9a0d600700 1 -- 172.21.15.115:0/799437546 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f99fc08f060 con 0x7f9a10074d20 2024-03-22T10:47:05.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.117+0000 7f9a14a00700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f99f806dfa0 0x7f99f8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:05.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.118+0000 7f9a14a00700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f99f806dfa0 0x7f99f8070450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a04005950 tx=0x7f9a0400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:05.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.126+0000 7f9a0d600700 1 -- 172.21.15.115:0/799437546 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99fc05bfc0 con 0x7f9a10074d20 2024-03-22T10:47:05.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:04 smithi115 ceph-mon[78672]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:05.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:04 smithi115 ceph-mon[78672]: from='osd.1 [v2:172.21.15.115:6810/3813430676,v1:172.21.15.115:6811/3813430676]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T10:47:05.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:04 smithi115 ceph-mon[78672]: osdmap e56: 6 total, 5 up, 6 in 2024-03-22T10:47:05.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:04 smithi115 ceph-mon[78672]: from='osd.1 [v2:172.21.15.115:6810/3813430676,v1:172.21.15.115:6811/3813430676]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:47:05.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:05.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:05.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:05.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:47:05.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:04 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 57/369 objects degraded (15.447%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:05.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.397+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9a1019ddc0 con 0x7f9a10074d20 2024-03-22T10:47:05.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.398+0000 7f9a0d600700 1 -- 172.21.15.115:0/799437546 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1924 (secure 0 0 0) 0x7f99fc00f460 con 0x7f9a10074d20 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout:e15 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 15 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:47:05.389 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:46:58.862168+0000 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:47:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 14454 members: 14454,24241 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:47:05.391 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f99f806dfa0 msgr2=0x7f99f8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f99f806dfa0 0x7f99f8070450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a04005950 tx=0x7f9a0400b410 comp rx=0 tx=0).stop 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10074d20 msgr2=0x7f9a10198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10074d20 0x7f9a10198090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00bd70 tx=0x7f99fc00bda0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 shutdown_connections 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f99f806dfa0 0x7f99f8070450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a10073750 0x7f9a10197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 --2- 172.21.15.115:0/799437546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a10074d20 0x7f9a10198090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 >> 172.21.15.115:0/799437546 conn(0x7f9a100fb6e0 msgr2=0x7f9a10105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 shutdown_connections 2024-03-22T10:47:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.402+0000 7f9a17400700 1 -- 172.21.15.115:0/799437546 wait complete. 2024-03-22T10:47:05.394 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 15 2024-03-22T10:47:05.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:04 smithi150 ceph-mon[65513]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:05.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:04 smithi150 ceph-mon[65513]: from='osd.1 [v2:172.21.15.115:6810/3813430676,v1:172.21.15.115:6811/3813430676]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T10:47:05.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:04 smithi150 ceph-mon[65513]: osdmap e56: 6 total, 5 up, 6 in 2024-03-22T10:47:05.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:04 smithi150 ceph-mon[65513]: from='osd.1 [v2:172.21.15.115:6810/3813430676,v1:172.21.15.115:6811/3813430676]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:47:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:47:05.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:04 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 57/369 objects degraded (15.447%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.556+0000 7feea8000700 1 -- 172.21.15.115:0/230682279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 msgr2=0x7feea01017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.556+0000 7feea8000700 1 --2- 172.21.15.115:0/230682279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 0x7feea01017a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009910 tx=0x7fee8c009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.556+0000 7feea8000700 1 -- 172.21.15.115:0/230682279 shutdown_connections 2024-03-22T10:47:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.556+0000 7feea8000700 1 --2- 172.21.15.115:0/230682279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 0x7feea01017a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.556+0000 7feea8000700 1 --2- 172.21.15.115:0/230682279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea0100150 0x7feea0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.556+0000 7feea8000700 1 -- 172.21.15.115:0/230682279 >> 172.21.15.115:0/230682279 conn(0x7feea00fb6e0 msgr2=0x7feea00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.556+0000 7feea8000700 1 -- 172.21.15.115:0/230682279 shutdown_connections 2024-03-22T10:47:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.556+0000 7feea8000700 1 -- 172.21.15.115:0/230682279 wait complete. 2024-03-22T10:47:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.557+0000 7feea8000700 1 Processor -- start 2024-03-22T10:47:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.557+0000 7feea8000700 1 -- start start 2024-03-22T10:47:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.558+0000 7feea8000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea0100150 0x7feea0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.558+0000 7feea8000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 0x7feea0073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.558+0000 7feea8000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea0073680 con 0x7feea0101350 2024-03-22T10:47:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.558+0000 7feea8000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00737c0 con 0x7feea0100150 2024-03-22T10:47:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.558+0000 7feea4c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 0x7feea0073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.558+0000 7feea4c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 0x7feea0073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34548/0 (socket says 172.21.15.115:34548) 2024-03-22T10:47:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.558+0000 7feea4c00700 1 -- 172.21.15.115:0/3086446798 learned_addr learned my addr 172.21.15.115:0/3086446798 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.558+0000 7feea5600700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea0100150 0x7feea0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.559+0000 7feea4c00700 1 -- 172.21.15.115:0/3086446798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea0100150 msgr2=0x7feea0074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.559+0000 7feea4c00700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea0100150 0x7feea0074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.559+0000 7feea4c00700 1 -- 172.21.15.115:0/3086446798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c0095f0 con 0x7feea0101350 2024-03-22T10:47:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.559+0000 7feea5600700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea0100150 0x7feea0074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:47:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.559+0000 7feea4c00700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 0x7feea0073140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00bd70 tx=0x7fee8c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.559+0000 7fee9e000700 1 -- 172.21.15.115:0/3086446798 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee8c01d070 con 0x7feea0101350 2024-03-22T10:47:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.559+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea0073a40 con 0x7feea0101350 2024-03-22T10:47:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.559+0000 7fee9e000700 1 -- 172.21.15.115:0/3086446798 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee8c022470 con 0x7feea0101350 2024-03-22T10:47:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.559+0000 7fee9e000700 1 -- 172.21.15.115:0/3086446798 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee8c00f670 con 0x7feea0101350 2024-03-22T10:47:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.560+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea0073f30 con 0x7feea0101350 2024-03-22T10:47:05.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.561+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea0108d40 con 0x7feea0101350 2024-03-22T10:47:05.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.561+0000 7fee9e000700 1 -- 172.21.15.115:0/3086446798 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fee8c022a50 con 0x7feea0101350 2024-03-22T10:47:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.562+0000 7fee9e000700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fee8806dfa0 0x7fee88070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.562+0000 7fee9e000700 1 -- 172.21.15.115:0/3086446798 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fee8c08f390 con 0x7feea0101350 2024-03-22T10:47:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.563+0000 7feea5600700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fee8806dfa0 0x7fee88070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:05.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.563+0000 7feea5600700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fee8806dfa0 0x7fee88070450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feea01011b0 tx=0x7fee94008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.567+0000 7fee9e000700 1 -- 172.21.15.115:0/3086446798 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee8c05c420 con 0x7feea0101350 2024-03-22T10:47:05.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.832+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feea0105ca0 con 0x7fee8806dfa0 2024-03-22T10:47:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.833+0000 7fee9e000700 1 -- 172.21.15.115:0/3086446798 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7feea0105ca0 con 0x7fee8806dfa0 2024-03-22T10:47:05.824 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "8/23 daemons upgraded", 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:47:05.825 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.836+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fee8806dfa0 msgr2=0x7fee88070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.836+0000 7feea8000700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fee8806dfa0 0x7fee88070450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feea01011b0 tx=0x7fee94008040 comp rx=0 tx=0).stop 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.836+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 msgr2=0x7feea0073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.836+0000 7feea8000700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 0x7feea0073140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00bd70 tx=0x7fee8c00be50 comp rx=0 tx=0).stop 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.836+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 shutdown_connections 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.836+0000 7feea8000700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fee8806dfa0 0x7fee88070450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.836+0000 7feea8000700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea0100150 0x7feea0074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.836+0000 7feea8000700 1 --2- 172.21.15.115:0/3086446798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea0101350 0x7feea0073140 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.836+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 >> 172.21.15.115:0/3086446798 conn(0x7feea00fb6e0 msgr2=0x7feea0104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.837+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 shutdown_connections 2024-03-22T10:47:05.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:05.837+0000 7feea8000700 1 -- 172.21.15.115:0/3086446798 wait complete. 2024-03-22T10:47:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.019+0000 7f1dae600700 1 -- 172.21.15.115:0/1621686536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 msgr2=0x7f1da81017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.019+0000 7f1dae600700 1 --2- 172.21.15.115:0/1621686536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 0x7f1da81017a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009910 tx=0x7f1d94009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.019+0000 7f1dae600700 1 -- 172.21.15.115:0/1621686536 shutdown_connections 2024-03-22T10:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.019+0000 7f1dae600700 1 --2- 172.21.15.115:0/1621686536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 0x7f1da81017a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.019+0000 7f1dae600700 1 --2- 172.21.15.115:0/1621686536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da8100150 0x7f1da8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.019+0000 7f1dae600700 1 -- 172.21.15.115:0/1621686536 >> 172.21.15.115:0/1621686536 conn(0x7f1da80fb6e0 msgr2=0x7f1da80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.020+0000 7f1dae600700 1 -- 172.21.15.115:0/1621686536 shutdown_connections 2024-03-22T10:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.020+0000 7f1dae600700 1 -- 172.21.15.115:0/1621686536 wait complete. 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.021+0000 7f1dae600700 1 Processor -- start 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.021+0000 7f1dae600700 1 -- start start 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.021+0000 7f1dae600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da8100150 0x7f1da8074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.021+0000 7f1dae600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 0x7f1da8073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.021+0000 7f1dae600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da8073680 con 0x7f1da8101350 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.021+0000 7f1dae600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80737c0 con 0x7f1da8100150 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.022+0000 7f1da6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 0x7f1da8073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.022+0000 7f1da6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 0x7f1da8073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34572/0 (socket says 172.21.15.115:34572) 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.022+0000 7f1da6a00700 1 -- 172.21.15.115:0/3721974540 learned_addr learned my addr 172.21.15.115:0/3721974540 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.022+0000 7f1da7400700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da8100150 0x7f1da8074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.022+0000 7f1da6a00700 1 -- 172.21.15.115:0/3721974540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da8100150 msgr2=0x7f1da8074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.022+0000 7f1da6a00700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da8100150 0x7f1da8074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.022+0000 7f1da6a00700 1 -- 172.21.15.115:0/3721974540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940095f0 con 0x7f1da8101350 2024-03-22T10:47:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.022+0000 7f1da6a00700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 0x7f1da8073140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b5c0 tx=0x7f1d9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.023+0000 7f1d9fe00700 1 -- 172.21.15.115:0/3721974540 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9401d070 con 0x7f1da8101350 2024-03-22T10:47:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.023+0000 7f1d9fe00700 1 -- 172.21.15.115:0/3721974540 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1d94022470 con 0x7f1da8101350 2024-03-22T10:47:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.023+0000 7f1d9fe00700 1 -- 172.21.15.115:0/3721974540 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9400f690 con 0x7f1da8101350 2024-03-22T10:47:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.023+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da8073a40 con 0x7f1da8101350 2024-03-22T10:47:06.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.023+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da8073ee0 con 0x7f1da8101350 2024-03-22T10:47:06.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.024+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da8108d40 con 0x7f1da8101350 2024-03-22T10:47:06.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.025+0000 7f1d9fe00700 1 -- 172.21.15.115:0/3721974540 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f1d94004b70 con 0x7f1da8101350 2024-03-22T10:47:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.026+0000 7f1d9fe00700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1d8c06dfa0 0x7f1d8c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.026+0000 7f1d9fe00700 1 -- 172.21.15.115:0/3721974540 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1d9408f510 con 0x7f1da8101350 2024-03-22T10:47:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.026+0000 7f1da7400700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1d8c06dfa0 0x7f1d8c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.027+0000 7f1da7400700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1d8c06dfa0 0x7f1d8c070450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d98006fd0 tx=0x7f1d98008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:06.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.031+0000 7f1d9fe00700 1 -- 172.21.15.115:0/3721974540 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d9405c520 con 0x7f1da8101350 2024-03-22T10:47:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: from='client.34276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: pgmap v101: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:47:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:47:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: osd.1 [v2:172.21.15.115:6810/3813430676,v1:172.21.15.115:6811/3813430676] boot 2024-03-22T10:47:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: osdmap e57: 6 total, 6 up, 6 in 2024-03-22T10:47:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:47:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3711074261' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/799437546' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:47:06.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:05 smithi115 ceph-mon[78672]: osdmap e58: 6 total, 6 up, 6 in 2024-03-22T10:47:06.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.345+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1da8108f30 con 0x7f1da8101350 2024-03-22T10:47:06.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.346+0000 7f1d9fe00700 1 -- 172.21.15.115:0/3721974540 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1261 (secure 0 0 0) 0x7f1d94027500 con 0x7f1da8101350 2024-03-22T10:47:06.337 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 57/369 objects degraded (15.447%), 16 pgs degraded 2024-03-22T10:47:06.337 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:47:06.337 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 57/369 objects degraded (15.447%), 16 pgs degraded 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-03-22T10:47:06.338 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-03-22T10:47:06.339 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-22T10:47:06.339 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-03-22T10:47:06.339 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-03-22T10:47:06.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.349+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1d8c06dfa0 msgr2=0x7f1d8c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:06.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.349+0000 7f1dae600700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1d8c06dfa0 0x7f1d8c070450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d98006fd0 tx=0x7f1d98008040 comp rx=0 tx=0).stop 2024-03-22T10:47:06.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.349+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 msgr2=0x7f1da8073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.349+0000 7f1dae600700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 0x7f1da8073140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b5c0 tx=0x7f1d9400be20 comp rx=0 tx=0).stop 2024-03-22T10:47:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.349+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 shutdown_connections 2024-03-22T10:47:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.349+0000 7f1dae600700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1d8c06dfa0 0x7f1d8c070450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.349+0000 7f1dae600700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da8100150 0x7f1da8074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.349+0000 7f1dae600700 1 --2- 172.21.15.115:0/3721974540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1da8101350 0x7f1da8073140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.349+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 >> 172.21.15.115:0/3721974540 conn(0x7f1da80fb6e0 msgr2=0x7f1da8104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.350+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 shutdown_connections 2024-03-22T10:47:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:06.350+0000 7f1dae600700 1 -- 172.21.15.115:0/3721974540 wait complete. 2024-03-22T10:47:06.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: from='client.34276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:06.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: pgmap v101: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 57/369 objects degraded (15.447%) 2024-03-22T10:47:06.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:06.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:47:06.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: osd.1 [v2:172.21.15.115:6810/3813430676,v1:172.21.15.115:6811/3813430676] boot 2024-03-22T10:47:06.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: osdmap e57: 6 total, 6 up, 6 in 2024-03-22T10:47:06.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T10:47:06.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3711074261' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:06.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/799437546' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:47:06.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:05 smithi150 ceph-mon[65513]: osdmap e58: 6 total, 6 up, 6 in 2024-03-22T10:47:07.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:06 smithi150 ceph-mon[65513]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:07.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:06 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:07.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:06 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:07.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:06 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3721974540' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:47:07.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:06 smithi115 ceph-mon[78672]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:07.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:06 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:07.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:06 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:07.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:06 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3721974540' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:47:08.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:07 smithi115 ceph-mon[78672]: pgmap v104: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24/369 objects degraded (6.504%) 2024-03-22T10:47:08.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:07 smithi150 ceph-mon[65513]: pgmap v104: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24/369 objects degraded (6.504%) 2024-03-22T10:47:09.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:08 smithi115 ceph-mon[78672]: pgmap v105: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24/369 objects degraded (6.504%) 2024-03-22T10:47:09.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:08 smithi150 ceph-mon[65513]: pgmap v105: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24/369 objects degraded (6.504%) 2024-03-22T10:47:10.252 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:09 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 24/369 objects degraded (6.504%), 7 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:10.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:09 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 24/369 objects degraded (6.504%), 7 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:11.611 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: pgmap v106: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T10:47:11.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: Upgrade: osd.2 is safe to restart 2024-03-22T10:47:11.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:11 smithi115 ceph-mon[78672]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/369 objects degraded (6.504%), 7 pgs degraded) 2024-03-22T10:47:11.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: pgmap v106: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: Upgrade: osd.2 is safe to restart 2024-03-22T10:47:11.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:11 smithi150 ceph-mon[65513]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/369 objects degraded (6.504%), 7 pgs degraded) 2024-03-22T10:47:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:13 smithi150 ceph-mon[65513]: pgmap v107: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T10:47:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:47:13.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:13 smithi115 ceph-mon[78672]: pgmap v107: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:13.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:13.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T10:47:13.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:47:14.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:14 smithi115 ceph-mon[78672]: Upgrade: Updating osd.2 2024-03-22T10:47:14.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:14 smithi115 ceph-mon[78672]: Deploying daemon osd.2 on smithi115 2024-03-22T10:47:14.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:14 smithi150 ceph-mon[65513]: Upgrade: Updating osd.2 2024-03-22T10:47:14.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:14 smithi150 ceph-mon[65513]: Deploying daemon osd.2 on smithi115 2024-03-22T10:47:15.622 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:15 smithi115 ceph-mon[78672]: pgmap v108: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:15.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:15 smithi150 ceph-mon[65513]: pgmap v108: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:17.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:17 smithi150 ceph-mon[65513]: pgmap v109: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:17.697 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:17 smithi115 ceph-mon[78672]: pgmap v109: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:18.010 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:17 smithi115 systemd[1]: Stopping Ceph osd.2 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:47:18.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:18 smithi115 ceph-mon[78672]: osd.2 marked itself down and dead 2024-03-22T10:47:18.511 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:18 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[55009]: 2024-03-22T10:47:18.178+0000 7f1dce000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T10:47:18.512 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:18 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[55009]: 2024-03-22T10:47:18.178+0000 7f1dce000700 -1 osd.2 58 *** Got signal Terminated *** 2024-03-22T10:47:18.512 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:18 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[55009]: 2024-03-22T10:47:18.178+0000 7f1dce000700 -1 osd.2 58 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T10:47:18.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:18 smithi150 ceph-mon[65513]: osd.2 marked itself down and dead 2024-03-22T10:47:19.244 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:18 smithi115 podman[91522]: 2024-03-22 10:47:18.939095874 +0000 UTC m=+0.981235365 container died 52d9f92f53a844ad108c4c685d275bcaed35a860a7930e16b7a74ae07d2063b0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1) 2024-03-22T10:47:19.510 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:19 smithi115 podman[91522]: 2024-03-22 10:47:19.379975926 +0000 UTC m=+1.422115457 container cleanup 52d9f92f53a844ad108c4c685d275bcaed35a860a7930e16b7a74ae07d2063b0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-22T10:47:19.511 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:19 smithi115 bash[91522]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2 2024-03-22T10:47:19.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:19 smithi115 ceph-mon[78672]: pgmap v110: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:19.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:19 smithi115 ceph-mon[78672]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:47:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:19 smithi115 ceph-mon[78672]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:47:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:19 smithi115 ceph-mon[78672]: osdmap e59: 6 total, 5 up, 6 in 2024-03-22T10:47:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:47:19.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:19.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:19 smithi150 ceph-mon[65513]: pgmap v110: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:19.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:19 smithi150 ceph-mon[65513]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:47:19.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:19 smithi150 ceph-mon[65513]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:active} 2 up:standby 2024-03-22T10:47:19.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:19 smithi150 ceph-mon[65513]: osdmap e59: 6 total, 5 up, 6 in 2024-03-22T10:47:19.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:19.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:47:19.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:20.010 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:19 smithi115 podman[91541]: 2024-03-22 10:47:19.58124253 +0000 UTC m=+0.626534950 container remove 52d9f92f53a844ad108c4c685d275bcaed35a860a7930e16b7a74ae07d2063b0 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T10:47:20.619 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:20 smithi115 ceph-mon[78672]: osdmap e60: 6 total, 5 up, 6 in 2024-03-22T10:47:20.619 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:20 smithi115 podman[91627]: 2024-03-22 10:47:20.267400603 +0000 UTC m=+0.031818326 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:47:20.619 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:20 smithi115 podman[91627]: 2024-03-22 10:47:20.373879231 +0000 UTC m=+0.138296995 container create 86b32688768dfce6b9dabb502b763e4b62c8ab32da6e17c1e0c298838e7dca08 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T10:47:20.619 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:20 smithi115 podman[91627]: 2024-03-22 10:47:20.627319829 +0000 UTC m=+0.391737597 container init 86b32688768dfce6b9dabb502b763e4b62c8ab32da6e17c1e0c298838e7dca08 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-22T10:47:20.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:20 smithi150 ceph-mon[65513]: osdmap e60: 6 total, 5 up, 6 in 2024-03-22T10:47:21.010 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:20 smithi115 podman[91627]: 2024-03-22 10:47:20.637455116 +0000 UTC m=+0.401872873 container start 86b32688768dfce6b9dabb502b763e4b62c8ab32da6e17c1e0c298838e7dca08 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-22T10:47:21.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:20 smithi115 podman[91627]: 2024-03-22 10:47:20.743050013 +0000 UTC m=+0.507467781 container attach 86b32688768dfce6b9dabb502b763e4b62c8ab32da6e17c1e0c298838e7dca08 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-22T10:47:21.401 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:21 smithi115 podman[91627]: 2024-03-22 10:47:21.084273235 +0000 UTC m=+0.848691026 container died 86b32688768dfce6b9dabb502b763e4b62c8ab32da6e17c1e0c298838e7dca08 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-22T10:47:21.712 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:21 smithi115 ceph-mon[78672]: pgmap v113: 65 pgs: 6 active+undersized+degraded, 15 peering, 6 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18/369 objects degraded (4.878%) 2024-03-22T10:47:21.712 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:21 smithi115 ceph-mon[78672]: Health check failed: Reduced data availability: 7 pgs peering (PG_AVAILABILITY) 2024-03-22T10:47:21.712 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:21 smithi115 ceph-mon[78672]: Health check failed: Degraded data redundancy: 18/369 objects degraded (4.878%), 6 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:21 smithi150 ceph-mon[65513]: pgmap v113: 65 pgs: 6 active+undersized+degraded, 15 peering, 6 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18/369 objects degraded (4.878%) 2024-03-22T10:47:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:21 smithi150 ceph-mon[65513]: Health check failed: Reduced data availability: 7 pgs peering (PG_AVAILABILITY) 2024-03-22T10:47:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:21 smithi150 ceph-mon[65513]: Health check failed: Degraded data redundancy: 18/369 objects degraded (4.878%), 6 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:22.010 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:21 smithi115 podman[91652]: 2024-03-22 10:47:21.721203159 +0000 UTC m=+0.616058389 container remove 86b32688768dfce6b9dabb502b763e4b62c8ab32da6e17c1e0c298838e7dca08 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-22T10:47:22.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:21 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-22T10:47:22.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:21 smithi115 systemd[1]: Stopped Ceph osd.2 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:47:22.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:21 smithi115 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.2.service: Consumed 1min 18.455s CPU time. 2024-03-22T10:47:22.510 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:22 smithi115 systemd[1]: Starting Ceph osd.2 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:47:22.809 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:22 smithi115 podman[91751]: 2024-03-22 10:47:22.451936741 +0000 UTC m=+0.032776504 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:47:22.809 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:22 smithi115 podman[91751]: 2024-03-22 10:47:22.555924064 +0000 UTC m=+0.136763864 container create ef73698b5915ab4ca74687f04189f0946c5c104d62020ac3c30374cc0fa9a512 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-22T10:47:22.810 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:22 smithi115 podman[91751]: 2024-03-22 10:47:22.818191394 +0000 UTC m=+0.399031201 container init ef73698b5915ab4ca74687f04189f0946c5c104d62020ac3c30374cc0fa9a512 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-22T10:47:23.260 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:22 smithi115 podman[91751]: 2024-03-22 10:47:22.830375528 +0000 UTC m=+0.411215365 container start ef73698b5915ab4ca74687f04189f0946c5c104d62020ac3c30374cc0fa9a512 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:47:23.260 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:22 smithi115 podman[91751]: 2024-03-22 10:47:22.916776892 +0000 UTC m=+0.497616738 container attach ef73698b5915ab4ca74687f04189f0946c5c104d62020ac3c30374cc0fa9a512 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-22T10:47:23.680 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:23 smithi115 ceph-mon[78672]: pgmap v114: 65 pgs: 6 active+undersized+degraded, 15 peering, 6 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18/369 objects degraded (4.878%) 2024-03-22T10:47:23.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:23 smithi150 ceph-mon[65513]: pgmap v114: 65 pgs: 6 active+undersized+degraded, 15 peering, 6 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18/369 objects degraded (4.878%) 2024-03-22T10:47:24.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate[91766]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T10:47:24.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate[91766]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T10:47:24.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 bash[91751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T10:47:24.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 bash[91751]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T10:47:24.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 bash[91751]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T10:47:24.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 bash[91751]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T10:47:24.011 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 bash[91751]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-22T10:47:24.012 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 bash[91751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T10:47:24.012 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 bash[91751]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-22T10:47:24.012 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate[91766]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T10:47:24.012 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate[91766]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T10:47:24.012 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate[91766]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-22T10:47:24.012 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate[91766]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T10:47:24.012 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate[91766]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-22T10:47:24.012 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:23 smithi115 podman[91751]: 2024-03-22 10:47:23.750418796 +0000 UTC m=+1.331258623 container died ef73698b5915ab4ca74687f04189f0946c5c104d62020ac3c30374cc0fa9a512 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T10:47:24.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:24 smithi115 ceph-mon[78672]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs peering) 2024-03-22T10:47:24.760 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:24 smithi115 podman[91890]: 2024-03-22 10:47:24.595847663 +0000 UTC m=+0.825414088 container remove ef73698b5915ab4ca74687f04189f0946c5c104d62020ac3c30374cc0fa9a512 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-22T10:47:24.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:24 smithi150 ceph-mon[65513]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs peering) 2024-03-22T10:47:25.233 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:24 smithi115 podman[91949]: 2024-03-22 10:47:24.845457489 +0000 UTC m=+0.033275128 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:47:25.233 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:24 smithi115 podman[91949]: 2024-03-22 10:47:24.954902737 +0000 UTC m=+0.142720400 container create 2e695f9baca8a550d04baaabbce66e3fd5e5ba7f7a1e5e89d00880ff8ff83f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T10:47:25.233 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:25 smithi115 podman[91949]: 2024-03-22 10:47:25.241916265 +0000 UTC m=+0.429733934 container init 2e695f9baca8a550d04baaabbce66e3fd5e5ba7f7a1e5e89d00880ff8ff83f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-22T10:47:25.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:25 smithi115 ceph-mon[78672]: pgmap v115: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:25.511 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:25 smithi115 podman[91949]: 2024-03-22 10:47:25.254239394 +0000 UTC m=+0.442057025 container start 2e695f9baca8a550d04baaabbce66e3fd5e5ba7f7a1e5e89d00880ff8ff83f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T10:47:25.511 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:25 smithi115 ceph-osd[91968]: -- 172.21.15.115:0/3027131753 <== mon.0 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5562de01d680 con 0x5562ddfe7000 2024-03-22T10:47:25.511 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:25 smithi115 bash[91949]: 2e695f9baca8a550d04baaabbce66e3fd5e5ba7f7a1e5e89d00880ff8ff83f52 2024-03-22T10:47:25.511 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:25 smithi115 systemd[1]: Started Ceph osd.2 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:47:25.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:25 smithi150 ceph-mon[65513]: pgmap v115: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:26.178 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:25 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[91964]: 2024-03-22T10:47:25.819+0000 7f157da3a7c0 -1 Falling back to public interface 2024-03-22T10:47:26.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:26.786 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:26.786 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:47:26.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:26.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:26.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:47:27.879 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:27 smithi115 ceph-mon[78672]: pgmap v116: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:27.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:27 smithi150 ceph-mon[65513]: pgmap v116: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:29.795 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:29 smithi115 ceph-mon[78672]: pgmap v117: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:29.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:29 smithi150 ceph-mon[65513]: pgmap v117: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:30.655 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:30 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 45/369 objects degraded (12.195%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:30.656 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:30 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[91964]: 2024-03-22T10:47:30.366+0000 7f157da3a7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-22T10:47:30.656 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:30 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[91964]: 2024-03-22T10:47:30.663+0000 7f157da3a7c0 -1 osd.2 58 log_to_monitors true 2024-03-22T10:47:30.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:30 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 45/369 objects degraded (12.195%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:31.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:31 smithi115 ceph-mon[78672]: pgmap v118: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:31.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:31 smithi115 ceph-mon[78672]: from='osd.2 [v2:172.21.15.115:6818/3285562920,v1:172.21.15.115:6819/3285562920]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T10:47:31.761 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 10:47:31 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[91964]: 2024-03-22T10:47:31.573+0000 7f156ce00700 -1 osd.2 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T10:47:31.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:31 smithi150 ceph-mon[65513]: pgmap v118: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:31.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:31 smithi150 ceph-mon[65513]: from='osd.2 [v2:172.21.15.115:6818/3285562920,v1:172.21.15.115:6819/3285562920]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T10:47:32.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:32 smithi150 ceph-mon[65513]: from='osd.2 [v2:172.21.15.115:6818/3285562920,v1:172.21.15.115:6819/3285562920]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T10:47:32.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:32 smithi150 ceph-mon[65513]: osdmap e61: 6 total, 5 up, 6 in 2024-03-22T10:47:32.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:32 smithi150 ceph-mon[65513]: from='osd.2 [v2:172.21.15.115:6818/3285562920,v1:172.21.15.115:6819/3285562920]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:47:32.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:32 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:33.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:32 smithi115 ceph-mon[78672]: from='osd.2 [v2:172.21.15.115:6818/3285562920,v1:172.21.15.115:6819/3285562920]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T10:47:33.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:32 smithi115 ceph-mon[78672]: osdmap e61: 6 total, 5 up, 6 in 2024-03-22T10:47:33.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:32 smithi115 ceph-mon[78672]: from='osd.2 [v2:172.21.15.115:6818/3285562920,v1:172.21.15.115:6819/3285562920]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-22T10:47:33.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:32 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:33.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:33 smithi150 ceph-mon[65513]: pgmap v120: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:33.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:33 smithi150 ceph-mon[65513]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:47:33.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:33 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:33.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:33 smithi150 ceph-mon[65513]: osd.2 [v2:172.21.15.115:6818/3285562920,v1:172.21.15.115:6819/3285562920] boot 2024-03-22T10:47:33.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:33 smithi150 ceph-mon[65513]: osdmap e62: 6 total, 6 up, 6 in 2024-03-22T10:47:33.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:33 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:47:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:33 smithi115 ceph-mon[78672]: pgmap v120: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-03-22T10:47:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:33 smithi115 ceph-mon[78672]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:47:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:33 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:33 smithi115 ceph-mon[78672]: osd.2 [v2:172.21.15.115:6818/3285562920,v1:172.21.15.115:6819/3285562920] boot 2024-03-22T10:47:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:33 smithi115 ceph-mon[78672]: osdmap e62: 6 total, 6 up, 6 in 2024-03-22T10:47:34.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:33 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T10:47:34.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:34 smithi150 ceph-mon[65513]: osdmap e63: 6 total, 6 up, 6 in 2024-03-22T10:47:34.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:34.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:47:34.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:34.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:35.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:34 smithi115 ceph-mon[78672]: osdmap e63: 6 total, 6 up, 6 in 2024-03-22T10:47:35.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:35.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:47:35.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:35.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:35.706 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:35 smithi115 ceph-mon[78672]: pgmap v123: 65 pgs: 10 active+undersized+degraded, 9 active+undersized, 46 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 33/369 objects degraded (8.943%) 2024-03-22T10:47:35.706 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:35 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 33/369 objects degraded (8.943%), 10 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:35.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:35 smithi150 ceph-mon[65513]: pgmap v123: 65 pgs: 10 active+undersized+degraded, 9 active+undersized, 46 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 33/369 objects degraded (8.943%) 2024-03-22T10:47:35.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:35 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 33/369 objects degraded (8.943%), 10 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:36.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.520+0000 7f5d0c400700 1 -- 172.21.15.115:0/1543109169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041079a0 msgr2=0x7f5d04107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:36.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.520+0000 7f5d0c400700 1 --2- 172.21.15.115:0/1543109169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041079a0 0x7f5d04107df0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009910 tx=0x7f5cf0009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:36.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.520+0000 7f5d0c400700 1 -- 172.21.15.115:0/1543109169 shutdown_connections 2024-03-22T10:47:36.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.520+0000 7f5d0c400700 1 --2- 172.21.15.115:0/1543109169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041079a0 0x7f5d04107df0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.520+0000 7f5d0c400700 1 --2- 172.21.15.115:0/1543109169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d041067a0 0x7f5d04106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.520+0000 7f5d0c400700 1 -- 172.21.15.115:0/1543109169 >> 172.21.15.115:0/1543109169 conn(0x7f5d04075910 msgr2=0x7f5d04077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:36.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.521+0000 7f5d0c400700 1 -- 172.21.15.115:0/1543109169 shutdown_connections 2024-03-22T10:47:36.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.521+0000 7f5d0c400700 1 -- 172.21.15.115:0/1543109169 wait complete. 2024-03-22T10:47:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.522+0000 7f5d0c400700 1 Processor -- start 2024-03-22T10:47:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.522+0000 7f5d0c400700 1 -- start start 2024-03-22T10:47:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.522+0000 7f5d0c400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041067a0 0x7f5d0419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.522+0000 7f5d0c400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d041079a0 0x7f5d0419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.522+0000 7f5d0c400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0419cb00 con 0x7f5d041067a0 2024-03-22T10:47:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.522+0000 7f5d0c400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0419cc40 con 0x7f5d041079a0 2024-03-22T10:47:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.522+0000 7f5d09a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041067a0 0x7f5d0419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.523+0000 7f5d09a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041067a0 0x7f5d0419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40324/0 (socket says 172.21.15.115:40324) 2024-03-22T10:47:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.523+0000 7f5d09a00700 1 -- 172.21.15.115:0/3519426311 learned_addr learned my addr 172.21.15.115:0/3519426311 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.523+0000 7f5d09000700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d041079a0 0x7f5d0419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.523+0000 7f5d09a00700 1 -- 172.21.15.115:0/3519426311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d041079a0 msgr2=0x7f5d0419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.523+0000 7f5d09a00700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d041079a0 0x7f5d0419c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.523+0000 7f5d09a00700 1 -- 172.21.15.115:0/3519426311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf00095f0 con 0x7f5d041067a0 2024-03-22T10:47:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.523+0000 7f5d09a00700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041067a0 0x7f5d0419bfa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800d900 tx=0x7f5cf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.524+0000 7f5d02000700 1 -- 172.21.15.115:0/3519426311 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf80041f0 con 0x7f5d041067a0 2024-03-22T10:47:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.524+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d041a16f0 con 0x7f5d041067a0 2024-03-22T10:47:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.524+0000 7f5d02000700 1 -- 172.21.15.115:0/3519426311 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5cf8004350 con 0x7f5d041067a0 2024-03-22T10:47:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.524+0000 7f5d02000700 1 -- 172.21.15.115:0/3519426311 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf8003da0 con 0x7f5d041067a0 2024-03-22T10:47:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.524+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d041a1c40 con 0x7f5d041067a0 2024-03-22T10:47:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.525+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0410f2f0 con 0x7f5d041067a0 2024-03-22T10:47:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.526+0000 7f5d02000700 1 -- 172.21.15.115:0/3519426311 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5cf8009730 con 0x7f5d041067a0 2024-03-22T10:47:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.527+0000 7f5d02000700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5cec06dfa0 0x7f5cec070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.527+0000 7f5d02000700 1 -- 172.21.15.115:0/3519426311 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5cf808de60 con 0x7f5d041067a0 2024-03-22T10:47:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.528+0000 7f5d09000700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5cec06dfa0 0x7f5cec070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.528+0000 7f5d09000700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5cec06dfa0 0x7f5cec070450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009fd0 tx=0x7f5cf000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:36.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.533+0000 7f5d02000700 1 -- 172.21.15.115:0/3519426311 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5cf805ae60 con 0x7f5d041067a0 2024-03-22T10:47:36.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.794+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d0410c2f0 con 0x7f5cec06dfa0 2024-03-22T10:47:36.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.796+0000 7f5d02000700 1 -- 172.21.15.115:0/3519426311 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5d0410c2f0 con 0x7f5cec06dfa0 2024-03-22T10:47:36.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5cec06dfa0 msgr2=0x7f5cec070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:36.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5cec06dfa0 0x7f5cec070450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009fd0 tx=0x7f5cf000b540 comp rx=0 tx=0).stop 2024-03-22T10:47:36.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041067a0 msgr2=0x7f5d0419bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:36.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041067a0 0x7f5d0419bfa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800d900 tx=0x7f5cf800dcc0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 shutdown_connections 2024-03-22T10:47:36.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5cec06dfa0 0x7f5cec070450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d041067a0 0x7f5d0419bfa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 --2- 172.21.15.115:0/3519426311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d041079a0 0x7f5d0419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 >> 172.21.15.115:0/3519426311 conn(0x7f5d04075910 msgr2=0x7f5d0410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:36.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.799+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 shutdown_connections 2024-03-22T10:47:36.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.800+0000 7f5d0c400700 1 -- 172.21.15.115:0/3519426311 wait complete. 2024-03-22T10:47:36.807 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:47:36.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.958+0000 7fa2b1800700 1 -- 172.21.15.115:0/616805546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 msgr2=0x7fa2ac1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:36.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.958+0000 7fa2b1800700 1 --2- 172.21.15.115:0/616805546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 0x7fa2ac1017a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa298009910 tx=0x7fa298009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:36.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.958+0000 7fa2b1800700 1 -- 172.21.15.115:0/616805546 shutdown_connections 2024-03-22T10:47:36.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.958+0000 7fa2b1800700 1 --2- 172.21.15.115:0/616805546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 0x7fa2ac1017a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.958+0000 7fa2b1800700 1 --2- 172.21.15.115:0/616805546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac100150 0x7fa2ac100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.958+0000 7fa2b1800700 1 -- 172.21.15.115:0/616805546 >> 172.21.15.115:0/616805546 conn(0x7fa2ac0fb6e0 msgr2=0x7fa2ac0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:36.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.958+0000 7fa2b1800700 1 -- 172.21.15.115:0/616805546 shutdown_connections 2024-03-22T10:47:36.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.958+0000 7fa2b1800700 1 -- 172.21.15.115:0/616805546 wait complete. 2024-03-22T10:47:36.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.960+0000 7fa2b1800700 1 Processor -- start 2024-03-22T10:47:36.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.960+0000 7fa2b1800700 1 -- start start 2024-03-22T10:47:36.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.960+0000 7fa2b1800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac100150 0x7fa2ac195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:36.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.960+0000 7fa2b1800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 0x7fa2ac195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:36.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.960+0000 7fa2b1800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1964c0 con 0x7fa2ac101350 2024-03-22T10:47:36.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.960+0000 7fa2b1800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac196600 con 0x7fa2ac100150 2024-03-22T10:47:36.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.961+0000 7fa2aa000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 0x7fa2ac195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:36.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.961+0000 7fa2aa000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 0x7fa2ac195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40342/0 (socket says 172.21.15.115:40342) 2024-03-22T10:47:36.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.961+0000 7fa2aa000700 1 -- 172.21.15.115:0/1730448229 learned_addr learned my addr 172.21.15.115:0/1730448229 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:36.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.961+0000 7fa2aaa00700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac100150 0x7fa2ac195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:36.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.961+0000 7fa2aa000700 1 -- 172.21.15.115:0/1730448229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac100150 msgr2=0x7fa2ac195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:36.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.961+0000 7fa2aa000700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac100150 0x7fa2ac195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:36.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.961+0000 7fa2aa000700 1 -- 172.21.15.115:0/1730448229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2980095f0 con 0x7fa2ac101350 2024-03-22T10:47:36.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.962+0000 7fa2aa000700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 0x7fa2ac195ea0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa298005b40 tx=0x7fa29800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:36.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.962+0000 7fa2a3400700 1 -- 172.21.15.115:0/1730448229 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29801d070 con 0x7fa2ac101350 2024-03-22T10:47:36.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.962+0000 7fa2a3400700 1 -- 172.21.15.115:0/1730448229 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa29800f460 con 0x7fa2ac101350 2024-03-22T10:47:36.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.962+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac19b050 con 0x7fa2ac101350 2024-03-22T10:47:36.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.962+0000 7fa2a3400700 1 -- 172.21.15.115:0/1730448229 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa298021640 con 0x7fa2ac101350 2024-03-22T10:47:36.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.962+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac19b540 con 0x7fa2ac101350 2024-03-22T10:47:36.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.963+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac108ca0 con 0x7fa2ac101350 2024-03-22T10:47:36.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.964+0000 7fa2a3400700 1 -- 172.21.15.115:0/1730448229 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa29800fa90 con 0x7fa2ac101350 2024-03-22T10:47:36.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.966+0000 7fa2a3400700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa29006dfa0 0x7fa290070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:36.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.966+0000 7fa2a3400700 1 -- 172.21.15.115:0/1730448229 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa29808eff0 con 0x7fa2ac101350 2024-03-22T10:47:36.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.966+0000 7fa2aaa00700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa29006dfa0 0x7fa290070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:36.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.967+0000 7fa2aaa00700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa29006dfa0 0x7fa290070450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005950 tx=0x7fa29c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:36.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:36.971+0000 7fa2a3400700 1 -- 172.21.15.115:0/1730448229 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa29805c030 con 0x7fa2ac101350 2024-03-22T10:47:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.230+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2ac19bc90 con 0x7fa29006dfa0 2024-03-22T10:47:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.233+0000 7fa2a3400700 1 -- 172.21.15.115:0/1730448229 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa2ac19bc90 con 0x7fa29006dfa0 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.235+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa29006dfa0 msgr2=0x7fa290070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.235+0000 7fa2b1800700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa29006dfa0 0x7fa290070450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005950 tx=0x7fa29c0058e0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.235+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 msgr2=0x7fa2ac195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.235+0000 7fa2b1800700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 0x7fa2ac195ea0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa298005b40 tx=0x7fa29800be20 comp rx=0 tx=0).stop 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.236+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 shutdown_connections 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.236+0000 7fa2b1800700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa29006dfa0 0x7fa290070450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.236+0000 7fa2b1800700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2ac100150 0x7fa2ac195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.236+0000 7fa2b1800700 1 --2- 172.21.15.115:0/1730448229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac101350 0x7fa2ac195ea0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.236+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 >> 172.21.15.115:0/1730448229 conn(0x7fa2ac0fb6e0 msgr2=0x7fa2ac104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.236+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 shutdown_connections 2024-03-22T10:47:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.236+0000 7fa2b1800700 1 -- 172.21.15.115:0/1730448229 wait complete. 2024-03-22T10:47:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.393+0000 7f41ae000700 1 -- 172.21.15.115:0/2810273716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 msgr2=0x7f41a8102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.393+0000 7f41ae000700 1 --2- 172.21.15.115:0/2810273716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 0x7f41a8102fc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4198009910 tx=0x7f4198009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.393+0000 7f41ae000700 1 -- 172.21.15.115:0/2810273716 shutdown_connections 2024-03-22T10:47:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.393+0000 7f41ae000700 1 --2- 172.21.15.115:0/2810273716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a8103500 0x7f41a81058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.393+0000 7f41ae000700 1 --2- 172.21.15.115:0/2810273716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 0x7f41a8102fc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.393+0000 7f41ae000700 1 -- 172.21.15.115:0/2810273716 >> 172.21.15.115:0/2810273716 conn(0x7f41a80fa5e0 msgr2=0x7f41a80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.393+0000 7f41ae000700 1 -- 172.21.15.115:0/2810273716 shutdown_connections 2024-03-22T10:47:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.394+0000 7f41ae000700 1 -- 172.21.15.115:0/2810273716 wait complete. 2024-03-22T10:47:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.395+0000 7f41ae000700 1 Processor -- start 2024-03-22T10:47:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.395+0000 7f41ae000700 1 -- start start 2024-03-22T10:47:37.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.395+0000 7f41ae000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 0x7f41a8195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:37.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.395+0000 7f41ae000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a8103500 0x7f41a8195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:37.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.395+0000 7f41ae000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a8196480 con 0x7f41a8100be0 2024-03-22T10:47:37.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.395+0000 7f41ae000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a81965c0 con 0x7f41a8103500 2024-03-22T10:47:37.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.396+0000 7f41a7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 0x7f41a8195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:37.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.396+0000 7f41a7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 0x7f41a8195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40370/0 (socket says 172.21.15.115:40370) 2024-03-22T10:47:37.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.396+0000 7f41a7400700 1 -- 172.21.15.115:0/2936892921 learned_addr learned my addr 172.21.15.115:0/2936892921 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:37.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.396+0000 7f41a6a00700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a8103500 0x7f41a8195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:37.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.396+0000 7f41a7400700 1 -- 172.21.15.115:0/2936892921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a8103500 msgr2=0x7f41a8195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:37.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.396+0000 7f41a7400700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a8103500 0x7f41a8195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.396+0000 7f41a7400700 1 -- 172.21.15.115:0/2936892921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41980095f0 con 0x7f41a8100be0 2024-03-22T10:47:37.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.397+0000 7f41a6a00700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a8103500 0x7f41a8195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:47:37.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.397+0000 7f41a7400700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 0x7f41a8195920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f419800b5c0 tx=0x7f419800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:37.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.397+0000 7f419fe00700 1 -- 172.21.15.115:0/2936892921 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f419801d070 con 0x7f41a8100be0 2024-03-22T10:47:37.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.397+0000 7f419fe00700 1 -- 172.21.15.115:0/2936892921 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f419800f460 con 0x7f41a8100be0 2024-03-22T10:47:37.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.397+0000 7f419fe00700 1 -- 172.21.15.115:0/2936892921 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4198021640 con 0x7f41a8100be0 2024-03-22T10:47:37.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.397+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a819b010 con 0x7f41a8100be0 2024-03-22T10:47:37.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.398+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a819b500 con 0x7f41a8100be0 2024-03-22T10:47:37.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.399+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a818fb70 con 0x7f41a8100be0 2024-03-22T10:47:37.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.401+0000 7f419fe00700 1 -- 172.21.15.115:0/2936892921 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f419800f5d0 con 0x7f41a8100be0 2024-03-22T10:47:37.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.402+0000 7f419fe00700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f418c06dfa0 0x7f418c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:37.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.402+0000 7f419fe00700 1 -- 172.21.15.115:0/2936892921 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f419808f0d0 con 0x7f41a8100be0 2024-03-22T10:47:37.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.402+0000 7f41a6a00700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f418c06dfa0 0x7f418c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:37.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.403+0000 7f41a6a00700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f418c06dfa0 0x7f418c070450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4194005950 tx=0x7f41940058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:37.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.407+0000 7f419fe00700 1 -- 172.21.15.115:0/2936892921 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f419805c0b0 con 0x7f41a8100be0 2024-03-22T10:47:37.638 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:37 smithi115 ceph-mon[78672]: pgmap v124: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 16/369 objects degraded (4.336%) 2024-03-22T10:47:37.638 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:37 smithi115 ceph-mon[78672]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:37.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.647+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f41a8061190 con 0x7f418c06dfa0 2024-03-22T10:47:37.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.664+0000 7f419fe00700 1 -- 172.21.15.115:0/2936892921 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f41a8061190 con 0x7f418c06dfa0 2024-03-22T10:47:37.655 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:47:37.655 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (14m) 5s ago 15m 14.2M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:47:37.655 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (16m) 5s ago 16m 8694k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:47:37.655 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (14m) 108s ago 14m 7952k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:47:37.655 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (2m) 5s ago 16m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:47:37.655 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (113s) 108s ago 14m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:47:37.655 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (13m) 5s ago 15m 73.7M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:47:37.655 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (9m) 5s ago 9m 127M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (8m) 5s ago 8m 16.9M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (8m) 108s ago 8m 15.5M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (9m) 108s ago 9m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (5m) 5s ago 17m 507M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (4m) 108s ago 14m 449M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (3m) 5s ago 18m 46.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (3m) 108s ago 14m 33.6M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (16m) 5s ago 16m 8238k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (14m) 108s ago 14m 7721k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (85s) 5s ago 13m 84.2M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (41s) 5s ago 13m 81.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (12s) 5s ago 12m 11.1M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (11m) 108s ago 11m 235M 2241M 18.2.0 dc2bc1663786 58dd20ca013a 2024-03-22T10:47:37.656 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (11m) 108s ago 11m 225M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:47:37.657 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (10m) 108s ago 10m 195M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:47:37.657 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (4m) 5s ago 15m 71.2M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:47:37.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.668+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f418c06dfa0 msgr2=0x7f418c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:37.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.668+0000 7f41ae000700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f418c06dfa0 0x7f418c070450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4194005950 tx=0x7f41940058e0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.668+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 msgr2=0x7f41a8195920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:37.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.668+0000 7f41ae000700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 0x7f41a8195920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f419800b5c0 tx=0x7f419800be20 comp rx=0 tx=0).stop 2024-03-22T10:47:37.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.668+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 shutdown_connections 2024-03-22T10:47:37.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.668+0000 7f41ae000700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f418c06dfa0 0x7f418c070450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.668+0000 7f41ae000700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41a8100be0 0x7f41a8195920 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.668+0000 7f41ae000700 1 --2- 172.21.15.115:0/2936892921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a8103500 0x7f41a8195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.668+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 >> 172.21.15.115:0/2936892921 conn(0x7f41a80fa5e0 msgr2=0x7f41a80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:37.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.669+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 shutdown_connections 2024-03-22T10:47:37.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.669+0000 7f41ae000700 1 -- 172.21.15.115:0/2936892921 wait complete. 2024-03-22T10:47:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.831+0000 7f38fe400700 1 -- 172.21.15.115:0/2964986183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 msgr2=0x7f38f81037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.831+0000 7f38fe400700 1 --2- 172.21.15.115:0/2964986183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 0x7f38f81037d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009910 tx=0x7f38e4009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.831+0000 7f38fe400700 1 -- 172.21.15.115:0/2964986183 shutdown_connections 2024-03-22T10:47:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.831+0000 7f38fe400700 1 --2- 172.21.15.115:0/2964986183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 0x7f38f81037d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.831+0000 7f38fe400700 1 --2- 172.21.15.115:0/2964986183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38f80fe9f0 0x7f38f8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.831+0000 7f38fe400700 1 -- 172.21.15.115:0/2964986183 >> 172.21.15.115:0/2964986183 conn(0x7f38f80fa600 msgr2=0x7f38f80fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.831+0000 7f38fe400700 1 -- 172.21.15.115:0/2964986183 shutdown_connections 2024-03-22T10:47:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.831+0000 7f38fe400700 1 -- 172.21.15.115:0/2964986183 wait complete. 2024-03-22T10:47:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.833+0000 7f38fe400700 1 Processor -- start 2024-03-22T10:47:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.833+0000 7f38fe400700 1 -- start start 2024-03-22T10:47:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.833+0000 7f38fe400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38f80fe9f0 0x7f38f819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.833+0000 7f38fe400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 0x7f38f819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.833+0000 7f38fe400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f819cad0 con 0x7f38f8101350 2024-03-22T10:47:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.833+0000 7f38fe400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f819cc10 con 0x7f38f80fe9f0 2024-03-22T10:47:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.834+0000 7f38f6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 0x7f38f819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:37.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.834+0000 7f38f6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 0x7f38f819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40398/0 (socket says 172.21.15.115:40398) 2024-03-22T10:47:37.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.834+0000 7f38f6a00700 1 -- 172.21.15.115:0/1713430413 learned_addr learned my addr 172.21.15.115:0/1713430413 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:37.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.834+0000 7f38f7400700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38f80fe9f0 0x7f38f819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:37.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.834+0000 7f38f6a00700 1 -- 172.21.15.115:0/1713430413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38f80fe9f0 msgr2=0x7f38f819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:37.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.834+0000 7f38f6a00700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38f80fe9f0 0x7f38f819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:37.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.834+0000 7f38f6a00700 1 -- 172.21.15.115:0/1713430413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e40095f0 con 0x7f38f8101350 2024-03-22T10:47:37.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.835+0000 7f38f6a00700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 0x7f38f819c4b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38e400bd70 tx=0x7f38e400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:37.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.835+0000 7f38efe00700 1 -- 172.21.15.115:0/1713430413 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e401d070 con 0x7f38f8101350 2024-03-22T10:47:37.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.835+0000 7f38efe00700 1 -- 172.21.15.115:0/1713430413 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f38e4022470 con 0x7f38f8101350 2024-03-22T10:47:37.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.835+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f81a1660 con 0x7f38f8101350 2024-03-22T10:47:37.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.835+0000 7f38efe00700 1 -- 172.21.15.115:0/1713430413 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e400f670 con 0x7f38f8101350 2024-03-22T10:47:37.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.835+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f81a1b50 con 0x7f38f8101350 2024-03-22T10:47:37.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.836+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f8196190 con 0x7f38f8101350 2024-03-22T10:47:37.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.837+0000 7f38efe00700 1 -- 172.21.15.115:0/1713430413 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f38e4022a50 con 0x7f38f8101350 2024-03-22T10:47:37.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.839+0000 7f38efe00700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f38dc06dfa0 0x7f38dc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:37.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.839+0000 7f38efe00700 1 -- 172.21.15.115:0/1713430413 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f38e408f2f0 con 0x7f38f8101350 2024-03-22T10:47:37.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.839+0000 7f38f7400700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f38dc06dfa0 0x7f38dc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:37.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.840+0000 7f38f7400700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f38dc06dfa0 0x7f38dc070450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38e8005950 tx=0x7f38e80058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:37.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:37.843+0000 7f38efe00700 1 -- 172.21.15.115:0/1713430413 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38e405c4a0 con 0x7f38f8101350 2024-03-22T10:47:37.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:37 smithi150 ceph-mon[65513]: pgmap v124: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 16/369 objects degraded (4.336%) 2024-03-22T10:47:37.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:37 smithi150 ceph-mon[65513]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.173+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f38f802cc30 con 0x7f38f8101350 2024-03-22T10:47:38.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.173+0000 7f38efe00700 1 -- 172.21.15.115:0/1713430413 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f38e405c030 con 0x7f38f8101350 2024-03-22T10:47:38.164 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:47:38.164 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:47:38.164 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:47:38.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 3 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 7, 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 7 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:47:38.165 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.177+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f38dc06dfa0 msgr2=0x7f38dc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.177+0000 7f38fe400700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f38dc06dfa0 0x7f38dc070450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38e8005950 tx=0x7f38e80058e0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.177+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 msgr2=0x7f38f819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.177+0000 7f38fe400700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 0x7f38f819c4b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38e400bd70 tx=0x7f38e400be50 comp rx=0 tx=0).stop 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.178+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 shutdown_connections 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.178+0000 7f38fe400700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f38dc06dfa0 0x7f38dc070450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.178+0000 7f38fe400700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38f80fe9f0 0x7f38f819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.178+0000 7f38fe400700 1 --2- 172.21.15.115:0/1713430413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f8101350 0x7f38f819c4b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.178+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 >> 172.21.15.115:0/1713430413 conn(0x7f38f80fa600 msgr2=0x7f38f80fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.178+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 shutdown_connections 2024-03-22T10:47:38.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.178+0000 7f38fe400700 1 -- 172.21.15.115:0/1713430413 wait complete. 2024-03-22T10:47:38.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.334+0000 7f6751a00700 1 -- 172.21.15.115:0/863445293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c074d80 msgr2=0x7f674c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:38.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.334+0000 7f6751a00700 1 --2- 172.21.15.115:0/863445293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c074d80 0x7f674c0731e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f673c009940 tx=0x7f673c009c50 comp rx=0 tx=0).stop 2024-03-22T10:47:38.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.334+0000 7f6751a00700 1 -- 172.21.15.115:0/863445293 shutdown_connections 2024-03-22T10:47:38.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.334+0000 7f6751a00700 1 --2- 172.21.15.115:0/863445293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f674c0737b0 0x7f674c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.334+0000 7f6751a00700 1 --2- 172.21.15.115:0/863445293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c074d80 0x7f674c0731e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.334+0000 7f6751a00700 1 -- 172.21.15.115:0/863445293 >> 172.21.15.115:0/863445293 conn(0x7f674c0fb6e0 msgr2=0x7f674c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:38.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.334+0000 7f6751a00700 1 -- 172.21.15.115:0/863445293 shutdown_connections 2024-03-22T10:47:38.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.334+0000 7f6751a00700 1 -- 172.21.15.115:0/863445293 wait complete. 2024-03-22T10:47:38.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.335+0000 7f6751a00700 1 Processor -- start 2024-03-22T10:47:38.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.336+0000 7f6751a00700 1 -- start start 2024-03-22T10:47:38.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.336+0000 7f6751a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c0737b0 0x7f674c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:38.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.336+0000 7f6751a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f674c074d80 0x7f674c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:38.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.336+0000 7f6751a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c1985e0 con 0x7f674c0737b0 2024-03-22T10:47:38.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.336+0000 7f6751a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c198720 con 0x7f674c074d80 2024-03-22T10:47:38.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.336+0000 7f674aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c0737b0 0x7f674c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:38.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.336+0000 7f674aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c0737b0 0x7f674c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40408/0 (socket says 172.21.15.115:40408) 2024-03-22T10:47:38.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.336+0000 7f674aa00700 1 -- 172.21.15.115:0/3180358934 learned_addr learned my addr 172.21.15.115:0/3180358934 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:38.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f6743e00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f674c074d80 0x7f674c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:38.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f674aa00700 1 -- 172.21.15.115:0/3180358934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f674c074d80 msgr2=0x7f674c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:38.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f674aa00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f674c074d80 0x7f674c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f674aa00700 1 -- 172.21.15.115:0/3180358934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c0095f0 con 0x7f674c0737b0 2024-03-22T10:47:38.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f6743e00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f674c074d80 0x7f674c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:47:38.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f674aa00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c0737b0 0x7f674c197b10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f673c0055f0 tx=0x7f673c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:38.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f6743400700 1 -- 172.21.15.115:0/3180358934 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f673c01d070 con 0x7f674c0737b0 2024-03-22T10:47:38.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f6743400700 1 -- 172.21.15.115:0/3180358934 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f673c003ba0 con 0x7f674c0737b0 2024-03-22T10:47:38.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f6743400700 1 -- 172.21.15.115:0/3180358934 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f673c021640 con 0x7f674c0737b0 2024-03-22T10:47:38.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.337+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674c19d180 con 0x7f674c0737b0 2024-03-22T10:47:38.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.338+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c19d6c0 con 0x7f674c0737b0 2024-03-22T10:47:38.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.339+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c10ae90 con 0x7f674c0737b0 2024-03-22T10:47:38.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.339+0000 7f6743400700 1 -- 172.21.15.115:0/3180358934 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f673c00fa10 con 0x7f674c0737b0 2024-03-22T10:47:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.340+0000 7f6743400700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f673006dfa0 0x7f6730070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.341+0000 7f6743400700 1 -- 172.21.15.115:0/3180358934 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f673c08f150 con 0x7f674c0737b0 2024-03-22T10:47:38.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.341+0000 7f6743e00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f673006dfa0 0x7f6730070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:38.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.341+0000 7f6743e00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f673006dfa0 0x7f6730070450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6734005fd0 tx=0x7f6734005e20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:38.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.345+0000 7f6743400700 1 -- 172.21.15.115:0/3180358934 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f673c05c130 con 0x7f674c0737b0 2024-03-22T10:47:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.634+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f674c19e180 con 0x7f674c0737b0 2024-03-22T10:47:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.635+0000 7f6743400700 1 -- 172.21.15.115:0/3180358934 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1907 (secure 0 0 0) 0x7f673c05bcc0 con 0x7f674c0737b0 2024-03-22T10:47:38.626 INFO:teuthology.orchestra.run.smithi115.stdout:e16 2024-03-22T10:47:38.626 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:47:38.626 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:47:38.626 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:47:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:47:38.626 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:47:38.626 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:47:38.626 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 16 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:47:17.573634+0000 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:47:38.627 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 14454 members: 24241,14454 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:47:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:47:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:47:38.629 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:47:38.629 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:47:38.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f673006dfa0 msgr2=0x7f6730070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:38.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f673006dfa0 0x7f6730070450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6734005fd0 tx=0x7f6734005e20 comp rx=0 tx=0).stop 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c0737b0 msgr2=0x7f674c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c0737b0 0x7f674c197b10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f673c0055f0 tx=0x7f673c0056d0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 shutdown_connections 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f673006dfa0 0x7f6730070450 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f674c0737b0 0x7f674c197b10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 --2- 172.21.15.115:0/3180358934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f674c074d80 0x7f674c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 >> 172.21.15.115:0/3180358934 conn(0x7f674c0fb6e0 msgr2=0x7f674c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 shutdown_connections 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.638+0000 7f6751a00700 1 -- 172.21.15.115:0/3180358934 wait complete. 2024-03-22T10:47:38.630 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2024-03-22T10:47:38.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.796+0000 7fde9d600700 1 -- 172.21.15.115:0/2155717087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98103580 msgr2=0x7fde981039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:38.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.796+0000 7fde9d600700 1 --2- 172.21.15.115:0/2155717087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98103580 0x7fde981039d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fde84009910 tx=0x7fde84009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:38.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.796+0000 7fde9d600700 1 -- 172.21.15.115:0/2155717087 shutdown_connections 2024-03-22T10:47:38.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.796+0000 7fde9d600700 1 --2- 172.21.15.115:0/2155717087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98103580 0x7fde981039d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.796+0000 7fde9d600700 1 --2- 172.21.15.115:0/2155717087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde98102380 0x7fde98102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.796+0000 7fde9d600700 1 -- 172.21.15.115:0/2155717087 >> 172.21.15.115:0/2155717087 conn(0x7fde980fd930 msgr2=0x7fde980ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:38.788 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:38 smithi115 ceph-mon[78672]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:38.788 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:38 smithi115 ceph-mon[78672]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:38.788 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:38 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1713430413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.796+0000 7fde9d600700 1 -- 172.21.15.115:0/2155717087 shutdown_connections 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.796+0000 7fde9d600700 1 -- 172.21.15.115:0/2155717087 wait complete. 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.797+0000 7fde9d600700 1 Processor -- start 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.798+0000 7fde9d600700 1 -- start start 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.798+0000 7fde9d600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98102380 0x7fde98197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.798+0000 7fde9d600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde98103580 0x7fde981980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.798+0000 7fde9d600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde981986e0 con 0x7fde98102380 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.798+0000 7fde9d600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98198820 con 0x7fde98103580 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.798+0000 7fde96a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98102380 0x7fde98197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.798+0000 7fde96a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98102380 0x7fde98197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40424/0 (socket says 172.21.15.115:40424) 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.798+0000 7fde96a00700 1 -- 172.21.15.115:0/1181849473 learned_addr learned my addr 172.21.15.115:0/1181849473 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:38.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.798+0000 7fde96000700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde98103580 0x7fde981980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.799+0000 7fde96a00700 1 -- 172.21.15.115:0/1181849473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde98103580 msgr2=0x7fde981980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.799+0000 7fde96a00700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde98103580 0x7fde981980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.799+0000 7fde96a00700 1 -- 172.21.15.115:0/1181849473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde840095f0 con 0x7fde98102380 2024-03-22T10:47:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.799+0000 7fde96a00700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98102380 0x7fde98197b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fde8800d900 tx=0x7fde8800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.799+0000 7fde8f400700 1 -- 172.21.15.115:0/1181849473 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde8800b920 con 0x7fde98102380 2024-03-22T10:47:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.799+0000 7fde8f400700 1 -- 172.21.15.115:0/1181849473 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fde88010460 con 0x7fde98102380 2024-03-22T10:47:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.799+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9819d2d0 con 0x7fde98102380 2024-03-22T10:47:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.799+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9819d820 con 0x7fde98102380 2024-03-22T10:47:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.799+0000 7fde8f400700 1 -- 172.21.15.115:0/1181849473 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde8800f590 con 0x7fde98102380 2024-03-22T10:47:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.801+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9810aed0 con 0x7fde98102380 2024-03-22T10:47:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.802+0000 7fde8f400700 1 -- 172.21.15.115:0/1181849473 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fde880105d0 con 0x7fde98102380 2024-03-22T10:47:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.802+0000 7fde8f400700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde7c06dfa0 0x7fde7c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.803+0000 7fde8f400700 1 -- 172.21.15.115:0/1181849473 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fde8808dce0 con 0x7fde98102380 2024-03-22T10:47:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.803+0000 7fde96000700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde7c06dfa0 0x7fde7c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.803+0000 7fde96000700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde7c06dfa0 0x7fde7c070450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde84006010 tx=0x7fde8400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:38.807+0000 7fde8f400700 1 -- 172.21.15.115:0/1181849473 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde8805ace0 con 0x7fde98102380 2024-03-22T10:47:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:38 smithi150 ceph-mon[65513]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:38 smithi150 ceph-mon[65513]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:38 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1713430413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.048+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde98107ed0 con 0x7fde7c06dfa0 2024-03-22T10:47:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.050+0000 7fde8f400700 1 -- 172.21.15.115:0/1181849473 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fde98107ed0 con 0x7fde7c06dfa0 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "9/23 daemons upgraded", 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:47:39.041 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:47:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.052+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde7c06dfa0 msgr2=0x7fde7c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.052+0000 7fde9d600700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde7c06dfa0 0x7fde7c070450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde84006010 tx=0x7fde8400b540 comp rx=0 tx=0).stop 2024-03-22T10:47:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.052+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98102380 msgr2=0x7fde98197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.052+0000 7fde9d600700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98102380 0x7fde98197b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fde8800d900 tx=0x7fde8800dc10 comp rx=0 tx=0).stop 2024-03-22T10:47:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.052+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 shutdown_connections 2024-03-22T10:47:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.052+0000 7fde9d600700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde7c06dfa0 0x7fde7c070450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.052+0000 7fde9d600700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98102380 0x7fde98197b80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.052+0000 7fde9d600700 1 --2- 172.21.15.115:0/1181849473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde98103580 0x7fde981980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.052+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 >> 172.21.15.115:0/1181849473 conn(0x7fde980fd930 msgr2=0x7fde981067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.053+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 shutdown_connections 2024-03-22T10:47:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.053+0000 7fde9d600700 1 -- 172.21.15.115:0/1181849473 wait complete. 2024-03-22T10:47:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.202+0000 7f203be00700 1 -- 172.21.15.115:0/2386545551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 msgr2=0x7f203c0fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.202+0000 7f203be00700 1 --2- 172.21.15.115:0/2386545551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 0x7f203c0fec40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f202c009910 tx=0x7f202c009c20 comp rx=0 tx=0).stop 2024-03-22T10:47:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.202+0000 7f203be00700 1 -- 172.21.15.115:0/2386545551 shutdown_connections 2024-03-22T10:47:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.202+0000 7f203be00700 1 --2- 172.21.15.115:0/2386545551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f203c0ff180 0x7f203c0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.202+0000 7f203be00700 1 --2- 172.21.15.115:0/2386545551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 0x7f203c0fec40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.202+0000 7f203be00700 1 -- 172.21.15.115:0/2386545551 >> 172.21.15.115:0/2386545551 conn(0x7f203c0fa480 msgr2=0x7f203c0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.202+0000 7f203be00700 1 -- 172.21.15.115:0/2386545551 shutdown_connections 2024-03-22T10:47:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.202+0000 7f203be00700 1 -- 172.21.15.115:0/2386545551 wait complete. 2024-03-22T10:47:39.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.203+0000 7f203be00700 1 Processor -- start 2024-03-22T10:47:39.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.203+0000 7f203be00700 1 -- start start 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.203+0000 7f203be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 0x7f203c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.203+0000 7f203be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f203c0ff180 0x7f203c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.203+0000 7f203be00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c198510 con 0x7f203c0fe830 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.203+0000 7f203be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c198650 con 0x7f203c0ff180 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.204+0000 7f203aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 0x7f203c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.204+0000 7f203aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 0x7f203c1979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40442/0 (socket says 172.21.15.115:40442) 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.204+0000 7f203aa00700 1 -- 172.21.15.115:0/1518788156 learned_addr learned my addr 172.21.15.115:0/1518788156 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.204+0000 7f203a000700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f203c0ff180 0x7f203c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.204+0000 7f203aa00700 1 -- 172.21.15.115:0/1518788156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f203c0ff180 msgr2=0x7f203c197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.204+0000 7f203aa00700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f203c0ff180 0x7f203c197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.204+0000 7f203aa00700 1 -- 172.21.15.115:0/1518788156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c0095f0 con 0x7f203c0fe830 2024-03-22T10:47:39.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.205+0000 7f203aa00700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 0x7f203c1979b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f202c005b40 tx=0x7f202c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:39.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.205+0000 7f2033400700 1 -- 172.21.15.115:0/1518788156 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f202c01d070 con 0x7f203c0fe830 2024-03-22T10:47:39.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.205+0000 7f2033400700 1 -- 172.21.15.115:0/1518788156 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f202c00f460 con 0x7f203c0fe830 2024-03-22T10:47:39.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.205+0000 7f2033400700 1 -- 172.21.15.115:0/1518788156 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f202c021640 con 0x7f203c0fe830 2024-03-22T10:47:39.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.205+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c19d0a0 con 0x7f203c0fe830 2024-03-22T10:47:39.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.205+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c19d590 con 0x7f203c0fe830 2024-03-22T10:47:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.206+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c10ace0 con 0x7f203c0fe830 2024-03-22T10:47:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.207+0000 7f2033400700 1 -- 172.21.15.115:0/1518788156 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f202c00fa90 con 0x7f203c0fe830 2024-03-22T10:47:39.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.208+0000 7f2033400700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f202006dfa0 0x7f2020070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:47:39.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.208+0000 7f2033400700 1 -- 172.21.15.115:0/1518788156 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f202c08f020 con 0x7f203c0fe830 2024-03-22T10:47:39.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.208+0000 7f203a000700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f202006dfa0 0x7f2020070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:47:39.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.209+0000 7f203a000700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f202006dfa0 0x7f2020070450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2028005950 tx=0x7f20280058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:47:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.213+0000 7f2033400700 1 -- 172.21.15.115:0/1518788156 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f202c05c0a0 con 0x7f203c0fe830 2024-03-22T10:47:39.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.509+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f203c10aed0 con 0x7f203c0fe830 2024-03-22T10:47:39.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.510+0000 7f2033400700 1 -- 172.21.15.115:0/1518788156 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f202c05bc30 con 0x7f203c0fe830 2024-03-22T10:47:39.501 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:47:39.501 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:47:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:47:39.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f202006dfa0 msgr2=0x7f2020070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:39.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f202006dfa0 0x7f2020070450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2028005950 tx=0x7f20280058e0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 msgr2=0x7f203c1979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:47:39.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 0x7f203c1979b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f202c005b40 tx=0x7f202c00be20 comp rx=0 tx=0).stop 2024-03-22T10:47:39.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 shutdown_connections 2024-03-22T10:47:39.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f202006dfa0 0x7f2020070450 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f203c0fe830 0x7f203c1979b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 --2- 172.21.15.115:0/1518788156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f203c0ff180 0x7f203c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:47:39.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 >> 172.21.15.115:0/1518788156 conn(0x7f203c0fa480 msgr2=0x7f203c106e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:47:39.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 shutdown_connections 2024-03-22T10:47:39.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:47:39.514+0000 7f203be00700 1 -- 172.21.15.115:0/1518788156 wait complete. 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: pgmap v125: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/369 objects degraded (4.336%), 5 pgs degraded) 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3180358934' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:39.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:39.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:39.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T10:47:39.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:39.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:39 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1518788156' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:47:39.872 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: pgmap v125: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:39.872 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/369 objects degraded (4.336%), 5 pgs degraded) 2024-03-22T10:47:39.872 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:39.872 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3180358934' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:47:39.872 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:39.872 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:47:39.873 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:47:39.873 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:39.873 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:47:39.873 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:39.873 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:39.873 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:47:39.873 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T10:47:39.873 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:39.873 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:39 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1518788156' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:47:40.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:40 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T10:47:40.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:40 smithi150 ceph-mon[65513]: Upgrade: osd.3 is safe to restart 2024-03-22T10:47:40.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:40 smithi150 ceph-mon[65513]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:41.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:40 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T10:47:41.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:40 smithi115 ceph-mon[78672]: Upgrade: osd.3 is safe to restart 2024-03-22T10:47:41.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:40 smithi115 ceph-mon[78672]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:47:41.886 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:41 smithi115 ceph-mon[78672]: pgmap v126: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:41.887 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:41.887 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T10:47:41.887 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:47:41.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:41 smithi150 ceph-mon[65513]: pgmap v126: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:41.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:41.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T10:47:41.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:47:42.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:42 smithi150 ceph-mon[65513]: Upgrade: Updating osd.3 2024-03-22T10:47:42.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:42 smithi150 ceph-mon[65513]: Deploying daemon osd.3 on smithi150 2024-03-22T10:47:43.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:42 smithi115 ceph-mon[78672]: Upgrade: Updating osd.3 2024-03-22T10:47:43.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:42 smithi115 ceph-mon[78672]: Deploying daemon osd.3 on smithi150 2024-03-22T10:47:43.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:43 smithi150 ceph-mon[65513]: pgmap v127: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:44.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:43 smithi115 ceph-mon[78672]: pgmap v127: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:46.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:45 smithi115 ceph-mon[78672]: pgmap v128: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:46.060 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:45 smithi150 ceph-mon[65513]: pgmap v128: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:46.398 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:46 smithi150 systemd[1]: Stopping Ceph osd.3 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:47:46.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:46 smithi150 ceph-mon[65513]: osd.3 marked itself down and dead 2024-03-22T10:47:46.898 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:46 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[40452]: 2024-03-22T10:47:46.519+0000 7fe3ec600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T10:47:46.898 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:46 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[40452]: 2024-03-22T10:47:46.519+0000 7fe3ec600700 -1 osd.3 63 *** Got signal Terminated *** 2024-03-22T10:47:46.898 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:46 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[40452]: 2024-03-22T10:47:46.519+0000 7fe3ec600700 -1 osd.3 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T10:47:47.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:46 smithi115 ceph-mon[78672]: osd.3 marked itself down and dead 2024-03-22T10:47:47.460 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:47 smithi150 podman[70217]: 2024-03-22 10:47:47.179199846 +0000 UTC m=+0.801757317 container died 58dd20ca013ab0436dfb4afb77947b3b4530949aea3ba030e1307d46785472c5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, ceph=True, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212) 2024-03-22T10:47:47.808 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:47 smithi150 ceph-mon[65513]: pgmap v129: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:47.808 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:47 smithi150 ceph-mon[65513]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:47:47.809 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:47 smithi150 ceph-mon[65513]: osdmap e64: 6 total, 5 up, 6 in 2024-03-22T10:47:47.809 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:47 smithi150 podman[70217]: 2024-03-22 10:47:47.580035237 +0000 UTC m=+1.202592745 container cleanup 58dd20ca013ab0436dfb4afb77947b3b4530949aea3ba030e1307d46785472c5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1) 2024-03-22T10:47:47.809 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:47 smithi150 bash[70217]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3 2024-03-22T10:47:48.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:47 smithi115 ceph-mon[78672]: pgmap v129: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:48.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:47 smithi115 ceph-mon[78672]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:47:48.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:47 smithi115 ceph-mon[78672]: osdmap e64: 6 total, 5 up, 6 in 2024-03-22T10:47:48.148 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:47 smithi150 podman[70238]: 2024-03-22 10:47:47.889938613 +0000 UTC m=+0.699968237 container remove 58dd20ca013ab0436dfb4afb77947b3b4530949aea3ba030e1307d46785472c5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3, RELEASE=HEAD, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0) 2024-03-22T10:47:48.928 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:48 smithi150 ceph-mon[65513]: osdmap e65: 6 total, 5 up, 6 in 2024-03-22T10:47:48.928 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:48 smithi150 podman[70326]: 2024-03-22 10:47:48.651475915 +0000 UTC m=+0.033125929 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:47:48.928 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:48 smithi150 podman[70326]: 2024-03-22 10:47:48.766127379 +0000 UTC m=+0.147777431 container create 7667690e49b5c73cbc3515bf336bf0197f3444b1c88ffdc2c415c31a85f986cf (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-22T10:47:49.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:48 smithi115 ceph-mon[78672]: osdmap e65: 6 total, 5 up, 6 in 2024-03-22T10:47:49.398 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:49 smithi150 podman[70326]: 2024-03-22 10:47:49.070020833 +0000 UTC m=+0.451670891 container init 7667690e49b5c73cbc3515bf336bf0197f3444b1c88ffdc2c415c31a85f986cf (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-22T10:47:49.398 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:49 smithi150 podman[70326]: 2024-03-22 10:47:49.082651402 +0000 UTC m=+0.464301487 container start 7667690e49b5c73cbc3515bf336bf0197f3444b1c88ffdc2c415c31a85f986cf (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-22T10:47:49.399 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:49 smithi150 podman[70326]: 2024-03-22 10:47:49.18539475 +0000 UTC m=+0.567044852 container attach 7667690e49b5c73cbc3515bf336bf0197f3444b1c88ffdc2c415c31a85f986cf (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:47:49.859 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:49 smithi150 ceph-mon[65513]: pgmap v132: 65 pgs: 16 stale+active+clean, 49 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:49.859 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:49.859 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:47:49.859 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:49 smithi150 podman[70326]: 2024-03-22 10:47:49.525278055 +0000 UTC m=+0.906928131 container died 7667690e49b5c73cbc3515bf336bf0197f3444b1c88ffdc2c415c31a85f986cf (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:47:50.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:49 smithi115 ceph-mon[78672]: pgmap v132: 65 pgs: 16 stale+active+clean, 49 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:47:50.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:50.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:47:50.644 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:50 smithi150 podman[70352]: 2024-03-22 10:47:50.247618898 +0000 UTC m=+0.704834940 container remove 7667690e49b5c73cbc3515bf336bf0197f3444b1c88ffdc2c415c31a85f986cf (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-22T10:47:50.644 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:50 smithi150 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-22T10:47:50.645 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:50 smithi150 systemd[1]: Stopped Ceph osd.3 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:47:50.645 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:50 smithi150 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.3.service: Consumed 1min 51.190s CPU time. 2024-03-22T10:47:50.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:50 smithi150 ceph-mon[65513]: Health check failed: Degraded data redundancy: 72/369 objects degraded (19.512%), 18 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:50.898 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:50 smithi150 systemd[1]: Starting Ceph osd.3 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:47:51.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:50 smithi115 ceph-mon[78672]: Health check failed: Degraded data redundancy: 72/369 objects degraded (19.512%), 18 pgs degraded (PG_DEGRADED) 2024-03-22T10:47:51.294 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:51 smithi150 podman[70451]: 2024-03-22 10:47:50.922112865 +0000 UTC m=+0.037256254 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:47:51.294 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:51 smithi150 podman[70451]: 2024-03-22 10:47:51.023616074 +0000 UTC m=+0.138759503 container create e74a94a6155e8d1b08a17437321b8e40febc8ebd26cc12ce45dae39ea5a57b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:47:51.294 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:51 smithi150 podman[70451]: 2024-03-22 10:47:51.302683656 +0000 UTC m=+0.417827088 container init e74a94a6155e8d1b08a17437321b8e40febc8ebd26cc12ce45dae39ea5a57b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-22T10:47:51.648 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:51 smithi150 podman[70451]: 2024-03-22 10:47:51.313164554 +0000 UTC m=+0.428307949 container start e74a94a6155e8d1b08a17437321b8e40febc8ebd26cc12ce45dae39ea5a57b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T10:47:51.648 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:51 smithi150 podman[70451]: 2024-03-22 10:47:51.384647765 +0000 UTC m=+0.499791238 container attach e74a94a6155e8d1b08a17437321b8e40febc8ebd26cc12ce45dae39ea5a57b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-22T10:47:52.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:51 smithi115 ceph-mon[78672]: pgmap v133: 65 pgs: 18 active+undersized+degraded, 1 stale+active+clean, 16 active+undersized, 30 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:47:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:51 smithi150 ceph-mon[65513]: pgmap v133: 65 pgs: 18 active+undersized+degraded, 1 stale+active+clean, 16 active+undersized, 30 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:47:52.576 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate[70465]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T10:47:52.576 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate[70465]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 bash[70451]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 bash[70451]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 bash[70451]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 bash[70451]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 bash[70451]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 bash[70451]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 bash[70451]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate[70465]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate[70465]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate[70465]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate[70465]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate[70465]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-22T10:47:52.577 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 podman[70451]: 2024-03-22 10:47:52.24040963 +0000 UTC m=+1.355553063 container died e74a94a6155e8d1b08a17437321b8e40febc8ebd26cc12ce45dae39ea5a57b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True) 2024-03-22T10:47:53.147 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:52 smithi150 podman[70601]: 2024-03-22 10:47:52.904445043 +0000 UTC m=+0.645253189 container remove e74a94a6155e8d1b08a17437321b8e40febc8ebd26cc12ce45dae39ea5a57b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:47:53.545 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:53 smithi150 podman[70663]: 2024-03-22 10:47:53.176857041 +0000 UTC m=+0.033595352 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:47:53.546 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:53 smithi150 podman[70663]: 2024-03-22 10:47:53.288885582 +0000 UTC m=+0.145623899 container create e01353665cf38622dacf031ec456182dd3b58e50bb79150b1fb472ac6f6c51bd (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-22T10:47:53.546 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:53 smithi150 podman[70663]: 2024-03-22 10:47:53.500561926 +0000 UTC m=+0.357300279 container init e01353665cf38622dacf031ec456182dd3b58e50bb79150b1fb472ac6f6c51bd (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-22T10:47:53.546 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:53 smithi150 podman[70663]: 2024-03-22 10:47:53.510938951 +0000 UTC m=+0.367677281 container start e01353665cf38622dacf031ec456182dd3b58e50bb79150b1fb472ac6f6c51bd (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-22T10:47:53.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:53 smithi150 ceph-mon[65513]: pgmap v134: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:47:53.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:53 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:53.898 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:53 smithi150 ceph-osd[70681]: -- 172.21.15.150:0/638225188 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5637db855680 con 0x5637db81ec00 2024-03-22T10:47:53.898 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:53 smithi150 bash[70663]: e01353665cf38622dacf031ec456182dd3b58e50bb79150b1fb472ac6f6c51bd 2024-03-22T10:47:53.898 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:53 smithi150 systemd[1]: Started Ceph osd.3 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:47:54.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:53 smithi115 ceph-mon[78672]: pgmap v134: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:47:54.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:53 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:54.398 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:54 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T10:47:54.078+0000 7f7a8663a7c0 -1 Falling back to public interface 2024-03-22T10:47:55.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:54 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:55.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:54 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:47:55.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:54 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:55.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:54 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:47:56.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:55 smithi115 ceph-mon[78672]: pgmap v135: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:47:56.068 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:55 smithi150 ceph-mon[65513]: pgmap v135: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:47:58.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:57 smithi150 ceph-mon[65513]: pgmap v136: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:47:58.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:57 smithi115 ceph-mon[78672]: pgmap v136: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:47:59.398 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:59 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T10:47:59.262+0000 7f7a8663a7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-22T10:47:59.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:59.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:59.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:47:59 smithi150 ceph-mon[65513]: pgmap v137: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:47:59.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:59.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:47:59.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:47:59 smithi115 ceph-mon[78672]: pgmap v137: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:48:00.120 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:47:59 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T10:47:59.703+0000 7f7a8663a7c0 -1 osd.3 63 log_to_monitors true 2024-03-22T10:48:00.398 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 10:48:00 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T10:48:00.175+0000 7f7a75a00700 -1 osd.3 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T10:48:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:00 smithi150 ceph-mon[65513]: from='osd.3 [v2:172.21.15.150:6800/2792834726,v1:172.21.15.150:6801/2792834726]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T10:48:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:00 smithi150 ceph-mon[65513]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T10:48:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:00 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 76/369 objects degraded (20.596%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T10:48:00.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:00 smithi115 ceph-mon[78672]: from='osd.3 [v2:172.21.15.150:6800/2792834726,v1:172.21.15.150:6801/2792834726]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T10:48:00.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:00 smithi115 ceph-mon[78672]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T10:48:00.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:00 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 76/369 objects degraded (20.596%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T10:48:01.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:01.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:01 smithi150 ceph-mon[65513]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T10:48:01.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:01 smithi150 ceph-mon[65513]: osdmap e66: 6 total, 5 up, 6 in 2024-03-22T10:48:01.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:01 smithi150 ceph-mon[65513]: from='osd.3 [v2:172.21.15.150:6800/2792834726,v1:172.21.15.150:6801/2792834726]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:48:01.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:01 smithi150 ceph-mon[65513]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:48:01.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:01.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:01 smithi150 ceph-mon[65513]: pgmap v139: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:48:01.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:01.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:01 smithi115 ceph-mon[78672]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T10:48:01.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:01 smithi115 ceph-mon[78672]: osdmap e66: 6 total, 5 up, 6 in 2024-03-22T10:48:01.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:01 smithi115 ceph-mon[78672]: from='osd.3 [v2:172.21.15.150:6800/2792834726,v1:172.21.15.150:6801/2792834726]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:48:01.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:01 smithi115 ceph-mon[78672]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:48:01.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:01.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:01 smithi115 ceph-mon[78672]: pgmap v139: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/369 objects degraded (20.596%) 2024-03-22T10:48:02.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:02 smithi115 ceph-mon[78672]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:48:02.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:02 smithi115 ceph-mon[78672]: osd.3 [v2:172.21.15.150:6800/2792834726,v1:172.21.15.150:6801/2792834726] boot 2024-03-22T10:48:02.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:02 smithi115 ceph-mon[78672]: osdmap e67: 6 total, 6 up, 6 in 2024-03-22T10:48:02.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:48:02.315 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:02 smithi150 ceph-mon[65513]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:48:02.316 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:02 smithi150 ceph-mon[65513]: osd.3 [v2:172.21.15.150:6800/2792834726,v1:172.21.15.150:6801/2792834726] boot 2024-03-22T10:48:02.316 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:02 smithi150 ceph-mon[65513]: osdmap e67: 6 total, 6 up, 6 in 2024-03-22T10:48:02.316 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T10:48:03.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:03 smithi150 ceph-mon[65513]: pgmap v142: 65 pgs: 3 peering, 18 active+undersized+degraded, 17 active+undersized, 27 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:48:03.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:03 smithi150 ceph-mon[65513]: osdmap e68: 6 total, 6 up, 6 in 2024-03-22T10:48:03.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:03 smithi115 ceph-mon[78672]: pgmap v142: 65 pgs: 3 peering, 18 active+undersized+degraded, 17 active+undersized, 27 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 72/369 objects degraded (19.512%) 2024-03-22T10:48:03.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:03 smithi115 ceph-mon[78672]: osdmap e68: 6 total, 6 up, 6 in 2024-03-22T10:48:05.253 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: pgmap v143: 65 pgs: 3 peering, 13 active+undersized+degraded, 11 active+undersized, 38 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:48:05.253 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:05.253 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:48:05.253 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:05.253 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:05.253 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:48:05.253 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:48:05.254 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:05.254 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 56/369 objects degraded (15.176%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T10:48:05.254 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:48:05.254 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:05.254 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:05.254 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:05.254 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T10:48:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: pgmap v143: 65 pgs: 3 peering, 13 active+undersized+degraded, 11 active+undersized, 38 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:48:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 56/369 objects degraded (15.176%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:05.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T10:48:06.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:06 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T10:48:06.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:06 smithi150 ceph-mon[65513]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-22T10:48:06.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:06 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T10:48:06.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:06 smithi115 ceph-mon[78672]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-22T10:48:07.522 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:07 smithi115 ceph-mon[78672]: pgmap v144: 65 pgs: 3 peering, 62 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:07.522 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:07 smithi115 ceph-mon[78672]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 56/369 objects degraded (15.176%), 13 pgs degraded) 2024-03-22T10:48:07.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:07 smithi150 ceph-mon[65513]: pgmap v144: 65 pgs: 3 peering, 62 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:07.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:07 smithi150 ceph-mon[65513]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 56/369 objects degraded (15.176%), 13 pgs degraded) 2024-03-22T10:48:09.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:09 smithi150 ceph-mon[65513]: pgmap v145: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:09.675 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:09 smithi115 ceph-mon[78672]: pgmap v145: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.683+0000 7f042c600700 1 -- 172.21.15.115:0/1652570336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 msgr2=0x7f0424106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.683+0000 7f042c600700 1 --2- 172.21.15.115:0/1652570336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 0x7f0424106bb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0418009910 tx=0x7f0418009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.683+0000 7f042c600700 1 -- 172.21.15.115:0/1652570336 shutdown_connections 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.683+0000 7f042c600700 1 --2- 172.21.15.115:0/1652570336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04241079a0 0x7f0424107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.683+0000 7f042c600700 1 --2- 172.21.15.115:0/1652570336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 0x7f0424106bb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.683+0000 7f042c600700 1 -- 172.21.15.115:0/1652570336 >> 172.21.15.115:0/1652570336 conn(0x7f0424075910 msgr2=0x7f0424077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.683+0000 7f042c600700 1 -- 172.21.15.115:0/1652570336 shutdown_connections 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.683+0000 7f042c600700 1 -- 172.21.15.115:0/1652570336 wait complete. 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.684+0000 7f042c600700 1 Processor -- start 2024-03-22T10:48:09.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.684+0000 7f042c600700 1 -- start start 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.684+0000 7f042c600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 0x7f042419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.684+0000 7f042c600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04241079a0 0x7f042419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f042c600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042419cb10 con 0x7f04241067a0 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f042c600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042419cc50 con 0x7f04241079a0 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f0429c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 0x7f042419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f0429c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 0x7f042419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44710/0 (socket says 172.21.15.115:44710) 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f0429c00700 1 -- 172.21.15.115:0/3494344810 learned_addr learned my addr 172.21.15.115:0/3494344810 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f0429200700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04241079a0 0x7f042419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f0429c00700 1 -- 172.21.15.115:0/3494344810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04241079a0 msgr2=0x7f042419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:09.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f0429c00700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04241079a0 0x7f042419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:09.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f0429c00700 1 -- 172.21.15.115:0/3494344810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180095f0 con 0x7f04241067a0 2024-03-22T10:48:09.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.685+0000 7f0429200700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04241079a0 0x7f042419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:48:09.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.686+0000 7f0429c00700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 0x7f042419bfb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f041800bda0 tx=0x7f041800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:09.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.686+0000 7f0422000700 1 -- 172.21.15.115:0/3494344810 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f041801d070 con 0x7f04241067a0 2024-03-22T10:48:09.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.686+0000 7f0422000700 1 -- 172.21.15.115:0/3494344810 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0418004c50 con 0x7f04241067a0 2024-03-22T10:48:09.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.686+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04241a16a0 con 0x7f04241067a0 2024-03-22T10:48:09.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.686+0000 7f0422000700 1 -- 172.21.15.115:0/3494344810 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f041800f690 con 0x7f04241067a0 2024-03-22T10:48:09.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.686+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04241a1b90 con 0x7f04241067a0 2024-03-22T10:48:09.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.688+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042410f2f0 con 0x7f04241067a0 2024-03-22T10:48:09.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.688+0000 7f0422000700 1 -- 172.21.15.115:0/3494344810 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f041800f900 con 0x7f04241067a0 2024-03-22T10:48:09.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.689+0000 7f0422000700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f040c06dfa0 0x7f040c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:09.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.689+0000 7f0422000700 1 -- 172.21.15.115:0/3494344810 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f041808f5a0 con 0x7f04241067a0 2024-03-22T10:48:09.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.690+0000 7f0429200700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f040c06dfa0 0x7f040c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:09.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.690+0000 7f0429200700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f040c06dfa0 0x7f040c070450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0410005fd0 tx=0x7f0410005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:09.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.695+0000 7f0422000700 1 -- 172.21.15.115:0/3494344810 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f041805c540 con 0x7f04241067a0 2024-03-22T10:48:09.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.941+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f042410c2f0 con 0x7f040c06dfa0 2024-03-22T10:48:09.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.943+0000 7f0422000700 1 -- 172.21.15.115:0/3494344810 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f042410c2f0 con 0x7f040c06dfa0 2024-03-22T10:48:09.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.946+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f040c06dfa0 msgr2=0x7f040c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:09.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.946+0000 7f042c600700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f040c06dfa0 0x7f040c070450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0410005fd0 tx=0x7f0410005dc0 comp rx=0 tx=0).stop 2024-03-22T10:48:09.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.946+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 msgr2=0x7f042419bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:09.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.946+0000 7f042c600700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 0x7f042419bfb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f041800bda0 tx=0x7f041800be80 comp rx=0 tx=0).stop 2024-03-22T10:48:09.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.947+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 shutdown_connections 2024-03-22T10:48:09.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.947+0000 7f042c600700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f040c06dfa0 0x7f040c070450 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:09.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.947+0000 7f042c600700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04241067a0 0x7f042419bfb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:09.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.947+0000 7f042c600700 1 --2- 172.21.15.115:0/3494344810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04241079a0 0x7f042419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:09.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.947+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 >> 172.21.15.115:0/3494344810 conn(0x7f0424075910 msgr2=0x7f042410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:09.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.947+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 shutdown_connections 2024-03-22T10:48:09.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:09.947+0000 7f042c600700 1 -- 172.21.15.115:0/3494344810 wait complete. 2024-03-22T10:48:09.956 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:48:10.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.108+0000 7fbac6400700 1 -- 172.21.15.115:0/3816454472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01021c0 msgr2=0x7fbac01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.108+0000 7fbac6400700 1 --2- 172.21.15.115:0/3816454472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01021c0 0x7fbac01025d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009910 tx=0x7fbab4009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:10.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.108+0000 7fbac6400700 1 -- 172.21.15.115:0/3816454472 shutdown_connections 2024-03-22T10:48:10.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.108+0000 7fbac6400700 1 --2- 172.21.15.115:0/3816454472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbac01033c0 0x7fbac0103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.108+0000 7fbac6400700 1 --2- 172.21.15.115:0/3816454472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01021c0 0x7fbac01025d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.108+0000 7fbac6400700 1 -- 172.21.15.115:0/3816454472 >> 172.21.15.115:0/3816454472 conn(0x7fbac00fd8d0 msgr2=0x7fbac00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:10.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.108+0000 7fbac6400700 1 -- 172.21.15.115:0/3816454472 shutdown_connections 2024-03-22T10:48:10.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.108+0000 7fbac6400700 1 -- 172.21.15.115:0/3816454472 wait complete. 2024-03-22T10:48:10.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.109+0000 7fbac6400700 1 Processor -- start 2024-03-22T10:48:10.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.109+0000 7fbac6400700 1 -- start start 2024-03-22T10:48:10.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbac6400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbac01021c0 0x7fbac01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:10.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbac6400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01033c0 0x7fbac0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:10.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbac6400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac0198520 con 0x7fbac01033c0 2024-03-22T10:48:10.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbac6400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac0198660 con 0x7fbac01021c0 2024-03-22T10:48:10.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbabfe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01033c0 0x7fbac0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:10.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbabfe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01033c0 0x7fbac0197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44726/0 (socket says 172.21.15.115:44726) 2024-03-22T10:48:10.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbabfe00700 1 -- 172.21.15.115:0/1340363919 learned_addr learned my addr 172.21.15.115:0/1340363919 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:10.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbac5000700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbac01021c0 0x7fbac01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:10.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbabfe00700 1 -- 172.21.15.115:0/1340363919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbac01021c0 msgr2=0x7fbac01979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbabfe00700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbac01021c0 0x7fbac01979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.110+0000 7fbabfe00700 1 -- 172.21.15.115:0/1340363919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab40095f0 con 0x7fbac01033c0 2024-03-22T10:48:10.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.111+0000 7fbabfe00700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01033c0 0x7fbac0197f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c8a0 tx=0x7fbaac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:10.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.111+0000 7fbabd600700 1 -- 172.21.15.115:0/1340363919 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaac00cea0 con 0x7fbac01033c0 2024-03-22T10:48:10.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.111+0000 7fbabd600700 1 -- 172.21.15.115:0/1340363919 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbaac004830 con 0x7fbac01033c0 2024-03-22T10:48:10.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.111+0000 7fbabd600700 1 -- 172.21.15.115:0/1340363919 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaac0055b0 con 0x7fbac01033c0 2024-03-22T10:48:10.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.111+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac019d110 con 0x7fbac01033c0 2024-03-22T10:48:10.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.111+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac019d610 con 0x7fbac01033c0 2024-03-22T10:48:10.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.112+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac010ad10 con 0x7fbac01033c0 2024-03-22T10:48:10.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.114+0000 7fbabd600700 1 -- 172.21.15.115:0/1340363919 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fbaac007ad0 con 0x7fbac01033c0 2024-03-22T10:48:10.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.115+0000 7fbabd600700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbaa806dfa0 0x7fbaa8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:10.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.115+0000 7fbabd600700 1 -- 172.21.15.115:0/1340363919 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbaac08dac0 con 0x7fbac01033c0 2024-03-22T10:48:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.115+0000 7fbac5000700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbaa806dfa0 0x7fbaa8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.116+0000 7fbac5000700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbaa806dfa0 0x7fbaa8070450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbab40059d0 tx=0x7fbab4005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.120+0000 7fbabd600700 1 -- 172.21.15.115:0/1340363919 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbaac05aa20 con 0x7fbac01033c0 2024-03-22T10:48:10.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.366+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbac0107d10 con 0x7fbaa806dfa0 2024-03-22T10:48:10.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.369+0000 7fbabd600700 1 -- 172.21.15.115:0/1340363919 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbac0107d10 con 0x7fbaa806dfa0 2024-03-22T10:48:10.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.371+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbaa806dfa0 msgr2=0x7fbaa8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.371+0000 7fbac6400700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbaa806dfa0 0x7fbaa8070450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbab40059d0 tx=0x7fbab4005960 comp rx=0 tx=0).stop 2024-03-22T10:48:10.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.371+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01033c0 msgr2=0x7fbac0197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.371+0000 7fbac6400700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01033c0 0x7fbac0197f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c8a0 tx=0x7fbaac00cc60 comp rx=0 tx=0).stop 2024-03-22T10:48:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.371+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 shutdown_connections 2024-03-22T10:48:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.371+0000 7fbac6400700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbaa806dfa0 0x7fbaa8070450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.372+0000 7fbac6400700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbac01021c0 0x7fbac01979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.372+0000 7fbac6400700 1 --2- 172.21.15.115:0/1340363919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbac01033c0 0x7fbac0197f00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.372+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 >> 172.21.15.115:0/1340363919 conn(0x7fbac00fd8d0 msgr2=0x7fbac01065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.372+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 shutdown_connections 2024-03-22T10:48:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.372+0000 7fbac6400700 1 -- 172.21.15.115:0/1340363919 wait complete. 2024-03-22T10:48:10.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.518+0000 7f64aaa00700 1 -- 172.21.15.115:0/3297681580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a40737b0 msgr2=0x7f64a4073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.518+0000 7f64aaa00700 1 --2- 172.21.15.115:0/3297681580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a40737b0 0x7f64a4073c20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f648c009940 tx=0x7f648c009c50 comp rx=0 tx=0).stop 2024-03-22T10:48:10.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.518+0000 7f64aaa00700 1 -- 172.21.15.115:0/3297681580 shutdown_connections 2024-03-22T10:48:10.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.518+0000 7f64aaa00700 1 --2- 172.21.15.115:0/3297681580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a40737b0 0x7f64a4073c20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.518+0000 7f64aaa00700 1 --2- 172.21.15.115:0/3297681580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a4074d80 0x7f64a40731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.518+0000 7f64aaa00700 1 -- 172.21.15.115:0/3297681580 >> 172.21.15.115:0/3297681580 conn(0x7f64a40fb6e0 msgr2=0x7f64a40fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:10.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.519+0000 7f64aaa00700 1 -- 172.21.15.115:0/3297681580 shutdown_connections 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.519+0000 7f64aaa00700 1 -- 172.21.15.115:0/3297681580 wait complete. 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.520+0000 7f64aaa00700 1 Processor -- start 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.520+0000 7f64aaa00700 1 -- start start 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.520+0000 7f64aaa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a40737b0 0x7f64a4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.520+0000 7f64aaa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a4074d80 0x7f64a41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.520+0000 7f64aaa00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a41986c0 con 0x7f64a4074d80 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.520+0000 7f64aaa00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4198800 con 0x7f64a40737b0 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.520+0000 7f649b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a4074d80 0x7f64a41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f649b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a4074d80 0x7f64a41980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44738/0 (socket says 172.21.15.115:44738) 2024-03-22T10:48:10.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f649b400700 1 -- 172.21.15.115:0/1423440406 learned_addr learned my addr 172.21.15.115:0/1423440406 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:10.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f649b400700 1 -- 172.21.15.115:0/1423440406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a40737b0 msgr2=0x7f64a4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f649b400700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a40737b0 0x7f64a4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f649b400700 1 -- 172.21.15.115:0/1423440406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c0095f0 con 0x7f64a4074d80 2024-03-22T10:48:10.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f649b400700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a4074d80 0x7f64a41980a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f648c0055f0 tx=0x7f648c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:10.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f64a1600700 1 -- 172.21.15.115:0/1423440406 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f648c01d070 con 0x7f64a4074d80 2024-03-22T10:48:10.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f64a1600700 1 -- 172.21.15.115:0/1423440406 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f648c00f460 con 0x7f64a4074d80 2024-03-22T10:48:10.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a419d250 con 0x7f64a4074d80 2024-03-22T10:48:10.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.521+0000 7f64a1600700 1 -- 172.21.15.115:0/1423440406 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f648c021620 con 0x7f64a4074d80 2024-03-22T10:48:10.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.522+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a419d7c0 con 0x7f64a4074d80 2024-03-22T10:48:10.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.523+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a410ae90 con 0x7f64a4074d80 2024-03-22T10:48:10.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.524+0000 7f64a1600700 1 -- 172.21.15.115:0/1423440406 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f648c00fa50 con 0x7f64a4074d80 2024-03-22T10:48:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.524+0000 7f64a1600700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f648806dfa0 0x7f6488070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.524+0000 7f64a1600700 1 -- 172.21.15.115:0/1423440406 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f648c08efb0 con 0x7f64a4074d80 2024-03-22T10:48:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.525+0000 7f64a3e00700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f648806dfa0 0x7f6488070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.525+0000 7f64a3e00700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f648806dfa0 0x7f6488070450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64940060b0 tx=0x7f6494006040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:10.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.530+0000 7f64a1600700 1 -- 172.21.15.115:0/1423440406 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f648c0c3740 con 0x7f64a4074d80 2024-03-22T10:48:10.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.750+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f64a4107650 con 0x7f648806dfa0 2024-03-22T10:48:10.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.767+0000 7f64a1600700 1 -- 172.21.15.115:0/1423440406 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f64a4107650 con 0x7f648806dfa0 2024-03-22T10:48:10.759 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:48:10.759 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (14m) 38s ago 16m 14.2M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:48:10.759 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (16m) 38s ago 16m 8694k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:48:10.759 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (15m) 12s ago 15m 8120k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:48:10.759 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (2m) 38s ago 16m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:48:10.759 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (2m) 12s ago 15m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (14m) 38s ago 16m 73.7M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (9m) 38s ago 9m 127M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (9m) 38s ago 9m 16.9M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (9m) 12s ago 9m 16.4M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (9m) 12s ago 9m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (6m) 38s ago 18m 507M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (4m) 12s ago 15m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (4m) 38s ago 18m 46.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (3m) 12s ago 15m 37.0M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (16m) 38s ago 16m 8238k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (15m) 12s ago 15m 8140k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (118s) 38s ago 14m 84.2M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (74s) 38s ago 13m 81.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (45s) 38s ago 12m 11.1M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:48:10.760 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (17s) 12s ago 12m 11.1M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:48:10.761 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (11m) 12s ago 11m 228M 2241M 18.2.0 dc2bc1663786 a6c769e3c602 2024-03-22T10:48:10.761 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (11m) 12s ago 11m 196M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:48:10.761 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (4m) 38s ago 15m 71.2M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:48:10.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f648806dfa0 msgr2=0x7f6488070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f648806dfa0 0x7f6488070450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64940060b0 tx=0x7f6494006040 comp rx=0 tx=0).stop 2024-03-22T10:48:10.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a4074d80 msgr2=0x7f64a41980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a4074d80 0x7f64a41980a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f648c0055f0 tx=0x7f648c0056d0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 shutdown_connections 2024-03-22T10:48:10.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f648806dfa0 0x7f6488070450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a40737b0 0x7f64a4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 --2- 172.21.15.115:0/1423440406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a4074d80 0x7f64a41980a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 >> 172.21.15.115:0/1423440406 conn(0x7f64a40fb6e0 msgr2=0x7f64a4105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:10.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 shutdown_connections 2024-03-22T10:48:10.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.770+0000 7f64aaa00700 1 -- 172.21.15.115:0/1423440406 wait complete. 2024-03-22T10:48:10.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.923+0000 7f14f8600700 1 -- 172.21.15.115:0/583507128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0073730 msgr2=0x7f14f0073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.923+0000 7f14f8600700 1 --2- 172.21.15.115:0/583507128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0073730 0x7f14f0073ba0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009910 tx=0x7f14dc009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:10.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.923+0000 7f14f8600700 1 -- 172.21.15.115:0/583507128 shutdown_connections 2024-03-22T10:48:10.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.923+0000 7f14f8600700 1 --2- 172.21.15.115:0/583507128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0073730 0x7f14f0073ba0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.923+0000 7f14f8600700 1 --2- 172.21.15.115:0/583507128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14f0074d00 0x7f14f0073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.923+0000 7f14f8600700 1 -- 172.21.15.115:0/583507128 >> 172.21.15.115:0/583507128 conn(0x7f14f00fb6e0 msgr2=0x7f14f00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:10.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.923+0000 7f14f8600700 1 -- 172.21.15.115:0/583507128 shutdown_connections 2024-03-22T10:48:10.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.923+0000 7f14f8600700 1 -- 172.21.15.115:0/583507128 wait complete. 2024-03-22T10:48:10.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.924+0000 7f14f8600700 1 Processor -- start 2024-03-22T10:48:10.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.925+0000 7f14f8600700 1 -- start start 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.925+0000 7f14f8600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14f0073730 0x7f14f0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.925+0000 7f14f8600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0074d00 0x7f14f01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.925+0000 7f14f8600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f0198700 con 0x7f14f0074d00 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.925+0000 7f14f8600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f0198840 con 0x7f14f0073730 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.925+0000 7f14f5200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0074d00 0x7f14f01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.925+0000 7f14f5200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0074d00 0x7f14f01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44760/0 (socket says 172.21.15.115:44760) 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.925+0000 7f14f5200700 1 -- 172.21.15.115:0/4127262919 learned_addr learned my addr 172.21.15.115:0/4127262919 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.925+0000 7f14f5c00700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14f0073730 0x7f14f0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.926+0000 7f14f5200700 1 -- 172.21.15.115:0/4127262919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14f0073730 msgr2=0x7f14f0197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.926+0000 7f14f5200700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14f0073730 0x7f14f0197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.926+0000 7f14f5200700 1 -- 172.21.15.115:0/4127262919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc0095f0 con 0x7f14f0074d00 2024-03-22T10:48:10.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.926+0000 7f14f5200700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0074d00 0x7f14f01980e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00b5c0 tx=0x7f14dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:10.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.926+0000 7f14ee000700 1 -- 172.21.15.115:0/4127262919 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14dc01d070 con 0x7f14f0074d00 2024-03-22T10:48:10.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.926+0000 7f14ee000700 1 -- 172.21.15.115:0/4127262919 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f14dc022470 con 0x7f14f0074d00 2024-03-22T10:48:10.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.926+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f019d290 con 0x7f14f0074d00 2024-03-22T10:48:10.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.926+0000 7f14ee000700 1 -- 172.21.15.115:0/4127262919 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14dc00f690 con 0x7f14f0074d00 2024-03-22T10:48:10.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.926+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f019d800 con 0x7f14f0074d00 2024-03-22T10:48:10.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.928+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f010af20 con 0x7f14f0074d00 2024-03-22T10:48:10.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.929+0000 7f14ee000700 1 -- 172.21.15.115:0/4127262919 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f14dc004b70 con 0x7f14f0074d00 2024-03-22T10:48:10.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.929+0000 7f14ee000700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f14d806dfa0 0x7f14d8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:10.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.930+0000 7f14ee000700 1 -- 172.21.15.115:0/4127262919 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f14dc08f460 con 0x7f14f0074d00 2024-03-22T10:48:10.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.930+0000 7f14f5c00700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f14d806dfa0 0x7f14d8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:10.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.930+0000 7f14f5c00700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f14d806dfa0 0x7f14d8070450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14e4006fd0 tx=0x7f14e4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:10.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:10.934+0000 7f14ee000700 1 -- 172.21.15.115:0/4127262919 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f14dc05c4f0 con 0x7f14f0074d00 2024-03-22T10:48:11.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.249+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f14f019df50 con 0x7f14f0074d00 2024-03-22T10:48:11.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.250+0000 7f14ee000700 1 -- 172.21.15.115:0/4127262919 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f14dc05c080 con 0x7f14f0074d00 2024-03-22T10:48:11.242 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:48:11.242 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:48:11.243 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:48:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6, 2024-03-22T10:48:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 8 2024-03-22T10:48:11.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:48:11.244 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f14d806dfa0 msgr2=0x7f14d8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f14d806dfa0 0x7f14d8070450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14e4006fd0 tx=0x7f14e4008040 comp rx=0 tx=0).stop 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0074d00 msgr2=0x7f14f01980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0074d00 0x7f14f01980e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00b5c0 tx=0x7f14dc00be20 comp rx=0 tx=0).stop 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 shutdown_connections 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f14d806dfa0 0x7f14d8070450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14f0073730 0x7f14f0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 --2- 172.21.15.115:0/4127262919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14f0074d00 0x7f14f01980e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 >> 172.21.15.115:0/4127262919 conn(0x7f14f00fb6e0 msgr2=0x7f14f0105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 shutdown_connections 2024-03-22T10:48:11.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.253+0000 7f14f8600700 1 -- 172.21.15.115:0/4127262919 wait complete. 2024-03-22T10:48:11.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.399+0000 7f791b600700 1 -- 172.21.15.115:0/328014398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914100be0 msgr2=0x7f7914102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:11.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.399+0000 7f791b600700 1 --2- 172.21.15.115:0/328014398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914100be0 0x7f7914102fc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7908009910 tx=0x7f7908009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:11.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.399+0000 7f791b600700 1 -- 172.21.15.115:0/328014398 shutdown_connections 2024-03-22T10:48:11.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.399+0000 7f791b600700 1 --2- 172.21.15.115:0/328014398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914103500 0x7f79141058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.399+0000 7f791b600700 1 --2- 172.21.15.115:0/328014398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914100be0 0x7f7914102fc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.399+0000 7f791b600700 1 -- 172.21.15.115:0/328014398 >> 172.21.15.115:0/328014398 conn(0x7f79140fa5e0 msgr2=0x7f79140fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:11.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.399+0000 7f791b600700 1 -- 172.21.15.115:0/328014398 shutdown_connections 2024-03-22T10:48:11.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.400+0000 7f791b600700 1 -- 172.21.15.115:0/328014398 wait complete. 2024-03-22T10:48:11.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.401+0000 7f791b600700 1 Processor -- start 2024-03-22T10:48:11.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.401+0000 7f791b600700 1 -- start start 2024-03-22T10:48:11.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.401+0000 7f791b600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914100be0 0x7f7914197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:11.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.401+0000 7f791b600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914103500 0x7f7914198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:11.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.401+0000 7f791b600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914198660 con 0x7f7914103500 2024-03-22T10:48:11.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.401+0000 7f791b600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79141987a0 con 0x7f7914100be0 2024-03-22T10:48:11.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.401+0000 7f7913e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914103500 0x7f7914198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.401+0000 7f7913e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914103500 0x7f7914198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44780/0 (socket says 172.21.15.115:44780) 2024-03-22T10:48:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.401+0000 7f7913e00700 1 -- 172.21.15.115:0/1183950755 learned_addr learned my addr 172.21.15.115:0/1183950755 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f7918c00700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914100be0 0x7f7914197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f7913e00700 1 -- 172.21.15.115:0/1183950755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914100be0 msgr2=0x7f7914197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f7913e00700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914100be0 0x7f7914197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f7913e00700 1 -- 172.21.15.115:0/1183950755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080095f0 con 0x7f7914103500 2024-03-22T10:48:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f7918c00700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914100be0 0x7f7914197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:48:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f7913e00700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914103500 0x7f7914198040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f790000d8d0 tx=0x7f790000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f7911600700 1 -- 172.21.15.115:0/1183950755 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7900009940 con 0x7f7914103500 2024-03-22T10:48:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f7911600700 1 -- 172.21.15.115:0/1183950755 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7900010460 con 0x7f7914103500 2024-03-22T10:48:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791419d250 con 0x7f7914103500 2024-03-22T10:48:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.402+0000 7f7911600700 1 -- 172.21.15.115:0/1183950755 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f790000f5f0 con 0x7f7914103500 2024-03-22T10:48:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.403+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791419d7a0 con 0x7f7914103500 2024-03-22T10:48:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.404+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79140fc1c0 con 0x7f7914103500 2024-03-22T10:48:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.404+0000 7f7911600700 1 -- 172.21.15.115:0/1183950755 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f79000105d0 con 0x7f7914103500 2024-03-22T10:48:11.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.405+0000 7f7911600700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f78fc06dfa0 0x7f78fc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:11.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.405+0000 7f7911600700 1 -- 172.21.15.115:0/1183950755 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f790008dd70 con 0x7f7914103500 2024-03-22T10:48:11.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.406+0000 7f7918c00700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f78fc06dfa0 0x7f78fc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:11.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.406+0000 7f7918c00700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f78fc06dfa0 0x7f78fc070450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7908006010 tx=0x7f790800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:11.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.410+0000 7f7911600700 1 -- 172.21.15.115:0/1183950755 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f790005acd0 con 0x7f7914103500 2024-03-22T10:48:11.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:11 smithi115 ceph-mon[78672]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:11.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:11 smithi115 ceph-mon[78672]: pgmap v146: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:11.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:11 smithi115 ceph-mon[78672]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:11.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:11 smithi115 ceph-mon[78672]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:11.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:11 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/4127262919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:11.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:11 smithi150 ceph-mon[65513]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:11.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:11 smithi150 ceph-mon[65513]: pgmap v146: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:11.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:11 smithi150 ceph-mon[65513]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:11.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:11 smithi150 ceph-mon[65513]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:11.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:11 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/4127262919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:11.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.667+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f791402cc30 con 0x7f7914103500 2024-03-22T10:48:11.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.668+0000 7f7911600700 1 -- 172.21.15.115:0/1183950755 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1907 (secure 0 0 0) 0x7f790005a860 con 0x7f7914103500 2024-03-22T10:48:11.660 INFO:teuthology.orchestra.run.smithi115.stdout:e16 2024-03-22T10:48:11.660 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:48:11.660 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:48:11.660 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 16 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:47:17.573634+0000 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:48:11.661 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:48:11.662 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 14454 members: 24241,14454 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:48:11.663 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f78fc06dfa0 msgr2=0x7f78fc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f78fc06dfa0 0x7f78fc070450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7908006010 tx=0x7f790800b540 comp rx=0 tx=0).stop 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914103500 msgr2=0x7f7914198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914103500 0x7f7914198040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f790000d8d0 tx=0x7f790000dc90 comp rx=0 tx=0).stop 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 shutdown_connections 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f78fc06dfa0 0x7f78fc070450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7914100be0 0x7f7914197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 --2- 172.21.15.115:0/1183950755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914103500 0x7f7914198040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 >> 172.21.15.115:0/1183950755 conn(0x7f79140fa5e0 msgr2=0x7f79140ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:11.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.671+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 shutdown_connections 2024-03-22T10:48:11.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.672+0000 7f791b600700 1 -- 172.21.15.115:0/1183950755 wait complete. 2024-03-22T10:48:11.665 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2024-03-22T10:48:11.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.824+0000 7ff39be00700 1 -- 172.21.15.115:0/2298362940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3941035a0 msgr2=0x7ff3941039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:11.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.824+0000 7ff39be00700 1 --2- 172.21.15.115:0/2298362940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3941035a0 0x7ff3941039f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff380009910 tx=0x7ff380009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:11.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.824+0000 7ff39be00700 1 -- 172.21.15.115:0/2298362940 shutdown_connections 2024-03-22T10:48:11.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.824+0000 7ff39be00700 1 --2- 172.21.15.115:0/2298362940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3941035a0 0x7ff3941039f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.824+0000 7ff39be00700 1 --2- 172.21.15.115:0/2298362940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3941023a0 0x7ff3941027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.824+0000 7ff39be00700 1 -- 172.21.15.115:0/2298362940 >> 172.21.15.115:0/2298362940 conn(0x7ff3940fd990 msgr2=0x7ff3940ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:11.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.825+0000 7ff39be00700 1 -- 172.21.15.115:0/2298362940 shutdown_connections 2024-03-22T10:48:11.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.825+0000 7ff39be00700 1 -- 172.21.15.115:0/2298362940 wait complete. 2024-03-22T10:48:11.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.826+0000 7ff39be00700 1 Processor -- start 2024-03-22T10:48:11.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.826+0000 7ff39be00700 1 -- start start 2024-03-22T10:48:11.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.826+0000 7ff39be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3941023a0 0x7ff394197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:11.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.826+0000 7ff39be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394198320 0x7ff39419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:11.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff39be00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff394198820 con 0x7ff394198320 2024-03-22T10:48:11.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff39be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff394198990 con 0x7ff3941023a0 2024-03-22T10:48:11.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff398a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394198320 0x7ff39419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:11.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff398a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394198320 0x7ff39419d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60596/0 (socket says 172.21.15.115:60596) 2024-03-22T10:48:11.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff398a00700 1 -- 172.21.15.115:0/3292109311 learned_addr learned my addr 172.21.15.115:0/3292109311 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:11.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff399400700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3941023a0 0x7ff394197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:11.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff398a00700 1 -- 172.21.15.115:0/3292109311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3941023a0 msgr2=0x7ff394197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:11.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff398a00700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3941023a0 0x7ff394197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:11.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff398a00700 1 -- 172.21.15.115:0/3292109311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3800095f0 con 0x7ff394198320 2024-03-22T10:48:11.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.827+0000 7ff398a00700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394198320 0x7ff39419d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff380000c00 tx=0x7ff38000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:11.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.828+0000 7ff392000700 1 -- 172.21.15.115:0/3292109311 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff38001d070 con 0x7ff394198320 2024-03-22T10:48:11.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.828+0000 7ff392000700 1 -- 172.21.15.115:0/3292109311 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff38000f460 con 0x7ff394198320 2024-03-22T10:48:11.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.828+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39419d8d0 con 0x7ff394198320 2024-03-22T10:48:11.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.828+0000 7ff392000700 1 -- 172.21.15.115:0/3292109311 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff380021640 con 0x7ff394198320 2024-03-22T10:48:11.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.828+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39419dd90 con 0x7ff394198320 2024-03-22T10:48:11.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.829+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39410aef0 con 0x7ff394198320 2024-03-22T10:48:11.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.830+0000 7ff392000700 1 -- 172.21.15.115:0/3292109311 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff38000fad0 con 0x7ff394198320 2024-03-22T10:48:11.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.831+0000 7ff392000700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37c06dfa0 0x7ff37c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:11.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.831+0000 7ff392000700 1 -- 172.21.15.115:0/3292109311 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff38008ef10 con 0x7ff394198320 2024-03-22T10:48:11.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.831+0000 7ff399400700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37c06dfa0 0x7ff37c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:11.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.832+0000 7ff399400700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37c06dfa0 0x7ff37c070450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff388005950 tx=0x7ff3880058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:11.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:11.836+0000 7ff392000700 1 -- 172.21.15.115:0/3292109311 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff38005bef0 con 0x7ff394198320 2024-03-22T10:48:12.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.082+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff394107ef0 con 0x7ff37c06dfa0 2024-03-22T10:48:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.085+0000 7ff392000700 1 -- 172.21.15.115:0/3292109311 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff394107ef0 con 0x7ff37c06dfa0 2024-03-22T10:48:12.077 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:48:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:48:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:48:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:48:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:48:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:48:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:48:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:48:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:48:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "10/23 daemons upgraded", 2024-03-22T10:48:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T10:48:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:48:12.078 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:48:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37c06dfa0 msgr2=0x7ff37c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37c06dfa0 0x7ff37c070450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff388005950 tx=0x7ff3880058e0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394198320 msgr2=0x7ff39419d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394198320 0x7ff39419d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff380000c00 tx=0x7ff38000be20 comp rx=0 tx=0).stop 2024-03-22T10:48:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 shutdown_connections 2024-03-22T10:48:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37c06dfa0 0x7ff37c070450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3941023a0 0x7ff394197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 --2- 172.21.15.115:0/3292109311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff394198320 0x7ff39419d390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 >> 172.21.15.115:0/3292109311 conn(0x7ff3940fd990 msgr2=0x7ff3941067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 shutdown_connections 2024-03-22T10:48:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.088+0000 7ff39be00700 1 -- 172.21.15.115:0/3292109311 wait complete. 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.241+0000 7f94f9c00700 1 -- 172.21.15.115:0/1225239002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 msgr2=0x7f94f41039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.241+0000 7f94f9c00700 1 --2- 172.21.15.115:0/1225239002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 0x7f94f41039f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009910 tx=0x7f94e0009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.241+0000 7f94f9c00700 1 -- 172.21.15.115:0/1225239002 shutdown_connections 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.241+0000 7f94f9c00700 1 --2- 172.21.15.115:0/1225239002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 0x7f94f41039f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.241+0000 7f94f9c00700 1 --2- 172.21.15.115:0/1225239002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94f41023a0 0x7f94f41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.241+0000 7f94f9c00700 1 -- 172.21.15.115:0/1225239002 >> 172.21.15.115:0/1225239002 conn(0x7f94f40fd990 msgr2=0x7f94f40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.241+0000 7f94f9c00700 1 -- 172.21.15.115:0/1225239002 shutdown_connections 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.241+0000 7f94f9c00700 1 -- 172.21.15.115:0/1225239002 wait complete. 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.242+0000 7f94f9c00700 1 Processor -- start 2024-03-22T10:48:12.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.242+0000 7f94f9c00700 1 -- start start 2024-03-22T10:48:12.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f9c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94f41023a0 0x7f94f4197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:12.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f9c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 0x7f94f41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:12.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f9c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f4198700 con 0x7f94f41035a0 2024-03-22T10:48:12.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f9c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f4198840 con 0x7f94f41023a0 2024-03-22T10:48:12.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f2000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 0x7f94f41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:12.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f2000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 0x7f94f41980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60622/0 (socket says 172.21.15.115:60622) 2024-03-22T10:48:12.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f2000700 1 -- 172.21.15.115:0/2874404776 learned_addr learned my addr 172.21.15.115:0/2874404776 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:12.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94f41023a0 0x7f94f4197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f2000700 1 -- 172.21.15.115:0/2874404776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94f41023a0 msgr2=0x7f94f4197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f2000700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94f41023a0 0x7f94f4197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.243+0000 7f94f2000700 1 -- 172.21.15.115:0/2874404776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e00095f0 con 0x7f94f41035a0 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.244+0000 7f94f2000700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 0x7f94f41980e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94e000b5c0 tx=0x7f94e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.244+0000 7f94eb400700 1 -- 172.21.15.115:0/2874404776 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e001d070 con 0x7f94f41035a0 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.244+0000 7f94eb400700 1 -- 172.21.15.115:0/2874404776 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94e0022470 con 0x7f94f41035a0 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.244+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f419d290 con 0x7f94f41035a0 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.244+0000 7f94eb400700 1 -- 172.21.15.115:0/2874404776 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e000f690 con 0x7f94f41035a0 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.244+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f419d780 con 0x7f94f41035a0 2024-03-22T10:48:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.246+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f410aef0 con 0x7f94f41035a0 2024-03-22T10:48:12.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.247+0000 7f94eb400700 1 -- 172.21.15.115:0/2874404776 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f94e00225e0 con 0x7f94f41035a0 2024-03-22T10:48:12.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.248+0000 7f94eb400700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f94d806dfa0 0x7f94d8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:12.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.248+0000 7f94eb400700 1 -- 172.21.15.115:0/2874404776 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f94e008f580 con 0x7f94f41035a0 2024-03-22T10:48:12.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.248+0000 7f94f2a00700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f94d806dfa0 0x7f94d8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:12.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.248+0000 7f94f2a00700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f94d806dfa0 0x7f94d8070450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94f4103400 tx=0x7f94e400a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:12.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.252+0000 7f94eb400700 1 -- 172.21.15.115:0/2874404776 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f94e005c4e0 con 0x7f94f41035a0 2024-03-22T10:48:12.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:12 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1183950755' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:48:12.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:12 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1183950755' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:48:12.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.550+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f94f419de80 con 0x7f94f41035a0 2024-03-22T10:48:12.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.550+0000 7f94eb400700 1 -- 172.21.15.115:0/2874404776 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f94e005c070 con 0x7f94f41035a0 2024-03-22T10:48:12.542 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:48:12.542 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:48:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:48:12.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f94d806dfa0 msgr2=0x7f94d8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:12.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f94d806dfa0 0x7f94d8070450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94f4103400 tx=0x7f94e400a380 comp rx=0 tx=0).stop 2024-03-22T10:48:12.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 msgr2=0x7f94f41980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:12.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 0x7f94f41980e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94e000b5c0 tx=0x7f94e000bd70 comp rx=0 tx=0).stop 2024-03-22T10:48:12.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 shutdown_connections 2024-03-22T10:48:12.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f94d806dfa0 0x7f94d8070450 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94f41023a0 0x7f94f4197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 --2- 172.21.15.115:0/2874404776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94f41035a0 0x7f94f41980e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:12.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 >> 172.21.15.115:0/2874404776 conn(0x7f94f40fd990 msgr2=0x7f94f41067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:12.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 shutdown_connections 2024-03-22T10:48:12.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:12.553+0000 7f94f9c00700 1 -- 172.21.15.115:0/2874404776 wait complete. 2024-03-22T10:48:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:13 smithi150 ceph-mon[65513]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:13 smithi150 ceph-mon[65513]: pgmap v147: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:13 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2874404776' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:48:13.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:13 smithi115 ceph-mon[78672]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:13.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:13 smithi115 ceph-mon[78672]: pgmap v147: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:13.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:13 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2874404776' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:48:15.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:15 smithi150 ceph-mon[65513]: pgmap v148: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:15.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:15.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:15 smithi115 ceph-mon[78672]: pgmap v148: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:15.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:17.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:17 smithi150 ceph-mon[65513]: pgmap v149: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:17.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:17 smithi115 ceph-mon[78672]: pgmap v149: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:19.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:19 smithi150 ceph-mon[65513]: pgmap v150: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:19.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:19.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:48:19.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:19 smithi115 ceph-mon[78672]: pgmap v150: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:19.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:19.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:48:20.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:20 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T10:48:20.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:20 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T10:48:21.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:21 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T10:48:21.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:21 smithi150 ceph-mon[65513]: Upgrade: osd.4 is safe to restart 2024-03-22T10:48:21.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:21 smithi150 ceph-mon[65513]: pgmap v151: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:21.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:21 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T10:48:21.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:21 smithi115 ceph-mon[78672]: Upgrade: osd.4 is safe to restart 2024-03-22T10:48:21.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:21 smithi115 ceph-mon[78672]: pgmap v151: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:23.388 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:23 smithi150 ceph-mon[65513]: pgmap v152: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:23.388 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:23 smithi150 ceph-mon[65513]: Upgrade: Updating osd.4 2024-03-22T10:48:23.388 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:23 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:23.388 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:23 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T10:48:23.388 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:23 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:48:23.388 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:23 smithi150 ceph-mon[65513]: Deploying daemon osd.4 on smithi150 2024-03-22T10:48:23.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:23 smithi115 ceph-mon[78672]: pgmap v152: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:23 smithi115 ceph-mon[78672]: Upgrade: Updating osd.4 2024-03-22T10:48:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T10:48:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:48:23.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:23 smithi115 ceph-mon[78672]: Deploying daemon osd.4 on smithi150 2024-03-22T10:48:26.095 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:25 smithi150 ceph-mon[65513]: pgmap v153: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:26.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:25 smithi115 ceph-mon[78672]: pgmap v153: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:27.648 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:27 smithi150 systemd[1]: Stopping Ceph osd.4 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:48:28.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:27 smithi150 ceph-mon[65513]: pgmap v154: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:28.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:27 smithi150 ceph-mon[65513]: osd.4 marked itself down and dead 2024-03-22T10:48:28.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:27 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[45143]: 2024-03-22T10:48:27.749+0000 7f7778600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T10:48:28.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:27 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[45143]: 2024-03-22T10:48:27.749+0000 7f7778600700 -1 osd.4 68 *** Got signal Terminated *** 2024-03-22T10:48:28.149 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:27 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[45143]: 2024-03-22T10:48:27.749+0000 7f7778600700 -1 osd.4 68 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T10:48:28.202 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:27 smithi115 ceph-mon[78672]: pgmap v154: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:28.277 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:27 smithi115 ceph-mon[78672]: osd.4 marked itself down and dead 2024-03-22T10:48:28.498 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:28 smithi150 podman[73105]: 2024-03-22 10:48:28.224476599 +0000 UTC m=+0.618943766 container died a6c769e3c6022451a54ee203db5164cc4f7bc63d951fbbde977009db91499d05 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=HEAD) 2024-03-22T10:48:28.801 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:28 smithi150 podman[73105]: 2024-03-22 10:48:28.618382865 +0000 UTC m=+1.012850069 container cleanup a6c769e3c6022451a54ee203db5164cc4f7bc63d951fbbde977009db91499d05 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, ceph=True) 2024-03-22T10:48:28.801 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:28 smithi150 bash[73105]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4 2024-03-22T10:48:29.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:28 smithi150 ceph-mon[65513]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:48:29.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:28 smithi150 ceph-mon[65513]: osdmap e69: 6 total, 5 up, 6 in 2024-03-22T10:48:29.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:28 smithi150 ceph-mon[65513]: osdmap e70: 6 total, 5 up, 6 in 2024-03-22T10:48:29.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:28 smithi150 podman[73125]: 2024-03-22 10:48:28.919899995 +0000 UTC m=+0.685000266 container remove a6c769e3c6022451a54ee203db5164cc4f7bc63d951fbbde977009db91499d05 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-22T10:48:29.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:28 smithi115 ceph-mon[78672]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:48:29.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:28 smithi115 ceph-mon[78672]: osdmap e69: 6 total, 5 up, 6 in 2024-03-22T10:48:29.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:28 smithi115 ceph-mon[78672]: osdmap e70: 6 total, 5 up, 6 in 2024-03-22T10:48:29.778 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:29 smithi150 podman[73213]: 2024-03-22 10:48:29.345595572 +0000 UTC m=+0.036027258 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:48:29.778 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:29 smithi150 podman[73213]: 2024-03-22 10:48:29.45876288 +0000 UTC m=+0.149194588 container create 4708e55ea3cfec1eb94cb500da2a6ec515b5ff9d61cce4e73731aedbadc63f6f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-22T10:48:30.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:29 smithi150 ceph-mon[65513]: pgmap v156: 65 pgs: 6 stale+active+clean, 59 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:30.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:29 smithi150 podman[73213]: 2024-03-22 10:48:29.916509532 +0000 UTC m=+0.606941247 container init 4708e55ea3cfec1eb94cb500da2a6ec515b5ff9d61cce4e73731aedbadc63f6f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-22T10:48:30.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:29 smithi150 podman[73213]: 2024-03-22 10:48:29.927669318 +0000 UTC m=+0.618101059 container start 4708e55ea3cfec1eb94cb500da2a6ec515b5ff9d61cce4e73731aedbadc63f6f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-22T10:48:30.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:30 smithi150 podman[73213]: 2024-03-22 10:48:30.015334229 +0000 UTC m=+0.705765991 container attach 4708e55ea3cfec1eb94cb500da2a6ec515b5ff9d61cce4e73731aedbadc63f6f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:48:30.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:29 smithi115 ceph-mon[78672]: pgmap v156: 65 pgs: 6 stale+active+clean, 59 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:30.648 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:30 smithi150 podman[73213]: 2024-03-22 10:48:30.370058807 +0000 UTC m=+1.060490535 container died 4708e55ea3cfec1eb94cb500da2a6ec515b5ff9d61cce4e73731aedbadc63f6f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-22T10:48:31.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:30 smithi150 ceph-mon[65513]: Health check failed: Degraded data redundancy: 63/369 objects degraded (17.073%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T10:48:31.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:30 smithi115 ceph-mon[78672]: Health check failed: Degraded data redundancy: 63/369 objects degraded (17.073%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T10:48:31.612 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:31 smithi150 podman[73239]: 2024-03-22 10:48:31.210837663 +0000 UTC m=+0.825365355 container remove 4708e55ea3cfec1eb94cb500da2a6ec515b5ff9d61cce4e73731aedbadc63f6f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-22T10:48:31.612 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:31 smithi150 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-22T10:48:31.612 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:31 smithi150 systemd[1]: Stopped Ceph osd.4 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:48:31.612 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:31 smithi150 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.4.service: Consumed 1min 38.020s CPU time. 2024-03-22T10:48:31.873 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:31 smithi150 systemd[1]: Starting Ceph osd.4 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:48:32.129 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:31 smithi150 ceph-mon[65513]: pgmap v158: 65 pgs: 16 active+undersized+degraded, 1 stale+active+clean, 16 active+undersized, 32 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:32.130 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:31 smithi150 podman[73343]: 2024-03-22 10:48:31.885459702 +0000 UTC m=+0.036451465 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:48:32.130 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:31 smithi150 podman[73343]: 2024-03-22 10:48:31.995099749 +0000 UTC m=+0.146091548 container create fcb64b2757bc8ef79821f36b7b26bd76c2d6647119e16943ffe0945aaa9bdbbc (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:48:32.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:31 smithi115 ceph-mon[78672]: pgmap v158: 65 pgs: 16 active+undersized+degraded, 1 stale+active+clean, 16 active+undersized, 32 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:32.398 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:32 smithi150 podman[73343]: 2024-03-22 10:48:32.266057732 +0000 UTC m=+0.417049538 container init fcb64b2757bc8ef79821f36b7b26bd76c2d6647119e16943ffe0945aaa9bdbbc (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-22T10:48:32.398 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:32 smithi150 podman[73343]: 2024-03-22 10:48:32.277375399 +0000 UTC m=+0.428367170 container start fcb64b2757bc8ef79821f36b7b26bd76c2d6647119e16943ffe0945aaa9bdbbc (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-22T10:48:32.398 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:32 smithi150 podman[73343]: 2024-03-22 10:48:32.364478068 +0000 UTC m=+0.515469889 container attach fcb64b2757bc8ef79821f36b7b26bd76c2d6647119e16943ffe0945aaa9bdbbc (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T10:48:33.399 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate[73358]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T10:48:33.399 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate[73358]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T10:48:33.399 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 bash[73343]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T10:48:33.399 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 bash[73343]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T10:48:33.399 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 bash[73343]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T10:48:33.399 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 bash[73343]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T10:48:33.399 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 bash[73343]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-22T10:48:33.399 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 bash[73343]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T10:48:33.399 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 bash[73343]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-22T10:48:33.400 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate[73358]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T10:48:33.400 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate[73358]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T10:48:33.400 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate[73358]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-22T10:48:33.400 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate[73358]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T10:48:33.400 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate[73358]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-22T10:48:33.400 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 podman[73343]: 2024-03-22 10:48:33.181933334 +0000 UTC m=+1.332925143 container died fcb64b2757bc8ef79821f36b7b26bd76c2d6647119e16943ffe0945aaa9bdbbc (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-22T10:48:34.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:33 smithi150 ceph-mon[65513]: pgmap v159: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:34.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:33 smithi150 podman[73484]: 2024-03-22 10:48:33.934552821 +0000 UTC m=+0.732952827 container remove fcb64b2757bc8ef79821f36b7b26bd76c2d6647119e16943ffe0945aaa9bdbbc (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-22T10:48:34.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:33 smithi115 ceph-mon[78672]: pgmap v159: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:34.472 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:34 smithi150 podman[73544]: 2024-03-22 10:48:34.201140667 +0000 UTC m=+0.033600859 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:48:34.472 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:34 smithi150 podman[73544]: 2024-03-22 10:48:34.319088319 +0000 UTC m=+0.151548535 container create c7c7943cac44275f987843c8df9552d1b35b152eaa7d83954d489ed5bc2a83f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-22T10:48:35.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:34 smithi150 podman[73544]: 2024-03-22 10:48:34.697403169 +0000 UTC m=+0.529863404 container init c7c7943cac44275f987843c8df9552d1b35b152eaa7d83954d489ed5bc2a83f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-22T10:48:35.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:34 smithi150 podman[73544]: 2024-03-22 10:48:34.711122203 +0000 UTC m=+0.543582437 container start c7c7943cac44275f987843c8df9552d1b35b152eaa7d83954d489ed5bc2a83f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-22T10:48:35.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:34 smithi150 ceph-osd[73562]: -- 172.21.15.150:0/770607154 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559c2152d680 con 0x559c214f6c00 2024-03-22T10:48:35.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:34 smithi150 bash[73544]: c7c7943cac44275f987843c8df9552d1b35b152eaa7d83954d489ed5bc2a83f9 2024-03-22T10:48:35.148 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:34 smithi150 systemd[1]: Started Ceph osd.4 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:48:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:35 smithi115 ceph-mon[78672]: pgmap v160: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:48:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:35.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:35.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:48:35.648 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:35 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T10:48:35.277+0000 7f9667f937c0 -1 Falling back to public interface 2024-03-22T10:48:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:35 smithi150 ceph-mon[65513]: pgmap v160: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:48:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:48:37.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:37 smithi115 ceph-mon[78672]: pgmap v161: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:37.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:37 smithi150 ceph-mon[65513]: pgmap v161: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:39.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:39 smithi150 ceph-mon[65513]: pgmap v162: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:39.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:39 smithi115 ceph-mon[78672]: pgmap v162: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:40.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:40.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:40.895 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:40.895 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:40.895 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:40 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T10:48:40.492+0000 7f9667f937c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-22T10:48:40.895 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:40 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T10:48:40.903+0000 7f9667f937c0 -1 osd.4 68 log_to_monitors true 2024-03-22T10:48:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:41 smithi115 ceph-mon[78672]: pgmap v163: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:41 smithi115 ceph-mon[78672]: from='osd.4 [v2:172.21.15.150:6808/231697863,v1:172.21.15.150:6809/231697863]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T10:48:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:41 smithi115 ceph-mon[78672]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T10:48:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:41.826 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:41 smithi150 ceph-mon[65513]: pgmap v163: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:41.826 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:41 smithi150 ceph-mon[65513]: from='osd.4 [v2:172.21.15.150:6808/231697863,v1:172.21.15.150:6809/231697863]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T10:48:41.826 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:41 smithi150 ceph-mon[65513]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T10:48:41.826 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:41.826 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:41.826 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 10:48:41 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T10:48:41.470+0000 7f9657400700 -1 osd.4 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T10:48:42.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:42 smithi150 ceph-mon[65513]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T10:48:42.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:42 smithi150 ceph-mon[65513]: osdmap e71: 6 total, 5 up, 6 in 2024-03-22T10:48:42.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:42 smithi150 ceph-mon[65513]: from='osd.4 [v2:172.21.15.150:6808/231697863,v1:172.21.15.150:6809/231697863]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:48:42.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:42 smithi150 ceph-mon[65513]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:48:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.720+0000 7f52e9e00700 1 -- 172.21.15.115:0/2945029740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 msgr2=0x7f52e4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.720+0000 7f52e9e00700 1 --2- 172.21.15.115:0/2945029740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 0x7f52e4106bb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009910 tx=0x7f52d4009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:42.711 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:42 smithi115 ceph-mon[78672]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T10:48:42.711 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:42 smithi115 ceph-mon[78672]: osdmap e71: 6 total, 5 up, 6 in 2024-03-22T10:48:42.711 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:42 smithi115 ceph-mon[78672]: from='osd.4 [v2:172.21.15.150:6808/231697863,v1:172.21.15.150:6809/231697863]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:48:42.712 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:42 smithi115 ceph-mon[78672]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:48:42.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.721+0000 7f52e9e00700 1 -- 172.21.15.115:0/2945029740 shutdown_connections 2024-03-22T10:48:42.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.721+0000 7f52e9e00700 1 --2- 172.21.15.115:0/2945029740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e41079a0 0x7f52e4107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:42.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.721+0000 7f52e9e00700 1 --2- 172.21.15.115:0/2945029740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 0x7f52e4106bb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.721+0000 7f52e9e00700 1 -- 172.21.15.115:0/2945029740 >> 172.21.15.115:0/2945029740 conn(0x7f52e4075910 msgr2=0x7f52e4077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.721+0000 7f52e9e00700 1 -- 172.21.15.115:0/2945029740 shutdown_connections 2024-03-22T10:48:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.721+0000 7f52e9e00700 1 -- 172.21.15.115:0/2945029740 wait complete. 2024-03-22T10:48:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.722+0000 7f52e9e00700 1 Processor -- start 2024-03-22T10:48:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.722+0000 7f52e9e00700 1 -- start start 2024-03-22T10:48:42.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.722+0000 7f52e9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 0x7f52e419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.722+0000 7f52e9e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e41079a0 0x7f52e419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.722+0000 7f52e9e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e419cb10 con 0x7f52e41067a0 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.722+0000 7f52e9e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e419cc50 con 0x7f52e41079a0 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.722+0000 7f52e2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 0x7f52e419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.723+0000 7f52e2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 0x7f52e419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58410/0 (socket says 172.21.15.115:58410) 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.723+0000 7f52e2a00700 1 -- 172.21.15.115:0/2755805950 learned_addr learned my addr 172.21.15.115:0/2755805950 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.723+0000 7f52e2000700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e41079a0 0x7f52e419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.723+0000 7f52e2a00700 1 -- 172.21.15.115:0/2755805950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e41079a0 msgr2=0x7f52e419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.723+0000 7f52e2a00700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e41079a0 0x7f52e419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.723+0000 7f52e2a00700 1 -- 172.21.15.115:0/2755805950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d40095f0 con 0x7f52e41067a0 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.723+0000 7f52e2a00700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 0x7f52e419bfb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f52d400f690 tx=0x7f52d400f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.723+0000 7f52e8a00700 1 -- 172.21.15.115:0/2755805950 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d401d070 con 0x7f52e41067a0 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.724+0000 7f52e8a00700 1 -- 172.21.15.115:0/2755805950 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f52d4005dc0 con 0x7f52e41067a0 2024-03-22T10:48:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.724+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e41a16a0 con 0x7f52e41067a0 2024-03-22T10:48:42.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.724+0000 7f52e8a00700 1 -- 172.21.15.115:0/2755805950 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d4017630 con 0x7f52e41067a0 2024-03-22T10:48:42.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.724+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e41a1b90 con 0x7f52e41067a0 2024-03-22T10:48:42.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.725+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e410f2f0 con 0x7f52e41067a0 2024-03-22T10:48:42.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.726+0000 7f52e8a00700 1 -- 172.21.15.115:0/2755805950 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f52d4003910 con 0x7f52e41067a0 2024-03-22T10:48:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.727+0000 7f52e8a00700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52c806dfa0 0x7f52c8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.727+0000 7f52e8a00700 1 -- 172.21.15.115:0/2755805950 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f52d408f0d0 con 0x7f52e41067a0 2024-03-22T10:48:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.727+0000 7f52e2000700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52c806dfa0 0x7f52c8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.728+0000 7f52e2000700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52c806dfa0 0x7f52c8070450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009c60 tx=0x7f52d0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:42.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.732+0000 7f52e8a00700 1 -- 172.21.15.115:0/2755805950 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52d405c150 con 0x7f52e41067a0 2024-03-22T10:48:42.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.976+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52e410c2f0 con 0x7f52c806dfa0 2024-03-22T10:48:42.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.979+0000 7f52e8a00700 1 -- 172.21.15.115:0/2755805950 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f52e410c2f0 con 0x7f52c806dfa0 2024-03-22T10:48:42.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52c806dfa0 msgr2=0x7f52c8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52c806dfa0 0x7f52c8070450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009c60 tx=0x7f52d0009260 comp rx=0 tx=0).stop 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 msgr2=0x7f52e419bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 0x7f52e419bfb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f52d400f690 tx=0x7f52d400f6c0 comp rx=0 tx=0).stop 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 shutdown_connections 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52c806dfa0 0x7f52c8070450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e41067a0 0x7f52e419bfb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 --2- 172.21.15.115:0/2755805950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52e41079a0 0x7f52e419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 >> 172.21.15.115:0/2755805950 conn(0x7f52e4075910 msgr2=0x7f52e410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 shutdown_connections 2024-03-22T10:48:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:42.982+0000 7f52e9e00700 1 -- 172.21.15.115:0/2755805950 wait complete. 2024-03-22T10:48:42.989 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:48:43.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.141+0000 7ff616e00700 1 -- 172.21.15.115:0/1273065704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 msgr2=0x7ff6101025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:43.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.141+0000 7ff616e00700 1 --2- 172.21.15.115:0/1273065704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 0x7ff6101025d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff604009910 tx=0x7ff604009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:43.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.141+0000 7ff616e00700 1 -- 172.21.15.115:0/1273065704 shutdown_connections 2024-03-22T10:48:43.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.141+0000 7ff616e00700 1 --2- 172.21.15.115:0/1273065704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101033c0 0x7ff610103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.141+0000 7ff616e00700 1 --2- 172.21.15.115:0/1273065704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 0x7ff6101025d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.141+0000 7ff616e00700 1 -- 172.21.15.115:0/1273065704 >> 172.21.15.115:0/1273065704 conn(0x7ff6100fd8d0 msgr2=0x7ff6100ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:43.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.141+0000 7ff616e00700 1 -- 172.21.15.115:0/1273065704 shutdown_connections 2024-03-22T10:48:43.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.141+0000 7ff616e00700 1 -- 172.21.15.115:0/1273065704 wait complete. 2024-03-22T10:48:43.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.142+0000 7ff616e00700 1 Processor -- start 2024-03-22T10:48:43.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.142+0000 7ff616e00700 1 -- start start 2024-03-22T10:48:43.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.142+0000 7ff616e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 0x7ff6101979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:43.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.142+0000 7ff616e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101033c0 0x7ff610197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:43.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.142+0000 7ff616e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610198520 con 0x7ff6101021c0 2024-03-22T10:48:43.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.142+0000 7ff616e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610198660 con 0x7ff6101033c0 2024-03-22T10:48:43.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.142+0000 7ff615a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 0x7ff6101979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.143+0000 7ff615a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 0x7ff6101979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58430/0 (socket says 172.21.15.115:58430) 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.143+0000 7ff615a00700 1 -- 172.21.15.115:0/3534583938 learned_addr learned my addr 172.21.15.115:0/3534583938 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.143+0000 7ff615a00700 1 -- 172.21.15.115:0/3534583938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101033c0 msgr2=0x7ff610197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.143+0000 7ff615a00700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101033c0 0x7ff610197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.143+0000 7ff615a00700 1 -- 172.21.15.115:0/3534583938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6040095f0 con 0x7ff6101021c0 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.143+0000 7ff615a00700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 0x7ff6101979c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff60400bd70 tx=0x7ff60400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.143+0000 7ff60e000700 1 -- 172.21.15.115:0/3534583938 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff60401d070 con 0x7ff6101021c0 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.144+0000 7ff60e000700 1 -- 172.21.15.115:0/3534583938 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff60400f460 con 0x7ff6101021c0 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.144+0000 7ff60e000700 1 -- 172.21.15.115:0/3534583938 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff604021640 con 0x7ff6101021c0 2024-03-22T10:48:43.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.144+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61019d0b0 con 0x7ff6101021c0 2024-03-22T10:48:43.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.144+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61019d550 con 0x7ff6101021c0 2024-03-22T10:48:43.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.145+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61010ad10 con 0x7ff6101021c0 2024-03-22T10:48:43.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.146+0000 7ff60e000700 1 -- 172.21.15.115:0/3534583938 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff60400fab0 con 0x7ff6101021c0 2024-03-22T10:48:43.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.147+0000 7ff60e000700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff5f806dfa0 0x7ff5f8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:43.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.147+0000 7ff60e000700 1 -- 172.21.15.115:0/3534583938 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff60408f170 con 0x7ff6101021c0 2024-03-22T10:48:43.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.147+0000 7ff615000700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff5f806dfa0 0x7ff5f8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:43.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.148+0000 7ff615000700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff5f806dfa0 0x7ff5f8070450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00a990 tx=0x7ff5fc005c80 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:43.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.152+0000 7ff60e000700 1 -- 172.21.15.115:0/3534583938 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff60405c150 con 0x7ff6101021c0 2024-03-22T10:48:43.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.398+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff610107d10 con 0x7ff5f806dfa0 2024-03-22T10:48:43.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.401+0000 7ff60e000700 1 -- 172.21.15.115:0/3534583938 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff610107d10 con 0x7ff5f806dfa0 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff5f806dfa0 msgr2=0x7ff5f8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff5f806dfa0 0x7ff5f8070450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00a990 tx=0x7ff5fc005c80 comp rx=0 tx=0).stop 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 msgr2=0x7ff6101979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 0x7ff6101979c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff60400bd70 tx=0x7ff60400be50 comp rx=0 tx=0).stop 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 shutdown_connections 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff5f806dfa0 0x7ff5f8070450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6101021c0 0x7ff6101979c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 --2- 172.21.15.115:0/3534583938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6101033c0 0x7ff610197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 >> 172.21.15.115:0/3534583938 conn(0x7ff6100fd8d0 msgr2=0x7ff6101065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 shutdown_connections 2024-03-22T10:48:43.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.404+0000 7ff616e00700 1 -- 172.21.15.115:0/3534583938 wait complete. 2024-03-22T10:48:43.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.572+0000 7fe97ea00700 1 -- 172.21.15.115:0/2286317393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9780737b0 msgr2=0x7fe978073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:43.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.572+0000 7fe97ea00700 1 --2- 172.21.15.115:0/2286317393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9780737b0 0x7fe978073c20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe964009910 tx=0x7fe964009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:43.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.572+0000 7fe97ea00700 1 -- 172.21.15.115:0/2286317393 shutdown_connections 2024-03-22T10:48:43.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.572+0000 7fe97ea00700 1 --2- 172.21.15.115:0/2286317393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9780737b0 0x7fe978073c20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.572+0000 7fe97ea00700 1 --2- 172.21.15.115:0/2286317393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe978074d80 0x7fe9780731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.572+0000 7fe97ea00700 1 -- 172.21.15.115:0/2286317393 >> 172.21.15.115:0/2286317393 conn(0x7fe9780fb6e0 msgr2=0x7fe9780fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:43.563 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:43 smithi115 ceph-mon[78672]: pgmap v165: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:43.563 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:43 smithi115 ceph-mon[78672]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:48:43.564 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:43 smithi115 ceph-mon[78672]: osd.4 [v2:172.21.15.150:6808/231697863,v1:172.21.15.150:6809/231697863] boot 2024-03-22T10:48:43.564 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:43 smithi115 ceph-mon[78672]: osdmap e72: 6 total, 6 up, 6 in 2024-03-22T10:48:43.564 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:43 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:48:43.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.572+0000 7fe97ea00700 1 -- 172.21.15.115:0/2286317393 shutdown_connections 2024-03-22T10:48:43.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.572+0000 7fe97ea00700 1 -- 172.21.15.115:0/2286317393 wait complete. 2024-03-22T10:48:43.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.573+0000 7fe97ea00700 1 Processor -- start 2024-03-22T10:48:43.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.573+0000 7fe97ea00700 1 -- start start 2024-03-22T10:48:43.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.574+0000 7fe97ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9780737b0 0x7fe97819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:43.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.574+0000 7fe97ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe978074d80 0x7fe97819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.574+0000 7fe97ea00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97819ca70 con 0x7fe978074d80 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.574+0000 7fe97ea00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97819cbb0 con 0x7fe9780737b0 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.574+0000 7fe977400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe978074d80 0x7fe97819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.574+0000 7fe977400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe978074d80 0x7fe97819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58448/0 (socket says 172.21.15.115:58448) 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.574+0000 7fe977400700 1 -- 172.21.15.115:0/4008627766 learned_addr learned my addr 172.21.15.115:0/4008627766 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.574+0000 7fe977e00700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9780737b0 0x7fe97819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.575+0000 7fe977400700 1 -- 172.21.15.115:0/4008627766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9780737b0 msgr2=0x7fe97819bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.575+0000 7fe977400700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9780737b0 0x7fe97819bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.575+0000 7fe977400700 1 -- 172.21.15.115:0/4008627766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640095f0 con 0x7fe978074d80 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.575+0000 7fe977e00700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9780737b0 0x7fe97819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:48:43.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.575+0000 7fe977400700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe978074d80 0x7fe97819c450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe964005b40 tx=0x7fe96400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:43.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.575+0000 7fe974c00700 1 -- 172.21.15.115:0/4008627766 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe96401d070 con 0x7fe978074d80 2024-03-22T10:48:43.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.575+0000 7fe974c00700 1 -- 172.21.15.115:0/4008627766 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe96400f460 con 0x7fe978074d80 2024-03-22T10:48:43.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.575+0000 7fe97ea00700 1 -- 172.21.15.115:0/4008627766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9781a1600 con 0x7fe978074d80 2024-03-22T10:48:43.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.575+0000 7fe974c00700 1 -- 172.21.15.115:0/4008627766 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe964021640 con 0x7fe978074d80 2024-03-22T10:48:43.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.576+0000 7fe97ea00700 1 -- 172.21.15.115:0/4008627766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9781a1af0 con 0x7fe978074d80 2024-03-22T10:48:43.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.577+0000 7fe96e000700 1 -- 172.21.15.115:0/4008627766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97810f290 con 0x7fe978074d80 2024-03-22T10:48:43.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.578+0000 7fe974c00700 1 -- 172.21.15.115:0/4008627766 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe96400fa90 con 0x7fe978074d80 2024-03-22T10:48:43.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.580+0000 7fe974c00700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe95c06dfa0 0x7fe95c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:43.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.580+0000 7fe974c00700 1 -- 172.21.15.115:0/4008627766 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe96408f0c0 con 0x7fe978074d80 2024-03-22T10:48:43.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.580+0000 7fe977e00700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe95c06dfa0 0x7fe95c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:43.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.581+0000 7fe977e00700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe95c06dfa0 0x7fe95c070450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe968005950 tx=0x7fe9680058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:43.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.584+0000 7fe974c00700 1 -- 172.21.15.115:0/4008627766 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe96405c0a0 con 0x7fe978074d80 2024-03-22T10:48:43.616 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:43 smithi150 ceph-mon[65513]: pgmap v165: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 63/369 objects degraded (17.073%) 2024-03-22T10:48:43.616 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:43 smithi150 ceph-mon[65513]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:48:43.616 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:43 smithi150 ceph-mon[65513]: osd.4 [v2:172.21.15.150:6808/231697863,v1:172.21.15.150:6809/231697863] boot 2024-03-22T10:48:43.616 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:43 smithi150 ceph-mon[65513]: osdmap e72: 6 total, 6 up, 6 in 2024-03-22T10:48:43.616 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:43 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T10:48:43.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.835+0000 7fe96e000700 1 -- 172.21.15.115:0/4008627766 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe97810f420 con 0x7fe95c06dfa0 2024-03-22T10:48:43.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.860+0000 7fe974c00700 1 -- 172.21.15.115:0/4008627766 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fe97810f420 con 0x7fe95c06dfa0 2024-03-22T10:48:43.851 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:48:43.851 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (15m) 71s ago 17m 14.2M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:48:43.851 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (17m) 71s ago 17m 8694k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:48:43.851 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (16m) 4s ago 16m 8392k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:48:43.851 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (3m) 71s ago 17m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (2m) 4s ago 15m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (14m) 71s ago 16m 73.7M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (10m) 71s ago 10m 127M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (10m) 71s ago 10m 16.9M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (9m) 4s ago 9m 16.7M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (10m) 4s ago 10m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (6m) 71s ago 18m 507M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (5m) 4s ago 15m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (4m) 71s ago 19m 46.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (4m) 4s ago 15m 37.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (17m) 71s ago 17m 8238k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (15m) 4s ago 15m 8284k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (2m) 71s ago 14m 84.2M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:48:43.852 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (107s) 71s ago 14m 81.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:48:43.853 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (78s) 71s ago 13m 11.1M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:48:43.853 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (50s) 4s ago 13m 87.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:48:43.853 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (9s) 4s ago 12m 11.1M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:48:43.853 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (11m) 4s ago 11m 197M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:48:43.853 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (5m) 71s ago 16m 71.2M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:48:43.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.863+0000 7fe96e000700 1 -- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe95c06dfa0 msgr2=0x7fe95c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:43.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.863+0000 7fe96e000700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe95c06dfa0 0x7fe95c070450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe968005950 tx=0x7fe9680058e0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.863+0000 7fe96e000700 1 -- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe978074d80 msgr2=0x7fe97819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:43.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.863+0000 7fe96e000700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe978074d80 0x7fe97819c450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe964005b40 tx=0x7fe96400be20 comp rx=0 tx=0).stop 2024-03-22T10:48:43.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.863+0000 7fe96e000700 1 -- 172.21.15.115:0/4008627766 shutdown_connections 2024-03-22T10:48:43.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.863+0000 7fe96e000700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe95c06dfa0 0x7fe95c070450 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.863+0000 7fe96e000700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9780737b0 0x7fe97819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.863+0000 7fe96e000700 1 --2- 172.21.15.115:0/4008627766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe978074d80 0x7fe97819c450 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:43.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.863+0000 7fe96e000700 1 -- 172.21.15.115:0/4008627766 >> 172.21.15.115:0/4008627766 conn(0x7fe9780fb6e0 msgr2=0x7fe9781019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:43.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.864+0000 7fe96e000700 1 -- 172.21.15.115:0/4008627766 shutdown_connections 2024-03-22T10:48:43.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:43.864+0000 7fe96e000700 1 -- 172.21.15.115:0/4008627766 wait complete. 2024-03-22T10:48:44.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.014+0000 7fb165800700 1 -- 172.21.15.115:0/3164789105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1600737b0 msgr2=0x7fb160073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.014+0000 7fb165800700 1 --2- 172.21.15.115:0/3164789105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1600737b0 0x7fb160073c20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009940 tx=0x7fb14c009c50 comp rx=0 tx=0).stop 2024-03-22T10:48:44.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.014+0000 7fb165800700 1 -- 172.21.15.115:0/3164789105 shutdown_connections 2024-03-22T10:48:44.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.014+0000 7fb165800700 1 --2- 172.21.15.115:0/3164789105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1600737b0 0x7fb160073c20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.014+0000 7fb165800700 1 --2- 172.21.15.115:0/3164789105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb160074d80 0x7fb1600731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.014+0000 7fb165800700 1 -- 172.21.15.115:0/3164789105 >> 172.21.15.115:0/3164789105 conn(0x7fb1600fb6e0 msgr2=0x7fb1600fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:44.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.014+0000 7fb165800700 1 -- 172.21.15.115:0/3164789105 shutdown_connections 2024-03-22T10:48:44.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.014+0000 7fb165800700 1 -- 172.21.15.115:0/3164789105 wait complete. 2024-03-22T10:48:44.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.016+0000 7fb165800700 1 Processor -- start 2024-03-22T10:48:44.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.016+0000 7fb165800700 1 -- start start 2024-03-22T10:48:44.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.016+0000 7fb165800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1600737b0 0x7fb160197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:44.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.016+0000 7fb165800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb160074d80 0x7fb1601980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:44.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.016+0000 7fb165800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1601986d0 con 0x7fb160074d80 2024-03-22T10:48:44.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.016+0000 7fb165800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb160198810 con 0x7fb1600737b0 2024-03-22T10:48:44.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.017+0000 7fb15e000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb160074d80 0x7fb1601980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:44.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.017+0000 7fb15e000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb160074d80 0x7fb1601980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58458/0 (socket says 172.21.15.115:58458) 2024-03-22T10:48:44.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.017+0000 7fb15e000700 1 -- 172.21.15.115:0/3880079089 learned_addr learned my addr 172.21.15.115:0/3880079089 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:44.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.017+0000 7fb15ea00700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1600737b0 0x7fb160197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:44.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.017+0000 7fb15e000700 1 -- 172.21.15.115:0/3880079089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1600737b0 msgr2=0x7fb160197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.017+0000 7fb15e000700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1600737b0 0x7fb160197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.017+0000 7fb15e000700 1 -- 172.21.15.115:0/3880079089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c0095f0 con 0x7fb160074d80 2024-03-22T10:48:44.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.018+0000 7fb15e000700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb160074d80 0x7fb1601980b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00b580 tx=0x7fb14c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:44.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.018+0000 7fb157400700 1 -- 172.21.15.115:0/3880079089 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb14c01d070 con 0x7fb160074d80 2024-03-22T10:48:44.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.018+0000 7fb157400700 1 -- 172.21.15.115:0/3880079089 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb14c00f460 con 0x7fb160074d80 2024-03-22T10:48:44.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.018+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16019d260 con 0x7fb160074d80 2024-03-22T10:48:44.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.018+0000 7fb157400700 1 -- 172.21.15.115:0/3880079089 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb14c021640 con 0x7fb160074d80 2024-03-22T10:48:44.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.018+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16019d7d0 con 0x7fb160074d80 2024-03-22T10:48:44.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.019+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16010aeb0 con 0x7fb160074d80 2024-03-22T10:48:44.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.020+0000 7fb157400700 1 -- 172.21.15.115:0/3880079089 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb14c00f5d0 con 0x7fb160074d80 2024-03-22T10:48:44.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.022+0000 7fb157400700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb14406dfa0 0x7fb144070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:44.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.022+0000 7fb157400700 1 -- 172.21.15.115:0/3880079089 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb14c08f1e0 con 0x7fb160074d80 2024-03-22T10:48:44.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.022+0000 7fb15ea00700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb14406dfa0 0x7fb144070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:44.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.023+0000 7fb15ea00700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb14406dfa0 0x7fb144070450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb150005950 tx=0x7fb1500058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:44.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.026+0000 7fb157400700 1 -- 172.21.15.115:0/3880079089 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb14c05c140 con 0x7fb160074d80 2024-03-22T10:48:44.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.331+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb16010b070 con 0x7fb160074d80 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.332+0000 7fb157400700 1 -- 172.21.15.115:0/3880079089 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fb14c05bcd0 con 0x7fb160074d80 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:48:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 5 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 9 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:48:44.324 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:48:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb14406dfa0 msgr2=0x7fb144070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb14406dfa0 0x7fb144070450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb150005950 tx=0x7fb1500058e0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb160074d80 msgr2=0x7fb1601980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb160074d80 0x7fb1601980b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00b580 tx=0x7fb14c0056a0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 shutdown_connections 2024-03-22T10:48:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb14406dfa0 0x7fb144070450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1600737b0 0x7fb160197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 --2- 172.21.15.115:0/3880079089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb160074d80 0x7fb1601980b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 >> 172.21.15.115:0/3880079089 conn(0x7fb1600fb6e0 msgr2=0x7fb160105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:44.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 shutdown_connections 2024-03-22T10:48:44.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.335+0000 7fb165800700 1 -- 172.21.15.115:0/3880079089 wait complete. 2024-03-22T10:48:44.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.483+0000 7f8e39c00700 1 -- 172.21.15.115:0/4218853478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34100be0 msgr2=0x7f8e34102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.483+0000 7f8e39c00700 1 --2- 172.21.15.115:0/4218853478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34100be0 0x7f8e34102fc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009910 tx=0x7f8e24009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:44.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.483+0000 7f8e39c00700 1 -- 172.21.15.115:0/4218853478 shutdown_connections 2024-03-22T10:48:44.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.483+0000 7f8e39c00700 1 --2- 172.21.15.115:0/4218853478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34103500 0x7f8e341058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.483+0000 7f8e39c00700 1 --2- 172.21.15.115:0/4218853478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34100be0 0x7f8e34102fc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.483+0000 7f8e39c00700 1 -- 172.21.15.115:0/4218853478 >> 172.21.15.115:0/4218853478 conn(0x7f8e340fa5e0 msgr2=0x7f8e340fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:44.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.483+0000 7f8e39c00700 1 -- 172.21.15.115:0/4218853478 shutdown_connections 2024-03-22T10:48:44.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.483+0000 7f8e39c00700 1 -- 172.21.15.115:0/4218853478 wait complete. 2024-03-22T10:48:44.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.485+0000 7f8e39c00700 1 Processor -- start 2024-03-22T10:48:44.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.485+0000 7f8e39c00700 1 -- start start 2024-03-22T10:48:44.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.485+0000 7f8e39c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100be0 0x7f8e34191d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.485+0000 7f8e39c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34103500 0x7f8e34192280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.485+0000 7f8e39c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3418e830 con 0x7f8e34103500 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.485+0000 7f8e39c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3418e9a0 con 0x7f8e34100be0 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.485+0000 7f8e32000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34103500 0x7f8e34192280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.485+0000 7f8e32000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34103500 0x7f8e34192280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58482/0 (socket says 172.21.15.115:58482) 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.485+0000 7f8e32000700 1 -- 172.21.15.115:0/555433626 learned_addr learned my addr 172.21.15.115:0/555433626 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.486+0000 7f8e32a00700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100be0 0x7f8e34191d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.486+0000 7f8e32000700 1 -- 172.21.15.115:0/555433626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100be0 msgr2=0x7f8e34191d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.486+0000 7f8e32000700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100be0 0x7f8e34191d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.486+0000 7f8e32000700 1 -- 172.21.15.115:0/555433626 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e240095f0 con 0x7f8e34103500 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.486+0000 7f8e32a00700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100be0 0x7f8e34191d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:48:44.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.486+0000 7f8e32000700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34103500 0x7f8e34192280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000d8d0 tx=0x7f8e2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.488+0000 7f8e2b400700 1 -- 172.21.15.115:0/555433626 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e20009940 con 0x7f8e34103500 2024-03-22T10:48:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.488+0000 7f8e2b400700 1 -- 172.21.15.115:0/555433626 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8e20010460 con 0x7f8e34103500 2024-03-22T10:48:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.488+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3418ec80 con 0x7f8e34103500 2024-03-22T10:48:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.488+0000 7f8e2b400700 1 -- 172.21.15.115:0/555433626 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e2000f5f0 con 0x7f8e34103500 2024-03-22T10:48:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.488+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3418f1d0 con 0x7f8e34103500 2024-03-22T10:48:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.489+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e340fc1c0 con 0x7f8e34103500 2024-03-22T10:48:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.490+0000 7f8e2b400700 1 -- 172.21.15.115:0/555433626 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f8e200105d0 con 0x7f8e34103500 2024-03-22T10:48:44.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.491+0000 7f8e2b400700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e1806dfa0 0x7f8e18070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:44.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.491+0000 7f8e2b400700 1 -- 172.21.15.115:0/555433626 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8e2008dbe0 con 0x7f8e34103500 2024-03-22T10:48:44.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.491+0000 7f8e32a00700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e1806dfa0 0x7f8e18070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:44.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.492+0000 7f8e32a00700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e1806dfa0 0x7f8e18070450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e2400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:44.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.496+0000 7f8e2b400700 1 -- 172.21.15.115:0/555433626 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8e2005ac60 con 0x7f8e34103500 2024-03-22T10:48:44.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:44 smithi150 ceph-mon[65513]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:44.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:44 smithi150 ceph-mon[65513]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:44.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:44 smithi150 ceph-mon[65513]: osdmap e73: 6 total, 6 up, 6 in 2024-03-22T10:48:44.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:44 smithi150 ceph-mon[65513]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:44.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:44 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3880079089' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:44.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.758+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8e3402cc30 con 0x7f8e34103500 2024-03-22T10:48:44.749 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:44 smithi115 ceph-mon[78672]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:44.750 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:44 smithi115 ceph-mon[78672]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:44.750 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:44 smithi115 ceph-mon[78672]: osdmap e73: 6 total, 6 up, 6 in 2024-03-22T10:48:44.750 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:44 smithi115 ceph-mon[78672]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:44.750 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:44 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3880079089' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:44.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.760+0000 7f8e2b400700 1 -- 172.21.15.115:0/555433626 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1907 (secure 0 0 0) 0x7f8e20016790 con 0x7f8e34103500 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:e16 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 16 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:47:17.573634+0000 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:48:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:48:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 14454 members: 24241,14454 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:48:44.753 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:48:44.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.763+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e1806dfa0 msgr2=0x7f8e18070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.763+0000 7f8e39c00700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e1806dfa0 0x7f8e18070450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e2400b540 comp rx=0 tx=0).stop 2024-03-22T10:48:44.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.763+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34103500 msgr2=0x7f8e34192280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.763+0000 7f8e39c00700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34103500 0x7f8e34192280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000d8d0 tx=0x7f8e2000dc90 comp rx=0 tx=0).stop 2024-03-22T10:48:44.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.763+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 shutdown_connections 2024-03-22T10:48:44.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.763+0000 7f8e39c00700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e1806dfa0 0x7f8e18070450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.763+0000 7f8e39c00700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100be0 0x7f8e34191d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.763+0000 7f8e39c00700 1 --2- 172.21.15.115:0/555433626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e34103500 0x7f8e34192280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.764+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 >> 172.21.15.115:0/555433626 conn(0x7f8e340fa5e0 msgr2=0x7f8e340ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:44.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.764+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 shutdown_connections 2024-03-22T10:48:44.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.764+0000 7f8e39c00700 1 -- 172.21.15.115:0/555433626 wait complete. 2024-03-22T10:48:44.756 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2024-03-22T10:48:44.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.917+0000 7f28e0c00700 1 -- 172.21.15.115:0/1373129355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1021c0 msgr2=0x7f28dc1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.917+0000 7f28e0c00700 1 --2- 172.21.15.115:0/1373129355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1021c0 0x7f28dc1025d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009910 tx=0x7f28cc009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:44.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.917+0000 7f28e0c00700 1 -- 172.21.15.115:0/1373129355 shutdown_connections 2024-03-22T10:48:44.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.917+0000 7f28e0c00700 1 --2- 172.21.15.115:0/1373129355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc1033c0 0x7f28dc103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.917+0000 7f28e0c00700 1 --2- 172.21.15.115:0/1373129355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1021c0 0x7f28dc1025d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.917+0000 7f28e0c00700 1 -- 172.21.15.115:0/1373129355 >> 172.21.15.115:0/1373129355 conn(0x7f28dc0fd8d0 msgr2=0x7f28dc0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:44.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.917+0000 7f28e0c00700 1 -- 172.21.15.115:0/1373129355 shutdown_connections 2024-03-22T10:48:44.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.917+0000 7f28e0c00700 1 -- 172.21.15.115:0/1373129355 wait complete. 2024-03-22T10:48:44.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.918+0000 7f28e0c00700 1 Processor -- start 2024-03-22T10:48:44.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.918+0000 7f28e0c00700 1 -- start start 2024-03-22T10:48:44.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.918+0000 7f28e0c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc1021c0 0x7f28dc1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:44.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.918+0000 7f28e0c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1033c0 0x7f28dc197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.918+0000 7f28e0c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc198520 con 0x7f28dc1033c0 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.918+0000 7f28e0c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc198660 con 0x7f28dc1021c0 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.919+0000 7f28daa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1033c0 0x7f28dc197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.919+0000 7f28daa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1033c0 0x7f28dc197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58502/0 (socket says 172.21.15.115:58502) 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.919+0000 7f28daa00700 1 -- 172.21.15.115:0/1600372291 learned_addr learned my addr 172.21.15.115:0/1600372291 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.919+0000 7f28db400700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc1021c0 0x7f28dc1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.919+0000 7f28daa00700 1 -- 172.21.15.115:0/1600372291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc1021c0 msgr2=0x7f28dc1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.919+0000 7f28daa00700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc1021c0 0x7f28dc1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.919+0000 7f28daa00700 1 -- 172.21.15.115:0/1600372291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0095f0 con 0x7f28dc1033c0 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.919+0000 7f28db400700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc1021c0 0x7f28dc1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.919+0000 7f28daa00700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1033c0 0x7f28dc197f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28c800d8d0 tx=0x7f28c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.920+0000 7f28d3e00700 1 -- 172.21.15.115:0/1600372291 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c8009940 con 0x7f28dc1033c0 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.920+0000 7f28d3e00700 1 -- 172.21.15.115:0/1600372291 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f28c8010460 con 0x7f28dc1033c0 2024-03-22T10:48:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.920+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc19d110 con 0x7f28dc1033c0 2024-03-22T10:48:44.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.920+0000 7f28d3e00700 1 -- 172.21.15.115:0/1600372291 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c800f5f0 con 0x7f28dc1033c0 2024-03-22T10:48:44.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.920+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc19d660 con 0x7f28dc1033c0 2024-03-22T10:48:44.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.921+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc10ad10 con 0x7f28dc1033c0 2024-03-22T10:48:44.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.922+0000 7f28d3e00700 1 -- 172.21.15.115:0/1600372291 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f28c80105d0 con 0x7f28dc1033c0 2024-03-22T10:48:44.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.923+0000 7f28d3e00700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f28c006dfa0 0x7f28c0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:44.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.923+0000 7f28d3e00700 1 -- 172.21.15.115:0/1600372291 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f28c808dd70 con 0x7f28dc1033c0 2024-03-22T10:48:44.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.923+0000 7f28db400700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f28c006dfa0 0x7f28c0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:44.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.924+0000 7f28db400700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f28c006dfa0 0x7f28c0070450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006010 tx=0x7f28cc00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:44.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:44.928+0000 7f28d3e00700 1 -- 172.21.15.115:0/1600372291 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f28c805acd0 con 0x7f28dc1033c0 2024-03-22T10:48:45.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.179+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28dc107d10 con 0x7f28c006dfa0 2024-03-22T10:48:45.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.181+0000 7f28d3e00700 1 -- 172.21.15.115:0/1600372291 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f28dc107d10 con 0x7f28c006dfa0 2024-03-22T10:48:45.172 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:48:45.172 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:48:45.172 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:48:45.172 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:48:45.172 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:48:45.172 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:48:45.172 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:48:45.173 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:48:45.173 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:48:45.173 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "11/23 daemons upgraded", 2024-03-22T10:48:45.173 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T10:48:45.173 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:48:45.173 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:48:45.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.184+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f28c006dfa0 msgr2=0x7f28c0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:45.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.184+0000 7f28e0c00700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f28c006dfa0 0x7f28c0070450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006010 tx=0x7f28cc00b540 comp rx=0 tx=0).stop 2024-03-22T10:48:45.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.184+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1033c0 msgr2=0x7f28dc197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:45.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.184+0000 7f28e0c00700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1033c0 0x7f28dc197f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28c800d8d0 tx=0x7f28c800dc90 comp rx=0 tx=0).stop 2024-03-22T10:48:45.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.185+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 shutdown_connections 2024-03-22T10:48:45.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.185+0000 7f28e0c00700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f28c006dfa0 0x7f28c0070450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:45.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.185+0000 7f28e0c00700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28dc1021c0 0x7f28dc1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:45.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.185+0000 7f28e0c00700 1 --2- 172.21.15.115:0/1600372291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28dc1033c0 0x7f28dc197f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:45.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.185+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 >> 172.21.15.115:0/1600372291 conn(0x7f28dc0fd8d0 msgr2=0x7f28dc1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:45.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.185+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 shutdown_connections 2024-03-22T10:48:45.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.185+0000 7f28e0c00700 1 -- 172.21.15.115:0/1600372291 wait complete. 2024-03-22T10:48:45.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.337+0000 7f20a2200700 1 -- 172.21.15.115:0/4237331320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0fe810 msgr2=0x7f209c100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:45.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.337+0000 7f20a2200700 1 --2- 172.21.15.115:0/4237331320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0fe810 0x7f209c100c30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2090009910 tx=0x7f2090009c20 comp rx=0 tx=0).stop 2024-03-22T10:48:45.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.337+0000 7f20a2200700 1 -- 172.21.15.115:0/4237331320 shutdown_connections 2024-03-22T10:48:45.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.337+0000 7f20a2200700 1 --2- 172.21.15.115:0/4237331320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209c101170 0x7f209c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:45.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.337+0000 7f20a2200700 1 --2- 172.21.15.115:0/4237331320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0fe810 0x7f209c100c30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:45.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.337+0000 7f20a2200700 1 -- 172.21.15.115:0/4237331320 >> 172.21.15.115:0/4237331320 conn(0x7f209c0fa440 msgr2=0x7f209c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:45.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.337+0000 7f20a2200700 1 -- 172.21.15.115:0/4237331320 shutdown_connections 2024-03-22T10:48:45.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.337+0000 7f20a2200700 1 -- 172.21.15.115:0/4237331320 wait complete. 2024-03-22T10:48:45.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.338+0000 7f20a2200700 1 Processor -- start 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.338+0000 7f20a2200700 1 -- start start 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.338+0000 7f20a2200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209c0fe810 0x7f209c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.338+0000 7f20a2200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c101170 0x7f209c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.338+0000 7f20a2200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c19c880 con 0x7f209c101170 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.338+0000 7f20a2200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c19c9f0 con 0x7f209c0fe810 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.339+0000 7f209be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c101170 0x7f209c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.339+0000 7f209be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c101170 0x7f209c19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58512/0 (socket says 172.21.15.115:58512) 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.339+0000 7f209be00700 1 -- 172.21.15.115:0/2638058197 learned_addr learned my addr 172.21.15.115:0/2638058197 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.339+0000 7f20a0e00700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209c0fe810 0x7f209c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.339+0000 7f209be00700 1 -- 172.21.15.115:0/2638058197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209c0fe810 msgr2=0x7f209c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:45.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.339+0000 7f209be00700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209c0fe810 0x7f209c19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:45.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.339+0000 7f209be00700 1 -- 172.21.15.115:0/2638058197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20900095f0 con 0x7f209c101170 2024-03-22T10:48:45.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.339+0000 7f209be00700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c101170 0x7f209c19c2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f208800c960 tx=0x7f208800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:45.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.340+0000 7f2099600700 1 -- 172.21.15.115:0/2638058197 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2088007a30 con 0x7f209c101170 2024-03-22T10:48:45.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.340+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c1a1440 con 0x7f209c101170 2024-03-22T10:48:45.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.340+0000 7f2099600700 1 -- 172.21.15.115:0/2638058197 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2088007b90 con 0x7f209c101170 2024-03-22T10:48:45.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.340+0000 7f2099600700 1 -- 172.21.15.115:0/2638058197 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2088018630 con 0x7f209c101170 2024-03-22T10:48:45.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.340+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c1a1990 con 0x7f209c101170 2024-03-22T10:48:45.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.341+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c195fb0 con 0x7f209c101170 2024-03-22T10:48:45.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.342+0000 7f2099600700 1 -- 172.21.15.115:0/2638058197 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f2088007d00 con 0x7f209c101170 2024-03-22T10:48:45.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.343+0000 7f2099600700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208406dfa0 0x7f2084070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:48:45.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.343+0000 7f2099600700 1 -- 172.21.15.115:0/2638058197 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f208808dd60 con 0x7f209c101170 2024-03-22T10:48:45.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.343+0000 7f20a0e00700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208406dfa0 0x7f2084070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:48:45.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.344+0000 7f20a0e00700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208406dfa0 0x7f2084070450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2090006010 tx=0x7f209000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:48:45.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.348+0000 7f2099600700 1 -- 172.21.15.115:0/2638058197 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2088092070 con 0x7f209c101170 2024-03-22T10:48:45.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.644+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f209c02cc30 con 0x7f209c101170 2024-03-22T10:48:45.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.645+0000 7f2099600700 1 -- 172.21.15.115:0/2638058197 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+873 (secure 0 0 0) 0x7f208805ad40 con 0x7f209c101170 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 37/369 objects degraded (10.027%), 9 pgs degraded 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 37/369 objects degraded (10.027%), 9 pgs degraded 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.11 is active+undersized+degraded, acting [3,1] 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-03-22T10:48:45.637 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208406dfa0 msgr2=0x7f2084070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208406dfa0 0x7f2084070450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2090006010 tx=0x7f209000b540 comp rx=0 tx=0).stop 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c101170 msgr2=0x7f209c19c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c101170 0x7f209c19c2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f208800c960 tx=0x7f208800cd20 comp rx=0 tx=0).stop 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 shutdown_connections 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208406dfa0 0x7f2084070450 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209c0fe810 0x7f209c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 --2- 172.21.15.115:0/2638058197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c101170 0x7f209c19c2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 >> 172.21.15.115:0/2638058197 conn(0x7f209c0fa440 msgr2=0x7f209c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 shutdown_connections 2024-03-22T10:48:45.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:48:45.649+0000 7f20a2200700 1 -- 172.21.15.115:0/2638058197 wait complete. 2024-03-22T10:48:45.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:45 smithi115 ceph-mon[78672]: pgmap v168: 65 pgs: 9 active+undersized+degraded, 12 active+undersized, 44 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 37/369 objects degraded (10.027%) 2024-03-22T10:48:45.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:45 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 37/369 objects degraded (10.027%), 9 pgs degraded (PG_DEGRADED) 2024-03-22T10:48:45.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:45 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/555433626' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:48:45.791 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:45 smithi150 ceph-mon[65513]: pgmap v168: 65 pgs: 9 active+undersized+degraded, 12 active+undersized, 44 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 37/369 objects degraded (10.027%) 2024-03-22T10:48:45.791 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:45 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 37/369 objects degraded (10.027%), 9 pgs degraded (PG_DEGRADED) 2024-03-22T10:48:45.791 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:45 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/555433626' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:48:46.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2638058197' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:46.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:46 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T10:48:46.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:48:46.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2638058197' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:48:46.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:46.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:46.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:48:46.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:48:46.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:46.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:48:46.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:46.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:46.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:48:46.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:46 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T10:48:48.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:47 smithi115 ceph-mon[78672]: pgmap v169: 65 pgs: 3 active+undersized+degraded, 4 active+undersized, 58 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7/369 objects degraded (1.897%) 2024-03-22T10:48:48.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:47 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T10:48:48.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:47 smithi115 ceph-mon[78672]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-03-22T10:48:48.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:47 smithi150 ceph-mon[65513]: pgmap v169: 65 pgs: 3 active+undersized+degraded, 4 active+undersized, 58 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7/369 objects degraded (1.897%) 2024-03-22T10:48:48.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:47 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T10:48:48.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:47 smithi150 ceph-mon[65513]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-03-22T10:48:49.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:48 smithi115 ceph-mon[78672]: pgmap v170: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:49.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:48 smithi115 ceph-mon[78672]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/369 objects degraded (1.897%), 3 pgs degraded) 2024-03-22T10:48:49.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:48 smithi150 ceph-mon[65513]: pgmap v170: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:49.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:48 smithi150 ceph-mon[65513]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/369 objects degraded (1.897%), 3 pgs degraded) 2024-03-22T10:48:50.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:50 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:50.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:50 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:48:50.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:50 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:50.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:50 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:48:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:51 smithi150 ceph-mon[65513]: pgmap v171: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:52.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:51 smithi115 ceph-mon[78672]: pgmap v171: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:54.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:53 smithi150 ceph-mon[65513]: pgmap v172: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:53 smithi115 ceph-mon[78672]: pgmap v172: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:55.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:55 smithi150 ceph-mon[65513]: pgmap v173: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:55.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:55 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:55.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:55 smithi115 ceph-mon[78672]: pgmap v173: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:55.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:55 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:48:57.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:57 smithi150 ceph-mon[65513]: pgmap v174: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:57.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:57 smithi115 ceph-mon[78672]: pgmap v174: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:59.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:48:59 smithi115 ceph-mon[78672]: pgmap v175: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:48:59.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:48:59 smithi150 ceph-mon[65513]: pgmap v175: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:01.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:01 smithi115 ceph-mon[78672]: pgmap v176: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:01.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T10:49:01.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:01 smithi150 ceph-mon[65513]: pgmap v176: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:01.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T10:49:02.662 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:02 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T10:49:02.663 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:02 smithi150 ceph-mon[65513]: Upgrade: osd.5 is safe to restart 2024-03-22T10:49:02.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:02 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T10:49:02.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:02 smithi115 ceph-mon[78672]: Upgrade: osd.5 is safe to restart 2024-03-22T10:49:03.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:03 smithi150 ceph-mon[65513]: pgmap v177: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:03.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:03 smithi115 ceph-mon[78672]: pgmap v177: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:04.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:04 smithi150 ceph-mon[65513]: Upgrade: Updating osd.5 2024-03-22T10:49:04.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:04.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T10:49:04.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:04.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:04 smithi150 ceph-mon[65513]: Deploying daemon osd.5 on smithi150 2024-03-22T10:49:04.661 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:49:04.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:04 smithi115 ceph-mon[78672]: Upgrade: Updating osd.5 2024-03-22T10:49:04.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:04.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T10:49:04.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:04.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:04 smithi115 ceph-mon[78672]: Deploying daemon osd.5 on smithi150 2024-03-22T10:49:04.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:49:05.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:05 smithi115 ceph-mon[78672]: pgmap v178: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:05.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:05 smithi150 ceph-mon[65513]: pgmap v178: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:07.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:07 smithi115 ceph-mon[78672]: pgmap v179: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:07.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:07 smithi150 ceph-mon[65513]: pgmap v179: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:08.648 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:08 smithi150 systemd[1]: Stopping Ceph osd.5 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:49:09.148 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:08 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[49908]: 2024-03-22T10:49:08.831+0000 7fc30cc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T10:49:09.149 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:08 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[49908]: 2024-03-22T10:49:08.831+0000 7fc30cc00700 -1 osd.5 73 *** Got signal Terminated *** 2024-03-22T10:49:09.149 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:08 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[49908]: 2024-03-22T10:49:08.831+0000 7fc30cc00700 -1 osd.5 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T10:49:09.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:09 smithi115 ceph-mon[78672]: pgmap v180: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:09.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:09 smithi115 ceph-mon[78672]: osd.5 marked itself down and dead 2024-03-22T10:49:09.783 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:09 smithi150 ceph-mon[65513]: pgmap v180: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:09.783 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:09 smithi150 ceph-mon[65513]: osd.5 marked itself down and dead 2024-03-22T10:49:10.479 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:10 smithi150 podman[75984]: 2024-03-22 10:49:10.17850793 +0000 UTC m=+1.467950651 container died 5a44f7b5c385a6a4a205a148031b4bb7b7818a62ca59221781155515030bcc4c (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5, GIT_CLEAN=True, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-22T10:49:10.847 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:10 smithi150 ceph-mon[65513]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:49:10.847 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:10 smithi150 ceph-mon[65513]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-22T10:49:10.847 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:10 smithi150 ceph-mon[65513]: osdmap e74: 6 total, 5 up, 6 in 2024-03-22T10:49:10.847 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:10 smithi150 podman[75984]: 2024-03-22 10:49:10.625052506 +0000 UTC m=+1.914495237 container cleanup 5a44f7b5c385a6a4a205a148031b4bb7b7818a62ca59221781155515030bcc4c (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS) 2024-03-22T10:49:10.847 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:10 smithi150 bash[75984]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5 2024-03-22T10:49:11.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:10 smithi115 ceph-mon[78672]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T10:49:11.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:10 smithi115 ceph-mon[78672]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-22T10:49:11.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:10 smithi115 ceph-mon[78672]: osdmap e74: 6 total, 5 up, 6 in 2024-03-22T10:49:11.147 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:10 smithi150 podman[76003]: 2024-03-22 10:49:10.851001428 +0000 UTC m=+0.652323100 container remove 5a44f7b5c385a6a4a205a148031b4bb7b7818a62ca59221781155515030bcc4c (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-22T10:49:11.865 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:11 smithi150 ceph-mon[65513]: pgmap v182: 65 pgs: 10 peering, 10 stale+active+clean, 45 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:11.865 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:11 smithi150 ceph-mon[65513]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-22T10:49:11.865 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:11 smithi150 ceph-mon[65513]: osdmap e75: 6 total, 5 up, 6 in 2024-03-22T10:49:11.865 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:11 smithi150 podman[76091]: 2024-03-22 10:49:11.605658813 +0000 UTC m=+0.036842291 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:49:11.866 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:11 smithi150 podman[76091]: 2024-03-22 10:49:11.727102125 +0000 UTC m=+0.158285612 container create 4fa082f189a282dc0894ca1dead9de775ffdeae19acb96b2424b925f8a8419a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T10:49:11.887 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:11 smithi115 ceph-mon[78672]: pgmap v182: 65 pgs: 10 peering, 10 stale+active+clean, 45 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:11.887 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:11 smithi115 ceph-mon[78672]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-22T10:49:11.887 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:11 smithi115 ceph-mon[78672]: osdmap e75: 6 total, 5 up, 6 in 2024-03-22T10:49:12.148 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:12 smithi150 podman[76091]: 2024-03-22 10:49:12.006211694 +0000 UTC m=+0.437395185 container init 4fa082f189a282dc0894ca1dead9de775ffdeae19acb96b2424b925f8a8419a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-22T10:49:12.148 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:12 smithi150 podman[76091]: 2024-03-22 10:49:12.017106247 +0000 UTC m=+0.448289738 container start 4fa082f189a282dc0894ca1dead9de775ffdeae19acb96b2424b925f8a8419a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-22T10:49:12.148 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:12 smithi150 podman[76091]: 2024-03-22 10:49:12.096541305 +0000 UTC m=+0.527724802 container attach 4fa082f189a282dc0894ca1dead9de775ffdeae19acb96b2424b925f8a8419a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-22T10:49:12.784 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:12 smithi150 ceph-mon[65513]: Health check failed: Degraded data redundancy: 17/369 objects degraded (4.607%), 6 pgs degraded (PG_DEGRADED) 2024-03-22T10:49:12.784 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:12 smithi150 podman[76091]: 2024-03-22 10:49:12.447933957 +0000 UTC m=+0.879117474 container died 4fa082f189a282dc0894ca1dead9de775ffdeae19acb96b2424b925f8a8419a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-22T10:49:13.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:12 smithi115 ceph-mon[78672]: Health check failed: Degraded data redundancy: 17/369 objects degraded (4.607%), 6 pgs degraded (PG_DEGRADED) 2024-03-22T10:49:13.574 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:13 smithi150 podman[76116]: 2024-03-22 10:49:13.208419198 +0000 UTC m=+0.746073040 container remove 4fa082f189a282dc0894ca1dead9de775ffdeae19acb96b2424b925f8a8419a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T10:49:13.574 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:13 smithi150 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-22T10:49:13.574 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:13 smithi150 systemd[1]: Stopped Ceph osd.5 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:49:13.574 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:13 smithi150 systemd[1]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.5.service: Consumed 1min 35.436s CPU time. 2024-03-22T10:49:13.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:13 smithi150 ceph-mon[65513]: pgmap v184: 65 pgs: 6 active+undersized+degraded, 10 peering, 3 stale+active+clean, 6 active+undersized, 40 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 17/369 objects degraded (4.607%) 2024-03-22T10:49:13.898 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:13 smithi150 systemd[1]: Starting Ceph osd.5 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T10:49:14.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:13 smithi115 ceph-mon[78672]: pgmap v184: 65 pgs: 6 active+undersized+degraded, 10 peering, 3 stale+active+clean, 6 active+undersized, 40 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 17/369 objects degraded (4.607%) 2024-03-22T10:49:14.313 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:14 smithi150 podman[76215]: 2024-03-22 10:49:13.908879643 +0000 UTC m=+0.037058785 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:49:14.325 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:14 smithi150 podman[76215]: 2024-03-22 10:49:14.059484575 +0000 UTC m=+0.187663751 container create aed71527e6b6e62d577787a703a37874b839898da6356134a903d5c378e2a0d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-22T10:49:14.648 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:14 smithi150 podman[76215]: 2024-03-22 10:49:14.32248434 +0000 UTC m=+0.450663501 container init aed71527e6b6e62d577787a703a37874b839898da6356134a903d5c378e2a0d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-22T10:49:14.648 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:14 smithi150 podman[76215]: 2024-03-22 10:49:14.335494759 +0000 UTC m=+0.463673900 container start aed71527e6b6e62d577787a703a37874b839898da6356134a903d5c378e2a0d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-22T10:49:14.648 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:14 smithi150 podman[76215]: 2024-03-22 10:49:14.420459376 +0000 UTC m=+0.548638546 container attach aed71527e6b6e62d577787a703a37874b839898da6356134a903d5c378e2a0d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-22T10:49:15.581 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate[76229]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T10:49:15.581 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate[76229]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T10:49:15.581 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 bash[76215]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 bash[76215]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 bash[76215]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 bash[76215]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 bash[76215]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 bash[76215]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 bash[76215]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate[76229]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate[76229]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate[76229]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate[76229]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate[76229]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-22T10:49:15.582 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:15 smithi150 podman[76215]: 2024-03-22 10:49:15.271966143 +0000 UTC m=+1.400145323 container died aed71527e6b6e62d577787a703a37874b839898da6356134a903d5c378e2a0d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-22T10:49:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.817+0000 7fd600000700 1 -- 172.21.15.115:0/1260634297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81035a0 msgr2=0x7fd5f81039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.817+0000 7fd600000700 1 --2- 172.21.15.115:0/1260634297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81035a0 0x7fd5f81039f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009910 tx=0x7fd5e4009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.817+0000 7fd600000700 1 -- 172.21.15.115:0/1260634297 shutdown_connections 2024-03-22T10:49:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.817+0000 7fd600000700 1 --2- 172.21.15.115:0/1260634297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81035a0 0x7fd5f81039f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.817+0000 7fd600000700 1 --2- 172.21.15.115:0/1260634297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f81023a0 0x7fd5f81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.817+0000 7fd600000700 1 -- 172.21.15.115:0/1260634297 >> 172.21.15.115:0/1260634297 conn(0x7fd5f80fd990 msgr2=0x7fd5f80ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.817+0000 7fd600000700 1 -- 172.21.15.115:0/1260634297 shutdown_connections 2024-03-22T10:49:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.817+0000 7fd600000700 1 -- 172.21.15.115:0/1260634297 wait complete. 2024-03-22T10:49:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.818+0000 7fd600000700 1 Processor -- start 2024-03-22T10:49:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.818+0000 7fd600000700 1 -- start start 2024-03-22T10:49:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd600000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81023a0 0x7fd5f8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd600000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f81035a0 0x7fd5f81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd600000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8198700 con 0x7fd5f81023a0 2024-03-22T10:49:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd600000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8198840 con 0x7fd5f81035a0 2024-03-22T10:49:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd5fd600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81023a0 0x7fd5f8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd5fd600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81023a0 0x7fd5f8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38256/0 (socket says 172.21.15.115:38256) 2024-03-22T10:49:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd5fd600700 1 -- 172.21.15.115:0/1735625245 learned_addr learned my addr 172.21.15.115:0/1735625245 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd5fcc00700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f81035a0 0x7fd5f81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd5fd600700 1 -- 172.21.15.115:0/1735625245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f81035a0 msgr2=0x7fd5f81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd5fd600700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f81035a0 0x7fd5f81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd5fd600700 1 -- 172.21.15.115:0/1735625245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e40095f0 con 0x7fd5f81023a0 2024-03-22T10:49:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.819+0000 7fd5fcc00700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f81035a0 0x7fd5f81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:49:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.820+0000 7fd5fd600700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81023a0 0x7fd5f8197ba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d900 tx=0x7fd5ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:15.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.820+0000 7fd5f6000700 1 -- 172.21.15.115:0/1735625245 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5ec0041f0 con 0x7fd5f81023a0 2024-03-22T10:49:15.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.820+0000 7fd5f6000700 1 -- 172.21.15.115:0/1735625245 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd5ec004350 con 0x7fd5f81023a0 2024-03-22T10:49:15.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.820+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f819d2f0 con 0x7fd5f81023a0 2024-03-22T10:49:15.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.820+0000 7fd5f6000700 1 -- 172.21.15.115:0/1735625245 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5ec003da0 con 0x7fd5f81023a0 2024-03-22T10:49:15.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.820+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f819d840 con 0x7fd5f81023a0 2024-03-22T10:49:15.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.821+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f810aef0 con 0x7fd5f81023a0 2024-03-22T10:49:15.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.822+0000 7fd5f6000700 1 -- 172.21.15.115:0/1735625245 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fd5ec009730 con 0x7fd5f81023a0 2024-03-22T10:49:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.823+0000 7fd5f6000700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd5e006dfa0 0x7fd5e0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.823+0000 7fd5f6000700 1 -- 172.21.15.115:0/1735625245 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd5ec08ded0 con 0x7fd5f81023a0 2024-03-22T10:49:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.824+0000 7fd5fcc00700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd5e006dfa0 0x7fd5e0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.825+0000 7fd5fcc00700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd5e006dfa0 0x7fd5e0070450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006010 tx=0x7fd5e400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:15.829+0000 7fd5f6000700 1 -- 172.21.15.115:0/1735625245 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd5ec05ae30 con 0x7fd5f81023a0 2024-03-22T10:49:15.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:15 smithi150 ceph-mon[65513]: pgmap v185: 65 pgs: 8 active+undersized+degraded, 10 peering, 10 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24/369 objects degraded (6.504%) 2024-03-22T10:49:16.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:15 smithi115 ceph-mon[78672]: pgmap v185: 65 pgs: 8 active+undersized+degraded, 10 peering, 10 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24/369 objects degraded (6.504%) 2024-03-22T10:49:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.072+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5f8107ef0 con 0x7fd5e006dfa0 2024-03-22T10:49:16.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.074+0000 7fd5f6000700 1 -- 172.21.15.115:0/1735625245 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd5f8107ef0 con 0x7fd5e006dfa0 2024-03-22T10:49:16.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd5e006dfa0 msgr2=0x7fd5e0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd5e006dfa0 0x7fd5e0070450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006010 tx=0x7fd5e400b540 comp rx=0 tx=0).stop 2024-03-22T10:49:16.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81023a0 msgr2=0x7fd5f8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81023a0 0x7fd5f8197ba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d900 tx=0x7fd5ec00dcc0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 shutdown_connections 2024-03-22T10:49:16.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd5e006dfa0 0x7fd5e0070450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5f81023a0 0x7fd5f8197ba0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 --2- 172.21.15.115:0/1735625245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5f81035a0 0x7fd5f81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 >> 172.21.15.115:0/1735625245 conn(0x7fd5f80fd990 msgr2=0x7fd5f81067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:16.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.077+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 shutdown_connections 2024-03-22T10:49:16.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.078+0000 7fd600000700 1 -- 172.21.15.115:0/1735625245 wait complete. 2024-03-22T10:49:16.085 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:49:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.237+0000 7f8e47e00700 1 -- 172.21.15.115:0/1247741145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 msgr2=0x7f8e48103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.237+0000 7f8e47e00700 1 --2- 172.21.15.115:0/1247741145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 0x7f8e48103830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009910 tx=0x7f8e34009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.237+0000 7f8e47e00700 1 -- 172.21.15.115:0/1247741145 shutdown_connections 2024-03-22T10:49:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.237+0000 7f8e47e00700 1 --2- 172.21.15.115:0/1247741145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 0x7f8e48103830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.237+0000 7f8e47e00700 1 --2- 172.21.15.115:0/1247741145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e481021e0 0x7f8e481025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.237+0000 7f8e47e00700 1 -- 172.21.15.115:0/1247741145 >> 172.21.15.115:0/1247741145 conn(0x7f8e480fd8d0 msgr2=0x7f8e480ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.237+0000 7f8e47e00700 1 -- 172.21.15.115:0/1247741145 shutdown_connections 2024-03-22T10:49:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.237+0000 7f8e47e00700 1 -- 172.21.15.115:0/1247741145 wait complete. 2024-03-22T10:49:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.238+0000 7f8e47e00700 1 Processor -- start 2024-03-22T10:49:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.238+0000 7f8e47e00700 1 -- start start 2024-03-22T10:49:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e47e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e481021e0 0x7f8e481979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:16.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e47e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 0x7f8e48197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:16.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e47e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48198540 con 0x7f8e481033e0 2024-03-22T10:49:16.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e47e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48198680 con 0x7f8e481021e0 2024-03-22T10:49:16.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e46000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 0x7f8e48197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:16.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e46000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 0x7f8e48197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38272/0 (socket says 172.21.15.115:38272) 2024-03-22T10:49:16.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e46000700 1 -- 172.21.15.115:0/3327836962 learned_addr learned my addr 172.21.15.115:0/3327836962 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e46a00700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e481021e0 0x7f8e481979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e46000700 1 -- 172.21.15.115:0/3327836962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e481021e0 msgr2=0x7f8e481979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e46000700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e481021e0 0x7f8e481979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.239+0000 7f8e46000700 1 -- 172.21.15.115:0/3327836962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e340095f0 con 0x7f8e481033e0 2024-03-22T10:49:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.240+0000 7f8e46000700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 0x7f8e48197f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e34005b40 tx=0x7f8e3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.240+0000 7f8e3f400700 1 -- 172.21.15.115:0/3327836962 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3401d070 con 0x7f8e481033e0 2024-03-22T10:49:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.240+0000 7f8e3f400700 1 -- 172.21.15.115:0/3327836962 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8e34022470 con 0x7f8e481033e0 2024-03-22T10:49:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.240+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4819d0d0 con 0x7f8e481033e0 2024-03-22T10:49:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.240+0000 7f8e3f400700 1 -- 172.21.15.115:0/3327836962 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3400f690 con 0x7f8e481033e0 2024-03-22T10:49:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.240+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4819d5c0 con 0x7f8e481033e0 2024-03-22T10:49:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.241+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4810ad30 con 0x7f8e481033e0 2024-03-22T10:49:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.242+0000 7f8e3f400700 1 -- 172.21.15.115:0/3327836962 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f8e340225e0 con 0x7f8e481033e0 2024-03-22T10:49:16.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.243+0000 7f8e3f400700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e2c06dfa0 0x7f8e2c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:16.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.243+0000 7f8e3f400700 1 -- 172.21.15.115:0/3327836962 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8e3408f4f0 con 0x7f8e481033e0 2024-03-22T10:49:16.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.244+0000 7f8e46a00700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e2c06dfa0 0x7f8e2c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:16.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.244+0000 7f8e46a00700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e2c06dfa0 0x7f8e2c070450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005fd0 tx=0x7f8e38005dd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:16.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.248+0000 7f8e3f400700 1 -- 172.21.15.115:0/3327836962 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8e3405c550 con 0x7f8e481033e0 2024-03-22T10:49:16.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.490+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e48107d30 con 0x7f8e2c06dfa0 2024-03-22T10:49:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.492+0000 7f8e3f400700 1 -- 172.21.15.115:0/3327836962 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f8e48107d30 con 0x7f8e2c06dfa0 2024-03-22T10:49:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.496+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e2c06dfa0 msgr2=0x7f8e2c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.496+0000 7f8e47e00700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e2c06dfa0 0x7f8e2c070450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005fd0 tx=0x7f8e38005dd0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.496+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 msgr2=0x7f8e48197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.496+0000 7f8e47e00700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 0x7f8e48197f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e34005b40 tx=0x7f8e3400be20 comp rx=0 tx=0).stop 2024-03-22T10:49:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.497+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 shutdown_connections 2024-03-22T10:49:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.497+0000 7f8e47e00700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8e2c06dfa0 0x7f8e2c070450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.497+0000 7f8e47e00700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e481021e0 0x7f8e481979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.497+0000 7f8e47e00700 1 --2- 172.21.15.115:0/3327836962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e481033e0 0x7f8e48197f20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.497+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 >> 172.21.15.115:0/3327836962 conn(0x7f8e480fd8d0 msgr2=0x7f8e48106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.497+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 shutdown_connections 2024-03-22T10:49:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.497+0000 7f8e47e00700 1 -- 172.21.15.115:0/3327836962 wait complete. 2024-03-22T10:49:16.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:16 smithi150 podman[76365]: 2024-03-22 10:49:16.308094103 +0000 UTC m=+1.016288012 container remove aed71527e6b6e62d577787a703a37874b839898da6356134a903d5c378e2a0d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-activate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T10:49:16.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.647+0000 7f6006a00700 1 -- 172.21.15.115:0/2566365354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000ffff0 msgr2=0x7f6000100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.647+0000 7f6006a00700 1 --2- 172.21.15.115:0/2566365354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000ffff0 0x7f6000100460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009910 tx=0x7f5fec009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:16.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.647+0000 7f6006a00700 1 -- 172.21.15.115:0/2566365354 shutdown_connections 2024-03-22T10:49:16.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.647+0000 7f6006a00700 1 --2- 172.21.15.115:0/2566365354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000ffff0 0x7f6000100460 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.647+0000 7f6006a00700 1 --2- 172.21.15.115:0/2566365354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60000fee90 0x7f60000ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.647+0000 7f6006a00700 1 -- 172.21.15.115:0/2566365354 >> 172.21.15.115:0/2566365354 conn(0x7f60000fa440 msgr2=0x7f60000fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:16.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.648+0000 7f6006a00700 1 -- 172.21.15.115:0/2566365354 shutdown_connections 2024-03-22T10:49:16.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.648+0000 7f6006a00700 1 -- 172.21.15.115:0/2566365354 wait complete. 2024-03-22T10:49:16.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.649+0000 7f6006a00700 1 Processor -- start 2024-03-22T10:49:16.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.649+0000 7f6006a00700 1 -- start start 2024-03-22T10:49:16.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.649+0000 7f6006a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000fee90 0x7f6000197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:16.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.649+0000 7f6006a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60000ffff0 0x7f6000197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:16.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.649+0000 7f6006a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000198580 con 0x7f60000fee90 2024-03-22T10:49:16.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.649+0000 7f6006a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60001986c0 con 0x7f60000ffff0 2024-03-22T10:49:16.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.650+0000 7f6005600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000fee90 0x7f6000197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:16.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.650+0000 7f6005600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000fee90 0x7f6000197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38296/0 (socket says 172.21.15.115:38296) 2024-03-22T10:49:16.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.650+0000 7f6005600700 1 -- 172.21.15.115:0/1959631535 learned_addr learned my addr 172.21.15.115:0/1959631535 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:16.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.650+0000 7f6004c00700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60000ffff0 0x7f6000197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:16.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.650+0000 7f6005600700 1 -- 172.21.15.115:0/1959631535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60000ffff0 msgr2=0x7f6000197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.650+0000 7f6005600700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60000ffff0 0x7f6000197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.650+0000 7f6005600700 1 -- 172.21.15.115:0/1959631535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0095f0 con 0x7f60000fee90 2024-03-22T10:49:16.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.650+0000 7f6005600700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000fee90 0x7f6000197a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400b4b0 tx=0x7f5ff400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:16.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.651+0000 7f5ffe000700 1 -- 172.21.15.115:0/1959631535 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ff4011800 con 0x7f60000fee90 2024-03-22T10:49:16.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.651+0000 7f5ffe000700 1 -- 172.21.15.115:0/1959631535 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ff4011e10 con 0x7f60000fee90 2024-03-22T10:49:16.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.651+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600019d170 con 0x7f60000fee90 2024-03-22T10:49:16.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.651+0000 7f5ffe000700 1 -- 172.21.15.115:0/1959631535 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ff400f2e0 con 0x7f60000fee90 2024-03-22T10:49:16.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.651+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600019d740 con 0x7f60000fee90 2024-03-22T10:49:16.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.652+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60001079e0 con 0x7f60000fee90 2024-03-22T10:49:16.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.653+0000 7f5ffe000700 1 -- 172.21.15.115:0/1959631535 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5ff4010380 con 0x7f60000fee90 2024-03-22T10:49:16.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.654+0000 7f5ffe000700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5fe806dfa0 0x7f5fe8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:16.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.654+0000 7f5ffe000700 1 -- 172.21.15.115:0/1959631535 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5ff408da10 con 0x7f60000fee90 2024-03-22T10:49:16.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.655+0000 7f6004c00700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5fe806dfa0 0x7f5fe8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:16.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.655+0000 7f6004c00700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5fe806dfa0 0x7f5fe8070450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fec006010 tx=0x7f5fec00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.659+0000 7f5ffe000700 1 -- 172.21.15.115:0/1959631535 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5ff405a970 con 0x7f60000fee90 2024-03-22T10:49:16.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:16 smithi115 ceph-mon[78672]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-22T10:49:16.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.892+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6000068ae0 con 0x7f5fe806dfa0 2024-03-22T10:49:16.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.904+0000 7f5ffe000700 1 -- 172.21.15.115:0/1959631535 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6000068ae0 con 0x7f5fe806dfa0 2024-03-22T10:49:16.896 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:49:16.897 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (15m) 104s ago 17m 14.2M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:49:16.897 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (17m) 104s ago 17m 8694k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:49:16.897 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (16m) 37s ago 16m 8392k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:49:16.897 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (3m) 104s ago 17m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:49:16.897 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (3m) 37s ago 16m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:49:16.897 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (15m) 104s ago 17m 73.7M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:49:16.898 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (10m) 104s ago 10m 127M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:49:16.898 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (10m) 104s ago 10m 16.9M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:49:16.898 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (10m) 37s ago 10m 16.7M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:49:16.898 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (10m) 37s ago 10m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:49:16.898 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (7m) 104s ago 19m 507M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:49:16.898 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (5m) 37s ago 16m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:49:16.898 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (5m) 104s ago 19m 46.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:49:16.899 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (4m) 37s ago 16m 37.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:49:16.899 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (17m) 104s ago 17m 8238k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:49:16.899 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (16m) 37s ago 16m 8284k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:49:16.899 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (3m) 104s ago 15m 84.2M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:49:16.899 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (2m) 104s ago 14m 81.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:49:16.899 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (112s) 104s ago 14m 11.1M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:49:16.900 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (83s) 37s ago 13m 87.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:49:16.900 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (42s) 37s ago 12m 11.1M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:49:16.900 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (12m) 37s ago 12m 197M 2241M 18.2.0 dc2bc1663786 5a44f7b5c385 2024-03-22T10:49:16.900 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (6m) 104s ago 17m 71.2M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:49:16.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:16 smithi150 ceph-mon[65513]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-22T10:49:16.902 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:16 smithi150 podman[76429]: 2024-03-22 10:49:16.556590527 +0000 UTC m=+0.032773426 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T10:49:16.902 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:16 smithi150 podman[76429]: 2024-03-22 10:49:16.675602538 +0000 UTC m=+0.151785489 container create 956493e4218861ccdfd039b61507a8a89dee7383beae26b6b1c72ff1ef07823b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-22T10:49:16.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.907+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5fe806dfa0 msgr2=0x7f5fe8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.907+0000 7f6006a00700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5fe806dfa0 0x7f5fe8070450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fec006010 tx=0x7f5fec00b540 comp rx=0 tx=0).stop 2024-03-22T10:49:16.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.907+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000fee90 msgr2=0x7f6000197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:16.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.907+0000 7f6006a00700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000fee90 0x7f6000197a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400b4b0 tx=0x7f5ff400b870 comp rx=0 tx=0).stop 2024-03-22T10:49:16.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.907+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 shutdown_connections 2024-03-22T10:49:16.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.907+0000 7f6006a00700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5fe806dfa0 0x7f5fe8070450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.907+0000 7f6006a00700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60000fee90 0x7f6000197a20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.907+0000 7f6006a00700 1 --2- 172.21.15.115:0/1959631535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60000ffff0 0x7f6000197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:16.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.907+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 >> 172.21.15.115:0/1959631535 conn(0x7f60000fa440 msgr2=0x7f60000683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:16.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.908+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 shutdown_connections 2024-03-22T10:49:16.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:16.908+0000 7f6006a00700 1 -- 172.21.15.115:0/1959631535 wait complete. 2024-03-22T10:49:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.059+0000 7f9271800700 1 -- 172.21.15.115:0/2475747029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 msgr2=0x7f926c073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.059+0000 7f9271800700 1 --2- 172.21.15.115:0/2475747029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 0x7f926c073bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9254009910 tx=0x7f9254009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.059+0000 7f9271800700 1 -- 172.21.15.115:0/2475747029 shutdown_connections 2024-03-22T10:49:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.059+0000 7f9271800700 1 --2- 172.21.15.115:0/2475747029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 0x7f926c073bc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.059+0000 7f9271800700 1 --2- 172.21.15.115:0/2475747029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f926c074d20 0x7f926c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.059+0000 7f9271800700 1 -- 172.21.15.115:0/2475747029 >> 172.21.15.115:0/2475747029 conn(0x7f926c0fb6e0 msgr2=0x7f926c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.059+0000 7f9271800700 1 -- 172.21.15.115:0/2475747029 shutdown_connections 2024-03-22T10:49:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.059+0000 7f9271800700 1 -- 172.21.15.115:0/2475747029 wait complete. 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.060+0000 7f9271800700 1 Processor -- start 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.060+0000 7f9271800700 1 -- start start 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f9271800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 0x7f926c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f9271800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f926c074d20 0x7f926c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f9271800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c198690 con 0x7f926c073750 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f9271800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c1987d0 con 0x7f926c074d20 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f926aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 0x7f926c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f926aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 0x7f926c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38322/0 (socket says 172.21.15.115:38322) 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f926aa00700 1 -- 172.21.15.115:0/2168380022 learned_addr learned my addr 172.21.15.115:0/2168380022 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f9263e00700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f926c074d20 0x7f926c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f926aa00700 1 -- 172.21.15.115:0/2168380022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f926c074d20 msgr2=0x7f926c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.061+0000 7f926aa00700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f926c074d20 0x7f926c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.062+0000 7f926aa00700 1 -- 172.21.15.115:0/2168380022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92540095f0 con 0x7f926c073750 2024-03-22T10:49:17.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.062+0000 7f926aa00700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 0x7f926c197b30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c8a0 tx=0x7f925c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:17.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.062+0000 7f9263400700 1 -- 172.21.15.115:0/2168380022 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f925c00cea0 con 0x7f926c073750 2024-03-22T10:49:17.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.062+0000 7f9263400700 1 -- 172.21.15.115:0/2168380022 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f925c004830 con 0x7f926c073750 2024-03-22T10:49:17.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.062+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c19d280 con 0x7f926c073750 2024-03-22T10:49:17.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.062+0000 7f9263400700 1 -- 172.21.15.115:0/2168380022 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f925c0055b0 con 0x7f926c073750 2024-03-22T10:49:17.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.062+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c19d7d0 con 0x7f926c073750 2024-03-22T10:49:17.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.063+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c10aeb0 con 0x7f926c073750 2024-03-22T10:49:17.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.064+0000 7f9263400700 1 -- 172.21.15.115:0/2168380022 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f925c007ad0 con 0x7f926c073750 2024-03-22T10:49:17.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.065+0000 7f9263400700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f925006dfa0 0x7f9250070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:17.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.065+0000 7f9263400700 1 -- 172.21.15.115:0/2168380022 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f925c08da50 con 0x7f926c073750 2024-03-22T10:49:17.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.065+0000 7f9263e00700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f925006dfa0 0x7f9250070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:17.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.066+0000 7f9263e00700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f925006dfa0 0x7f9250070450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9254006010 tx=0x7f925400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.070+0000 7f9263400700 1 -- 172.21.15.115:0/2168380022 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f925c05aa30 con 0x7f926c073750 2024-03-22T10:49:17.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.376+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f926c19dec0 con 0x7f926c073750 2024-03-22T10:49:17.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.378+0000 7f9263400700 1 -- 172.21.15.115:0/2168380022 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+700 (secure 0 0 0) 0x7f925c01d2d0 con 0x7f926c073750 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 5 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:49:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:49:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:49:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:49:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-03-22T10:49:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 9 2024-03-22T10:49:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:49:17.371 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:49:17.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.381+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f925006dfa0 msgr2=0x7f9250070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.381+0000 7f9271800700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f925006dfa0 0x7f9250070450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9254006010 tx=0x7f925400b540 comp rx=0 tx=0).stop 2024-03-22T10:49:17.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.381+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 msgr2=0x7f926c197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.381+0000 7f9271800700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 0x7f926c197b30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c8a0 tx=0x7f925c00cc60 comp rx=0 tx=0).stop 2024-03-22T10:49:17.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.381+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 shutdown_connections 2024-03-22T10:49:17.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.381+0000 7f9271800700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f925006dfa0 0x7f9250070450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.381+0000 7f9271800700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f926c073750 0x7f926c197b30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.381+0000 7f9271800700 1 --2- 172.21.15.115:0/2168380022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f926c074d20 0x7f926c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.381+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 >> 172.21.15.115:0/2168380022 conn(0x7f926c0fb6e0 msgr2=0x7f926c105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:17.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.382+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 shutdown_connections 2024-03-22T10:49:17.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.382+0000 7f9271800700 1 -- 172.21.15.115:0/2168380022 wait complete. 2024-03-22T10:49:17.397 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:16 smithi150 podman[76429]: 2024-03-22 10:49:16.929607634 +0000 UTC m=+0.405790591 container init 956493e4218861ccdfd039b61507a8a89dee7383beae26b6b1c72ff1ef07823b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-22T10:49:17.398 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:16 smithi150 podman[76429]: 2024-03-22 10:49:16.947623582 +0000 UTC m=+0.423806523 container start 956493e4218861ccdfd039b61507a8a89dee7383beae26b6b1c72ff1ef07823b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-22T10:49:17.398 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:16 smithi150 ceph-osd[76448]: -- 172.21.15.150:0/631253354 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bf37e71680 con 0x55bf37e3ac00 2024-03-22T10:49:17.398 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:17 smithi150 bash[76429]: 956493e4218861ccdfd039b61507a8a89dee7383beae26b6b1c72ff1ef07823b 2024-03-22T10:49:17.398 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:17 smithi150 systemd[1]: Started Ceph osd.5 for fd1caff4-e836-11ee-95cd-87774f69a715. 2024-03-22T10:49:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.524+0000 7f9853c00700 1 -- 172.21.15.115:0/420007806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c107980 msgr2=0x7f984c107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.524+0000 7f9853c00700 1 --2- 172.21.15.115:0/420007806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c107980 0x7f984c107dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9838009910 tx=0x7f9838009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.524+0000 7f9853c00700 1 -- 172.21.15.115:0/420007806 shutdown_connections 2024-03-22T10:49:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.524+0000 7f9853c00700 1 --2- 172.21.15.115:0/420007806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c107980 0x7f984c107dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.524+0000 7f9853c00700 1 --2- 172.21.15.115:0/420007806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c106780 0x7f984c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.524+0000 7f9853c00700 1 -- 172.21.15.115:0/420007806 >> 172.21.15.115:0/420007806 conn(0x7f984c075740 msgr2=0x7f984c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.524+0000 7f9853c00700 1 -- 172.21.15.115:0/420007806 shutdown_connections 2024-03-22T10:49:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.524+0000 7f9853c00700 1 -- 172.21.15.115:0/420007806 wait complete. 2024-03-22T10:49:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.525+0000 7f9853c00700 1 Processor -- start 2024-03-22T10:49:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.525+0000 7f9853c00700 1 -- start start 2024-03-22T10:49:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9853c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c106780 0x7f984c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9853c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c107980 0x7f984c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9853c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c19cb00 con 0x7f984c106780 2024-03-22T10:49:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9853c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c19cc40 con 0x7f984c107980 2024-03-22T10:49:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9851200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c106780 0x7f984c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:17.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9851200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c106780 0x7f984c19bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38346/0 (socket says 172.21.15.115:38346) 2024-03-22T10:49:17.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9851200700 1 -- 172.21.15.115:0/1501651949 learned_addr learned my addr 172.21.15.115:0/1501651949 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:17.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f984be00700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c107980 0x7f984c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9851200700 1 -- 172.21.15.115:0/1501651949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c107980 msgr2=0x7f984c19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9851200700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c107980 0x7f984c19c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.526+0000 7f9851200700 1 -- 172.21.15.115:0/1501651949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380095f0 con 0x7f984c106780 2024-03-22T10:49:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.527+0000 7f9851200700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c106780 0x7f984c19bfa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f984000e940 tx=0x7f984000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.527+0000 7f9849600700 1 -- 172.21.15.115:0/1501651949 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9840009780 con 0x7f984c106780 2024-03-22T10:49:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.527+0000 7f9849600700 1 -- 172.21.15.115:0/1501651949 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9840004020 con 0x7f984c106780 2024-03-22T10:49:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.527+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c1a16f0 con 0x7f984c106780 2024-03-22T10:49:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.527+0000 7f9849600700 1 -- 172.21.15.115:0/1501651949 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9840003de0 con 0x7f984c106780 2024-03-22T10:49:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.527+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c1a1c40 con 0x7f984c106780 2024-03-22T10:49:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.528+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c10f2d0 con 0x7f984c106780 2024-03-22T10:49:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.529+0000 7f9849600700 1 -- 172.21.15.115:0/1501651949 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f984000cb40 con 0x7f984c106780 2024-03-22T10:49:17.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.530+0000 7f9849600700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f983406dfa0 0x7f9834070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:17.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.531+0000 7f9849600700 1 -- 172.21.15.115:0/1501651949 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9840014070 con 0x7f984c106780 2024-03-22T10:49:17.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.531+0000 7f984be00700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f983406dfa0 0x7f9834070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:17.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.531+0000 7f984be00700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f983406dfa0 0x7f9834070450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f983800b6f0 tx=0x7f9838009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:17.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.535+0000 7f9849600700 1 -- 172.21.15.115:0/1501651949 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f984005b070 con 0x7f984c106780 2024-03-22T10:49:17.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.791+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f984c1a22d0 con 0x7f984c106780 2024-03-22T10:49:17.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.792+0000 7f9849600700 1 -- 172.21.15.115:0/1501651949 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1907 (secure 0 0 0) 0x7f984005ac00 con 0x7f984c106780 2024-03-22T10:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout:e16 2024-03-22T10:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 16 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:47:17.573634+0000 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 14454 members: 24241,14454 2024-03-22T10:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.794+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f983406dfa0 msgr2=0x7f9834070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.794+0000 7f9853c00700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f983406dfa0 0x7f9834070450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f983800b6f0 tx=0x7f9838009f70 comp rx=0 tx=0).stop 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.794+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c106780 msgr2=0x7f984c19bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.794+0000 7f9853c00700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c106780 0x7f984c19bfa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f984000e940 tx=0x7f984000ed00 comp rx=0 tx=0).stop 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.795+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 shutdown_connections 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.795+0000 7f9853c00700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f983406dfa0 0x7f9834070450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.795+0000 7f9853c00700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c106780 0x7f984c19bfa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.795+0000 7f9853c00700 1 --2- 172.21.15.115:0/1501651949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c107980 0x7f984c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.795+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 >> 172.21.15.115:0/1501651949 conn(0x7f984c075740 msgr2=0x7f984c10abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.795+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 shutdown_connections 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.795+0000 7f9853c00700 1 -- 172.21.15.115:0/1501651949 wait complete. 2024-03-22T10:49:17.788 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2024-03-22T10:49:17.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:17 smithi150 ceph-mon[65513]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:17.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:17 smithi150 ceph-mon[65513]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:17.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:17 smithi150 ceph-mon[65513]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:17.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:17 smithi150 ceph-mon[65513]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:17.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:17 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:17.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:17 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:17.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:17 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:17.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:17 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2168380022' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:17.898 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:17 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T10:49:17.521+0000 7f2f008827c0 -1 Falling back to public interface 2024-03-22T10:49:17.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.941+0000 7f35e6800700 1 -- 172.21.15.115:0/1979317607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 msgr2=0x7f35e00731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.941+0000 7f35e6800700 1 --2- 172.21.15.115:0/1979317607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 0x7f35e00731e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f35d000b0d0 tx=0x7f35d000b3e0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.941+0000 7f35e6800700 1 -- 172.21.15.115:0/1979317607 shutdown_connections 2024-03-22T10:49:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.941+0000 7f35e6800700 1 --2- 172.21.15.115:0/1979317607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e00737b0 0x7f35e0073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.941+0000 7f35e6800700 1 --2- 172.21.15.115:0/1979317607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 0x7f35e00731e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.941+0000 7f35e6800700 1 -- 172.21.15.115:0/1979317607 >> 172.21.15.115:0/1979317607 conn(0x7f35e00fb6e0 msgr2=0x7f35e00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:17.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:17 smithi115 ceph-mon[78672]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:17.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:17 smithi115 ceph-mon[78672]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:17.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:17 smithi115 ceph-mon[78672]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:17.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:17 smithi115 ceph-mon[78672]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:17.933 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:17 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:17.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:17 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:17.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:17 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:17.934 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:17 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2168380022' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:17.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.941+0000 7f35e6800700 1 -- 172.21.15.115:0/1979317607 shutdown_connections 2024-03-22T10:49:17.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.941+0000 7f35e6800700 1 -- 172.21.15.115:0/1979317607 wait complete. 2024-03-22T10:49:17.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.943+0000 7f35e6800700 1 Processor -- start 2024-03-22T10:49:17.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.943+0000 7f35e6800700 1 -- start start 2024-03-22T10:49:17.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.943+0000 7f35e6800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e00737b0 0x7f35e0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:17.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.943+0000 7f35e6800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 0x7f35e01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.943+0000 7f35e6800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e01986f0 con 0x7f35e0074d80 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.943+0000 7f35e6800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e0198830 con 0x7f35e00737b0 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.944+0000 7f35d6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 0x7f35e01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.944+0000 7f35d6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 0x7f35e01980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38354/0 (socket says 172.21.15.115:38354) 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.944+0000 7f35d6a00700 1 -- 172.21.15.115:0/1352711168 learned_addr learned my addr 172.21.15.115:0/1352711168 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.944+0000 7f35df400700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e00737b0 0x7f35e0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.944+0000 7f35d6a00700 1 -- 172.21.15.115:0/1352711168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e00737b0 msgr2=0x7f35e0197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.944+0000 7f35d6a00700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e00737b0 0x7f35e0197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.944+0000 7f35d6a00700 1 -- 172.21.15.115:0/1352711168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0009d00 con 0x7f35e0074d80 2024-03-22T10:49:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.944+0000 7f35df400700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e00737b0 0x7f35e0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T10:49:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.944+0000 7f35d6a00700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 0x7f35e01980d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e920 tx=0x7f35c800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.945+0000 7f35dcc00700 1 -- 172.21.15.115:0/1352711168 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35c800c9d0 con 0x7f35e0074d80 2024-03-22T10:49:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.945+0000 7f35dcc00700 1 -- 172.21.15.115:0/1352711168 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f35c8010430 con 0x7f35e0074d80 2024-03-22T10:49:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.945+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e019d2e0 con 0x7f35e0074d80 2024-03-22T10:49:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.945+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e019d8b0 con 0x7f35e0074d80 2024-03-22T10:49:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.945+0000 7f35dcc00700 1 -- 172.21.15.115:0/1352711168 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35c8018650 con 0x7f35e0074d80 2024-03-22T10:49:17.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.946+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc005320 con 0x7f35e0074d80 2024-03-22T10:49:17.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.947+0000 7f35dcc00700 1 -- 172.21.15.115:0/1352711168 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f35c800cb30 con 0x7f35e0074d80 2024-03-22T10:49:17.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.949+0000 7f35dcc00700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f35c406dfa0 0x7f35c4070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:17.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.949+0000 7f35dcc00700 1 -- 172.21.15.115:0/1352711168 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f35c8014070 con 0x7f35e0074d80 2024-03-22T10:49:17.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.949+0000 7f35df400700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f35c406dfa0 0x7f35c4070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:17.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.950+0000 7f35df400700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f35c406dfa0 0x7f35c4070450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f35d0000f80 tx=0x7f35d000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:17.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:17.953+0000 7f35dcc00700 1 -- 172.21.15.115:0/1352711168 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f35c8053740 con 0x7f35e0074d80 2024-03-22T10:49:18.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.179+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35bc000bf0 con 0x7f35c406dfa0 2024-03-22T10:49:18.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.181+0000 7f35dcc00700 1 -- 172.21.15.115:0/1352711168 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f35bc000bf0 con 0x7f35c406dfa0 2024-03-22T10:49:18.172 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:49:18.172 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "11/23 daemons upgraded", 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:49:18.173 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:49:18.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.183+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f35c406dfa0 msgr2=0x7f35c4070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:18.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.183+0000 7f35e6800700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f35c406dfa0 0x7f35c4070450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f35d0000f80 tx=0x7f35d000b040 comp rx=0 tx=0).stop 2024-03-22T10:49:18.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.183+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 msgr2=0x7f35e01980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:18.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.183+0000 7f35e6800700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 0x7f35e01980d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e920 tx=0x7f35c800ec30 comp rx=0 tx=0).stop 2024-03-22T10:49:18.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.184+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 shutdown_connections 2024-03-22T10:49:18.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.184+0000 7f35e6800700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f35c406dfa0 0x7f35c4070450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.184+0000 7f35e6800700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e00737b0 0x7f35e0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.184+0000 7f35e6800700 1 --2- 172.21.15.115:0/1352711168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35e0074d80 0x7f35e01980d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.184+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 >> 172.21.15.115:0/1352711168 conn(0x7f35e00fb6e0 msgr2=0x7f35e0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.184+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 shutdown_connections 2024-03-22T10:49:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.184+0000 7f35e6800700 1 -- 172.21.15.115:0/1352711168 wait complete. 2024-03-22T10:49:18.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.326+0000 7fad3dc00700 1 -- 172.21.15.115:0/862408385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 msgr2=0x7fad381027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.326+0000 7fad3dc00700 1 --2- 172.21.15.115:0/862408385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 0x7fad381027b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fad28009910 tx=0x7fad28009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.326+0000 7fad3dc00700 1 -- 172.21.15.115:0/862408385 shutdown_connections 2024-03-22T10:49:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.326+0000 7fad3dc00700 1 --2- 172.21.15.115:0/862408385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad381035a0 0x7fad381039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.326+0000 7fad3dc00700 1 --2- 172.21.15.115:0/862408385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 0x7fad381027b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.326+0000 7fad3dc00700 1 -- 172.21.15.115:0/862408385 >> 172.21.15.115:0/862408385 conn(0x7fad380fd990 msgr2=0x7fad380ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.326+0000 7fad3dc00700 1 -- 172.21.15.115:0/862408385 shutdown_connections 2024-03-22T10:49:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.326+0000 7fad3dc00700 1 -- 172.21.15.115:0/862408385 wait complete. 2024-03-22T10:49:18.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.327+0000 7fad3dc00700 1 Processor -- start 2024-03-22T10:49:18.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.327+0000 7fad3dc00700 1 -- start start 2024-03-22T10:49:18.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.327+0000 7fad3dc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 0x7fad38197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:18.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.327+0000 7fad3dc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad381035a0 0x7fad381980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:18.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.327+0000 7fad3dc00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad38198710 con 0x7fad381023a0 2024-03-22T10:49:18.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.327+0000 7fad3dc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad38198850 con 0x7fad381035a0 2024-03-22T10:49:18.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.328+0000 7fad36a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 0x7fad38197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:18.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.328+0000 7fad36a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 0x7fad38197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38364/0 (socket says 172.21.15.115:38364) 2024-03-22T10:49:18.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.328+0000 7fad36a00700 1 -- 172.21.15.115:0/2743503635 learned_addr learned my addr 172.21.15.115:0/2743503635 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:18.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.328+0000 7fad36a00700 1 -- 172.21.15.115:0/2743503635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad381035a0 msgr2=0x7fad381980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:18.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.328+0000 7fad2fe00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad381035a0 0x7fad381980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:18.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.328+0000 7fad36a00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad381035a0 0x7fad381980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.328+0000 7fad36a00700 1 -- 172.21.15.115:0/2743503635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280095f0 con 0x7fad381023a0 2024-03-22T10:49:18.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.328+0000 7fad36a00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 0x7fad38197bb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad2800bda0 tx=0x7fad2800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:18.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.329+0000 7fad2f400700 1 -- 172.21.15.115:0/2743503635 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad2801d070 con 0x7fad381023a0 2024-03-22T10:49:18.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.329+0000 7fad2f400700 1 -- 172.21.15.115:0/2743503635 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad2800f460 con 0x7fad381023a0 2024-03-22T10:49:18.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.329+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3819d2a0 con 0x7fad381023a0 2024-03-22T10:49:18.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.329+0000 7fad2f400700 1 -- 172.21.15.115:0/2743503635 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad28021620 con 0x7fad381023a0 2024-03-22T10:49:18.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.329+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3819d790 con 0x7fad381023a0 2024-03-22T10:49:18.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.330+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3810aef0 con 0x7fad381023a0 2024-03-22T10:49:18.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.331+0000 7fad2f400700 1 -- 172.21.15.115:0/2743503635 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fad2800f5d0 con 0x7fad381023a0 2024-03-22T10:49:18.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.332+0000 7fad2f400700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fad1806dfa0 0x7fad18070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:18.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.332+0000 7fad2f400700 1 -- 172.21.15.115:0/2743503635 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fad2808f060 con 0x7fad381023a0 2024-03-22T10:49:18.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.332+0000 7fad2fe00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fad1806dfa0 0x7fad18070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:18.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.333+0000 7fad2fe00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fad1806dfa0 0x7fad18070450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fad20005950 tx=0x7fad200058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:18.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.336+0000 7fad2f400700 1 -- 172.21.15.115:0/2743503635 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fad2805bfc0 con 0x7fad381023a0 2024-03-22T10:49:18.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.645+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fad3810b0b0 con 0x7fad381023a0 2024-03-22T10:49:18.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.646+0000 7fad2f400700 1 -- 172.21.15.115:0/2743503635 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1480 (secure 0 0 0) 0x7fad2805bb50 con 0x7fad381023a0 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 56/369 objects degraded (15.176%), 14 pgs degraded 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout: osd.5 (root=default,host=smithi150) is down 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 56/369 objects degraded (15.176%), 14 pgs degraded 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-03-22T10:49:18.638 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-03-22T10:49:18.639 INFO:teuthology.orchestra.run.smithi115.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-03-22T10:49:18.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fad1806dfa0 msgr2=0x7fad18070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fad1806dfa0 0x7fad18070450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fad20005950 tx=0x7fad200058e0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 msgr2=0x7fad38197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 0x7fad38197bb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad2800bda0 tx=0x7fad2800be80 comp rx=0 tx=0).stop 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 shutdown_connections 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fad1806dfa0 0x7fad18070450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad381023a0 0x7fad38197bb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 --2- 172.21.15.115:0/2743503635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad381035a0 0x7fad381980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 >> 172.21.15.115:0/2743503635 conn(0x7fad380fd990 msgr2=0x7fad381067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 shutdown_connections 2024-03-22T10:49:18.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:18.649+0000 7fad3dc00700 1 -- 172.21.15.115:0/2743503635 wait complete. 2024-03-22T10:49:18.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:18 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1501651949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:49:19.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:18 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1501651949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:49:20.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:19 smithi115 ceph-mon[78672]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:20.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:19 smithi115 ceph-mon[78672]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:20.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:19 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2743503635' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:49:20.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:20.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:49:20.053 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:19 smithi150 ceph-mon[65513]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:20.053 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:19 smithi150 ceph-mon[65513]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:20.053 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:19 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2743503635' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:49:20.053 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:20.053 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:49:21.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:20 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 56/369 objects degraded (15.176%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T10:49:21.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:20 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 56/369 objects degraded (15.176%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T10:49:22.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:21 smithi115 ceph-mon[78672]: pgmap v188: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:22.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:22.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:22.090 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:21 smithi150 ceph-mon[65513]: pgmap v188: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:22.091 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:22.091 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:22.398 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:22 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T10:49:22.099+0000 7f2f008827c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-22T10:49:22.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:22 smithi150 ceph-mon[65513]: from='osd.5 [v2:172.21.15.150:6816/2534589159,v1:172.21.15.150:6817/2534589159]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T10:49:22.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:22 smithi150 ceph-mon[65513]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T10:49:22.898 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:22 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T10:49:22.482+0000 7f2f008827c0 -1 osd.5 73 log_to_monitors true 2024-03-22T10:49:23.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:22 smithi115 ceph-mon[78672]: from='osd.5 [v2:172.21.15.150:6816/2534589159,v1:172.21.15.150:6817/2534589159]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T10:49:23.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:22 smithi115 ceph-mon[78672]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T10:49:24.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:23 smithi115 ceph-mon[78672]: pgmap v189: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:24.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:23 smithi115 ceph-mon[78672]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T10:49:24.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:23 smithi115 ceph-mon[78672]: osdmap e76: 6 total, 5 up, 6 in 2024-03-22T10:49:24.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:23 smithi115 ceph-mon[78672]: from='osd.5 [v2:172.21.15.150:6816/2534589159,v1:172.21.15.150:6817/2534589159]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:49:24.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:23 smithi115 ceph-mon[78672]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:49:24.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:24.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:23 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:24.148 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 10:49:23 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T10:49:23.799+0000 7f2eeac00700 -1 osd.5 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T10:49:24.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:23 smithi150 ceph-mon[65513]: pgmap v189: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:24.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:23 smithi150 ceph-mon[65513]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T10:49:24.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:23 smithi150 ceph-mon[65513]: osdmap e76: 6 total, 5 up, 6 in 2024-03-22T10:49:24.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:23 smithi150 ceph-mon[65513]: from='osd.5 [v2:172.21.15.150:6816/2534589159,v1:172.21.15.150:6817/2534589159]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:49:24.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:23 smithi150 ceph-mon[65513]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-22T10:49:24.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:23 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:24.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:23 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:25.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:24 smithi150 ceph-mon[65513]: from='osd.5 ' entity='osd.5' 2024-03-22T10:49:25.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:24 smithi115 ceph-mon[78672]: from='osd.5 ' entity='osd.5' 2024-03-22T10:49:26.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:25 smithi150 ceph-mon[65513]: pgmap v191: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:26.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:25 smithi150 ceph-mon[65513]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:49:26.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:25 smithi150 ceph-mon[65513]: osd.5 [v2:172.21.15.150:6816/2534589159,v1:172.21.15.150:6817/2534589159] boot 2024-03-22T10:49:26.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:25 smithi150 ceph-mon[65513]: osdmap e77: 6 total, 6 up, 6 in 2024-03-22T10:49:26.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:49:26.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:25 smithi115 ceph-mon[78672]: pgmap v191: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-03-22T10:49:26.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:25 smithi115 ceph-mon[78672]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T10:49:26.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:25 smithi115 ceph-mon[78672]: osd.5 [v2:172.21.15.150:6816/2534589159,v1:172.21.15.150:6817/2534589159] boot 2024-03-22T10:49:26.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:25 smithi115 ceph-mon[78672]: osdmap e77: 6 total, 6 up, 6 in 2024-03-22T10:49:26.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T10:49:27.133 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:26 smithi150 ceph-mon[65513]: osdmap e78: 6 total, 6 up, 6 in 2024-03-22T10:49:27.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:26 smithi115 ceph-mon[78672]: osdmap e78: 6 total, 6 up, 6 in 2024-03-22T10:49:28.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:27 smithi150 ceph-mon[65513]: pgmap v194: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28/369 objects degraded (7.588%) 2024-03-22T10:49:28.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:27 smithi150 ceph-mon[65513]: Health check update: Degraded data redundancy: 28/369 objects degraded (7.588%), 9 pgs degraded (PG_DEGRADED) 2024-03-22T10:49:28.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:27 smithi115 ceph-mon[78672]: pgmap v194: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28/369 objects degraded (7.588%) 2024-03-22T10:49:28.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:27 smithi115 ceph-mon[78672]: Health check update: Degraded data redundancy: 28/369 objects degraded (7.588%), 9 pgs degraded (PG_DEGRADED) 2024-03-22T10:49:29.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:29.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:29.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: pgmap v195: 65 pgs: 7 peering, 1 active+undersized+degraded, 2 active+undersized, 55 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4/369 objects degraded (1.084%) 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all osd 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-22T10:49:29.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: Upgrade: Setting require_osd_release to 19 squid 2024-03-22T10:49:29.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: pgmap v195: 65 pgs: 7 peering, 1 active+undersized+degraded, 2 active+undersized, 55 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4/369 objects degraded (1.084%) 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all osd 2024-03-22T10:49:29.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: Upgrade: Setting require_osd_release to 19 squid 2024-03-22T10:49:29.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-22T10:49:30.340 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-22T10:49:30.340 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-22T10:49:30.340 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: osdmap e79: 6 total, 6 up, 6 in 2024-03-22T10:49:30.340 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: Upgrade: Scaling down filesystem cephfs 2024-03-22T10:49:30.340 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:30.340 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-22T10:49:30.340 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: stopping daemon mds.cephfs.smithi150.zvyrzc 2024-03-22T10:49:30.340 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-22T10:49:30.340 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:stopping} 2 up:standby 2024-03-22T10:49:30.341 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:30.341 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:30.341 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:49:30.341 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:30.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: osdmap e79: 6 total, 6 up, 6 in 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: Upgrade: Scaling down filesystem cephfs 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: stopping daemon mds.cephfs.smithi150.zvyrzc 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:stopping} 2 up:standby 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:49:30.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:31.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:31.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:31.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:31.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:31.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:31.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:31 smithi150 ceph-mon[65513]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-03-22T10:49:31.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:31 smithi150 ceph-mon[65513]: pgmap v197: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:31.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:31.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:31.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:31.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:31.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:31.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:31 smithi115 ceph-mon[78672]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-03-22T10:49:31.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:31 smithi115 ceph-mon[78672]: pgmap v197: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:32.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:32 smithi115 ceph-mon[78672]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/369 objects degraded (1.084%), 1 pg degraded) 2024-03-22T10:49:32.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:32 smithi150 ceph-mon[65513]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/369 objects degraded (1.084%), 1 pg degraded) 2024-03-22T10:49:33.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:33 smithi150 ceph-mon[65513]: pgmap v198: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:33.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:33 smithi115 ceph-mon[78672]: pgmap v198: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T10:49:35.509 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:35 smithi150 ceph-mon[65513]: pgmap v199: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s wr, 2 op/s 2024-03-22T10:49:35.509 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:35.509 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:49:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:35 smithi115 ceph-mon[78672]: pgmap v199: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s wr, 2 op/s 2024-03-22T10:49:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:49:37.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:37 smithi115 ceph-mon[78672]: pgmap v200: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s wr, 2 op/s 2024-03-22T10:49:37.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:37 smithi150 ceph-mon[65513]: pgmap v200: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s wr, 2 op/s 2024-03-22T10:49:39.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:39 smithi150 ceph-mon[65513]: pgmap v201: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 8.9 KiB/s wr, 3 op/s 2024-03-22T10:49:39.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:39 smithi150 ceph-mon[65513]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:stopping} 2 up:standby 2024-03-22T10:49:39.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:39 smithi115 ceph-mon[78672]: pgmap v201: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 8.9 KiB/s wr, 3 op/s 2024-03-22T10:49:39.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:39 smithi115 ceph-mon[78672]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi150.zvyrzc=up:stopping} 2 up:standby 2024-03-22T10:49:40.677 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:40.677 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:40.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:40.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:42.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:41 smithi150 ceph-mon[65513]: pgmap v202: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s wr, 4 op/s 2024-03-22T10:49:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:49:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:42.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:41 smithi115 ceph-mon[78672]: pgmap v202: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s wr, 4 op/s 2024-03-22T10:49:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:49:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:42.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:42.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:43.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:42 smithi150 ceph-mon[65513]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-03-22T10:49:43.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:42 smithi115 ceph-mon[78672]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-03-22T10:49:44.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:43 smithi150 ceph-mon[65513]: pgmap v203: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 12 KiB/s wr, 6 op/s 2024-03-22T10:49:44.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:43 smithi115 ceph-mon[78672]: pgmap v203: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 12 KiB/s wr, 6 op/s 2024-03-22T10:49:46.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:45 smithi115 ceph-mon[78672]: pgmap v204: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 13 KiB/s wr, 6 op/s 2024-03-22T10:49:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:45 smithi150 ceph-mon[65513]: pgmap v204: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 13 KiB/s wr, 6 op/s 2024-03-22T10:49:48.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:47 smithi150 ceph-mon[65513]: pgmap v205: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 7.8 KiB/s wr, 4 op/s 2024-03-22T10:49:48.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:47 smithi115 ceph-mon[78672]: pgmap v205: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 7.8 KiB/s wr, 4 op/s 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.819+0000 7ff390a00700 1 -- 172.21.15.115:0/1509445811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1079a0 msgr2=0x7ff38c107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.819+0000 7ff390a00700 1 --2- 172.21.15.115:0/1509445811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1079a0 0x7ff38c107df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff378009910 tx=0x7ff378009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.819+0000 7ff390a00700 1 -- 172.21.15.115:0/1509445811 shutdown_connections 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.819+0000 7ff390a00700 1 --2- 172.21.15.115:0/1509445811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1079a0 0x7ff38c107df0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.819+0000 7ff390a00700 1 --2- 172.21.15.115:0/1509445811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff38c1067a0 0x7ff38c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.819+0000 7ff390a00700 1 -- 172.21.15.115:0/1509445811 >> 172.21.15.115:0/1509445811 conn(0x7ff38c075910 msgr2=0x7ff38c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.819+0000 7ff390a00700 1 -- 172.21.15.115:0/1509445811 shutdown_connections 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.819+0000 7ff390a00700 1 -- 172.21.15.115:0/1509445811 wait complete. 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.820+0000 7ff390a00700 1 Processor -- start 2024-03-22T10:49:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.820+0000 7ff390a00700 1 -- start start 2024-03-22T10:49:48.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.820+0000 7ff390a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1067a0 0x7ff38c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:48.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.820+0000 7ff390a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff38c1079a0 0x7ff38c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:48.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.820+0000 7ff390a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c19cb00 con 0x7ff38c1067a0 2024-03-22T10:49:48.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.820+0000 7ff390a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c19cc40 con 0x7ff38c1079a0 2024-03-22T10:49:48.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.821+0000 7ff389e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1067a0 0x7ff38c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:48.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.821+0000 7ff389e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1067a0 0x7ff38c19bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44872/0 (socket says 172.21.15.115:44872) 2024-03-22T10:49:48.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.821+0000 7ff389e00700 1 -- 172.21.15.115:0/3054084546 learned_addr learned my addr 172.21.15.115:0/3054084546 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:48.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.821+0000 7ff389400700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff38c1079a0 0x7ff38c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:48.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.821+0000 7ff389e00700 1 -- 172.21.15.115:0/3054084546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff38c1079a0 msgr2=0x7ff38c19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:48.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.821+0000 7ff389e00700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff38c1079a0 0x7ff38c19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:48.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.821+0000 7ff389e00700 1 -- 172.21.15.115:0/3054084546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3780095f0 con 0x7ff38c1067a0 2024-03-22T10:49:48.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.822+0000 7ff389e00700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1067a0 0x7ff38c19bfa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00c8f0 tx=0x7ff37c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:48.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.822+0000 7ff382a00700 1 -- 172.21.15.115:0/3054084546 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37c007a70 con 0x7ff38c1067a0 2024-03-22T10:49:48.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.822+0000 7ff382a00700 1 -- 172.21.15.115:0/3054084546 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff37c007bd0 con 0x7ff38c1067a0 2024-03-22T10:49:48.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.822+0000 7ff382a00700 1 -- 172.21.15.115:0/3054084546 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37c018630 con 0x7ff38c1067a0 2024-03-22T10:49:48.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.822+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c1a16f0 con 0x7ff38c1067a0 2024-03-22T10:49:48.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.822+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c1a1bf0 con 0x7ff38c1067a0 2024-03-22T10:49:48.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.823+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c10f2f0 con 0x7ff38c1067a0 2024-03-22T10:49:48.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.824+0000 7ff382a00700 1 -- 172.21.15.115:0/3054084546 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff37c00f450 con 0x7ff38c1067a0 2024-03-22T10:49:48.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.825+0000 7ff382a00700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37006dfa0 0x7ff370070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:48.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.825+0000 7ff382a00700 1 -- 172.21.15.115:0/3054084546 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff37c08dda0 con 0x7ff38c1067a0 2024-03-22T10:49:48.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.825+0000 7ff389400700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37006dfa0 0x7ff370070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:48.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.826+0000 7ff389400700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37006dfa0 0x7ff370070450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff378006010 tx=0x7ff37800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:48.830+0000 7ff382a00700 1 -- 172.21.15.115:0/3054084546 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff37c05adb0 con 0x7ff38c1067a0 2024-03-22T10:49:49.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.077+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff38c10c2f0 con 0x7ff37006dfa0 2024-03-22T10:49:49.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.079+0000 7ff382a00700 1 -- 172.21.15.115:0/3054084546 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff38c10c2f0 con 0x7ff37006dfa0 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37006dfa0 msgr2=0x7ff370070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37006dfa0 0x7ff370070450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff378006010 tx=0x7ff37800b540 comp rx=0 tx=0).stop 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1067a0 msgr2=0x7ff38c19bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1067a0 0x7ff38c19bfa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00c8f0 tx=0x7ff37c00ccb0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 shutdown_connections 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff37006dfa0 0x7ff370070450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c1067a0 0x7ff38c19bfa0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 --2- 172.21.15.115:0/3054084546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff38c1079a0 0x7ff38c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 >> 172.21.15.115:0/3054084546 conn(0x7ff38c075910 msgr2=0x7ff38c10abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 shutdown_connections 2024-03-22T10:49:49.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.083+0000 7ff390a00700 1 -- 172.21.15.115:0/3054084546 wait complete. 2024-03-22T10:49:49.088 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:49:49.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.239+0000 7f2320a00700 1 -- 172.21.15.115:0/555690712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 msgr2=0x7f231c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.239+0000 7f2320a00700 1 --2- 172.21.15.115:0/555690712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 0x7f231c073180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f230c009910 tx=0x7f230c009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:49.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.239+0000 7f2320a00700 1 -- 172.21.15.115:0/555690712 shutdown_connections 2024-03-22T10:49:49.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.239+0000 7f2320a00700 1 --2- 172.21.15.115:0/555690712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f231c073750 0x7f231c073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.239+0000 7f2320a00700 1 --2- 172.21.15.115:0/555690712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 0x7f231c073180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.239+0000 7f2320a00700 1 -- 172.21.15.115:0/555690712 >> 172.21.15.115:0/555690712 conn(0x7f231c0fb6e0 msgr2=0x7f231c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:49.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.239+0000 7f2320a00700 1 -- 172.21.15.115:0/555690712 shutdown_connections 2024-03-22T10:49:49.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.239+0000 7f2320a00700 1 -- 172.21.15.115:0/555690712 wait complete. 2024-03-22T10:49:49.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.240+0000 7f2320a00700 1 Processor -- start 2024-03-22T10:49:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.240+0000 7f2320a00700 1 -- start start 2024-03-22T10:49:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.241+0000 7f2320a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f231c073750 0x7f231c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.241+0000 7f2320a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 0x7f231c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.241+0000 7f2320a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1986c0 con 0x7f231c074d20 2024-03-22T10:49:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.241+0000 7f2320a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c198800 con 0x7f231c073750 2024-03-22T10:49:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.241+0000 7f2319400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 0x7f231c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.241+0000 7f2319400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 0x7f231c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44886/0 (socket says 172.21.15.115:44886) 2024-03-22T10:49:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.241+0000 7f2319400700 1 -- 172.21.15.115:0/4198489701 learned_addr learned my addr 172.21.15.115:0/4198489701 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.241+0000 7f2319e00700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f231c073750 0x7f231c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.242+0000 7f2319400700 1 -- 172.21.15.115:0/4198489701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f231c073750 msgr2=0x7f231c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.242+0000 7f2319400700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f231c073750 0x7f231c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.242+0000 7f2319400700 1 -- 172.21.15.115:0/4198489701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c0095f0 con 0x7f231c074d20 2024-03-22T10:49:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.242+0000 7f2319400700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 0x7f231c1980a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f230800c8a0 tx=0x7f230800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.242+0000 7f2312a00700 1 -- 172.21.15.115:0/4198489701 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f230800cea0 con 0x7f231c074d20 2024-03-22T10:49:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.242+0000 7f2312a00700 1 -- 172.21.15.115:0/4198489701 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2308004830 con 0x7f231c074d20 2024-03-22T10:49:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.242+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c19d2b0 con 0x7f231c074d20 2024-03-22T10:49:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.243+0000 7f2312a00700 1 -- 172.21.15.115:0/4198489701 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23080055b0 con 0x7f231c074d20 2024-03-22T10:49:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.243+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231c19d800 con 0x7f231c074d20 2024-03-22T10:49:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.244+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231c10aeb0 con 0x7f231c074d20 2024-03-22T10:49:49.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.245+0000 7f2312a00700 1 -- 172.21.15.115:0/4198489701 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f2308007ad0 con 0x7f231c074d20 2024-03-22T10:49:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.246+0000 7f2312a00700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f230006dfa0 0x7f2300070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.246+0000 7f2312a00700 1 -- 172.21.15.115:0/4198489701 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f230808da50 con 0x7f231c074d20 2024-03-22T10:49:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.246+0000 7f2319e00700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f230006dfa0 0x7f2300070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:49.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.247+0000 7f2319e00700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f230006dfa0 0x7f2300070450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f230c006010 tx=0x7f230c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:49.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.250+0000 7f2312a00700 1 -- 172.21.15.115:0/4198489701 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f230805aa30 con 0x7f231c074d20 2024-03-22T10:49:49.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.481+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f231c1076d0 con 0x7f230006dfa0 2024-03-22T10:49:49.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.483+0000 7f2312a00700 1 -- 172.21.15.115:0/4198489701 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f231c1076d0 con 0x7f230006dfa0 2024-03-22T10:49:49.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f230006dfa0 msgr2=0x7f2300070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f230006dfa0 0x7f2300070450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f230c006010 tx=0x7f230c00b540 comp rx=0 tx=0).stop 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 msgr2=0x7f231c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 0x7f231c1980a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f230800c8a0 tx=0x7f230800cc60 comp rx=0 tx=0).stop 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 shutdown_connections 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f230006dfa0 0x7f2300070450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f231c073750 0x7f231c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 --2- 172.21.15.115:0/4198489701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f231c074d20 0x7f231c1980a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 >> 172.21.15.115:0/4198489701 conn(0x7f231c0fb6e0 msgr2=0x7f231c105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 shutdown_connections 2024-03-22T10:49:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.486+0000 7f2320a00700 1 -- 172.21.15.115:0/4198489701 wait complete. 2024-03-22T10:49:49.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.641+0000 7f9051600700 1 -- 172.21.15.115:0/817628388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1079a0 msgr2=0x7f904c107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.641+0000 7f9051600700 1 --2- 172.21.15.115:0/817628388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1079a0 0x7f904c107df0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9034009910 tx=0x7f9034009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:49.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.641+0000 7f9051600700 1 -- 172.21.15.115:0/817628388 shutdown_connections 2024-03-22T10:49:49.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.641+0000 7f9051600700 1 --2- 172.21.15.115:0/817628388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1079a0 0x7f904c107df0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.641+0000 7f9051600700 1 --2- 172.21.15.115:0/817628388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f904c1067a0 0x7f904c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.641+0000 7f9051600700 1 -- 172.21.15.115:0/817628388 >> 172.21.15.115:0/817628388 conn(0x7f904c075910 msgr2=0x7f904c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:49.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.641+0000 7f9051600700 1 -- 172.21.15.115:0/817628388 shutdown_connections 2024-03-22T10:49:49.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.641+0000 7f9051600700 1 -- 172.21.15.115:0/817628388 wait complete. 2024-03-22T10:49:49.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.642+0000 7f9051600700 1 Processor -- start 2024-03-22T10:49:49.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.642+0000 7f9051600700 1 -- start start 2024-03-22T10:49:49.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f9051600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1067a0 0x7f904c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:49.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f9051600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f904c1079a0 0x7f904c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:49.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f9051600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c19cb00 con 0x7f904c1067a0 2024-03-22T10:49:49.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f9051600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c19cc40 con 0x7f904c1079a0 2024-03-22T10:49:49.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f904aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1067a0 0x7f904c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:49.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f904aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1067a0 0x7f904c19bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44896/0 (socket says 172.21.15.115:44896) 2024-03-22T10:49:49.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f904aa00700 1 -- 172.21.15.115:0/3671381165 learned_addr learned my addr 172.21.15.115:0/3671381165 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f9043e00700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f904c1079a0 0x7f904c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f904aa00700 1 -- 172.21.15.115:0/3671381165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f904c1079a0 msgr2=0x7f904c19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f904aa00700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f904c1079a0 0x7f904c19c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.643+0000 7f904aa00700 1 -- 172.21.15.115:0/3671381165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90340095f0 con 0x7f904c1067a0 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.644+0000 7f904aa00700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1067a0 0x7f904c19bfa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f903c00c8f0 tx=0x7f903c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.644+0000 7f9043400700 1 -- 172.21.15.115:0/3671381165 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903c007a70 con 0x7f904c1067a0 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.644+0000 7f9043400700 1 -- 172.21.15.115:0/3671381165 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f903c007bd0 con 0x7f904c1067a0 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.644+0000 7f9043400700 1 -- 172.21.15.115:0/3671381165 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903c018630 con 0x7f904c1067a0 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.644+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904c1a16f0 con 0x7f904c1067a0 2024-03-22T10:49:49.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.644+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904c1a1bf0 con 0x7f904c1067a0 2024-03-22T10:49:49.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.645+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904c10f2f0 con 0x7f904c1067a0 2024-03-22T10:49:49.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.646+0000 7f9043400700 1 -- 172.21.15.115:0/3671381165 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f903c00f450 con 0x7f904c1067a0 2024-03-22T10:49:49.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.647+0000 7f9043400700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f903006dfa0 0x7f9030070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:49.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.647+0000 7f9043400700 1 -- 172.21.15.115:0/3671381165 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f903c08dda0 con 0x7f904c1067a0 2024-03-22T10:49:49.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.648+0000 7f9043e00700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f903006dfa0 0x7f9030070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:49.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.648+0000 7f9043e00700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f903006dfa0 0x7f9030070450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9034000c00 tx=0x7f9034005fb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:49.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.652+0000 7f9043400700 1 -- 172.21.15.115:0/3671381165 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f903c05adb0 con 0x7f904c1067a0 2024-03-22T10:49:49.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.892+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f904c10c2f0 con 0x7f903006dfa0 2024-03-22T10:49:49.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.908+0000 7f9043400700 1 -- 172.21.15.115:0/3671381165 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f904c10c2f0 con 0x7f903006dfa0 2024-03-22T10:49:49.899 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:49:49.899 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (16m) 2m ago 18m 14.2M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:49:49.899 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (18m) 2m ago 18m 8694k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:49:49.899 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (17m) 28s ago 17m 8644k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (4m) 2m ago 18m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (4m) 28s ago 17m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (16m) 2m ago 17m 73.7M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (11m) 2m ago 11m 127M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (11m) 2m ago 11m 16.9M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (11m) 28s ago 11m 17.0M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (11m) 28s ago 11m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (7m) 2m ago 19m 507M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (6m) 28s ago 16m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (6m) 2m ago 20m 46.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (5m) 28s ago 16m 38.8M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (18m) 2m ago 18m 8238k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (16m) 28s ago 16m 8212k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (3m) 2m ago 15m 84.2M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (2m) 2m ago 15m 81.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:49:49.900 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (2m) 2m ago 14m 11.1M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:49:49.901 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (116s) 28s ago 14m 87.8M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:49:49.901 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (75s) 28s ago 13m 84.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:49:49.901 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (33s) 28s ago 12m 11.1M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:49:49.901 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (6m) 2m ago 17m 71.2M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:49:49.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f903006dfa0 msgr2=0x7f9030070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f903006dfa0 0x7f9030070450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9034000c00 tx=0x7f9034005fb0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1067a0 msgr2=0x7f904c19bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:49.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1067a0 0x7f904c19bfa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f903c00c8f0 tx=0x7f903c00ccb0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 shutdown_connections 2024-03-22T10:49:49.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f903006dfa0 0x7f9030070450 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f904c1067a0 0x7f904c19bfa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 --2- 172.21.15.115:0/3671381165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f904c1079a0 0x7f904c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:49.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 >> 172.21.15.115:0/3671381165 conn(0x7f904c075910 msgr2=0x7f904c10abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:49.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.911+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 shutdown_connections 2024-03-22T10:49:49.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:49.912+0000 7f9051600700 1 -- 172.21.15.115:0/3671381165 wait complete. 2024-03-22T10:49:50.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.071+0000 7f1668a00700 1 -- 172.21.15.115:0/3096831000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 msgr2=0x7f1664105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.071+0000 7f1668a00700 1 --2- 172.21.15.115:0/3096831000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 0x7f1664105910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1650009910 tx=0x7f1650009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:50.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.072+0000 7f1668a00700 1 -- 172.21.15.115:0/3096831000 shutdown_connections 2024-03-22T10:49:50.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.072+0000 7f1668a00700 1 --2- 172.21.15.115:0/3096831000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 0x7f1664105910 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.072+0000 7f1668a00700 1 --2- 172.21.15.115:0/3096831000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1664100c10 0x7f1664102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.072+0000 7f1668a00700 1 -- 172.21.15.115:0/3096831000 >> 172.21.15.115:0/3096831000 conn(0x7f16640fa5d0 msgr2=0x7f16640fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:50.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.072+0000 7f1668a00700 1 -- 172.21.15.115:0/3096831000 shutdown_connections 2024-03-22T10:49:50.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.072+0000 7f1668a00700 1 -- 172.21.15.115:0/3096831000 wait complete. 2024-03-22T10:49:50.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.073+0000 7f1668a00700 1 Processor -- start 2024-03-22T10:49:50.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.073+0000 7f1668a00700 1 -- start start 2024-03-22T10:49:50.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.073+0000 7f1668a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1664100c10 0x7f16641958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:50.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.073+0000 7f1668a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 0x7f1664195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:50.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.073+0000 7f1668a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1664196370 con 0x7f1664103530 2024-03-22T10:49:50.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.073+0000 7f1668a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16641964b0 con 0x7f1664100c10 2024-03-22T10:49:50.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.074+0000 7f1661400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 0x7f1664195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:50.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.074+0000 7f1661400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 0x7f1664195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44914/0 (socket says 172.21.15.115:44914) 2024-03-22T10:49:50.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.074+0000 7f1661400700 1 -- 172.21.15.115:0/583580795 learned_addr learned my addr 172.21.15.115:0/583580795 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.074+0000 7f1661e00700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1664100c10 0x7f16641958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.074+0000 7f1661400700 1 -- 172.21.15.115:0/583580795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1664100c10 msgr2=0x7f16641958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.074+0000 7f1661400700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1664100c10 0x7f16641958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.074+0000 7f1661400700 1 -- 172.21.15.115:0/583580795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16500095f0 con 0x7f1664103530 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.075+0000 7f1661400700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 0x7f1664195de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1650005b40 tx=0x7f165000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.075+0000 7f165aa00700 1 -- 172.21.15.115:0/583580795 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165001d070 con 0x7f1664103530 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.075+0000 7f165aa00700 1 -- 172.21.15.115:0/583580795 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1650022470 con 0x7f1664103530 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.075+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166419af10 con 0x7f1664103530 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.075+0000 7f165aa00700 1 -- 172.21.15.115:0/583580795 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165000f690 con 0x7f1664103530 2024-03-22T10:49:50.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.075+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166419b3d0 con 0x7f1664103530 2024-03-22T10:49:50.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.076+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166418fb40 con 0x7f1664103530 2024-03-22T10:49:50.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.077+0000 7f165aa00700 1 -- 172.21.15.115:0/583580795 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f16500225e0 con 0x7f1664103530 2024-03-22T10:49:50.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.078+0000 7f165aa00700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f164806dfa0 0x7f1648070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:50.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.078+0000 7f165aa00700 1 -- 172.21.15.115:0/583580795 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f165008f510 con 0x7f1664103530 2024-03-22T10:49:50.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.078+0000 7f1661e00700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f164806dfa0 0x7f1648070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:50.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.079+0000 7f1661e00700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f164806dfa0 0x7f1648070450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1654005ea0 tx=0x7f1654005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:50.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.083+0000 7f165aa00700 1 -- 172.21.15.115:0/583580795 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f165005c570 con 0x7f1664103530 2024-03-22T10:49:50.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:49 smithi150 ceph-mon[65513]: pgmap v206: 65 pgs: 65 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 8.4 KiB/s wr, 6 op/s 2024-03-22T10:49:50.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:49:50.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:49 smithi115 ceph-mon[78672]: pgmap v206: 65 pgs: 65 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 8.4 KiB/s wr, 6 op/s 2024-03-22T10:49:50.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:49:50.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.378+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f166402cc30 con 0x7f1664103530 2024-03-22T10:49:50.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.378+0000 7f165aa00700 1 -- 172.21.15.115:0/583580795 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+701 (secure 0 0 0) 0x7f165005c100 con 0x7f1664103530 2024-03-22T10:49:50.369 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:49:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:49:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:49:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:49:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:49:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:49:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:49:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 10 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:49:50.370 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:49:50.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f164806dfa0 msgr2=0x7f1648070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f164806dfa0 0x7f1648070450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1654005ea0 tx=0x7f1654005e30 comp rx=0 tx=0).stop 2024-03-22T10:49:50.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 msgr2=0x7f1664195de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 0x7f1664195de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1650005b40 tx=0x7f165000be20 comp rx=0 tx=0).stop 2024-03-22T10:49:50.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 shutdown_connections 2024-03-22T10:49:50.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f164806dfa0 0x7f1648070450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1664100c10 0x7f16641958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 --2- 172.21.15.115:0/583580795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1664103530 0x7f1664195de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 >> 172.21.15.115:0/583580795 conn(0x7f16640fa5d0 msgr2=0x7f16640fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:50.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.381+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 shutdown_connections 2024-03-22T10:49:50.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.382+0000 7f1668a00700 1 -- 172.21.15.115:0/583580795 wait complete. 2024-03-22T10:49:50.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.525+0000 7fa002000700 1 -- 172.21.15.115:0/1904776534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1021c0 msgr2=0x7f9ffc1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.525+0000 7fa002000700 1 --2- 172.21.15.115:0/1904776534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1021c0 0x7f9ffc1025d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009910 tx=0x7f9ff0009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:50.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.525+0000 7fa002000700 1 -- 172.21.15.115:0/1904776534 shutdown_connections 2024-03-22T10:49:50.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.525+0000 7fa002000700 1 --2- 172.21.15.115:0/1904776534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ffc1033c0 0x7f9ffc103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.525+0000 7fa002000700 1 --2- 172.21.15.115:0/1904776534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1021c0 0x7f9ffc1025d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.525+0000 7fa002000700 1 -- 172.21.15.115:0/1904776534 >> 172.21.15.115:0/1904776534 conn(0x7f9ffc0fd8d0 msgr2=0x7f9ffc0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:50.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.525+0000 7fa002000700 1 -- 172.21.15.115:0/1904776534 shutdown_connections 2024-03-22T10:49:50.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.525+0000 7fa002000700 1 -- 172.21.15.115:0/1904776534 wait complete. 2024-03-22T10:49:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.526+0000 7fa002000700 1 Processor -- start 2024-03-22T10:49:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7fa002000700 1 -- start start 2024-03-22T10:49:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7fa002000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ffc1021c0 0x7f9ffc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7fa002000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1033c0 0x7f9ffc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7fa002000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc1985a0 con 0x7f9ffc1033c0 2024-03-22T10:49:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7fa002000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc1986e0 con 0x7f9ffc1021c0 2024-03-22T10:49:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7f9ffbe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1033c0 0x7f9ffc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7f9ffbe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1033c0 0x7f9ffc197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44926/0 (socket says 172.21.15.115:44926) 2024-03-22T10:49:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7f9ffbe00700 1 -- 172.21.15.115:0/2515558090 learned_addr learned my addr 172.21.15.115:0/2515558090 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7fa000c00700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ffc1021c0 0x7f9ffc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7f9ffbe00700 1 -- 172.21.15.115:0/2515558090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ffc1021c0 msgr2=0x7f9ffc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7f9ffbe00700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ffc1021c0 0x7f9ffc197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.527+0000 7f9ffbe00700 1 -- 172.21.15.115:0/2515558090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff00095f0 con 0x7f9ffc1033c0 2024-03-22T10:49:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.528+0000 7f9ffbe00700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1033c0 0x7f9ffc197f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b4b0 tx=0x7f9fec00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.528+0000 7f9ff9600700 1 -- 172.21.15.115:0/2515558090 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fec011800 con 0x7f9ffc1033c0 2024-03-22T10:49:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.528+0000 7f9ff9600700 1 -- 172.21.15.115:0/2515558090 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9fec011e10 con 0x7f9ffc1033c0 2024-03-22T10:49:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.528+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc19d190 con 0x7f9ffc1033c0 2024-03-22T10:49:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.528+0000 7f9ff9600700 1 -- 172.21.15.115:0/2515558090 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fec00f2e0 con 0x7f9ffc1033c0 2024-03-22T10:49:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.528+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc19d760 con 0x7f9ffc1033c0 2024-03-22T10:49:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.529+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc10ad90 con 0x7f9ffc1033c0 2024-03-22T10:49:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.530+0000 7f9ff9600700 1 -- 172.21.15.115:0/2515558090 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f9fec010380 con 0x7f9ffc1033c0 2024-03-22T10:49:50.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.531+0000 7f9ff9600700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9fe406dfa0 0x7f9fe4070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:50.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.531+0000 7f9ff9600700 1 -- 172.21.15.115:0/2515558090 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9fec08d8c0 con 0x7f9ffc1033c0 2024-03-22T10:49:50.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.531+0000 7fa000c00700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9fe406dfa0 0x7f9fe4070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:50.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.532+0000 7fa000c00700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9fe406dfa0 0x7f9fe4070450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0006010 tx=0x7f9ff000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:50.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.536+0000 7f9ff9600700 1 -- 172.21.15.115:0/2515558090 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9fec05a940 con 0x7f9ffc1033c0 2024-03-22T10:49:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.789+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9ffc10af50 con 0x7f9ffc1033c0 2024-03-22T10:49:50.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.790+0000 7f9ff9600700 1 -- 172.21.15.115:0/2515558090 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f9fec05a4d0 con 0x7f9ffc1033c0 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:e18 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 18 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:49:37.577141+0000 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:49:50.781 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 1 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14454,1=24241} 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:49:50.782 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 14454 members: 14454 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:14454} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.115:6826/3803356871,v1:172.21.15.115:6827/3803356871] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{1:24241} state up:stopping seq 5 export targets 0 join_fscid=1 addr [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:49:50.783 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:49:50.784 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:14468} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:49:50.784 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:49:50.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.792+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9fe406dfa0 msgr2=0x7f9fe4070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.792+0000 7fa002000700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9fe406dfa0 0x7f9fe4070450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0006010 tx=0x7f9ff000b540 comp rx=0 tx=0).stop 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.792+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1033c0 msgr2=0x7f9ffc197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.792+0000 7fa002000700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1033c0 0x7f9ffc197f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b4b0 tx=0x7f9fec00b870 comp rx=0 tx=0).stop 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.793+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 shutdown_connections 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.793+0000 7fa002000700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f9fe406dfa0 0x7f9fe4070450 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.793+0000 7fa002000700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ffc1021c0 0x7f9ffc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.793+0000 7fa002000700 1 --2- 172.21.15.115:0/2515558090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ffc1033c0 0x7f9ffc197f80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.793+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 >> 172.21.15.115:0/2515558090 conn(0x7f9ffc0fd8d0 msgr2=0x7f9ffc1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.793+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 shutdown_connections 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.793+0000 7fa002000700 1 -- 172.21.15.115:0/2515558090 wait complete. 2024-03-22T10:49:50.785 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 18 2024-03-22T10:49:50.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.955+0000 7ffa52000700 1 -- 172.21.15.115:0/1622310953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c100a60 msgr2=0x7ffa4c102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.955+0000 7ffa52000700 1 --2- 172.21.15.115:0/1622310953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c100a60 0x7ffa4c102e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000b0a0 tx=0x7ffa4000b3b0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.955+0000 7ffa52000700 1 -- 172.21.15.115:0/1622310953 shutdown_connections 2024-03-22T10:49:50.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.955+0000 7ffa52000700 1 --2- 172.21.15.115:0/1622310953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa4c103380 0x7ffa4c105760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.955+0000 7ffa52000700 1 --2- 172.21.15.115:0/1622310953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c100a60 0x7ffa4c102e40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.955+0000 7ffa52000700 1 -- 172.21.15.115:0/1622310953 >> 172.21.15.115:0/1622310953 conn(0x7ffa4c0fa480 msgr2=0x7ffa4c0fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:50.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.955+0000 7ffa52000700 1 -- 172.21.15.115:0/1622310953 shutdown_connections 2024-03-22T10:49:50.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.955+0000 7ffa52000700 1 -- 172.21.15.115:0/1622310953 wait complete. 2024-03-22T10:49:50.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.956+0000 7ffa52000700 1 Processor -- start 2024-03-22T10:49:50.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.957+0000 7ffa52000700 1 -- start start 2024-03-22T10:49:50.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.957+0000 7ffa52000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa4c100a60 0x7ffa4c1958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:50.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.957+0000 7ffa52000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c103380 0x7ffa4c195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:50.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.957+0000 7ffa52000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c196410 con 0x7ffa4c103380 2024-03-22T10:49:50.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.957+0000 7ffa52000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c196550 con 0x7ffa4c100a60 2024-03-22T10:49:50.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.957+0000 7ffa4be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c103380 0x7ffa4c195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:50.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.957+0000 7ffa4be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c103380 0x7ffa4c195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44938/0 (socket says 172.21.15.115:44938) 2024-03-22T10:49:50.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.957+0000 7ffa4be00700 1 -- 172.21.15.115:0/161663527 learned_addr learned my addr 172.21.15.115:0/161663527 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:50.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.958+0000 7ffa50c00700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa4c100a60 0x7ffa4c1958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:50.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.958+0000 7ffa4be00700 1 -- 172.21.15.115:0/161663527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa4c100a60 msgr2=0x7ffa4c1958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:50.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.958+0000 7ffa4be00700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa4c100a60 0x7ffa4c1958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:50.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.958+0000 7ffa4be00700 1 -- 172.21.15.115:0/161663527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa40009d00 con 0x7ffa4c103380 2024-03-22T10:49:50.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.958+0000 7ffa4be00700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c103380 0x7ffa4c195df0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800b4b0 tx=0x7ffa3800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:50.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.958+0000 7ffa49600700 1 -- 172.21.15.115:0/161663527 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa38011800 con 0x7ffa4c103380 2024-03-22T10:49:50.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.958+0000 7ffa49600700 1 -- 172.21.15.115:0/161663527 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffa38011e10 con 0x7ffa4c103380 2024-03-22T10:49:50.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.958+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c19b000 con 0x7ffa4c103380 2024-03-22T10:49:50.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.958+0000 7ffa49600700 1 -- 172.21.15.115:0/161663527 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3800f2e0 con 0x7ffa4c103380 2024-03-22T10:49:50.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.959+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c19b5a0 con 0x7ffa4c103380 2024-03-22T10:49:50.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.960+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c18fa70 con 0x7ffa4c103380 2024-03-22T10:49:50.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.961+0000 7ffa49600700 1 -- 172.21.15.115:0/161663527 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ffa38011960 con 0x7ffa4c103380 2024-03-22T10:49:50.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.961+0000 7ffa49600700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffa3406dfa0 0x7ffa34070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:50.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.962+0000 7ffa49600700 1 -- 172.21.15.115:0/161663527 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ffa3808d960 con 0x7ffa4c103380 2024-03-22T10:49:50.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.962+0000 7ffa50c00700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffa3406dfa0 0x7ffa34070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:50.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.962+0000 7ffa50c00700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffa3406dfa0 0x7ffa34070450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa40015040 tx=0x7ffa40000f40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:50.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:50.966+0000 7ffa49600700 1 -- 172.21.15.115:0/161663527 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa3805a940 con 0x7ffa4c103380 2024-03-22T10:49:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:50 smithi150 ceph-mon[65513]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:50 smithi150 ceph-mon[65513]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:50 smithi150 ceph-mon[65513]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:50 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/583580795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:50 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2515558090' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:49:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.209+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa4c061190 con 0x7ffa3406dfa0 2024-03-22T10:49:51.201 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:50 smithi115 ceph-mon[78672]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:51.201 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:50 smithi115 ceph-mon[78672]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:51.201 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:50 smithi115 ceph-mon[78672]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:51.201 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:50 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/583580795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:51.201 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:50 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2515558090' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:49:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.212+0000 7ffa49600700 1 -- 172.21.15.115:0/161663527 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffa4c061190 con 0x7ffa3406dfa0 2024-03-22T10:49:51.202 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:49:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:49:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:49:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "osd", 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "12/23 daemons upgraded", 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:49:51.203 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:49:51.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.216+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffa3406dfa0 msgr2=0x7ffa34070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.216+0000 7ffa52000700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffa3406dfa0 0x7ffa34070450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa40015040 tx=0x7ffa40000f40 comp rx=0 tx=0).stop 2024-03-22T10:49:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.216+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c103380 msgr2=0x7ffa4c195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.216+0000 7ffa52000700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c103380 0x7ffa4c195df0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800b4b0 tx=0x7ffa3800b870 comp rx=0 tx=0).stop 2024-03-22T10:49:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.216+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 shutdown_connections 2024-03-22T10:49:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.216+0000 7ffa52000700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffa3406dfa0 0x7ffa34070450 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.217+0000 7ffa52000700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa4c100a60 0x7ffa4c1958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.217+0000 7ffa52000700 1 --2- 172.21.15.115:0/161663527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c103380 0x7ffa4c195df0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.217+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 >> 172.21.15.115:0/161663527 conn(0x7ffa4c0fa480 msgr2=0x7ffa4c0fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.217+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 shutdown_connections 2024-03-22T10:49:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.217+0000 7ffa52000700 1 -- 172.21.15.115:0/161663527 wait complete. 2024-03-22T10:49:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.378+0000 7fee1e200700 1 -- 172.21.15.115:0/2227828839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 msgr2=0x7fee181027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.378+0000 7fee1e200700 1 --2- 172.21.15.115:0/2227828839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 0x7fee181027b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fee08009910 tx=0x7fee08009c20 comp rx=0 tx=0).stop 2024-03-22T10:49:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.378+0000 7fee1e200700 1 -- 172.21.15.115:0/2227828839 shutdown_connections 2024-03-22T10:49:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.378+0000 7fee1e200700 1 --2- 172.21.15.115:0/2227828839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee181035a0 0x7fee181039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.378+0000 7fee1e200700 1 --2- 172.21.15.115:0/2227828839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 0x7fee181027b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.378+0000 7fee1e200700 1 -- 172.21.15.115:0/2227828839 >> 172.21.15.115:0/2227828839 conn(0x7fee180fd990 msgr2=0x7fee180ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.378+0000 7fee1e200700 1 -- 172.21.15.115:0/2227828839 shutdown_connections 2024-03-22T10:49:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.378+0000 7fee1e200700 1 -- 172.21.15.115:0/2227828839 wait complete. 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.379+0000 7fee1e200700 1 Processor -- start 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.379+0000 7fee1e200700 1 -- start start 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.379+0000 7fee1e200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 0x7fee18197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.379+0000 7fee1e200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee181035a0 0x7fee181980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.379+0000 7fee1e200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee18198710 con 0x7fee181023a0 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.379+0000 7fee1e200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee18198850 con 0x7fee181035a0 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.380+0000 7fee17400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 0x7fee18197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.380+0000 7fee17400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 0x7fee18197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44956/0 (socket says 172.21.15.115:44956) 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.380+0000 7fee17400700 1 -- 172.21.15.115:0/2125325393 learned_addr learned my addr 172.21.15.115:0/2125325393 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.380+0000 7fee0fe00700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee181035a0 0x7fee181980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.380+0000 7fee17400700 1 -- 172.21.15.115:0/2125325393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee181035a0 msgr2=0x7fee181980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.380+0000 7fee17400700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee181035a0 0x7fee181980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.380+0000 7fee17400700 1 -- 172.21.15.115:0/2125325393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee080095f0 con 0x7fee181023a0 2024-03-22T10:49:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.380+0000 7fee17400700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 0x7fee18197bb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fee0800bd70 tx=0x7fee0800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:51.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.381+0000 7fee14c00700 1 -- 172.21.15.115:0/2125325393 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee0801d070 con 0x7fee181023a0 2024-03-22T10:49:51.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.381+0000 7fee14c00700 1 -- 172.21.15.115:0/2125325393 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee0800f460 con 0x7fee181023a0 2024-03-22T10:49:51.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.381+0000 7fee14c00700 1 -- 172.21.15.115:0/2125325393 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee08021640 con 0x7fee181023a0 2024-03-22T10:49:51.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.381+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1819d2a0 con 0x7fee181023a0 2024-03-22T10:49:51.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.381+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1819d790 con 0x7fee181023a0 2024-03-22T10:49:51.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.382+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1810aef0 con 0x7fee181023a0 2024-03-22T10:49:51.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.383+0000 7fee14c00700 1 -- 172.21.15.115:0/2125325393 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fee0800fab0 con 0x7fee181023a0 2024-03-22T10:49:51.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.384+0000 7fee14c00700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fedfc06dfa0 0x7fedfc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:49:51.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.384+0000 7fee14c00700 1 -- 172.21.15.115:0/2125325393 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fee0808f170 con 0x7fee181023a0 2024-03-22T10:49:51.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.384+0000 7fee0fe00700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fedfc06dfa0 0x7fedfc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:49:51.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.385+0000 7fee0fe00700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fedfc06dfa0 0x7fedfc070450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee00005950 tx=0x7fee00005cf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:49:51.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.389+0000 7fee14c00700 1 -- 172.21.15.115:0/2125325393 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee0805c150 con 0x7fee181023a0 2024-03-22T10:49:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.695+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fee1810b0e0 con 0x7fee181023a0 2024-03-22T10:49:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.696+0000 7fee14c00700 1 -- 172.21.15.115:0/2125325393 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fee0805bce0 con 0x7fee181023a0 2024-03-22T10:49:51.687 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:49:51.687 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:49:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:49:51.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fedfc06dfa0 msgr2=0x7fedfc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:51.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fedfc06dfa0 0x7fedfc070450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee00005950 tx=0x7fee00005cf0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 msgr2=0x7fee18197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:49:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 0x7fee18197bb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fee0800bd70 tx=0x7fee0800be50 comp rx=0 tx=0).stop 2024-03-22T10:49:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 shutdown_connections 2024-03-22T10:49:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fedfc06dfa0 0x7fedfc070450 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee181023a0 0x7fee18197bb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 --2- 172.21.15.115:0/2125325393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee181035a0 0x7fee181980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:49:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 >> 172.21.15.115:0/2125325393 conn(0x7fee180fd990 msgr2=0x7fee181067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:49:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 shutdown_connections 2024-03-22T10:49:51.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:49:51.699+0000 7fee1e200700 1 -- 172.21.15.115:0/2125325393 wait complete. 2024-03-22T10:49:52.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:51 smithi115 ceph-mon[78672]: pgmap v207: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-22T10:49:52.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:51 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2125325393' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:49:52.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:52.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:52.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:49:52.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:51 smithi150 ceph-mon[65513]: pgmap v207: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-22T10:49:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:51 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2125325393' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:49:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:49:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:49:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:49:53.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:52 smithi150 ceph-mon[65513]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:53.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:53.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:53.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:53.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:53.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:53.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:52 smithi115 ceph-mon[78672]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:49:53.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:49:53.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:53.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:53.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:53.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:49:54.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:53 smithi150 ceph-mon[65513]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-03-22T10:49:54.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:53 smithi150 ceph-mon[65513]: pgmap v208: 65 pgs: 65 active+clean; 263 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 3.2 KiB/s wr, 8 op/s 2024-03-22T10:49:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:53 smithi115 ceph-mon[78672]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-03-22T10:49:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:53 smithi115 ceph-mon[78672]: pgmap v208: 65 pgs: 65 active+clean; 263 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 426 B/s rd, 3.2 KiB/s wr, 8 op/s 2024-03-22T10:49:56.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:55 smithi150 ceph-mon[65513]: pgmap v209: 65 pgs: 65 active+clean; 235 MiB data, 923 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-22T10:49:56.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:55 smithi115 ceph-mon[78672]: pgmap v209: 65 pgs: 65 active+clean; 235 MiB data, 923 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-22T10:49:58.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:57 smithi150 ceph-mon[65513]: pgmap v210: 65 pgs: 65 active+clean; 199 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1023 B/s wr, 7 op/s 2024-03-22T10:49:58.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:57 smithi115 ceph-mon[78672]: pgmap v210: 65 pgs: 65 active+clean; 199 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1023 B/s wr, 7 op/s 2024-03-22T10:50:00.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:49:59 smithi150 ceph-mon[65513]: pgmap v211: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-22T10:50:00.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:49:59 smithi115 ceph-mon[78672]: pgmap v211: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-22T10:50:01.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:00 smithi115 ceph-mon[78672]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:50:01.356 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:00 smithi150 ceph-mon[65513]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:50:02.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:01 smithi115 ceph-mon[78672]: pgmap v212: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 597 B/s wr, 6 op/s 2024-03-22T10:50:02.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:01 smithi150 ceph-mon[65513]: pgmap v212: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 597 B/s wr, 6 op/s 2024-03-22T10:50:03.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:03.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:03.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:50:03.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:03.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:03.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:03.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:03.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:03.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:03.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:02 smithi115 ceph-mon[78672]: daemon mds.cephfs.smithi150.zvyrzc finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-03-22T10:50:03.276 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:03.276 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:03.276 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:50:03.276 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:03.276 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:03.276 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:03.277 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:03.277 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:03.277 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:03.277 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:02 smithi150 ceph-mon[65513]: daemon mds.cephfs.smithi150.zvyrzc finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-03-22T10:50:04.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:03 smithi115 ceph-mon[78672]: pgmap v213: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 426 B/s wr, 2 op/s 2024-03-22T10:50:04.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:03 smithi115 ceph-mon[78672]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-03-22T10:50:04.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:03 smithi115 ceph-mon[78672]: mds.1 [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] down:stopped 2024-03-22T10:50:04.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:03 smithi115 ceph-mon[78672]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:active} 2 up:standby 2024-03-22T10:50:04.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:03 smithi150 ceph-mon[65513]: pgmap v213: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 426 B/s wr, 2 op/s 2024-03-22T10:50:04.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:03 smithi150 ceph-mon[65513]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-03-22T10:50:04.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:03 smithi150 ceph-mon[65513]: mds.1 [v2:172.21.15.150:6824/1103557222,v1:172.21.15.150:6825/1103557222] down:stopped 2024-03-22T10:50:04.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:03 smithi150 ceph-mon[65513]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:active} 2 up:standby 2024-03-22T10:50:05.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:05 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.150:6824/2616517800,v1:172.21.15.150:6825/2616517800] up:boot 2024-03-22T10:50:05.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:05 smithi115 ceph-mon[78672]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:active} 3 up:standby 2024-03-22T10:50:05.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:50:05.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:05 smithi115 ceph-mon[78672]: pgmap v214: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-03-22T10:50:05.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:50:05.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:05 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.150:6824/2616517800,v1:172.21.15.150:6825/2616517800] up:boot 2024-03-22T10:50:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:05 smithi150 ceph-mon[65513]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:active} 3 up:standby 2024-03-22T10:50:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:50:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:05 smithi150 ceph-mon[65513]: pgmap v214: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-03-22T10:50:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:50:07.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:07 smithi150 ceph-mon[65513]: pgmap v215: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 1 op/s 2024-03-22T10:50:07.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:07 smithi115 ceph-mon[78672]: pgmap v215: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 1 op/s 2024-03-22T10:50:09.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:09 smithi150 ceph-mon[65513]: pgmap v216: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T10:50:09.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:09 smithi115 ceph-mon[78672]: pgmap v216: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T10:50:11.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:11 smithi150 ceph-mon[65513]: pgmap v217: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:11.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:11 smithi115 ceph-mon[78672]: pgmap v217: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: pgmap v218: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:50:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:13.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:13.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:13.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:13.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:13.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:13.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi115.ggxfzi"]}]: dispatch 2024-03-22T10:50:13.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:13 smithi150 ceph-mon[65513]: Upgrade: It appears safe to stop mds.cephfs.smithi115.ggxfzi 2024-03-22T10:50:13.661 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: pgmap v218: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:13.661 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:13.661 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:13.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:50:13.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:13.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:13.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:13.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:13.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:13.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:13.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi115.ggxfzi"]}]: dispatch 2024-03-22T10:50:13.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:13 smithi115 ceph-mon[78672]: Upgrade: It appears safe to stop mds.cephfs.smithi115.ggxfzi 2024-03-22T10:50:15.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:15 smithi150 ceph-mon[65513]: pgmap v219: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:15.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:15.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.ggxfzi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:50:15.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:15.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:15 smithi115 ceph-mon[78672]: pgmap v219: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:15.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:15.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.ggxfzi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:50:15.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:16.625 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:16 smithi115 ceph-mon[78672]: Upgrade: Updating mds.cephfs.smithi115.ggxfzi 2024-03-22T10:50:16.625 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:16 smithi115 ceph-mon[78672]: Deploying daemon mds.cephfs.smithi115.ggxfzi on smithi115 2024-03-22T10:50:16.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:16 smithi150 ceph-mon[65513]: Upgrade: Updating mds.cephfs.smithi115.ggxfzi 2024-03-22T10:50:16.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:16 smithi150 ceph-mon[65513]: Deploying daemon mds.cephfs.smithi115.ggxfzi on smithi115 2024-03-22T10:50:17.624 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:17 smithi115 ceph-mon[78672]: pgmap v220: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:17.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:17 smithi150 ceph-mon[65513]: pgmap v220: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:18.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:18 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[78668]: 2024-03-22T10:50:18.120+0000 7f7b87c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:50:18.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:18 smithi150 ceph-mon[65513]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T10:50:18.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:18 smithi150 ceph-mon[65513]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:50:18.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:18 smithi150 ceph-mon[65513]: osdmap e80: 6 total, 6 up, 6 in 2024-03-22T10:50:18.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:18 smithi150 ceph-mon[65513]: Standby daemon mds.cephfs.smithi115.wjotmn assigned to filesystem cephfs as rank 0 2024-03-22T10:50:18.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:18 smithi150 ceph-mon[65513]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T10:50:18.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:18 smithi150 ceph-mon[65513]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T10:50:18.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:18 smithi150 ceph-mon[65513]: fsmap cephfs:1/1 {0=cephfs.smithi115.wjotmn=up:replay} 2 up:standby 2024-03-22T10:50:18.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:18 smithi115 ceph-mon[78672]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T10:50:18.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:18 smithi115 ceph-mon[78672]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:50:18.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:18 smithi115 ceph-mon[78672]: osdmap e80: 6 total, 6 up, 6 in 2024-03-22T10:50:18.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:18 smithi115 ceph-mon[78672]: Standby daemon mds.cephfs.smithi115.wjotmn assigned to filesystem cephfs as rank 0 2024-03-22T10:50:18.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:18 smithi115 ceph-mon[78672]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T10:50:18.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:18 smithi115 ceph-mon[78672]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T10:50:18.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:18 smithi115 ceph-mon[78672]: fsmap cephfs:1/1 {0=cephfs.smithi115.wjotmn=up:replay} 2 up:standby 2024-03-22T10:50:19.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:19 smithi150 ceph-mon[65513]: pgmap v222: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:19.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:19.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:50:19.759 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:19 smithi115 ceph-mon[78672]: pgmap v222: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:19.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:19.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:50:20.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:20 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:50:20.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:20 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:50:21.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:21 smithi150 ceph-mon[65513]: pgmap v223: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s rd, 2 op/s 2024-03-22T10:50:21.759 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:21 smithi115 ceph-mon[78672]: pgmap v223: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s rd, 2 op/s 2024-03-22T10:50:21.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.872+0000 7fc727e00700 1 -- 172.21.15.115:0/3477286542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728100a20 msgr2=0x7fc728102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:21.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.872+0000 7fc727e00700 1 --2- 172.21.15.115:0/3477286542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728100a20 0x7fc728102e00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc718009910 tx=0x7fc718009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:21.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.872+0000 7fc727e00700 1 -- 172.21.15.115:0/3477286542 shutdown_connections 2024-03-22T10:50:21.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.872+0000 7fc727e00700 1 --2- 172.21.15.115:0/3477286542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728103340 0x7fc728105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:21.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.872+0000 7fc727e00700 1 --2- 172.21.15.115:0/3477286542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728100a20 0x7fc728102e00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:21.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.872+0000 7fc727e00700 1 -- 172.21.15.115:0/3477286542 >> 172.21.15.115:0/3477286542 conn(0x7fc7280fa440 msgr2=0x7fc7280fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:21.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.872+0000 7fc727e00700 1 -- 172.21.15.115:0/3477286542 shutdown_connections 2024-03-22T10:50:21.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.872+0000 7fc727e00700 1 -- 172.21.15.115:0/3477286542 wait complete. 2024-03-22T10:50:21.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.874+0000 7fc727e00700 1 Processor -- start 2024-03-22T10:50:21.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.874+0000 7fc727e00700 1 -- start start 2024-03-22T10:50:21.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.875+0000 7fc727e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728100a20 0x7fc7281979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:21.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.875+0000 7fc727e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728103340 0x7fc728197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:21.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.875+0000 7fc727e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7281984c0 con 0x7fc728103340 2024-03-22T10:50:21.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.875+0000 7fc727e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc728198600 con 0x7fc728100a20 2024-03-22T10:50:21.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.875+0000 7fc726000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728103340 0x7fc728197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:21.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.875+0000 7fc726000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728103340 0x7fc728197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34874/0 (socket says 172.21.15.115:34874) 2024-03-22T10:50:21.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.875+0000 7fc726000700 1 -- 172.21.15.115:0/1332952549 learned_addr learned my addr 172.21.15.115:0/1332952549 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:21.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.875+0000 7fc726a00700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728100a20 0x7fc7281979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.876+0000 7fc726000700 1 -- 172.21.15.115:0/1332952549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728100a20 msgr2=0x7fc7281979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.876+0000 7fc726000700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728100a20 0x7fc7281979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.876+0000 7fc726000700 1 -- 172.21.15.115:0/1332952549 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7180095f0 con 0x7fc728103340 2024-03-22T10:50:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.876+0000 7fc726000700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728103340 0x7fc728197f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c930 tx=0x7fc71400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.877+0000 7fc71f400700 1 -- 172.21.15.115:0/1332952549 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc714007ad0 con 0x7fc728103340 2024-03-22T10:50:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.877+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72819d0c0 con 0x7fc728103340 2024-03-22T10:50:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.877+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72819d5e0 con 0x7fc728103340 2024-03-22T10:50:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.877+0000 7fc71f400700 1 -- 172.21.15.115:0/1332952549 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc71400ce80 con 0x7fc728103340 2024-03-22T10:50:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.877+0000 7fc71f400700 1 -- 172.21.15.115:0/1332952549 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc714018630 con 0x7fc728103340 2024-03-22T10:50:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.878+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc708005320 con 0x7fc728103340 2024-03-22T10:50:21.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.879+0000 7fc71f400700 1 -- 172.21.15.115:0/1332952549 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc71400f450 con 0x7fc728103340 2024-03-22T10:50:21.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.880+0000 7fc71f400700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc70c06dfa0 0x7fc70c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:21.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.880+0000 7fc71f400700 1 -- 172.21.15.115:0/1332952549 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc71408dc70 con 0x7fc728103340 2024-03-22T10:50:21.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.881+0000 7fc726a00700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc70c06dfa0 0x7fc70c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.882+0000 7fc726a00700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc70c06dfa0 0x7fc70c070450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b5c0 tx=0x7fc7180052a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:21.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:21.890+0000 7fc71f400700 1 -- 172.21.15.115:0/1332952549 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc714094080 con 0x7fc728103340 2024-03-22T10:50:22.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.139+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc708000bf0 con 0x7fc70c06dfa0 2024-03-22T10:50:22.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.141+0000 7fc71f400700 1 -- 172.21.15.115:0/1332952549 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc708000bf0 con 0x7fc70c06dfa0 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc70c06dfa0 msgr2=0x7fc70c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc70c06dfa0 0x7fc70c070450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b5c0 tx=0x7fc7180052a0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728103340 msgr2=0x7fc728197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728103340 0x7fc728197f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c930 tx=0x7fc71400cc40 comp rx=0 tx=0).stop 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 shutdown_connections 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc70c06dfa0 0x7fc70c070450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728100a20 0x7fc7281979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 --2- 172.21.15.115:0/1332952549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc728103340 0x7fc728197f30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 >> 172.21.15.115:0/1332952549 conn(0x7fc7280fa440 msgr2=0x7fc7280ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:22.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 shutdown_connections 2024-03-22T10:50:22.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.145+0000 7fc727e00700 1 -- 172.21.15.115:0/1332952549 wait complete. 2024-03-22T10:50:22.159 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:50:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.307+0000 7f1a1b600700 1 -- 172.21.15.115:0/4228478060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 msgr2=0x7f1a141017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.307+0000 7f1a1b600700 1 --2- 172.21.15.115:0/4228478060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 0x7f1a141017a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009910 tx=0x7f1a00009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.308+0000 7f1a1b600700 1 -- 172.21.15.115:0/4228478060 shutdown_connections 2024-03-22T10:50:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.308+0000 7f1a1b600700 1 --2- 172.21.15.115:0/4228478060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 0x7f1a141017a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.308+0000 7f1a1b600700 1 --2- 172.21.15.115:0/4228478060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a14100150 0x7f1a14100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.308+0000 7f1a1b600700 1 -- 172.21.15.115:0/4228478060 >> 172.21.15.115:0/4228478060 conn(0x7f1a140fb6e0 msgr2=0x7f1a140fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.308+0000 7f1a1b600700 1 -- 172.21.15.115:0/4228478060 shutdown_connections 2024-03-22T10:50:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.308+0000 7f1a1b600700 1 -- 172.21.15.115:0/4228478060 wait complete. 2024-03-22T10:50:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.309+0000 7f1a1b600700 1 Processor -- start 2024-03-22T10:50:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.309+0000 7f1a1b600700 1 -- start start 2024-03-22T10:50:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.310+0000 7f1a1b600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a14100150 0x7f1a14195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.310+0000 7f1a1b600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 0x7f1a14195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.310+0000 7f1a1b600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14196560 con 0x7f1a14101350 2024-03-22T10:50:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.310+0000 7f1a1b600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a141966a0 con 0x7f1a14100150 2024-03-22T10:50:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.310+0000 7f1a13e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 0x7f1a14195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.310+0000 7f1a13e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 0x7f1a14195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34896/0 (socket says 172.21.15.115:34896) 2024-03-22T10:50:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.310+0000 7f1a13e00700 1 -- 172.21.15.115:0/4028865011 learned_addr learned my addr 172.21.15.115:0/4028865011 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.310+0000 7f1a18c00700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a14100150 0x7f1a14195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.311+0000 7f1a13e00700 1 -- 172.21.15.115:0/4028865011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a14100150 msgr2=0x7f1a14195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.311+0000 7f1a13e00700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a14100150 0x7f1a14195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.311+0000 7f1a13e00700 1 -- 172.21.15.115:0/4028865011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000095f0 con 0x7f1a14101350 2024-03-22T10:50:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.311+0000 7f1a13e00700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 0x7f1a14195f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000b5c0 tx=0x7f1a0000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.311+0000 7f1a11600700 1 -- 172.21.15.115:0/4028865011 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a0001d070 con 0x7f1a14101350 2024-03-22T10:50:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.311+0000 7f1a11600700 1 -- 172.21.15.115:0/4028865011 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a00022470 con 0x7f1a14101350 2024-03-22T10:50:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.311+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1419b0f0 con 0x7f1a14101350 2024-03-22T10:50:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.312+0000 7f1a11600700 1 -- 172.21.15.115:0/4028865011 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a0000f690 con 0x7f1a14101350 2024-03-22T10:50:22.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.312+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1419b660 con 0x7f1a14101350 2024-03-22T10:50:22.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.313+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a14108d40 con 0x7f1a14101350 2024-03-22T10:50:22.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.315+0000 7f1a11600700 1 -- 172.21.15.115:0/4028865011 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f1a00004b70 con 0x7f1a14101350 2024-03-22T10:50:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.316+0000 7f1a11600700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f19fc06dfa0 0x7f19fc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.316+0000 7f1a11600700 1 -- 172.21.15.115:0/4028865011 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1a0008f570 con 0x7f1a14101350 2024-03-22T10:50:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.316+0000 7f1a18c00700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f19fc06dfa0 0x7f19fc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:22.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.317+0000 7f1a18c00700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f19fc06dfa0 0x7f19fc070450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a08006fd0 tx=0x7f1a08008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:22.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.321+0000 7f1a11600700 1 -- 172.21.15.115:0/4028865011 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1a0005c520 con 0x7f1a14101350 2024-03-22T10:50:22.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.563+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a1419bd60 con 0x7f19fc06dfa0 2024-03-22T10:50:22.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.565+0000 7f1a11600700 1 -- 172.21.15.115:0/4028865011 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1a1419bd60 con 0x7f19fc06dfa0 2024-03-22T10:50:22.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.567+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f19fc06dfa0 msgr2=0x7f19fc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.567+0000 7f1a1b600700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f19fc06dfa0 0x7f19fc070450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a08006fd0 tx=0x7f1a08008040 comp rx=0 tx=0).stop 2024-03-22T10:50:22.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.567+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 msgr2=0x7f1a14195f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.567+0000 7f1a1b600700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 0x7f1a14195f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000b5c0 tx=0x7f1a0000be20 comp rx=0 tx=0).stop 2024-03-22T10:50:22.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.567+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 shutdown_connections 2024-03-22T10:50:22.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.567+0000 7f1a1b600700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f19fc06dfa0 0x7f19fc070450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.567+0000 7f1a1b600700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a14100150 0x7f1a14195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.567+0000 7f1a1b600700 1 --2- 172.21.15.115:0/4028865011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a14101350 0x7f1a14195f40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.568+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 >> 172.21.15.115:0/4028865011 conn(0x7f1a140fb6e0 msgr2=0x7f1a14104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.568+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 shutdown_connections 2024-03-22T10:50:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.568+0000 7f1a1b600700 1 -- 172.21.15.115:0/4028865011 wait complete. 2024-03-22T10:50:22.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.745+0000 7fdcfe000700 1 -- 172.21.15.115:0/2715232283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 msgr2=0x7fdcf8103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.745+0000 7fdcfe000700 1 --2- 172.21.15.115:0/2715232283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 0x7fdcf8103020 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009910 tx=0x7fdce8009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:22.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.745+0000 7fdcfe000700 1 -- 172.21.15.115:0/2715232283 shutdown_connections 2024-03-22T10:50:22.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.745+0000 7fdcfe000700 1 --2- 172.21.15.115:0/2715232283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8103560 0x7fdcf8105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.745+0000 7fdcfe000700 1 --2- 172.21.15.115:0/2715232283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 0x7fdcf8103020 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.745+0000 7fdcfe000700 1 -- 172.21.15.115:0/2715232283 >> 172.21.15.115:0/2715232283 conn(0x7fdcf80fa5e0 msgr2=0x7fdcf80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:22.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.745+0000 7fdcfe000700 1 -- 172.21.15.115:0/2715232283 shutdown_connections 2024-03-22T10:50:22.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.746+0000 7fdcfe000700 1 -- 172.21.15.115:0/2715232283 wait complete. 2024-03-22T10:50:22.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.747+0000 7fdcfe000700 1 Processor -- start 2024-03-22T10:50:22.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.747+0000 7fdcfe000700 1 -- start start 2024-03-22T10:50:22.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.747+0000 7fdcfe000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 0x7fdcf81005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:22.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.747+0000 7fdcfe000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8103560 0x7fdcf80febf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:22.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.747+0000 7fdcfe000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80ff130 con 0x7fdcf8100c40 2024-03-22T10:50:22.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.747+0000 7fdcfe000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80ff270 con 0x7fdcf8103560 2024-03-22T10:50:22.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.748+0000 7fdcf7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 0x7fdcf81005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:22.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.748+0000 7fdcf7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 0x7fdcf81005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34920/0 (socket says 172.21.15.115:34920) 2024-03-22T10:50:22.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.748+0000 7fdcf7400700 1 -- 172.21.15.115:0/17500614 learned_addr learned my addr 172.21.15.115:0/17500614 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:22.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.748+0000 7fdcf6a00700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8103560 0x7fdcf80febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:22.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.748+0000 7fdcf7400700 1 -- 172.21.15.115:0/17500614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8103560 msgr2=0x7fdcf80febf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.748+0000 7fdcf7400700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8103560 0x7fdcf80febf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.748+0000 7fdcf7400700 1 -- 172.21.15.115:0/17500614 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80095f0 con 0x7fdcf8100c40 2024-03-22T10:50:22.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.748+0000 7fdcf6a00700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8103560 0x7fdcf80febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:50:22.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.749+0000 7fdcf7400700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 0x7fdcf81005a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdce8005b40 tx=0x7fdce800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:22.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.749+0000 7fdcefe00700 1 -- 172.21.15.115:0/17500614 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdce801d070 con 0x7fdcf8100c40 2024-03-22T10:50:22.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.749+0000 7fdcefe00700 1 -- 172.21.15.115:0/17500614 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdce8022470 con 0x7fdcf8100c40 2024-03-22T10:50:22.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.749+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf80ff4f0 con 0x7fdcf8100c40 2024-03-22T10:50:22.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.749+0000 7fdcefe00700 1 -- 172.21.15.115:0/17500614 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdce800f690 con 0x7fdcf8100c40 2024-03-22T10:50:22.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.749+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf80ff9e0 con 0x7fdcf8100c40 2024-03-22T10:50:22.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.750+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf818fb70 con 0x7fdcf8100c40 2024-03-22T10:50:22.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.752+0000 7fdcefe00700 1 -- 172.21.15.115:0/17500614 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fdce80225e0 con 0x7fdcf8100c40 2024-03-22T10:50:22.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.752+0000 7fdcefe00700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdcdc06dfa0 0x7fdcdc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:22.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.753+0000 7fdcefe00700 1 -- 172.21.15.115:0/17500614 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdce808f6b0 con 0x7fdcf8100c40 2024-03-22T10:50:22.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.753+0000 7fdcf6a00700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdcdc06dfa0 0x7fdcdc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:22.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.754+0000 7fdcf6a00700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdcdc06dfa0 0x7fdcdc070450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005fd0 tx=0x7fdce4005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:22.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.757+0000 7fdcefe00700 1 -- 172.21.15.115:0/17500614 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdce805c630 con 0x7fdcf8100c40 2024-03-22T10:50:22.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:22.987+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcf8061190 con 0x7fdcdc06dfa0 2024-03-22T10:50:22.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.002+0000 7fdcefe00700 1 -- 172.21.15.115:0/17500614 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fdcf8061190 con 0x7fdcdc06dfa0 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (16m) 2m ago 18m 14.2M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (19m) 2m ago 19m 8694k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (17m) 61s ago 17m 8644k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (4m) 2m ago 19m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (4m) 61s ago 17m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (16m) 2m ago 18m 73.7M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (11m) 2m ago 11m 127M - 18.2.0 dc2bc1663786 dc1020df8486 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (11m) 2m ago 11m 16.9M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (11m) 61s ago 11m 17.0M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (11m) 61s ago 11m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (8m) 2m ago 20m 507M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:50:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (6m) 61s ago 17m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (6m) 2m ago 20m 46.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (5m) 61s ago 17m 38.8M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (18m) 2m ago 18m 8238k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (17m) 61s ago 17m 8212k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (4m) 2m ago 16m 84.2M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (3m) 2m ago 15m 81.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (2m) 2m ago 15m 11.1M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (2m) 61s ago 14m 87.8M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (108s) 61s ago 14m 84.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (66s) 61s ago 13m 11.1M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:50:22.994 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (7m) 2m ago 18m 71.2M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:50:22.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdcdc06dfa0 msgr2=0x7fdcdc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdcdc06dfa0 0x7fdcdc070450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005fd0 tx=0x7fdce4005dc0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 msgr2=0x7fdcf81005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:22.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 0x7fdcf81005a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdce8005b40 tx=0x7fdce800be20 comp rx=0 tx=0).stop 2024-03-22T10:50:22.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 shutdown_connections 2024-03-22T10:50:22.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fdcdc06dfa0 0x7fdcdc070450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcf8100c40 0x7fdcf81005a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 --2- 172.21.15.115:0/17500614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8103560 0x7fdcf80febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:22.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 >> 172.21.15.115:0/17500614 conn(0x7fdcf80fa5e0 msgr2=0x7fdcf80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:22.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.005+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 shutdown_connections 2024-03-22T10:50:22.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.006+0000 7fdcfe000700 1 -- 172.21.15.115:0/17500614 wait complete. 2024-03-22T10:50:23.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.146+0000 7f0a35e00700 1 -- 172.21.15.115:0/1304124293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 msgr2=0x7f0a30073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:23.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.146+0000 7f0a35e00700 1 --2- 172.21.15.115:0/1304124293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 0x7f0a30073ba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009910 tx=0x7f0a1c009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:23.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.147+0000 7f0a35e00700 1 -- 172.21.15.115:0/1304124293 shutdown_connections 2024-03-22T10:50:23.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.147+0000 7f0a35e00700 1 --2- 172.21.15.115:0/1304124293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 0x7f0a30073ba0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.147+0000 7f0a35e00700 1 --2- 172.21.15.115:0/1304124293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a30074d00 0x7f0a30073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.147+0000 7f0a35e00700 1 -- 172.21.15.115:0/1304124293 >> 172.21.15.115:0/1304124293 conn(0x7f0a300fb6e0 msgr2=0x7f0a300fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:23.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.147+0000 7f0a35e00700 1 -- 172.21.15.115:0/1304124293 shutdown_connections 2024-03-22T10:50:23.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.147+0000 7f0a35e00700 1 -- 172.21.15.115:0/1304124293 wait complete. 2024-03-22T10:50:23.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.148+0000 7f0a35e00700 1 Processor -- start 2024-03-22T10:50:23.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.148+0000 7f0a35e00700 1 -- start start 2024-03-22T10:50:23.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.148+0000 7f0a35e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 0x7f0a30197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:23.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.148+0000 7f0a35e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a30074d00 0x7f0a301980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:23.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.148+0000 7f0a35e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a30198710 con 0x7f0a30073730 2024-03-22T10:50:23.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.148+0000 7f0a35e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a30198850 con 0x7f0a30074d00 2024-03-22T10:50:23.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.149+0000 7f0a2ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 0x7f0a30197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:23.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.149+0000 7f0a2ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 0x7f0a30197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34930/0 (socket says 172.21.15.115:34930) 2024-03-22T10:50:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.149+0000 7f0a2ea00700 1 -- 172.21.15.115:0/2182187867 learned_addr learned my addr 172.21.15.115:0/2182187867 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.149+0000 7f0a2e000700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a30074d00 0x7f0a301980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.149+0000 7f0a2ea00700 1 -- 172.21.15.115:0/2182187867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a30074d00 msgr2=0x7f0a301980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.149+0000 7f0a2ea00700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a30074d00 0x7f0a301980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.149+0000 7f0a2ea00700 1 -- 172.21.15.115:0/2182187867 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c0095f0 con 0x7f0a30073730 2024-03-22T10:50:23.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.150+0000 7f0a2ea00700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 0x7f0a30197bb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000b4b0 tx=0x7f0a2000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:23.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.150+0000 7f0a34a00700 1 -- 172.21.15.115:0/2182187867 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a20011800 con 0x7f0a30073730 2024-03-22T10:50:23.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.150+0000 7f0a34a00700 1 -- 172.21.15.115:0/2182187867 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a20011e10 con 0x7f0a30073730 2024-03-22T10:50:23.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.150+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3019d300 con 0x7f0a30073730 2024-03-22T10:50:23.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.150+0000 7f0a34a00700 1 -- 172.21.15.115:0/2182187867 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a2000f2e0 con 0x7f0a30073730 2024-03-22T10:50:23.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.150+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3019d8d0 con 0x7f0a30073730 2024-03-22T10:50:23.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.151+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3010af00 con 0x7f0a30073730 2024-03-22T10:50:23.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.153+0000 7f0a34a00700 1 -- 172.21.15.115:0/2182187867 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f0a20010380 con 0x7f0a30073730 2024-03-22T10:50:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.155+0000 7f0a34a00700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a1406dfa0 0x7f0a14070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.155+0000 7f0a34a00700 1 -- 172.21.15.115:0/2182187867 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0a2008d980 con 0x7f0a30073730 2024-03-22T10:50:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.155+0000 7f0a2e000700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a1406dfa0 0x7f0a14070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:23.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.156+0000 7f0a2e000700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a1406dfa0 0x7f0a14070450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c006010 tx=0x7f0a1c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:23.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.163+0000 7f0a34a00700 1 -- 172.21.15.115:0/2182187867 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a2005a940 con 0x7f0a30073730 2024-03-22T10:50:23.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.471+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0a3010b0f0 con 0x7f0a30073730 2024-03-22T10:50:23.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.472+0000 7f0a34a00700 1 -- 172.21.15.115:0/2182187867 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+701 (secure 0 0 0) 0x7f0a2005a4d0 con 0x7f0a30073730 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:50:23.463 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3 2024-03-22T10:50:23.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:50:23.464 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:50:23.464 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-03-22T10:50:23.464 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 10 2024-03-22T10:50:23.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:50:23.464 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:50:23.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a1406dfa0 msgr2=0x7f0a14070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:23.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a1406dfa0 0x7f0a14070450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c006010 tx=0x7f0a1c00b540 comp rx=0 tx=0).stop 2024-03-22T10:50:23.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 msgr2=0x7f0a30197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:23.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 0x7f0a30197bb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000b4b0 tx=0x7f0a2000b870 comp rx=0 tx=0).stop 2024-03-22T10:50:23.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 shutdown_connections 2024-03-22T10:50:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a1406dfa0 0x7f0a14070450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a30073730 0x7f0a30197bb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 --2- 172.21.15.115:0/2182187867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a30074d00 0x7f0a301980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 >> 172.21.15.115:0/2182187867 conn(0x7f0a300fb6e0 msgr2=0x7f0a30105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.475+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 shutdown_connections 2024-03-22T10:50:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.476+0000 7f0a35e00700 1 -- 172.21.15.115:0/2182187867 wait complete. 2024-03-22T10:50:23.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.632+0000 7f2b70000700 1 -- 172.21.15.115:0/2904544821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b681035a0 msgr2=0x7f2b68105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:23.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.632+0000 7f2b70000700 1 --2- 172.21.15.115:0/2904544821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b681035a0 0x7f2b68105980 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009910 tx=0x7f2b54009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:23.623 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:23 smithi115 ceph-mon[78672]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:23.623 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:23 smithi115 ceph-mon[78672]: pgmap v224: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-03-22T10:50:23.623 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:23 smithi115 ceph-mon[78672]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:23.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.632+0000 7f2b70000700 1 -- 172.21.15.115:0/2904544821 shutdown_connections 2024-03-22T10:50:23.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.632+0000 7f2b70000700 1 --2- 172.21.15.115:0/2904544821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b681035a0 0x7f2b68105980 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.632+0000 7f2b70000700 1 --2- 172.21.15.115:0/2904544821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b68100c80 0x7f2b68103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.632+0000 7f2b70000700 1 -- 172.21.15.115:0/2904544821 >> 172.21.15.115:0/2904544821 conn(0x7f2b680fa5e0 msgr2=0x7f2b680fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:23.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.632+0000 7f2b70000700 1 -- 172.21.15.115:0/2904544821 shutdown_connections 2024-03-22T10:50:23.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.633+0000 7f2b70000700 1 -- 172.21.15.115:0/2904544821 wait complete. 2024-03-22T10:50:23.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.634+0000 7f2b70000700 1 Processor -- start 2024-03-22T10:50:23.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.634+0000 7f2b70000700 1 -- start start 2024-03-22T10:50:23.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.634+0000 7f2b70000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b68100c80 0x7f2b68195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:23.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.634+0000 7f2b70000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b681035a0 0x7f2b68195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:23.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.634+0000 7f2b70000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b681964c0 con 0x7f2b68100c80 2024-03-22T10:50:23.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.634+0000 7f2b70000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68196600 con 0x7f2b681035a0 2024-03-22T10:50:23.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.634+0000 7f2b6d600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b68100c80 0x7f2b68195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:23.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.634+0000 7f2b6d600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b68100c80 0x7f2b68195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34958/0 (socket says 172.21.15.115:34958) 2024-03-22T10:50:23.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.634+0000 7f2b6d600700 1 -- 172.21.15.115:0/26591984 learned_addr learned my addr 172.21.15.115:0/26591984 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:23.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.635+0000 7f2b6d600700 1 -- 172.21.15.115:0/26591984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b681035a0 msgr2=0x7f2b68195ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-22T10:50:23.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.635+0000 7f2b6d600700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b681035a0 0x7f2b68195ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.635+0000 7f2b6d600700 1 -- 172.21.15.115:0/26591984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540095f0 con 0x7f2b68100c80 2024-03-22T10:50:23.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.636+0000 7f2b6d600700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b68100c80 0x7f2b68195960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c8a0 tx=0x7f2b5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:23.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.636+0000 7f2b66000700 1 -- 172.21.15.115:0/26591984 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b5c00cea0 con 0x7f2b68100c80 2024-03-22T10:50:23.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.636+0000 7f2b66000700 1 -- 172.21.15.115:0/26591984 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c004830 con 0x7f2b68100c80 2024-03-22T10:50:23.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.636+0000 7f2b66000700 1 -- 172.21.15.115:0/26591984 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b5c0055b0 con 0x7f2b68100c80 2024-03-22T10:50:23.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.636+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6819b0b0 con 0x7f2b68100c80 2024-03-22T10:50:23.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.636+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6819b600 con 0x7f2b68100c80 2024-03-22T10:50:23.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.638+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6818fb80 con 0x7f2b68100c80 2024-03-22T10:50:23.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.639+0000 7f2b66000700 1 -- 172.21.15.115:0/26591984 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f2b5c007ad0 con 0x7f2b68100c80 2024-03-22T10:50:23.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.640+0000 7f2b66000700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2b5006dfa0 0x7f2b50070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:23.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.640+0000 7f2b66000700 1 -- 172.21.15.115:0/26591984 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2b5c08d9d0 con 0x7f2b68100c80 2024-03-22T10:50:23.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.640+0000 7f2b6cc00700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2b5006dfa0 0x7f2b50070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:23.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.641+0000 7f2b6cc00700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2b5006dfa0 0x7f2b50070450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005090 tx=0x7f2b54005fb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:23.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.645+0000 7f2b66000700 1 -- 172.21.15.115:0/26591984 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2b5c05a9f0 con 0x7f2b68100c80 2024-03-22T10:50:23.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:23 smithi150 ceph-mon[65513]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:23.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:23 smithi150 ceph-mon[65513]: pgmap v224: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-03-22T10:50:23.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:23 smithi150 ceph-mon[65513]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:23.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.962+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2b68061a00 con 0x7f2b68100c80 2024-03-22T10:50:23.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.963+0000 7f2b66000700 1 -- 172.21.15.115:0/26591984 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1708 (secure 0 0 0) 0x7f2b5c05a580 con 0x7f2b68100c80 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:e22 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 22 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:50:18.128814+0000 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:50:23.954 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 80 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 1 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:in 0 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=14468} 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 1 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:50:23.955 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 0 members: 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{0:14468} state up:replay seq 3 join_fscid=1 addr [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:50:23.956 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{-1:44271} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6824/2616517800,v1:172.21.15.150:6825/2616517800] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2b5006dfa0 msgr2=0x7f2b50070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2b5006dfa0 0x7f2b50070450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005090 tx=0x7f2b54005fb0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b68100c80 msgr2=0x7f2b68195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b68100c80 0x7f2b68195960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c8a0 tx=0x7f2b5c00cc60 comp rx=0 tx=0).stop 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 shutdown_connections 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2b5006dfa0 0x7f2b50070450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b68100c80 0x7f2b68195960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 --2- 172.21.15.115:0/26591984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b681035a0 0x7f2b68195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 >> 172.21.15.115:0/26591984 conn(0x7f2b680fa5e0 msgr2=0x7f2b680fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 shutdown_connections 2024-03-22T10:50:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:23.966+0000 7f2b70000700 1 -- 172.21.15.115:0/26591984 wait complete. 2024-03-22T10:50:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 22 2024-03-22T10:50:24.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.117+0000 7fa125200700 1 -- 172.21.15.115:0/2873097788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 msgr2=0x7fa120073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:24.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.117+0000 7fa125200700 1 --2- 172.21.15.115:0/2873097788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 0x7fa120073ba0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009910 tx=0x7fa10c009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:24.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.117+0000 7fa125200700 1 -- 172.21.15.115:0/2873097788 shutdown_connections 2024-03-22T10:50:24.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.117+0000 7fa125200700 1 --2- 172.21.15.115:0/2873097788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 0x7fa120073ba0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.117+0000 7fa125200700 1 --2- 172.21.15.115:0/2873097788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa120074d00 0x7fa120073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.117+0000 7fa125200700 1 -- 172.21.15.115:0/2873097788 >> 172.21.15.115:0/2873097788 conn(0x7fa1200fb6e0 msgr2=0x7fa1200fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:24.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.117+0000 7fa125200700 1 -- 172.21.15.115:0/2873097788 shutdown_connections 2024-03-22T10:50:24.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.117+0000 7fa125200700 1 -- 172.21.15.115:0/2873097788 wait complete. 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.118+0000 7fa125200700 1 Processor -- start 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.118+0000 7fa125200700 1 -- start start 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.118+0000 7fa125200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 0x7fa120197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.118+0000 7fa125200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa120074d00 0x7fa1201980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.118+0000 7fa125200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa120198710 con 0x7fa120073730 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.118+0000 7fa125200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa120198850 con 0x7fa120074d00 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.119+0000 7fa11de00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 0x7fa120197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.119+0000 7fa11de00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 0x7fa120197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34972/0 (socket says 172.21.15.115:34972) 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.119+0000 7fa11de00700 1 -- 172.21.15.115:0/2622298221 learned_addr learned my addr 172.21.15.115:0/2622298221 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.119+0000 7fa11de00700 1 -- 172.21.15.115:0/2622298221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa120074d00 msgr2=0x7fa1201980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:24.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.119+0000 7fa11de00700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa120074d00 0x7fa1201980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.119+0000 7fa11de00700 1 -- 172.21.15.115:0/2622298221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0095f0 con 0x7fa120073730 2024-03-22T10:50:24.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.119+0000 7fa11de00700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 0x7fa120197bb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b4b0 tx=0x7fa11000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:24.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.119+0000 7fa116a00700 1 -- 172.21.15.115:0/2622298221 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa110011800 con 0x7fa120073730 2024-03-22T10:50:24.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.120+0000 7fa116a00700 1 -- 172.21.15.115:0/2622298221 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa110011e10 con 0x7fa120073730 2024-03-22T10:50:24.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.120+0000 7fa116a00700 1 -- 172.21.15.115:0/2622298221 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa11000f2e0 con 0x7fa120073730 2024-03-22T10:50:24.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.120+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12019d300 con 0x7fa120073730 2024-03-22T10:50:24.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.120+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12019d8d0 con 0x7fa120073730 2024-03-22T10:50:24.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.121+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12010af00 con 0x7fa120073730 2024-03-22T10:50:24.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.122+0000 7fa116a00700 1 -- 172.21.15.115:0/2622298221 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa110010380 con 0x7fa120073730 2024-03-22T10:50:24.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.122+0000 7fa116a00700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa10406dfa0 0x7fa104070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:24.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.123+0000 7fa116a00700 1 -- 172.21.15.115:0/2622298221 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa11008d9c0 con 0x7fa120073730 2024-03-22T10:50:24.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.123+0000 7fa11d400700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa10406dfa0 0x7fa104070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:24.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.123+0000 7fa11d400700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa10406dfa0 0x7fa104070450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa10c005090 tx=0x7fa10c005fb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:24.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.127+0000 7fa116a00700 1 -- 172.21.15.115:0/2622298221 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa11005a940 con 0x7fa120073730 2024-03-22T10:50:24.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.366+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1201076d0 con 0x7fa10406dfa0 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.368+0000 7fa116a00700 1 -- 172.21.15.115:0/2622298221 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa1201076d0 con 0x7fa10406dfa0 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout: "osd", 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:50:24.359 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:50:24.360 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "12/23 daemons upgraded", 2024-03-22T10:50:24.360 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T10:50:24.360 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:50:24.360 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:50:24.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.371+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa10406dfa0 msgr2=0x7fa104070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:24.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.371+0000 7fa125200700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa10406dfa0 0x7fa104070450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa10c005090 tx=0x7fa10c005fb0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.371+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 msgr2=0x7fa120197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:24.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.371+0000 7fa125200700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 0x7fa120197bb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b4b0 tx=0x7fa11000b870 comp rx=0 tx=0).stop 2024-03-22T10:50:24.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.371+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 shutdown_connections 2024-03-22T10:50:24.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.371+0000 7fa125200700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa10406dfa0 0x7fa104070450 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.371+0000 7fa125200700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa120073730 0x7fa120197bb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.371+0000 7fa125200700 1 --2- 172.21.15.115:0/2622298221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa120074d00 0x7fa1201980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.371+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 >> 172.21.15.115:0/2622298221 conn(0x7fa1200fb6e0 msgr2=0x7fa120105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:24.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.372+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 shutdown_connections 2024-03-22T10:50:24.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.372+0000 7fa125200700 1 -- 172.21.15.115:0/2622298221 wait complete. 2024-03-22T10:50:24.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.517+0000 7f0a26000700 1 -- 172.21.15.115:0/4192709002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201077b0 msgr2=0x7f0a20107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:24.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.517+0000 7f0a26000700 1 --2- 172.21.15.115:0/4192709002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201077b0 0x7f0a20107c00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009910 tx=0x7f0a0c009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:24.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.517+0000 7f0a26000700 1 -- 172.21.15.115:0/4192709002 shutdown_connections 2024-03-22T10:50:24.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.517+0000 7f0a26000700 1 --2- 172.21.15.115:0/4192709002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201077b0 0x7f0a20107c00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.517+0000 7f0a26000700 1 --2- 172.21.15.115:0/4192709002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a201065b0 0x7f0a201069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.517+0000 7f0a26000700 1 -- 172.21.15.115:0/4192709002 >> 172.21.15.115:0/4192709002 conn(0x7f0a20075740 msgr2=0x7f0a20077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:24.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.517+0000 7f0a26000700 1 -- 172.21.15.115:0/4192709002 shutdown_connections 2024-03-22T10:50:24.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.517+0000 7f0a26000700 1 -- 172.21.15.115:0/4192709002 wait complete. 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.518+0000 7f0a26000700 1 Processor -- start 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.518+0000 7f0a26000700 1 -- start start 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a26000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201065b0 0x7f0a2019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a26000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a201077b0 0x7f0a2019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a26000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2019ca50 con 0x7f0a201065b0 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a26000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2019cb90 con 0x7f0a201077b0 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a24c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201065b0 0x7f0a2019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a24c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201065b0 0x7f0a2019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34982/0 (socket says 172.21.15.115:34982) 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a24c00700 1 -- 172.21.15.115:0/31283833 learned_addr learned my addr 172.21.15.115:0/31283833 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a1fe00700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a201077b0 0x7f0a2019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a24c00700 1 -- 172.21.15.115:0/31283833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a201077b0 msgr2=0x7f0a2019c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a24c00700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a201077b0 0x7f0a2019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.519+0000 7f0a24c00700 1 -- 172.21.15.115:0/31283833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0095f0 con 0x7f0a201065b0 2024-03-22T10:50:24.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.520+0000 7f0a24c00700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201065b0 0x7f0a2019bef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400b480 tx=0x7f0a1400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:24.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.520+0000 7f0a1d600700 1 -- 172.21.15.115:0/31283833 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a140118c0 con 0x7f0a201065b0 2024-03-22T10:50:24.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.520+0000 7f0a1d600700 1 -- 172.21.15.115:0/31283833 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a14010380 con 0x7f0a201065b0 2024-03-22T10:50:24.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.520+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a201a1640 con 0x7f0a201065b0 2024-03-22T10:50:24.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.520+0000 7f0a1d600700 1 -- 172.21.15.115:0/31283833 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a1400f4f0 con 0x7f0a201065b0 2024-03-22T10:50:24.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.520+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a201a1c10 con 0x7f0a201065b0 2024-03-22T10:50:24.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.521+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2010f1a0 con 0x7f0a201065b0 2024-03-22T10:50:24.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.522+0000 7f0a1d600700 1 -- 172.21.15.115:0/31283833 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f0a140104f0 con 0x7f0a201065b0 2024-03-22T10:50:24.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.523+0000 7f0a1d600700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a0806dfa0 0x7f0a08070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:24.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.523+0000 7f0a1d600700 1 -- 172.21.15.115:0/31283833 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0a1408da40 con 0x7f0a201065b0 2024-03-22T10:50:24.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.524+0000 7f0a1fe00700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a0806dfa0 0x7f0a08070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:24.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.524+0000 7f0a1fe00700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a0806dfa0 0x7f0a08070450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c006010 tx=0x7f0a0c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:24.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.528+0000 7f0a1d600700 1 -- 172.21.15.115:0/31283833 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a1405a940 con 0x7f0a201065b0 2024-03-22T10:50:24.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:24 smithi150 ceph-mon[65513]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:24.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:24 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2182187867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:24.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:24 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/26591984' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:50:24.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:24 smithi115 ceph-mon[78672]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:24.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:24 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2182187867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:24.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:24 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/26591984' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:50:24.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.825+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0a2010f390 con 0x7f0a201065b0 2024-03-22T10:50:24.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.825+0000 7f0a1d600700 1 -- 172.21.15.115:0/31283833 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f0a1405a4d0 con 0x7f0a201065b0 2024-03-22T10:50:24.816 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-03-22T10:50:24.816 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-22T10:50:24.816 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs is degraded 2024-03-22T10:50:24.816 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:50:24.816 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a0806dfa0 msgr2=0x7f0a08070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a0806dfa0 0x7f0a08070450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c006010 tx=0x7f0a0c00b540 comp rx=0 tx=0).stop 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201065b0 msgr2=0x7f0a2019bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201065b0 0x7f0a2019bef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400b480 tx=0x7f0a1400b840 comp rx=0 tx=0).stop 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 shutdown_connections 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0a0806dfa0 0x7f0a08070450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a201065b0 0x7f0a2019bef0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 --2- 172.21.15.115:0/31283833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a201077b0 0x7f0a2019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 >> 172.21.15.115:0/31283833 conn(0x7f0a20075740 msgr2=0x7f0a2010a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.828+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 shutdown_connections 2024-03-22T10:50:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:24.829+0000 7f0a26000700 1 -- 172.21.15.115:0/31283833 wait complete. 2024-03-22T10:50:25.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:25 smithi115 ceph-mon[78672]: pgmap v225: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-03-22T10:50:25.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:25 smithi115 ceph-mon[78672]: reconnect by client.14516 192.168.0.1:0/629163332 after 0.00199996 2024-03-22T10:50:25.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:25 smithi115 ceph-mon[78672]: reconnect by client.24289 192.168.0.1:0/3857176443 after 0.00299994 2024-03-22T10:50:25.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:25 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:reconnect 2024-03-22T10:50:25.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:25 smithi115 ceph-mon[78672]: fsmap cephfs:1/1 {0=cephfs.smithi115.wjotmn=up:reconnect} 2 up:standby 2024-03-22T10:50:25.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:25 smithi115 ceph-mon[78672]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:25.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:25 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/31283833' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:50:25.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:25 smithi150 ceph-mon[65513]: pgmap v225: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-03-22T10:50:25.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:25 smithi150 ceph-mon[65513]: reconnect by client.14516 192.168.0.1:0/629163332 after 0.00199996 2024-03-22T10:50:25.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:25 smithi150 ceph-mon[65513]: reconnect by client.24289 192.168.0.1:0/3857176443 after 0.00299994 2024-03-22T10:50:25.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:25 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:reconnect 2024-03-22T10:50:25.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:25 smithi150 ceph-mon[65513]: fsmap cephfs:1/1 {0=cephfs.smithi115.wjotmn=up:reconnect} 2 up:standby 2024-03-22T10:50:25.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:25 smithi150 ceph-mon[65513]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:25.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:25 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/31283833' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:50:26.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:26 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:rejoin 2024-03-22T10:50:26.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:26 smithi115 ceph-mon[78672]: fsmap cephfs:1/1 {0=cephfs.smithi115.wjotmn=up:rejoin} 2 up:standby 2024-03-22T10:50:26.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:26 smithi115 ceph-mon[78672]: daemon mds.cephfs.smithi115.wjotmn is now active in filesystem cephfs as rank 0 2024-03-22T10:50:26.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:26 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:rejoin 2024-03-22T10:50:26.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:26 smithi150 ceph-mon[65513]: fsmap cephfs:1/1 {0=cephfs.smithi115.wjotmn=up:rejoin} 2 up:standby 2024-03-22T10:50:26.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:26 smithi150 ceph-mon[65513]: daemon mds.cephfs.smithi115.wjotmn is now active in filesystem cephfs as rank 0 2024-03-22T10:50:27.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:27 smithi115 ceph-mon[78672]: pgmap v226: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-22T10:50:27.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:27 smithi115 ceph-mon[78672]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T10:50:27.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:27 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:active 2024-03-22T10:50:27.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:27 smithi115 ceph-mon[78672]: fsmap cephfs:1 {0=cephfs.smithi115.wjotmn=up:active} 2 up:standby 2024-03-22T10:50:27.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:27 smithi150 ceph-mon[65513]: pgmap v226: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-22T10:50:27.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:27 smithi150 ceph-mon[65513]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T10:50:27.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:27 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.115:6828/4173107662,v1:172.21.15.115:6829/4173107662] up:active 2024-03-22T10:50:27.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:27 smithi150 ceph-mon[65513]: fsmap cephfs:1 {0=cephfs.smithi115.wjotmn=up:active} 2 up:standby 2024-03-22T10:50:29.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:29 smithi115 ceph-mon[78672]: pgmap v227: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 810 B/s wr, 8 op/s 2024-03-22T10:50:29.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:29 smithi150 ceph-mon[65513]: pgmap v227: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 810 B/s wr, 8 op/s 2024-03-22T10:50:31.753 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:31 smithi115 ceph-mon[78672]: pgmap v228: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 938 B/s wr, 9 op/s 2024-03-22T10:50:31.753 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:31 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] up:boot 2024-03-22T10:50:31.753 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:31 smithi115 ceph-mon[78672]: fsmap cephfs:1 {0=cephfs.smithi115.wjotmn=up:active} 3 up:standby 2024-03-22T10:50:31.753 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:50:31.754 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:31.754 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:31.754 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:31.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:31 smithi150 ceph-mon[65513]: pgmap v228: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 938 B/s wr, 9 op/s 2024-03-22T10:50:31.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:31 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] up:boot 2024-03-22T10:50:31.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:31 smithi150 ceph-mon[65513]: fsmap cephfs:1 {0=cephfs.smithi115.wjotmn=up:active} 3 up:standby 2024-03-22T10:50:31.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:50:31.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:31.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:31.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:33.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:33 smithi115 ceph-mon[78672]: pgmap v229: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-03-22T10:50:33.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:33 smithi150 ceph-mon[65513]: pgmap v229: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-03-22T10:50:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:35 smithi115 ceph-mon[78672]: pgmap v230: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-03-22T10:50:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:35.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:50:35.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:35 smithi150 ceph-mon[65513]: pgmap v230: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-03-22T10:50:35.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:35.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:50:37.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:37 smithi115 ceph-mon[78672]: pgmap v231: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 7.2 KiB/s wr, 6 op/s 2024-03-22T10:50:37.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:37 smithi150 ceph-mon[65513]: pgmap v231: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 7.2 KiB/s wr, 6 op/s 2024-03-22T10:50:38.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:38.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:39.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:39.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:39.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:39 smithi150 ceph-mon[65513]: pgmap v232: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 195 KiB/s rd, 7.2 KiB/s wr, 6 op/s 2024-03-22T10:50:40.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:39 smithi115 ceph-mon[78672]: pgmap v232: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 195 KiB/s rd, 7.2 KiB/s wr, 6 op/s 2024-03-22T10:50:41.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:41.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:41.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:41.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:41.887 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:41 smithi115 ceph-mon[78672]: pgmap v233: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 6.5 KiB/s wr, 3 op/s 2024-03-22T10:50:42.049 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:41 smithi150 ceph-mon[65513]: pgmap v233: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 6.5 KiB/s wr, 3 op/s 2024-03-22T10:50:43.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:43 smithi150 ceph-mon[65513]: pgmap v234: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s wr, 1 op/s 2024-03-22T10:50:44.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:43 smithi115 ceph-mon[78672]: pgmap v234: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s wr, 1 op/s 2024-03-22T10:50:46.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: pgmap v235: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:46.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:46.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi115.wjotmn"]}]: dispatch 2024-03-22T10:50:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: pgmap v235: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:50:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:50:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:46.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:46.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:46.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:46.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi115.wjotmn"]}]: dispatch 2024-03-22T10:50:46.796 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:46 smithi115 ceph-mon[78672]: Detected new or changed devices on smithi115 2024-03-22T10:50:46.796 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:46 smithi115 ceph-mon[78672]: Upgrade: It appears safe to stop mds.cephfs.smithi115.wjotmn 2024-03-22T10:50:47.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:46 smithi150 ceph-mon[65513]: Detected new or changed devices on smithi115 2024-03-22T10:50:47.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:46 smithi150 ceph-mon[65513]: Upgrade: It appears safe to stop mds.cephfs.smithi115.wjotmn 2024-03-22T10:50:48.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:47 smithi115 ceph-mon[78672]: pgmap v236: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:48.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:47 smithi150 ceph-mon[65513]: pgmap v236: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:49.085 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:48 smithi115 ceph-mon[78672]: Upgrade: Updating mds.cephfs.smithi115.wjotmn 2024-03-22T10:50:49.085 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:48 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:49.085 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:48 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.wjotmn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:50:49.085 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:48 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:49.085 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:48 smithi115 ceph-mon[78672]: Deploying daemon mds.cephfs.smithi115.wjotmn on smithi115 2024-03-22T10:50:49.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:48 smithi150 ceph-mon[65513]: Upgrade: Updating mds.cephfs.smithi115.wjotmn 2024-03-22T10:50:49.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:48 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:50:49.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:48 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.wjotmn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:50:49.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:48 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:50:49.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:48 smithi150 ceph-mon[65513]: Deploying daemon mds.cephfs.smithi115.wjotmn on smithi115 2024-03-22T10:50:50.259 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:49 smithi115 ceph-mon[78672]: pgmap v237: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:50.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:50:50.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:49 smithi150 ceph-mon[65513]: pgmap v237: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:50.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:50:50.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:50 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[78668]: 2024-03-22T10:50:50.659+0000 7f7b87c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:50:51.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:50 smithi115 ceph-mon[78672]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T10:50:51.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:50 smithi115 ceph-mon[78672]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:50:51.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:50 smithi115 ceph-mon[78672]: osdmap e81: 6 total, 6 up, 6 in 2024-03-22T10:50:51.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:50 smithi115 ceph-mon[78672]: Standby daemon mds.cephfs.smithi150.slzwwb assigned to filesystem cephfs as rank 0 2024-03-22T10:50:51.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:50 smithi115 ceph-mon[78672]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T10:50:51.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:50 smithi115 ceph-mon[78672]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T10:50:51.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:50 smithi115 ceph-mon[78672]: fsmap cephfs:1/1 {0=cephfs.smithi150.slzwwb=up:replay} 2 up:standby 2024-03-22T10:50:51.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:50 smithi150 ceph-mon[65513]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T10:50:51.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:50 smithi150 ceph-mon[65513]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:50:51.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:50 smithi150 ceph-mon[65513]: osdmap e81: 6 total, 6 up, 6 in 2024-03-22T10:50:51.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:50 smithi150 ceph-mon[65513]: Standby daemon mds.cephfs.smithi150.slzwwb assigned to filesystem cephfs as rank 0 2024-03-22T10:50:51.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:50 smithi150 ceph-mon[65513]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T10:50:51.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:50 smithi150 ceph-mon[65513]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T10:50:51.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:50 smithi150 ceph-mon[65513]: fsmap cephfs:1/1 {0=cephfs.smithi150.slzwwb=up:replay} 2 up:standby 2024-03-22T10:50:52.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:51 smithi115 ceph-mon[78672]: pgmap v238: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:52.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:51 smithi150 ceph-mon[65513]: pgmap v238: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:50:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:53 smithi115 ceph-mon[78672]: pgmap v240: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 410 KiB/s rd, 0 op/s 2024-03-22T10:50:54.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:53 smithi150 ceph-mon[65513]: pgmap v240: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 410 KiB/s rd, 0 op/s 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:54.999+0000 7fa3aa000700 1 -- 172.21.15.115:0/1461427588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41079a0 msgr2=0x7fa3a4107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:54.999+0000 7fa3aa000700 1 --2- 172.21.15.115:0/1461427588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41079a0 0x7fa3a4107df0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa390009910 tx=0x7fa390009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:54.999+0000 7fa3aa000700 1 -- 172.21.15.115:0/1461427588 shutdown_connections 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:54.999+0000 7fa3aa000700 1 --2- 172.21.15.115:0/1461427588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41079a0 0x7fa3a4107df0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:54.999+0000 7fa3aa000700 1 --2- 172.21.15.115:0/1461427588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3a41067a0 0x7fa3a4106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:54.999+0000 7fa3aa000700 1 -- 172.21.15.115:0/1461427588 >> 172.21.15.115:0/1461427588 conn(0x7fa3a4075910 msgr2=0x7fa3a4077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:54.999+0000 7fa3aa000700 1 -- 172.21.15.115:0/1461427588 shutdown_connections 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:54.999+0000 7fa3aa000700 1 -- 172.21.15.115:0/1461427588 wait complete. 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.000+0000 7fa3aa000700 1 Processor -- start 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.000+0000 7fa3aa000700 1 -- start start 2024-03-22T10:50:54.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.000+0000 7fa3aa000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41067a0 0x7fa3a419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.000+0000 7fa3aa000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3a41079a0 0x7fa3a419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.000+0000 7fa3aa000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a419cb00 con 0x7fa3a41067a0 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.000+0000 7fa3aa000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a419cc40 con 0x7fa3a41079a0 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.001+0000 7fa3a3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41067a0 0x7fa3a419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.001+0000 7fa3a3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41067a0 0x7fa3a419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53284/0 (socket says 172.21.15.115:53284) 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.001+0000 7fa3a3400700 1 -- 172.21.15.115:0/4261122169 learned_addr learned my addr 172.21.15.115:0/4261122169 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.001+0000 7fa3a2a00700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3a41079a0 0x7fa3a419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.001+0000 7fa3a3400700 1 -- 172.21.15.115:0/4261122169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3a41079a0 msgr2=0x7fa3a419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.001+0000 7fa3a3400700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3a41079a0 0x7fa3a419c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.001+0000 7fa3a3400700 1 -- 172.21.15.115:0/4261122169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900095f0 con 0x7fa3a41067a0 2024-03-22T10:50:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.001+0000 7fa3a3400700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41067a0 0x7fa3a419bfa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d900 tx=0x7fa39400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:54.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.002+0000 7fa39be00700 1 -- 172.21.15.115:0/4261122169 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3940041f0 con 0x7fa3a41067a0 2024-03-22T10:50:54.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.002+0000 7fa39be00700 1 -- 172.21.15.115:0/4261122169 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa394004350 con 0x7fa3a41067a0 2024-03-22T10:50:54.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.002+0000 7fa39be00700 1 -- 172.21.15.115:0/4261122169 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa394003d20 con 0x7fa3a41067a0 2024-03-22T10:50:54.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.002+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a41a16f0 con 0x7fa3a41067a0 2024-03-22T10:50:54.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.002+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a41a1c40 con 0x7fa3a41067a0 2024-03-22T10:50:54.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.003+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a410f2f0 con 0x7fa3a41067a0 2024-03-22T10:50:54.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.004+0000 7fa39be00700 1 -- 172.21.15.115:0/4261122169 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa394009730 con 0x7fa3a41067a0 2024-03-22T10:50:54.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.005+0000 7fa39be00700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa38806dfa0 0x7fa388070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:54.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.005+0000 7fa39be00700 1 -- 172.21.15.115:0/4261122169 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa39408dc80 con 0x7fa3a41067a0 2024-03-22T10:50:54.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.005+0000 7fa3a2a00700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa38806dfa0 0x7fa388070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:54.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.006+0000 7fa3a2a00700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa38806dfa0 0x7fa388070450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa390006010 tx=0x7fa39000b560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:55.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.010+0000 7fa39be00700 1 -- 172.21.15.115:0/4261122169 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa39405abd0 con 0x7fa3a41067a0 2024-03-22T10:50:55.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.264+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3a410c2f0 con 0x7fa38806dfa0 2024-03-22T10:50:55.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.266+0000 7fa39be00700 1 -- 172.21.15.115:0/4261122169 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa3a410c2f0 con 0x7fa38806dfa0 2024-03-22T10:50:55.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.269+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa38806dfa0 msgr2=0x7fa388070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:55.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.269+0000 7fa3aa000700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa38806dfa0 0x7fa388070450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa390006010 tx=0x7fa39000b560 comp rx=0 tx=0).stop 2024-03-22T10:50:55.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.269+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41067a0 msgr2=0x7fa3a419bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:55.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.269+0000 7fa3aa000700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41067a0 0x7fa3a419bfa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d900 tx=0x7fa39400dc10 comp rx=0 tx=0).stop 2024-03-22T10:50:55.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.269+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 shutdown_connections 2024-03-22T10:50:55.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.269+0000 7fa3aa000700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa38806dfa0 0x7fa388070450 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.269+0000 7fa3aa000700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a41067a0 0x7fa3a419bfa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.269+0000 7fa3aa000700 1 --2- 172.21.15.115:0/4261122169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3a41079a0 0x7fa3a419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.269+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 >> 172.21.15.115:0/4261122169 conn(0x7fa3a4075910 msgr2=0x7fa3a410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:55.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.270+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 shutdown_connections 2024-03-22T10:50:55.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.270+0000 7fa3aa000700 1 -- 172.21.15.115:0/4261122169 wait complete. 2024-03-22T10:50:55.277 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:50:55.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.435+0000 7fd1ed200700 1 -- 172.21.15.115:0/1414876768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 msgr2=0x7fd1e8100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:55.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.435+0000 7fd1ed200700 1 --2- 172.21.15.115:0/1414876768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 0x7fd1e8100590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009920 tx=0x7fd1d8009c30 comp rx=0 tx=0).stop 2024-03-22T10:50:55.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.436+0000 7fd1ed200700 1 -- 172.21.15.115:0/1414876768 shutdown_connections 2024-03-22T10:50:55.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.436+0000 7fd1ed200700 1 --2- 172.21.15.115:0/1414876768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e8101380 0x7fd1e81017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.436+0000 7fd1ed200700 1 --2- 172.21.15.115:0/1414876768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 0x7fd1e8100590 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.436+0000 7fd1ed200700 1 -- 172.21.15.115:0/1414876768 >> 172.21.15.115:0/1414876768 conn(0x7fd1e80fb710 msgr2=0x7fd1e80fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:55.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.436+0000 7fd1ed200700 1 -- 172.21.15.115:0/1414876768 shutdown_connections 2024-03-22T10:50:55.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.436+0000 7fd1ed200700 1 -- 172.21.15.115:0/1414876768 wait complete. 2024-03-22T10:50:55.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.437+0000 7fd1ed200700 1 Processor -- start 2024-03-22T10:50:55.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.437+0000 7fd1ed200700 1 -- start start 2024-03-22T10:50:55.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.437+0000 7fd1ed200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 0x7fd1e81959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:55.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.437+0000 7fd1ed200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e8101380 0x7fd1e8195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:55.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.437+0000 7fd1ed200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e8196500 con 0x7fd1e8100180 2024-03-22T10:50:55.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.437+0000 7fd1ed200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e8196640 con 0x7fd1e8101380 2024-03-22T10:50:55.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.437+0000 7fd1e5e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 0x7fd1e81959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:55.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.438+0000 7fd1e5e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 0x7fd1e81959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53304/0 (socket says 172.21.15.115:53304) 2024-03-22T10:50:55.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.438+0000 7fd1e5e00700 1 -- 172.21.15.115:0/3473072372 learned_addr learned my addr 172.21.15.115:0/3473072372 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:55.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.438+0000 7fd1e5400700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e8101380 0x7fd1e8195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:55.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.438+0000 7fd1e5e00700 1 -- 172.21.15.115:0/3473072372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e8101380 msgr2=0x7fd1e8195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:55.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.438+0000 7fd1e5e00700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e8101380 0x7fd1e8195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.438+0000 7fd1e5e00700 1 -- 172.21.15.115:0/3473072372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d8009580 con 0x7fd1e8100180 2024-03-22T10:50:55.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.438+0000 7fd1e5e00700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 0x7fd1e81959a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009550 tx=0x7fd1d8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:55.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.439+0000 7fd1dea00700 1 -- 172.21.15.115:0/3473072372 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1d801d070 con 0x7fd1e8100180 2024-03-22T10:50:55.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.439+0000 7fd1dea00700 1 -- 172.21.15.115:0/3473072372 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd1d800be30 con 0x7fd1e8100180 2024-03-22T10:50:55.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.439+0000 7fd1dea00700 1 -- 172.21.15.115:0/3473072372 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1d800f460 con 0x7fd1e8100180 2024-03-22T10:50:55.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.439+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e819b090 con 0x7fd1e8100180 2024-03-22T10:50:55.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.439+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e819b600 con 0x7fd1e8100180 2024-03-22T10:50:55.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.440+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e8108cd0 con 0x7fd1e8100180 2024-03-22T10:50:55.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.441+0000 7fd1dea00700 1 -- 172.21.15.115:0/3473072372 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fd1d8022820 con 0x7fd1e8100180 2024-03-22T10:50:55.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.442+0000 7fd1dea00700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd1cc06dfa0 0x7fd1cc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:55.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.442+0000 7fd1dea00700 1 -- 172.21.15.115:0/3473072372 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd1d808f100 con 0x7fd1e8100180 2024-03-22T10:50:55.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.442+0000 7fd1e5400700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd1cc06dfa0 0x7fd1cc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:55.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.443+0000 7fd1e5400700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd1cc06dfa0 0x7fd1cc070450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009c60 tx=0x7fd1d4009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:55.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.447+0000 7fd1dea00700 1 -- 172.21.15.115:0/3473072372 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd1d805c0e0 con 0x7fd1e8100180 2024-03-22T10:50:55.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.691+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1e8105cd0 con 0x7fd1cc06dfa0 2024-03-22T10:50:55.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.694+0000 7fd1dea00700 1 -- 172.21.15.115:0/3473072372 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd1e8105cd0 con 0x7fd1cc06dfa0 2024-03-22T10:50:55.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd1cc06dfa0 msgr2=0x7fd1cc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:55.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd1cc06dfa0 0x7fd1cc070450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009c60 tx=0x7fd1d4009260 comp rx=0 tx=0).stop 2024-03-22T10:50:55.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 msgr2=0x7fd1e81959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:55.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 0x7fd1e81959a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009550 tx=0x7fd1d8004af0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 shutdown_connections 2024-03-22T10:50:55.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd1cc06dfa0 0x7fd1cc070450 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1e8100180 0x7fd1e81959a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 --2- 172.21.15.115:0/3473072372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1e8101380 0x7fd1e8195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 >> 172.21.15.115:0/3473072372 conn(0x7fd1e80fb710 msgr2=0x7fd1e81045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:55.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 shutdown_connections 2024-03-22T10:50:55.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.697+0000 7fd1ed200700 1 -- 172.21.15.115:0/3473072372 wait complete. 2024-03-22T10:50:55.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.855+0000 7fcf89800700 1 -- 172.21.15.115:0/49404332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841067a0 msgr2=0x7fcf84106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:55.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.855+0000 7fcf89800700 1 --2- 172.21.15.115:0/49404332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841067a0 0x7fcf84106bb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009910 tx=0x7fcf74009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:55.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.855+0000 7fcf89800700 1 -- 172.21.15.115:0/49404332 shutdown_connections 2024-03-22T10:50:55.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.855+0000 7fcf89800700 1 --2- 172.21.15.115:0/49404332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf841079a0 0x7fcf84107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.855+0000 7fcf89800700 1 --2- 172.21.15.115:0/49404332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841067a0 0x7fcf84106bb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.855+0000 7fcf89800700 1 -- 172.21.15.115:0/49404332 >> 172.21.15.115:0/49404332 conn(0x7fcf84075910 msgr2=0x7fcf84077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:55.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.856+0000 7fcf89800700 1 -- 172.21.15.115:0/49404332 shutdown_connections 2024-03-22T10:50:55.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.856+0000 7fcf89800700 1 -- 172.21.15.115:0/49404332 wait complete. 2024-03-22T10:50:55.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.857+0000 7fcf89800700 1 Processor -- start 2024-03-22T10:50:55.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.857+0000 7fcf89800700 1 -- start start 2024-03-22T10:50:55.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.857+0000 7fcf89800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf841067a0 0x7fcf8419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:55.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.857+0000 7fcf89800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841079a0 0x7fcf8419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:55.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.857+0000 7fcf89800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8419cb80 con 0x7fcf841079a0 2024-03-22T10:50:55.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.857+0000 7fcf89800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8419ccc0 con 0x7fcf841067a0 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.857+0000 7fcf82000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841079a0 0x7fcf8419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.858+0000 7fcf82000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841079a0 0x7fcf8419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53328/0 (socket says 172.21.15.115:53328) 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.858+0000 7fcf82000700 1 -- 172.21.15.115:0/1476351527 learned_addr learned my addr 172.21.15.115:0/1476351527 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.858+0000 7fcf82a00700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf841067a0 0x7fcf8419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.858+0000 7fcf82000700 1 -- 172.21.15.115:0/1476351527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf841067a0 msgr2=0x7fcf8419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.858+0000 7fcf82000700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf841067a0 0x7fcf8419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.858+0000 7fcf82000700 1 -- 172.21.15.115:0/1476351527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740095f0 con 0x7fcf841079a0 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.858+0000 7fcf82000700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841079a0 0x7fcf8419c560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000b4b0 tx=0x7fcf7000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.858+0000 7fcf7b400700 1 -- 172.21.15.115:0/1476351527 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf70011800 con 0x7fcf841079a0 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.858+0000 7fcf7b400700 1 -- 172.21.15.115:0/1476351527 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcf70011e10 con 0x7fcf841079a0 2024-03-22T10:50:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.859+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf841a1770 con 0x7fcf841079a0 2024-03-22T10:50:55.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.859+0000 7fcf7b400700 1 -- 172.21.15.115:0/1476351527 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf7000f2e0 con 0x7fcf841079a0 2024-03-22T10:50:55.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.859+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf841a1d40 con 0x7fcf841079a0 2024-03-22T10:50:55.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.860+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8410f370 con 0x7fcf841079a0 2024-03-22T10:50:55.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.861+0000 7fcf7b400700 1 -- 172.21.15.115:0/1476351527 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fcf70010380 con 0x7fcf841079a0 2024-03-22T10:50:55.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.862+0000 7fcf7b400700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcf6806dfa0 0x7fcf68070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:55.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.862+0000 7fcf7b400700 1 -- 172.21.15.115:0/1476351527 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcf7008da10 con 0x7fcf841079a0 2024-03-22T10:50:55.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.862+0000 7fcf82a00700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcf6806dfa0 0x7fcf68070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:55.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.863+0000 7fcf82a00700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcf6806dfa0 0x7fcf68070450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006010 tx=0x7fcf7400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:55.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:55.866+0000 7fcf7b400700 1 -- 172.21.15.115:0/1476351527 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcf7005a940 con 0x7fcf841079a0 2024-03-22T10:50:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.101+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf8410c2f0 con 0x7fcf6806dfa0 2024-03-22T10:50:56.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.132+0000 7fcf7b400700 1 -- 172.21.15.115:0/1476351527 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcf8410c2f0 con 0x7fcf6806dfa0 2024-03-22T10:50:56.124 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:50:56.124 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (17m) 18s ago 19m 14.7M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:50:56.124 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (19m) 18s ago 19m 9286k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:50:56.124 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (18m) 94s ago 18m 8644k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (5m) 18s ago 19m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (5m) 94s ago 18m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (17m) 18s ago 18m 72.8M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (26s) 18s ago 12m 10.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 44a0769ebe8e 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (12m) 18s ago 12m 70.3M - 18.2.0 dc2bc1663786 424a5bb97c80 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (12m) 94s ago 12m 17.0M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (12m) 94s ago 12m 126M - 18.2.0 dc2bc1663786 b1abec81e45c 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (8m) 18s ago 21m 511M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (7m) 94s ago 17m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (7m) 18s ago 21m 52.9M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (6m) 94s ago 17m 38.8M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (19m) 18s ago 19m 8308k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (18m) 94s ago 18m 8212k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (4m) 18s ago 16m 96.3M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (3m) 18s ago 16m 92.8M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:50:56.125 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (3m) 18s ago 15m 81.3M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:50:56.126 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (3m) 94s ago 15m 87.8M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:50:56.126 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (2m) 94s ago 14m 84.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:50:56.126 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (99s) 94s ago 14m 11.1M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:50:56.126 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (7m) 18s ago 18m 74.2M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.135+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcf6806dfa0 msgr2=0x7fcf68070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.135+0000 7fcf89800700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcf6806dfa0 0x7fcf68070450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006010 tx=0x7fcf7400b540 comp rx=0 tx=0).stop 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.135+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841079a0 msgr2=0x7fcf8419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.135+0000 7fcf89800700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841079a0 0x7fcf8419c560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000b4b0 tx=0x7fcf7000b870 comp rx=0 tx=0).stop 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.135+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 shutdown_connections 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.135+0000 7fcf89800700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcf6806dfa0 0x7fcf68070450 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.135+0000 7fcf89800700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf841067a0 0x7fcf8419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.135+0000 7fcf89800700 1 --2- 172.21.15.115:0/1476351527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf841079a0 0x7fcf8419c560 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.135+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 >> 172.21.15.115:0/1476351527 conn(0x7fcf84075910 msgr2=0x7fcf8410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:56.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.136+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 shutdown_connections 2024-03-22T10:50:56.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.136+0000 7fcf89800700 1 -- 172.21.15.115:0/1476351527 wait complete. 2024-03-22T10:50:56.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:55 smithi115 ceph-mon[78672]: pgmap v241: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 2 op/s 2024-03-22T10:50:56.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.293+0000 7f06cac00700 1 -- 172.21.15.115:0/3021622695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 msgr2=0x7f06c4103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:56.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.293+0000 7f06cac00700 1 --2- 172.21.15.115:0/3021622695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 0x7f06c4103800 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009910 tx=0x7f06b0009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:56.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.293+0000 7f06cac00700 1 -- 172.21.15.115:0/3021622695 shutdown_connections 2024-03-22T10:50:56.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.293+0000 7f06cac00700 1 --2- 172.21.15.115:0/3021622695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 0x7f06c4103800 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.293+0000 7f06cac00700 1 --2- 172.21.15.115:0/3021622695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c41021b0 0x7f06c41025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.293+0000 7f06cac00700 1 -- 172.21.15.115:0/3021622695 >> 172.21.15.115:0/3021622695 conn(0x7f06c40fd760 msgr2=0x7f06c40ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:56.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.293+0000 7f06cac00700 1 -- 172.21.15.115:0/3021622695 shutdown_connections 2024-03-22T10:50:56.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.293+0000 7f06cac00700 1 -- 172.21.15.115:0/3021622695 wait complete. 2024-03-22T10:50:56.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.294+0000 7f06cac00700 1 Processor -- start 2024-03-22T10:50:56.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.295+0000 7f06cac00700 1 -- start start 2024-03-22T10:50:56.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.295+0000 7f06cac00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c41021b0 0x7f06c41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:56.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.295+0000 7f06cac00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 0x7f06c4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:56.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.295+0000 7f06cac00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c4198480 con 0x7f06c41033b0 2024-03-22T10:50:56.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.295+0000 7f06cac00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c41985c0 con 0x7f06c41021b0 2024-03-22T10:50:56.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.295+0000 7f06c8e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 0x7f06c4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:56.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.295+0000 7f06c8e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 0x7f06c4197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53348/0 (socket says 172.21.15.115:53348) 2024-03-22T10:50:56.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.295+0000 7f06c8e00700 1 -- 172.21.15.115:0/1445728477 learned_addr learned my addr 172.21.15.115:0/1445728477 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:56.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.295+0000 7f06c9800700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c41021b0 0x7f06c41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:56.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.296+0000 7f06c8e00700 1 -- 172.21.15.115:0/1445728477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c41021b0 msgr2=0x7f06c41979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:56.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.296+0000 7f06c8e00700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c41021b0 0x7f06c41979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.296+0000 7f06c8e00700 1 -- 172.21.15.115:0/1445728477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b00095f0 con 0x7f06c41033b0 2024-03-22T10:50:56.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.296+0000 7f06c8e00700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 0x7f06c4197ef0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b5c0 tx=0x7f06b000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:56.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.296+0000 7f06c2000700 1 -- 172.21.15.115:0/1445728477 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b001d070 con 0x7f06c41033b0 2024-03-22T10:50:56.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.296+0000 7f06c2000700 1 -- 172.21.15.115:0/1445728477 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f06b0022470 con 0x7f06c41033b0 2024-03-22T10:50:56.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.296+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c419d020 con 0x7f06c41033b0 2024-03-22T10:50:56.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.297+0000 7f06c2000700 1 -- 172.21.15.115:0/1445728477 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b000f690 con 0x7f06c41033b0 2024-03-22T10:50:56.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.297+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c419d4e0 con 0x7f06c41033b0 2024-03-22T10:50:56.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.298+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c410ad00 con 0x7f06c41033b0 2024-03-22T10:50:56.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.299+0000 7f06c2000700 1 -- 172.21.15.115:0/1445728477 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f06b00225e0 con 0x7f06c41033b0 2024-03-22T10:50:56.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.300+0000 7f06c2000700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f06ac06dfa0 0x7f06ac070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:56.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.300+0000 7f06c2000700 1 -- 172.21.15.115:0/1445728477 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f06b008f590 con 0x7f06c41033b0 2024-03-22T10:50:56.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.300+0000 7f06c9800700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f06ac06dfa0 0x7f06ac070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:56.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.301+0000 7f06c9800700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f06ac06dfa0 0x7f06ac070450 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f06c4103210 tx=0x7f06b800a3b0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:56.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.304+0000 7f06c2000700 1 -- 172.21.15.115:0/1445728477 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f06b005c4e0 con 0x7f06c41033b0 2024-03-22T10:50:56.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:55 smithi150 ceph-mon[65513]: pgmap v241: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 2 op/s 2024-03-22T10:50:56.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.617+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f06c419dbe0 con 0x7f06c41033b0 2024-03-22T10:50:56.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.618+0000 7f06c2000700 1 -- 172.21.15.115:0/1445728477 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f06b005c070 con 0x7f06c41033b0 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:50:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-03-22T10:50:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-22T10:50:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:50:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:50:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-03-22T10:50:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 11 2024-03-22T10:50:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:50:56.611 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:50:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f06ac06dfa0 msgr2=0x7f06ac070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f06ac06dfa0 0x7f06ac070450 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f06c4103210 tx=0x7f06b800a3b0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 msgr2=0x7f06c4197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 0x7f06c4197ef0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b5c0 tx=0x7f06b000bd70 comp rx=0 tx=0).stop 2024-03-22T10:50:56.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 shutdown_connections 2024-03-22T10:50:56.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f06ac06dfa0 0x7f06ac070450 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c41021b0 0x7f06c41979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 --2- 172.21.15.115:0/1445728477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06c41033b0 0x7f06c4197ef0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 >> 172.21.15.115:0/1445728477 conn(0x7f06c40fd760 msgr2=0x7f06c41065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:56.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.622+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 shutdown_connections 2024-03-22T10:50:56.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.623+0000 7f06cac00700 1 -- 172.21.15.115:0/1445728477 wait complete. 2024-03-22T10:50:56.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.771+0000 7f32c2400700 1 -- 172.21.15.115:0/982586089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 msgr2=0x7f32bc073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:56.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.771+0000 7f32c2400700 1 --2- 172.21.15.115:0/982586089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 0x7f32bc073c20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009940 tx=0x7f32a8009c50 comp rx=0 tx=0).stop 2024-03-22T10:50:56.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.771+0000 7f32c2400700 1 -- 172.21.15.115:0/982586089 shutdown_connections 2024-03-22T10:50:56.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.771+0000 7f32c2400700 1 --2- 172.21.15.115:0/982586089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 0x7f32bc073c20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.771+0000 7f32c2400700 1 --2- 172.21.15.115:0/982586089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32bc074d80 0x7f32bc0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.772+0000 7f32c2400700 1 -- 172.21.15.115:0/982586089 >> 172.21.15.115:0/982586089 conn(0x7f32bc0fb500 msgr2=0x7f32bc0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:56.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.772+0000 7f32c2400700 1 -- 172.21.15.115:0/982586089 shutdown_connections 2024-03-22T10:50:56.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.772+0000 7f32c2400700 1 -- 172.21.15.115:0/982586089 wait complete. 2024-03-22T10:50:56.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.773+0000 7f32c2400700 1 Processor -- start 2024-03-22T10:50:56.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.773+0000 7f32c2400700 1 -- start start 2024-03-22T10:50:56.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.773+0000 7f32c2400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 0x7f32bc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:56.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.773+0000 7f32c2400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32bc074d80 0x7f32bc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:56.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.773+0000 7f32c2400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc198600 con 0x7f32bc0737b0 2024-03-22T10:50:56.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.773+0000 7f32c2400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc198740 con 0x7f32bc074d80 2024-03-22T10:50:56.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.773+0000 7f32c1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 0x7f32bc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:56.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.773+0000 7f32c1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 0x7f32bc197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53376/0 (socket says 172.21.15.115:53376) 2024-03-22T10:50:56.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.773+0000 7f32c1000700 1 -- 172.21.15.115:0/3928973078 learned_addr learned my addr 172.21.15.115:0/3928973078 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:56.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.774+0000 7f32c1000700 1 -- 172.21.15.115:0/3928973078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32bc074d80 msgr2=0x7f32bc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:56.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.774+0000 7f32c1000700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32bc074d80 0x7f32bc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:56.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.774+0000 7f32c1000700 1 -- 172.21.15.115:0/3928973078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a80095f0 con 0x7f32bc0737b0 2024-03-22T10:50:56.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.774+0000 7f32c1000700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 0x7f32bc197aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b4b0 tx=0x7f32ac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:56.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.774+0000 7f32ba000700 1 -- 172.21.15.115:0/3928973078 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32ac011800 con 0x7f32bc0737b0 2024-03-22T10:50:56.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.774+0000 7f32ba000700 1 -- 172.21.15.115:0/3928973078 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32ac011e10 con 0x7f32bc0737b0 2024-03-22T10:50:56.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.774+0000 7f32ba000700 1 -- 172.21.15.115:0/3928973078 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32ac00f2e0 con 0x7f32bc0737b0 2024-03-22T10:50:56.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.774+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc19d1f0 con 0x7f32bc0737b0 2024-03-22T10:50:56.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.775+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc19d7c0 con 0x7f32bc0737b0 2024-03-22T10:50:56.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.776+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc10ad50 con 0x7f32bc0737b0 2024-03-22T10:50:56.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.777+0000 7f32ba000700 1 -- 172.21.15.115:0/3928973078 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f32ac010380 con 0x7f32bc0737b0 2024-03-22T10:50:56.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.777+0000 7f32ba000700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f32a006dfa0 0x7f32a0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:56.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.778+0000 7f32ba000700 1 -- 172.21.15.115:0/3928973078 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f32ac08da10 con 0x7f32bc0737b0 2024-03-22T10:50:56.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.778+0000 7f32b3e00700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f32a006dfa0 0x7f32a0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:56.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.778+0000 7f32b3e00700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f32a006dfa0 0x7f32a0070450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006010 tx=0x7f32a8000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:56.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:56.782+0000 7f32ba000700 1 -- 172.21.15.115:0/3928973078 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f32ac05a940 con 0x7f32bc0737b0 2024-03-22T10:50:57.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.108+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f32bc19e2b0 con 0x7f32bc0737b0 2024-03-22T10:50:57.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.109+0000 7f32ba000700 1 -- 172.21.15.115:0/3928973078 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1713 (secure 0 0 0) 0x7f32ac05a4d0 con 0x7f32bc0737b0 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:e29 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 29 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:50:56.374327+0000 2024-03-22T10:50:57.101 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 81 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 1 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:in 0 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=24255} 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:50:57.102 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 1 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 0 members: 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{0:24255} state up:reconnect seq 184 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{-1:34476} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:50:57.103 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{-1:44271} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6824/2616517800,v1:172.21.15.150:6825/2616517800] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:50:57.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f32a006dfa0 msgr2=0x7f32a0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f32a006dfa0 0x7f32a0070450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006010 tx=0x7f32a8000bc0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 msgr2=0x7f32bc197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 0x7f32bc197aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b4b0 tx=0x7f32ac00b870 comp rx=0 tx=0).stop 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 shutdown_connections 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f32a006dfa0 0x7f32a0070450 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32bc0737b0 0x7f32bc197aa0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 --2- 172.21.15.115:0/3928973078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32bc074d80 0x7f32bc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 >> 172.21.15.115:0/3928973078 conn(0x7f32bc0fb500 msgr2=0x7f32bc105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 shutdown_connections 2024-03-22T10:50:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.113+0000 7f32c2400700 1 -- 172.21.15.115:0/3928973078 wait complete. 2024-03-22T10:50:57.106 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 29 2024-03-22T10:50:57.151 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:56 smithi115 ceph-mon[78672]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:57.151 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:56 smithi115 ceph-mon[78672]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:57.151 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:56 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1445728477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.265+0000 7fa95b200700 1 -- 172.21.15.115:0/1619333311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 msgr2=0x7fa95c1025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.265+0000 7fa95b200700 1 --2- 172.21.15.115:0/1619333311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 0x7fa95c1025f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009910 tx=0x7fa94c009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.265+0000 7fa95b200700 1 -- 172.21.15.115:0/1619333311 shutdown_connections 2024-03-22T10:50:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.265+0000 7fa95b200700 1 --2- 172.21.15.115:0/1619333311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c1033e0 0x7fa95c103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.265+0000 7fa95b200700 1 --2- 172.21.15.115:0/1619333311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 0x7fa95c1025f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.265+0000 7fa95b200700 1 -- 172.21.15.115:0/1619333311 >> 172.21.15.115:0/1619333311 conn(0x7fa95c0fd8d0 msgr2=0x7fa95c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.265+0000 7fa95b200700 1 -- 172.21.15.115:0/1619333311 shutdown_connections 2024-03-22T10:50:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.265+0000 7fa95b200700 1 -- 172.21.15.115:0/1619333311 wait complete. 2024-03-22T10:50:57.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.266+0000 7fa95b200700 1 Processor -- start 2024-03-22T10:50:57.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa95b200700 1 -- start start 2024-03-22T10:50:57.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa95b200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 0x7fa95c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:57.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa95b200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c1033e0 0x7fa95c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:57.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa95b200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c198590 con 0x7fa95c1021e0 2024-03-22T10:50:57.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa95b200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1986d0 con 0x7fa95c1033e0 2024-03-22T10:50:57.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa959e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 0x7fa95c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:57.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa959e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 0x7fa95c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53400/0 (socket says 172.21.15.115:53400) 2024-03-22T10:50:57.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa959e00700 1 -- 172.21.15.115:0/3625154916 learned_addr learned my addr 172.21.15.115:0/3625154916 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa959e00700 1 -- 172.21.15.115:0/3625154916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c1033e0 msgr2=0x7fa95c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa959e00700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c1033e0 0x7fa95c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.267+0000 7fa959e00700 1 -- 172.21.15.115:0/3625154916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0095f0 con 0x7fa95c1021e0 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.268+0000 7fa959e00700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 0x7fa95c197a30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bd70 tx=0x7fa94c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.268+0000 7fa952a00700 1 -- 172.21.15.115:0/3625154916 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94c01d070 con 0x7fa95c1021e0 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.268+0000 7fa952a00700 1 -- 172.21.15.115:0/3625154916 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa94c00f460 con 0x7fa95c1021e0 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.268+0000 7fa952a00700 1 -- 172.21.15.115:0/3625154916 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94c021640 con 0x7fa95c1021e0 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.268+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c19d120 con 0x7fa95c1021e0 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.268+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c19d690 con 0x7fa95c1021e0 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.269+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c10adb0 con 0x7fa95c1021e0 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.270+0000 7fa952a00700 1 -- 172.21.15.115:0/3625154916 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa94c00fab0 con 0x7fa95c1021e0 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.271+0000 7fa952a00700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa94006dfa0 0x7fa940070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.271+0000 7fa952a00700 1 -- 172.21.15.115:0/3625154916 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa94c08f220 con 0x7fa95c1021e0 2024-03-22T10:50:57.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.272+0000 7fa959400700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa94006dfa0 0x7fa940070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:57.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.272+0000 7fa959400700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa94006dfa0 0x7fa940070450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9480097b0 tx=0x7fa948006d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:57.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.276+0000 7fa952a00700 1 -- 172.21.15.115:0/3625154916 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa94c05c150 con 0x7fa95c1021e0 2024-03-22T10:50:57.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:56 smithi150 ceph-mon[65513]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:57.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:56 smithi150 ceph-mon[65513]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:57.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:56 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1445728477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:50:57.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.544+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa95c107d30 con 0x7fa94006dfa0 2024-03-22T10:50:57.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.546+0000 7fa952a00700 1 -- 172.21.15.115:0/3625154916 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa95c107d30 con 0x7fa94006dfa0 2024-03-22T10:50:57.538 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:50:57.538 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:50:57.538 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:50:57.538 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:50:57.538 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:50:57.538 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:50:57.538 INFO:teuthology.orchestra.run.smithi115.stdout: "osd", 2024-03-22T10:50:57.538 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:50:57.538 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:50:57.539 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:50:57.539 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "13/23 daemons upgraded", 2024-03-22T10:50:57.539 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T10:50:57.539 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:50:57.539 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:50:57.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.549+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa94006dfa0 msgr2=0x7fa940070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:57.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.549+0000 7fa95b200700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa94006dfa0 0x7fa940070450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9480097b0 tx=0x7fa948006d20 comp rx=0 tx=0).stop 2024-03-22T10:50:57.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.549+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 msgr2=0x7fa95c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:57.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.549+0000 7fa95b200700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 0x7fa95c197a30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bd70 tx=0x7fa94c00be50 comp rx=0 tx=0).stop 2024-03-22T10:50:57.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.549+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 shutdown_connections 2024-03-22T10:50:57.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.549+0000 7fa95b200700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa94006dfa0 0x7fa940070450 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.549+0000 7fa95b200700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95c1021e0 0x7fa95c197a30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.549+0000 7fa95b200700 1 --2- 172.21.15.115:0/3625154916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c1033e0 0x7fa95c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.549+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 >> 172.21.15.115:0/3625154916 conn(0x7fa95c0fd8d0 msgr2=0x7fa95c106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:57.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.550+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 shutdown_connections 2024-03-22T10:50:57.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.550+0000 7fa95b200700 1 -- 172.21.15.115:0/3625154916 wait complete. 2024-03-22T10:50:57.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.728+0000 7f6ca0200700 1 -- 172.21.15.115:0/2120423779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 msgr2=0x7f6c98073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:57.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.728+0000 7f6ca0200700 1 --2- 172.21.15.115:0/2120423779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 0x7f6c98073180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009910 tx=0x7f6c8c009c20 comp rx=0 tx=0).stop 2024-03-22T10:50:57.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.728+0000 7f6ca0200700 1 -- 172.21.15.115:0/2120423779 shutdown_connections 2024-03-22T10:50:57.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.728+0000 7f6ca0200700 1 --2- 172.21.15.115:0/2120423779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c98073750 0x7f6c98073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.728+0000 7f6ca0200700 1 --2- 172.21.15.115:0/2120423779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 0x7f6c98073180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.728+0000 7f6ca0200700 1 -- 172.21.15.115:0/2120423779 >> 172.21.15.115:0/2120423779 conn(0x7f6c980fb6e0 msgr2=0x7f6c980fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:57.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.729+0000 7f6ca0200700 1 -- 172.21.15.115:0/2120423779 shutdown_connections 2024-03-22T10:50:57.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.729+0000 7f6ca0200700 1 -- 172.21.15.115:0/2120423779 wait complete. 2024-03-22T10:50:57.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.730+0000 7f6ca0200700 1 Processor -- start 2024-03-22T10:50:57.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.730+0000 7f6ca0200700 1 -- start start 2024-03-22T10:50:57.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6ca0200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c98073750 0x7f6c98197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:57.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6ca0200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 0x7f6c98198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:57.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6ca0200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981986b0 con 0x7f6c98074d20 2024-03-22T10:50:57.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6ca0200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981987f0 con 0x7f6c98073750 2024-03-22T10:50:57.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6c9ce00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 0x7f6c98198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:57.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6c9ce00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 0x7f6c98198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53424/0 (socket says 172.21.15.115:53424) 2024-03-22T10:50:57.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6c9ce00700 1 -- 172.21.15.115:0/306726005 learned_addr learned my addr 172.21.15.115:0/306726005 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:50:57.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6c9d800700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c98073750 0x7f6c98197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:57.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6c9ce00700 1 -- 172.21.15.115:0/306726005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c98073750 msgr2=0x7f6c98197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:57.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6c9ce00700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c98073750 0x7f6c98197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:57.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.731+0000 7f6c9ce00700 1 -- 172.21.15.115:0/306726005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c0095f0 con 0x7f6c98074d20 2024-03-22T10:50:57.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.732+0000 7f6c9ce00700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 0x7f6c98198090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c8a0 tx=0x7f6c8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:57.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.732+0000 7f6c96000700 1 -- 172.21.15.115:0/306726005 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c8400cea0 con 0x7f6c98074d20 2024-03-22T10:50:57.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.732+0000 7f6c96000700 1 -- 172.21.15.115:0/306726005 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6c84004830 con 0x7f6c98074d20 2024-03-22T10:50:57.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.732+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9819d2a0 con 0x7f6c98074d20 2024-03-22T10:50:57.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.732+0000 7f6c96000700 1 -- 172.21.15.115:0/306726005 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c840055b0 con 0x7f6c98074d20 2024-03-22T10:50:57.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.732+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9819d7f0 con 0x7f6c98074d20 2024-03-22T10:50:57.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.734+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9810ae90 con 0x7f6c98074d20 2024-03-22T10:50:57.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.735+0000 7f6c96000700 1 -- 172.21.15.115:0/306726005 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6c84007ad0 con 0x7f6c98074d20 2024-03-22T10:50:57.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.735+0000 7f6c96000700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6c8006dfa0 0x7f6c80070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:50:57.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.736+0000 7f6c96000700 1 -- 172.21.15.115:0/306726005 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6c8408db00 con 0x7f6c98074d20 2024-03-22T10:50:57.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.736+0000 7f6c9d800700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6c8006dfa0 0x7f6c80070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:50:57.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.736+0000 7f6c9d800700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6c8006dfa0 0x7f6c80070450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0059d0 tx=0x7f6c8c005920 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:50:57.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:57.740+0000 7f6c96000700 1 -- 172.21.15.115:0/306726005 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6c8405aa30 con 0x7f6c98074d20 2024-03-22T10:50:58.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:57 smithi115 ceph-mon[78672]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:58.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:57 smithi115 ceph-mon[78672]: pgmap v242: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-22T10:50:58.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:57 smithi115 ceph-mon[78672]: reconnect by client.14516 192.168.0.1:0/629163332 after 0.00299994 2024-03-22T10:50:58.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:57 smithi115 ceph-mon[78672]: reconnect by client.24289 192.168.0.1:0/3857176443 after 0.00299994 2024-03-22T10:50:58.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:57 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:reconnect 2024-03-22T10:50:58.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:57 smithi115 ceph-mon[78672]: fsmap cephfs:1/1 {0=cephfs.smithi150.slzwwb=up:reconnect} 2 up:standby 2024-03-22T10:50:58.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:57 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3928973078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:50:58.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.086+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6c9810b050 con 0x7f6c98074d20 2024-03-22T10:50:58.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.088+0000 7f6c96000700 1 -- 172.21.15.115:0/306726005 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f6c8401d090 con 0x7f6c98074d20 2024-03-22T10:50:58.080 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-03-22T10:50:58.080 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-22T10:50:58.080 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs is degraded 2024-03-22T10:50:58.080 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:50:58.080 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:50:58.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6c8006dfa0 msgr2=0x7f6c80070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:58.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6c8006dfa0 0x7f6c80070450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0059d0 tx=0x7f6c8c005920 comp rx=0 tx=0).stop 2024-03-22T10:50:58.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 msgr2=0x7f6c98198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:50:58.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 0x7f6c98198090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c8a0 tx=0x7f6c8400cc60 comp rx=0 tx=0).stop 2024-03-22T10:50:58.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 shutdown_connections 2024-03-22T10:50:58.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6c8006dfa0 0x7f6c80070450 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:58.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c98073750 0x7f6c98197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:58.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 --2- 172.21.15.115:0/306726005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c98074d20 0x7f6c98198090 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:50:58.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 >> 172.21.15.115:0/306726005 conn(0x7f6c980fb6e0 msgr2=0x7f6c98105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:50:58.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 shutdown_connections 2024-03-22T10:50:58.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:50:58.092+0000 7f6ca0200700 1 -- 172.21.15.115:0/306726005 wait complete. 2024-03-22T10:50:58.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:57 smithi150 ceph-mon[65513]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:58.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:57 smithi150 ceph-mon[65513]: pgmap v242: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-22T10:50:58.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:57 smithi150 ceph-mon[65513]: reconnect by client.14516 192.168.0.1:0/629163332 after 0.00299994 2024-03-22T10:50:58.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:57 smithi150 ceph-mon[65513]: reconnect by client.24289 192.168.0.1:0/3857176443 after 0.00299994 2024-03-22T10:50:58.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:57 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:reconnect 2024-03-22T10:50:58.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:57 smithi150 ceph-mon[65513]: fsmap cephfs:1/1 {0=cephfs.smithi150.slzwwb=up:reconnect} 2 up:standby 2024-03-22T10:50:58.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:57 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3928973078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:50:59.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:58 smithi115 ceph-mon[78672]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:59.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:58 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:rejoin 2024-03-22T10:50:59.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:58 smithi115 ceph-mon[78672]: fsmap cephfs:1/1 {0=cephfs.smithi150.slzwwb=up:rejoin} 2 up:standby 2024-03-22T10:50:59.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:58 smithi115 ceph-mon[78672]: daemon mds.cephfs.smithi150.slzwwb is now active in filesystem cephfs as rank 0 2024-03-22T10:50:59.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:58 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/306726005' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:50:59.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:58 smithi150 ceph-mon[65513]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:50:59.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:58 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:rejoin 2024-03-22T10:50:59.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:58 smithi150 ceph-mon[65513]: fsmap cephfs:1/1 {0=cephfs.smithi150.slzwwb=up:rejoin} 2 up:standby 2024-03-22T10:50:59.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:58 smithi150 ceph-mon[65513]: daemon mds.cephfs.smithi150.slzwwb is now active in filesystem cephfs as rank 0 2024-03-22T10:50:59.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:58 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/306726005' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:51:00.256 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:59 smithi115 ceph-mon[78672]: pgmap v243: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-22T10:51:00.256 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:59 smithi115 ceph-mon[78672]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T10:51:00.256 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:00.256 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:59 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:active 2024-03-22T10:51:00.256 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:59 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] up:boot 2024-03-22T10:51:00.256 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:59 smithi115 ceph-mon[78672]: fsmap cephfs:1 {0=cephfs.smithi150.slzwwb=up:active} 3 up:standby 2024-03-22T10:51:00.257 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:51:00.257 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:00.257 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:50:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:59 smithi150 ceph-mon[65513]: pgmap v243: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-22T10:51:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:59 smithi150 ceph-mon[65513]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T10:51:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:59 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] up:active 2024-03-22T10:51:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:59 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] up:boot 2024-03-22T10:51:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:59 smithi150 ceph-mon[65513]: fsmap cephfs:1 {0=cephfs.smithi150.slzwwb=up:active} 3 up:standby 2024-03-22T10:51:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:51:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:00.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:50:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:01.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:00 smithi115 ceph-mon[78672]: pgmap v244: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 102 B/s wr, 6 op/s 2024-03-22T10:51:01.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:00 smithi150 ceph-mon[65513]: pgmap v244: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 102 B/s wr, 6 op/s 2024-03-22T10:51:03.479 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:03 smithi115 ceph-mon[78672]: pgmap v245: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-03-22T10:51:03.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:03 smithi150 ceph-mon[65513]: pgmap v245: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-03-22T10:51:04.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:51:04.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:51:05.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:05 smithi115 ceph-mon[78672]: pgmap v246: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-22T10:51:05.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:05 smithi150 ceph-mon[65513]: pgmap v246: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-22T10:51:07.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:07 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:07.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:07 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:07.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:07 smithi150 ceph-mon[65513]: pgmap v247: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-03-22T10:51:07.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:07 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:07.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:07 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:07.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:07 smithi115 ceph-mon[78672]: pgmap v247: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-03-22T10:51:09.348 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:09 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:09.348 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:09 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:09.348 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:09 smithi115 ceph-mon[78672]: pgmap v248: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-22T10:51:09.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:09 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:09.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:09 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:09.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:09 smithi150 ceph-mon[65513]: pgmap v248: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-22T10:51:11.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:11 smithi150 ceph-mon[65513]: pgmap v249: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 686 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-22T10:51:11.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:11 smithi115 ceph-mon[78672]: pgmap v249: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 686 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-22T10:51:13.598 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:13 smithi150 ceph-mon[65513]: pgmap v250: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-03-22T10:51:13.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:13 smithi115 ceph-mon[78672]: pgmap v250: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-03-22T10:51:14.817 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi150.zvyrzc"]}]: dispatch 2024-03-22T10:51:14.818 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:14 smithi150 ceph-mon[65513]: Upgrade: It appears safe to stop mds.cephfs.smithi150.zvyrzc 2024-03-22T10:51:15.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:15.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi150.zvyrzc"]}]: dispatch 2024-03-22T10:51:15.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:14 smithi115 ceph-mon[78672]: Upgrade: It appears safe to stop mds.cephfs.smithi150.zvyrzc 2024-03-22T10:51:15.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:15 smithi115 ceph-mon[78672]: pgmap v251: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 0 op/s 2024-03-22T10:51:15.878 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:15 smithi150 ceph-mon[65513]: pgmap v251: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 0 op/s 2024-03-22T10:51:17.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:17 smithi150 ceph-mon[65513]: Upgrade: Updating mds.cephfs.smithi150.zvyrzc 2024-03-22T10:51:17.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:17 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:17.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:17 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.zvyrzc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:51:17.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:17 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:51:17.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:17 smithi150 ceph-mon[65513]: Deploying daemon mds.cephfs.smithi150.zvyrzc on smithi150 2024-03-22T10:51:17.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:17 smithi150 ceph-mon[65513]: pgmap v252: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:17.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:17 smithi115 ceph-mon[78672]: Upgrade: Updating mds.cephfs.smithi150.zvyrzc 2024-03-22T10:51:17.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:17 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:17.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:17 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.zvyrzc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:51:17.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:17 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:51:17.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:17 smithi115 ceph-mon[78672]: Deploying daemon mds.cephfs.smithi150.zvyrzc on smithi150 2024-03-22T10:51:17.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:17 smithi115 ceph-mon[78672]: pgmap v252: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:19.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:19 smithi150 ceph-mon[65513]: pgmap v253: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:19.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:51:19.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:19 smithi115 ceph-mon[78672]: pgmap v253: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:19.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:51:20.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:20 smithi150 ceph-mon[65513]: osdmap e82: 6 total, 6 up, 6 in 2024-03-22T10:51:20.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:20 smithi150 ceph-mon[65513]: fsmap cephfs:1 {0=cephfs.smithi150.slzwwb=up:active} 2 up:standby 2024-03-22T10:51:20.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:20 smithi115 ceph-mon[78672]: osdmap e82: 6 total, 6 up, 6 in 2024-03-22T10:51:20.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:20 smithi115 ceph-mon[78672]: fsmap cephfs:1 {0=cephfs.smithi150.slzwwb=up:active} 2 up:standby 2024-03-22T10:51:21.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:21 smithi150 ceph-mon[65513]: pgmap v255: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:21.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:21 smithi115 ceph-mon[78672]: pgmap v255: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:23.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:23 smithi150 ceph-mon[65513]: pgmap v256: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:23.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:23 smithi115 ceph-mon[78672]: pgmap v256: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:25.373 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:25 smithi150 ceph-mon[65513]: pgmap v257: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:25.716 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:25 smithi115 ceph-mon[78672]: pgmap v257: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:27.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:27.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:27.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:27.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:26 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.150:6824/1804848703,v1:172.21.15.150:6825/1804848703] up:boot 2024-03-22T10:51:27.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:26 smithi115 ceph-mon[78672]: fsmap cephfs:1 {0=cephfs.smithi150.slzwwb=up:active} 3 up:standby 2024-03-22T10:51:27.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:51:27.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:27.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:27.063 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:27.063 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:27.063 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:27.063 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:26 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.150:6824/1804848703,v1:172.21.15.150:6825/1804848703] up:boot 2024-03-22T10:51:27.063 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:26 smithi150 ceph-mon[65513]: fsmap cephfs:1 {0=cephfs.smithi150.slzwwb=up:active} 3 up:standby 2024-03-22T10:51:27.063 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:51:27.063 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:27.063 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:27.936 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:27 smithi150 ceph-mon[65513]: pgmap v258: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:28.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:27 smithi115 ceph-mon[78672]: pgmap v258: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.273+0000 7fe7aac00700 1 -- 172.21.15.115:0/2892941895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 msgr2=0x7fe7a4100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.273+0000 7fe7aac00700 1 --2- 172.21.15.115:0/2892941895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 0x7fe7a4100580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe794009910 tx=0x7fe794009c20 comp rx=0 tx=0).stop 2024-03-22T10:51:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.273+0000 7fe7aac00700 1 -- 172.21.15.115:0/2892941895 shutdown_connections 2024-03-22T10:51:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.273+0000 7fe7aac00700 1 --2- 172.21.15.115:0/2892941895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7a4101370 0x7fe7a41017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.273+0000 7fe7aac00700 1 --2- 172.21.15.115:0/2892941895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 0x7fe7a4100580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.273+0000 7fe7aac00700 1 -- 172.21.15.115:0/2892941895 >> 172.21.15.115:0/2892941895 conn(0x7fe7a40fb6e0 msgr2=0x7fe7a40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.273+0000 7fe7aac00700 1 -- 172.21.15.115:0/2892941895 shutdown_connections 2024-03-22T10:51:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.273+0000 7fe7aac00700 1 -- 172.21.15.115:0/2892941895 wait complete. 2024-03-22T10:51:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.274+0000 7fe7aac00700 1 Processor -- start 2024-03-22T10:51:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.274+0000 7fe7aac00700 1 -- start start 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.274+0000 7fe7aac00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 0x7fe7a4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.274+0000 7fe7aac00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7a4101370 0x7fe7a41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.274+0000 7fe7aac00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a4198660 con 0x7fe7a4100170 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.274+0000 7fe7aac00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a41987a0 con 0x7fe7a4101370 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.275+0000 7fe7a3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 0x7fe7a4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.275+0000 7fe7a3e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 0x7fe7a4197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53368/0 (socket says 172.21.15.115:53368) 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.275+0000 7fe7a3e00700 1 -- 172.21.15.115:0/2950100785 learned_addr learned my addr 172.21.15.115:0/2950100785 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.275+0000 7fe79b400700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7a4101370 0x7fe7a41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.275+0000 7fe7a3e00700 1 -- 172.21.15.115:0/2950100785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7a4101370 msgr2=0x7fe7a41980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.275+0000 7fe7a3e00700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7a4101370 0x7fe7a41980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.275+0000 7fe7a3e00700 1 -- 172.21.15.115:0/2950100785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940095f0 con 0x7fe7a4100170 2024-03-22T10:51:28.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.276+0000 7fe7a3e00700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 0x7fe7a4197b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe79400bda0 tx=0x7fe79400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:28.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.276+0000 7fe7a1600700 1 -- 172.21.15.115:0/2950100785 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79401d070 con 0x7fe7a4100170 2024-03-22T10:51:28.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.276+0000 7fe7a1600700 1 -- 172.21.15.115:0/2950100785 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe794004c50 con 0x7fe7a4100170 2024-03-22T10:51:28.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.276+0000 7fe7a1600700 1 -- 172.21.15.115:0/2950100785 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79400f690 con 0x7fe7a4100170 2024-03-22T10:51:28.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.276+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a419d200 con 0x7fe7a4100170 2024-03-22T10:51:28.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.276+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a419d670 con 0x7fe7a4100170 2024-03-22T10:51:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.277+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a4105450 con 0x7fe7a4100170 2024-03-22T10:51:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.278+0000 7fe7a1600700 1 -- 172.21.15.115:0/2950100785 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe794022a20 con 0x7fe7a4100170 2024-03-22T10:51:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.279+0000 7fe7a1600700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe78806dfa0 0x7fe788070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.279+0000 7fe7a1600700 1 -- 172.21.15.115:0/2950100785 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe79408f580 con 0x7fe7a4100170 2024-03-22T10:51:28.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.280+0000 7fe79b400700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe78806dfa0 0x7fe788070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:28.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.280+0000 7fe79b400700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe78806dfa0 0x7fe788070450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00ba60 tx=0x7fe78c005d50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:28.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.285+0000 7fe7a1600700 1 -- 172.21.15.115:0/2950100785 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe79405c600 con 0x7fe7a4100170 2024-03-22T10:51:28.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.534+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7a4061190 con 0x7fe78806dfa0 2024-03-22T10:51:28.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.537+0000 7fe7a1600700 1 -- 172.21.15.115:0/2950100785 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe7a4061190 con 0x7fe78806dfa0 2024-03-22T10:51:28.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.540+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe78806dfa0 msgr2=0x7fe788070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.540+0000 7fe7aac00700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe78806dfa0 0x7fe788070450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00ba60 tx=0x7fe78c005d50 comp rx=0 tx=0).stop 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.540+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 msgr2=0x7fe7a4197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.540+0000 7fe7aac00700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 0x7fe7a4197b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe79400bda0 tx=0x7fe79400be80 comp rx=0 tx=0).stop 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.540+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 shutdown_connections 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.540+0000 7fe7aac00700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe78806dfa0 0x7fe788070450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.540+0000 7fe7aac00700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7a4100170 0x7fe7a4197b90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.540+0000 7fe7aac00700 1 --2- 172.21.15.115:0/2950100785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7a4101370 0x7fe7a41980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.540+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 >> 172.21.15.115:0/2950100785 conn(0x7fe7a40fb6e0 msgr2=0x7fe7a4102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.541+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 shutdown_connections 2024-03-22T10:51:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.541+0000 7fe7aac00700 1 -- 172.21.15.115:0/2950100785 wait complete. 2024-03-22T10:51:28.546 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:51:28.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.695+0000 7f8304e00700 1 -- 172.21.15.115:0/1661331062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8300101490 msgr2=0x7f83001018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:28.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.695+0000 7f8304e00700 1 --2- 172.21.15.115:0/1661331062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8300101490 0x7f83001018e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009910 tx=0x7f82ec009c20 comp rx=0 tx=0).stop 2024-03-22T10:51:28.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.695+0000 7f8304e00700 1 -- 172.21.15.115:0/1661331062 shutdown_connections 2024-03-22T10:51:28.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.695+0000 7f8304e00700 1 --2- 172.21.15.115:0/1661331062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8300101490 0x7f83001018e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.695+0000 7f8304e00700 1 --2- 172.21.15.115:0/1661331062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83000fe410 0x7f83000fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.695+0000 7f8304e00700 1 -- 172.21.15.115:0/1661331062 >> 172.21.15.115:0/1661331062 conn(0x7f83000f9d80 msgr2=0x7f83000fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:28.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.695+0000 7f8304e00700 1 -- 172.21.15.115:0/1661331062 shutdown_connections 2024-03-22T10:51:28.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.695+0000 7f8304e00700 1 -- 172.21.15.115:0/1661331062 wait complete. 2024-03-22T10:51:28.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.696+0000 7f8304e00700 1 Processor -- start 2024-03-22T10:51:28.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.697+0000 7f8304e00700 1 -- start start 2024-03-22T10:51:28.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.697+0000 7f8304e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83000fe410 0x7f8300197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:28.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.697+0000 7f8304e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8300101490 0x7f83001980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:28.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.697+0000 7f8304e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8300198650 con 0x7f83000fe410 2024-03-22T10:51:28.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.697+0000 7f8304e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8300198790 con 0x7f8300101490 2024-03-22T10:51:28.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.697+0000 7f82fde00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83000fe410 0x7f8300197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:28.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.697+0000 7f82fde00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83000fe410 0x7f8300197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53392/0 (socket says 172.21.15.115:53392) 2024-03-22T10:51:28.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.697+0000 7f82fde00700 1 -- 172.21.15.115:0/184446236 learned_addr learned my addr 172.21.15.115:0/184446236 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:51:28.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.698+0000 7f82fd400700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8300101490 0x7f83001980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:28.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.698+0000 7f82fde00700 1 -- 172.21.15.115:0/184446236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8300101490 msgr2=0x7f83001980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:28.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.698+0000 7f82fde00700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8300101490 0x7f83001980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.698+0000 7f82fde00700 1 -- 172.21.15.115:0/184446236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec0095f0 con 0x7f83000fe410 2024-03-22T10:51:28.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.698+0000 7f82fde00700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83000fe410 0x7f8300197b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c910 tx=0x7f82f000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:28.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.698+0000 7f82f6a00700 1 -- 172.21.15.115:0/184446236 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f00041f0 con 0x7f83000fe410 2024-03-22T10:51:28.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.699+0000 7f82f6a00700 1 -- 172.21.15.115:0/184446236 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f82f0007db0 con 0x7f83000fe410 2024-03-22T10:51:28.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.699+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830019d250 con 0x7f83000fe410 2024-03-22T10:51:28.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.699+0000 7f82f6a00700 1 -- 172.21.15.115:0/184446236 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f0003d60 con 0x7f83000fe410 2024-03-22T10:51:28.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.699+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830019d770 con 0x7f83000fe410 2024-03-22T10:51:28.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.700+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8300105570 con 0x7f83000fe410 2024-03-22T10:51:28.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.701+0000 7f82f6a00700 1 -- 172.21.15.115:0/184446236 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f82f0007900 con 0x7f83000fe410 2024-03-22T10:51:28.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.702+0000 7f82f6a00700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f82e406dfa0 0x7f82e4070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:28.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.702+0000 7f82f6a00700 1 -- 172.21.15.115:0/184446236 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f82f008db90 con 0x7f83000fe410 2024-03-22T10:51:28.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.702+0000 7f82fd400700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f82e406dfa0 0x7f82e4070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:28.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.703+0000 7f82fd400700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f82e406dfa0 0x7f82e4070450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82ec006010 tx=0x7f82ec00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:28.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.707+0000 7f82f6a00700 1 -- 172.21.15.115:0/184446236 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f82f005aa40 con 0x7f83000fe410 2024-03-22T10:51:28.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.938+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8300061190 con 0x7f82e406dfa0 2024-03-22T10:51:28.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.941+0000 7f82f6a00700 1 -- 172.21.15.115:0/184446236 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8300061190 con 0x7f82e406dfa0 2024-03-22T10:51:28.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f82e406dfa0 msgr2=0x7f82e4070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:28.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f82e406dfa0 0x7f82e4070450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82ec006010 tx=0x7f82ec00b540 comp rx=0 tx=0).stop 2024-03-22T10:51:28.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83000fe410 msgr2=0x7f8300197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:28.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83000fe410 0x7f8300197b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c910 tx=0x7f82f000ccd0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 shutdown_connections 2024-03-22T10:51:28.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f82e406dfa0 0x7f82e4070450 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83000fe410 0x7f8300197b80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 --2- 172.21.15.115:0/184446236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8300101490 0x7f83001980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:28.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 >> 172.21.15.115:0/184446236 conn(0x7f83000f9d80 msgr2=0x7f83001026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:28.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 shutdown_connections 2024-03-22T10:51:28.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:28.945+0000 7f8304e00700 1 -- 172.21.15.115:0/184446236 wait complete. 2024-03-22T10:51:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.099+0000 7fecb3600700 1 -- 172.21.15.115:0/1470542608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac074d80 msgr2=0x7fecac0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.099+0000 7fecb3600700 1 --2- 172.21.15.115:0/1470542608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac074d80 0x7fecac0731e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009940 tx=0x7fec9c009c50 comp rx=0 tx=0).stop 2024-03-22T10:51:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.099+0000 7fecb3600700 1 -- 172.21.15.115:0/1470542608 shutdown_connections 2024-03-22T10:51:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.099+0000 7fecb3600700 1 --2- 172.21.15.115:0/1470542608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecac0737b0 0x7fecac073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.099+0000 7fecb3600700 1 --2- 172.21.15.115:0/1470542608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac074d80 0x7fecac0731e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.100+0000 7fecb3600700 1 -- 172.21.15.115:0/1470542608 >> 172.21.15.115:0/1470542608 conn(0x7fecac0fb6e0 msgr2=0x7fecac0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.100+0000 7fecb3600700 1 -- 172.21.15.115:0/1470542608 shutdown_connections 2024-03-22T10:51:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.100+0000 7fecb3600700 1 -- 172.21.15.115:0/1470542608 wait complete. 2024-03-22T10:51:29.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.101+0000 7fecb3600700 1 Processor -- start 2024-03-22T10:51:29.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.101+0000 7fecb3600700 1 -- start start 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.101+0000 7fecb3600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac0737b0 0x7fecac197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.101+0000 7fecb3600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecac074d80 0x7fecac198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.101+0000 7fecb3600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac198670 con 0x7fecac0737b0 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.101+0000 7fecb3600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac1987b0 con 0x7fecac074d80 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.102+0000 7fecb0c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac0737b0 0x7fecac197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.102+0000 7fecb0c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac0737b0 0x7fecac197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53416/0 (socket says 172.21.15.115:53416) 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.102+0000 7fecb0c00700 1 -- 172.21.15.115:0/3405334644 learned_addr learned my addr 172.21.15.115:0/3405334644 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.102+0000 7feca3e00700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecac074d80 0x7fecac198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.102+0000 7fecb0c00700 1 -- 172.21.15.115:0/3405334644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecac074d80 msgr2=0x7fecac198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.102+0000 7fecb0c00700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecac074d80 0x7fecac198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.102+0000 7fecb0c00700 1 -- 172.21.15.115:0/3405334644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0095f0 con 0x7fecac0737b0 2024-03-22T10:51:29.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.102+0000 7fecb0c00700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac0737b0 0x7fecac197b10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0055f0 tx=0x7fec9c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:29.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.103+0000 7fecaa000700 1 -- 172.21.15.115:0/3405334644 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec9c01d070 con 0x7fecac0737b0 2024-03-22T10:51:29.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.103+0000 7fecaa000700 1 -- 172.21.15.115:0/3405334644 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fec9c00f460 con 0x7fecac0737b0 2024-03-22T10:51:29.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.103+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac19d200 con 0x7fecac0737b0 2024-03-22T10:51:29.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.103+0000 7fecaa000700 1 -- 172.21.15.115:0/3405334644 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec9c021640 con 0x7fecac0737b0 2024-03-22T10:51:29.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.103+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac19d770 con 0x7fecac0737b0 2024-03-22T10:51:29.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.104+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac10ae90 con 0x7fecac0737b0 2024-03-22T10:51:29.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.105+0000 7fecaa000700 1 -- 172.21.15.115:0/3405334644 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fec9c00fad0 con 0x7fecac0737b0 2024-03-22T10:51:29.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.106+0000 7fecaa000700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fec9006dfa0 0x7fec90070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:29.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.106+0000 7fecaa000700 1 -- 172.21.15.115:0/3405334644 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fec9c08f0a0 con 0x7fecac0737b0 2024-03-22T10:51:29.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.106+0000 7feca3e00700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fec9006dfa0 0x7fec90070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:29.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.107+0000 7feca3e00700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fec9006dfa0 0x7fec90070450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fec94005950 tx=0x7fec940058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.111+0000 7fecaa000700 1 -- 172.21.15.115:0/3405334644 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec9c05bfd0 con 0x7fecac0737b0 2024-03-22T10:51:29.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.331+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fecac19dec0 con 0x7fec9006dfa0 2024-03-22T10:51:29.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.362+0000 7fecaa000700 1 -- 172.21.15.115:0/3405334644 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fecac19dec0 con 0x7fec9006dfa0 2024-03-22T10:51:29.353 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:51:29.353 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (17m) 23s ago 19m 14.7M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:51:29.353 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (20m) 23s ago 20m 9323k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:51:29.353 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (18m) 2m ago 18m 8644k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:51:29.353 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (5m) 23s ago 20m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:51:29.353 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (5m) 2m ago 18m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:51:29.353 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (17m) 23s ago 19m 73.8M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:51:29.353 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (59s) 23s ago 12m 12.0M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 44a0769ebe8e 2024-03-22T10:51:29.353 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (30s) 23s ago 12m 10.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51d700f6eb18 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (12m) 2m ago 12m 17.0M - 18.2.0 dc2bc1663786 155f3b745b6b 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 starting - - - - 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (9m) 23s ago 21m 512M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (8m) 2m ago 18m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (7m) 23s ago 21m 55.1M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (6m) 2m ago 18m 38.8M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (20m) 23s ago 20m 8921k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (18m) 2m ago 18m 8212k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (5m) 23s ago 17m 96.9M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (4m) 23s ago 16m 95.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (4m) 23s ago 16m 83.0M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (3m) 2m ago 15m 87.8M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (2m) 2m ago 15m 84.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (2m) 2m ago 14m 11.1M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:51:29.354 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (8m) 23s ago 19m 74.3M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:51:29.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fec9006dfa0 msgr2=0x7fec90070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fec9006dfa0 0x7fec90070450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fec94005950 tx=0x7fec940058e0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac0737b0 msgr2=0x7fecac197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac0737b0 0x7fecac197b10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0055f0 tx=0x7fec9c0056d0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 shutdown_connections 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fec9006dfa0 0x7fec90070450 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecac0737b0 0x7fecac197b10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 --2- 172.21.15.115:0/3405334644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecac074d80 0x7fecac198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 >> 172.21.15.115:0/3405334644 conn(0x7fecac0fb6e0 msgr2=0x7fecac105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.365+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 shutdown_connections 2024-03-22T10:51:29.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.366+0000 7fecb3600700 1 -- 172.21.15.115:0/3405334644 wait complete. 2024-03-22T10:51:29.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.518+0000 7ffad9a00700 1 -- 172.21.15.115:0/1746031727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41023b0 msgr2=0x7ffad41027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.518+0000 7ffad9a00700 1 --2- 172.21.15.115:0/1746031727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41023b0 0x7ffad41027c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009910 tx=0x7ffac4009c20 comp rx=0 tx=0).stop 2024-03-22T10:51:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.518+0000 7ffad9a00700 1 -- 172.21.15.115:0/1746031727 shutdown_connections 2024-03-22T10:51:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.518+0000 7ffad9a00700 1 --2- 172.21.15.115:0/1746031727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad41035b0 0x7ffad4103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.518+0000 7ffad9a00700 1 --2- 172.21.15.115:0/1746031727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41023b0 0x7ffad41027c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.518+0000 7ffad9a00700 1 -- 172.21.15.115:0/1746031727 >> 172.21.15.115:0/1746031727 conn(0x7ffad40fd960 msgr2=0x7ffad40ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.518+0000 7ffad9a00700 1 -- 172.21.15.115:0/1746031727 shutdown_connections 2024-03-22T10:51:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.518+0000 7ffad9a00700 1 -- 172.21.15.115:0/1746031727 wait complete. 2024-03-22T10:51:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.519+0000 7ffad9a00700 1 Processor -- start 2024-03-22T10:51:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.519+0000 7ffad9a00700 1 -- start start 2024-03-22T10:51:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad9a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad41023b0 0x7ffad4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad9a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41035b0 0x7ffad41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad9a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4198710 con 0x7ffad41035b0 2024-03-22T10:51:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad9a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4198850 con 0x7ffad41023b0 2024-03-22T10:51:29.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad2000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41035b0 0x7ffad41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:29.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad2000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41035b0 0x7ffad41980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53446/0 (socket says 172.21.15.115:53446) 2024-03-22T10:51:29.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad2000700 1 -- 172.21.15.115:0/2906292798 learned_addr learned my addr 172.21.15.115:0/2906292798 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:51:29.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad2a00700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad41023b0 0x7ffad4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:29.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad2000700 1 -- 172.21.15.115:0/2906292798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad41023b0 msgr2=0x7ffad4197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad2000700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad41023b0 0x7ffad4197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.520+0000 7ffad2000700 1 -- 172.21.15.115:0/2906292798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40095f0 con 0x7ffad41035b0 2024-03-22T10:51:29.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.521+0000 7ffad2000700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41035b0 0x7ffad41980f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffac000e920 tx=0x7ffac000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:29.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.521+0000 7ffacb400700 1 -- 172.21.15.115:0/2906292798 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac000c9d0 con 0x7ffad41035b0 2024-03-22T10:51:29.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.521+0000 7ffacb400700 1 -- 172.21.15.115:0/2906292798 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffac0010430 con 0x7ffad41035b0 2024-03-22T10:51:29.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.521+0000 7ffacb400700 1 -- 172.21.15.115:0/2906292798 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac0018650 con 0x7ffad41035b0 2024-03-22T10:51:29.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.521+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad419d300 con 0x7ffad41035b0 2024-03-22T10:51:29.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.521+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad419d850 con 0x7ffad41035b0 2024-03-22T10:51:29.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.522+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad410af00 con 0x7ffad41035b0 2024-03-22T10:51:29.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.523+0000 7ffacb400700 1 -- 172.21.15.115:0/2906292798 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ffac00105a0 con 0x7ffad41035b0 2024-03-22T10:51:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.524+0000 7ffacb400700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffab806dfa0 0x7ffab8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.524+0000 7ffacb400700 1 -- 172.21.15.115:0/2906292798 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ffac0014070 con 0x7ffad41035b0 2024-03-22T10:51:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.525+0000 7ffad2a00700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffab806dfa0 0x7ffab8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.525+0000 7ffad2a00700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffab806dfa0 0x7ffab8070450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b6f0 tx=0x7ffac4005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:29.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.529+0000 7ffacb400700 1 -- 172.21.15.115:0/2906292798 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffac0053740 con 0x7ffad41035b0 2024-03-22T10:51:29.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.838+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffad419e310 con 0x7ffad41035b0 2024-03-22T10:51:29.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.839+0000 7ffacb400700 1 -- 172.21.15.115:0/2906292798 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7ffac005af80 con 0x7ffad41035b0 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:51:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:51:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-03-22T10:51:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 3 2024-03-22T10:51:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:51:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:51:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-03-22T10:51:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 13 2024-03-22T10:51:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:51:29.831 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:51:29.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffab806dfa0 msgr2=0x7ffab8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffab806dfa0 0x7ffab8070450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b6f0 tx=0x7ffac4005940 comp rx=0 tx=0).stop 2024-03-22T10:51:29.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41035b0 msgr2=0x7ffad41980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41035b0 0x7ffad41980f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffac000e920 tx=0x7ffac000ec30 comp rx=0 tx=0).stop 2024-03-22T10:51:29.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 shutdown_connections 2024-03-22T10:51:29.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ffab806dfa0 0x7ffab8070450 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad41023b0 0x7ffad4197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 --2- 172.21.15.115:0/2906292798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad41035b0 0x7ffad41980f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 >> 172.21.15.115:0/2906292798 conn(0x7ffad40fd960 msgr2=0x7ffad41067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:29.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 shutdown_connections 2024-03-22T10:51:29.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.842+0000 7ffad9a00700 1 -- 172.21.15.115:0/2906292798 wait complete. 2024-03-22T10:51:29.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.993+0000 7f98ec800700 1 -- 172.21.15.115:0/2245106406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41035a0 msgr2=0x7f98e41039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.993+0000 7f98ec800700 1 --2- 172.21.15.115:0/2245106406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41035a0 0x7f98e41039f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009910 tx=0x7f98d0009c20 comp rx=0 tx=0).stop 2024-03-22T10:51:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:29 smithi115 ceph-mon[78672]: pgmap v259: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:29 smithi115 ceph-mon[78672]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:51:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:29 smithi115 ceph-mon[78672]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:51:29.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.993+0000 7f98ec800700 1 -- 172.21.15.115:0/2245106406 shutdown_connections 2024-03-22T10:51:29.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.993+0000 7f98ec800700 1 --2- 172.21.15.115:0/2245106406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41035a0 0x7f98e41039f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.993+0000 7f98ec800700 1 --2- 172.21.15.115:0/2245106406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e41023a0 0x7f98e41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.993+0000 7f98ec800700 1 -- 172.21.15.115:0/2245106406 >> 172.21.15.115:0/2245106406 conn(0x7f98e40fd990 msgr2=0x7f98e40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:29.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.994+0000 7f98ec800700 1 -- 172.21.15.115:0/2245106406 shutdown_connections 2024-03-22T10:51:29.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.994+0000 7f98ec800700 1 -- 172.21.15.115:0/2245106406 wait complete. 2024-03-22T10:51:29.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.995+0000 7f98ec800700 1 Processor -- start 2024-03-22T10:51:29.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.995+0000 7f98ec800700 1 -- start start 2024-03-22T10:51:29.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.995+0000 7f98ec800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41023a0 0x7f98e4197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.995+0000 7f98ec800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e41035a0 0x7f98e41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.995+0000 7f98ec800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4198700 con 0x7f98e41023a0 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.995+0000 7f98ec800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4198840 con 0x7f98e41035a0 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.995+0000 7f98e9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41023a0 0x7f98e4197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.995+0000 7f98e9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41023a0 0x7f98e4197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53456/0 (socket says 172.21.15.115:53456) 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.995+0000 7f98e9e00700 1 -- 172.21.15.115:0/1624683998 learned_addr learned my addr 172.21.15.115:0/1624683998 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.996+0000 7f98e9400700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e41035a0 0x7f98e41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.996+0000 7f98e9e00700 1 -- 172.21.15.115:0/1624683998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e41035a0 msgr2=0x7f98e41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.996+0000 7f98e9e00700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e41035a0 0x7f98e41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.996+0000 7f98e9e00700 1 -- 172.21.15.115:0/1624683998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00095f0 con 0x7f98e41023a0 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.996+0000 7f98e9e00700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41023a0 0x7f98e4197ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f98d800e8f0 tx=0x7f98d800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.996+0000 7f98dea00700 1 -- 172.21.15.115:0/1624683998 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d8009790 con 0x7f98e41023a0 2024-03-22T10:51:29.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.997+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e419d2f0 con 0x7f98e41023a0 2024-03-22T10:51:29.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.997+0000 7f98dea00700 1 -- 172.21.15.115:0/1624683998 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f98d800c810 con 0x7f98e41023a0 2024-03-22T10:51:29.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.997+0000 7f98dea00700 1 -- 172.21.15.115:0/1624683998 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d8003d20 con 0x7f98e41023a0 2024-03-22T10:51:29.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.997+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e419d840 con 0x7f98e41023a0 2024-03-22T10:51:29.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.998+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c4005320 con 0x7f98e41023a0 2024-03-22T10:51:29.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:29.999+0000 7f98dea00700 1 -- 172.21.15.115:0/1624683998 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f98d80041f0 con 0x7f98e41023a0 2024-03-22T10:51:29.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.000+0000 7f98dea00700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f98cc06dfa0 0x7f98cc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:29.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.000+0000 7f98dea00700 1 -- 172.21.15.115:0/1624683998 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f98d8014070 con 0x7f98e41023a0 2024-03-22T10:51:29.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.000+0000 7f98e9400700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f98cc06dfa0 0x7f98cc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:29.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.001+0000 7f98e9400700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f98cc06dfa0 0x7f98cc070450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009fd0 tx=0x7f98d0005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:29.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.005+0000 7f98dea00700 1 -- 172.21.15.115:0/1624683998 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f98d805af80 con 0x7f98e41023a0 2024-03-22T10:51:30.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:29 smithi150 ceph-mon[65513]: pgmap v259: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:30.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:29 smithi150 ceph-mon[65513]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:51:30.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:29 smithi150 ceph-mon[65513]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:51:30.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.278+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f98c4006200 con 0x7f98e41023a0 2024-03-22T10:51:30.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.278+0000 7f98dea00700 1 -- 172.21.15.115:0/1624683998 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1897 (secure 0 0 0) 0x7f98d805ab10 con 0x7f98e41023a0 2024-03-22T10:51:30.269 INFO:teuthology.orchestra.run.smithi115.stdout:e33 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 31 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:50:58.971074+0000 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:51:30.270 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 81 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 1 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:in 0 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=24255} 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 1 2024-03-22T10:51:30.271 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 24255 members: 24255 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{0:24255} state up:active seq 186 join_fscid=1 addr [v2:172.21.15.150:6826/2497876489,v1:172.21.15.150:6827/2497876489] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{-1:34476} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:34508} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:51:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6824/1804848703,v1:172.21.15.150:6825/1804848703] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:51:30.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f98cc06dfa0 msgr2=0x7f98cc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:30.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f98cc06dfa0 0x7f98cc070450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009fd0 tx=0x7f98d0005fd0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41023a0 msgr2=0x7f98e4197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:30.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41023a0 0x7f98e4197ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f98d800e8f0 tx=0x7f98d800ec00 comp rx=0 tx=0).stop 2024-03-22T10:51:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 shutdown_connections 2024-03-22T10:51:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f98cc06dfa0 0x7f98cc070450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98e41023a0 0x7f98e4197ba0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 --2- 172.21.15.115:0/1624683998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e41035a0 0x7f98e41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 >> 172.21.15.115:0/1624683998 conn(0x7f98e40fd990 msgr2=0x7f98e41067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 shutdown_connections 2024-03-22T10:51:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.281+0000 7f98ec800700 1 -- 172.21.15.115:0/1624683998 wait complete. 2024-03-22T10:51:30.274 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 33 2024-03-22T10:51:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.431+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2895293661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 msgr2=0x7f8fc0107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.431+0000 7f8fc4c00700 1 --2- 172.21.15.115:0/2895293661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 0x7f8fc0107df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009910 tx=0x7f8fac009c20 comp rx=0 tx=0).stop 2024-03-22T10:51:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.431+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2895293661 shutdown_connections 2024-03-22T10:51:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.431+0000 7f8fc4c00700 1 --2- 172.21.15.115:0/2895293661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 0x7f8fc0107df0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.431+0000 7f8fc4c00700 1 --2- 172.21.15.115:0/2895293661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fc01067a0 0x7f8fc0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.431+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2895293661 >> 172.21.15.115:0/2895293661 conn(0x7f8fc0075910 msgr2=0x7f8fc0077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.431+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2895293661 shutdown_connections 2024-03-22T10:51:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.431+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2895293661 wait complete. 2024-03-22T10:51:30.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.432+0000 7f8fc4c00700 1 Processor -- start 2024-03-22T10:51:30.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.432+0000 7f8fc4c00700 1 -- start start 2024-03-22T10:51:30.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.432+0000 7f8fc4c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fc01067a0 0x7f8fc019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:30.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.432+0000 7f8fc4c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 0x7f8fc019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:30.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.432+0000 7f8fc4c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc019cb00 con 0x7f8fc01079a0 2024-03-22T10:51:30.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.432+0000 7f8fc4c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc019cc40 con 0x7f8fc01067a0 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.433+0000 7f8fbd400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 0x7f8fc019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.433+0000 7f8fbd400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 0x7f8fc019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53468/0 (socket says 172.21.15.115:53468) 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.433+0000 7f8fbd400700 1 -- 172.21.15.115:0/2175592248 learned_addr learned my addr 172.21.15.115:0/2175592248 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.433+0000 7f8fbde00700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fc01067a0 0x7f8fc019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.433+0000 7f8fbd400700 1 -- 172.21.15.115:0/2175592248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fc01067a0 msgr2=0x7f8fc019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.433+0000 7f8fbd400700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fc01067a0 0x7f8fc019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.433+0000 7f8fbd400700 1 -- 172.21.15.115:0/2175592248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac0095f0 con 0x7f8fc01079a0 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.434+0000 7f8fbd400700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 0x7f8fc019c4e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00bd70 tx=0x7f8fac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.434+0000 7f8fb6a00700 1 -- 172.21.15.115:0/2175592248 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fac01d070 con 0x7f8fc01079a0 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.434+0000 7f8fb6a00700 1 -- 172.21.15.115:0/2175592248 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8fac022470 con 0x7f8fc01079a0 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.434+0000 7f8fb6a00700 1 -- 172.21.15.115:0/2175592248 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fac00f670 con 0x7f8fc01079a0 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.434+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc01a1690 con 0x7f8fc01079a0 2024-03-22T10:51:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.434+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc01a1b80 con 0x7f8fc01079a0 2024-03-22T10:51:30.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.435+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc010f2f0 con 0x7f8fc01079a0 2024-03-22T10:51:30.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.436+0000 7f8fb6a00700 1 -- 172.21.15.115:0/2175592248 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f8fac022a50 con 0x7f8fc01079a0 2024-03-22T10:51:30.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.437+0000 7f8fb6a00700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8fa406dfa0 0x7f8fa4070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:30.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.437+0000 7f8fb6a00700 1 -- 172.21.15.115:0/2175592248 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8fac08f300 con 0x7f8fc01079a0 2024-03-22T10:51:30.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.437+0000 7f8fbde00700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8fa406dfa0 0x7f8fa4070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:30.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.438+0000 7f8fbde00700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8fa406dfa0 0x7f8fa4070450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005950 tx=0x7f8fb00058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:30.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.441+0000 7f8fb6a00700 1 -- 172.21.15.115:0/2175592248 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8fac05c4a0 con 0x7f8fc01079a0 2024-03-22T10:51:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.684+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fc010c2f0 con 0x7f8fa406dfa0 2024-03-22T10:51:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.687+0000 7f8fb6a00700 1 -- 172.21.15.115:0/2175592248 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8fc010c2f0 con 0x7f8fa406dfa0 2024-03-22T10:51:30.678 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:51:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:51:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:51:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:51:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:51:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:51:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "osd", 2024-03-22T10:51:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:51:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "crash" 2024-03-22T10:51:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:51:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "15/23 daemons upgraded", 2024-03-22T10:51:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T10:51:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:51:30.679 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:51:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8fa406dfa0 msgr2=0x7f8fa4070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8fa406dfa0 0x7f8fa4070450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005950 tx=0x7f8fb00058e0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 msgr2=0x7f8fc019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 0x7f8fc019c4e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00bd70 tx=0x7f8fac00be50 comp rx=0 tx=0).stop 2024-03-22T10:51:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 shutdown_connections 2024-03-22T10:51:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8fa406dfa0 0x7f8fa4070450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fc01067a0 0x7f8fc019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 --2- 172.21.15.115:0/2175592248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fc01079a0 0x7f8fc019c4e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 >> 172.21.15.115:0/2175592248 conn(0x7f8fc0075910 msgr2=0x7f8fc010abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 shutdown_connections 2024-03-22T10:51:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.691+0000 7f8fc4c00700 1 -- 172.21.15.115:0/2175592248 wait complete. 2024-03-22T10:51:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.842+0000 7fcb9d400700 1 -- 172.21.15.115:0/1503221796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981067a0 msgr2=0x7fcb98106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:30.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.842+0000 7fcb9d400700 1 --2- 172.21.15.115:0/1503221796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981067a0 0x7fcb98106bb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009910 tx=0x7fcb88009c20 comp rx=0 tx=0).stop 2024-03-22T10:51:30.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.842+0000 7fcb9d400700 1 -- 172.21.15.115:0/1503221796 shutdown_connections 2024-03-22T10:51:30.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.842+0000 7fcb9d400700 1 --2- 172.21.15.115:0/1503221796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb981079a0 0x7fcb98107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.842+0000 7fcb9d400700 1 --2- 172.21.15.115:0/1503221796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981067a0 0x7fcb98106bb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.842+0000 7fcb9d400700 1 -- 172.21.15.115:0/1503221796 >> 172.21.15.115:0/1503221796 conn(0x7fcb98075910 msgr2=0x7fcb98077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:30.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.842+0000 7fcb9d400700 1 -- 172.21.15.115:0/1503221796 shutdown_connections 2024-03-22T10:51:30.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.842+0000 7fcb9d400700 1 -- 172.21.15.115:0/1503221796 wait complete. 2024-03-22T10:51:30.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.843+0000 7fcb9d400700 1 Processor -- start 2024-03-22T10:51:30.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.843+0000 7fcb9d400700 1 -- start start 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb9d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb981067a0 0x7fcb9819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb9d400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981079a0 0x7fcb9819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb9d400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9819cb30 con 0x7fcb981079a0 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb9d400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9819cc70 con 0x7fcb981067a0 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb95e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981079a0 0x7fcb9819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb95e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981079a0 0x7fcb9819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53492/0 (socket says 172.21.15.115:53492) 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb95e00700 1 -- 172.21.15.115:0/2410074954 learned_addr learned my addr 172.21.15.115:0/2410074954 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb96800700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb981067a0 0x7fcb9819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb95e00700 1 -- 172.21.15.115:0/2410074954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb981067a0 msgr2=0x7fcb9819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb95e00700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb981067a0 0x7fcb9819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.844+0000 7fcb95e00700 1 -- 172.21.15.115:0/2410074954 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb880095f0 con 0x7fcb981079a0 2024-03-22T10:51:30.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.845+0000 7fcb95e00700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981079a0 0x7fcb9819c510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400d900 tx=0x7fcb8400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:30.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.845+0000 7fcb8f400700 1 -- 172.21.15.115:0/2410074954 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb84004a00 con 0x7fcb981079a0 2024-03-22T10:51:30.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.845+0000 7fcb8f400700 1 -- 172.21.15.115:0/2410074954 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcb840054d0 con 0x7fcb981079a0 2024-03-22T10:51:30.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.845+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb981a1720 con 0x7fcb981079a0 2024-03-22T10:51:30.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.845+0000 7fcb8f400700 1 -- 172.21.15.115:0/2410074954 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb84009d00 con 0x7fcb981079a0 2024-03-22T10:51:30.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.845+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb981a1c70 con 0x7fcb981079a0 2024-03-22T10:51:30.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.846+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9810f310 con 0x7fcb981079a0 2024-03-22T10:51:30.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.847+0000 7fcb8f400700 1 -- 172.21.15.115:0/2410074954 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fcb84005020 con 0x7fcb981079a0 2024-03-22T10:51:30.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.848+0000 7fcb8f400700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcb7c06dfa0 0x7fcb7c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:51:30.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.848+0000 7fcb8f400700 1 -- 172.21.15.115:0/2410074954 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcb8408de00 con 0x7fcb981079a0 2024-03-22T10:51:30.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.848+0000 7fcb96800700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcb7c06dfa0 0x7fcb7c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:51:30.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.849+0000 7fcb96800700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcb7c06dfa0 0x7fcb7c070450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb88006010 tx=0x7fcb88005ec0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:51:30.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:30.852+0000 7fcb8f400700 1 -- 172.21.15.115:0/2410074954 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb8405acb0 con 0x7fcb981079a0 2024-03-22T10:51:31.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:30 smithi115 ceph-mon[78672]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:51:31.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:30 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2906292798' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:31.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:31.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:30 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:31.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:30 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1624683998' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:51:31.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.146+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcb981a22b0 con 0x7fcb981079a0 2024-03-22T10:51:31.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.146+0000 7fcb8f400700 1 -- 172.21.15.115:0/2410074954 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fcb84009e60 con 0x7fcb981079a0 2024-03-22T10:51:31.137 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:51:31.137 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:51:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:51:31.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.149+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcb7c06dfa0 msgr2=0x7fcb7c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:31.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.149+0000 7fcb9d400700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcb7c06dfa0 0x7fcb7c070450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb88006010 tx=0x7fcb88005ec0 comp rx=0 tx=0).stop 2024-03-22T10:51:31.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.149+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981079a0 msgr2=0x7fcb9819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:51:31.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.149+0000 7fcb9d400700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981079a0 0x7fcb9819c510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400d900 tx=0x7fcb8400dc10 comp rx=0 tx=0).stop 2024-03-22T10:51:31.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.149+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 shutdown_connections 2024-03-22T10:51:31.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.149+0000 7fcb9d400700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcb7c06dfa0 0x7fcb7c070450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:31.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.149+0000 7fcb9d400700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb981067a0 0x7fcb9819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:31.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.149+0000 7fcb9d400700 1 --2- 172.21.15.115:0/2410074954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb981079a0 0x7fcb9819c510 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:51:31.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.149+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 >> 172.21.15.115:0/2410074954 conn(0x7fcb98075910 msgr2=0x7fcb9810abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:51:31.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.150+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 shutdown_connections 2024-03-22T10:51:31.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:51:31.150+0000 7fcb9d400700 1 -- 172.21.15.115:0/2410074954 wait complete. 2024-03-22T10:51:31.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:30 smithi150 ceph-mon[65513]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:51:31.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:30 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2906292798' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:31.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:31.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:30 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:31.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:30 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1624683998' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:51:32.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:31 smithi115 ceph-mon[78672]: pgmap v260: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:32.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:31 smithi115 ceph-mon[78672]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:51:32.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:31 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2410074954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:51:32.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:31 smithi150 ceph-mon[65513]: pgmap v260: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:32.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:31 smithi150 ceph-mon[65513]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:51:32.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:31 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2410074954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:51:33.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:33 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:33.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:33 smithi115 ceph-mon[78672]: pgmap v261: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:33.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:33 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:33.548 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:33 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:33.548 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:33 smithi150 ceph-mon[65513]: pgmap v261: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:33.548 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:33 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:34.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:51:34.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:51:35.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:35 smithi150 ceph-mon[65513]: pgmap v262: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:35.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:35 smithi115 ceph-mon[78672]: pgmap v262: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:37.541 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:37 smithi150 ceph-mon[65513]: pgmap v263: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:37.759 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:37 smithi115 ceph-mon[78672]: pgmap v263: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: Detected new or changed devices on smithi150 2024-03-22T10:51:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:51:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:51:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:38.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:38.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:38.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:38.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi150.slzwwb"]}]: dispatch 2024-03-22T10:51:39.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: Detected new or changed devices on smithi150 2024-03-22T10:51:39.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:39.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:39.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:51:39.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:51:39.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:39.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:39.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:39.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:39.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:39.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:51:39.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi150.slzwwb"]}]: dispatch 2024-03-22T10:51:39.957 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:39 smithi150 ceph-mon[65513]: Upgrade: It appears safe to stop mds.cephfs.smithi150.slzwwb 2024-03-22T10:51:39.958 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:39 smithi150 ceph-mon[65513]: pgmap v264: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:40.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:39 smithi115 ceph-mon[78672]: Upgrade: It appears safe to stop mds.cephfs.smithi150.slzwwb 2024-03-22T10:51:40.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:39 smithi115 ceph-mon[78672]: pgmap v264: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:41.579 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:41 smithi150 ceph-mon[65513]: pgmap v265: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:41.579 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:41 smithi150 ceph-mon[65513]: Upgrade: Updating mds.cephfs.smithi150.slzwwb 2024-03-22T10:51:41.579 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:41.579 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.slzwwb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:51:41.579 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:51:41.579 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:41 smithi150 ceph-mon[65513]: Deploying daemon mds.cephfs.smithi150.slzwwb on smithi150 2024-03-22T10:51:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:41 smithi115 ceph-mon[78672]: pgmap v265: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:41 smithi115 ceph-mon[78672]: Upgrade: Updating mds.cephfs.smithi150.slzwwb 2024-03-22T10:51:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi150.slzwwb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T10:51:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:51:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:41 smithi115 ceph-mon[78672]: Deploying daemon mds.cephfs.smithi150.slzwwb on smithi150 2024-03-22T10:51:43.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:43 smithi150 ceph-mon[65513]: pgmap v266: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:43.661 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:43 smithi115 ceph-mon[78672]: pgmap v266: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:51:43.661 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:43 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[78668]: 2024-03-22T10:51:43.567+0000 7f7b87c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:51:44.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:44 smithi150 ceph-mon[65513]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T10:51:44.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:44 smithi150 ceph-mon[65513]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:51:44.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:44 smithi150 ceph-mon[65513]: osdmap e83: 6 total, 6 up, 6 in 2024-03-22T10:51:44.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:44 smithi150 ceph-mon[65513]: Standby daemon mds.cephfs.smithi115.ggxfzi assigned to filesystem cephfs as rank 0 2024-03-22T10:51:44.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:44 smithi150 ceph-mon[65513]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T10:51:44.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:44 smithi150 ceph-mon[65513]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T10:51:44.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:44 smithi150 ceph-mon[65513]: fsmap cephfs:1/1 {0=cephfs.smithi115.ggxfzi=up:replay} 2 up:standby 2024-03-22T10:51:44.685 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:44 smithi115 ceph-mon[78672]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T10:51:44.686 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:44 smithi115 ceph-mon[78672]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T10:51:44.686 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:44 smithi115 ceph-mon[78672]: osdmap e83: 6 total, 6 up, 6 in 2024-03-22T10:51:44.686 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:44 smithi115 ceph-mon[78672]: Standby daemon mds.cephfs.smithi115.ggxfzi assigned to filesystem cephfs as rank 0 2024-03-22T10:51:44.686 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:44 smithi115 ceph-mon[78672]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T10:51:44.686 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:44 smithi115 ceph-mon[78672]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-22T10:51:44.686 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:44 smithi115 ceph-mon[78672]: fsmap cephfs:1/1 {0=cephfs.smithi115.ggxfzi=up:replay} 2 up:standby 2024-03-22T10:51:45.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:45 smithi150 ceph-mon[65513]: pgmap v268: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-03-22T10:51:45.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:45 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:51:46.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:45 smithi115 ceph-mon[78672]: pgmap v268: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-03-22T10:51:46.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:45 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:51:47.897 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:47 smithi150 ceph-mon[65513]: pgmap v269: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s rd, 2 op/s 2024-03-22T10:51:48.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:47 smithi115 ceph-mon[78672]: pgmap v269: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s rd, 2 op/s 2024-03-22T10:51:49.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:49 smithi150 ceph-mon[65513]: pgmap v270: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-03-22T10:51:49.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:49.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:51:50.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:49 smithi115 ceph-mon[78672]: pgmap v270: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-03-22T10:51:50.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:50.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:51:51.498 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:51 smithi150 ceph-mon[65513]: pgmap v271: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-22T10:51:51.498 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:51 smithi150 ceph-mon[65513]: reconnect by client.14516 192.168.0.1:0/629163332 after 0.00299994 2024-03-22T10:51:51.498 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:51 smithi150 ceph-mon[65513]: reconnect by client.24289 192.168.0.1:0/3857176443 after 0.00299994 2024-03-22T10:51:51.498 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:51 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] up:reconnect 2024-03-22T10:51:51.498 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:51 smithi150 ceph-mon[65513]: fsmap cephfs:1/1 {0=cephfs.smithi115.ggxfzi=up:reconnect} 2 up:standby 2024-03-22T10:51:51.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:51 smithi115 ceph-mon[78672]: pgmap v271: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-22T10:51:51.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:51 smithi115 ceph-mon[78672]: reconnect by client.14516 192.168.0.1:0/629163332 after 0.00299994 2024-03-22T10:51:51.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:51 smithi115 ceph-mon[78672]: reconnect by client.24289 192.168.0.1:0/3857176443 after 0.00299994 2024-03-22T10:51:51.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:51 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] up:reconnect 2024-03-22T10:51:51.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:51 smithi115 ceph-mon[78672]: fsmap cephfs:1/1 {0=cephfs.smithi115.ggxfzi=up:reconnect} 2 up:standby 2024-03-22T10:51:52.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:52 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] up:rejoin 2024-03-22T10:51:52.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:52 smithi150 ceph-mon[65513]: fsmap cephfs:1/1 {0=cephfs.smithi115.ggxfzi=up:rejoin} 2 up:standby 2024-03-22T10:51:52.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:52 smithi150 ceph-mon[65513]: daemon mds.cephfs.smithi115.ggxfzi is now active in filesystem cephfs as rank 0 2024-03-22T10:51:52.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:52.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:52.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:52 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:52.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:52 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] up:rejoin 2024-03-22T10:51:52.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:52 smithi115 ceph-mon[78672]: fsmap cephfs:1/1 {0=cephfs.smithi115.ggxfzi=up:rejoin} 2 up:standby 2024-03-22T10:51:52.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:52 smithi115 ceph-mon[78672]: daemon mds.cephfs.smithi115.ggxfzi is now active in filesystem cephfs as rank 0 2024-03-22T10:51:52.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:52.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:52.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:52 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:51:53.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:53 smithi150 ceph-mon[65513]: pgmap v272: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 102 B/s wr, 5 op/s 2024-03-22T10:51:53.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:53 smithi150 ceph-mon[65513]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T10:51:53.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:53 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] up:active 2024-03-22T10:51:53.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:53 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.150:6826/2053213512,v1:172.21.15.150:6827/2053213512] up:boot 2024-03-22T10:51:53.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:53 smithi150 ceph-mon[65513]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:active} 3 up:standby 2024-03-22T10:51:53.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:53 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:51:53.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:53 smithi115 ceph-mon[78672]: pgmap v272: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 102 B/s wr, 5 op/s 2024-03-22T10:51:53.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:53 smithi115 ceph-mon[78672]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T10:51:53.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:53 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] up:active 2024-03-22T10:51:53.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:53 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.150:6826/2053213512,v1:172.21.15.150:6827/2053213512] up:boot 2024-03-22T10:51:53.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:53 smithi115 ceph-mon[78672]: fsmap cephfs:1 {0=cephfs.smithi115.ggxfzi=up:active} 3 up:standby 2024-03-22T10:51:53.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:53 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:51:55.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:55 smithi150 ceph-mon[65513]: pgmap v273: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 95 B/s wr, 6 op/s 2024-03-22T10:51:55.679 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:55 smithi115 ceph-mon[78672]: pgmap v273: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 95 B/s wr, 6 op/s 2024-03-22T10:51:57.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:57 smithi150 ceph-mon[65513]: pgmap v274: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-22T10:51:57.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:57 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:57.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:57 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:57.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:57 smithi115 ceph-mon[78672]: pgmap v274: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-22T10:51:57.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:57 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:57.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:57 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:59.618 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:59 smithi150 ceph-mon[65513]: pgmap v275: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-03-22T10:51:59.618 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:59.619 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:51:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:59.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:59 smithi115 ceph-mon[78672]: pgmap v275: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-03-22T10:51:59.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:51:59.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:51:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.317+0000 7f16c4400700 1 -- 172.21.15.115:0/1836601201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1067a0 msgr2=0x7f16bc106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.317+0000 7f16c4400700 1 --2- 172.21.15.115:0/1836601201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1067a0 0x7f16bc106bb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009910 tx=0x7f16b0009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.317+0000 7f16c4400700 1 -- 172.21.15.115:0/1836601201 shutdown_connections 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.317+0000 7f16c4400700 1 --2- 172.21.15.115:0/1836601201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc1079a0 0x7f16bc107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.317+0000 7f16c4400700 1 --2- 172.21.15.115:0/1836601201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1067a0 0x7f16bc106bb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.317+0000 7f16c4400700 1 -- 172.21.15.115:0/1836601201 >> 172.21.15.115:0/1836601201 conn(0x7f16bc075910 msgr2=0x7f16bc077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.317+0000 7f16c4400700 1 -- 172.21.15.115:0/1836601201 shutdown_connections 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.317+0000 7f16c4400700 1 -- 172.21.15.115:0/1836601201 wait complete. 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.318+0000 7f16c4400700 1 Processor -- start 2024-03-22T10:52:01.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.318+0000 7f16c4400700 1 -- start start 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c4400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc1067a0 0x7f16bc19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c4400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1079a0 0x7f16bc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c4400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19cb10 con 0x7f16bc1079a0 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c4400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19cc50 con 0x7f16bc1067a0 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1079a0 0x7f16bc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1079a0 0x7f16bc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40320/0 (socket says 172.21.15.115:40320) 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c1000700 1 -- 172.21.15.115:0/1962223334 learned_addr learned my addr 172.21.15.115:0/1962223334 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c1a00700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc1067a0 0x7f16bc19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c1000700 1 -- 172.21.15.115:0/1962223334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc1067a0 msgr2=0x7f16bc19bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c1000700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc1067a0 0x7f16bc19bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.319+0000 7f16c1000700 1 -- 172.21.15.115:0/1962223334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b00095f0 con 0x7f16bc1079a0 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.320+0000 7f16c1a00700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc1067a0 0x7f16bc19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:52:01.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.320+0000 7f16c1000700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1079a0 0x7f16bc19c4f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d8d0 tx=0x7f16a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:01.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.320+0000 7f16ba000700 1 -- 172.21.15.115:0/1962223334 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a8009940 con 0x7f16bc1079a0 2024-03-22T10:52:01.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.320+0000 7f16ba000700 1 -- 172.21.15.115:0/1962223334 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f16a8010460 con 0x7f16bc1079a0 2024-03-22T10:52:01.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.320+0000 7f16ba000700 1 -- 172.21.15.115:0/1962223334 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a800f5f0 con 0x7f16bc1079a0 2024-03-22T10:52:01.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.320+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc1a1700 con 0x7f16bc1079a0 2024-03-22T10:52:01.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.320+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc1a1c50 con 0x7f16bc1079a0 2024-03-22T10:52:01.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.321+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc10f2f0 con 0x7f16bc1079a0 2024-03-22T10:52:01.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.323+0000 7f16ba000700 1 -- 172.21.15.115:0/1962223334 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f16a80105d0 con 0x7f16bc1079a0 2024-03-22T10:52:01.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.323+0000 7f16ba000700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f16a406dfa0 0x7f16a4070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:01.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.324+0000 7f16ba000700 1 -- 172.21.15.115:0/1962223334 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f16a808d7a0 con 0x7f16bc1079a0 2024-03-22T10:52:01.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.324+0000 7f16c1a00700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f16a406dfa0 0x7f16a4070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:01.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.324+0000 7f16c1a00700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f16a406dfa0 0x7f16a4070450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006010 tx=0x7f16b000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:01.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.332+0000 7f16ba000700 1 -- 172.21.15.115:0/1962223334 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16a805a720 con 0x7f16bc1079a0 2024-03-22T10:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.582+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16bc10c2f0 con 0x7f16a406dfa0 2024-03-22T10:52:01.574 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:01 smithi115 ceph-mon[78672]: pgmap v276: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-22T10:52:01.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.584+0000 7f16ba000700 1 -- 172.21.15.115:0/1962223334 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f16bc10c2f0 con 0x7f16a406dfa0 2024-03-22T10:52:01.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f16a406dfa0 msgr2=0x7f16a4070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:01.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f16a406dfa0 0x7f16a4070450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006010 tx=0x7f16b000b540 comp rx=0 tx=0).stop 2024-03-22T10:52:01.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1079a0 msgr2=0x7f16bc19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:01.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1079a0 0x7f16bc19c4f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d8d0 tx=0x7f16a800dc90 comp rx=0 tx=0).stop 2024-03-22T10:52:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 shutdown_connections 2024-03-22T10:52:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f16a406dfa0 0x7f16a4070450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc1067a0 0x7f16bc19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 --2- 172.21.15.115:0/1962223334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16bc1079a0 0x7f16bc19c4f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 >> 172.21.15.115:0/1962223334 conn(0x7f16bc075910 msgr2=0x7f16bc10abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 shutdown_connections 2024-03-22T10:52:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.588+0000 7f16c4400700 1 -- 172.21.15.115:0/1962223334 wait complete. 2024-03-22T10:52:01.596 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:52:01.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:01 smithi150 ceph-mon[65513]: pgmap v276: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-22T10:52:01.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.746+0000 7f8b31600700 1 -- 172.21.15.115:0/3186658479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 msgr2=0x7f8b2c0ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:01.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.746+0000 7f8b31600700 1 --2- 172.21.15.115:0/3186658479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 0x7f8b2c0ff2a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009910 tx=0x7f8b1c009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:01.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.746+0000 7f8b31600700 1 -- 172.21.15.115:0/3186658479 shutdown_connections 2024-03-22T10:52:01.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.746+0000 7f8b31600700 1 --2- 172.21.15.115:0/3186658479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b2c0ffff0 0x7f8b2c100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:01.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.746+0000 7f8b31600700 1 --2- 172.21.15.115:0/3186658479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 0x7f8b2c0ff2a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:01.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.746+0000 7f8b31600700 1 -- 172.21.15.115:0/3186658479 >> 172.21.15.115:0/3186658479 conn(0x7f8b2c0fa400 msgr2=0x7f8b2c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:01.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.746+0000 7f8b31600700 1 -- 172.21.15.115:0/3186658479 shutdown_connections 2024-03-22T10:52:01.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.746+0000 7f8b31600700 1 -- 172.21.15.115:0/3186658479 wait complete. 2024-03-22T10:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.748+0000 7f8b31600700 1 Processor -- start 2024-03-22T10:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.748+0000 7f8b31600700 1 -- start start 2024-03-22T10:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.748+0000 7f8b31600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 0x7f8b2c069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.748+0000 7f8b31600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b2c0ffff0 0x7f8b2c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.748+0000 7f8b31600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c19aa80 con 0x7f8b2c0fee90 2024-03-22T10:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.749+0000 7f8b31600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c19abc0 con 0x7f8b2c0ffff0 2024-03-22T10:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.749+0000 7f8b23e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 0x7f8b2c069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.749+0000 7f8b23e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 0x7f8b2c069820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40338/0 (socket says 172.21.15.115:40338) 2024-03-22T10:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.749+0000 7f8b23e00700 1 -- 172.21.15.115:0/990188110 learned_addr learned my addr 172.21.15.115:0/990188110 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.749+0000 7f8b2be00700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b2c0ffff0 0x7f8b2c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.749+0000 7f8b23e00700 1 -- 172.21.15.115:0/990188110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b2c0ffff0 msgr2=0x7f8b2c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.749+0000 7f8b23e00700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b2c0ffff0 0x7f8b2c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.749+0000 7f8b23e00700 1 -- 172.21.15.115:0/990188110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c0095f0 con 0x7f8b2c0fee90 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.749+0000 7f8b2be00700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b2c0ffff0 0x7f8b2c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.750+0000 7f8b23e00700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 0x7f8b2c069820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00bd70 tx=0x7f8b1c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.750+0000 7f8b29600700 1 -- 172.21.15.115:0/990188110 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1c01d070 con 0x7f8b2c0fee90 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.750+0000 7f8b29600700 1 -- 172.21.15.115:0/990188110 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8b1c022470 con 0x7f8b2c0fee90 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.750+0000 7f8b29600700 1 -- 172.21.15.115:0/990188110 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1c00f670 con 0x7f8b2c0fee90 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.750+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c19ae40 con 0x7f8b2c0fee90 2024-03-22T10:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.750+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2c19b2b0 con 0x7f8b2c0fee90 2024-03-22T10:52:01.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.752+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2c107960 con 0x7f8b2c0fee90 2024-03-22T10:52:01.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.752+0000 7f8b29600700 1 -- 172.21.15.115:0/990188110 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f8b1c022a50 con 0x7f8b2c0fee90 2024-03-22T10:52:01.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.753+0000 7f8b29600700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b1006dfa0 0x7f8b10070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:01.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.753+0000 7f8b29600700 1 -- 172.21.15.115:0/990188110 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8b1c08f3d0 con 0x7f8b2c0fee90 2024-03-22T10:52:01.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.754+0000 7f8b2be00700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b1006dfa0 0x7f8b10070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.754+0000 7f8b2be00700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b1006dfa0 0x7f8b10070450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b14005fd0 tx=0x7f8b14005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:01.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:01.759+0000 7f8b29600700 1 -- 172.21.15.115:0/990188110 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b1c05c4a0 con 0x7f8b2c0fee90 2024-03-22T10:52:02.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.013+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b2c1049d0 con 0x7f8b1006dfa0 2024-03-22T10:52:02.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.015+0000 7f8b29600700 1 -- 172.21.15.115:0/990188110 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8b2c1049d0 con 0x7f8b1006dfa0 2024-03-22T10:52:02.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b1006dfa0 msgr2=0x7f8b10070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b1006dfa0 0x7f8b10070450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b14005fd0 tx=0x7f8b14005ee0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 msgr2=0x7f8b2c069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 0x7f8b2c069820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00bd70 tx=0x7f8b1c00be50 comp rx=0 tx=0).stop 2024-03-22T10:52:02.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 shutdown_connections 2024-03-22T10:52:02.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b1006dfa0 0x7f8b10070450 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b2c0fee90 0x7f8b2c069820 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 --2- 172.21.15.115:0/990188110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b2c0ffff0 0x7f8b2c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 >> 172.21.15.115:0/990188110 conn(0x7f8b2c0fa400 msgr2=0x7f8b2c1032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:02.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 shutdown_connections 2024-03-22T10:52:02.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.018+0000 7f8b31600700 1 -- 172.21.15.115:0/990188110 wait complete. 2024-03-22T10:52:02.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.170+0000 7fe6ae600700 1 -- 172.21.15.115:0/2876578913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 msgr2=0x7fe6a8100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.170+0000 7fe6ae600700 1 --2- 172.21.15.115:0/2876578913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 0x7fe6a8100560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe698009910 tx=0x7fe698009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:02.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.171+0000 7fe6ae600700 1 -- 172.21.15.115:0/2876578913 shutdown_connections 2024-03-22T10:52:02.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.171+0000 7fe6ae600700 1 --2- 172.21.15.115:0/2876578913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6a8101350 0x7fe6a81017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.171+0000 7fe6ae600700 1 --2- 172.21.15.115:0/2876578913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 0x7fe6a8100560 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.171+0000 7fe6ae600700 1 -- 172.21.15.115:0/2876578913 >> 172.21.15.115:0/2876578913 conn(0x7fe6a80fb6e0 msgr2=0x7fe6a80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:02.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.171+0000 7fe6ae600700 1 -- 172.21.15.115:0/2876578913 shutdown_connections 2024-03-22T10:52:02.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.171+0000 7fe6ae600700 1 -- 172.21.15.115:0/2876578913 wait complete. 2024-03-22T10:52:02.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.172+0000 7fe6ae600700 1 Processor -- start 2024-03-22T10:52:02.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.172+0000 7fe6ae600700 1 -- start start 2024-03-22T10:52:02.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.172+0000 7fe6ae600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 0x7fe6a8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:02.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.172+0000 7fe6ae600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6a8101350 0x7fe6a81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:02.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.172+0000 7fe6ae600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a8198700 con 0x7fe6a8100150 2024-03-22T10:52:02.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.172+0000 7fe6ae600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a8198840 con 0x7fe6a8101350 2024-03-22T10:52:02.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.173+0000 7fe6a7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 0x7fe6a8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:02.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.173+0000 7fe6a7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 0x7fe6a8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49382/0 (socket says 172.21.15.115:49382) 2024-03-22T10:52:02.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.173+0000 7fe6a7400700 1 -- 172.21.15.115:0/1714398353 learned_addr learned my addr 172.21.15.115:0/1714398353 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:02.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.173+0000 7fe6a6a00700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6a8101350 0x7fe6a81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:02.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.173+0000 7fe6a7400700 1 -- 172.21.15.115:0/1714398353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6a8101350 msgr2=0x7fe6a81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.173+0000 7fe6a7400700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6a8101350 0x7fe6a81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.173+0000 7fe6a7400700 1 -- 172.21.15.115:0/1714398353 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980095f0 con 0x7fe6a8100150 2024-03-22T10:52:02.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.173+0000 7fe6a7400700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 0x7fe6a8197ba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe69800bd70 tx=0x7fe69800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:02.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.174+0000 7fe69fe00700 1 -- 172.21.15.115:0/1714398353 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe69801d070 con 0x7fe6a8100150 2024-03-22T10:52:02.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.174+0000 7fe69fe00700 1 -- 172.21.15.115:0/1714398353 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe69800f460 con 0x7fe6a8100150 2024-03-22T10:52:02.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.174+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a819d290 con 0x7fe6a8100150 2024-03-22T10:52:02.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.174+0000 7fe69fe00700 1 -- 172.21.15.115:0/1714398353 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe698021640 con 0x7fe6a8100150 2024-03-22T10:52:02.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.174+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a819d780 con 0x7fe6a8100150 2024-03-22T10:52:02.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.175+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a8105430 con 0x7fe6a8100150 2024-03-22T10:52:02.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.176+0000 7fe69fe00700 1 -- 172.21.15.115:0/1714398353 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe69800fab0 con 0x7fe6a8100150 2024-03-22T10:52:02.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.178+0000 7fe69fe00700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe68c06dfa0 0x7fe68c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:02.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.178+0000 7fe69fe00700 1 -- 172.21.15.115:0/1714398353 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe69808f1d0 con 0x7fe6a8100150 2024-03-22T10:52:02.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.178+0000 7fe6a6a00700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe68c06dfa0 0x7fe68c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:02.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.179+0000 7fe6a6a00700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe68c06dfa0 0x7fe68c070450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe694005fd0 tx=0x7fe69400a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:02.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.185+0000 7fe69fe00700 1 -- 172.21.15.115:0/1714398353 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe69805c150 con 0x7fe6a8100150 2024-03-22T10:52:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.425+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6a8061190 con 0x7fe68c06dfa0 2024-03-22T10:52:02.416 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:02 smithi115 ceph-mon[78672]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:02.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.438+0000 7fe69fe00700 1 -- 172.21.15.115:0/1714398353 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe6a8061190 con 0x7fe68c06dfa0 2024-03-22T10:52:02.428 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:52:02.428 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (18m) 56s ago 20m 14.7M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:52:02.428 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (20m) 56s ago 20m 9323k - 18.2.0 dc2bc1663786 8130761e2d41 2024-03-22T10:52:02.428 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (19m) 6s ago 19m 9281k - 18.2.0 dc2bc1663786 5fbb5f1c5f83 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (6m) 56s ago 20m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (6m) 6s ago 19m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (18m) 56s ago 20m 73.8M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (92s) 56s ago 13m 12.0M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 44a0769ebe8e 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (64s) 56s ago 13m 10.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51d700f6eb18 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (11s) 6s ago 13m 10.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 82b6b216f754 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (37s) 6s ago 13m 11.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9605445d5b2b 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (9m) 56s ago 22m 512M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (8m) 6s ago 19m 452M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (8m) 56s ago 22m 55.1M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (7m) 6s ago 18m 43.3M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (20m) 56s ago 20m 8921k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (19m) 6s ago 19m 8208k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (5m) 56s ago 18m 96.9M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:52:02.429 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (5m) 56s ago 17m 95.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:52:02.430 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (4m) 56s ago 16m 83.0M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:52:02.430 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (4m) 6s ago 16m 103M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:52:02.430 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (3m) 6s ago 15m 90.5M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:52:02.430 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (2m) 6s ago 15m 90.5M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:52:02.430 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (8m) 56s ago 19m 74.3M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.440+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe68c06dfa0 msgr2=0x7fe68c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.440+0000 7fe6ae600700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe68c06dfa0 0x7fe68c070450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe694005fd0 tx=0x7fe69400a560 comp rx=0 tx=0).stop 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.440+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 msgr2=0x7fe6a8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.440+0000 7fe6ae600700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 0x7fe6a8197ba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe69800bd70 tx=0x7fe69800be50 comp rx=0 tx=0).stop 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.441+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 shutdown_connections 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.441+0000 7fe6ae600700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe68c06dfa0 0x7fe68c070450 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.441+0000 7fe6ae600700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a8100150 0x7fe6a8197ba0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.441+0000 7fe6ae600700 1 --2- 172.21.15.115:0/1714398353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6a8101350 0x7fe6a81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.441+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 >> 172.21.15.115:0/1714398353 conn(0x7fe6a80fb6e0 msgr2=0x7fe6a8102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.441+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 shutdown_connections 2024-03-22T10:52:02.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.441+0000 7fe6ae600700 1 -- 172.21.15.115:0/1714398353 wait complete. 2024-03-22T10:52:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.593+0000 7f8d5be00700 1 -- 172.21.15.115:0/811307012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54100150 msgr2=0x7f8d54100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.593+0000 7f8d5be00700 1 --2- 172.21.15.115:0/811307012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54100150 0x7f8d54100560 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009910 tx=0x7f8d48009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.593+0000 7f8d5be00700 1 -- 172.21.15.115:0/811307012 shutdown_connections 2024-03-22T10:52:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.593+0000 7f8d5be00700 1 --2- 172.21.15.115:0/811307012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d54101350 0x7f8d541017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.593+0000 7f8d5be00700 1 --2- 172.21.15.115:0/811307012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54100150 0x7f8d54100560 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.593+0000 7f8d5be00700 1 -- 172.21.15.115:0/811307012 >> 172.21.15.115:0/811307012 conn(0x7f8d540fb6e0 msgr2=0x7f8d540fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.593+0000 7f8d5be00700 1 -- 172.21.15.115:0/811307012 shutdown_connections 2024-03-22T10:52:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.593+0000 7f8d5be00700 1 -- 172.21.15.115:0/811307012 wait complete. 2024-03-22T10:52:02.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.595+0000 7f8d5be00700 1 Processor -- start 2024-03-22T10:52:02.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.595+0000 7f8d5be00700 1 -- start start 2024-03-22T10:52:02.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.595+0000 7f8d5be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d54100150 0x7f8d541959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:02.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.595+0000 7f8d5be00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54101350 0x7f8d54195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:02.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.595+0000 7f8d5be00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d54196500 con 0x7f8d54101350 2024-03-22T10:52:02.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.595+0000 7f8d5be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d54196640 con 0x7f8d54100150 2024-03-22T10:52:02.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.596+0000 7f8d58a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54101350 0x7f8d54195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:02.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.596+0000 7f8d58a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54101350 0x7f8d54195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49394/0 (socket says 172.21.15.115:49394) 2024-03-22T10:52:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.596+0000 7f8d58a00700 1 -- 172.21.15.115:0/4173837174 learned_addr learned my addr 172.21.15.115:0/4173837174 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.596+0000 7f8d59400700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d54100150 0x7f8d541959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.596+0000 7f8d58a00700 1 -- 172.21.15.115:0/4173837174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d54100150 msgr2=0x7f8d541959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.596+0000 7f8d58a00700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d54100150 0x7f8d541959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.596+0000 7f8d58a00700 1 -- 172.21.15.115:0/4173837174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d480095f0 con 0x7f8d54101350 2024-03-22T10:52:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.597+0000 7f8d58a00700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54101350 0x7f8d54195ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000e920 tx=0x7f8d4000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.597+0000 7f8d52000700 1 -- 172.21.15.115:0/4173837174 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4000c9d0 con 0x7f8d54101350 2024-03-22T10:52:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.597+0000 7f8d52000700 1 -- 172.21.15.115:0/4173837174 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8d40010430 con 0x7f8d54101350 2024-03-22T10:52:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.597+0000 7f8d52000700 1 -- 172.21.15.115:0/4173837174 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d40018650 con 0x7f8d54101350 2024-03-22T10:52:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.597+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5419b0f0 con 0x7f8d54101350 2024-03-22T10:52:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.597+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5419b640 con 0x7f8d54101350 2024-03-22T10:52:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.598+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d54108ca0 con 0x7f8d54101350 2024-03-22T10:52:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.600+0000 7f8d52000700 1 -- 172.21.15.115:0/4173837174 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f8d400105a0 con 0x7f8d54101350 2024-03-22T10:52:02.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.601+0000 7f8d52000700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8d3c06dfa0 0x7f8d3c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:02.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.601+0000 7f8d52000700 1 -- 172.21.15.115:0/4173837174 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8d40014070 con 0x7f8d54101350 2024-03-22T10:52:02.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.601+0000 7f8d59400700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8d3c06dfa0 0x7f8d3c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:02.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.602+0000 7f8d59400700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8d3c06dfa0 0x7f8d3c070450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009fd0 tx=0x7f8d48005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.606+0000 7f8d52000700 1 -- 172.21.15.115:0/4173837174 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8d40053740 con 0x7f8d54101350 2024-03-22T10:52:02.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:02 smithi150 ceph-mon[65513]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:02.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.917+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d5419bcd0 con 0x7f8d54101350 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.917+0000 7f8d52000700 1 -- 172.21.15.115:0/4173837174 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8d4005afe0 con 0x7f8d54101350 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:52:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:52:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:52:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:52:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:52:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-22T10:52:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:52:02.909 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:52:02.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.920+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8d3c06dfa0 msgr2=0x7f8d3c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.920+0000 7f8d5be00700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8d3c06dfa0 0x7f8d3c070450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009fd0 tx=0x7f8d48005fd0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.920+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54101350 msgr2=0x7f8d54195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:02.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.920+0000 7f8d5be00700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54101350 0x7f8d54195ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000e920 tx=0x7f8d4000ec30 comp rx=0 tx=0).stop 2024-03-22T10:52:02.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.921+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 shutdown_connections 2024-03-22T10:52:02.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.921+0000 7f8d5be00700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8d3c06dfa0 0x7f8d3c070450 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.921+0000 7f8d5be00700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d54100150 0x7f8d541959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.921+0000 7f8d5be00700 1 --2- 172.21.15.115:0/4173837174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d54101350 0x7f8d54195ee0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:02.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.921+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 >> 172.21.15.115:0/4173837174 conn(0x7f8d540fb6e0 msgr2=0x7f8d54104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:02.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.921+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 shutdown_connections 2024-03-22T10:52:02.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:02.921+0000 7f8d5be00700 1 -- 172.21.15.115:0/4173837174 wait complete. 2024-03-22T10:52:03.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.087+0000 7ff9d1000700 1 -- 172.21.15.115:0/608226126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 msgr2=0x7ff9cc0ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:03.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.087+0000 7ff9d1000700 1 --2- 172.21.15.115:0/608226126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 0x7ff9cc0ff490 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009910 tx=0x7ff9bc009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:03.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.087+0000 7ff9d1000700 1 -- 172.21.15.115:0/608226126 shutdown_connections 2024-03-22T10:52:03.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.087+0000 7ff9d1000700 1 --2- 172.21.15.115:0/608226126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9cc100280 0x7ff9cc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.087+0000 7ff9d1000700 1 --2- 172.21.15.115:0/608226126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 0x7ff9cc0ff490 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.087+0000 7ff9d1000700 1 -- 172.21.15.115:0/608226126 >> 172.21.15.115:0/608226126 conn(0x7ff9cc0fa630 msgr2=0x7ff9cc0fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:03.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.087+0000 7ff9d1000700 1 -- 172.21.15.115:0/608226126 shutdown_connections 2024-03-22T10:52:03.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.087+0000 7ff9d1000700 1 -- 172.21.15.115:0/608226126 wait complete. 2024-03-22T10:52:03.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.088+0000 7ff9d1000700 1 Processor -- start 2024-03-22T10:52:03.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.088+0000 7ff9d1000700 1 -- start start 2024-03-22T10:52:03.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9d1000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 0x7ff9cc103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9d1000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9cc100280 0x7ff9cc101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9d1000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc101ba0 con 0x7ff9cc0ff080 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9d1000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc101ce0 con 0x7ff9cc100280 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9c9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 0x7ff9cc103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9c9e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 0x7ff9cc103010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49402/0 (socket says 172.21.15.115:49402) 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9c9e00700 1 -- 172.21.15.115:0/726190321 learned_addr learned my addr 172.21.15.115:0/726190321 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9c9400700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9cc100280 0x7ff9cc101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9c9e00700 1 -- 172.21.15.115:0/726190321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9cc100280 msgr2=0x7ff9cc101660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9c9e00700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9cc100280 0x7ff9cc101660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9c9e00700 1 -- 172.21.15.115:0/726190321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc0095f0 con 0x7ff9cc0ff080 2024-03-22T10:52:03.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.089+0000 7ff9c9400700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9cc100280 0x7ff9cc101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:52:03.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.090+0000 7ff9c9e00700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 0x7ff9cc103010 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00bda0 tx=0x7ff9bc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:03.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.090+0000 7ff9c2a00700 1 -- 172.21.15.115:0/726190321 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9bc01d070 con 0x7ff9cc0ff080 2024-03-22T10:52:03.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.090+0000 7ff9c2a00700 1 -- 172.21.15.115:0/726190321 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff9bc004c50 con 0x7ff9cc0ff080 2024-03-22T10:52:03.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.090+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc101f60 con 0x7ff9cc0ff080 2024-03-22T10:52:03.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.090+0000 7ff9c2a00700 1 -- 172.21.15.115:0/726190321 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9bc00f690 con 0x7ff9cc0ff080 2024-03-22T10:52:03.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.090+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc102450 con 0x7ff9cc0ff080 2024-03-22T10:52:03.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.092+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc107bd0 con 0x7ff9cc0ff080 2024-03-22T10:52:03.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.093+0000 7ff9c2a00700 1 -- 172.21.15.115:0/726190321 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff9bc0229b0 con 0x7ff9cc0ff080 2024-03-22T10:52:03.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.094+0000 7ff9c2a00700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff9b006dfa0 0x7ff9b0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:03.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.094+0000 7ff9c2a00700 1 -- 172.21.15.115:0/726190321 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff9bc08f630 con 0x7ff9cc0ff080 2024-03-22T10:52:03.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.095+0000 7ff9c9400700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff9b006dfa0 0x7ff9b0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:03.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.095+0000 7ff9c9400700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff9b006dfa0 0x7ff9b0070450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005fd0 tx=0x7ff9b8005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:03.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.099+0000 7ff9c2a00700 1 -- 172.21.15.115:0/726190321 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9bc05c650 con 0x7ff9cc0ff080 2024-03-22T10:52:03.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.395+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff9cc19e300 con 0x7ff9cc0ff080 2024-03-22T10:52:03.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.396+0000 7ff9c2a00700 1 -- 172.21.15.115:0/726190321 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1918 (secure 0 0 0) 0x7ff9bc027790 con 0x7ff9cc0ff080 2024-03-22T10:52:03.386 INFO:teuthology.orchestra.run.smithi115.stdout:e38 2024-03-22T10:52:03.386 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:52:03.386 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:52:03.386 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 38 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:51:52.280217+0000 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:52:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 83 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 1 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:in 0 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=34476} 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 1 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:52:03.388 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 34476 members: 34476 2024-03-22T10:52:03.389 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:34476} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:52:03.389 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:52:03.389 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:52:03.389 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:52:03.389 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:52:03.389 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{-1:34508} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:52:03.389 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6824/1804848703,v1:172.21.15.150:6825/1804848703] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:52:03.389 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6826/2053213512,v1:172.21.15.150:6827/2053213512] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.398+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff9b006dfa0 msgr2=0x7ff9b0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.398+0000 7ff9d1000700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff9b006dfa0 0x7ff9b0070450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005fd0 tx=0x7ff9b8005dc0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.398+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 msgr2=0x7ff9cc103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.398+0000 7ff9d1000700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 0x7ff9cc103010 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00bda0 tx=0x7ff9bc00be80 comp rx=0 tx=0).stop 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.399+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 shutdown_connections 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.399+0000 7ff9d1000700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff9b006dfa0 0x7ff9b0070450 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.399+0000 7ff9d1000700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9cc0ff080 0x7ff9cc103010 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.399+0000 7ff9d1000700 1 --2- 172.21.15.115:0/726190321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9cc100280 0x7ff9cc101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.399+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 >> 172.21.15.115:0/726190321 conn(0x7ff9cc0fa630 msgr2=0x7ff9cc068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.399+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 shutdown_connections 2024-03-22T10:52:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.399+0000 7ff9d1000700 1 -- 172.21.15.115:0/726190321 wait complete. 2024-03-22T10:52:03.391 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 38 2024-03-22T10:52:03.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.565+0000 7f3828e00700 1 -- 172.21.15.115:0/4266080245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 msgr2=0x7f38241015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:03.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.565+0000 7f3828e00700 1 --2- 172.21.15.115:0/4266080245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 0x7f38241015e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3810009910 tx=0x7f3810009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:03.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.565+0000 7f3828e00700 1 -- 172.21.15.115:0/4266080245 shutdown_connections 2024-03-22T10:52:03.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.565+0000 7f3828e00700 1 --2- 172.21.15.115:0/4266080245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 0x7f38241015e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.565+0000 7f3828e00700 1 --2- 172.21.15.115:0/4266080245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38240fff90 0x7f38241003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.565+0000 7f3828e00700 1 -- 172.21.15.115:0/4266080245 >> 172.21.15.115:0/4266080245 conn(0x7f38240fb500 msgr2=0x7f38240fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:03.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.565+0000 7f3828e00700 1 -- 172.21.15.115:0/4266080245 shutdown_connections 2024-03-22T10:52:03.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.565+0000 7f3828e00700 1 -- 172.21.15.115:0/4266080245 wait complete. 2024-03-22T10:52:03.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.566+0000 7f3828e00700 1 Processor -- start 2024-03-22T10:52:03.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.566+0000 7f3828e00700 1 -- start start 2024-03-22T10:52:03.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3828e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38240fff90 0x7f38241979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:03.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3828e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 0x7f3824197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:03.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3828e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3824198510 con 0x7f3824101190 2024-03-22T10:52:03.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3828e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3824198650 con 0x7f38240fff90 2024-03-22T10:52:03.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3822a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 0x7f3824197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:03.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3822a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 0x7f3824197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49420/0 (socket says 172.21.15.115:49420) 2024-03-22T10:52:03.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3822a00700 1 -- 172.21.15.115:0/1501115671 learned_addr learned my addr 172.21.15.115:0/1501115671 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:03.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3823400700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38240fff90 0x7f38241979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3822a00700 1 -- 172.21.15.115:0/1501115671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38240fff90 msgr2=0x7f38241979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3822a00700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38240fff90 0x7f38241979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.567+0000 7f3822a00700 1 -- 172.21.15.115:0/1501115671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38100095f0 con 0x7f3824101190 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.568+0000 7f3822a00700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 0x7f3824197ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3810005b40 tx=0x7f381000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.568+0000 7f381be00700 1 -- 172.21.15.115:0/1501115671 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381001d070 con 0x7f3824101190 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.568+0000 7f381be00700 1 -- 172.21.15.115:0/1501115671 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f381000f460 con 0x7f3824101190 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.568+0000 7f381be00700 1 -- 172.21.15.115:0/1501115671 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3810021640 con 0x7f3824101190 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.568+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382419d0a0 con 0x7f3824101190 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.568+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382419d540 con 0x7f3824101190 2024-03-22T10:52:03.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.570+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3824105270 con 0x7f3824101190 2024-03-22T10:52:03.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.571+0000 7f381be00700 1 -- 172.21.15.115:0/1501115671 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f381000fa90 con 0x7f3824101190 2024-03-22T10:52:03.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.572+0000 7f381be00700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f380806dfa0 0x7f3808070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:03.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.572+0000 7f381be00700 1 -- 172.21.15.115:0/1501115671 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f381008f1b0 con 0x7f3824101190 2024-03-22T10:52:03.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.572+0000 7f3823400700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f380806dfa0 0x7f3808070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:03.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.573+0000 7f3823400700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f380806dfa0 0x7f3808070450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3814005950 tx=0x7f38140058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:03.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.577+0000 7f381be00700 1 -- 172.21.15.115:0/1501115671 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f381005c0b0 con 0x7f3824101190 2024-03-22T10:52:03.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:03 smithi150 ceph-mon[65513]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:03.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:03 smithi150 ceph-mon[65513]: pgmap v277: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 885 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-22T10:52:03.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:03 smithi150 ceph-mon[65513]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:03.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:03 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/4173837174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:03.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:03 smithi115 ceph-mon[78672]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:03.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:03 smithi115 ceph-mon[78672]: pgmap v277: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 885 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-22T10:52:03.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:03 smithi115 ceph-mon[78672]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:03.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:03 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/4173837174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:03.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.827+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3824061190 con 0x7f380806dfa0 2024-03-22T10:52:03.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.828+0000 7f381be00700 1 -- 172.21.15.115:0/1501115671 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3824061190 con 0x7f380806dfa0 2024-03-22T10:52:03.819 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:52:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:52:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:52:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:52:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:52:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:52:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:52:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-03-22T10:52:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osd", 2024-03-22T10:52:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "mds" 2024-03-22T10:52:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:52:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "16/23 daemons upgraded", 2024-03-22T10:52:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T10:52:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:52:03.820 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:52:03.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.832+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f380806dfa0 msgr2=0x7f3808070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:03.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.832+0000 7f3828e00700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f380806dfa0 0x7f3808070450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3814005950 tx=0x7f38140058e0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.832+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 msgr2=0x7f3824197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:03.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.832+0000 7f3828e00700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 0x7f3824197ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3810005b40 tx=0x7f381000be20 comp rx=0 tx=0).stop 2024-03-22T10:52:03.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.833+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 shutdown_connections 2024-03-22T10:52:03.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.833+0000 7f3828e00700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f380806dfa0 0x7f3808070450 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.833+0000 7f3828e00700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38240fff90 0x7f38241979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.833+0000 7f3828e00700 1 --2- 172.21.15.115:0/1501115671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3824101190 0x7f3824197ef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:03.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.833+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 >> 172.21.15.115:0/1501115671 conn(0x7f38240fb500 msgr2=0x7f38241023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:03.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.833+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 shutdown_connections 2024-03-22T10:52:03.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:03.833+0000 7f3828e00700 1 -- 172.21.15.115:0/1501115671 wait complete. 2024-03-22T10:52:04.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.030+0000 7ff44ac00700 1 -- 172.21.15.115:0/1463447098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440ff2f0 msgr2=0x7ff4440ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:04.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.030+0000 7ff44ac00700 1 --2- 172.21.15.115:0/1463447098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440ff2f0 0x7ff4440ff760 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff430009910 tx=0x7ff430009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:04.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.030+0000 7ff44ac00700 1 -- 172.21.15.115:0/1463447098 shutdown_connections 2024-03-22T10:52:04.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.030+0000 7ff44ac00700 1 --2- 172.21.15.115:0/1463447098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440ff2f0 0x7ff4440ff760 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:04.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.030+0000 7ff44ac00700 1 --2- 172.21.15.115:0/1463447098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4440fe9a0 0x7ff4440fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:04.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.030+0000 7ff44ac00700 1 -- 172.21.15.115:0/1463447098 >> 172.21.15.115:0/1463447098 conn(0x7ff4440fa5f0 msgr2=0x7ff4440fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:04.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.030+0000 7ff44ac00700 1 -- 172.21.15.115:0/1463447098 shutdown_connections 2024-03-22T10:52:04.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.030+0000 7ff44ac00700 1 -- 172.21.15.115:0/1463447098 wait complete. 2024-03-22T10:52:04.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.031+0000 7ff44ac00700 1 Processor -- start 2024-03-22T10:52:04.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.031+0000 7ff44ac00700 1 -- start start 2024-03-22T10:52:04.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff44ac00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440fe9a0 0x7ff444104520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:04.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff44ac00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4440ff2f0 0x7ff444104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:04.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff44ac00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444101010 con 0x7ff4440fe9a0 2024-03-22T10:52:04.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff44ac00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444101180 con 0x7ff4440ff2f0 2024-03-22T10:52:04.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff443e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440fe9a0 0x7ff444104520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:04.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff443e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440fe9a0 0x7ff444104520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49432/0 (socket says 172.21.15.115:49432) 2024-03-22T10:52:04.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff443e00700 1 -- 172.21.15.115:0/370749532 learned_addr learned my addr 172.21.15.115:0/370749532 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff443400700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4440ff2f0 0x7ff444104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff443e00700 1 -- 172.21.15.115:0/370749532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4440ff2f0 msgr2=0x7ff444104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff443e00700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4440ff2f0 0x7ff444104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.032+0000 7ff443e00700 1 -- 172.21.15.115:0/370749532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4300095f0 con 0x7ff4440fe9a0 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.033+0000 7ff443e00700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440fe9a0 0x7ff444104520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff43400d8d0 tx=0x7ff43400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.033+0000 7ff440c00700 1 -- 172.21.15.115:0/370749532 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff434009940 con 0x7ff4440fe9a0 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.033+0000 7ff440c00700 1 -- 172.21.15.115:0/370749532 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff434010460 con 0x7ff4440fe9a0 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.033+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff444101460 con 0x7ff4440fe9a0 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.033+0000 7ff440c00700 1 -- 172.21.15.115:0/370749532 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff43400f5f0 con 0x7ff4440fe9a0 2024-03-22T10:52:04.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.033+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4441019b0 con 0x7ff4440fe9a0 2024-03-22T10:52:04.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.034+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44410ae50 con 0x7ff4440fe9a0 2024-03-22T10:52:04.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.035+0000 7ff440c00700 1 -- 172.21.15.115:0/370749532 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff4340105d0 con 0x7ff4440fe9a0 2024-03-22T10:52:04.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.036+0000 7ff440c00700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff42806dfa0 0x7ff428070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:04.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.036+0000 7ff440c00700 1 -- 172.21.15.115:0/370749532 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff43408dce0 con 0x7ff4440fe9a0 2024-03-22T10:52:04.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.037+0000 7ff443400700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff42806dfa0 0x7ff428070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:04.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.037+0000 7ff443400700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff42806dfa0 0x7ff428070450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff430006010 tx=0x7ff43000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:04.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.042+0000 7ff440c00700 1 -- 172.21.15.115:0/370749532 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff434092070 con 0x7ff4440fe9a0 2024-03-22T10:52:04.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.397+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff44410b040 con 0x7ff4440fe9a0 2024-03-22T10:52:04.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.398+0000 7ff440c00700 1 -- 172.21.15.115:0/370749532 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff4340160e0 con 0x7ff4440fe9a0 2024-03-22T10:52:04.388 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:52:04.388 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:52:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:52:04.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.401+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff42806dfa0 msgr2=0x7ff428070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:04.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.401+0000 7ff44ac00700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff42806dfa0 0x7ff428070450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff430006010 tx=0x7ff43000b540 comp rx=0 tx=0).stop 2024-03-22T10:52:04.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.401+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440fe9a0 msgr2=0x7ff444104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:04.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.401+0000 7ff44ac00700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440fe9a0 0x7ff444104520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff43400d8d0 tx=0x7ff43400dc90 comp rx=0 tx=0).stop 2024-03-22T10:52:04.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.401+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 shutdown_connections 2024-03-22T10:52:04.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.401+0000 7ff44ac00700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff42806dfa0 0x7ff428070450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:04.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.401+0000 7ff44ac00700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440fe9a0 0x7ff444104520 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:04.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.401+0000 7ff44ac00700 1 --2- 172.21.15.115:0/370749532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4440ff2f0 0x7ff444104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:04.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.401+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 >> 172.21.15.115:0/370749532 conn(0x7ff4440fa5f0 msgr2=0x7ff444106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:04.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.402+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 shutdown_connections 2024-03-22T10:52:04.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:04.402+0000 7ff44ac00700 1 -- 172.21.15.115:0/370749532 wait complete. 2024-03-22T10:52:04.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/726190321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:52:04.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:04.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi115.ggxfzi"}]': finished 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:52:04.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi115.wjotmn"}]': finished 2024-03-22T10:52:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:52:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi150.slzwwb"}]': finished 2024-03-22T10:52:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:52:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi150.zvyrzc"}]': finished 2024-03-22T10:52:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-22T10:52:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:52:04.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/726190321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:52:04.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:04.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:52:04.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi115.ggxfzi"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi115.ggxfzi"}]': finished 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi115.wjotmn"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi115.wjotmn"}]': finished 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi150.slzwwb"}]: dispatch 2024-03-22T10:52:04.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi150.slzwwb"}]': finished 2024-03-22T10:52:04.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi150.zvyrzc"}]: dispatch 2024-03-22T10:52:04.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi150.zvyrzc"}]': finished 2024-03-22T10:52:04.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-22T10:52:04.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:04.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:52:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all mds 2024-03-22T10:52:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-03-22T10:52:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: pgmap v278: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 345 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-22T10:52:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/370749532' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:52:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: daemon mds.cephfs.smithi115.wjotmn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T10:52:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-22T10:52:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi115.wjotmn=up:starting} 2 up:standby 2024-03-22T10:52:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all rgw 2024-03-22T10:52:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: daemon mds.cephfs.smithi115.wjotmn is now active in filesystem cephfs as rank 1 2024-03-22T10:52:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T10:52:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:05.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all mds 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: pgmap v278: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 345 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/370749532' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: daemon mds.cephfs.smithi115.wjotmn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi115.wjotmn=up:starting} 2 up:standby 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all rgw 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: daemon mds.cephfs.smithi115.wjotmn is now active in filesystem cephfs as rank 1 2024-03-22T10:52:05.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:05.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:05.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T10:52:05.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:05.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:07.259 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:06 smithi115 ceph-mon[78672]: mds.? [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] up:active 2024-03-22T10:52:07.259 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:06 smithi115 ceph-mon[78672]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi115.wjotmn=up:active} 2 up:standby 2024-03-22T10:52:07.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:06 smithi150 ceph-mon[65513]: mds.? [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] up:active 2024-03-22T10:52:07.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:06 smithi150 ceph-mon[65513]: fsmap cephfs:2 {0=cephfs.smithi115.ggxfzi=up:active,1=cephfs.smithi115.wjotmn=up:active} 2 up:standby 2024-03-22T10:52:08.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:07 smithi115 ceph-mon[78672]: pgmap v279: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-03-22T10:52:08.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:07 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:08.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:07 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:52:08.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:07 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:52:08.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:07 smithi150 ceph-mon[65513]: pgmap v279: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-03-22T10:52:08.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:07 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:08.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:07 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi115", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:52:08.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:07 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:52:09.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:08 smithi115 ceph-mon[78672]: Upgrade: Updating ceph-exporter.smithi115 (1/2) 2024-03-22T10:52:09.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:08 smithi115 ceph-mon[78672]: Deploying daemon ceph-exporter.smithi115 on smithi115 2024-03-22T10:52:09.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:08 smithi150 ceph-mon[65513]: Upgrade: Updating ceph-exporter.smithi115 (1/2) 2024-03-22T10:52:09.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:08 smithi150 ceph-mon[65513]: Deploying daemon ceph-exporter.smithi115 on smithi115 2024-03-22T10:52:10.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:09 smithi115 ceph-mon[78672]: pgmap v280: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 4.0 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-03-22T10:52:10.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:09 smithi150 ceph-mon[65513]: pgmap v280: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 4.0 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-03-22T10:52:12.159 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:11 smithi115 ceph-mon[78672]: pgmap v281: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-22T10:52:12.397 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:11 smithi150 ceph-mon[65513]: pgmap v281: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-22T10:52:13.780 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:13 smithi150 ceph-mon[65513]: pgmap v282: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-22T10:52:13.780 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:13.780 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:14.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:13 smithi115 ceph-mon[78672]: pgmap v282: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-22T10:52:14.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:14.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:16.049 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:15 smithi150 ceph-mon[65513]: pgmap v283: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T10:52:16.049 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:15 smithi150 ceph-mon[65513]: Upgrade: Updating ceph-exporter.smithi150 (2/2) 2024-03-22T10:52:16.049 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:16.049 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:52:16.049 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:15 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:52:16.049 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:15 smithi150 ceph-mon[65513]: Deploying daemon ceph-exporter.smithi150 on smithi150 2024-03-22T10:52:16.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:15 smithi115 ceph-mon[78672]: pgmap v283: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T10:52:16.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:15 smithi115 ceph-mon[78672]: Upgrade: Updating ceph-exporter.smithi150 (2/2) 2024-03-22T10:52:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi150", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-22T10:52:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:15 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:52:16.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:15 smithi115 ceph-mon[78672]: Deploying daemon ceph-exporter.smithi150 on smithi150 2024-03-22T10:52:18.243 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:17 smithi150 ceph-mon[65513]: pgmap v284: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T10:52:18.259 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:17 smithi115 ceph-mon[78672]: pgmap v284: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T10:52:20.194 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:19 smithi150 ceph-mon[65513]: pgmap v285: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T10:52:20.194 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:52:20.259 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:19 smithi115 ceph-mon[78672]: pgmap v285: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-22T10:52:20.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:52:21.687 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:21 smithi115 ceph-mon[78672]: pgmap v286: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T10:52:21.687 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:21.687 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:21.687 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:52:21.774 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:21 smithi150 ceph-mon[65513]: pgmap v286: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T10:52:21.774 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:21.774 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:21.774 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:52:23.759 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:23 smithi115 ceph-mon[78672]: pgmap v287: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T10:52:23.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:23 smithi150 ceph-mon[65513]: pgmap v287: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T10:52:25.308 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:25.309 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:25.309 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:25 smithi150 ceph-mon[65513]: pgmap v288: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T10:52:25.509 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:25.509 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:25.509 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:25 smithi115 ceph-mon[78672]: pgmap v288: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T10:52:27.223 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:27.223 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:27.223 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:26 smithi150 ceph-mon[65513]: pgmap v289: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:27.223 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:27.223 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:26 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:27.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:27.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:27.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:26 smithi115 ceph-mon[78672]: pgmap v289: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:27.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:27.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:26 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:30.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:29 smithi115 ceph-mon[78672]: pgmap v290: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:30.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:30.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:30.112 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:29 smithi150 ceph-mon[65513]: pgmap v290: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:30.112 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:30.112 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:32.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:31 smithi150 ceph-mon[65513]: pgmap v291: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:32.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:32.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:32.259 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:31 smithi115 ceph-mon[78672]: pgmap v291: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:32.259 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:32.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:33 smithi115 ceph-mon[78672]: pgmap v292: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:33 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:33 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:33 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:52:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:33 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:52:34.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:33 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:34.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:33 smithi150 ceph-mon[65513]: pgmap v292: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:34.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:33 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:34.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:33 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:34.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:33 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:52:34.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:33 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:52:34.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:33 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.595+0000 7f51d0e00700 1 -- 172.21.15.115:0/3770922629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 msgr2=0x7f51cc0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.595+0000 7f51d0e00700 1 --2- 172.21.15.115:0/3770922629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 0x7f51cc0731e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009940 tx=0x7f51bc009c50 comp rx=0 tx=0).stop 2024-03-22T10:52:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.596+0000 7f51d0e00700 1 -- 172.21.15.115:0/3770922629 shutdown_connections 2024-03-22T10:52:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.596+0000 7f51d0e00700 1 --2- 172.21.15.115:0/3770922629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51cc0737b0 0x7f51cc073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.596+0000 7f51d0e00700 1 --2- 172.21.15.115:0/3770922629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 0x7f51cc0731e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.596+0000 7f51d0e00700 1 -- 172.21.15.115:0/3770922629 >> 172.21.15.115:0/3770922629 conn(0x7f51cc0fb6e0 msgr2=0x7f51cc0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.596+0000 7f51d0e00700 1 -- 172.21.15.115:0/3770922629 shutdown_connections 2024-03-22T10:52:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.596+0000 7f51d0e00700 1 -- 172.21.15.115:0/3770922629 wait complete. 2024-03-22T10:52:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.598+0000 7f51d0e00700 1 Processor -- start 2024-03-22T10:52:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.598+0000 7f51d0e00700 1 -- start start 2024-03-22T10:52:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.598+0000 7f51d0e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51cc0737b0 0x7f51cc197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.598+0000 7f51d0e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 0x7f51cc1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:34.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.598+0000 7f51d0e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc1987d0 con 0x7f51cc074d80 2024-03-22T10:52:34.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.598+0000 7f51d0e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc19d1e0 con 0x7f51cc0737b0 2024-03-22T10:52:34.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.599+0000 7f51c9400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 0x7f51cc1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:34.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.599+0000 7f51c9400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 0x7f51cc1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49156/0 (socket says 172.21.15.115:49156) 2024-03-22T10:52:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.599+0000 7f51c9400700 1 -- 172.21.15.115:0/2760002711 learned_addr learned my addr 172.21.15.115:0/2760002711 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.599+0000 7f51c9e00700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51cc0737b0 0x7f51cc197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.599+0000 7f51c9400700 1 -- 172.21.15.115:0/2760002711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51cc0737b0 msgr2=0x7f51cc197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.599+0000 7f51c9400700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51cc0737b0 0x7f51cc197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.599+0000 7f51c9400700 1 -- 172.21.15.115:0/2760002711 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0095f0 con 0x7f51cc074d80 2024-03-22T10:52:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.600+0000 7f51c9400700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 0x7f51cc1981b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51b800b4b0 tx=0x7f51b800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.600+0000 7f51c2a00700 1 -- 172.21.15.115:0/2760002711 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b8011800 con 0x7f51cc074d80 2024-03-22T10:52:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.600+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc19d3e0 con 0x7f51cc074d80 2024-03-22T10:52:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.600+0000 7f51c2a00700 1 -- 172.21.15.115:0/2760002711 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51b8011e10 con 0x7f51cc074d80 2024-03-22T10:52:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.600+0000 7f51c2a00700 1 -- 172.21.15.115:0/2760002711 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b800f2e0 con 0x7f51cc074d80 2024-03-22T10:52:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.601+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc19d980 con 0x7f51cc074d80 2024-03-22T10:52:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.602+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51cc10af10 con 0x7f51cc074d80 2024-03-22T10:52:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.603+0000 7f51c2a00700 1 -- 172.21.15.115:0/2760002711 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f51b8010380 con 0x7f51cc074d80 2024-03-22T10:52:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.604+0000 7f51c2a00700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f51b006dfa0 0x7f51b0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.604+0000 7f51c2a00700 1 -- 172.21.15.115:0/2760002711 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f51b808da90 con 0x7f51cc074d80 2024-03-22T10:52:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.605+0000 7f51c9e00700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f51b006dfa0 0x7f51b0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.606+0000 7f51c9e00700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f51b006dfa0 0x7f51b0070450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f51bc006010 tx=0x7f51bc00b9a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.610+0000 7f51c2a00700 1 -- 172.21.15.115:0/2760002711 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f51b8056e60 con 0x7f51cc074d80 2024-03-22T10:52:34.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.935+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51cc107650 con 0x7f51b006dfa0 2024-03-22T10:52:34.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.937+0000 7f51c2a00700 1 -- 172.21.15.115:0/2760002711 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f51cc107650 con 0x7f51b006dfa0 2024-03-22T10:52:34.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f51b006dfa0 msgr2=0x7f51b0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:34.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f51b006dfa0 0x7f51b0070450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f51bc006010 tx=0x7f51bc00b9a0 comp rx=0 tx=0).stop 2024-03-22T10:52:34.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 msgr2=0x7f51cc1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:34.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 0x7f51cc1981b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51b800b4b0 tx=0x7f51b800b870 comp rx=0 tx=0).stop 2024-03-22T10:52:34.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 shutdown_connections 2024-03-22T10:52:34.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f51b006dfa0 0x7f51b0070450 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:34.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51cc0737b0 0x7f51cc197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:34.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 --2- 172.21.15.115:0/2760002711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc074d80 0x7f51cc1981b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:34.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 >> 172.21.15.115:0/2760002711 conn(0x7f51cc0fb6e0 msgr2=0x7f51cc105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:34.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 shutdown_connections 2024-03-22T10:52:34.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:34.940+0000 7f51d0e00700 1 -- 172.21.15.115:0/2760002711 wait complete. 2024-03-22T10:52:34.951 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:52:35.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.128+0000 7fc891e00700 1 -- 172.21.15.115:0/688045656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 msgr2=0x7fc88c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:35.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.128+0000 7fc891e00700 1 --2- 172.21.15.115:0/688045656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 0x7fc88c073c20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc878009940 tx=0x7fc878009c50 comp rx=0 tx=0).stop 2024-03-22T10:52:35.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.128+0000 7fc891e00700 1 -- 172.21.15.115:0/688045656 shutdown_connections 2024-03-22T10:52:35.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.128+0000 7fc891e00700 1 --2- 172.21.15.115:0/688045656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 0x7fc88c073c20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.128+0000 7fc891e00700 1 --2- 172.21.15.115:0/688045656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc88c074d80 0x7fc88c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.128+0000 7fc891e00700 1 -- 172.21.15.115:0/688045656 >> 172.21.15.115:0/688045656 conn(0x7fc88c0fb6e0 msgr2=0x7fc88c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:35.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.128+0000 7fc891e00700 1 -- 172.21.15.115:0/688045656 shutdown_connections 2024-03-22T10:52:35.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.128+0000 7fc891e00700 1 -- 172.21.15.115:0/688045656 wait complete. 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi115"}]: dispatch 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi115"}]': finished 2024-03-22T10:52:35.121 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi150"}]: dispatch 2024-03-22T10:52:35.122 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi150"}]': finished 2024-03-22T10:52:35.122 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.122 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:35.122 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.122 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:35.122 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.122 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:35.122 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:52:35.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.130+0000 7fc891e00700 1 Processor -- start 2024-03-22T10:52:35.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.130+0000 7fc891e00700 1 -- start start 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.130+0000 7fc891e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 0x7fc88c103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.130+0000 7fc891e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc88c074d80 0x7fc88c1039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.130+0000 7fc891e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0fff70 con 0x7fc88c0737b0 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.130+0000 7fc891e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c1000e0 con 0x7fc88c074d80 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.131+0000 7fc88aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 0x7fc88c103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.131+0000 7fc88aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 0x7fc88c103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49168/0 (socket says 172.21.15.115:49168) 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.131+0000 7fc88aa00700 1 -- 172.21.15.115:0/17820869 learned_addr learned my addr 172.21.15.115:0/17820869 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.131+0000 7fc88a000700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc88c074d80 0x7fc88c1039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.131+0000 7fc88aa00700 1 -- 172.21.15.115:0/17820869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc88c074d80 msgr2=0x7fc88c1039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.131+0000 7fc88aa00700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc88c074d80 0x7fc88c1039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.131+0000 7fc88aa00700 1 -- 172.21.15.115:0/17820869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780095f0 con 0x7fc88c0737b0 2024-03-22T10:52:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.132+0000 7fc88aa00700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 0x7fc88c103480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00d8d0 tx=0x7fc87c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:35.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.132+0000 7fc890a00700 1 -- 172.21.15.115:0/17820869 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc87c009940 con 0x7fc88c0737b0 2024-03-22T10:52:35.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.132+0000 7fc890a00700 1 -- 172.21.15.115:0/17820869 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc87c010460 con 0x7fc88c0737b0 2024-03-22T10:52:35.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.132+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c1003c0 con 0x7fc88c0737b0 2024-03-22T10:52:35.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.132+0000 7fc890a00700 1 -- 172.21.15.115:0/17820869 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc87c00f5f0 con 0x7fc88c0737b0 2024-03-22T10:52:35.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.132+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c100990 con 0x7fc88c0737b0 2024-03-22T10:52:35.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.133+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c10ae90 con 0x7fc88c0737b0 2024-03-22T10:52:35.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.135+0000 7fc890a00700 1 -- 172.21.15.115:0/17820869 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc87c0105d0 con 0x7fc88c0737b0 2024-03-22T10:52:35.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.135+0000 7fc890a00700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc87006dfa0 0x7fc870070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:35.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.136+0000 7fc890a00700 1 -- 172.21.15.115:0/17820869 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc87c08de70 con 0x7fc88c0737b0 2024-03-22T10:52:35.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.136+0000 7fc88a000700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc87006dfa0 0x7fc870070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:35.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.137+0000 7fc88a000700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc87006dfa0 0x7fc870070450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc878005b40 tx=0x7fc878005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:35.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.142+0000 7fc890a00700 1 -- 172.21.15.115:0/17820869 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc87c05acf0 con 0x7fc88c0737b0 2024-03-22T10:52:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:52:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi115"}]: dispatch 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi115"}]': finished 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi150"}]: dispatch 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi150"}]': finished 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:35.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:52:35.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.476+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc88c1a23d0 con 0x7fc87006dfa0 2024-03-22T10:52:35.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.478+0000 7fc890a00700 1 -- 172.21.15.115:0/17820869 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fc88c1a23d0 con 0x7fc87006dfa0 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc87006dfa0 msgr2=0x7fc870070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc87006dfa0 0x7fc870070450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc878005b40 tx=0x7fc878005a90 comp rx=0 tx=0).stop 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 msgr2=0x7fc88c103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 0x7fc88c103480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00d8d0 tx=0x7fc87c00dc90 comp rx=0 tx=0).stop 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 shutdown_connections 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc87006dfa0 0x7fc870070450 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0737b0 0x7fc88c103480 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 --2- 172.21.15.115:0/17820869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc88c074d80 0x7fc88c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 >> 172.21.15.115:0/17820869 conn(0x7fc88c0fb6e0 msgr2=0x7fc88c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 shutdown_connections 2024-03-22T10:52:35.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.481+0000 7fc891e00700 1 -- 172.21.15.115:0/17820869 wait complete. 2024-03-22T10:52:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.661+0000 7f530f200700 1 -- 172.21.15.115:0/1903405427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 msgr2=0x7f5310105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:35.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.661+0000 7f530f200700 1 --2- 172.21.15.115:0/1903405427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 0x7f5310105720 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009910 tx=0x7f52f8009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:35.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.661+0000 7f530f200700 1 -- 172.21.15.115:0/1903405427 shutdown_connections 2024-03-22T10:52:35.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.661+0000 7f530f200700 1 --2- 172.21.15.115:0/1903405427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 0x7f5310105720 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.661+0000 7f530f200700 1 --2- 172.21.15.115:0/1903405427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5310100a20 0x7f5310102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.661+0000 7f530f200700 1 -- 172.21.15.115:0/1903405427 >> 172.21.15.115:0/1903405427 conn(0x7f53100fa440 msgr2=0x7f53100fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:35.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.661+0000 7f530f200700 1 -- 172.21.15.115:0/1903405427 shutdown_connections 2024-03-22T10:52:35.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.661+0000 7f530f200700 1 -- 172.21.15.115:0/1903405427 wait complete. 2024-03-22T10:52:35.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.662+0000 7f530f200700 1 Processor -- start 2024-03-22T10:52:35.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.663+0000 7f530f200700 1 -- start start 2024-03-22T10:52:35.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.663+0000 7f530f200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5310100a20 0x7f5310195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:35.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.663+0000 7f530f200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 0x7f5310195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:35.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.663+0000 7f530f200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5310196240 con 0x7f5310103340 2024-03-22T10:52:35.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.663+0000 7f530f200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5310196380 con 0x7f5310100a20 2024-03-22T10:52:35.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.663+0000 7f5307e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 0x7f5310195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:35.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.663+0000 7f5307e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 0x7f5310195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49176/0 (socket says 172.21.15.115:49176) 2024-03-22T10:52:35.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.663+0000 7f5307e00700 1 -- 172.21.15.115:0/3911677784 learned_addr learned my addr 172.21.15.115:0/3911677784 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:35.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.664+0000 7f530de00700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5310100a20 0x7f5310195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:35.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.664+0000 7f5307e00700 1 -- 172.21.15.115:0/3911677784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5310100a20 msgr2=0x7f5310195770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:35.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.664+0000 7f5307e00700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5310100a20 0x7f5310195770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.664+0000 7f5307e00700 1 -- 172.21.15.115:0/3911677784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80095f0 con 0x7f5310103340 2024-03-22T10:52:35.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.664+0000 7f530de00700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5310100a20 0x7f5310195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:52:35.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.665+0000 7f5307e00700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 0x7f5310195cb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52f800b5c0 tx=0x7f52f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.665+0000 7f5306a00700 1 -- 172.21.15.115:0/3911677784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52f801d070 con 0x7f5310103340 2024-03-22T10:52:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.665+0000 7f5306a00700 1 -- 172.21.15.115:0/3911677784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f52f8022470 con 0x7f5310103340 2024-03-22T10:52:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.665+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531019ade0 con 0x7f5310103340 2024-03-22T10:52:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.665+0000 7f5306a00700 1 -- 172.21.15.115:0/3911677784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52f800f690 con 0x7f5310103340 2024-03-22T10:52:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.665+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53100fe9e0 con 0x7f5310103340 2024-03-22T10:52:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.666+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531018f9b0 con 0x7f5310103340 2024-03-22T10:52:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.668+0000 7f5306a00700 1 -- 172.21.15.115:0/3911677784 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f52f8004b70 con 0x7f5310103340 2024-03-22T10:52:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.669+0000 7f5306a00700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52f40723b0 0x7f52f4074860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.669+0000 7f5306a00700 1 -- 172.21.15.115:0/3911677784 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f52f808f5f0 con 0x7f5310103340 2024-03-22T10:52:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.670+0000 7f530de00700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52f40723b0 0x7f52f4074860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.671+0000 7f530de00700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52f40723b0 0x7f52f4074860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5300005fd0 tx=0x7f5300005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:35.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.675+0000 7f5306a00700 1 -- 172.21.15.115:0/3911677784 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52f805c520 con 0x7f5310103340 2024-03-22T10:52:35.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.956+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5310061190 con 0x7f52f40723b0 2024-03-22T10:52:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.975+0000 7f5306a00700 1 -- 172.21.15.115:0/3911677784 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5310061190 con 0x7f52f40723b0 2024-03-22T10:52:35.966 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:52:35.966 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (19m) 9s ago 20m 14.7M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (23s) 9s ago 21m 7868k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e11b0bdaf069 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (16s) 11s ago 19m 6975k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 ef9665eef7b1 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (6m) 9s ago 21m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (6m) 11s ago 19m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (18m) 9s ago 20m 73.8M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (2m) 9s ago 14m 66.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 44a0769ebe8e 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (97s) 9s ago 13m 15.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51d700f6eb18 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (44s) 11s ago 13m 12.0M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 82b6b216f754 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (70s) 11s ago 13m 12.1M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9605445d5b2b 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (10m) 9s ago 22m 514M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (9m) 11s ago 19m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (8m) 9s ago 22m 57.8M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (8m) 11s ago 19m 44.4M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (21m) 9s ago 21m 7902k - 1.5.0 0da6a335fe13 4369bde227f2 2024-03-22T10:52:35.967 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (19m) 11s ago 19m 8414k - 1.5.0 0da6a335fe13 c71131425028 2024-03-22T10:52:35.968 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (6m) 9s ago 18m 97.9M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:52:35.968 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (5m) 9s ago 18m 96.3M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:52:35.968 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (5m) 9s ago 17m 83.3M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:52:35.968 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (4m) 11s ago 16m 103M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:52:35.968 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (4m) 11s ago 16m 92.5M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:52:35.968 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (3m) 11s ago 15m 90.7M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:52:35.968 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (9m) 9s ago 20m 79.3M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:52:35.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.977+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52f40723b0 msgr2=0x7f52f4074860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:35.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.977+0000 7f530f200700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52f40723b0 0x7f52f4074860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5300005fd0 tx=0x7f5300005e30 comp rx=0 tx=0).stop 2024-03-22T10:52:35.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.977+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 msgr2=0x7f5310195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:35.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.977+0000 7f530f200700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 0x7f5310195cb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52f800b5c0 tx=0x7f52f800be20 comp rx=0 tx=0).stop 2024-03-22T10:52:35.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.977+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 shutdown_connections 2024-03-22T10:52:35.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.978+0000 7f530f200700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f52f40723b0 0x7f52f4074860 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.978+0000 7f530f200700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5310100a20 0x7f5310195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.978+0000 7f530f200700 1 --2- 172.21.15.115:0/3911677784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5310103340 0x7f5310195cb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:35.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.978+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 >> 172.21.15.115:0/3911677784 conn(0x7f53100fa440 msgr2=0x7f53100fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:35.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.978+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 shutdown_connections 2024-03-22T10:52:35.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:35.978+0000 7f530f200700 1 -- 172.21.15.115:0/3911677784 wait complete. 2024-03-22T10:52:36.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.137+0000 7f5403200700 1 -- 172.21.15.115:0/777214936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 msgr2=0x7f5404100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:36.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.137+0000 7f5403200700 1 --2- 172.21.15.115:0/777214936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 0x7f5404100420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009910 tx=0x7f53f4009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:36.129 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:35 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T10:52:36.129 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:35 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all iscsi 2024-03-22T10:52:36.129 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:35 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all nfs 2024-03-22T10:52:36.129 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:35 smithi115 ceph-mon[78672]: Upgrade: Setting container_image for all nvmeof 2024-03-22T10:52:36.129 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:35 smithi115 ceph-mon[78672]: pgmap v293: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:36.129 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:35 smithi115 ceph-mon[78672]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:36.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.137+0000 7f5403200700 1 -- 172.21.15.115:0/777214936 shutdown_connections 2024-03-22T10:52:36.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.137+0000 7f5403200700 1 --2- 172.21.15.115:0/777214936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5404101210 0x7f5404101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:36.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.137+0000 7f5403200700 1 --2- 172.21.15.115:0/777214936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 0x7f5404100420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:36.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.137+0000 7f5403200700 1 -- 172.21.15.115:0/777214936 >> 172.21.15.115:0/777214936 conn(0x7f54040fb5a0 msgr2=0x7f54040fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:36.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.137+0000 7f5403200700 1 -- 172.21.15.115:0/777214936 shutdown_connections 2024-03-22T10:52:36.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.137+0000 7f5403200700 1 -- 172.21.15.115:0/777214936 wait complete. 2024-03-22T10:52:36.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.139+0000 7f5403200700 1 Processor -- start 2024-03-22T10:52:36.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.139+0000 7f5403200700 1 -- start start 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.139+0000 7f5403200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 0x7f54041979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.139+0000 7f5403200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5404101210 0x7f5404197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.139+0000 7f5403200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5404198490 con 0x7f5404100010 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.139+0000 7f5403200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041985d0 con 0x7f5404101210 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.140+0000 7f5401e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 0x7f54041979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.140+0000 7f5401e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 0x7f54041979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49198/0 (socket says 172.21.15.115:49198) 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.140+0000 7f5401e00700 1 -- 172.21.15.115:0/4014623152 learned_addr learned my addr 172.21.15.115:0/4014623152 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.140+0000 7f53f9400700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5404101210 0x7f5404197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.140+0000 7f5401e00700 1 -- 172.21.15.115:0/4014623152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5404101210 msgr2=0x7f5404197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.140+0000 7f5401e00700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5404101210 0x7f5404197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.140+0000 7f5401e00700 1 -- 172.21.15.115:0/4014623152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40095f0 con 0x7f5404100010 2024-03-22T10:52:36.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.140+0000 7f53f9400700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5404101210 0x7f5404197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:52:36.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.141+0000 7f5401e00700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 0x7f54041979c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53f400bda0 tx=0x7f53f400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:36.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.141+0000 7f53fb400700 1 -- 172.21.15.115:0/4014623152 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53f401d070 con 0x7f5404100010 2024-03-22T10:52:36.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.141+0000 7f53fb400700 1 -- 172.21.15.115:0/4014623152 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f53f4004c50 con 0x7f5404100010 2024-03-22T10:52:36.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.141+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540419d030 con 0x7f5404100010 2024-03-22T10:52:36.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.141+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540419d4f0 con 0x7f5404100010 2024-03-22T10:52:36.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.142+0000 7f53fb400700 1 -- 172.21.15.115:0/4014623152 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53f400f690 con 0x7f5404100010 2024-03-22T10:52:36.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.143+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e0005320 con 0x7f5404100010 2024-03-22T10:52:36.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.144+0000 7f53fb400700 1 -- 172.21.15.115:0/4014623152 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f53f400f7f0 con 0x7f5404100010 2024-03-22T10:52:36.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.145+0000 7f53fb400700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f53e806dfa0 0x7f53e8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:36.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.145+0000 7f53fb400700 1 -- 172.21.15.115:0/4014623152 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f53f408f5c0 con 0x7f5404100010 2024-03-22T10:52:36.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.145+0000 7f53f9400700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f53e806dfa0 0x7f53e8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:36.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.146+0000 7f53f9400700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f53e806dfa0 0x7f53e8070450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53ec005fd0 tx=0x7f53ec005e20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:36.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.150+0000 7f53fb400700 1 -- 172.21.15.115:0/4014623152 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f53f405c610 con 0x7f5404100010 2024-03-22T10:52:36.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:35 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T10:52:36.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:35 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all iscsi 2024-03-22T10:52:36.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:35 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all nfs 2024-03-22T10:52:36.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:35 smithi150 ceph-mon[65513]: Upgrade: Setting container_image for all nvmeof 2024-03-22T10:52:36.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:35 smithi150 ceph-mon[65513]: pgmap v293: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:36.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:35 smithi150 ceph-mon[65513]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:36.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.577+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f53e0006200 con 0x7f5404100010 2024-03-22T10:52:36.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.578+0000 7f53fb400700 1 -- 172.21.15.115:0/4014623152 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f53f405c1a0 con 0x7f5404100010 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:52:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:52:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:52:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:52:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-22T10:52:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:52:36.571 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:52:36.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f53e806dfa0 msgr2=0x7f53e8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f53e806dfa0 0x7f53e8070450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53ec005fd0 tx=0x7f53ec005e20 comp rx=0 tx=0).stop 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 msgr2=0x7f54041979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 0x7f54041979c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53f400bda0 tx=0x7f53f400be80 comp rx=0 tx=0).stop 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 shutdown_connections 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f53e806dfa0 0x7f53e8070450 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5404100010 0x7f54041979c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 --2- 172.21.15.115:0/4014623152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5404101210 0x7f5404197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 >> 172.21.15.115:0/4014623152 conn(0x7f54040fb5a0 msgr2=0x7f5404102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 shutdown_connections 2024-03-22T10:52:36.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.582+0000 7f5403200700 1 -- 172.21.15.115:0/4014623152 wait complete. 2024-03-22T10:52:36.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.777+0000 7f83fc600700 1 -- 172.21.15.115:0/2481760714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4100c40 msgr2=0x7f83f4103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:36.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.777+0000 7f83fc600700 1 --2- 172.21.15.115:0/2481760714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4100c40 0x7f83f4103020 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009910 tx=0x7f83e8009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:36.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.777+0000 7f83fc600700 1 -- 172.21.15.115:0/2481760714 shutdown_connections 2024-03-22T10:52:36.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.777+0000 7f83fc600700 1 --2- 172.21.15.115:0/2481760714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4103560 0x7f83f4105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:36.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.777+0000 7f83fc600700 1 --2- 172.21.15.115:0/2481760714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4100c40 0x7f83f4103020 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:36.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.777+0000 7f83fc600700 1 -- 172.21.15.115:0/2481760714 >> 172.21.15.115:0/2481760714 conn(0x7f83f40fa5e0 msgr2=0x7f83f40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:36.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.778+0000 7f83fc600700 1 -- 172.21.15.115:0/2481760714 shutdown_connections 2024-03-22T10:52:36.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.778+0000 7f83fc600700 1 -- 172.21.15.115:0/2481760714 wait complete. 2024-03-22T10:52:36.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.779+0000 7f83fc600700 1 Processor -- start 2024-03-22T10:52:36.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.779+0000 7f83fc600700 1 -- start start 2024-03-22T10:52:36.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.780+0000 7f83fc600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4100c40 0x7f83f41005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:36.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.780+0000 7f83fc600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4103560 0x7f83f40febf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:36.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.780+0000 7f83fc600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40ff130 con 0x7f83f4103560 2024-03-22T10:52:36.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.780+0000 7f83fc600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40ff270 con 0x7f83f4100c40 2024-03-22T10:52:36.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.780+0000 7f83f9200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4103560 0x7f83f40febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:36.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.780+0000 7f83f9200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4103560 0x7f83f40febf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49218/0 (socket says 172.21.15.115:49218) 2024-03-22T10:52:36.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.780+0000 7f83f9200700 1 -- 172.21.15.115:0/3390588665 learned_addr learned my addr 172.21.15.115:0/3390588665 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:36.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.780+0000 7f83f9c00700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4100c40 0x7f83f41005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:36.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.781+0000 7f83f9200700 1 -- 172.21.15.115:0/3390588665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4100c40 msgr2=0x7f83f41005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:36.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.781+0000 7f83f9200700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4100c40 0x7f83f41005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:36.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.781+0000 7f83f9200700 1 -- 172.21.15.115:0/3390588665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80095f0 con 0x7f83f4103560 2024-03-22T10:52:36.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.781+0000 7f83f9200700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4103560 0x7f83f40febf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83e000c8a0 tx=0x7f83e000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:36.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.782+0000 7f83f2000700 1 -- 172.21.15.115:0/3390588665 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83e00078e0 con 0x7f83f4103560 2024-03-22T10:52:36.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.782+0000 7f83f2000700 1 -- 172.21.15.115:0/3390588665 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f83e000f450 con 0x7f83f4103560 2024-03-22T10:52:36.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.782+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f40ff550 con 0x7f83f4103560 2024-03-22T10:52:36.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.782+0000 7f83f2000700 1 -- 172.21.15.115:0/3390588665 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83e000e5e0 con 0x7f83f4103560 2024-03-22T10:52:36.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.782+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f419d980 con 0x7f83f4103560 2024-03-22T10:52:36.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.783+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f418fb70 con 0x7f83f4103560 2024-03-22T10:52:36.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.784+0000 7f83f2000700 1 -- 172.21.15.115:0/3390588665 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f83e000f5c0 con 0x7f83f4103560 2024-03-22T10:52:36.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.786+0000 7f83f2000700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f83dc06dfa0 0x7f83dc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:36.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.786+0000 7f83f2000700 1 -- 172.21.15.115:0/3390588665 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f83e008dd00 con 0x7f83f4103560 2024-03-22T10:52:36.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.786+0000 7f83f9c00700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f83dc06dfa0 0x7f83dc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:36.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.787+0000 7f83f9c00700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f83dc06dfa0 0x7f83dc070450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f83e8006010 tx=0x7f83e800b560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:36.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:36.792+0000 7f83f2000700 1 -- 172.21.15.115:0/3390588665 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f83e005ac00 con 0x7f83f4103560 2024-03-22T10:52:37.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.089+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f83f402cc30 con 0x7f83f4103560 2024-03-22T10:52:37.081 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:36 smithi115 ceph-mon[78672]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:37.081 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:36 smithi115 ceph-mon[78672]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:37.081 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:36 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/4014623152' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:37.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.089+0000 7f83f2000700 1 -- 172.21.15.115:0/3390588665 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1932 (secure 0 0 0) 0x7f83e005a790 con 0x7f83f4103560 2024-03-22T10:52:37.082 INFO:teuthology.orchestra.run.smithi115.stdout:e40 2024-03-22T10:52:37.082 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:52:37.082 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:52:37.082 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 40 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:52:05.899739+0000 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:52:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 83 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=34476,1=34508} 2024-03-22T10:52:37.084 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 34476 members: 34476,34508 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:34476} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:52:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{1:34508} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:52:37.086 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:52:37.086 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:52:37.086 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:52:37.086 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:52:37.086 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6824/1804848703,v1:172.21.15.150:6825/1804848703] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:52:37.086 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6826/2053213512,v1:172.21.15.150:6827/2053213512] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:52:37.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.093+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f83dc06dfa0 msgr2=0x7f83dc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:37.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.093+0000 7f83fc600700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f83dc06dfa0 0x7f83dc070450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f83e8006010 tx=0x7f83e800b560 comp rx=0 tx=0).stop 2024-03-22T10:52:37.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.093+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4103560 msgr2=0x7f83f40febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:37.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.093+0000 7f83fc600700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4103560 0x7f83f40febf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83e000c8a0 tx=0x7f83e000cbb0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.094+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 shutdown_connections 2024-03-22T10:52:37.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.094+0000 7f83fc600700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f83dc06dfa0 0x7f83dc070450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.094+0000 7f83fc600700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4100c40 0x7f83f41005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.094+0000 7f83fc600700 1 --2- 172.21.15.115:0/3390588665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83f4103560 0x7f83f40febf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.094+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 >> 172.21.15.115:0/3390588665 conn(0x7f83f40fa5e0 msgr2=0x7f83f40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:37.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.094+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 shutdown_connections 2024-03-22T10:52:37.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.094+0000 7f83fc600700 1 -- 172.21.15.115:0/3390588665 wait complete. 2024-03-22T10:52:37.088 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2024-03-22T10:52:37.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:36 smithi150 ceph-mon[65513]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:37.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:36 smithi150 ceph-mon[65513]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:37.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:36 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/4014623152' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:52:37.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.248+0000 7f1f48e00700 1 -- 172.21.15.115:0/2184198052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 msgr2=0x7f1f44100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:37.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.248+0000 7f1f48e00700 1 --2- 172.21.15.115:0/2184198052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 0x7f1f44100560 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009910 tx=0x7f1f34009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:37.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.248+0000 7f1f48e00700 1 -- 172.21.15.115:0/2184198052 shutdown_connections 2024-03-22T10:52:37.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.248+0000 7f1f48e00700 1 --2- 172.21.15.115:0/2184198052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f44101350 0x7f1f441017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.248+0000 7f1f48e00700 1 --2- 172.21.15.115:0/2184198052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 0x7f1f44100560 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.248+0000 7f1f48e00700 1 -- 172.21.15.115:0/2184198052 >> 172.21.15.115:0/2184198052 conn(0x7f1f440fb6e0 msgr2=0x7f1f440fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:37.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.248+0000 7f1f48e00700 1 -- 172.21.15.115:0/2184198052 shutdown_connections 2024-03-22T10:52:37.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.248+0000 7f1f48e00700 1 -- 172.21.15.115:0/2184198052 wait complete. 2024-03-22T10:52:37.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.250+0000 7f1f48e00700 1 Processor -- start 2024-03-22T10:52:37.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.250+0000 7f1f48e00700 1 -- start start 2024-03-22T10:52:37.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.250+0000 7f1f48e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 0x7f1f441959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:37.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.251+0000 7f1f48e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f44101350 0x7f1f44195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:37.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.251+0000 7f1f48e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44196500 con 0x7f1f44100150 2024-03-22T10:52:37.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.251+0000 7f1f48e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44196640 con 0x7f1f44101350 2024-03-22T10:52:37.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.251+0000 7f1f41e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 0x7f1f441959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.251+0000 7f1f41e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 0x7f1f441959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49238/0 (socket says 172.21.15.115:49238) 2024-03-22T10:52:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.251+0000 7f1f41e00700 1 -- 172.21.15.115:0/2808937175 learned_addr learned my addr 172.21.15.115:0/2808937175 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.251+0000 7f1f41400700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f44101350 0x7f1f44195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.252+0000 7f1f41e00700 1 -- 172.21.15.115:0/2808937175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f44101350 msgr2=0x7f1f44195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.252+0000 7f1f41e00700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f44101350 0x7f1f44195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.252+0000 7f1f41e00700 1 -- 172.21.15.115:0/2808937175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340095f0 con 0x7f1f44100150 2024-03-22T10:52:37.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.252+0000 7f1f41e00700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 0x7f1f441959a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400bda0 tx=0x7f1f3400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.253+0000 7f1f3aa00700 1 -- 172.21.15.115:0/2808937175 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f3401d070 con 0x7f1f44100150 2024-03-22T10:52:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.253+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4419b090 con 0x7f1f44100150 2024-03-22T10:52:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.253+0000 7f1f3aa00700 1 -- 172.21.15.115:0/2808937175 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1f3400f460 con 0x7f1f44100150 2024-03-22T10:52:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.253+0000 7f1f3aa00700 1 -- 172.21.15.115:0/2808937175 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f34021620 con 0x7f1f44100150 2024-03-22T10:52:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.254+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4419b580 con 0x7f1f44100150 2024-03-22T10:52:37.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.255+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f44108ca0 con 0x7f1f44100150 2024-03-22T10:52:37.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.256+0000 7f1f3aa00700 1 -- 172.21.15.115:0/2808937175 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f1f3400f5d0 con 0x7f1f44100150 2024-03-22T10:52:37.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.257+0000 7f1f3aa00700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1f2806dfa0 0x7f1f28070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:37.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.257+0000 7f1f3aa00700 1 -- 172.21.15.115:0/2808937175 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1f3408ee20 con 0x7f1f44100150 2024-03-22T10:52:37.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.257+0000 7f1f41400700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1f2806dfa0 0x7f1f28070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:37.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.258+0000 7f1f41400700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1f2806dfa0 0x7f1f28070450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f30007c60 tx=0x7f1f300073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:37.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.262+0000 7f1f3aa00700 1 -- 172.21.15.115:0/2808937175 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f3405bdd0 con 0x7f1f44100150 2024-03-22T10:52:37.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.588+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f44105ca0 con 0x7f1f2806dfa0 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.590+0000 7f1f3aa00700 1 -- 172.21.15.115:0/2808937175 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f1f44105ca0 con 0x7f1f2806dfa0 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout: "osd", 2024-03-22T10:52:37.582 INFO:teuthology.orchestra.run.smithi115.stdout: "mds", 2024-03-22T10:52:37.583 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph-exporter" 2024-03-22T10:52:37.583 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:52:37.583 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "18/23 daemons upgraded", 2024-03-22T10:52:37.583 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-22T10:52:37.583 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:52:37.583 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:52:37.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1f2806dfa0 msgr2=0x7f1f28070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1f2806dfa0 0x7f1f28070450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f30007c60 tx=0x7f1f300073d0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 msgr2=0x7f1f441959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 0x7f1f441959a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400bda0 tx=0x7f1f3400bdd0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 shutdown_connections 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1f2806dfa0 0x7f1f28070450 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f44100150 0x7f1f441959a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 --2- 172.21.15.115:0/2808937175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f44101350 0x7f1f44195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 >> 172.21.15.115:0/2808937175 conn(0x7f1f440fb6e0 msgr2=0x7f1f44104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 shutdown_connections 2024-03-22T10:52:37.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.594+0000 7f1f48e00700 1 -- 172.21.15.115:0/2808937175 wait complete. 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.772+0000 7f87b1400700 1 -- 172.21.15.115:0/3069055627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 msgr2=0x7f87ac102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.772+0000 7f87b1400700 1 --2- 172.21.15.115:0/3069055627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 0x7f87ac102fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f879c009910 tx=0x7f879c009c20 comp rx=0 tx=0).stop 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.772+0000 7f87b1400700 1 -- 172.21.15.115:0/3069055627 shutdown_connections 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.772+0000 7f87b1400700 1 --2- 172.21.15.115:0/3069055627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ac103500 0x7f87ac1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.772+0000 7f87b1400700 1 --2- 172.21.15.115:0/3069055627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 0x7f87ac102fc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.772+0000 7f87b1400700 1 -- 172.21.15.115:0/3069055627 >> 172.21.15.115:0/3069055627 conn(0x7f87ac0fa5e0 msgr2=0x7f87ac0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.772+0000 7f87b1400700 1 -- 172.21.15.115:0/3069055627 shutdown_connections 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.772+0000 7f87b1400700 1 -- 172.21.15.115:0/3069055627 wait complete. 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.773+0000 7f87b1400700 1 Processor -- start 2024-03-22T10:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.774+0000 7f87b1400700 1 -- start start 2024-03-22T10:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.774+0000 7f87b1400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 0x7f87ac071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.774+0000 7f87b1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ac103500 0x7f87ac072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.774+0000 7f87b1400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac072840 con 0x7f87ac100be0 2024-03-22T10:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.774+0000 7f87b1400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac072980 con 0x7f87ac103500 2024-03-22T10:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.775+0000 7f87aa800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 0x7f87ac071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.775+0000 7f87aa800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 0x7f87ac071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49258/0 (socket says 172.21.15.115:49258) 2024-03-22T10:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.775+0000 7f87aa800700 1 -- 172.21.15.115:0/1567650010 learned_addr learned my addr 172.21.15.115:0/1567650010 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.775+0000 7f87a9e00700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ac103500 0x7f87ac072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:37.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.775+0000 7f87aa800700 1 -- 172.21.15.115:0/1567650010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ac103500 msgr2=0x7f87ac072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:37.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.775+0000 7f87aa800700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ac103500 0x7f87ac072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:37.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.775+0000 7f87aa800700 1 -- 172.21.15.115:0/1567650010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0095f0 con 0x7f87ac100be0 2024-03-22T10:52:37.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.776+0000 7f87aa800700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 0x7f87ac071ce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f879c000c00 tx=0x7f879c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:37.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.776+0000 7f87a3400700 1 -- 172.21.15.115:0/1567650010 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f879c01d070 con 0x7f87ac100be0 2024-03-22T10:52:37.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.776+0000 7f87a3400700 1 -- 172.21.15.115:0/1567650010 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f879c005dc0 con 0x7f87ac100be0 2024-03-22T10:52:37.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.776+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac0fedc0 con 0x7f87ac100be0 2024-03-22T10:52:37.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.776+0000 7f87a3400700 1 -- 172.21.15.115:0/1567650010 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f879c0226a0 con 0x7f87ac100be0 2024-03-22T10:52:37.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.777+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac0ff2b0 con 0x7f87ac100be0 2024-03-22T10:52:37.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.779+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac108c90 con 0x7f87ac100be0 2024-03-22T10:52:37.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.781+0000 7f87a3400700 1 -- 172.21.15.115:0/1567650010 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f879c0038f0 con 0x7f87ac100be0 2024-03-22T10:52:37.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.782+0000 7f87a3400700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f879006dff0 0x7f87900704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:52:37.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.783+0000 7f87a3400700 1 -- 172.21.15.115:0/1567650010 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f879c08ec20 con 0x7f87ac100be0 2024-03-22T10:52:37.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.783+0000 7f87a9e00700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f879006dff0 0x7f87900704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:52:37.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.784+0000 7f87a9e00700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f879006dff0 0x7f87900704a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8798009c60 tx=0x7f8798009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:52:37.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:37.788+0000 7f87a3400700 1 -- 172.21.15.115:0/1567650010 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f879c05bc60 con 0x7f87ac100be0 2024-03-22T10:52:38.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:37 smithi115 ceph-mon[78672]: pgmap v294: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:38.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:37 smithi115 ceph-mon[78672]: Upgrade: Updating node-exporter.smithi115 (1/2) 2024-03-22T10:52:38.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:37 smithi115 ceph-mon[78672]: Deploying daemon node-exporter.smithi115 on smithi115 2024-03-22T10:52:38.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:37 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3390588665' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:52:38.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.147+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f87ac06bca0 con 0x7f87ac100be0 2024-03-22T10:52:38.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.148+0000 7f87a3400700 1 -- 172.21.15.115:0/1567650010 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f879c05b7f0 con 0x7f87ac100be0 2024-03-22T10:52:38.139 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:52:38.139 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:52:38.139 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:52:38.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.150+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f879006dff0 msgr2=0x7f87900704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:38.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.150+0000 7f87b1400700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f879006dff0 0x7f87900704a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8798009c60 tx=0x7f8798009260 comp rx=0 tx=0).stop 2024-03-22T10:52:38.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.150+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 msgr2=0x7f87ac071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:52:38.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.150+0000 7f87b1400700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 0x7f87ac071ce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f879c000c00 tx=0x7f879c00f690 comp rx=0 tx=0).stop 2024-03-22T10:52:38.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.151+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 shutdown_connections 2024-03-22T10:52:38.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.151+0000 7f87b1400700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f879006dff0 0x7f87900704a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:38.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.151+0000 7f87b1400700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87ac100be0 0x7f87ac071ce0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:38.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.151+0000 7f87b1400700 1 --2- 172.21.15.115:0/1567650010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ac103500 0x7f87ac072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:52:38.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.151+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 >> 172.21.15.115:0/1567650010 conn(0x7f87ac0fa5e0 msgr2=0x7f87ac0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:52:38.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.151+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 shutdown_connections 2024-03-22T10:52:38.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:52:38.151+0000 7f87b1400700 1 -- 172.21.15.115:0/1567650010 wait complete. 2024-03-22T10:52:38.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:37 smithi150 ceph-mon[65513]: pgmap v294: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:38.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:37 smithi150 ceph-mon[65513]: Upgrade: Updating node-exporter.smithi115 (1/2) 2024-03-22T10:52:38.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:37 smithi150 ceph-mon[65513]: Deploying daemon node-exporter.smithi115 on smithi115 2024-03-22T10:52:38.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:37 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3390588665' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:52:39.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:38 smithi150 ceph-mon[65513]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:39.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:38 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1567650010' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:52:39.177 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:38 smithi115 ceph-mon[78672]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:52:39.177 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:38 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1567650010' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:52:40.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:39 smithi150 ceph-mon[65513]: pgmap v295: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:40.178 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:39 smithi115 ceph-mon[78672]: pgmap v295: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:41 smithi115 ceph-mon[78672]: pgmap v296: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:41.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:41.782 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:41 smithi150 ceph-mon[65513]: pgmap v296: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:41.782 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:41.782 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:43.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:43 smithi115 ceph-mon[78672]: pgmap v297: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:43.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:43 smithi115 ceph-mon[78672]: Upgrade: Updating node-exporter.smithi150 (2/2) 2024-03-22T10:52:43.761 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:43 smithi115 ceph-mon[78672]: Deploying daemon node-exporter.smithi150 on smithi150 2024-03-22T10:52:43.765 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:43 smithi150 ceph-mon[65513]: pgmap v297: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:43.765 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:43 smithi150 ceph-mon[65513]: Upgrade: Updating node-exporter.smithi150 (2/2) 2024-03-22T10:52:43.765 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:43 smithi150 ceph-mon[65513]: Deploying daemon node-exporter.smithi150 on smithi150 2024-03-22T10:52:45.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:45 smithi150 ceph-mon[65513]: pgmap v298: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:46.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:45 smithi115 ceph-mon[78672]: pgmap v298: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:48.139 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:47 smithi115 ceph-mon[78672]: pgmap v299: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:48.139 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:47 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:48.139 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:47 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:48.140 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:47 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:52:48.140 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:47 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:48.140 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:47 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:48.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:47 smithi150 ceph-mon[65513]: pgmap v299: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:48.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:47 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:48.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:47 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:48.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:47 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:52:48.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:47 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:48.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:47 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:50.130 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:49 smithi115 ceph-mon[78672]: pgmap v300: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:50.130 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:52:50.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:49 smithi150 ceph-mon[65513]: pgmap v300: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:50.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:52:52.147 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:51 smithi150 ceph-mon[65513]: pgmap v301: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:51 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:52.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:51 smithi115 ceph-mon[78672]: pgmap v301: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:52.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:52.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:51 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:54.244 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:53 smithi150 ceph-mon[65513]: pgmap v302: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:54.244 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:53 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:54.244 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:53 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:54.244 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:53 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:54.245 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:53 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:53 smithi115 ceph-mon[78672]: pgmap v302: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:53 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:53 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:53 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:54.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:53 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:56.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:55 smithi150 ceph-mon[65513]: pgmap v303: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:56.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:55 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:56.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:55 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:56.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:55 smithi115 ceph-mon[78672]: pgmap v303: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:56.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:55 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:56.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:55 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:52:58.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:57 smithi115 ceph-mon[78672]: pgmap v304: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:52:58.335 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:57 smithi150 ceph-mon[65513]: pgmap v304: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:00.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:59 smithi115 ceph-mon[78672]: pgmap v305: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:00.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:00.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:52:59 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:00.095 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:59 smithi150 ceph-mon[65513]: pgmap v305: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:00.095 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:00.095 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:52:59 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:01.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: pgmap v306: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:01.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:01.612 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.613 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.614 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.614 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.614 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:01 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: pgmap v306: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:01.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:01.649 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.650 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:01.651 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:01 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:03.644 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:03 smithi115 ceph-mon[78672]: pgmap v307: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:03.647 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:03 smithi150 ceph-mon[65513]: pgmap v307: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:04.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:04 smithi115 ceph-mon[78672]: Upgrade: Updating prometheus.smithi115 2024-03-22T10:53:04.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:04 smithi115 ceph-mon[78672]: Deploying daemon prometheus.smithi115 on smithi115 2024-03-22T10:53:04.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:53:04.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:04 smithi150 ceph-mon[65513]: Upgrade: Updating prometheus.smithi115 2024-03-22T10:53:04.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:04 smithi150 ceph-mon[65513]: Deploying daemon prometheus.smithi115 on smithi115 2024-03-22T10:53:04.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:53:05.648 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:05 smithi150 ceph-mon[65513]: pgmap v308: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:05.669 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:05 smithi115 ceph-mon[78672]: pgmap v308: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:07.760 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:07 smithi115 ceph-mon[78672]: pgmap v309: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:08.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:07 smithi150 ceph-mon[65513]: pgmap v309: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:08.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.340+0000 7f583e600700 1 -- 172.21.15.115:0/552300006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5838074d80 msgr2=0x7f58380731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:08.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.340+0000 7f583e600700 1 --2- 172.21.15.115:0/552300006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5838074d80 0x7f58380731e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5828009940 tx=0x7f5828009c50 comp rx=0 tx=0).stop 2024-03-22T10:53:08.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.341+0000 7f583e600700 1 -- 172.21.15.115:0/552300006 shutdown_connections 2024-03-22T10:53:08.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.341+0000 7f583e600700 1 --2- 172.21.15.115:0/552300006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58380737b0 0x7f5838073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:08.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.341+0000 7f583e600700 1 --2- 172.21.15.115:0/552300006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5838074d80 0x7f58380731e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:08.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.341+0000 7f583e600700 1 -- 172.21.15.115:0/552300006 >> 172.21.15.115:0/552300006 conn(0x7f58380fb6e0 msgr2=0x7f58380fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:08.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.341+0000 7f583e600700 1 -- 172.21.15.115:0/552300006 shutdown_connections 2024-03-22T10:53:08.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.341+0000 7f583e600700 1 -- 172.21.15.115:0/552300006 wait complete. 2024-03-22T10:53:08.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.342+0000 7f583e600700 1 Processor -- start 2024-03-22T10:53:08.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.342+0000 7f583e600700 1 -- start start 2024-03-22T10:53:08.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.343+0000 7f583e600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58380737b0 0x7f5838103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:08.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.343+0000 7f583e600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5838074d80 0x7f58381039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:08.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.343+0000 7f583e600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380ffee0 con 0x7f58380737b0 2024-03-22T10:53:08.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.343+0000 7f583e600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5838100050 con 0x7f5838074d80 2024-03-22T10:53:08.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.343+0000 7f5837400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58380737b0 0x7f5838103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:08.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.343+0000 7f5837400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58380737b0 0x7f5838103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49528/0 (socket says 172.21.15.115:49528) 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.343+0000 7f5837400700 1 -- 172.21.15.115:0/2585403190 learned_addr learned my addr 172.21.15.115:0/2585403190 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.343+0000 7f5836a00700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5838074d80 0x7f58381039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.344+0000 7f5837400700 1 -- 172.21.15.115:0/2585403190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5838074d80 msgr2=0x7f58381039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.344+0000 7f5837400700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5838074d80 0x7f58381039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.344+0000 7f5837400700 1 -- 172.21.15.115:0/2585403190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58280095f0 con 0x7f58380737b0 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.344+0000 7f5837400700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58380737b0 0x7f5838103480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5828005620 tx=0x7f5828005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.344+0000 7f582fe00700 1 -- 172.21.15.115:0/2585403190 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582801d070 con 0x7f58380737b0 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.345+0000 7f582fe00700 1 -- 172.21.15.115:0/2585403190 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f582800f460 con 0x7f58380737b0 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.345+0000 7f582fe00700 1 -- 172.21.15.115:0/2585403190 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5828021640 con 0x7f58380737b0 2024-03-22T10:53:08.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.345+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58381002d0 con 0x7f58380737b0 2024-03-22T10:53:08.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.345+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5838100840 con 0x7f58380737b0 2024-03-22T10:53:08.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.346+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5838100b20 con 0x7f58380737b0 2024-03-22T10:53:08.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.349+0000 7f582fe00700 1 -- 172.21.15.115:0/2585403190 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f582800f5d0 con 0x7f58380737b0 2024-03-22T10:53:08.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.350+0000 7f582fe00700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f581c06dfa0 0x7f581c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:08.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.351+0000 7f582fe00700 1 -- 172.21.15.115:0/2585403190 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f582808f580 con 0x7f58380737b0 2024-03-22T10:53:08.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.351+0000 7f5836a00700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f581c06dfa0 0x7f581c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:08.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.352+0000 7f5836a00700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f581c06dfa0 0x7f581c070450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5824005ea0 tx=0x7f5824005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:08.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.358+0000 7f582fe00700 1 -- 172.21.15.115:0/2585403190 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f582805c620 con 0x7f58380737b0 2024-03-22T10:53:08.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.716+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5838107650 con 0x7f581c06dfa0 2024-03-22T10:53:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.718+0000 7f582fe00700 1 -- 172.21.15.115:0/2585403190 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f5838107650 con 0x7f581c06dfa0 2024-03-22T10:53:08.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.721+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f581c06dfa0 msgr2=0x7f581c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:08.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.721+0000 7f583e600700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f581c06dfa0 0x7f581c070450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5824005ea0 tx=0x7f5824005e30 comp rx=0 tx=0).stop 2024-03-22T10:53:08.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.721+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58380737b0 msgr2=0x7f5838103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:08.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.721+0000 7f583e600700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58380737b0 0x7f5838103480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5828005620 tx=0x7f5828005700 comp rx=0 tx=0).stop 2024-03-22T10:53:08.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.722+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 shutdown_connections 2024-03-22T10:53:08.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.722+0000 7f583e600700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f581c06dfa0 0x7f581c070450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:08.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.722+0000 7f583e600700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58380737b0 0x7f5838103480 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:08.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.722+0000 7f583e600700 1 --2- 172.21.15.115:0/2585403190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5838074d80 0x7f58381039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:08.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.722+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 >> 172.21.15.115:0/2585403190 conn(0x7f58380fb6e0 msgr2=0x7f5838105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:08.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.722+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 shutdown_connections 2024-03-22T10:53:08.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.722+0000 7f583e600700 1 -- 172.21.15.115:0/2585403190 wait complete. 2024-03-22T10:53:08.731 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:53:08.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.908+0000 7fd922200700 1 -- 172.21.15.115:0/224716825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c074d20 msgr2=0x7fd91c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:08.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.908+0000 7fd922200700 1 --2- 172.21.15.115:0/224716825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c074d20 0x7fd91c073180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009910 tx=0x7fd90c009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:08.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.909+0000 7fd922200700 1 -- 172.21.15.115:0/224716825 shutdown_connections 2024-03-22T10:53:08.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.909+0000 7fd922200700 1 --2- 172.21.15.115:0/224716825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd91c073750 0x7fd91c073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:08.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.909+0000 7fd922200700 1 --2- 172.21.15.115:0/224716825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c074d20 0x7fd91c073180 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:08.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.909+0000 7fd922200700 1 -- 172.21.15.115:0/224716825 >> 172.21.15.115:0/224716825 conn(0x7fd91c0fb740 msgr2=0x7fd91c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:08.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.909+0000 7fd922200700 1 -- 172.21.15.115:0/224716825 shutdown_connections 2024-03-22T10:53:08.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.909+0000 7fd922200700 1 -- 172.21.15.115:0/224716825 wait complete. 2024-03-22T10:53:08.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.910+0000 7fd922200700 1 Processor -- start 2024-03-22T10:53:08.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.910+0000 7fd922200700 1 -- start start 2024-03-22T10:53:08.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.910+0000 7fd922200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c073750 0x7fd91c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:08.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.910+0000 7fd922200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd91c074d20 0x7fd91c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:08.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.910+0000 7fd922200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1986b0 con 0x7fd91c073750 2024-03-22T10:53:08.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.910+0000 7fd922200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1987f0 con 0x7fd91c074d20 2024-03-22T10:53:08.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.911+0000 7fd91b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c073750 0x7fd91c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:08.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.911+0000 7fd91b400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c073750 0x7fd91c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49548/0 (socket says 172.21.15.115:49548) 2024-03-22T10:53:08.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.911+0000 7fd91b400700 1 -- 172.21.15.115:0/812383167 learned_addr learned my addr 172.21.15.115:0/812383167 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:08.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.911+0000 7fd91aa00700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd91c074d20 0x7fd91c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:08.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.911+0000 7fd91b400700 1 -- 172.21.15.115:0/812383167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd91c074d20 msgr2=0x7fd91c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:08.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.911+0000 7fd91b400700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd91c074d20 0x7fd91c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:08.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.911+0000 7fd91b400700 1 -- 172.21.15.115:0/812383167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0095f0 con 0x7fd91c073750 2024-03-22T10:53:08.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.912+0000 7fd91b400700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c073750 0x7fd91c197b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd90c005b40 tx=0x7fd90c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:08.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.912+0000 7fd913e00700 1 -- 172.21.15.115:0/812383167 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd90c01d070 con 0x7fd91c073750 2024-03-22T10:53:08.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.912+0000 7fd913e00700 1 -- 172.21.15.115:0/812383167 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd90c00f460 con 0x7fd91c073750 2024-03-22T10:53:08.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.912+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c19d240 con 0x7fd91c073750 2024-03-22T10:53:08.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.912+0000 7fd913e00700 1 -- 172.21.15.115:0/812383167 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd90c021640 con 0x7fd91c073750 2024-03-22T10:53:08.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.912+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c19d730 con 0x7fd91c073750 2024-03-22T10:53:08.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.914+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91c10aed0 con 0x7fd91c073750 2024-03-22T10:53:08.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.915+0000 7fd913e00700 1 -- 172.21.15.115:0/812383167 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fd90c00fa90 con 0x7fd91c073750 2024-03-22T10:53:08.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.916+0000 7fd913e00700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd90006dfa0 0x7fd900070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:08.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.917+0000 7fd913e00700 1 -- 172.21.15.115:0/812383167 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd90c08f150 con 0x7fd91c073750 2024-03-22T10:53:08.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.917+0000 7fd91aa00700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd90006dfa0 0x7fd900070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:08.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.919+0000 7fd91aa00700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd90006dfa0 0x7fd900070450 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd908005950 tx=0x7fd9080058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:08.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:08.926+0000 7fd913e00700 1 -- 172.21.15.115:0/812383167 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd90c05c050 con 0x7fd91c073750 2024-03-22T10:53:09.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.203+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd91c1076f0 con 0x7fd90006dfa0 2024-03-22T10:53:09.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.205+0000 7fd913e00700 1 -- 172.21.15.115:0/812383167 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7fd91c1076f0 con 0x7fd90006dfa0 2024-03-22T10:53:09.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.208+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd90006dfa0 msgr2=0x7fd900070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:09.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.208+0000 7fd922200700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd90006dfa0 0x7fd900070450 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd908005950 tx=0x7fd9080058e0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.208+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c073750 msgr2=0x7fd91c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:09.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.208+0000 7fd922200700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c073750 0x7fd91c197b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd90c005b40 tx=0x7fd90c00be20 comp rx=0 tx=0).stop 2024-03-22T10:53:09.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.208+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 shutdown_connections 2024-03-22T10:53:09.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.208+0000 7fd922200700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd90006dfa0 0x7fd900070450 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.208+0000 7fd922200700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd91c073750 0x7fd91c197b50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.208+0000 7fd922200700 1 --2- 172.21.15.115:0/812383167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd91c074d20 0x7fd91c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.208+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 >> 172.21.15.115:0/812383167 conn(0x7fd91c0fb740 msgr2=0x7fd91c105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:09.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.209+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 shutdown_connections 2024-03-22T10:53:09.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.209+0000 7fd922200700 1 -- 172.21.15.115:0/812383167 wait complete. 2024-03-22T10:53:09.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.379+0000 7fd25bc00700 1 -- 172.21.15.115:0/152489269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 msgr2=0x7fd254100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:09.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.379+0000 7fd25bc00700 1 --2- 172.21.15.115:0/152489269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 0x7fd254100560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd248009910 tx=0x7fd248009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:09.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.379+0000 7fd25bc00700 1 -- 172.21.15.115:0/152489269 shutdown_connections 2024-03-22T10:53:09.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.379+0000 7fd25bc00700 1 --2- 172.21.15.115:0/152489269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd254101350 0x7fd2541017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.379+0000 7fd25bc00700 1 --2- 172.21.15.115:0/152489269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 0x7fd254100560 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.379+0000 7fd25bc00700 1 -- 172.21.15.115:0/152489269 >> 172.21.15.115:0/152489269 conn(0x7fd2540fb6e0 msgr2=0x7fd2540fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:09.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.379+0000 7fd25bc00700 1 -- 172.21.15.115:0/152489269 shutdown_connections 2024-03-22T10:53:09.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.379+0000 7fd25bc00700 1 -- 172.21.15.115:0/152489269 wait complete. 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.381+0000 7fd25bc00700 1 Processor -- start 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.381+0000 7fd25bc00700 1 -- start start 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.381+0000 7fd25bc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 0x7fd254197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.381+0000 7fd25bc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd254101350 0x7fd254198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.381+0000 7fd25bc00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541987a0 con 0x7fd254100150 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.381+0000 7fd25bc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25419d1b0 con 0x7fd254101350 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.382+0000 7fd259200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 0x7fd254197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.382+0000 7fd259200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 0x7fd254197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49566/0 (socket says 172.21.15.115:49566) 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.382+0000 7fd259200700 1 -- 172.21.15.115:0/3603427899 learned_addr learned my addr 172.21.15.115:0/3603427899 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.382+0000 7fd253e00700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd254101350 0x7fd254198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.382+0000 7fd259200700 1 -- 172.21.15.115:0/3603427899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd254101350 msgr2=0x7fd254198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:09.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.382+0000 7fd259200700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd254101350 0x7fd254198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.382+0000 7fd259200700 1 -- 172.21.15.115:0/3603427899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480095f0 con 0x7fd254100150 2024-03-22T10:53:09.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.382+0000 7fd259200700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 0x7fd254197c40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd24800bd70 tx=0x7fd24800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:09.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.383+0000 7fd251600700 1 -- 172.21.15.115:0/3603427899 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd24801d070 con 0x7fd254100150 2024-03-22T10:53:09.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.383+0000 7fd251600700 1 -- 172.21.15.115:0/3603427899 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd24800f460 con 0x7fd254100150 2024-03-22T10:53:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.383+0000 7fd251600700 1 -- 172.21.15.115:0/3603427899 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd248021640 con 0x7fd254100150 2024-03-22T10:53:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.383+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25419d350 con 0x7fd254100150 2024-03-22T10:53:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.383+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25419d890 con 0x7fd254100150 2024-03-22T10:53:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.384+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd254105430 con 0x7fd254100150 2024-03-22T10:53:09.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.385+0000 7fd251600700 1 -- 172.21.15.115:0/3603427899 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fd24800fab0 con 0x7fd254100150 2024-03-22T10:53:09.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.386+0000 7fd251600700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd23c06dfa0 0x7fd23c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.386+0000 7fd251600700 1 -- 172.21.15.115:0/3603427899 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd24808f1d0 con 0x7fd254100150 2024-03-22T10:53:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.387+0000 7fd253e00700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd23c06dfa0 0x7fd23c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.387+0000 7fd253e00700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd23c06dfa0 0x7fd23c070450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd240006fd0 tx=0x7fd240008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:09.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.391+0000 7fd251600700 1 -- 172.21.15.115:0/3603427899 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd248058620 con 0x7fd254100150 2024-03-22T10:53:09.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.748+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd254061190 con 0x7fd23c06dfa0 2024-03-22T10:53:09.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.763+0000 7fd251600700 1 -- 172.21.15.115:0/3603427899 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd254061190 con 0x7fd23c06dfa0 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (19m) 16s ago 21m 14.7M - 0.25.0 c8568f914cd2 d303a12b571d 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (57s) 16s ago 21m 7998k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e11b0bdaf069 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (49s) 18s ago 20m 7662k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 ef9665eef7b1 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (7m) 16s ago 21m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (7m) 18s ago 20m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (19m) 16s ago 21m 73.8M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (2m) 16s ago 14m 67.0M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 44a0769ebe8e 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (2m) 16s ago 14m 16.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51d700f6eb18 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (78s) 18s ago 14m 12.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 82b6b216f754 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (104s) 18s ago 14m 12.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9605445d5b2b 2024-03-22T10:53:09.755 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (11m) 16s ago 23m 514M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (9m) 18s ago 20m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (9m) 16s ago 23m 58.9M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (8m) 18s ago 20m 45.7M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (29s) 16s ago 21m 6832k - 1.5.0 0da6a335fe13 a98cb20e74e1 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (23s) 18s ago 20m 2935k - 1.5.0 0da6a335fe13 6c3ba9d80db0 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (6m) 16s ago 19m 97.8M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (6m) 16s ago 18m 96.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (5m) 16s ago 17m 83.4M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (5m) 18s ago 17m 103M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (4m) 18s ago 16m 91.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (3m) 18s ago 16m 90.9M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:53:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (9m) 16s ago 20m 79.3M - 2.43.0 a07b618ecd1d ccbe30a2875f 2024-03-22T10:53:09.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.766+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd23c06dfa0 msgr2=0x7fd23c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:09.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.766+0000 7fd25bc00700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd23c06dfa0 0x7fd23c070450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd240006fd0 tx=0x7fd240008040 comp rx=0 tx=0).stop 2024-03-22T10:53:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.766+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 msgr2=0x7fd254197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.766+0000 7fd25bc00700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 0x7fd254197c40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd24800bd70 tx=0x7fd24800be50 comp rx=0 tx=0).stop 2024-03-22T10:53:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.767+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 shutdown_connections 2024-03-22T10:53:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.767+0000 7fd25bc00700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd23c06dfa0 0x7fd23c070450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.767+0000 7fd25bc00700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd254100150 0x7fd254197c40 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd24800bd70 tx=0x7fd24800be50 comp rx=0 tx=0).stop 2024-03-22T10:53:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.767+0000 7fd25bc00700 1 --2- 172.21.15.115:0/3603427899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd254101350 0x7fd254198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.767+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 >> 172.21.15.115:0/3603427899 conn(0x7fd2540fb6e0 msgr2=0x7fd254102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.767+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 shutdown_connections 2024-03-22T10:53:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.767+0000 7fd25bc00700 1 -- 172.21.15.115:0/3603427899 wait complete. 2024-03-22T10:53:09.935 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:09 smithi115 ceph-mon[78672]: pgmap v310: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:09.935 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:09 smithi115 ceph-mon[78672]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:09.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.940+0000 7f457c600700 1 -- 172.21.15.115:0/511895817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45740737b0 msgr2=0x7f4574073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:09.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.940+0000 7f457c600700 1 --2- 172.21.15.115:0/511895817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45740737b0 0x7f4574073c20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4560009910 tx=0x7f4560009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:09.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.940+0000 7f457c600700 1 -- 172.21.15.115:0/511895817 shutdown_connections 2024-03-22T10:53:09.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.940+0000 7f457c600700 1 --2- 172.21.15.115:0/511895817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45740737b0 0x7f4574073c20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.940+0000 7f457c600700 1 --2- 172.21.15.115:0/511895817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4574074d80 0x7f45740731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.940+0000 7f457c600700 1 -- 172.21.15.115:0/511895817 >> 172.21.15.115:0/511895817 conn(0x7f45740fb6e0 msgr2=0x7f45740fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.941+0000 7f457c600700 1 -- 172.21.15.115:0/511895817 shutdown_connections 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.941+0000 7f457c600700 1 -- 172.21.15.115:0/511895817 wait complete. 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.942+0000 7f457c600700 1 Processor -- start 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.942+0000 7f457c600700 1 -- start start 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.942+0000 7f457c600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45740737b0 0x7f457419c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.942+0000 7f457c600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4574074d80 0x7f457419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.942+0000 7f457c600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457419cbc0 con 0x7f4574074d80 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.942+0000 7f457c600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457419cd00 con 0x7f45740737b0 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.943+0000 7f4579200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4574074d80 0x7f457419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.943+0000 7f4579200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4574074d80 0x7f457419c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49584/0 (socket says 172.21.15.115:49584) 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.943+0000 7f4579200700 1 -- 172.21.15.115:0/4105356480 learned_addr learned my addr 172.21.15.115:0/4105356480 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.943+0000 7f4579c00700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45740737b0 0x7f457419c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.944+0000 7f4579c00700 1 -- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4574074d80 msgr2=0x7f457419c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.944+0000 7f4579c00700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4574074d80 0x7f457419c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.944+0000 7f4579c00700 1 -- 172.21.15.115:0/4105356480 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45600095f0 con 0x7f45740737b0 2024-03-22T10:53:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.944+0000 7f4579200700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4574074d80 0x7f457419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T10:53:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.945+0000 7f4579c00700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45740737b0 0x7f457419c060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f456800b3d0 tx=0x7f456800b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.946+0000 7f4572000700 1 -- 172.21.15.115:0/4105356480 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4568011800 con 0x7f45740737b0 2024-03-22T10:53:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.946+0000 7f4572000700 1 -- 172.21.15.115:0/4105356480 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f456800be90 con 0x7f45740737b0 2024-03-22T10:53:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.946+0000 7f4572000700 1 -- 172.21.15.115:0/4105356480 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4568010e10 con 0x7f45740737b0 2024-03-22T10:53:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.947+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45741a17b0 con 0x7f45740737b0 2024-03-22T10:53:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.947+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45741a1d30 con 0x7f45740737b0 2024-03-22T10:53:09.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.950+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457410f300 con 0x7f45740737b0 2024-03-22T10:53:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.954+0000 7f4572000700 1 -- 172.21.15.115:0/4105356480 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4568039ac0 con 0x7f45740737b0 2024-03-22T10:53:09.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.956+0000 7f4572000700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f455c06e110 0x7f455c0705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:09.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.956+0000 7f4572000700 1 -- 172.21.15.115:0/4105356480 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f456808dae0 con 0x7f45740737b0 2024-03-22T10:53:09.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.957+0000 7f4579200700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f455c06e110 0x7f455c0705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:09.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.959+0000 7f4579200700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f455c06e110 0x7f455c0705c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4560009fd0 tx=0x7f45600058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:09.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:09.969+0000 7f4572000700 1 -- 172.21.15.115:0/4105356480 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f456805aa60 con 0x7f45740737b0 2024-03-22T10:53:10.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:09 smithi150 ceph-mon[65513]: pgmap v310: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:10.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:09 smithi150 ceph-mon[65513]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.340+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f45741a2830 con 0x7f45740737b0 2024-03-22T10:53:10.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.341+0000 7f4572000700 1 -- 172.21.15.115:0/4105356480 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f456805a5f0 con 0x7f45740737b0 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:53:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:53:10.334 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-22T10:53:10.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:53:10.334 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:53:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f455c06e110 msgr2=0x7f455c0705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f455c06e110 0x7f455c0705c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4560009fd0 tx=0x7f45600058e0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45740737b0 msgr2=0x7f457419c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45740737b0 0x7f457419c060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f456800b3d0 tx=0x7f456800b6e0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 shutdown_connections 2024-03-22T10:53:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f455c06e110 0x7f455c0705c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45740737b0 0x7f457419c060 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 --2- 172.21.15.115:0/4105356480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4574074d80 0x7f457419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 >> 172.21.15.115:0/4105356480 conn(0x7f45740fb6e0 msgr2=0x7f4574101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.344+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 shutdown_connections 2024-03-22T10:53:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.345+0000 7f457c600700 1 -- 172.21.15.115:0/4105356480 wait complete. 2024-03-22T10:53:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.524+0000 7f36efc00700 1 -- 172.21.15.115:0/1603817960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8101360 msgr2=0x7f36e81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.524+0000 7f36efc00700 1 --2- 172.21.15.115:0/1603817960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8101360 0x7f36e81017b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009910 tx=0x7f36d4009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.524+0000 7f36efc00700 1 -- 172.21.15.115:0/1603817960 shutdown_connections 2024-03-22T10:53:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.524+0000 7f36efc00700 1 --2- 172.21.15.115:0/1603817960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8101360 0x7f36e81017b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.524+0000 7f36efc00700 1 --2- 172.21.15.115:0/1603817960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e8100160 0x7f36e8100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.524+0000 7f36efc00700 1 -- 172.21.15.115:0/1603817960 >> 172.21.15.115:0/1603817960 conn(0x7f36e80fb6f0 msgr2=0x7f36e80fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.524+0000 7f36efc00700 1 -- 172.21.15.115:0/1603817960 shutdown_connections 2024-03-22T10:53:10.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.525+0000 7f36efc00700 1 -- 172.21.15.115:0/1603817960 wait complete. 2024-03-22T10:53:10.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.526+0000 7f36efc00700 1 Processor -- start 2024-03-22T10:53:10.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.526+0000 7f36efc00700 1 -- start start 2024-03-22T10:53:10.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.527+0000 7f36efc00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8100160 0x7f36e8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:10.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.527+0000 7f36efc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e8198300 0x7f36e819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:10.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.527+0000 7f36efc00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8198800 con 0x7f36e8100160 2024-03-22T10:53:10.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.527+0000 7f36efc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8198970 con 0x7f36e8198300 2024-03-22T10:53:10.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.527+0000 7f36ed200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8100160 0x7f36e8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:10.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.527+0000 7f36ed200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8100160 0x7f36e8197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49606/0 (socket says 172.21.15.115:49606) 2024-03-22T10:53:10.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.527+0000 7f36ed200700 1 -- 172.21.15.115:0/2306185022 learned_addr learned my addr 172.21.15.115:0/2306185022 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:10.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.527+0000 7f36e7e00700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e8198300 0x7f36e819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:10.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.528+0000 7f36ed200700 1 -- 172.21.15.115:0/2306185022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e8198300 msgr2=0x7f36e819d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:10.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.528+0000 7f36ed200700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e8198300 0x7f36e819d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.528+0000 7f36ed200700 1 -- 172.21.15.115:0/2306185022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d40095f0 con 0x7f36e8100160 2024-03-22T10:53:10.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.528+0000 7f36e7e00700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e8198300 0x7f36e819d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:53:10.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.528+0000 7f36ed200700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8100160 0x7f36e8197dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00d900 tx=0x7f36dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:10.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.528+0000 7f36e5600700 1 -- 172.21.15.115:0/2306185022 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36dc0041f0 con 0x7f36e8100160 2024-03-22T10:53:10.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.529+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e819d910 con 0x7f36e8100160 2024-03-22T10:53:10.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.529+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e819de30 con 0x7f36e8100160 2024-03-22T10:53:10.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.529+0000 7f36e5600700 1 -- 172.21.15.115:0/2306185022 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f36dc004350 con 0x7f36e8100160 2024-03-22T10:53:10.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.529+0000 7f36e5600700 1 -- 172.21.15.115:0/2306185022 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36dc003da0 con 0x7f36e8100160 2024-03-22T10:53:10.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.530+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e8105440 con 0x7f36e8100160 2024-03-22T10:53:10.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.532+0000 7f36e5600700 1 -- 172.21.15.115:0/2306185022 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f36dc009730 con 0x7f36e8100160 2024-03-22T10:53:10.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.533+0000 7f36e5600700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f36d006dfa0 0x7f36d0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:10.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.533+0000 7f36e5600700 1 -- 172.21.15.115:0/2306185022 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f36dc08dd50 con 0x7f36e8100160 2024-03-22T10:53:10.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.533+0000 7f36e7e00700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f36d006dfa0 0x7f36d0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:10.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.534+0000 7f36e7e00700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f36d006dfa0 0x7f36d0070450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009fd0 tx=0x7f36d400b560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:10.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.542+0000 7f36e5600700 1 -- 172.21.15.115:0/2306185022 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f36dc05ac50 con 0x7f36e8100160 2024-03-22T10:53:10.807 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:10 smithi115 ceph-mon[78672]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:10.808 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:10 smithi115 ceph-mon[78672]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:10.808 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:10 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/4105356480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:10.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.859+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f36e8061a60 con 0x7f36e8100160 2024-03-22T10:53:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.860+0000 7f36e5600700 1 -- 172.21.15.115:0/2306185022 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1932 (secure 0 0 0) 0x7f36dc05a7e0 con 0x7f36e8100160 2024-03-22T10:53:10.851 INFO:teuthology.orchestra.run.smithi115.stdout:e40 2024-03-22T10:53:10.851 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:53:10.851 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:53:10.851 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:53:10.851 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:53:10.851 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:53:10.851 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 40 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:52:05.899739+0000 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:53:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 83 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=34476,1=34508} 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:53:10.853 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 34476 members: 34476,34508 2024-03-22T10:53:10.854 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:34476} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:53:10.854 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{1:34508} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:53:10.854 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:53:10.854 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:53:10.854 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:53:10.854 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:53:10.854 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6824/1804848703,v1:172.21.15.150:6825/1804848703] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:53:10.854 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6826/2053213512,v1:172.21.15.150:6827/2053213512] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:53:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f36d006dfa0 msgr2=0x7f36d0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f36d006dfa0 0x7f36d0070450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009fd0 tx=0x7f36d400b560 comp rx=0 tx=0).stop 2024-03-22T10:53:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8100160 msgr2=0x7f36e8197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8100160 0x7f36e8197dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00d900 tx=0x7f36dc00dcc0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 shutdown_connections 2024-03-22T10:53:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f36d006dfa0 0x7f36d0070450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36e8100160 0x7f36e8197dc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 --2- 172.21.15.115:0/2306185022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e8198300 0x7f36e819d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 >> 172.21.15.115:0/2306185022 conn(0x7f36e80fb6f0 msgr2=0x7f36e8102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 shutdown_connections 2024-03-22T10:53:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:10.864+0000 7f36efc00700 1 -- 172.21.15.115:0/2306185022 wait complete. 2024-03-22T10:53:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2024-03-22T10:53:11.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.044+0000 7feb93e00700 1 -- 172.21.15.115:0/2244357831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1067a0 msgr2=0x7feb8c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:11.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.044+0000 7feb93e00700 1 --2- 172.21.15.115:0/2244357831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1067a0 0x7feb8c106bb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feb80009910 tx=0x7feb80009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:11.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.044+0000 7feb93e00700 1 -- 172.21.15.115:0/2244357831 shutdown_connections 2024-03-22T10:53:11.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.044+0000 7feb93e00700 1 --2- 172.21.15.115:0/2244357831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c1079a0 0x7feb8c107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.044+0000 7feb93e00700 1 --2- 172.21.15.115:0/2244357831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1067a0 0x7feb8c106bb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.044+0000 7feb93e00700 1 -- 172.21.15.115:0/2244357831 >> 172.21.15.115:0/2244357831 conn(0x7feb8c069ae0 msgr2=0x7feb8c069f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:11.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.045+0000 7feb93e00700 1 -- 172.21.15.115:0/2244357831 shutdown_connections 2024-03-22T10:53:11.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.045+0000 7feb93e00700 1 -- 172.21.15.115:0/2244357831 wait complete. 2024-03-22T10:53:11.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.046+0000 7feb93e00700 1 Processor -- start 2024-03-22T10:53:11.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.046+0000 7feb93e00700 1 -- start start 2024-03-22T10:53:11.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.047+0000 7feb93e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c1067a0 0x7feb8c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:11.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.047+0000 7feb93e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1079a0 0x7feb8c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:11.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.047+0000 7feb93e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c19cb00 con 0x7feb8c1079a0 2024-03-22T10:53:11.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.047+0000 7feb93e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c19cc40 con 0x7feb8c1067a0 2024-03-22T10:53:11.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.047+0000 7feb90a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1079a0 0x7feb8c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:11.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.048+0000 7feb90a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1079a0 0x7feb8c19c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49620/0 (socket says 172.21.15.115:49620) 2024-03-22T10:53:11.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.048+0000 7feb90a00700 1 -- 172.21.15.115:0/1555773902 learned_addr learned my addr 172.21.15.115:0/1555773902 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:11.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.048+0000 7feb91400700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c1067a0 0x7feb8c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:11.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.048+0000 7feb90a00700 1 -- 172.21.15.115:0/1555773902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c1067a0 msgr2=0x7feb8c19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:11.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.048+0000 7feb90a00700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c1067a0 0x7feb8c19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.048+0000 7feb90a00700 1 -- 172.21.15.115:0/1555773902 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb800095f0 con 0x7feb8c1079a0 2024-03-22T10:53:11.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.049+0000 7feb90a00700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1079a0 0x7feb8c19c4e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00e920 tx=0x7feb7c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:11.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.049+0000 7feb8a000700 1 -- 172.21.15.115:0/1555773902 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb7c00c9d0 con 0x7feb8c1079a0 2024-03-22T10:53:11.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.050+0000 7feb8a000700 1 -- 172.21.15.115:0/1555773902 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb7c010430 con 0x7feb8c1079a0 2024-03-22T10:53:11.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.050+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c1a16f0 con 0x7feb8c1079a0 2024-03-22T10:53:11.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.050+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c1a1c40 con 0x7feb8c1079a0 2024-03-22T10:53:11.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.050+0000 7feb8a000700 1 -- 172.21.15.115:0/1555773902 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb7c018650 con 0x7feb8c1079a0 2024-03-22T10:53:11.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.052+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb70005320 con 0x7feb8c1079a0 2024-03-22T10:53:11.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.054+0000 7feb8a000700 1 -- 172.21.15.115:0/1555773902 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7feb7c00cb30 con 0x7feb8c1079a0 2024-03-22T10:53:11.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.057+0000 7feb8a000700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feb7406dfa0 0x7feb74070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:11.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.057+0000 7feb8a000700 1 -- 172.21.15.115:0/1555773902 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7feb7c014070 con 0x7feb8c1079a0 2024-03-22T10:53:11.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.057+0000 7feb91400700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feb7406dfa0 0x7feb74070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:11.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.058+0000 7feb91400700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feb7406dfa0 0x7feb74070450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb80009fd0 tx=0x7feb80005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:11.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.062+0000 7feb8a000700 1 -- 172.21.15.115:0/1555773902 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feb7c053740 con 0x7feb8c1079a0 2024-03-22T10:53:11.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:10 smithi150 ceph-mon[65513]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:11.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:10 smithi150 ceph-mon[65513]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:11.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:10 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/4105356480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:11.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.398+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb70000bf0 con 0x7feb7406dfa0 2024-03-22T10:53:11.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.400+0000 7feb8a000700 1 -- 172.21.15.115:0/1555773902 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7feb70000bf0 con 0x7feb7406dfa0 2024-03-22T10:53:11.391 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:53:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:53:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "osd", 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "mds", 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph-exporter" 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "18/23 daemons upgraded", 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:53:11.392 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:53:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.403+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feb7406dfa0 msgr2=0x7feb74070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.403+0000 7feb93e00700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feb7406dfa0 0x7feb74070450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb80009fd0 tx=0x7feb80005fd0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.403+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1079a0 msgr2=0x7feb8c19c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:11.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.403+0000 7feb93e00700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1079a0 0x7feb8c19c4e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00e920 tx=0x7feb7c00ec30 comp rx=0 tx=0).stop 2024-03-22T10:53:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.404+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 shutdown_connections 2024-03-22T10:53:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.404+0000 7feb93e00700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feb7406dfa0 0x7feb74070450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.404+0000 7feb93e00700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c1067a0 0x7feb8c19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.404+0000 7feb93e00700 1 --2- 172.21.15.115:0/1555773902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c1079a0 0x7feb8c19c4e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.404+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 >> 172.21.15.115:0/1555773902 conn(0x7feb8c069ae0 msgr2=0x7feb8c10abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.404+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 shutdown_connections 2024-03-22T10:53:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.404+0000 7feb93e00700 1 -- 172.21.15.115:0/1555773902 wait complete. 2024-03-22T10:53:11.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.555+0000 7fbe4a200700 1 -- 172.21.15.115:0/1559713622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44103500 msgr2=0x7fbe441058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:11.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.555+0000 7fbe4a200700 1 --2- 172.21.15.115:0/1559713622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44103500 0x7fbe441058e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009910 tx=0x7fbe30009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:11.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.555+0000 7fbe4a200700 1 -- 172.21.15.115:0/1559713622 shutdown_connections 2024-03-22T10:53:11.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.555+0000 7fbe4a200700 1 --2- 172.21.15.115:0/1559713622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44103500 0x7fbe441058e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.555+0000 7fbe4a200700 1 --2- 172.21.15.115:0/1559713622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe44100be0 0x7fbe44102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.555+0000 7fbe4a200700 1 -- 172.21.15.115:0/1559713622 >> 172.21.15.115:0/1559713622 conn(0x7fbe440fa5e0 msgr2=0x7fbe440fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:11.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.555+0000 7fbe4a200700 1 -- 172.21.15.115:0/1559713622 shutdown_connections 2024-03-22T10:53:11.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.555+0000 7fbe4a200700 1 -- 172.21.15.115:0/1559713622 wait complete. 2024-03-22T10:53:11.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.557+0000 7fbe4a200700 1 Processor -- start 2024-03-22T10:53:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.557+0000 7fbe4a200700 1 -- start start 2024-03-22T10:53:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.557+0000 7fbe4a200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44100be0 0x7fbe44100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.557+0000 7fbe4a200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe44103500 0x7fbe440feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.557+0000 7fbe4a200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440ff0d0 con 0x7fbe44100be0 2024-03-22T10:53:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.557+0000 7fbe4a200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440ff210 con 0x7fbe44103500 2024-03-22T10:53:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.557+0000 7fbe43400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44100be0 0x7fbe44100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.557+0000 7fbe43400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44100be0 0x7fbe44100540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49640/0 (socket says 172.21.15.115:49640) 2024-03-22T10:53:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.558+0000 7fbe43400700 1 -- 172.21.15.115:0/654591152 learned_addr learned my addr 172.21.15.115:0/654591152 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.558+0000 7fbe42a00700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe44103500 0x7fbe440feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.558+0000 7fbe43400700 1 -- 172.21.15.115:0/654591152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe44103500 msgr2=0x7fbe440feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.558+0000 7fbe43400700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe44103500 0x7fbe440feb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.558+0000 7fbe43400700 1 -- 172.21.15.115:0/654591152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300095f0 con 0x7fbe44100be0 2024-03-22T10:53:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.558+0000 7fbe43400700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44100be0 0x7fbe44100540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400d8d0 tx=0x7fbe3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.559+0000 7fbe3be00700 1 -- 172.21.15.115:0/654591152 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe34009940 con 0x7fbe44100be0 2024-03-22T10:53:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.559+0000 7fbe3be00700 1 -- 172.21.15.115:0/654591152 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe34010460 con 0x7fbe44100be0 2024-03-22T10:53:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.559+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe440ff4f0 con 0x7fbe44100be0 2024-03-22T10:53:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.559+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4419d990 con 0x7fbe44100be0 2024-03-22T10:53:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.559+0000 7fbe3be00700 1 -- 172.21.15.115:0/654591152 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe3400f5f0 con 0x7fbe44100be0 2024-03-22T10:53:11.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.560+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4418fb70 con 0x7fbe44100be0 2024-03-22T10:53:11.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.561+0000 7fbe3be00700 1 -- 172.21.15.115:0/654591152 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fbe340105d0 con 0x7fbe44100be0 2024-03-22T10:53:11.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.563+0000 7fbe3be00700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbe2806dfa0 0x7fbe28070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:11.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.563+0000 7fbe3be00700 1 -- 172.21.15.115:0/654591152 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fbe3408dd60 con 0x7fbe44100be0 2024-03-22T10:53:11.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.563+0000 7fbe42a00700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbe2806dfa0 0x7fbe28070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:11.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.564+0000 7fbe42a00700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbe2806dfa0 0x7fbe28070450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbe30006010 tx=0x7fbe3000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:11.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.571+0000 7fbe3be00700 1 -- 172.21.15.115:0/654591152 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbe34092070 con 0x7fbe44100be0 2024-03-22T10:53:11.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.958+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbe4402cc30 con 0x7fbe44100be0 2024-03-22T10:53:11.950 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:11 smithi115 ceph-mon[78672]: pgmap v311: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:11.950 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:11 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2306185022' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:53:11.950 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:11.950 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:11.951 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:11 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:11.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.959+0000 7fbe3be00700 1 -- 172.21.15.115:0/654591152 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fbe340160e0 con 0x7fbe44100be0 2024-03-22T10:53:11.951 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:53:11.952 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:53:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:53:11.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.962+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbe2806dfa0 msgr2=0x7fbe28070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:11.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.962+0000 7fbe4a200700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbe2806dfa0 0x7fbe28070450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbe30006010 tx=0x7fbe3000b540 comp rx=0 tx=0).stop 2024-03-22T10:53:11.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.962+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44100be0 msgr2=0x7fbe44100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:11.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.962+0000 7fbe4a200700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44100be0 0x7fbe44100540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400d8d0 tx=0x7fbe3400dc90 comp rx=0 tx=0).stop 2024-03-22T10:53:11.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.963+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 shutdown_connections 2024-03-22T10:53:11.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.963+0000 7fbe4a200700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fbe2806dfa0 0x7fbe28070450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.963+0000 7fbe4a200700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe44100be0 0x7fbe44100540 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.963+0000 7fbe4a200700 1 --2- 172.21.15.115:0/654591152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe44103500 0x7fbe440feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:11.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.963+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 >> 172.21.15.115:0/654591152 conn(0x7fbe440fa5e0 msgr2=0x7fbe440fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:11.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.963+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 shutdown_connections 2024-03-22T10:53:11.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:11.963+0000 7fbe4a200700 1 -- 172.21.15.115:0/654591152 wait complete. 2024-03-22T10:53:12.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:11 smithi150 ceph-mon[65513]: pgmap v311: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:12.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:11 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2306185022' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:53:12.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:12.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:12.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:11 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:13.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:12 smithi115 ceph-mon[78672]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:13.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:12 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/654591152' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:53:13.051 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:12 smithi150 ceph-mon[65513]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:13.051 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:12 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/654591152' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:53:13.966 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:13 smithi115 ceph-mon[78672]: pgmap v312: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:14.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:13 smithi150 ceph-mon[65513]: pgmap v312: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:15.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:15 smithi115 ceph-mon[78672]: pgmap v313: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:16.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:15 smithi150 ceph-mon[65513]: pgmap v313: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:18.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:17 smithi115 ceph-mon[78672]: pgmap v314: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:18.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:17 smithi150 ceph-mon[65513]: pgmap v314: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:19.007 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:18 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:19.007 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:18 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:19.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:18 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:19.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:18 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:19 smithi150 ceph-mon[65513]: pgmap v315: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:20.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:53:20.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:19 smithi115 ceph-mon[78672]: pgmap v315: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:20.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:53:21.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:20 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:21.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:20 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:21.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:20 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:21.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:20 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:21.898 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:21 smithi150 ceph-mon[65513]: pgmap v316: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:22.185 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:21 smithi115 ceph-mon[78672]: pgmap v316: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:24.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:23 smithi115 ceph-mon[78672]: pgmap v317: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:24.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:23 smithi150 ceph-mon[65513]: pgmap v317: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:26.105 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: pgmap v318: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:26.105 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:26.105 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.106 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.107 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.107 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.107 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.107 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.107 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.107 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:25 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: pgmap v318: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:26.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:26.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:26.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:53:26.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:53:26.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:26.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:25 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:27.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:26 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:53:27.152 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:26 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T10:53:28.146 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:27 smithi115 ceph-mon[78672]: pgmap v319: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:28.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:27 smithi150 ceph-mon[65513]: pgmap v319: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:29.114 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:28 smithi115 ceph-mon[78672]: Upgrade: Updating alertmanager.smithi115 2024-03-22T10:53:29.115 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:28 smithi115 ceph-mon[78672]: Deploying daemon alertmanager.smithi115 on smithi115 2024-03-22T10:53:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:28 smithi150 ceph-mon[65513]: Upgrade: Updating alertmanager.smithi115 2024-03-22T10:53:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:28 smithi150 ceph-mon[65513]: Deploying daemon alertmanager.smithi115 on smithi115 2024-03-22T10:53:30.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:29 smithi115 ceph-mon[78672]: pgmap v320: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:30.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:29 smithi150 ceph-mon[65513]: pgmap v320: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:32.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:31 smithi150 ceph-mon[65513]: pgmap v321: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:32.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:31 smithi115 ceph-mon[78672]: pgmap v321: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:34.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:33 smithi115 ceph-mon[78672]: pgmap v322: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:34.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:33 smithi150 ceph-mon[65513]: pgmap v322: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:35.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:53:35.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:53:36.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:35 smithi150 ceph-mon[65513]: pgmap v323: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:36.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:36.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:36.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:36.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:35 smithi115 ceph-mon[78672]: pgmap v323: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:36.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:36.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:36.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:38.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:37 smithi150 ceph-mon[65513]: pgmap v324: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:38.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:37 smithi115 ceph-mon[78672]: pgmap v324: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:40.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:39 smithi150 ceph-mon[65513]: pgmap v325: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:40.260 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:39 smithi115 ceph-mon[78672]: pgmap v325: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:42.142 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:41 smithi115 ceph-mon[78672]: pgmap v326: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:42.142 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:42.142 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:41 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.149+0000 7fef96200700 1 -- 172.21.15.115:0/4007937854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90100150 msgr2=0x7fef90100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.149+0000 7fef96200700 1 --2- 172.21.15.115:0/4007937854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90100150 0x7fef90100560 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fef80009910 tx=0x7fef80009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.149+0000 7fef96200700 1 -- 172.21.15.115:0/4007937854 shutdown_connections 2024-03-22T10:53:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.149+0000 7fef96200700 1 --2- 172.21.15.115:0/4007937854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90101350 0x7fef901017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.149+0000 7fef96200700 1 --2- 172.21.15.115:0/4007937854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90100150 0x7fef90100560 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.149+0000 7fef96200700 1 -- 172.21.15.115:0/4007937854 >> 172.21.15.115:0/4007937854 conn(0x7fef900fb6e0 msgr2=0x7fef900fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.149+0000 7fef96200700 1 -- 172.21.15.115:0/4007937854 shutdown_connections 2024-03-22T10:53:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.149+0000 7fef96200700 1 -- 172.21.15.115:0/4007937854 wait complete. 2024-03-22T10:53:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.151+0000 7fef96200700 1 Processor -- start 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.151+0000 7fef96200700 1 -- start start 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.151+0000 7fef96200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90100150 0x7fef90074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.151+0000 7fef96200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90101350 0x7fef90073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.151+0000 7fef96200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90073680 con 0x7fef90101350 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.151+0000 7fef96200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900737c0 con 0x7fef90100150 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.152+0000 7fef8ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90101350 0x7fef90073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.152+0000 7fef8ea00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90101350 0x7fef90073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57892/0 (socket says 172.21.15.115:57892) 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.152+0000 7fef8ea00700 1 -- 172.21.15.115:0/3096835019 learned_addr learned my addr 172.21.15.115:0/3096835019 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.152+0000 7fef8f400700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90100150 0x7fef90074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.152+0000 7fef8ea00700 1 -- 172.21.15.115:0/3096835019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90100150 msgr2=0x7fef90074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.152+0000 7fef8ea00700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90100150 0x7fef90074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.152+0000 7fef8ea00700 1 -- 172.21.15.115:0/3096835019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800095f0 con 0x7fef90101350 2024-03-22T10:53:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.153+0000 7fef8ea00700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90101350 0x7fef90073140 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00c930 tx=0x7fef7c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.153+0000 7fef87e00700 1 -- 172.21.15.115:0/3096835019 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef7c007ad0 con 0x7fef90101350 2024-03-22T10:53:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.153+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef90073aa0 con 0x7fef90101350 2024-03-22T10:53:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.153+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9019da00 con 0x7fef90101350 2024-03-22T10:53:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.153+0000 7fef87e00700 1 -- 172.21.15.115:0/3096835019 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef7c007c30 con 0x7fef90101350 2024-03-22T10:53:42.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.153+0000 7fef87e00700 1 -- 172.21.15.115:0/3096835019 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef7c018630 con 0x7fef90101350 2024-03-22T10:53:42.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.156+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef70005320 con 0x7fef90101350 2024-03-22T10:53:42.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.157+0000 7fef87e00700 1 -- 172.21.15.115:0/3096835019 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fef7c0188b0 con 0x7fef90101350 2024-03-22T10:53:42.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.158+0000 7fef87e00700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fef7406dfa0 0x7fef74070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:42.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.158+0000 7fef87e00700 1 -- 172.21.15.115:0/3096835019 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fef7c08e0f0 con 0x7fef90101350 2024-03-22T10:53:42.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.158+0000 7fef8f400700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fef7406dfa0 0x7fef74070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:42.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.159+0000 7fef8f400700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fef7406dfa0 0x7fef74070450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fef80006010 tx=0x7fef8000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:42.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.167+0000 7fef87e00700 1 -- 172.21.15.115:0/3096835019 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef7c05bf10 con 0x7fef90101350 2024-03-22T10:53:42.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:41 smithi150 ceph-mon[65513]: pgmap v326: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:42.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:42.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:41 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:42.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.442+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef70000bf0 con 0x7fef7406dfa0 2024-03-22T10:53:42.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.444+0000 7fef87e00700 1 -- 172.21.15.115:0/3096835019 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+465 (secure 0 0 0) 0x7fef70000bf0 con 0x7fef7406dfa0 2024-03-22T10:53:42.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.447+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fef7406dfa0 msgr2=0x7fef74070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:42.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.447+0000 7fef96200700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fef7406dfa0 0x7fef74070450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fef80006010 tx=0x7fef8000b540 comp rx=0 tx=0).stop 2024-03-22T10:53:42.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.447+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90101350 msgr2=0x7fef90073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:42.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.447+0000 7fef96200700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90101350 0x7fef90073140 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00c930 tx=0x7fef7c00ccf0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.447+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 shutdown_connections 2024-03-22T10:53:42.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.447+0000 7fef96200700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fef7406dfa0 0x7fef74070450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.447+0000 7fef96200700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90100150 0x7fef90074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.447+0000 7fef96200700 1 --2- 172.21.15.115:0/3096835019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef90101350 0x7fef90073140 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.447+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 >> 172.21.15.115:0/3096835019 conn(0x7fef900fb6e0 msgr2=0x7fef90104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:42.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.448+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 shutdown_connections 2024-03-22T10:53:42.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.448+0000 7fef96200700 1 -- 172.21.15.115:0/3096835019 wait complete. 2024-03-22T10:53:42.456 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:53:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.624+0000 7fea49800700 1 -- 172.21.15.115:0/2087340689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44103580 msgr2=0x7fea44105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.624+0000 7fea49800700 1 --2- 172.21.15.115:0/2087340689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44103580 0x7fea44105960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009910 tx=0x7fea2c009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.624+0000 7fea49800700 1 -- 172.21.15.115:0/2087340689 shutdown_connections 2024-03-22T10:53:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.624+0000 7fea49800700 1 --2- 172.21.15.115:0/2087340689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44103580 0x7fea44105960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.624+0000 7fea49800700 1 --2- 172.21.15.115:0/2087340689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44100c60 0x7fea44103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.624+0000 7fea49800700 1 -- 172.21.15.115:0/2087340689 >> 172.21.15.115:0/2087340689 conn(0x7fea440fa600 msgr2=0x7fea440fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.624+0000 7fea49800700 1 -- 172.21.15.115:0/2087340689 shutdown_connections 2024-03-22T10:53:42.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.624+0000 7fea49800700 1 -- 172.21.15.115:0/2087340689 wait complete. 2024-03-22T10:53:42.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.625+0000 7fea49800700 1 Processor -- start 2024-03-22T10:53:42.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.625+0000 7fea49800700 1 -- start start 2024-03-22T10:53:42.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea49800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44100c60 0x7fea44191d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea49800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44103580 0x7fea44192280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea49800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4418e830 con 0x7fea44100c60 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea49800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4418e9a0 con 0x7fea44103580 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea42a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44100c60 0x7fea44191d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea42a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44100c60 0x7fea44191d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57920/0 (socket says 172.21.15.115:57920) 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea42a00700 1 -- 172.21.15.115:0/731472335 learned_addr learned my addr 172.21.15.115:0/731472335 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea42000700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44103580 0x7fea44192280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea42a00700 1 -- 172.21.15.115:0/731472335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44103580 msgr2=0x7fea44192280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.626+0000 7fea42a00700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44103580 0x7fea44192280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.627+0000 7fea42a00700 1 -- 172.21.15.115:0/731472335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0095f0 con 0x7fea44100c60 2024-03-22T10:53:42.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.627+0000 7fea42a00700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44100c60 0x7fea44191d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c8a0 tx=0x7fea3400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:42.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.627+0000 7fea3b400700 1 -- 172.21.15.115:0/731472335 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea3400cea0 con 0x7fea44100c60 2024-03-22T10:53:42.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.627+0000 7fea3b400700 1 -- 172.21.15.115:0/731472335 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fea34004830 con 0x7fea44100c60 2024-03-22T10:53:42.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.627+0000 7fea3b400700 1 -- 172.21.15.115:0/731472335 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea340055b0 con 0x7fea44100c60 2024-03-22T10:53:42.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.627+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4418ec80 con 0x7fea44100c60 2024-03-22T10:53:42.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.627+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4418f1d0 con 0x7fea44100c60 2024-03-22T10:53:42.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.629+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea440fc1c0 con 0x7fea44100c60 2024-03-22T10:53:42.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.630+0000 7fea3b400700 1 -- 172.21.15.115:0/731472335 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fea34007ad0 con 0x7fea44100c60 2024-03-22T10:53:42.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.631+0000 7fea3b400700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fea2806dfa0 0x7fea28070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:42.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.631+0000 7fea3b400700 1 -- 172.21.15.115:0/731472335 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fea3408dac0 con 0x7fea44100c60 2024-03-22T10:53:42.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.632+0000 7fea42000700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fea2806dfa0 0x7fea28070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:42.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.633+0000 7fea42000700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fea2806dfa0 0x7fea28070450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006010 tx=0x7fea2c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:42.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.636+0000 7fea3b400700 1 -- 172.21.15.115:0/731472335 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fea3405a9c0 con 0x7fea44100c60 2024-03-22T10:53:42.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.934+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea44061190 con 0x7fea2806dfa0 2024-03-22T10:53:42.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.936+0000 7fea3b400700 1 -- 172.21.15.115:0/731472335 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+465 (secure 0 0 0) 0x7fea44061190 con 0x7fea2806dfa0 2024-03-22T10:53:42.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.938+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fea2806dfa0 msgr2=0x7fea28070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:42.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.938+0000 7fea49800700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fea2806dfa0 0x7fea28070450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006010 tx=0x7fea2c00b540 comp rx=0 tx=0).stop 2024-03-22T10:53:42.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.938+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44100c60 msgr2=0x7fea44191d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:42.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.938+0000 7fea49800700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44100c60 0x7fea44191d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c8a0 tx=0x7fea3400cc60 comp rx=0 tx=0).stop 2024-03-22T10:53:42.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.939+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 shutdown_connections 2024-03-22T10:53:42.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.939+0000 7fea49800700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fea2806dfa0 0x7fea28070450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.939+0000 7fea49800700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea44100c60 0x7fea44191d40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.939+0000 7fea49800700 1 --2- 172.21.15.115:0/731472335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44103580 0x7fea44192280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:42.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.939+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 >> 172.21.15.115:0/731472335 conn(0x7fea440fa600 msgr2=0x7fea441041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:42.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.939+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 shutdown_connections 2024-03-22T10:53:42.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:42.939+0000 7fea49800700 1 -- 172.21.15.115:0/731472335 wait complete. 2024-03-22T10:53:43.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.094+0000 7f2978a00700 1 -- 172.21.15.115:0/1372424889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974074d80 msgr2=0x7f29740731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:43.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.094+0000 7f2978a00700 1 --2- 172.21.15.115:0/1372424889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974074d80 0x7f29740731e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2964009910 tx=0x7f2964009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.095+0000 7f2978a00700 1 -- 172.21.15.115:0/1372424889 shutdown_connections 2024-03-22T10:53:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.095+0000 7f2978a00700 1 --2- 172.21.15.115:0/1372424889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29740737b0 0x7f2974073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.095+0000 7f2978a00700 1 --2- 172.21.15.115:0/1372424889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974074d80 0x7f29740731e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.095+0000 7f2978a00700 1 -- 172.21.15.115:0/1372424889 >> 172.21.15.115:0/1372424889 conn(0x7f29740fb6e0 msgr2=0x7f29740fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.095+0000 7f2978a00700 1 -- 172.21.15.115:0/1372424889 shutdown_connections 2024-03-22T10:53:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.095+0000 7f2978a00700 1 -- 172.21.15.115:0/1372424889 wait complete. 2024-03-22T10:53:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.096+0000 7f2978a00700 1 Processor -- start 2024-03-22T10:53:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.096+0000 7f2978a00700 1 -- start start 2024-03-22T10:53:43.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2978a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29740737b0 0x7f297419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:43.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2978a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974074d80 0x7f297419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:43.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2978a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297419ca90 con 0x7f29740737b0 2024-03-22T10:53:43.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2978a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297419cbd0 con 0x7f2974074d80 2024-03-22T10:53:43.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2971e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29740737b0 0x7f297419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:43.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2971e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29740737b0 0x7f297419bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57936/0 (socket says 172.21.15.115:57936) 2024-03-22T10:53:43.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2971e00700 1 -- 172.21.15.115:0/4144894845 learned_addr learned my addr 172.21.15.115:0/4144894845 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:43.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2971e00700 1 -- 172.21.15.115:0/4144894845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974074d80 msgr2=0x7f297419c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:43.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2971400700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974074d80 0x7f297419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:43.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2971e00700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974074d80 0x7f297419c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.097+0000 7f2971e00700 1 -- 172.21.15.115:0/4144894845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640095f0 con 0x7f29740737b0 2024-03-22T10:53:43.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.098+0000 7f2971400700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974074d80 0x7f297419c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:53:43.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.098+0000 7f2971e00700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29740737b0 0x7f297419bf30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2964005b40 tx=0x7f296400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:43.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.098+0000 7f296aa00700 1 -- 172.21.15.115:0/4144894845 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296401d070 con 0x7f29740737b0 2024-03-22T10:53:43.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.098+0000 7f296aa00700 1 -- 172.21.15.115:0/4144894845 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2964022470 con 0x7f29740737b0 2024-03-22T10:53:43.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.098+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29741a1620 con 0x7f29740737b0 2024-03-22T10:53:43.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.099+0000 7f296aa00700 1 -- 172.21.15.115:0/4144894845 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296400f690 con 0x7f29740737b0 2024-03-22T10:53:43.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.099+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29741a1b10 con 0x7f29740737b0 2024-03-22T10:53:43.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.100+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297410f280 con 0x7f29740737b0 2024-03-22T10:53:43.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.102+0000 7f296aa00700 1 -- 172.21.15.115:0/4144894845 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f29640225e0 con 0x7f29740737b0 2024-03-22T10:53:43.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.103+0000 7f296aa00700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806dfa0 0x7f2958070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:43.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.104+0000 7f296aa00700 1 -- 172.21.15.115:0/4144894845 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f296408f510 con 0x7f29740737b0 2024-03-22T10:53:43.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.104+0000 7f2971400700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806dfa0 0x7f2958070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:43.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.105+0000 7f2971400700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806dfa0 0x7f2958070450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2960005fd0 tx=0x7f2960005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:43.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.112+0000 7f296aa00700 1 -- 172.21.15.115:0/4144894845 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f296405c4c0 con 0x7f29740737b0 2024-03-22T10:53:43.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.393+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2974103200 con 0x7f295806dfa0 2024-03-22T10:53:43.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.406+0000 7f296aa00700 1 -- 172.21.15.115:0/4144894845 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2974103200 con 0x7f295806dfa0 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (9s) 1s ago 22m 14.5M - 0.25.0 c8568f914cd2 ff3d0ae1fb82 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (91s) 1s ago 22m 8102k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e11b0bdaf069 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (83s) 52s ago 21m 7662k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 ef9665eef7b1 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (8m) 1s ago 22m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (7m) 52s ago 20m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (19m) 1s ago 21m 74.4M - 9.4.7 954c08fa6188 9f827f5b8420 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (3m) 1s ago 15m 67.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 44a0769ebe8e 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (2m) 1s ago 15m 17.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51d700f6eb18 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (112s) 52s ago 14m 12.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 82b6b216f754 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (2m) 52s ago 15m 12.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9605445d5b2b 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (11m) 1s ago 23m 516M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (10m) 52s ago 20m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:53:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (9m) 1s ago 24m 60.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (9m) 52s ago 20m 45.7M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (63s) 1s ago 22m 7377k - 1.5.0 0da6a335fe13 a98cb20e74e1 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (57s) 52s ago 20m 2935k - 1.5.0 0da6a335fe13 6c3ba9d80db0 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (7m) 1s ago 19m 98.0M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (6m) 1s ago 19m 96.7M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (6m) 1s ago 18m 83.7M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (5m) 52s ago 18m 103M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (5m) 52s ago 17m 91.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (4m) 52s ago 16m 90.9M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:53:43.400 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (32s) 1s ago 21m 62.5M - 2.43.0 a07b618ecd1d 8561aedaa7b5 2024-03-22T10:53:43.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.409+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806dfa0 msgr2=0x7f2958070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:43.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.409+0000 7f2978a00700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806dfa0 0x7f2958070450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2960005fd0 tx=0x7f2960005dc0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.409+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29740737b0 msgr2=0x7f297419bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:43.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.409+0000 7f2978a00700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29740737b0 0x7f297419bf30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2964005b40 tx=0x7f296400be20 comp rx=0 tx=0).stop 2024-03-22T10:53:43.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.409+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 shutdown_connections 2024-03-22T10:53:43.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.409+0000 7f2978a00700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806dfa0 0x7f2958070450 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.409+0000 7f2978a00700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29740737b0 0x7f297419bf30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.409+0000 7f2978a00700 1 --2- 172.21.15.115:0/4144894845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974074d80 0x7f297419c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.409+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 >> 172.21.15.115:0/4144894845 conn(0x7f29740fb6e0 msgr2=0x7f2974101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:43.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.410+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 shutdown_connections 2024-03-22T10:53:43.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.410+0000 7f2978a00700 1 -- 172.21.15.115:0/4144894845 wait complete. 2024-03-22T10:53:43.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.617+0000 7fba02200700 1 -- 172.21.15.115:0/1459199837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 msgr2=0x7fb9fc0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:43.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.617+0000 7fba02200700 1 --2- 172.21.15.115:0/1459199837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 0x7fb9fc0ff790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009910 tx=0x7fb9e8009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:43.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.618+0000 7fba02200700 1 -- 172.21.15.115:0/1459199837 shutdown_connections 2024-03-22T10:53:43.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.618+0000 7fba02200700 1 --2- 172.21.15.115:0/1459199837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 0x7fb9fc0ff790 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.618+0000 7fba02200700 1 --2- 172.21.15.115:0/1459199837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9fc0fe9d0 0x7fb9fc0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.618+0000 7fba02200700 1 -- 172.21.15.115:0/1459199837 >> 172.21.15.115:0/1459199837 conn(0x7fb9fc0fa600 msgr2=0x7fb9fc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:43.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.618+0000 7fba02200700 1 -- 172.21.15.115:0/1459199837 shutdown_connections 2024-03-22T10:53:43.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.618+0000 7fba02200700 1 -- 172.21.15.115:0/1459199837 wait complete. 2024-03-22T10:53:43.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.619+0000 7fba02200700 1 Processor -- start 2024-03-22T10:53:43.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.619+0000 7fba02200700 1 -- start start 2024-03-22T10:53:43.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.620+0000 7fba02200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9fc0fe9d0 0x7fb9fc071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:43.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.620+0000 7fba02200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 0x7fb9fc072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:43.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.620+0000 7fba02200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc072760 con 0x7fb9fc0ff320 2024-03-22T10:53:43.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.620+0000 7fba02200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0728d0 con 0x7fb9fc0fe9d0 2024-03-22T10:53:43.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.620+0000 7fb9faa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 0x7fb9fc072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:43.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.621+0000 7fb9faa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 0x7fb9fc072220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57950/0 (socket says 172.21.15.115:57950) 2024-03-22T10:53:43.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.621+0000 7fb9faa00700 1 -- 172.21.15.115:0/965708451 learned_addr learned my addr 172.21.15.115:0/965708451 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:43.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.621+0000 7fb9fb400700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9fc0fe9d0 0x7fb9fc071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.621+0000 7fb9faa00700 1 -- 172.21.15.115:0/965708451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9fc0fe9d0 msgr2=0x7fb9fc071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.621+0000 7fb9faa00700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9fc0fe9d0 0x7fb9fc071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.621+0000 7fb9faa00700 1 -- 172.21.15.115:0/965708451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e80095f0 con 0x7fb9fc0ff320 2024-03-22T10:53:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.622+0000 7fb9faa00700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 0x7fb9fc072220 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800b5c0 tx=0x7fb9e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.622+0000 7fb9f3e00700 1 -- 172.21.15.115:0/965708451 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e801d070 con 0x7fb9fc0ff320 2024-03-22T10:53:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.622+0000 7fb9f3e00700 1 -- 172.21.15.115:0/965708451 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb9e800f460 con 0x7fb9fc0ff320 2024-03-22T10:53:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.622+0000 7fb9f3e00700 1 -- 172.21.15.115:0/965708451 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e8021640 con 0x7fb9fc0ff320 2024-03-22T10:53:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.622+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc1a9e40 con 0x7fb9fc0ff320 2024-03-22T10:53:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.623+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc1aa300 con 0x7fb9fc0ff320 2024-03-22T10:53:43.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.624+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc10ae80 con 0x7fb9fc0ff320 2024-03-22T10:53:43.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.625+0000 7fb9f3e00700 1 -- 172.21.15.115:0/965708451 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb9e800fa60 con 0x7fb9fc0ff320 2024-03-22T10:53:43.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.626+0000 7fb9f3e00700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9e006dfa0 0x7fb9e0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:43.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.627+0000 7fb9f3e00700 1 -- 172.21.15.115:0/965708451 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb9e808eff0 con 0x7fb9fc0ff320 2024-03-22T10:53:43.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.627+0000 7fb9fb400700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9e006dfa0 0x7fb9e0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:43.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.628+0000 7fb9fb400700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9e006dfa0 0x7fb9e0070450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec005950 tx=0x7fb9ec00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:43.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:43.632+0000 7fb9f3e00700 1 -- 172.21.15.115:0/965708451 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb9e805bf30 con 0x7fb9fc0ff320 2024-03-22T10:53:44.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.051+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb9fc10b040 con 0x7fb9fc0ff320 2024-03-22T10:53:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.051+0000 7fb9f3e00700 1 -- 172.21.15.115:0/965708451 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb9e805bac0 con 0x7fb9fc0ff320 2024-03-22T10:53:44.044 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:53:44.044 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:53:44.044 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:53:44.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:53:44.044 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:53:44.044 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:53:44.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:53:44.045 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:53:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.054+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9e006dfa0 msgr2=0x7fb9e0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.054+0000 7fba02200700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9e006dfa0 0x7fb9e0070450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec005950 tx=0x7fb9ec00b410 comp rx=0 tx=0).stop 2024-03-22T10:53:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.054+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 msgr2=0x7fb9fc072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.054+0000 7fba02200700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 0x7fb9fc072220 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800b5c0 tx=0x7fb9e800bd70 comp rx=0 tx=0).stop 2024-03-22T10:53:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.054+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 shutdown_connections 2024-03-22T10:53:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.054+0000 7fba02200700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9e006dfa0 0x7fb9e0070450 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.054+0000 7fba02200700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9fc0fe9d0 0x7fb9fc071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.054+0000 7fba02200700 1 --2- 172.21.15.115:0/965708451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9fc0ff320 0x7fb9fc072220 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.054+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 >> 172.21.15.115:0/965708451 conn(0x7fb9fc0fa600 msgr2=0x7fb9fc107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:44.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.055+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 shutdown_connections 2024-03-22T10:53:44.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.055+0000 7fba02200700 1 -- 172.21.15.115:0/965708451 wait complete. 2024-03-22T10:53:44.202 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:43 smithi150 ceph-mon[65513]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:44.202 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:43 smithi150 ceph-mon[65513]: pgmap v327: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:44.202 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:43 smithi150 ceph-mon[65513]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:44.202 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:43 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:44.202 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:43 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.225+0000 7f20a6000700 1 -- 172.21.15.115:0/3204792666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a0100a80 msgr2=0x7f20a0102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.225+0000 7f20a6000700 1 --2- 172.21.15.115:0/3204792666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a0100a80 0x7f20a0102e60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2094009910 tx=0x7f2094009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:44.218 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:43 smithi115 ceph-mon[78672]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:44.218 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:43 smithi115 ceph-mon[78672]: pgmap v327: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:44.218 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:43 smithi115 ceph-mon[78672]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:44.218 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:43 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:44.218 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:43 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.225+0000 7f20a6000700 1 -- 172.21.15.115:0/3204792666 shutdown_connections 2024-03-22T10:53:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.225+0000 7f20a6000700 1 --2- 172.21.15.115:0/3204792666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a01033a0 0x7f20a0105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.225+0000 7f20a6000700 1 --2- 172.21.15.115:0/3204792666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a0100a80 0x7f20a0102e60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.225+0000 7f20a6000700 1 -- 172.21.15.115:0/3204792666 >> 172.21.15.115:0/3204792666 conn(0x7f20a00fa440 msgr2=0x7f20a00fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.225+0000 7f20a6000700 1 -- 172.21.15.115:0/3204792666 shutdown_connections 2024-03-22T10:53:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.225+0000 7f20a6000700 1 -- 172.21.15.115:0/3204792666 wait complete. 2024-03-22T10:53:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.227+0000 7f20a6000700 1 Processor -- start 2024-03-22T10:53:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.227+0000 7f20a6000700 1 -- start start 2024-03-22T10:53:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.227+0000 7f20a6000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a0100a80 0x7f20a0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.228+0000 7f20a6000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a01033a0 0x7f20a0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.228+0000 7f20a6000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0198570 con 0x7f20a01033a0 2024-03-22T10:53:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.228+0000 7f20a6000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01986b0 con 0x7f20a0100a80 2024-03-22T10:53:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.229+0000 7f209fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a01033a0 0x7f20a0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.229+0000 7f209fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a01033a0 0x7f20a0197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57962/0 (socket says 172.21.15.115:57962) 2024-03-22T10:53:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.229+0000 7f209fe00700 1 -- 172.21.15.115:0/3590533078 learned_addr learned my addr 172.21.15.115:0/3590533078 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.229+0000 7f20a4c00700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a0100a80 0x7f20a0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.229+0000 7f209fe00700 1 -- 172.21.15.115:0/3590533078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a0100a80 msgr2=0x7f20a0197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.229+0000 7f209fe00700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a0100a80 0x7f20a0197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.229+0000 7f209fe00700 1 -- 172.21.15.115:0/3590533078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940095f0 con 0x7f20a01033a0 2024-03-22T10:53:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.230+0000 7f209fe00700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a01033a0 0x7f20a0197f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f208c00b4b0 tx=0x7f208c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.231+0000 7f209d600700 1 -- 172.21.15.115:0/3590533078 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f208c011800 con 0x7f20a01033a0 2024-03-22T10:53:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.231+0000 7f209d600700 1 -- 172.21.15.115:0/3590533078 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f208c011e10 con 0x7f20a01033a0 2024-03-22T10:53:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.231+0000 7f209d600700 1 -- 172.21.15.115:0/3590533078 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f208c00f2e0 con 0x7f20a01033a0 2024-03-22T10:53:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.231+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a019d160 con 0x7f20a01033a0 2024-03-22T10:53:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.231+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a019d730 con 0x7f20a01033a0 2024-03-22T10:53:44.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.235+0000 7f209d600700 1 -- 172.21.15.115:0/3590533078 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f208c010380 con 0x7f20a01033a0 2024-03-22T10:53:44.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.236+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a00fc000 con 0x7f20a01033a0 2024-03-22T10:53:44.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.237+0000 7f209d600700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208806e0c0 0x7f2088070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:44.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.237+0000 7f209d600700 1 -- 172.21.15.115:0/3590533078 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f208c08d920 con 0x7f20a01033a0 2024-03-22T10:53:44.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.237+0000 7f20a4c00700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208806e0c0 0x7f2088070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:44.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.239+0000 7f20a4c00700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208806e0c0 0x7f2088070570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2094006010 tx=0x7f209400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:44.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.243+0000 7f209d600700 1 -- 172.21.15.115:0/3590533078 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f208c05a8a0 con 0x7f20a01033a0 2024-03-22T10:53:44.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.617+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f20a002cc30 con 0x7f20a01033a0 2024-03-22T10:53:44.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.618+0000 7f209d600700 1 -- 172.21.15.115:0/3590533078 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1932 (secure 0 0 0) 0x7f208c0157a0 con 0x7f20a01033a0 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:e40 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 40 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:52:05.899739+0000 2024-03-22T10:53:44.611 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 83 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=34476,1=34508} 2024-03-22T10:53:44.612 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 34476 members: 34476,34508 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:34476} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{1:34508} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:53:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:53:44.614 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:53:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:53:44.614 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6824/1804848703,v1:172.21.15.150:6825/1804848703] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:53:44.614 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6826/2053213512,v1:172.21.15.150:6827/2053213512] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:53:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.621+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208806e0c0 msgr2=0x7f2088070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.621+0000 7f20a6000700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208806e0c0 0x7f2088070570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2094006010 tx=0x7f209400b540 comp rx=0 tx=0).stop 2024-03-22T10:53:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.621+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a01033a0 msgr2=0x7f20a0197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.621+0000 7f20a6000700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a01033a0 0x7f20a0197f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f208c00b4b0 tx=0x7f208c00b870 comp rx=0 tx=0).stop 2024-03-22T10:53:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.621+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 shutdown_connections 2024-03-22T10:53:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.621+0000 7f20a6000700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f208806e0c0 0x7f2088070570 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.621+0000 7f20a6000700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a0100a80 0x7f20a0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.621+0000 7f20a6000700 1 --2- 172.21.15.115:0/3590533078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20a01033a0 0x7f20a0197f50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.621+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 >> 172.21.15.115:0/3590533078 conn(0x7f20a00fa440 msgr2=0x7f20a00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.622+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 shutdown_connections 2024-03-22T10:53:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.622+0000 7f20a6000700 1 -- 172.21.15.115:0/3590533078 wait complete. 2024-03-22T10:53:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2024-03-22T10:53:44.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.802+0000 7fcff3400700 1 -- 172.21.15.115:0/1252691644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec100be0 msgr2=0x7fcfec102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.802+0000 7fcff3400700 1 --2- 172.21.15.115:0/1252691644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec100be0 0x7fcfec102fc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009910 tx=0x7fcfe0009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.802+0000 7fcff3400700 1 -- 172.21.15.115:0/1252691644 shutdown_connections 2024-03-22T10:53:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.802+0000 7fcff3400700 1 --2- 172.21.15.115:0/1252691644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfec103500 0x7fcfec1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.802+0000 7fcff3400700 1 --2- 172.21.15.115:0/1252691644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec100be0 0x7fcfec102fc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.802+0000 7fcff3400700 1 -- 172.21.15.115:0/1252691644 >> 172.21.15.115:0/1252691644 conn(0x7fcfec0fa5e0 msgr2=0x7fcfec0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.803+0000 7fcff3400700 1 -- 172.21.15.115:0/1252691644 shutdown_connections 2024-03-22T10:53:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.803+0000 7fcff3400700 1 -- 172.21.15.115:0/1252691644 wait complete. 2024-03-22T10:53:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.804+0000 7fcff3400700 1 Processor -- start 2024-03-22T10:53:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.805+0000 7fcff3400700 1 -- start start 2024-03-22T10:53:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.805+0000 7fcff3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfec100be0 0x7fcfec197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.805+0000 7fcff3400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec103500 0x7fcfec198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.805+0000 7fcff3400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec198660 con 0x7fcfec103500 2024-03-22T10:53:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.805+0000 7fcff3400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec1987a0 con 0x7fcfec100be0 2024-03-22T10:53:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.806+0000 7fcfebe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec103500 0x7fcfec198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.806+0000 7fcfebe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec103500 0x7fcfec198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57986/0 (socket says 172.21.15.115:57986) 2024-03-22T10:53:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.806+0000 7fcfebe00700 1 -- 172.21.15.115:0/3692797081 learned_addr learned my addr 172.21.15.115:0/3692797081 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.806+0000 7fcff0a00700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfec100be0 0x7fcfec197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.806+0000 7fcfebe00700 1 -- 172.21.15.115:0/3692797081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfec100be0 msgr2=0x7fcfec197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.806+0000 7fcfebe00700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfec100be0 0x7fcfec197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.806+0000 7fcfebe00700 1 -- 172.21.15.115:0/3692797081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe00095f0 con 0x7fcfec103500 2024-03-22T10:53:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.807+0000 7fcfebe00700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec103500 0x7fcfec198040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800c8a0 tx=0x7fcfd800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.807+0000 7fcfe9600700 1 -- 172.21.15.115:0/3692797081 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfd800cea0 con 0x7fcfec103500 2024-03-22T10:53:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.807+0000 7fcfe9600700 1 -- 172.21.15.115:0/3692797081 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcfd8004830 con 0x7fcfec103500 2024-03-22T10:53:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.807+0000 7fcfe9600700 1 -- 172.21.15.115:0/3692797081 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfd80055b0 con 0x7fcfec103500 2024-03-22T10:53:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.807+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec19d250 con 0x7fcfec103500 2024-03-22T10:53:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.808+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec19d7a0 con 0x7fcfec103500 2024-03-22T10:53:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.809+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec0fc1c0 con 0x7fcfec103500 2024-03-22T10:53:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.810+0000 7fcfe9600700 1 -- 172.21.15.115:0/3692797081 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fcfd8007ad0 con 0x7fcfec103500 2024-03-22T10:53:44.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.812+0000 7fcfe9600700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcfd406dfa0 0x7fcfd4070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:44.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.812+0000 7fcfe9600700 1 -- 172.21.15.115:0/3692797081 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcfd808daf0 con 0x7fcfec103500 2024-03-22T10:53:44.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.812+0000 7fcff0a00700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcfd406dfa0 0x7fcfd4070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:44.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.813+0000 7fcff0a00700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcfd406dfa0 0x7fcfd4070450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0006010 tx=0x7fcfe000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:44.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:44.816+0000 7fcfe9600700 1 -- 172.21.15.115:0/3692797081 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcfd805a9f0 con 0x7fcfec103500 2024-03-22T10:53:45.046 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:44 smithi115 ceph-mon[78672]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:45.046 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:44 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/965708451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:45.046 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:44 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3590533078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:53:45.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.201+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfec061190 con 0x7fcfd406dfa0 2024-03-22T10:53:45.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.203+0000 7fcfe9600700 1 -- 172.21.15.115:0/3692797081 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+465 (secure 0 0 0) 0x7fcfec061190 con 0x7fcfd406dfa0 2024-03-22T10:53:45.196 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:53:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:53:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:53:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:53:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:53:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:53:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:53:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-03-22T10:53:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "osd", 2024-03-22T10:53:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "mds", 2024-03-22T10:53:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph-exporter" 2024-03-22T10:53:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:53:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "18/23 daemons upgraded", 2024-03-22T10:53:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading alertmanager daemons", 2024-03-22T10:53:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:53:45.197 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:53:45.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.206+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcfd406dfa0 msgr2=0x7fcfd4070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:45.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.206+0000 7fcff3400700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcfd406dfa0 0x7fcfd4070450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0006010 tx=0x7fcfe000b540 comp rx=0 tx=0).stop 2024-03-22T10:53:45.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.206+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec103500 msgr2=0x7fcfec198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:45.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.206+0000 7fcff3400700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec103500 0x7fcfec198040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800c8a0 tx=0x7fcfd800cc60 comp rx=0 tx=0).stop 2024-03-22T10:53:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.207+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 shutdown_connections 2024-03-22T10:53:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.207+0000 7fcff3400700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcfd406dfa0 0x7fcfd4070450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.207+0000 7fcff3400700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfec100be0 0x7fcfec197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.207+0000 7fcff3400700 1 --2- 172.21.15.115:0/3692797081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfec103500 0x7fcfec198040 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.207+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 >> 172.21.15.115:0/3692797081 conn(0x7fcfec0fa5e0 msgr2=0x7fcfec0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.207+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 shutdown_connections 2024-03-22T10:53:45.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.207+0000 7fcff3400700 1 -- 172.21.15.115:0/3692797081 wait complete. 2024-03-22T10:53:45.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.363+0000 7f39f9600700 1 -- 172.21.15.115:0/856620430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 msgr2=0x7f39f4103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.363+0000 7f39f9600700 1 --2- 172.21.15.115:0/856620430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 0x7f39f4103020 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009910 tx=0x7f39e4009c20 comp rx=0 tx=0).stop 2024-03-22T10:53:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.363+0000 7f39f9600700 1 -- 172.21.15.115:0/856620430 shutdown_connections 2024-03-22T10:53:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.363+0000 7f39f9600700 1 --2- 172.21.15.115:0/856620430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f4103560 0x7f39f4105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.363+0000 7f39f9600700 1 --2- 172.21.15.115:0/856620430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 0x7f39f4103020 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.363+0000 7f39f9600700 1 -- 172.21.15.115:0/856620430 >> 172.21.15.115:0/856620430 conn(0x7f39f40fa5e0 msgr2=0x7f39f40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.363+0000 7f39f9600700 1 -- 172.21.15.115:0/856620430 shutdown_connections 2024-03-22T10:53:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.363+0000 7f39f9600700 1 -- 172.21.15.115:0/856620430 wait complete. 2024-03-22T10:53:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.364+0000 7f39f9600700 1 Processor -- start 2024-03-22T10:53:45.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.364+0000 7f39f9600700 1 -- start start 2024-03-22T10:53:45.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.365+0000 7f39f9600700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 0x7f39f41005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:45.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.365+0000 7f39f9600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f4103560 0x7f39f40febf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:45.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.365+0000 7f39f9600700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f40ff130 con 0x7f39f4100c40 2024-03-22T10:53:45.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.365+0000 7f39f9600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f40ff270 con 0x7f39f4103560 2024-03-22T10:53:45.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.365+0000 7f39f2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 0x7f39f41005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:45.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.365+0000 7f39f2a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 0x7f39f41005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58016/0 (socket says 172.21.15.115:58016) 2024-03-22T10:53:45.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.365+0000 7f39f2a00700 1 -- 172.21.15.115:0/1020840658 learned_addr learned my addr 172.21.15.115:0/1020840658 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:53:45.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.365+0000 7f39f2000700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f4103560 0x7f39f40febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.365+0000 7f39f2a00700 1 -- 172.21.15.115:0/1020840658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f4103560 msgr2=0x7f39f40febf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.366+0000 7f39f2a00700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f4103560 0x7f39f40febf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.366+0000 7f39f2a00700 1 -- 172.21.15.115:0/1020840658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e40095f0 con 0x7f39f4100c40 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.366+0000 7f39f2000700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f4103560 0x7f39f40febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.366+0000 7f39f2a00700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 0x7f39f41005a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b5c0 tx=0x7f39e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.366+0000 7f39eb400700 1 -- 172.21.15.115:0/1020840658 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e401d070 con 0x7f39f4100c40 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.366+0000 7f39eb400700 1 -- 172.21.15.115:0/1020840658 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f39e4022470 con 0x7f39f4100c40 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.366+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f40ff4f0 con 0x7f39f4100c40 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.366+0000 7f39eb400700 1 -- 172.21.15.115:0/1020840658 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e400f690 con 0x7f39f4100c40 2024-03-22T10:53:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.367+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f40ff9e0 con 0x7f39f4100c40 2024-03-22T10:53:45.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.368+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f418fbf0 con 0x7f39f4100c40 2024-03-22T10:53:45.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.369+0000 7f39eb400700 1 -- 172.21.15.115:0/1020840658 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f39e4004b70 con 0x7f39f4100c40 2024-03-22T10:53:45.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.370+0000 7f39eb400700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f39d806dfa0 0x7f39d8070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:53:45.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.370+0000 7f39eb400700 1 -- 172.21.15.115:0/1020840658 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f39e408f570 con 0x7f39f4100c40 2024-03-22T10:53:45.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.371+0000 7f39f2000700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f39d806dfa0 0x7f39d8070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:53:45.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.372+0000 7f39f2000700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f39d806dfa0 0x7f39d8070450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39e0007900 tx=0x7f39e0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:53:45.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.377+0000 7f39eb400700 1 -- 172.21.15.115:0/1020840658 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f39e405c520 con 0x7f39f4100c40 2024-03-22T10:53:45.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:44 smithi150 ceph-mon[65513]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:45.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:44 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/965708451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:45.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:44 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3590533078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:53:45.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.687+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f39f402ce90 con 0x7f39f4100c40 2024-03-22T10:53:45.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.687+0000 7f39eb400700 1 -- 172.21.15.115:0/1020840658 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f39e405c0b0 con 0x7f39f4100c40 2024-03-22T10:53:45.680 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:53:45.680 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:53:45.681 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:53:45.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f39d806dfa0 msgr2=0x7f39d8070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:45.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f39d806dfa0 0x7f39d8070450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39e0007900 tx=0x7f39e0008040 comp rx=0 tx=0).stop 2024-03-22T10:53:45.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 msgr2=0x7f39f41005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:53:45.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 0x7f39f41005a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b5c0 tx=0x7f39e400be20 comp rx=0 tx=0).stop 2024-03-22T10:53:45.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 shutdown_connections 2024-03-22T10:53:45.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f39d806dfa0 0x7f39d8070450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:45.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f4100c40 0x7f39f41005a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:45.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 --2- 172.21.15.115:0/1020840658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f4103560 0x7f39f40febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:53:45.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 >> 172.21.15.115:0/1020840658 conn(0x7f39f40fa5e0 msgr2=0x7f39f40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:53:45.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.690+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 shutdown_connections 2024-03-22T10:53:45.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:53:45.691+0000 7f39f9600700 1 -- 172.21.15.115:0/1020840658 wait complete. 2024-03-22T10:53:46.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:45 smithi115 ceph-mon[78672]: pgmap v328: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:46.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:45 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1020840658' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:53:46.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:45 smithi150 ceph-mon[65513]: pgmap v328: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:46.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:45 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1020840658' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:53:47.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:46 smithi115 ceph-mon[78672]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:47.317 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:46 smithi150 ceph-mon[65513]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:53:48.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:47 smithi115 ceph-mon[78672]: pgmap v329: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:48.398 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:47 smithi150 ceph-mon[65513]: pgmap v329: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: pgmap v330: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:53:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: pgmap v330: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:53:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:53:50.946 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:50 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T10:53:51.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:50 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T10:53:52.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:51 smithi115 ceph-mon[78672]: pgmap v331: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:52.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:51 smithi150 ceph-mon[65513]: pgmap v331: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:52 smithi115 ceph-mon[78672]: Upgrade: Updating grafana.smithi115 2024-03-22T10:53:53.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:52 smithi115 ceph-mon[78672]: Deploying daemon grafana.smithi115 on smithi115 2024-03-22T10:53:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:52 smithi150 ceph-mon[65513]: Upgrade: Updating grafana.smithi115 2024-03-22T10:53:53.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:52 smithi150 ceph-mon[65513]: Deploying daemon grafana.smithi115 on smithi115 2024-03-22T10:53:54.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:53 smithi115 ceph-mon[78672]: pgmap v332: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:54.148 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:53 smithi150 ceph-mon[65513]: pgmap v332: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:56.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:55 smithi115 ceph-mon[78672]: pgmap v333: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:56.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:55 smithi150 ceph-mon[65513]: pgmap v333: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:57.899 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:57 smithi150 ceph-mon[65513]: pgmap v334: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:53:58.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:57 smithi115 ceph-mon[78672]: pgmap v334: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:00.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:53:59 smithi150 ceph-mon[65513]: pgmap v335: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:00.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:53:59 smithi115 ceph-mon[78672]: pgmap v335: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:02.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:01 smithi150 ceph-mon[65513]: pgmap v336: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:02.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:01 smithi115 ceph-mon[78672]: pgmap v336: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:04.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:03 smithi150 ceph-mon[65513]: pgmap v337: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:04.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:03 smithi115 ceph-mon[78672]: pgmap v337: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:05.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:54:05.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:54:06.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:05 smithi150 ceph-mon[65513]: pgmap v338: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:06.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:05 smithi115 ceph-mon[78672]: pgmap v338: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:08.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:07 smithi115 ceph-mon[78672]: pgmap v339: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:08.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:07 smithi150 ceph-mon[65513]: pgmap v339: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:10.056 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:09 smithi115 ceph-mon[78672]: pgmap v340: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:10.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:09 smithi150 ceph-mon[65513]: pgmap v340: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:12.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:11 smithi150 ceph-mon[65513]: pgmap v341: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:12.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:11 smithi115 ceph-mon[78672]: pgmap v341: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:14.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:13 smithi150 ceph-mon[65513]: pgmap v342: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:14.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:14.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:14.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:13 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:54:14.153 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:13 smithi115 ceph-mon[78672]: pgmap v342: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:14.153 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:14.153 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:14.154 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:13 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:54:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.870+0000 7ff889200700 1 -- 172.21.15.115:0/3070475497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884100be0 msgr2=0x7ff884102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.870+0000 7ff889200700 1 --2- 172.21.15.115:0/3070475497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884100be0 0x7ff884102fc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff874009910 tx=0x7ff874009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.871+0000 7ff889200700 1 -- 172.21.15.115:0/3070475497 shutdown_connections 2024-03-22T10:54:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.871+0000 7ff889200700 1 --2- 172.21.15.115:0/3070475497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff884103500 0x7ff8841058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.871+0000 7ff889200700 1 --2- 172.21.15.115:0/3070475497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884100be0 0x7ff884102fc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.871+0000 7ff889200700 1 -- 172.21.15.115:0/3070475497 >> 172.21.15.115:0/3070475497 conn(0x7ff8840fa5e0 msgr2=0x7ff8840fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.871+0000 7ff889200700 1 -- 172.21.15.115:0/3070475497 shutdown_connections 2024-03-22T10:54:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.871+0000 7ff889200700 1 -- 172.21.15.115:0/3070475497 wait complete. 2024-03-22T10:54:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.872+0000 7ff889200700 1 Processor -- start 2024-03-22T10:54:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.872+0000 7ff889200700 1 -- start start 2024-03-22T10:54:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.872+0000 7ff889200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff884100be0 0x7ff884100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.872+0000 7ff889200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884103500 0x7ff8840feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.872+0000 7ff889200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840ff0d0 con 0x7ff884103500 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.872+0000 7ff889200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840ff210 con 0x7ff884100be0 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.873+0000 7ff881400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884103500 0x7ff8840feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.873+0000 7ff881400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884103500 0x7ff8840feb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49960/0 (socket says 172.21.15.115:49960) 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.873+0000 7ff881400700 1 -- 172.21.15.115:0/2756501974 learned_addr learned my addr 172.21.15.115:0/2756501974 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.873+0000 7ff881e00700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff884100be0 0x7ff884100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.873+0000 7ff881400700 1 -- 172.21.15.115:0/2756501974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff884100be0 msgr2=0x7ff884100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.873+0000 7ff881400700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff884100be0 0x7ff884100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.873+0000 7ff881400700 1 -- 172.21.15.115:0/2756501974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740095f0 con 0x7ff884103500 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.873+0000 7ff881e00700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff884100be0 0x7ff884100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:54:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.874+0000 7ff881400700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884103500 0x7ff8840feb90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff87000d8d0 tx=0x7ff87000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:15.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.874+0000 7ff87aa00700 1 -- 172.21.15.115:0/2756501974 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff870009940 con 0x7ff884103500 2024-03-22T10:54:15.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.874+0000 7ff87aa00700 1 -- 172.21.15.115:0/2756501974 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff870010460 con 0x7ff884103500 2024-03-22T10:54:15.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.874+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8840ff4f0 con 0x7ff884103500 2024-03-22T10:54:15.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.874+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88419d980 con 0x7ff884103500 2024-03-22T10:54:15.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.875+0000 7ff87aa00700 1 -- 172.21.15.115:0/2756501974 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff87000f5f0 con 0x7ff884103500 2024-03-22T10:54:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.876+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff864005320 con 0x7ff884103500 2024-03-22T10:54:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.877+0000 7ff87aa00700 1 -- 172.21.15.115:0/2756501974 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff870009aa0 con 0x7ff884103500 2024-03-22T10:54:15.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.878+0000 7ff87aa00700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff86806dff0 0x7ff8680704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:15.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.878+0000 7ff87aa00700 1 -- 172.21.15.115:0/2756501974 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff87008dcf0 con 0x7ff884103500 2024-03-22T10:54:15.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.878+0000 7ff881e00700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff86806dff0 0x7ff8680704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:15.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.879+0000 7ff881e00700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff86806dff0 0x7ff8680704a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff874006010 tx=0x7ff87400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:15.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:15.887+0000 7ff87aa00700 1 -- 172.21.15.115:0/2756501974 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff87005a9e0 con 0x7ff884103500 2024-03-22T10:54:16.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:15 smithi150 ceph-mon[65513]: pgmap v343: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.169+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff864000bf0 con 0x7ff86806dff0 2024-03-22T10:54:16.162 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:15 smithi115 ceph-mon[78672]: pgmap v343: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.172+0000 7ff87aa00700 1 -- 172.21.15.115:0/2756501974 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7ff864000bf0 con 0x7ff86806dff0 2024-03-22T10:54:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.174+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff86806dff0 msgr2=0x7ff8680704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.174+0000 7ff889200700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff86806dff0 0x7ff8680704a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff874006010 tx=0x7ff87400b540 comp rx=0 tx=0).stop 2024-03-22T10:54:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.174+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884103500 msgr2=0x7ff8840feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.174+0000 7ff889200700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884103500 0x7ff8840feb90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff87000d8d0 tx=0x7ff87000dc90 comp rx=0 tx=0).stop 2024-03-22T10:54:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.175+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 shutdown_connections 2024-03-22T10:54:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.175+0000 7ff889200700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff86806dff0 0x7ff8680704a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.175+0000 7ff889200700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff884100be0 0x7ff884100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.175+0000 7ff889200700 1 --2- 172.21.15.115:0/2756501974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff884103500 0x7ff8840feb90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.175+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 >> 172.21.15.115:0/2756501974 conn(0x7ff8840fa5e0 msgr2=0x7ff8840fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.175+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 shutdown_connections 2024-03-22T10:54:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.175+0000 7ff889200700 1 -- 172.21.15.115:0/2756501974 wait complete. 2024-03-22T10:54:16.182 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:54:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.326+0000 7f0b38200700 1 -- 172.21.15.115:0/3486676254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30103560 msgr2=0x7f0b30105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.326+0000 7f0b38200700 1 --2- 172.21.15.115:0/3486676254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30103560 0x7f0b30105940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009910 tx=0x7f0b1c009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.326+0000 7f0b38200700 1 -- 172.21.15.115:0/3486676254 shutdown_connections 2024-03-22T10:54:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.326+0000 7f0b38200700 1 --2- 172.21.15.115:0/3486676254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30103560 0x7f0b30105940 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.326+0000 7f0b38200700 1 --2- 172.21.15.115:0/3486676254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b30100c40 0x7f0b30103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.326+0000 7f0b38200700 1 -- 172.21.15.115:0/3486676254 >> 172.21.15.115:0/3486676254 conn(0x7f0b300fa5e0 msgr2=0x7f0b300fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.326+0000 7f0b38200700 1 -- 172.21.15.115:0/3486676254 shutdown_connections 2024-03-22T10:54:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.326+0000 7f0b38200700 1 -- 172.21.15.115:0/3486676254 wait complete. 2024-03-22T10:54:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.327+0000 7f0b38200700 1 Processor -- start 2024-03-22T10:54:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.327+0000 7f0b38200700 1 -- start start 2024-03-22T10:54:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.327+0000 7f0b38200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30100c40 0x7f0b30197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.327+0000 7f0b38200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b30103560 0x7f0b301980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.328+0000 7f0b38200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b301986e0 con 0x7f0b30100c40 2024-03-22T10:54:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.328+0000 7f0b38200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30198820 con 0x7f0b30103560 2024-03-22T10:54:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.328+0000 7f0b35800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30100c40 0x7f0b30197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.328+0000 7f0b35800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30100c40 0x7f0b30197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49966/0 (socket says 172.21.15.115:49966) 2024-03-22T10:54:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.328+0000 7f0b35800700 1 -- 172.21.15.115:0/4207697176 learned_addr learned my addr 172.21.15.115:0/4207697176 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.328+0000 7f0b34e00700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b30103560 0x7f0b301980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.328+0000 7f0b35800700 1 -- 172.21.15.115:0/4207697176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b30103560 msgr2=0x7f0b301980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.328+0000 7f0b35800700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b30103560 0x7f0b301980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.328+0000 7f0b35800700 1 -- 172.21.15.115:0/4207697176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c0095f0 con 0x7f0b30100c40 2024-03-22T10:54:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.329+0000 7f0b35800700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30100c40 0x7f0b30197b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400b4b0 tx=0x7f0b2400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.329+0000 7f0b2e000700 1 -- 172.21.15.115:0/4207697176 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b24011800 con 0x7f0b30100c40 2024-03-22T10:54:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.329+0000 7f0b2e000700 1 -- 172.21.15.115:0/4207697176 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0b24011e10 con 0x7f0b30100c40 2024-03-22T10:54:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.329+0000 7f0b2e000700 1 -- 172.21.15.115:0/4207697176 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b2400f2e0 con 0x7f0b30100c40 2024-03-22T10:54:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.329+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3019d2d0 con 0x7f0b30100c40 2024-03-22T10:54:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.330+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3019d8a0 con 0x7f0b30100c40 2024-03-22T10:54:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.331+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b300fc1c0 con 0x7f0b30100c40 2024-03-22T10:54:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.332+0000 7f0b2e000700 1 -- 172.21.15.115:0/4207697176 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f0b24010380 con 0x7f0b30100c40 2024-03-22T10:54:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.333+0000 7f0b2e000700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0b1806dfa0 0x7f0b18070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.333+0000 7f0b2e000700 1 -- 172.21.15.115:0/4207697176 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f0b2408d9c0 con 0x7f0b30100c40 2024-03-22T10:54:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.333+0000 7f0b34e00700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0b1806dfa0 0x7f0b18070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:16.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.334+0000 7f0b34e00700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0b1806dfa0 0x7f0b18070450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c006010 tx=0x7f0b1c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.341+0000 7f0b2e000700 1 -- 172.21.15.115:0/4207697176 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b2405a8c0 con 0x7f0b30100c40 2024-03-22T10:54:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.706+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b30061190 con 0x7f0b1806dfa0 2024-03-22T10:54:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.707+0000 7f0b2e000700 1 -- 172.21.15.115:0/4207697176 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7f0b30061190 con 0x7f0b1806dfa0 2024-03-22T10:54:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.710+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0b1806dfa0 msgr2=0x7f0b18070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0b1806dfa0 0x7f0b18070450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c006010 tx=0x7f0b1c00b540 comp rx=0 tx=0).stop 2024-03-22T10:54:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30100c40 msgr2=0x7f0b30197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30100c40 0x7f0b30197b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400b4b0 tx=0x7f0b2400b870 comp rx=0 tx=0).stop 2024-03-22T10:54:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 shutdown_connections 2024-03-22T10:54:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0b1806dfa0 0x7f0b18070450 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b30100c40 0x7f0b30197b80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 --2- 172.21.15.115:0/4207697176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b30103560 0x7f0b301980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 >> 172.21.15.115:0/4207697176 conn(0x7f0b300fa5e0 msgr2=0x7f0b300ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 shutdown_connections 2024-03-22T10:54:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.711+0000 7f0b38200700 1 -- 172.21.15.115:0/4207697176 wait complete. 2024-03-22T10:54:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.874+0000 7f3723200700 1 -- 172.21.15.115:0/3969509809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c0fffa0 msgr2=0x7f371c1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.874+0000 7f3723200700 1 --2- 172.21.15.115:0/3969509809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c0fffa0 0x7f371c1003b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3710009910 tx=0x7f3710009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.874+0000 7f3723200700 1 -- 172.21.15.115:0/3969509809 shutdown_connections 2024-03-22T10:54:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.874+0000 7f3723200700 1 --2- 172.21.15.115:0/3969509809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f371c1011a0 0x7f371c1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.874+0000 7f3723200700 1 --2- 172.21.15.115:0/3969509809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c0fffa0 0x7f371c1003b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.874+0000 7f3723200700 1 -- 172.21.15.115:0/3969509809 >> 172.21.15.115:0/3969509809 conn(0x7f371c0fb510 msgr2=0x7f371c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.874+0000 7f3723200700 1 -- 172.21.15.115:0/3969509809 shutdown_connections 2024-03-22T10:54:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.874+0000 7f3723200700 1 -- 172.21.15.115:0/3969509809 wait complete. 2024-03-22T10:54:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.875+0000 7f3723200700 1 Processor -- start 2024-03-22T10:54:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.876+0000 7f3723200700 1 -- start start 2024-03-22T10:54:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.876+0000 7f3723200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f371c0fffa0 0x7f371c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.876+0000 7f3723200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c1011a0 0x7f371c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.876+0000 7f3723200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c198520 con 0x7f371c1011a0 2024-03-22T10:54:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.876+0000 7f3723200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c198660 con 0x7f371c0fffa0 2024-03-22T10:54:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.876+0000 7f3721400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c1011a0 0x7f371c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.877+0000 7f3721400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c1011a0 0x7f371c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49990/0 (socket says 172.21.15.115:49990) 2024-03-22T10:54:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.877+0000 7f3721400700 1 -- 172.21.15.115:0/206383181 learned_addr learned my addr 172.21.15.115:0/206383181 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.877+0000 7f3721e00700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f371c0fffa0 0x7f371c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.877+0000 7f3721400700 1 -- 172.21.15.115:0/206383181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f371c0fffa0 msgr2=0x7f371c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.877+0000 7f3721400700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f371c0fffa0 0x7f371c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.877+0000 7f3721400700 1 -- 172.21.15.115:0/206383181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37100095f0 con 0x7f371c1011a0 2024-03-22T10:54:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.878+0000 7f3721400700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c1011a0 0x7f371c197f00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f370c00d8d0 tx=0x7f370c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.878+0000 7f3716a00700 1 -- 172.21.15.115:0/206383181 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f370c009940 con 0x7f371c1011a0 2024-03-22T10:54:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.878+0000 7f3716a00700 1 -- 172.21.15.115:0/206383181 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f370c010460 con 0x7f371c1011a0 2024-03-22T10:54:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.878+0000 7f3716a00700 1 -- 172.21.15.115:0/206383181 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f370c00f5f0 con 0x7f371c1011a0 2024-03-22T10:54:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.878+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c19d110 con 0x7f371c1011a0 2024-03-22T10:54:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.878+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c19d660 con 0x7f371c1011a0 2024-03-22T10:54:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.880+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c105280 con 0x7f371c1011a0 2024-03-22T10:54:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.881+0000 7f3716a00700 1 -- 172.21.15.115:0/206383181 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f370c0105d0 con 0x7f371c1011a0 2024-03-22T10:54:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.882+0000 7f3716a00700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f370406dfa0 0x7f3704070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.882+0000 7f3716a00700 1 -- 172.21.15.115:0/206383181 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f370c08dd60 con 0x7f371c1011a0 2024-03-22T10:54:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.883+0000 7f3721e00700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f370406dfa0 0x7f3704070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.883+0000 7f3721e00700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f370406dfa0 0x7f3704070450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3710006010 tx=0x7f371000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:16.888+0000 7f3716a00700 1 -- 172.21.15.115:0/206383181 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f370c092070 con 0x7f371c1011a0 2024-03-22T10:54:17.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.128+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f371c061190 con 0x7f370406dfa0 2024-03-22T10:54:17.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.140+0000 7f3716a00700 1 -- 172.21.15.115:0/206383181 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f371c061190 con 0x7f370406dfa0 2024-03-22T10:54:17.132 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:54:17.132 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (42s) 35s ago 22m 14.5M - 0.25.0 c8568f914cd2 ff3d0ae1fb82 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (2m) 35s ago 22m 8102k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e11b0bdaf069 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (117s) 85s ago 21m 7662k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 ef9665eef7b1 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (8m) 35s ago 22m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (8m) 85s ago 21m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 starting - - - - 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (3m) 35s ago 15m 67.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 44a0769ebe8e 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (3m) 35s ago 15m 17.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51d700f6eb18 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (2m) 85s ago 15m 12.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 82b6b216f754 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (2m) 85s ago 15m 12.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9605445d5b2b 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (12m) 35s ago 24m 516M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (10m) 85s ago 21m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (10m) 35s ago 24m 60.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (9m) 85s ago 21m 45.7M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:54:17.133 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (97s) 35s ago 22m 7377k - 1.5.0 0da6a335fe13 a98cb20e74e1 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (90s) 85s ago 21m 2935k - 1.5.0 0da6a335fe13 6c3ba9d80db0 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (8m) 35s ago 20m 98.0M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (7m) 35s ago 19m 96.7M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (6m) 35s ago 19m 83.7M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (6m) 85s ago 18m 103M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (5m) 85s ago 17m 91.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (5m) 85s ago 17m 90.9M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (66s) 35s ago 22m 62.5M - 2.43.0 a07b618ecd1d 8561aedaa7b5 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f370406dfa0 msgr2=0x7f3704070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:17.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f370406dfa0 0x7f3704070450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3710006010 tx=0x7f371000b540 comp rx=0 tx=0).stop 2024-03-22T10:54:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c1011a0 msgr2=0x7f371c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c1011a0 0x7f371c197f00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f370c00d8d0 tx=0x7f370c00dc90 comp rx=0 tx=0).stop 2024-03-22T10:54:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 shutdown_connections 2024-03-22T10:54:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f370406dfa0 0x7f3704070450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f371c0fffa0 0x7f371c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 --2- 172.21.15.115:0/206383181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f371c1011a0 0x7f371c197f00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 >> 172.21.15.115:0/206383181 conn(0x7f371c0fb510 msgr2=0x7f371c1023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 shutdown_connections 2024-03-22T10:54:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.143+0000 7f3723200700 1 -- 172.21.15.115:0/206383181 wait complete. 2024-03-22T10:54:17.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.301+0000 7f40e6e00700 1 -- 172.21.15.115:0/4097772460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 msgr2=0x7f40e0100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:17.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.301+0000 7f40e6e00700 1 --2- 172.21.15.115:0/4097772460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 0x7f40e0100560 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009910 tx=0x7f40d0009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:17.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.302+0000 7f40e6e00700 1 -- 172.21.15.115:0/4097772460 shutdown_connections 2024-03-22T10:54:17.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.302+0000 7f40e6e00700 1 --2- 172.21.15.115:0/4097772460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e0101350 0x7f40e01017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.302+0000 7f40e6e00700 1 --2- 172.21.15.115:0/4097772460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 0x7f40e0100560 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.302+0000 7f40e6e00700 1 -- 172.21.15.115:0/4097772460 >> 172.21.15.115:0/4097772460 conn(0x7f40e00fb6e0 msgr2=0x7f40e00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:17.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.302+0000 7f40e6e00700 1 -- 172.21.15.115:0/4097772460 shutdown_connections 2024-03-22T10:54:17.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.302+0000 7f40e6e00700 1 -- 172.21.15.115:0/4097772460 wait complete. 2024-03-22T10:54:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.303+0000 7f40e6e00700 1 Processor -- start 2024-03-22T10:54:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.304+0000 7f40e6e00700 1 -- start start 2024-03-22T10:54:17.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.304+0000 7f40e6e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 0x7f40e0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:17.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.304+0000 7f40e6e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e0101350 0x7f40e0073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:17.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.304+0000 7f40e6e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e0073680 con 0x7f40e0100150 2024-03-22T10:54:17.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.304+0000 7f40e6e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e00737c0 con 0x7f40e0101350 2024-03-22T10:54:17.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.304+0000 7f40dfe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 0x7f40e0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:17.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.304+0000 7f40dfe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 0x7f40e0074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50008/0 (socket says 172.21.15.115:50008) 2024-03-22T10:54:17.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.305+0000 7f40dfe00700 1 -- 172.21.15.115:0/1355054997 learned_addr learned my addr 172.21.15.115:0/1355054997 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:17.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.305+0000 7f40df400700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e0101350 0x7f40e0073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:17.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.305+0000 7f40dfe00700 1 -- 172.21.15.115:0/1355054997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e0101350 msgr2=0x7f40e0073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:17.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.305+0000 7f40dfe00700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e0101350 0x7f40e0073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.305+0000 7f40dfe00700 1 -- 172.21.15.115:0/1355054997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00095f0 con 0x7f40e0100150 2024-03-22T10:54:17.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.305+0000 7f40dfe00700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 0x7f40e0074af0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f40d000bda0 tx=0x7f40d000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:17.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.306+0000 7f40dcc00700 1 -- 172.21.15.115:0/1355054997 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40d001d070 con 0x7f40e0100150 2024-03-22T10:54:17.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.306+0000 7f40dcc00700 1 -- 172.21.15.115:0/1355054997 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f40d000f460 con 0x7f40e0100150 2024-03-22T10:54:17.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.306+0000 7f40dcc00700 1 -- 172.21.15.115:0/1355054997 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40d0021620 con 0x7f40e0100150 2024-03-22T10:54:17.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.306+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e0073a40 con 0x7f40e0100150 2024-03-22T10:54:17.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.306+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e0073ee0 con 0x7f40e0100150 2024-03-22T10:54:17.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.307+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e0108ca0 con 0x7f40e0100150 2024-03-22T10:54:17.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.309+0000 7f40dcc00700 1 -- 172.21.15.115:0/1355054997 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f40d000f5d0 con 0x7f40e0100150 2024-03-22T10:54:17.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.310+0000 7f40dcc00700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f40c406dfa0 0x7f40c4070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:17.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.310+0000 7f40dcc00700 1 -- 172.21.15.115:0/1355054997 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f40d008f280 con 0x7f40e0100150 2024-03-22T10:54:17.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.312+0000 7f40df400700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f40c406dfa0 0x7f40c4070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:17.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.312+0000 7f40df400700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f40c406dfa0 0x7f40c4070450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40cc005950 tx=0x7f40cc0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:17.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.316+0000 7f40dcc00700 1 -- 172.21.15.115:0/1355054997 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f40d005c100 con 0x7f40e0100150 2024-03-22T10:54:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.678+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f40e019de80 con 0x7f40e0100150 2024-03-22T10:54:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.679+0000 7f40dcc00700 1 -- 172.21.15.115:0/1355054997 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f40d005bc90 con 0x7f40e0100150 2024-03-22T10:54:17.672 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:54:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:54:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:54:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:54:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:54:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:54:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:54:17.672 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:54:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:54:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:54:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:54:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:54:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:54:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:54:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-22T10:54:17.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:54:17.673 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:54:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.682+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f40c406dfa0 msgr2=0x7f40c4070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.682+0000 7f40e6e00700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f40c406dfa0 0x7f40c4070450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40cc005950 tx=0x7f40cc0058e0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.682+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 msgr2=0x7f40e0074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.682+0000 7f40e6e00700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 0x7f40e0074af0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f40d000bda0 tx=0x7f40d000be80 comp rx=0 tx=0).stop 2024-03-22T10:54:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.683+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 shutdown_connections 2024-03-22T10:54:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.683+0000 7f40e6e00700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f40c406dfa0 0x7f40c4070450 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.683+0000 7f40e6e00700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e0100150 0x7f40e0074af0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.683+0000 7f40e6e00700 1 --2- 172.21.15.115:0/1355054997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e0101350 0x7f40e0073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.683+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 >> 172.21.15.115:0/1355054997 conn(0x7f40e00fb6e0 msgr2=0x7f40e0104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.683+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 shutdown_connections 2024-03-22T10:54:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.683+0000 7f40e6e00700 1 -- 172.21.15.115:0/1355054997 wait complete. 2024-03-22T10:54:17.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.878+0000 7fa7c4a00700 1 -- 172.21.15.115:0/2107294233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00ff250 msgr2=0x7fa7c00ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:17.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.878+0000 7fa7c4a00700 1 --2- 172.21.15.115:0/2107294233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00ff250 0x7fa7c00ff6c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009910 tx=0x7fa7ac009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:17.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.878+0000 7fa7c4a00700 1 -- 172.21.15.115:0/2107294233 shutdown_connections 2024-03-22T10:54:17.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.878+0000 7fa7c4a00700 1 --2- 172.21.15.115:0/2107294233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00ff250 0x7fa7c00ff6c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.878+0000 7fa7c4a00700 1 --2- 172.21.15.115:0/2107294233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7c00fe900 0x7fa7c00fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.878+0000 7fa7c4a00700 1 -- 172.21.15.115:0/2107294233 >> 172.21.15.115:0/2107294233 conn(0x7fa7c00fa4b0 msgr2=0x7fa7c00fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:17.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.878+0000 7fa7c4a00700 1 -- 172.21.15.115:0/2107294233 shutdown_connections 2024-03-22T10:54:17.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.878+0000 7fa7c4a00700 1 -- 172.21.15.115:0/2107294233 wait complete. 2024-03-22T10:54:17.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.879+0000 7fa7c4a00700 1 Processor -- start 2024-03-22T10:54:17.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.879+0000 7fa7c4a00700 1 -- start start 2024-03-22T10:54:17.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.880+0000 7fa7c4a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00fe900 0x7fa7c0197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:17.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.880+0000 7fa7c4a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7c00ff250 0x7fa7c0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:17.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.880+0000 7fa7c4a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01984b0 con 0x7fa7c00fe900 2024-03-22T10:54:17.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.880+0000 7fa7c4a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01985f0 con 0x7fa7c00ff250 2024-03-22T10:54:17.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.880+0000 7fa7be800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00fe900 0x7fa7c0197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:17.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.880+0000 7fa7be800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00fe900 0x7fa7c0197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50030/0 (socket says 172.21.15.115:50030) 2024-03-22T10:54:17.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.881+0000 7fa7be800700 1 -- 172.21.15.115:0/804210678 learned_addr learned my addr 172.21.15.115:0/804210678 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:17.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.881+0000 7fa7bde00700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7c00ff250 0x7fa7c0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:17.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.881+0000 7fa7be800700 1 -- 172.21.15.115:0/804210678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7c00ff250 msgr2=0x7fa7c0197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:17.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.881+0000 7fa7be800700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7c00ff250 0x7fa7c0197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:17.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.881+0000 7fa7be800700 1 -- 172.21.15.115:0/804210678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac0095f0 con 0x7fa7c00fe900 2024-03-22T10:54:17.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.882+0000 7fa7be800700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00fe900 0x7fa7c0197950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000d8d0 tx=0x7fa7b000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:17.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.882+0000 7fa7b7400700 1 -- 172.21.15.115:0/804210678 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7b00098a0 con 0x7fa7c00fe900 2024-03-22T10:54:17.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.882+0000 7fa7b7400700 1 -- 172.21.15.115:0/804210678 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa7b0010460 con 0x7fa7c00fe900 2024-03-22T10:54:17.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.882+0000 7fa7b7400700 1 -- 172.21.15.115:0/804210678 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7b000f5f0 con 0x7fa7c00fe900 2024-03-22T10:54:17.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.882+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c019d0a0 con 0x7fa7c00fe900 2024-03-22T10:54:17.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.882+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c019d5f0 con 0x7fa7c00fe900 2024-03-22T10:54:17.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.883+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c010acf0 con 0x7fa7c00fe900 2024-03-22T10:54:17.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.886+0000 7fa7b7400700 1 -- 172.21.15.115:0/804210678 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa7b0009a00 con 0x7fa7c00fe900 2024-03-22T10:54:17.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.888+0000 7fa7b7400700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa7a406dfa0 0x7fa7a4070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:17.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.888+0000 7fa7b7400700 1 -- 172.21.15.115:0/804210678 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa7b008dbc0 con 0x7fa7c00fe900 2024-03-22T10:54:17.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.888+0000 7fa7bde00700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa7a406dfa0 0x7fa7a4070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:17.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.889+0000 7fa7bde00700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa7a406dfa0 0x7fa7a4070450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006010 tx=0x7fa7ac00b560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:17.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:17.893+0000 7fa7b7400700 1 -- 172.21.15.115:0/804210678 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7b005acb0 con 0x7fa7c00fe900 2024-03-22T10:54:18.126 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:17 smithi150 ceph-mon[65513]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:18.126 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:17 smithi150 ceph-mon[65513]: pgmap v344: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:18.126 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:17 smithi150 ceph-mon[65513]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:18.126 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:17 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1355054997' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:18.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.186+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa7c019e140 con 0x7fa7c00fe900 2024-03-22T10:54:18.179 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:17 smithi115 ceph-mon[78672]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:18.179 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:17 smithi115 ceph-mon[78672]: pgmap v344: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:18.179 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:17 smithi115 ceph-mon[78672]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:18.179 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:17 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1355054997' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.186+0000 7fa7b7400700 1 -- 172.21.15.115:0/804210678 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1932 (secure 0 0 0) 0x7fa7b005a840 con 0x7fa7c00fe900 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stdout:e40 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 40 2024-03-22T10:54:18.180 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:52:05.899739+0000 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 83 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=34476,1=34508} 2024-03-22T10:54:18.181 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 34476 members: 34476,34508 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:34476} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{1:34508} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:54:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6824/1804848703,v1:172.21.15.150:6825/1804848703] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6826/2053213512,v1:172.21.15.150:6827/2053213512] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa7a406dfa0 msgr2=0x7fa7a4070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa7a406dfa0 0x7fa7a4070450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006010 tx=0x7fa7ac00b560 comp rx=0 tx=0).stop 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00fe900 msgr2=0x7fa7c0197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00fe900 0x7fa7c0197950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000d8d0 tx=0x7fa7b000dbe0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 shutdown_connections 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa7a406dfa0 0x7fa7a4070450 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c00fe900 0x7fa7c0197950 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 --2- 172.21.15.115:0/804210678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7c00ff250 0x7fa7c0197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 >> 172.21.15.115:0/804210678 conn(0x7fa7c00fa4b0 msgr2=0x7fa7c0106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.189+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 shutdown_connections 2024-03-22T10:54:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.190+0000 7fa7c4a00700 1 -- 172.21.15.115:0/804210678 wait complete. 2024-03-22T10:54:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2024-03-22T10:54:18.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.343+0000 7fb501000700 1 -- 172.21.15.115:0/328689903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 msgr2=0x7fb4fc101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:18.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.343+0000 7fb501000700 1 --2- 172.21.15.115:0/328689903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 0x7fb4fc101600 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009910 tx=0x7fb4e8009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:18.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.343+0000 7fb501000700 1 -- 172.21.15.115:0/328689903 shutdown_connections 2024-03-22T10:54:18.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.343+0000 7fb501000700 1 --2- 172.21.15.115:0/328689903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 0x7fb4fc101600 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.343+0000 7fb501000700 1 --2- 172.21.15.115:0/328689903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4fc0fffb0 0x7fb4fc1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.343+0000 7fb501000700 1 -- 172.21.15.115:0/328689903 >> 172.21.15.115:0/328689903 conn(0x7fb4fc0fb560 msgr2=0x7fb4fc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:18.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.344+0000 7fb501000700 1 -- 172.21.15.115:0/328689903 shutdown_connections 2024-03-22T10:54:18.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.344+0000 7fb501000700 1 -- 172.21.15.115:0/328689903 wait complete. 2024-03-22T10:54:18.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.345+0000 7fb501000700 1 Processor -- start 2024-03-22T10:54:18.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.345+0000 7fb501000700 1 -- start start 2024-03-22T10:54:18.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.345+0000 7fb501000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4fc0fffb0 0x7fb4fc1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:18.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.345+0000 7fb501000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 0x7fb4fc197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:18.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.345+0000 7fb501000700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc198520 con 0x7fb4fc1011b0 2024-03-22T10:54:18.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.345+0000 7fb501000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc198660 con 0x7fb4fc0fffb0 2024-03-22T10:54:18.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.346+0000 7fb4faa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 0x7fb4fc197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:18.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.346+0000 7fb4faa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 0x7fb4fc197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50052/0 (socket says 172.21.15.115:50052) 2024-03-22T10:54:18.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.346+0000 7fb4faa00700 1 -- 172.21.15.115:0/506161961 learned_addr learned my addr 172.21.15.115:0/506161961 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:18.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.346+0000 7fb4fb400700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4fc0fffb0 0x7fb4fc1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:18.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.346+0000 7fb4faa00700 1 -- 172.21.15.115:0/506161961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4fc0fffb0 msgr2=0x7fb4fc1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:18.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.346+0000 7fb4faa00700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4fc0fffb0 0x7fb4fc1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.346+0000 7fb4faa00700 1 -- 172.21.15.115:0/506161961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e80095f0 con 0x7fb4fc1011b0 2024-03-22T10:54:18.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.347+0000 7fb4faa00700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 0x7fb4fc197f00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8006010 tx=0x7fb4e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:18.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.347+0000 7fb4f3e00700 1 -- 172.21.15.115:0/506161961 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4e801d070 con 0x7fb4fc1011b0 2024-03-22T10:54:18.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.347+0000 7fb4f3e00700 1 -- 172.21.15.115:0/506161961 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb4e800f460 con 0x7fb4fc1011b0 2024-03-22T10:54:18.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.347+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc19d0b0 con 0x7fb4fc1011b0 2024-03-22T10:54:18.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.347+0000 7fb4f3e00700 1 -- 172.21.15.115:0/506161961 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4e8017630 con 0x7fb4fc1011b0 2024-03-22T10:54:18.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.347+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc19d5a0 con 0x7fb4fc1011b0 2024-03-22T10:54:18.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.349+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc105290 con 0x7fb4fc1011b0 2024-03-22T10:54:18.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.350+0000 7fb4f3e00700 1 -- 172.21.15.115:0/506161961 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb4e8003a60 con 0x7fb4fc1011b0 2024-03-22T10:54:18.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.351+0000 7fb4f3e00700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb4e006dfa0 0x7fb4e0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:18.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.352+0000 7fb4f3e00700 1 -- 172.21.15.115:0/506161961 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb4e808f310 con 0x7fb4fc1011b0 2024-03-22T10:54:18.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.352+0000 7fb4fb400700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb4e006dfa0 0x7fb4e0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:18.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.353+0000 7fb4fb400700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb4e006dfa0 0x7fb4e0070450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec005950 tx=0x7fb4ec0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:18.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.358+0000 7fb4f3e00700 1 -- 172.21.15.115:0/506161961 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb4e805c210 con 0x7fb4fc1011b0 2024-03-22T10:54:18.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.633+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4fc061190 con 0x7fb4e006dfa0 2024-03-22T10:54:18.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.636+0000 7fb4f3e00700 1 -- 172.21.15.115:0/506161961 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7fb4fc061190 con 0x7fb4e006dfa0 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": true, 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [ 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr", 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "mon", 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osd", 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "mds", 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph-exporter" 2024-03-22T10:54:18.629 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-03-22T10:54:18.630 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": "18/23 daemons upgraded", 2024-03-22T10:54:18.630 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "Currently upgrading grafana daemons", 2024-03-22T10:54:18.630 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:54:18.630 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:54:18.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.638+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb4e006dfa0 msgr2=0x7fb4e0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:18.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.638+0000 7fb501000700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb4e006dfa0 0x7fb4e0070450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec005950 tx=0x7fb4ec0058e0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.638+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 msgr2=0x7fb4fc197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:18.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.638+0000 7fb501000700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 0x7fb4fc197f00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8006010 tx=0x7fb4e800be20 comp rx=0 tx=0).stop 2024-03-22T10:54:18.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.638+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 shutdown_connections 2024-03-22T10:54:18.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.638+0000 7fb501000700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb4e006dfa0 0x7fb4e0070450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.638+0000 7fb501000700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4fc0fffb0 0x7fb4fc1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.638+0000 7fb501000700 1 --2- 172.21.15.115:0/506161961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4fc1011b0 0x7fb4fc197f00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.638+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 >> 172.21.15.115:0/506161961 conn(0x7fb4fc0fb560 msgr2=0x7fb4fc1023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:18.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.639+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 shutdown_connections 2024-03-22T10:54:18.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.639+0000 7fb501000700 1 -- 172.21.15.115:0/506161961 wait complete. 2024-03-22T10:54:18.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.792+0000 7fa399e00700 1 -- 172.21.15.115:0/2073751806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 msgr2=0x7fa3941027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:18.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.792+0000 7fa399e00700 1 --2- 172.21.15.115:0/2073751806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 0x7fa3941027c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa384009910 tx=0x7fa384009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:18.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.793+0000 7fa399e00700 1 -- 172.21.15.115:0/2073751806 shutdown_connections 2024-03-22T10:54:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.793+0000 7fa399e00700 1 --2- 172.21.15.115:0/2073751806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3941035b0 0x7fa394103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.793+0000 7fa399e00700 1 --2- 172.21.15.115:0/2073751806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 0x7fa3941027c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.793+0000 7fa399e00700 1 -- 172.21.15.115:0/2073751806 >> 172.21.15.115:0/2073751806 conn(0x7fa3940fd960 msgr2=0x7fa3940ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.793+0000 7fa399e00700 1 -- 172.21.15.115:0/2073751806 shutdown_connections 2024-03-22T10:54:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.793+0000 7fa399e00700 1 -- 172.21.15.115:0/2073751806 wait complete. 2024-03-22T10:54:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.794+0000 7fa399e00700 1 Processor -- start 2024-03-22T10:54:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.794+0000 7fa399e00700 1 -- start start 2024-03-22T10:54:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.795+0000 7fa399e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 0x7fa394197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.795+0000 7fa399e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3941035b0 0x7fa3941980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.795+0000 7fa399e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa394198710 con 0x7fa3941023b0 2024-03-22T10:54:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.795+0000 7fa399e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa394198850 con 0x7fa3941035b0 2024-03-22T10:54:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.795+0000 7fa392a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 0x7fa394197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.795+0000 7fa392a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 0x7fa394197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50078/0 (socket says 172.21.15.115:50078) 2024-03-22T10:54:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.795+0000 7fa392a00700 1 -- 172.21.15.115:0/2702277628 learned_addr learned my addr 172.21.15.115:0/2702277628 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.796+0000 7fa392000700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3941035b0 0x7fa3941980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.796+0000 7fa392a00700 1 -- 172.21.15.115:0/2702277628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3941035b0 msgr2=0x7fa3941980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.796+0000 7fa392a00700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3941035b0 0x7fa3941980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.796+0000 7fa392a00700 1 -- 172.21.15.115:0/2702277628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3840095f0 con 0x7fa3941023b0 2024-03-22T10:54:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.796+0000 7fa392a00700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 0x7fa394197bb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa38400bd70 tx=0x7fa38400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.797+0000 7fa398a00700 1 -- 172.21.15.115:0/2702277628 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38401d070 con 0x7fa3941023b0 2024-03-22T10:54:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.797+0000 7fa398a00700 1 -- 172.21.15.115:0/2702277628 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa38400f460 con 0x7fa3941023b0 2024-03-22T10:54:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.797+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39419d2a0 con 0x7fa3941023b0 2024-03-22T10:54:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.797+0000 7fa398a00700 1 -- 172.21.15.115:0/2702277628 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa384021640 con 0x7fa3941023b0 2024-03-22T10:54:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.797+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39419d790 con 0x7fa3941023b0 2024-03-22T10:54:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.798+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39410af00 con 0x7fa3941023b0 2024-03-22T10:54:18.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.800+0000 7fa398a00700 1 -- 172.21.15.115:0/2702277628 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa38400fab0 con 0x7fa3941023b0 2024-03-22T10:54:18.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.801+0000 7fa398a00700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa37806dfa0 0x7fa378070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:18.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.801+0000 7fa398a00700 1 -- 172.21.15.115:0/2702277628 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa38408f250 con 0x7fa3941023b0 2024-03-22T10:54:18.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.802+0000 7fa392000700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa37806dfa0 0x7fa378070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:18.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.802+0000 7fa392000700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa37806dfa0 0x7fa378070450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa380005fd0 tx=0x7fa38000a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:18.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:18.810+0000 7fa398a00700 1 -- 172.21.15.115:0/2702277628 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa38405c150 con 0x7fa3941023b0 2024-03-22T10:54:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:18 smithi150 ceph-mon[65513]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:19.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:18 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/804210678' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:54:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.222+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa39410b0f0 con 0x7fa3941023b0 2024-03-22T10:54:19.215 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:18 smithi115 ceph-mon[78672]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:19.216 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:18 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/804210678' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:54:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.223+0000 7fa398a00700 1 -- 172.21.15.115:0/2702277628 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fa38405bce0 con 0x7fa3941023b0 2024-03-22T10:54:19.216 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:54:19.216 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:54:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:54:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.226+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa37806dfa0 msgr2=0x7fa378070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.226+0000 7fa399e00700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa37806dfa0 0x7fa378070450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa380005fd0 tx=0x7fa38000a560 comp rx=0 tx=0).stop 2024-03-22T10:54:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.226+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 msgr2=0x7fa394197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.226+0000 7fa399e00700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 0x7fa394197bb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa38400bd70 tx=0x7fa38400be50 comp rx=0 tx=0).stop 2024-03-22T10:54:19.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.227+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 shutdown_connections 2024-03-22T10:54:19.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.227+0000 7fa399e00700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa37806dfa0 0x7fa378070450 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:19.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.227+0000 7fa399e00700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941023b0 0x7fa394197bb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:19.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.227+0000 7fa399e00700 1 --2- 172.21.15.115:0/2702277628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3941035b0 0x7fa3941980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:19.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.227+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 >> 172.21.15.115:0/2702277628 conn(0x7fa3940fd960 msgr2=0x7fa3941067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:19.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.227+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 shutdown_connections 2024-03-22T10:54:19.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:19.227+0000 7fa399e00700 1 -- 172.21.15.115:0/2702277628 wait complete. 2024-03-22T10:54:20.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:19 smithi150 ceph-mon[65513]: pgmap v345: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:20.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:19 smithi150 ceph-mon[65513]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:20.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:19 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2702277628' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:54:20.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:54:20.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:20.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:20.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:19 smithi115 ceph-mon[78672]: pgmap v345: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:20.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:19 smithi115 ceph-mon[78672]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:20.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:19 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2702277628' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:54:20.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:54:20.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:20.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:22.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:21 smithi150 ceph-mon[65513]: pgmap v346: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:22.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:22.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:21 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:21 smithi115 ceph-mon[78672]: pgmap v346: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:22.261 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:21 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:23.929 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:23 smithi115 ceph-mon[78672]: pgmap v347: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:24.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:23 smithi150 ceph-mon[65513]: pgmap v347: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:26.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:25 smithi150 ceph-mon[65513]: pgmap v348: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:26.196 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:25 smithi115 ceph-mon[78672]: pgmap v348: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: pgmap v349: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:54:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-22T10:54:28.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:54:28.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: pgmap v349: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:28.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:54:28.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T10:54:28.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T10:54:28.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T10:54:28.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:54:28.155 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:54:28.156 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:54:28.156 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:28.156 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:54:28.156 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:54:28.156 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:54:28.156 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:28 smithi115 ceph-mon[78672]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T10:54:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:28 smithi115 ceph-mon[78672]: Upgrade: Finalizing container_image settings 2024-03-22T10:54:29.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:28 smithi115 ceph-mon[78672]: Upgrade: Complete! 2024-03-22T10:54:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:28 smithi150 ceph-mon[65513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T10:54:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:28 smithi150 ceph-mon[65513]: Upgrade: Finalizing container_image settings 2024-03-22T10:54:29.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:28 smithi150 ceph-mon[65513]: Upgrade: Complete! 2024-03-22T10:54:30.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:29 smithi115 ceph-mon[78672]: pgmap v350: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:30.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:29 smithi150 ceph-mon[65513]: pgmap v350: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:30.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:54:32.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:31 smithi115 ceph-mon[78672]: pgmap v351: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:32.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:31 smithi150 ceph-mon[65513]: pgmap v351: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:34.223 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:33 smithi115 ceph-mon[78672]: pgmap v352: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:34.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:33 smithi150 ceph-mon[65513]: pgmap v352: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:35.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:54:35.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:54:36.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:35 smithi115 ceph-mon[78672]: pgmap v353: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:36.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:35 smithi150 ceph-mon[65513]: pgmap v353: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:38.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:37 smithi115 ceph-mon[78672]: pgmap v354: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:38.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:37 smithi150 ceph-mon[65513]: pgmap v354: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:40.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:39 smithi115 ceph-mon[78672]: pgmap v355: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:40.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:39 smithi150 ceph-mon[65513]: pgmap v355: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:42.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:42 smithi115 ceph-mon[78672]: pgmap v356: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:42.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:42 smithi150 ceph-mon[65513]: pgmap v356: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:44.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:44 smithi150 ceph-mon[65513]: pgmap v357: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:44.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:44 smithi115 ceph-mon[78672]: pgmap v357: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:45.399 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:45 smithi150 ceph-mon[65513]: pgmap v358: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:45.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:45 smithi115 ceph-mon[78672]: pgmap v358: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:48.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:47 smithi115 ceph-mon[78672]: pgmap v359: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:48.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:47 smithi150 ceph-mon[65513]: pgmap v359: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:49.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.442+0000 7fd7e0400700 1 -- 172.21.15.115:0/2284774033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80ff380 msgr2=0x7fd7d80ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:49.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.442+0000 7fd7e0400700 1 --2- 172.21.15.115:0/2284774033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80ff380 0x7fd7d80ff7f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009910 tx=0x7fd7c4009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:49.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.442+0000 7fd7e0400700 1 -- 172.21.15.115:0/2284774033 shutdown_connections 2024-03-22T10:54:49.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.442+0000 7fd7e0400700 1 --2- 172.21.15.115:0/2284774033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80ff380 0x7fd7d80ff7f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:49.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.442+0000 7fd7e0400700 1 --2- 172.21.15.115:0/2284774033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7d80fea30 0x7fd7d80fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:49.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.442+0000 7fd7e0400700 1 -- 172.21.15.115:0/2284774033 >> 172.21.15.115:0/2284774033 conn(0x7fd7d80fa600 msgr2=0x7fd7d80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:49.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.442+0000 7fd7e0400700 1 -- 172.21.15.115:0/2284774033 shutdown_connections 2024-03-22T10:54:49.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.442+0000 7fd7e0400700 1 -- 172.21.15.115:0/2284774033 wait complete. 2024-03-22T10:54:49.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.443+0000 7fd7e0400700 1 Processor -- start 2024-03-22T10:54:49.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.443+0000 7fd7e0400700 1 -- start start 2024-03-22T10:54:49.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.443+0000 7fd7e0400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80fea30 0x7fd7d8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:49.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7e0400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7d80ff380 0x7fd7d8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:49.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7e0400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d81986a0 con 0x7fd7d80fea30 2024-03-22T10:54:49.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7e0400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d81987e0 con 0x7fd7d80ff380 2024-03-22T10:54:49.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7dda00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80fea30 0x7fd7d8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:49.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7dda00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80fea30 0x7fd7d8197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33996/0 (socket says 172.21.15.115:33996) 2024-03-22T10:54:49.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7dda00700 1 -- 172.21.15.115:0/2887931672 learned_addr learned my addr 172.21.15.115:0/2887931672 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:49.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7dd000700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7d80ff380 0x7fd7d8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:49.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7dda00700 1 -- 172.21.15.115:0/2887931672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7d80ff380 msgr2=0x7fd7d8198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:49.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7dda00700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7d80ff380 0x7fd7d8198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:49.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.444+0000 7fd7dda00700 1 -- 172.21.15.115:0/2887931672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40095f0 con 0x7fd7d80fea30 2024-03-22T10:54:49.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.445+0000 7fd7dda00700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80fea30 0x7fd7d8197b40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00d8d0 tx=0x7fd7cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:49.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.445+0000 7fd7d6000700 1 -- 172.21.15.115:0/2887931672 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7cc009940 con 0x7fd7d80fea30 2024-03-22T10:54:49.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.445+0000 7fd7d6000700 1 -- 172.21.15.115:0/2887931672 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7cc010460 con 0x7fd7d80fea30 2024-03-22T10:54:49.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.445+0000 7fd7d6000700 1 -- 172.21.15.115:0/2887931672 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7cc00f5f0 con 0x7fd7d80fea30 2024-03-22T10:54:49.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.445+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d819d290 con 0x7fd7d80fea30 2024-03-22T10:54:49.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.445+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d819d860 con 0x7fd7d80fea30 2024-03-22T10:54:49.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.446+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d810ae80 con 0x7fd7d80fea30 2024-03-22T10:54:49.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.449+0000 7fd7d6000700 1 -- 172.21.15.115:0/2887931672 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fd7cc0105d0 con 0x7fd7d80fea30 2024-03-22T10:54:49.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.450+0000 7fd7d6000700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd7c006dfa0 0x7fd7c0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:49.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.450+0000 7fd7d6000700 1 -- 172.21.15.115:0/2887931672 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd7cc08dcd0 con 0x7fd7d80fea30 2024-03-22T10:54:49.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.451+0000 7fd7dd000700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd7c006dfa0 0x7fd7c0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:49.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.451+0000 7fd7dd000700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd7c006dfa0 0x7fd7c0070450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4006010 tx=0x7fd7c400b580 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:49.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.454+0000 7fd7d6000700 1 -- 172.21.15.115:0/2887931672 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd7cc05adc0 con 0x7fd7d80fea30 2024-03-22T10:54:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.833+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7d8108780 con 0x7fd7c006dfa0 2024-03-22T10:54:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.835+0000 7fd7d6000700 1 -- 172.21.15.115:0/2887931672 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fd7d8108780 con 0x7fd7c006dfa0 2024-03-22T10:54:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.838+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd7c006dfa0 msgr2=0x7fd7c0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.838+0000 7fd7e0400700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd7c006dfa0 0x7fd7c0070450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4006010 tx=0x7fd7c400b580 comp rx=0 tx=0).stop 2024-03-22T10:54:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.838+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80fea30 msgr2=0x7fd7d8197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.838+0000 7fd7e0400700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80fea30 0x7fd7d8197b40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00d8d0 tx=0x7fd7cc00dc90 comp rx=0 tx=0).stop 2024-03-22T10:54:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.839+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 shutdown_connections 2024-03-22T10:54:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.839+0000 7fd7e0400700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd7c006dfa0 0x7fd7c0070450 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.839+0000 7fd7e0400700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7d80fea30 0x7fd7d8197b40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.839+0000 7fd7e0400700 1 --2- 172.21.15.115:0/2887931672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7d80ff380 0x7fd7d8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.839+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 >> 172.21.15.115:0/2887931672 conn(0x7fd7d80fa600 msgr2=0x7fd7d8107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.839+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 shutdown_connections 2024-03-22T10:54:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:49.839+0000 7fd7e0400700 1 -- 172.21.15.115:0/2887931672 wait complete. 2024-03-22T10:54:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:49 smithi115 ceph-mon[78672]: pgmap v360: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:50.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:54:50.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:49 smithi150 ceph-mon[65513]: pgmap v360: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:50.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:54:50.551 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch ps' 2024-03-22T10:54:50.762 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:50 smithi115 ceph-mon[78672]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:51.019 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:54:51.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:50 smithi150 ceph-mon[65513]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:51.887 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:51 smithi115 ceph-mon[78672]: pgmap v361: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:52.149 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:51 smithi150 ceph-mon[65513]: pgmap v361: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.147+0000 7fcc45c00700 1 -- 172.21.15.115:0/1018756172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102600 msgr2=0x7fcc401029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.147+0000 7fcc45c00700 1 --2- 172.21.15.115:0/1018756172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102600 0x7fcc401029d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009910 tx=0x7fcc30009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.147+0000 7fcc45c00700 1 -- 172.21.15.115:0/1018756172 shutdown_connections 2024-03-22T10:54:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.147+0000 7fcc45c00700 1 --2- 172.21.15.115:0/1018756172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40102fa0 0x7fcc40106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:53.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.147+0000 7fcc45c00700 1 --2- 172.21.15.115:0/1018756172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102600 0x7fcc401029d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.147+0000 7fcc45c00700 1 -- 172.21.15.115:0/1018756172 >> 172.21.15.115:0/1018756172 conn(0x7fcc400fdec0 msgr2=0x7fcc401002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.147+0000 7fcc45c00700 1 -- 172.21.15.115:0/1018756172 shutdown_connections 2024-03-22T10:54:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.147+0000 7fcc45c00700 1 -- 172.21.15.115:0/1018756172 wait complete. 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.148+0000 7fcc45c00700 1 Processor -- start 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.148+0000 7fcc45c00700 1 -- start start 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.149+0000 7fcc45c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40102600 0x7fcc40197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.149+0000 7fcc45c00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102fa0 0x7fcc401984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.149+0000 7fcc45c00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40198bd0 con 0x7fcc40102fa0 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.149+0000 7fcc45c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4019c960 con 0x7fcc40102600 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.149+0000 7fcc3e000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102fa0 0x7fcc401984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.149+0000 7fcc3e000700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102fa0 0x7fcc401984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32936/0 (socket says 172.21.15.115:32936) 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.149+0000 7fcc3e000700 1 -- 172.21.15.115:0/1678882936 learned_addr learned my addr 172.21.15.115:0/1678882936 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.150+0000 7fcc3ea00700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40102600 0x7fcc40197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:53.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.150+0000 7fcc3e000700 1 -- 172.21.15.115:0/1678882936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40102600 msgr2=0x7fcc40197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:53.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.150+0000 7fcc3e000700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40102600 0x7fcc40197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:53.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.150+0000 7fcc3e000700 1 -- 172.21.15.115:0/1678882936 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc300095f0 con 0x7fcc40102fa0 2024-03-22T10:54:53.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.150+0000 7fcc3e000700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102fa0 0x7fcc401984f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00d900 tx=0x7fcc2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.150+0000 7fcc37400700 1 -- 172.21.15.115:0/1678882936 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc2c009900 con 0x7fcc40102fa0 2024-03-22T10:54:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.150+0000 7fcc37400700 1 -- 172.21.15.115:0/1678882936 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc2c010460 con 0x7fcc40102fa0 2024-03-22T10:54:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.151+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4019cc40 con 0x7fcc40102fa0 2024-03-22T10:54:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.151+0000 7fcc37400700 1 -- 172.21.15.115:0/1678882936 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc2c00f5f0 con 0x7fcc40102fa0 2024-03-22T10:54:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.151+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4019d190 con 0x7fcc40102fa0 2024-03-22T10:54:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.152+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4004ea50 con 0x7fcc40102fa0 2024-03-22T10:54:53.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.154+0000 7fcc37400700 1 -- 172.21.15.115:0/1678882936 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fcc2c0105d0 con 0x7fcc40102fa0 2024-03-22T10:54:53.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.155+0000 7fcc37400700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc2406dfa0 0x7fcc24070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:53.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.155+0000 7fcc37400700 1 -- 172.21.15.115:0/1678882936 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcc2c08dcc0 con 0x7fcc40102fa0 2024-03-22T10:54:53.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.155+0000 7fcc3ea00700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc2406dfa0 0x7fcc24070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:53.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.156+0000 7fcc3ea00700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc2406dfa0 0x7fcc24070450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006010 tx=0x7fcc3001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:53.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.160+0000 7fcc37400700 1 -- 172.21.15.115:0/1678882936 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc2c093020 con 0x7fcc40102fa0 2024-03-22T10:54:53.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.440+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc4019d670 con 0x7fcc2406dfa0 2024-03-22T10:54:53.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.452+0000 7fcc37400700 1 -- 172.21.15.115:0/1678882936 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fcc4019d670 con 0x7fcc2406dfa0 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (79s) 34s ago 23m 14.6M - 0.25.0 c8568f914cd2 ff3d0ae1fb82 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (2m) 34s ago 23m 8174k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e11b0bdaf069 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (2m) 2m ago 22m 7662k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 ef9665eef7b1 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (9m) 34s ago 23m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (9m) 2m ago 22m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (41s) 34s ago 22m 52.6M - 9.4.12 a0ea5e3c46d9 a3b2e6b0b3f8 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (4m) 34s ago 16m 67.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 44a0769ebe8e 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (3m) 34s ago 16m 18.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51d700f6eb18 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (3m) 2m ago 16m 12.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 82b6b216f754 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (3m) 2m ago 16m 12.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9605445d5b2b 2024-03-22T10:54:53.445 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (12m) 34s ago 24m 517M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (11m) 2m ago 21m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (11m) 34s ago 25m 61.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (10m) 2m ago 21m 45.7M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (2m) 34s ago 23m 7608k - 1.5.0 0da6a335fe13 a98cb20e74e1 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (2m) 2m ago 22m 2935k - 1.5.0 0da6a335fe13 6c3ba9d80db0 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (8m) 34s ago 20m 98.2M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (7m) 34s ago 20m 96.9M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (7m) 34s ago 19m 83.9M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (7m) 2m ago 19m 103M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (6m) 2m ago 18m 91.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (5m) 2m ago 18m 90.9M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:54:53.446 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (102s) 34s ago 22m 66.9M - 2.43.0 a07b618ecd1d 8561aedaa7b5 2024-03-22T10:54:53.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc2406dfa0 msgr2=0x7fcc24070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:53.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc2406dfa0 0x7fcc24070450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006010 tx=0x7fcc3001a040 comp rx=0 tx=0).stop 2024-03-22T10:54:53.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102fa0 msgr2=0x7fcc401984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:53.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102fa0 0x7fcc401984f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00d900 tx=0x7fcc2c00dcc0 comp rx=0 tx=0).stop 2024-03-22T10:54:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 shutdown_connections 2024-03-22T10:54:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc2406dfa0 0x7fcc24070450 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40102600 0x7fcc40197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 --2- 172.21.15.115:0/1678882936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc40102fa0 0x7fcc401984f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 >> 172.21.15.115:0/1678882936 conn(0x7fcc400fdec0 msgr2=0x7fcc400ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 shutdown_connections 2024-03-22T10:54:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:53.455+0000 7fcc45c00700 1 -- 172.21.15.115:0/1678882936 wait complete. 2024-03-22T10:54:54.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:53 smithi115 ceph-mon[78672]: pgmap v362: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:54.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:53 smithi150 ceph-mon[65513]: pgmap v362: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:54.262 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch upgrade status' 2024-03-22T10:54:54.692 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:54:55.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:54 smithi115 ceph-mon[78672]: from='client.34690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:55.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:54 smithi150 ceph-mon[65513]: from='client.34690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:55.889 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:55 smithi115 ceph-mon[78672]: pgmap v363: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:56.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:55 smithi150 ceph-mon[65513]: pgmap v363: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:56.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.732+0000 7f980b200700 1 -- 172.21.15.115:0/738779488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 msgr2=0x7f980c106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:56.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.732+0000 7f980b200700 1 --2- 172.21.15.115:0/738779488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 0x7f980c106360 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009910 tx=0x7f97fc009c20 comp rx=0 tx=0).stop 2024-03-22T10:54:56.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.732+0000 7f980b200700 1 -- 172.21.15.115:0/738779488 shutdown_connections 2024-03-22T10:54:56.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.732+0000 7f980b200700 1 --2- 172.21.15.115:0/738779488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c0683d0 0x7f980c068840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:56.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.732+0000 7f980b200700 1 --2- 172.21.15.115:0/738779488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 0x7f980c106360 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:56.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.732+0000 7f980b200700 1 -- 172.21.15.115:0/738779488 >> 172.21.15.115:0/738779488 conn(0x7f980c074d00 msgr2=0x7f980c075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:56.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.732+0000 7f980b200700 1 -- 172.21.15.115:0/738779488 shutdown_connections 2024-03-22T10:54:56.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.732+0000 7f980b200700 1 -- 172.21.15.115:0/738779488 wait complete. 2024-03-22T10:54:56.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.733+0000 7f980b200700 1 Processor -- start 2024-03-22T10:54:56.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.734+0000 7f980b200700 1 -- start start 2024-03-22T10:54:56.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.734+0000 7f980b200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c0683d0 0x7f980c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:56.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.734+0000 7f980b200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 0x7f980c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:56.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.734+0000 7f980b200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c19c490 con 0x7f980c105f90 2024-03-22T10:54:56.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.734+0000 7f980b200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1959d0 con 0x7f980c0683d0 2024-03-22T10:54:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.735+0000 7f9809400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 0x7f980c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.735+0000 7f9809400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 0x7f980c19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32960/0 (socket says 172.21.15.115:32960) 2024-03-22T10:54:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.735+0000 7f9809400700 1 -- 172.21.15.115:0/2804004548 learned_addr learned my addr 172.21.15.115:0/2804004548 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:54:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.735+0000 7f9809e00700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c0683d0 0x7f980c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.735+0000 7f9809400700 1 -- 172.21.15.115:0/2804004548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c0683d0 msgr2=0x7f980c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.735+0000 7f9809400700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c0683d0 0x7f980c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.735+0000 7f9809400700 1 -- 172.21.15.115:0/2804004548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0095f0 con 0x7f980c105f90 2024-03-22T10:54:56.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.736+0000 7f9809400700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 0x7f980c19be90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f97f800c930 tx=0x7f97f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:56.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.736+0000 7f9802a00700 1 -- 172.21.15.115:0/2804004548 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97f8007ad0 con 0x7f980c105f90 2024-03-22T10:54:56.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.736+0000 7f9802a00700 1 -- 172.21.15.115:0/2804004548 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97f800ce80 con 0x7f980c105f90 2024-03-22T10:54:56.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.736+0000 7f9802a00700 1 -- 172.21.15.115:0/2804004548 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97f8018630 con 0x7f980c105f90 2024-03-22T10:54:56.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.736+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c195cb0 con 0x7f980c105f90 2024-03-22T10:54:56.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.736+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c196200 con 0x7f980c105f90 2024-03-22T10:54:56.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.737+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c108470 con 0x7f980c105f90 2024-03-22T10:54:56.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.740+0000 7f9802a00700 1 -- 172.21.15.115:0/2804004548 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f97f8007c30 con 0x7f980c105f90 2024-03-22T10:54:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.741+0000 7f9802a00700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f97f006dfa0 0x7f97f0070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:54:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.741+0000 7f9802a00700 1 -- 172.21.15.115:0/2804004548 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f97f808de00 con 0x7f980c105f90 2024-03-22T10:54:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.741+0000 7f9809e00700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f97f006dfa0 0x7f97f0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:54:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.742+0000 7f9809e00700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f97f006dfa0 0x7f97f0070450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006010 tx=0x7f97fc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:54:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.746+0000 7f9802a00700 1 -- 172.21.15.115:0/2804004548 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f97f80c0740 con 0x7f980c105f90 2024-03-22T10:54:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.990+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f980c196530 con 0x7f97f006dfa0 2024-03-22T10:54:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.992+0000 7f9802a00700 1 -- 172.21.15.115:0/2804004548 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f980c196530 con 0x7f97f006dfa0 2024-03-22T10:54:56.985 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:54:56.985 INFO:teuthology.orchestra.run.smithi115.stdout: "target_image": null, 2024-03-22T10:54:56.985 INFO:teuthology.orchestra.run.smithi115.stdout: "in_progress": false, 2024-03-22T10:54:56.985 INFO:teuthology.orchestra.run.smithi115.stdout: "which": "", 2024-03-22T10:54:56.985 INFO:teuthology.orchestra.run.smithi115.stdout: "services_complete": [], 2024-03-22T10:54:56.986 INFO:teuthology.orchestra.run.smithi115.stdout: "progress": null, 2024-03-22T10:54:56.986 INFO:teuthology.orchestra.run.smithi115.stdout: "message": "", 2024-03-22T10:54:56.986 INFO:teuthology.orchestra.run.smithi115.stdout: "is_paused": false 2024-03-22T10:54:56.986 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:54:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.996+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f97f006dfa0 msgr2=0x7f97f0070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.996+0000 7f980b200700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f97f006dfa0 0x7f97f0070450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006010 tx=0x7f97fc01a040 comp rx=0 tx=0).stop 2024-03-22T10:54:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.996+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 msgr2=0x7f980c19be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:54:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.996+0000 7f980b200700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 0x7f980c19be90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f97f800c930 tx=0x7f97f800cc40 comp rx=0 tx=0).stop 2024-03-22T10:54:56.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.997+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 shutdown_connections 2024-03-22T10:54:56.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.997+0000 7f980b200700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f97f006dfa0 0x7f97f0070450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:56.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.997+0000 7f980b200700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c0683d0 0x7f980c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:56.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.997+0000 7f980b200700 1 --2- 172.21.15.115:0/2804004548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f980c105f90 0x7f980c19be90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:54:56.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.997+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 >> 172.21.15.115:0/2804004548 conn(0x7f980c074d00 msgr2=0x7f980c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:54:56.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.997+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 shutdown_connections 2024-03-22T10:54:56.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:54:56.997+0000 7f980b200700 1 -- 172.21.15.115:0/2804004548 wait complete. 2024-03-22T10:54:57.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph health detail' 2024-03-22T10:54:58.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:57 smithi115 ceph-mon[78672]: pgmap v364: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:58.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:57 smithi115 ceph-mon[78672]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:54:58.121 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:54:58.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:57 smithi150 ceph-mon[65513]: pgmap v364: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:54:58.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:57 smithi150 ceph-mon[65513]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:55:00.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:54:59 smithi115 ceph-mon[78672]: pgmap v365: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:00.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:54:59 smithi150 ceph-mon[65513]: pgmap v365: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:00.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.190+0000 7f95a7e00700 1 -- 172.21.15.115:0/654649397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 msgr2=0x7f95a810c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:55:00.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.190+0000 7f95a7e00700 1 --2- 172.21.15.115:0/654649397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 0x7f95a810c5e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9590009910 tx=0x7f9590009c20 comp rx=0 tx=0).stop 2024-03-22T10:55:00.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.191+0000 7f95a7e00700 1 -- 172.21.15.115:0/654649397 shutdown_connections 2024-03-22T10:55:00.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.191+0000 7f95a7e00700 1 --2- 172.21.15.115:0/654649397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 0x7f95a810c5e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:00.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.191+0000 7f95a7e00700 1 --2- 172.21.15.115:0/654649397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95a80683d0 0x7f95a80687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:00.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.191+0000 7f95a7e00700 1 -- 172.21.15.115:0/654649397 >> 172.21.15.115:0/654649397 conn(0x7f95a8074d00 msgr2=0x7f95a8075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:55:00.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.191+0000 7f95a7e00700 1 -- 172.21.15.115:0/654649397 shutdown_connections 2024-03-22T10:55:00.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.191+0000 7f95a7e00700 1 -- 172.21.15.115:0/654649397 wait complete. 2024-03-22T10:55:00.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.192+0000 7f95a7e00700 1 Processor -- start 2024-03-22T10:55:00.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.192+0000 7f95a7e00700 1 -- start start 2024-03-22T10:55:00.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.193+0000 7f95a7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95a80683d0 0x7f95a819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:55:00.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.193+0000 7f95a7e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 0x7f95a819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:55:00.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.193+0000 7f95a7e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a819e600 con 0x7f95a8068ce0 2024-03-22T10:55:00.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.193+0000 7f95a7e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a8197b40 con 0x7f95a80683d0 2024-03-22T10:55:00.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.193+0000 7f959fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 0x7f95a819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:55:00.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.193+0000 7f959fe00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 0x7f95a819e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32970/0 (socket says 172.21.15.115:32970) 2024-03-22T10:55:00.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.193+0000 7f959fe00700 1 -- 172.21.15.115:0/1390920757 learned_addr learned my addr 172.21.15.115:0/1390920757 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.194+0000 7f95a6a00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95a80683d0 0x7f95a819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.194+0000 7f959fe00700 1 -- 172.21.15.115:0/1390920757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95a80683d0 msgr2=0x7f95a819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.194+0000 7f959fe00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95a80683d0 0x7f95a819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.194+0000 7f959fe00700 1 -- 172.21.15.115:0/1390920757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900095f0 con 0x7f95a8068ce0 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.194+0000 7f959fe00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 0x7f95a819e000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9590006010 tx=0x7f959000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.195+0000 7f959f400700 1 -- 172.21.15.115:0/1390920757 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f959001d070 con 0x7f95a8068ce0 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.195+0000 7f959f400700 1 -- 172.21.15.115:0/1390920757 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9590004fb0 con 0x7f95a8068ce0 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.195+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a8197e20 con 0x7f95a8068ce0 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.195+0000 7f959f400700 1 -- 172.21.15.115:0/1390920757 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f959000f690 con 0x7f95a8068ce0 2024-03-22T10:55:00.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.195+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a8198370 con 0x7f95a8068ce0 2024-03-22T10:55:00.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.196+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a8109d50 con 0x7f95a8068ce0 2024-03-22T10:55:00.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.197+0000 7f959f400700 1 -- 172.21.15.115:0/1390920757 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f9590004af0 con 0x7f95a8068ce0 2024-03-22T10:55:00.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.198+0000 7f959f400700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f958c06dfa0 0x7f958c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:55:00.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.198+0000 7f959f400700 1 -- 172.21.15.115:0/1390920757 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f959008f600 con 0x7f95a8068ce0 2024-03-22T10:55:00.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.199+0000 7f95a6a00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f958c06dfa0 0x7f958c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:55:00.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.199+0000 7f95a6a00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f958c06dfa0 0x7f958c070450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9598005ea0 tx=0x7f9598005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:55:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.208+0000 7f959f400700 1 -- 172.21.15.115:0/1390920757 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f959005c500 con 0x7f95a8068ce0 2024-03-22T10:55:00.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.537+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f95a81990d0 con 0x7f95a8068ce0 2024-03-22T10:55:00.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.538+0000 7f959f400700 1 -- 172.21.15.115:0/1390920757 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f95900270e0 con 0x7f95a8068ce0 2024-03-22T10:55:00.531 INFO:teuthology.orchestra.run.smithi115.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T10:55:00.531 INFO:teuthology.orchestra.run.smithi115.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-22T10:55:00.531 INFO:teuthology.orchestra.run.smithi115.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-22T10:55:00.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f958c06dfa0 msgr2=0x7f958c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:55:00.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f958c06dfa0 0x7f958c070450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9598005ea0 tx=0x7f9598005e30 comp rx=0 tx=0).stop 2024-03-22T10:55:00.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 msgr2=0x7f95a819e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:55:00.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 0x7f95a819e000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9590006010 tx=0x7f959000be20 comp rx=0 tx=0).stop 2024-03-22T10:55:00.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 shutdown_connections 2024-03-22T10:55:00.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f958c06dfa0 0x7f958c070450 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:00.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95a80683d0 0x7f95a819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:00.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 --2- 172.21.15.115:0/1390920757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a8068ce0 0x7f95a819e000 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:00.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 >> 172.21.15.115:0/1390920757 conn(0x7f95a8074d00 msgr2=0x7f95a80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:55:00.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 shutdown_connections 2024-03-22T10:55:00.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:00.541+0000 7f95a7e00700 1 -- 172.21.15.115:0/1390920757 wait complete. 2024-03-22T10:55:00.937 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:00 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1390920757' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:55:01.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:00 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1390920757' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T10:55:01.384 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions' 2024-03-22T10:55:01.821 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:55:02.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:01 smithi115 ceph-mon[78672]: pgmap v366: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:02.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:01 smithi150 ceph-mon[65513]: pgmap v366: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:03.879 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:03 smithi115 ceph-mon[78672]: pgmap v367: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:04.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.069+0000 7fab2d200700 1 -- 172.21.15.115:0/3117817267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 msgr2=0x7fab281015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:55:04.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.069+0000 7fab2d200700 1 --2- 172.21.15.115:0/3117817267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 0x7fab281015c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fab18009910 tx=0x7fab18009c20 comp rx=0 tx=0).stop 2024-03-22T10:55:04.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.069+0000 7fab2d200700 1 -- 172.21.15.115:0/3117817267 shutdown_connections 2024-03-22T10:55:04.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.069+0000 7fab2d200700 1 --2- 172.21.15.115:0/3117817267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28101b00 0x7fab2810a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:04.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.069+0000 7fab2d200700 1 --2- 172.21.15.115:0/3117817267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 0x7fab281015c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:04.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.069+0000 7fab2d200700 1 -- 172.21.15.115:0/3117817267 >> 172.21.15.115:0/3117817267 conn(0x7fab280fab60 msgr2=0x7fab280fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:55:04.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.069+0000 7fab2d200700 1 -- 172.21.15.115:0/3117817267 shutdown_connections 2024-03-22T10:55:04.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.070+0000 7fab2d200700 1 -- 172.21.15.115:0/3117817267 wait complete. 2024-03-22T10:55:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.071+0000 7fab2d200700 1 Processor -- start 2024-03-22T10:55:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.071+0000 7fab2d200700 1 -- start start 2024-03-22T10:55:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.072+0000 7fab2d200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 0x7fab28198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:55:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.072+0000 7fab2d200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28101b00 0x7fab28198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:55:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.072+0000 7fab2d200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab28198c30 con 0x7fab281011f0 2024-03-22T10:55:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.072+0000 7fab2d200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2819c9c0 con 0x7fab28101b00 2024-03-22T10:55:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.072+0000 7fab25e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 0x7fab28198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:55:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.072+0000 7fab25e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 0x7fab28198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40098/0 (socket says 172.21.15.115:40098) 2024-03-22T10:55:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.072+0000 7fab25e00700 1 -- 172.21.15.115:0/2716812474 learned_addr learned my addr 172.21.15.115:0/2716812474 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:55:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab25400700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28101b00 0x7fab28198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:55:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab25e00700 1 -- 172.21.15.115:0/2716812474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28101b00 msgr2=0x7fab28198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:55:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab25e00700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28101b00 0x7fab28198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab25e00700 1 -- 172.21.15.115:0/2716812474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180095f0 con 0x7fab281011f0 2024-03-22T10:55:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab25400700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28101b00 0x7fab28198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:55:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab25e00700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 0x7fab28198010 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fab18006010 tx=0x7fab1800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:55:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab1ea00700 1 -- 172.21.15.115:0/2716812474 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab1801d070 con 0x7fab281011f0 2024-03-22T10:55:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab1ea00700 1 -- 172.21.15.115:0/2716812474 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab18004fb0 con 0x7fab281011f0 2024-03-22T10:55:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab1ea00700 1 -- 172.21.15.115:0/2716812474 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab1800f690 con 0x7fab281011f0 2024-03-22T10:55:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.073+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2819cc40 con 0x7fab281011f0 2024-03-22T10:55:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.074+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2819d130 con 0x7fab281011f0 2024-03-22T10:55:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.075+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab28109cd0 con 0x7fab281011f0 2024-03-22T10:55:04.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.076+0000 7fab1ea00700 1 -- 172.21.15.115:0/2716812474 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fab180228a0 con 0x7fab281011f0 2024-03-22T10:55:04.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.078+0000 7fab1ea00700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab0c06dfa0 0x7fab0c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:55:04.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.078+0000 7fab1ea00700 1 -- 172.21.15.115:0/2716812474 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fab1808f650 con 0x7fab281011f0 2024-03-22T10:55:04.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.078+0000 7fab25400700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab0c06dfa0 0x7fab0c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:55:04.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.079+0000 7fab25400700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab0c06dfa0 0x7fab0c070450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab28199630 tx=0x7fab14008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:55:04.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.082+0000 7fab1ea00700 1 -- 172.21.15.115:0/2716812474 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab18058a20 con 0x7fab281011f0 2024-03-22T10:55:04.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:03 smithi150 ceph-mon[65513]: pgmap v367: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:04.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.592+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab2804ea50 con 0x7fab281011f0 2024-03-22T10:55:04.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.593+0000 7fab1ea00700 1 -- 172.21.15.115:0/2716812474 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fab1805c0e0 con 0x7fab281011f0 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:55:04.586 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:55:04.587 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:55:04.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:55:04.587 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:55:04.587 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-22T10:55:04.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:55:04.587 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab0c06dfa0 msgr2=0x7fab0c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab0c06dfa0 0x7fab0c070450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab28199630 tx=0x7fab14008040 comp rx=0 tx=0).stop 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 msgr2=0x7fab28198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 0x7fab28198010 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fab18006010 tx=0x7fab1800be20 comp rx=0 tx=0).stop 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 shutdown_connections 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab0c06dfa0 0x7fab0c070450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab281011f0 0x7fab28198010 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 --2- 172.21.15.115:0/2716812474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28101b00 0x7fab28198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 >> 172.21.15.115:0/2716812474 conn(0x7fab280fab60 msgr2=0x7fab280ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:55:04.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 shutdown_connections 2024-03-22T10:55:04.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:55:04.598+0000 7fab2d200700 1 -- 172.21.15.115:0/2716812474 wait complete. 2024-03-22T10:55:04.978 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:55:04.978 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:04 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2716812474' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:55:05.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:55:05.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:04 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2716812474' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:55:05.406 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-22T10:55:05.835 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:55:06.119 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:05 smithi115 ceph-mon[78672]: pgmap v368: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:06.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:05 smithi150 ceph-mon[65513]: pgmap v368: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:07.893 INFO:teuthology.orchestra.run.smithi115.stdout:wait for servicemap items w/ changing names to refresh 2024-03-22T10:55:08.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:07 smithi115 ceph-mon[78672]: pgmap v369: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:08.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:07 smithi150 ceph-mon[65513]: pgmap v369: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:08.699 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'sleep 60' 2024-03-22T10:55:09.168 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:55:09.900 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:09 smithi150 ceph-mon[65513]: pgmap v370: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:10.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:09 smithi115 ceph-mon[78672]: pgmap v370: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:12.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:11 smithi150 ceph-mon[65513]: pgmap v371: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:12.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:11 smithi115 ceph-mon[78672]: pgmap v371: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:14.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:13 smithi150 ceph-mon[65513]: pgmap v372: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:14.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:13 smithi115 ceph-mon[78672]: pgmap v372: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:16.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:15 smithi115 ceph-mon[78672]: pgmap v373: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:16.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:15 smithi150 ceph-mon[65513]: pgmap v373: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:18.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:17 smithi150 ceph-mon[65513]: pgmap v374: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:18.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:17 smithi115 ceph-mon[78672]: pgmap v374: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:20.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:19 smithi150 ceph-mon[65513]: pgmap v375: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:20.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:55:20.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:19 smithi115 ceph-mon[78672]: pgmap v375: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:20.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:55:22.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:21 smithi115 ceph-mon[78672]: pgmap v376: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:22.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:21 smithi150 ceph-mon[65513]: pgmap v376: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:24.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:23 smithi150 ceph-mon[65513]: pgmap v377: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:24.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:23 smithi115 ceph-mon[78672]: pgmap v377: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:26.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:25 smithi150 ceph-mon[65513]: pgmap v378: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:26.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:25 smithi115 ceph-mon[78672]: pgmap v378: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:28.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:27 smithi150 ceph-mon[65513]: pgmap v379: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:28.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:27 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:55:28.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:27 smithi115 ceph-mon[78672]: pgmap v379: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:28.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:27 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:55:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:29 smithi150 ceph-mon[65513]: pgmap v380: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:55:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:55:30.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:55:30.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:29 smithi115 ceph-mon[78672]: pgmap v380: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:30.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:55:30.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:55:30.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:55:32.118 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:31 smithi115 ceph-mon[78672]: pgmap v381: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:32.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:31 smithi150 ceph-mon[65513]: pgmap v381: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:34.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:33 smithi150 ceph-mon[65513]: pgmap v382: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:34.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:33 smithi115 ceph-mon[78672]: pgmap v382: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:35.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:55:35.262 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:55:36.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:35 smithi150 ceph-mon[65513]: pgmap v383: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:36.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:35 smithi115 ceph-mon[78672]: pgmap v383: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:38.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:37 smithi150 ceph-mon[65513]: pgmap v384: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:38.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:37 smithi115 ceph-mon[78672]: pgmap v384: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:40.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:39 smithi150 ceph-mon[65513]: pgmap v385: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:40.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:39 smithi115 ceph-mon[78672]: pgmap v385: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:42.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:41 smithi150 ceph-mon[65513]: pgmap v386: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:42.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:41 smithi115 ceph-mon[78672]: pgmap v386: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:44.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:43 smithi150 ceph-mon[65513]: pgmap v387: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:44.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:43 smithi115 ceph-mon[78672]: pgmap v387: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:46.150 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:45 smithi150 ceph-mon[65513]: pgmap v388: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:46.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:45 smithi115 ceph-mon[78672]: pgmap v388: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:48.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:47 smithi115 ceph-mon[78672]: pgmap v389: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:48.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:47 smithi150 ceph-mon[65513]: pgmap v389: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:50.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:49 smithi115 ceph-mon[78672]: pgmap v390: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:50.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:55:50.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:49 smithi150 ceph-mon[65513]: pgmap v390: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:50.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:55:52.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:51 smithi115 ceph-mon[78672]: pgmap v391: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:52.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:51 smithi150 ceph-mon[65513]: pgmap v391: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:54.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:53 smithi115 ceph-mon[78672]: pgmap v392: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:54.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:53 smithi150 ceph-mon[65513]: pgmap v392: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:56.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:55 smithi115 ceph-mon[78672]: pgmap v393: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:56.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:55 smithi150 ceph-mon[65513]: pgmap v393: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:58.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:57 smithi115 ceph-mon[78672]: pgmap v394: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:55:58.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:57 smithi150 ceph-mon[65513]: pgmap v394: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:00.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:55:59 smithi115 ceph-mon[78672]: pgmap v395: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:00.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:55:59 smithi150 ceph-mon[65513]: pgmap v395: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:02.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:01 smithi115 ceph-mon[78672]: pgmap v396: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:02.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:01 smithi150 ceph-mon[65513]: pgmap v396: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:04.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:03 smithi115 ceph-mon[78672]: pgmap v397: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:04.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:03 smithi150 ceph-mon[65513]: pgmap v397: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:05.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:56:05.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:56:06.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:05 smithi115 ceph-mon[78672]: pgmap v398: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:06.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:05 smithi150 ceph-mon[65513]: pgmap v398: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:08.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:07 smithi115 ceph-mon[78672]: pgmap v399: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:08.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:07 smithi150 ceph-mon[65513]: pgmap v399: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:10.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:09 smithi115 ceph-mon[78672]: pgmap v400: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:10.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:10 smithi150 ceph-mon[65513]: pgmap v400: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:12.059 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch ps' 2024-03-22T10:56:12.146 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:12 smithi115 ceph-mon[78672]: pgmap v401: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:12.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:12 smithi150 ceph-mon[65513]: pgmap v401: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:12.500 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:56:14.156 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:14 smithi115 ceph-mon[78672]: pgmap v402: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:14.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:14 smithi150 ceph-mon[65513]: pgmap v402: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:14.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.677+0000 7f7bad400700 1 -- 172.21.15.115:0/3388864108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100200 msgr2=0x7f7ba81005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:14.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.677+0000 7f7bad400700 1 --2- 172.21.15.115:0/3388864108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100200 0x7f7ba81005d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009910 tx=0x7f7b98009c20 comp rx=0 tx=0).stop 2024-03-22T10:56:14.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.677+0000 7f7bad400700 1 -- 172.21.15.115:0/3388864108 shutdown_connections 2024-03-22T10:56:14.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.677+0000 7f7bad400700 1 --2- 172.21.15.115:0/3388864108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8100ba0 0x7f7ba8104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:14.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.677+0000 7f7bad400700 1 --2- 172.21.15.115:0/3388864108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100200 0x7f7ba81005d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:14.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.677+0000 7f7bad400700 1 -- 172.21.15.115:0/3388864108 >> 172.21.15.115:0/3388864108 conn(0x7f7ba80fba80 msgr2=0x7f7ba80fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:14.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.677+0000 7f7bad400700 1 -- 172.21.15.115:0/3388864108 shutdown_connections 2024-03-22T10:56:14.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.677+0000 7f7bad400700 1 -- 172.21.15.115:0/3388864108 wait complete. 2024-03-22T10:56:14.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.679+0000 7f7bad400700 1 Processor -- start 2024-03-22T10:56:14.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.679+0000 7f7bad400700 1 -- start start 2024-03-22T10:56:14.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.679+0000 7f7bad400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8100200 0x7f7ba819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:14.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.680+0000 7f7bad400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100ba0 0x7f7ba819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:14.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.680+0000 7f7bad400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba819c5a0 con 0x7f7ba8100ba0 2024-03-22T10:56:14.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.680+0000 7f7bad400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba8195a50 con 0x7f7ba8100200 2024-03-22T10:56:14.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.680+0000 7f7ba7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100ba0 0x7f7ba819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:14.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.680+0000 7f7ba7400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100ba0 0x7f7ba819bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57756/0 (socket says 172.21.15.115:57756) 2024-03-22T10:56:14.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.680+0000 7f7ba7400700 1 -- 172.21.15.115:0/1262315908 learned_addr learned my addr 172.21.15.115:0/1262315908 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:14.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.680+0000 7f7ba7e00700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8100200 0x7f7ba819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:14.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.681+0000 7f7ba7400700 1 -- 172.21.15.115:0/1262315908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8100200 msgr2=0x7f7ba819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:14.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.681+0000 7f7ba7400700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8100200 0x7f7ba819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:14.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.681+0000 7f7ba7400700 1 -- 172.21.15.115:0/1262315908 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980095f0 con 0x7f7ba8100ba0 2024-03-22T10:56:14.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.681+0000 7f7ba7400700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100ba0 0x7f7ba819bf10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c8f0 tx=0x7f7b9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:14.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.682+0000 7f7ba4c00700 1 -- 172.21.15.115:0/1262315908 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b94007a50 con 0x7f7ba8100ba0 2024-03-22T10:56:14.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.682+0000 7f7ba4c00700 1 -- 172.21.15.115:0/1262315908 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b94007bb0 con 0x7f7ba8100ba0 2024-03-22T10:56:14.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.682+0000 7f7ba4c00700 1 -- 172.21.15.115:0/1262315908 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b94018630 con 0x7f7ba8100ba0 2024-03-22T10:56:14.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.682+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba8195d30 con 0x7f7ba8100ba0 2024-03-22T10:56:14.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.682+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba8196280 con 0x7f7ba8100ba0 2024-03-22T10:56:14.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.683+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba8108510 con 0x7f7ba8100ba0 2024-03-22T10:56:14.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.686+0000 7f7ba4c00700 1 -- 172.21.15.115:0/1262315908 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f7b9400f9a0 con 0x7f7ba8100ba0 2024-03-22T10:56:14.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.687+0000 7f7ba4c00700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7b8c06dfa0 0x7f7b8c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:14.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.687+0000 7f7ba4c00700 1 -- 172.21.15.115:0/1262315908 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f7b9408e050 con 0x7f7ba8100ba0 2024-03-22T10:56:14.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.688+0000 7f7ba7e00700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7b8c06dfa0 0x7f7b8c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:14.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.688+0000 7f7ba7e00700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7b8c06dfa0 0x7f7b8c070450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7b98006010 tx=0x7f7b9801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:14.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.691+0000 7f7ba4c00700 1 -- 172.21.15.115:0/1262315908 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b9405b080 con 0x7f7ba8100ba0 2024-03-22T10:56:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.930+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 --> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ba81965c0 con 0x7f7b8c06dfa0 2024-03-22T10:56:14.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.957+0000 7f7ba4c00700 1 -- 172.21.15.115:0/1262315908 <== mgr.34102 v2:172.21.15.115:6800/3030573715 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7ba81965c0 con 0x7f7b8c06dfa0 2024-03-22T10:56:14.952 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T10:56:14.952 INFO:teuthology.orchestra.run.smithi115.stdout:alertmanager.smithi115 smithi115 *:9093,9094 running (2m) 115s ago 24m 14.6M - 0.25.0 c8568f914cd2 ff3d0ae1fb82 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi115 smithi115 running (4m) 115s ago 24m 8174k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e11b0bdaf069 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-exporter.smithi150 smithi150 running (3m) 3m ago 23m 7662k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 ef9665eef7b1 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi115 smithi115 running (10m) 115s ago 24m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2b394e46f2d1 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:crash.smithi150 smithi150 running (10m) 3m ago 23m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5de7c8db23aa 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:grafana.smithi115 smithi115 *:3000 running (2m) 115s ago 24m 52.6M - 9.4.12 a0ea5e3c46d9 a3b2e6b0b3f8 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.ggxfzi smithi115 running (5m) 115s ago 17m 67.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 44a0769ebe8e 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi115.wjotmn smithi115 running (5m) 115s ago 17m 18.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51d700f6eb18 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.slzwwb smithi150 running (4m) 3m ago 17m 12.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 82b6b216f754 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:mds.cephfs.smithi150.zvyrzc smithi150 running (4m) 3m ago 17m 12.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9605445d5b2b 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi115.iqzwsz smithi115 *:8443,9283,8765 running (14m) 115s ago 26m 517M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5cb4cdfcf028 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.smithi150.fmoumv smithi150 *:8443,9283,8765 running (12m) 3m ago 23m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 51357739918f 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi115 smithi115 running (12m) 115s ago 26m 61.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 52dfd9dbb5ad 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:mon.smithi150 smithi150 running (11m) 3m ago 23m 45.7M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f35dc3320e85 2024-03-22T10:56:14.953 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi115 smithi115 *:9100 running (3m) 115s ago 24m 7608k - 1.5.0 0da6a335fe13 a98cb20e74e1 2024-03-22T10:56:14.954 INFO:teuthology.orchestra.run.smithi115.stdout:node-exporter.smithi150 smithi150 *:9100 running (3m) 3m ago 23m 2935k - 1.5.0 0da6a335fe13 6c3ba9d80db0 2024-03-22T10:56:14.954 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (10m) 115s ago 22m 98.2M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f8691567d116 2024-03-22T10:56:14.954 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi115 running (9m) 115s ago 21m 96.9M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 4e693719f2d7 2024-03-22T10:56:14.954 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi115 running (8m) 115s ago 21m 83.9M 1217M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2e695f9baca8 2024-03-22T10:56:14.954 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi150 running (8m) 3m ago 20m 103M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e01353665cf3 2024-03-22T10:56:14.954 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi150 running (7m) 3m ago 19m 91.0M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c7c7943cac44 2024-03-22T10:56:14.954 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi150 running (6m) 3m ago 19m 90.9M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 956493e42188 2024-03-22T10:56:14.954 INFO:teuthology.orchestra.run.smithi115.stdout:prometheus.smithi115 smithi115 *:9095 running (3m) 115s ago 23m 66.9M - 2.43.0 a07b618ecd1d 8561aedaa7b5 2024-03-22T10:56:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.961+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7b8c06dfa0 msgr2=0x7f7b8c070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.961+0000 7f7bad400700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7b8c06dfa0 0x7f7b8c070450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7b98006010 tx=0x7f7b9801a040 comp rx=0 tx=0).stop 2024-03-22T10:56:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.961+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100ba0 msgr2=0x7f7ba819bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.961+0000 7f7bad400700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100ba0 0x7f7ba819bf10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c8f0 tx=0x7f7b9400ccb0 comp rx=0 tx=0).stop 2024-03-22T10:56:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.961+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 shutdown_connections 2024-03-22T10:56:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.961+0000 7f7bad400700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7b8c06dfa0 0x7f7b8c070450 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.961+0000 7f7bad400700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8100200 0x7f7ba819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.961+0000 7f7bad400700 1 --2- 172.21.15.115:0/1262315908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ba8100ba0 0x7f7ba819bf10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.961+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 >> 172.21.15.115:0/1262315908 conn(0x7f7ba80fba80 msgr2=0x7f7ba80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.962+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 shutdown_connections 2024-03-22T10:56:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:14.962+0000 7f7bad400700 1 -- 172.21.15.115:0/1262315908 wait complete. 2024-03-22T10:56:15.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions' 2024-03-22T10:56:16.203 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:56:16.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:16 smithi115 ceph-mon[78672]: pgmap v403: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:16.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:16 smithi115 ceph-mon[78672]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:56:16.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:16 smithi150 ceph-mon[65513]: pgmap v403: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:16.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:16 smithi150 ceph-mon[65513]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T10:56:18.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.259+0000 7f7090a00700 1 -- 172.21.15.115:0/2555841099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 msgr2=0x7f708c110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:18.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.259+0000 7f7090a00700 1 --2- 172.21.15.115:0/2555841099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 0x7f708c110a80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7078009910 tx=0x7f7078009c20 comp rx=0 tx=0).stop 2024-03-22T10:56:18.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.259+0000 7f7090a00700 1 -- 172.21.15.115:0/2555841099 shutdown_connections 2024-03-22T10:56:18.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.259+0000 7f7090a00700 1 --2- 172.21.15.115:0/2555841099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 0x7f708c110a80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:18.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.259+0000 7f7090a00700 1 --2- 172.21.15.115:0/2555841099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c0730f0 0x7f708c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:18.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.259+0000 7f7090a00700 1 -- 172.21.15.115:0/2555841099 >> 172.21.15.115:0/2555841099 conn(0x7f708c0fbab0 msgr2=0x7f708c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:18.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.259+0000 7f7090a00700 1 -- 172.21.15.115:0/2555841099 shutdown_connections 2024-03-22T10:56:18.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.259+0000 7f7090a00700 1 -- 172.21.15.115:0/2555841099 wait complete. 2024-03-22T10:56:18.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.260+0000 7f7090a00700 1 Processor -- start 2024-03-22T10:56:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.260+0000 7f7090a00700 1 -- start start 2024-03-22T10:56:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.261+0000 7f7090a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c0730f0 0x7f708c19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.261+0000 7f7090a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 0x7f708c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.261+0000 7f7090a00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c19cd00 con 0x7f708c073a00 2024-03-22T10:56:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.261+0000 7f7090a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1a0a90 con 0x7f708c0730f0 2024-03-22T10:56:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.261+0000 7f708aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 0x7f708c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.261+0000 7f708aa00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 0x7f708c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57764/0 (socket says 172.21.15.115:57764) 2024-03-22T10:56:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.261+0000 7f708aa00700 1 -- 172.21.15.115:0/1241228749 learned_addr learned my addr 172.21.15.115:0/1241228749 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.261+0000 7f708b400700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c0730f0 0x7f708c19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.262+0000 7f708aa00700 1 -- 172.21.15.115:0/1241228749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c0730f0 msgr2=0x7f708c19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.262+0000 7f708aa00700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c0730f0 0x7f708c19c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.262+0000 7f708aa00700 1 -- 172.21.15.115:0/1241228749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780095f0 con 0x7f708c073a00 2024-03-22T10:56:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.262+0000 7f708aa00700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 0x7f708c19c620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f707800b5c0 tx=0x7f707800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.262+0000 7f7083e00700 1 -- 172.21.15.115:0/1241228749 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707801d070 con 0x7f708c073a00 2024-03-22T10:56:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.262+0000 7f7083e00700 1 -- 172.21.15.115:0/1241228749 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7078005e00 con 0x7f708c073a00 2024-03-22T10:56:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.262+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c1a0d70 con 0x7f708c073a00 2024-03-22T10:56:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.262+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c1a12c0 con 0x7f708c073a00 2024-03-22T10:56:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.263+0000 7f7083e00700 1 -- 172.21.15.115:0/1241228749 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7078017400 con 0x7f708c073a00 2024-03-22T10:56:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.264+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706c005320 con 0x7f708c073a00 2024-03-22T10:56:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.266+0000 7f7083e00700 1 -- 172.21.15.115:0/1241228749 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f7078003a60 con 0x7f708c073a00 2024-03-22T10:56:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.267+0000 7f7083e00700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f707006dfa0 0x7f7070070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.267+0000 7f7083e00700 1 -- 172.21.15.115:0/1241228749 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f707808f190 con 0x7f708c073a00 2024-03-22T10:56:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.267+0000 7f708b400700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f707006dfa0 0x7f7070070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.268+0000 7f708b400700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f707006dfa0 0x7f7070070450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f707c005950 tx=0x7f707c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.275+0000 7f7083e00700 1 -- 172.21.15.115:0/1241228749 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7078058560 con 0x7f708c073a00 2024-03-22T10:56:18.270 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:18 smithi115 ceph-mon[78672]: pgmap v404: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:18.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:18 smithi150 ceph-mon[65513]: pgmap v404: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.758+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f706c006200 con 0x7f708c073a00 2024-03-22T10:56:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.759+0000 7f7083e00700 1 -- 172.21.15.115:0/1241228749 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f707805bc20 con 0x7f708c073a00 2024-03-22T10:56:18.754 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-22T10:56:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": { 2024-03-22T10:56:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:56:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:56:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": { 2024-03-22T10:56:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-22T10:56:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": { 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "overall": { 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-22T10:56:18.755 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-22T10:56:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.763+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f707006dfa0 msgr2=0x7f7070070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.763+0000 7f7090a00700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f707006dfa0 0x7f7070070450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f707c005950 tx=0x7f707c0058e0 comp rx=0 tx=0).stop 2024-03-22T10:56:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.763+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 msgr2=0x7f708c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.763+0000 7f7090a00700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 0x7f708c19c620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f707800b5c0 tx=0x7f707800be20 comp rx=0 tx=0).stop 2024-03-22T10:56:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.763+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 shutdown_connections 2024-03-22T10:56:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.763+0000 7f7090a00700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f707006dfa0 0x7f7070070450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.763+0000 7f7090a00700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f708c0730f0 0x7f708c19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.763+0000 7f7090a00700 1 --2- 172.21.15.115:0/1241228749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f708c073a00 0x7f708c19c620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.763+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 >> 172.21.15.115:0/1241228749 conn(0x7f708c0fbab0 msgr2=0x7f708c102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.764+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 shutdown_connections 2024-03-22T10:56:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:18.764+0000 7f7090a00700 1 -- 172.21.15.115:0/1241228749 wait complete. 2024-03-22T10:56:19.400 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:19 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1241228749' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:56:19.439 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:19 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1241228749' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:56:19.502 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-22T10:56:19.900 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:56:20.299 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:20 smithi115 ceph-mon[78672]: pgmap v405: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:20.299 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:20 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:56:20.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:20 smithi150 ceph-mon[65513]: pgmap v405: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:20.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:20 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:56:21.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:21 smithi150 ceph-mon[65513]: pgmap v406: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:21.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:21 smithi115 ceph-mon[78672]: pgmap v406: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:21.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.986+0000 7f589a400700 1 -- 172.21.15.115:0/2528558779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 msgr2=0x7f5894068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:21.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.986+0000 7f589a400700 1 --2- 172.21.15.115:0/2528558779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 0x7f5894068a00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5884009910 tx=0x7f5884009c20 comp rx=0 tx=0).stop 2024-03-22T10:56:21.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.986+0000 7f589a400700 1 -- 172.21.15.115:0/2528558779 shutdown_connections 2024-03-22T10:56:21.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.986+0000 7f589a400700 1 --2- 172.21.15.115:0/2528558779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5894068fd0 0x7f5894104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:21.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.986+0000 7f589a400700 1 --2- 172.21.15.115:0/2528558779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 0x7f5894068a00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:21.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.986+0000 7f589a400700 1 -- 172.21.15.115:0/2528558779 >> 172.21.15.115:0/2528558779 conn(0x7f5894074f50 msgr2=0x7f5894075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.986+0000 7f589a400700 1 -- 172.21.15.115:0/2528558779 shutdown_connections 2024-03-22T10:56:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.986+0000 7f589a400700 1 -- 172.21.15.115:0/2528558779 wait complete. 2024-03-22T10:56:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.987+0000 7f589a400700 1 Processor -- start 2024-03-22T10:56:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.988+0000 7f589a400700 1 -- start start 2024-03-22T10:56:21.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.988+0000 7f589a400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 0x7f58940ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:21.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.988+0000 7f589a400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5894068fd0 0x7f58940ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:21.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.988+0000 7f589a400700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58940ffc80 con 0x7f5894068630 2024-03-22T10:56:21.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.988+0000 7f589a400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58940ffdc0 con 0x7f5894068fd0 2024-03-22T10:56:21.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.989+0000 7f5893400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 0x7f58940ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:21.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.989+0000 7f5893400700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 0x7f58940ff200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45598/0 (socket says 172.21.15.115:45598) 2024-03-22T10:56:21.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.989+0000 7f5893400700 1 -- 172.21.15.115:0/1205371813 learned_addr learned my addr 172.21.15.115:0/1205371813 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:21.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.989+0000 7f5892a00700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5894068fd0 0x7f58940ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:21.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.989+0000 7f5893400700 1 -- 172.21.15.115:0/1205371813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5894068fd0 msgr2=0x7f58940ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:21.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.989+0000 7f5893400700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5894068fd0 0x7f58940ff740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.989+0000 7f5893400700 1 -- 172.21.15.115:0/1205371813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840095f0 con 0x7f5894068630 2024-03-22T10:56:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.990+0000 7f5893400700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 0x7f58940ff200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f588400b5c0 tx=0x7f588400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.990+0000 7f588be00700 1 -- 172.21.15.115:0/1205371813 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f588401d070 con 0x7f5894068630 2024-03-22T10:56:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.990+0000 7f588be00700 1 -- 172.21.15.115:0/1205371813 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f588400f460 con 0x7f5894068630 2024-03-22T10:56:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.990+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5894100040 con 0x7f5894068630 2024-03-22T10:56:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.990+0000 7f588be00700 1 -- 172.21.15.115:0/1205371813 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5884017610 con 0x7f5894068630 2024-03-22T10:56:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.990+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58941a2420 con 0x7f5894068630 2024-03-22T10:56:21.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.992+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58941086d0 con 0x7f5894068630 2024-03-22T10:56:21.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.993+0000 7f588be00700 1 -- 172.21.15.115:0/1205371813 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f588400f5d0 con 0x7f5894068630 2024-03-22T10:56:21.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.994+0000 7f588be00700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f587806dff0 0x7f58780704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:21.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.994+0000 7f588be00700 1 -- 172.21.15.115:0/1205371813 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f588408f210 con 0x7f5894068630 2024-03-22T10:56:21.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.994+0000 7f5892a00700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f587806dff0 0x7f58780704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:21.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.995+0000 7f5892a00700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f587806dff0 0x7f58780704a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5880006fd0 tx=0x7f5880008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:21.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:21.999+0000 7f588be00700 1 -- 172.21.15.115:0/1205371813 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5884058720 con 0x7f5894068630 2024-03-22T10:56:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.407+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f58941007d0 con 0x7f5894068630 2024-03-22T10:56:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.408+0000 7f588be00700 1 -- 172.21.15.115:0/1205371813 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f588405bde0 con 0x7f5894068630 2024-03-22T10:56:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.410+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f587806dff0 msgr2=0x7f58780704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.410+0000 7f589a400700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f587806dff0 0x7f58780704a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5880006fd0 tx=0x7f5880008040 comp rx=0 tx=0).stop 2024-03-22T10:56:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.410+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 msgr2=0x7f58940ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.411+0000 7f589a400700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 0x7f58940ff200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f588400b5c0 tx=0x7f588400be00 comp rx=0 tx=0).stop 2024-03-22T10:56:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.411+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 shutdown_connections 2024-03-22T10:56:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.411+0000 7f589a400700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f587806dff0 0x7f58780704a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.411+0000 7f589a400700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5894068630 0x7f58940ff200 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.411+0000 7f589a400700 1 --2- 172.21.15.115:0/1205371813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5894068fd0 0x7f58940ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.411+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 >> 172.21.15.115:0/1205371813 conn(0x7f5894074f50 msgr2=0x7f58940fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.411+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 shutdown_connections 2024-03-22T10:56:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:22.411+0000 7f589a400700 1 -- 172.21.15.115:0/1205371813 wait complete. 2024-03-22T10:56:22.421 INFO:teuthology.orchestra.run.smithi115.stdout:true 2024-03-22T10:56:22.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:22 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1205371813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:56:22.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:22 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1205371813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:56:23.254 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-22T10:56:23.665 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:56:23.763 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:23 smithi115 ceph-mon[78672]: pgmap v407: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:23.901 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:23 smithi150 ceph-mon[65513]: pgmap v407: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:25.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.638+0000 7f5710e00700 1 -- 172.21.15.115:0/3116071298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c100d70 msgr2=0x7f570c104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:25.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.638+0000 7f5710e00700 1 --2- 172.21.15.115:0/3116071298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c100d70 0x7f570c104d40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009910 tx=0x7f56f4009c20 comp rx=0 tx=0).stop 2024-03-22T10:56:25.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.638+0000 7f5710e00700 1 -- 172.21.15.115:0/3116071298 shutdown_connections 2024-03-22T10:56:25.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.638+0000 7f5710e00700 1 --2- 172.21.15.115:0/3116071298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c100d70 0x7f570c104d40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:25.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.638+0000 7f5710e00700 1 --2- 172.21.15.115:0/3116071298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f570c1003d0 0x7f570c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:25.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.638+0000 7f5710e00700 1 -- 172.21.15.115:0/3116071298 >> 172.21.15.115:0/3116071298 conn(0x7f570c0fbc90 msgr2=0x7f570c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:25.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.638+0000 7f5710e00700 1 -- 172.21.15.115:0/3116071298 shutdown_connections 2024-03-22T10:56:25.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.638+0000 7f5710e00700 1 -- 172.21.15.115:0/3116071298 wait complete. 2024-03-22T10:56:25.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.640+0000 7f5710e00700 1 Processor -- start 2024-03-22T10:56:25.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.640+0000 7f5710e00700 1 -- start start 2024-03-22T10:56:25.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.640+0000 7f5710e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c1003d0 0x7f570c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:25.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.640+0000 7f5710e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f570c100d70 0x7f570c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:25.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.640+0000 7f5710e00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c073ad0 con 0x7f570c1003d0 2024-03-22T10:56:25.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.640+0000 7f5710e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c073c10 con 0x7f570c100d70 2024-03-22T10:56:25.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.641+0000 7f5709e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c1003d0 0x7f570c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:25.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.641+0000 7f5709e00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c1003d0 0x7f570c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45618/0 (socket says 172.21.15.115:45618) 2024-03-22T10:56:25.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.641+0000 7f5709e00700 1 -- 172.21.15.115:0/2186783120 learned_addr learned my addr 172.21.15.115:0/2186783120 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:25.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.641+0000 7f5701400700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f570c100d70 0x7f570c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:25.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.641+0000 7f5709e00700 1 -- 172.21.15.115:0/2186783120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f570c100d70 msgr2=0x7f570c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:25.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.641+0000 7f5709e00700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f570c100d70 0x7f570c073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:25.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.641+0000 7f5709e00700 1 -- 172.21.15.115:0/2186783120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f40095f0 con 0x7f570c1003d0 2024-03-22T10:56:25.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.642+0000 7f5709e00700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c1003d0 0x7f570c073050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c8a0 tx=0x7f56fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:25.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.642+0000 7f5703400700 1 -- 172.21.15.115:0/2186783120 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56fc00cea0 con 0x7f570c1003d0 2024-03-22T10:56:25.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.642+0000 7f5703400700 1 -- 172.21.15.115:0/2186783120 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56fc004830 con 0x7f570c1003d0 2024-03-22T10:56:25.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.642+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570c1a1f10 con 0x7f570c1003d0 2024-03-22T10:56:25.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.642+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570c1a2320 con 0x7f570c1003d0 2024-03-22T10:56:25.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.642+0000 7f5703400700 1 -- 172.21.15.115:0/2186783120 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56fc0055b0 con 0x7f570c1003d0 2024-03-22T10:56:25.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.643+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e8005320 con 0x7f570c1003d0 2024-03-22T10:56:25.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.645+0000 7f5703400700 1 -- 172.21.15.115:0/2186783120 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f56fc007ad0 con 0x7f570c1003d0 2024-03-22T10:56:25.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.645+0000 7f5703400700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f56ec06dfa0 0x7f56ec070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:25.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.646+0000 7f5703400700 1 -- 172.21.15.115:0/2186783120 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f56fc08d9b0 con 0x7f570c1003d0 2024-03-22T10:56:25.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.646+0000 7f5701400700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f56ec06dfa0 0x7f56ec070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:25.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.647+0000 7f5701400700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f56ec06dfa0 0x7f56ec070450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56f4000c00 tx=0x7f56f4005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:25.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:25.651+0000 7f5703400700 1 -- 172.21.15.115:0/2186783120 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f56fc056e00 con 0x7f570c1003d0 2024-03-22T10:56:26.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:25 smithi115 ceph-mon[78672]: pgmap v408: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:26.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.025+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f56e8006200 con 0x7f570c1003d0 2024-03-22T10:56:26.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.025+0000 7f5703400700 1 -- 172.21.15.115:0/2186783120 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f56fc05a4c0 con 0x7f570c1003d0 2024-03-22T10:56:26.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.028+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f56ec06dfa0 msgr2=0x7f56ec070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:26.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.028+0000 7f5710e00700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f56ec06dfa0 0x7f56ec070450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56f4000c00 tx=0x7f56f4005fd0 comp rx=0 tx=0).stop 2024-03-22T10:56:26.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.028+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c1003d0 msgr2=0x7f570c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:26.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.028+0000 7f5710e00700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c1003d0 0x7f570c073050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c8a0 tx=0x7f56fc00cc60 comp rx=0 tx=0).stop 2024-03-22T10:56:26.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.029+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 shutdown_connections 2024-03-22T10:56:26.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.029+0000 7f5710e00700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f56ec06dfa0 0x7f56ec070450 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:26.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.029+0000 7f5710e00700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570c1003d0 0x7f570c073050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:26.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.029+0000 7f5710e00700 1 --2- 172.21.15.115:0/2186783120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f570c100d70 0x7f570c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:26.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.029+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 >> 172.21.15.115:0/2186783120 conn(0x7f570c0fbc90 msgr2=0x7f570c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:26.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.029+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 shutdown_connections 2024-03-22T10:56:26.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:26.029+0000 7f5710e00700 1 -- 172.21.15.115:0/2186783120 wait complete. 2024-03-22T10:56:26.041 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)" 2024-03-22T10:56:26.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:25 smithi150 ceph-mon[65513]: pgmap v408: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:26.772 DEBUG:teuthology.parallel:result is None 2024-03-22T10:56:26.772 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T10:56:26.783 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T10:56:26.783 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd1caff4-e836-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T10:56:27.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:26 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2186783120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:56:27.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:26 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2186783120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T10:56:27.181 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/mon.smithi115/config 2024-03-22T10:56:28.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:27 smithi115 ceph-mon[78672]: pgmap v409: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:27 smithi150 ceph-mon[65513]: pgmap v409: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:29.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.309+0000 7f7eece00700 1 -- 172.21.15.115:0/1537956057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 msgr2=0x7f7ee8104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:29.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.309+0000 7f7eece00700 1 --2- 172.21.15.115:0/1537956057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 0x7f7ee8104c00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009910 tx=0x7f7ed4009c20 comp rx=0 tx=0).stop 2024-03-22T10:56:29.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.309+0000 7f7eece00700 1 -- 172.21.15.115:0/1537956057 shutdown_connections 2024-03-22T10:56:29.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.309+0000 7f7eece00700 1 --2- 172.21.15.115:0/1537956057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 0x7f7ee8104c00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:29.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.309+0000 7f7eece00700 1 --2- 172.21.15.115:0/1537956057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ee8100290 0x7f7ee8100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:29.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.309+0000 7f7eece00700 1 -- 172.21.15.115:0/1537956057 >> 172.21.15.115:0/1537956057 conn(0x7f7ee80fbb90 msgr2=0x7f7ee80fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:29.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.309+0000 7f7eece00700 1 -- 172.21.15.115:0/1537956057 shutdown_connections 2024-03-22T10:56:29.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.309+0000 7f7eece00700 1 -- 172.21.15.115:0/1537956057 wait complete. 2024-03-22T10:56:29.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.310+0000 7f7eece00700 1 Processor -- start 2024-03-22T10:56:29.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.310+0000 7f7eece00700 1 -- start start 2024-03-22T10:56:29.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.311+0000 7f7eece00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ee8100290 0x7f7ee8197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:29.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.311+0000 7f7eece00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 0x7f7ee81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:29.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.311+0000 7f7eece00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee8198980 con 0x7f7ee8100c30 2024-03-22T10:56:29.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.311+0000 7f7eece00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee819c710 con 0x7f7ee8100290 2024-03-22T10:56:29.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.311+0000 7f7ee6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 0x7f7ee81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:29.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.311+0000 7f7ee6a00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 0x7f7ee81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45638/0 (socket says 172.21.15.115:45638) 2024-03-22T10:56:29.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.311+0000 7f7ee6a00700 1 -- 172.21.15.115:0/3612176232 learned_addr learned my addr 172.21.15.115:0/3612176232 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:29.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.311+0000 7f7ee7400700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ee8100290 0x7f7ee8197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:29.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.312+0000 7f7ee6a00700 1 -- 172.21.15.115:0/3612176232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ee8100290 msgr2=0x7f7ee8197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:29.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.312+0000 7f7ee6a00700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ee8100290 0x7f7ee8197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:29.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.312+0000 7f7ee6a00700 1 -- 172.21.15.115:0/3612176232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed40095f0 con 0x7f7ee8100c30 2024-03-22T10:56:29.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.312+0000 7f7ee6a00700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 0x7f7ee81982a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40092e0 tx=0x7f7ed400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:29.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.312+0000 7f7edfe00700 1 -- 172.21.15.115:0/3612176232 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ed401d070 con 0x7f7ee8100c30 2024-03-22T10:56:29.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.313+0000 7f7edfe00700 1 -- 172.21.15.115:0/3612176232 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ed4005dc0 con 0x7f7ee8100c30 2024-03-22T10:56:29.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.313+0000 7f7edfe00700 1 -- 172.21.15.115:0/3612176232 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ed4017400 con 0x7f7ee8100c30 2024-03-22T10:56:29.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.313+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee819c990 con 0x7f7ee8100c30 2024-03-22T10:56:29.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.313+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee819ce30 con 0x7f7ee8100c30 2024-03-22T10:56:29.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.314+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee8104380 con 0x7f7ee8100c30 2024-03-22T10:56:29.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.315+0000 7f7edfe00700 1 -- 172.21.15.115:0/3612176232 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f7ed400f460 con 0x7f7ee8100c30 2024-03-22T10:56:29.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.316+0000 7f7edfe00700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7ecc06dfa0 0x7f7ecc070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:29.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.317+0000 7f7edfe00700 1 -- 172.21.15.115:0/3612176232 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f7ed408f380 con 0x7f7ee8100c30 2024-03-22T10:56:29.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.317+0000 7f7ee7400700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7ecc06dfa0 0x7f7ecc070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:29.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.318+0000 7f7ee7400700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7ecc06dfa0 0x7f7ecc070450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005950 tx=0x7f7ed80058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:29.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.321+0000 7f7edfe00700 1 -- 172.21.15.115:0/3612176232 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ed40586d0 con 0x7f7ee8100c30 2024-03-22T10:56:29.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.629+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7ee804f2a0 con 0x7f7ee8100c30 2024-03-22T10:56:29.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.630+0000 7f7edfe00700 1 -- 172.21.15.115:0/3612176232 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1932 (secure 0 0 0) 0x7f7ed405bd90 con 0x7f7ee8100c30 2024-03-22T10:56:29.624 INFO:teuthology.orchestra.run.smithi115.stdout:e40 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 40 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-03-22T10:38:29.359957+0000 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-03-22T10:52:05.899739+0000 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-03-22T10:56:29.625 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 60 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 83 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 2 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=34476,1=34508} 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-03-22T10:56:29.626 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3] 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data enabled 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 1 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 34476 members: 34476,34508 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.ggxfzi{0:34476} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.115:6826/2107356580,v1:172.21.15.115:6827/2107356580] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi115.wjotmn{1:34508} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.115:6828/3270789943,v1:172.21.15.115:6829/3270789943] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:29.627 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.zvyrzc{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6824/1804848703,v1:172.21.15.150:6825/1804848703] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:56:29.628 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.cephfs.smithi150.slzwwb{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.150:6826/2053213512,v1:172.21.15.150:6827/2053213512] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T10:56:29.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7ecc06dfa0 msgr2=0x7f7ecc070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:29.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7ecc06dfa0 0x7f7ecc070450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005950 tx=0x7f7ed80058e0 comp rx=0 tx=0).stop 2024-03-22T10:56:29.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 msgr2=0x7f7ee81982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:29.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 0x7f7ee81982a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40092e0 tx=0x7f7ed400be20 comp rx=0 tx=0).stop 2024-03-22T10:56:29.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 shutdown_connections 2024-03-22T10:56:29.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f7ecc06dfa0 0x7f7ecc070450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:29.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ee8100290 0x7f7ee8197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:29.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 --2- 172.21.15.115:0/3612176232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8100c30 0x7f7ee81982a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:29.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 >> 172.21.15.115:0/3612176232 conn(0x7f7ee80fbb90 msgr2=0x7f7ee80fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:29.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 shutdown_connections 2024-03-22T10:56:29.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:29.634+0000 7f7eece00700 1 -- 172.21.15.115:0/3612176232 wait complete. 2024-03-22T10:56:29.629 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2024-03-22T10:56:30.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:29 smithi115 ceph-mon[78672]: pgmap v410: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:30.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:29 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:56:30.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:29 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3612176232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:56:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:29 smithi150 ceph-mon[65513]: pgmap v410: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:29 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:56:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:29 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3612176232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T10:56:30.365 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-22T10:56:30.376 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T10:56:30.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.605+0000 7f3937e00640 1 -- 172.21.15.115:0/1256276095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 msgr2=0x7f3938104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:30.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.605+0000 7f3937e00640 1 --2- 172.21.15.115:0/1256276095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 0x7f3938104020 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3924009880 tx=0x7f392402bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:30.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.605+0000 7f3937e00640 1 -- 172.21.15.115:0/1256276095 shutdown_connections 2024-03-22T10:56:30.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.605+0000 7f3937e00640 1 --2- 172.21.15.115:0/1256276095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 0x7f3938104020 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:30.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.605+0000 7f3937e00640 1 --2- 172.21.15.115:0/1256276095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39380ff6c0 0x7f39380ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:30.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.605+0000 7f3937e00640 1 -- 172.21.15.115:0/1256276095 >> 172.21.15.115:0/1256276095 conn(0x7f39380faf50 msgr2=0x7f39380fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:30.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.606+0000 7f3937e00640 1 -- 172.21.15.115:0/1256276095 shutdown_connections 2024-03-22T10:56:30.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.606+0000 7f3937e00640 1 -- 172.21.15.115:0/1256276095 wait complete. 2024-03-22T10:56:30.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.607+0000 7f3937e00640 1 Processor -- start 2024-03-22T10:56:30.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.607+0000 7f3937e00640 1 -- start start 2024-03-22T10:56:30.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.608+0000 7f3937e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39380ff6c0 0x7f3938197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:30.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.608+0000 7f3937e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 0x7f3938197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:30.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.608+0000 7f3937e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3938197ff0 con 0x7f3938100060 2024-03-22T10:56:30.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.608+0000 7f3937e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393819bdc0 con 0x7f39380ff6c0 2024-03-22T10:56:30.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.608+0000 7f3936000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 0x7f3938197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:30.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.608+0000 7f3936000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 0x7f3938197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45652/0 (socket says 172.21.15.115:45652) 2024-03-22T10:56:30.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.608+0000 7f3936000640 1 -- 172.21.15.115:0/4235567794 learned_addr learned my addr 172.21.15.115:0/4235567794 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:30.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.608+0000 7f3936a00640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39380ff6c0 0x7f3938197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:30.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.609+0000 7f3936000640 1 -- 172.21.15.115:0/4235567794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39380ff6c0 msgr2=0x7f3938197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:30.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.609+0000 7f3936000640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39380ff6c0 0x7f3938197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:30.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.609+0000 7f3936000640 1 -- 172.21.15.115:0/4235567794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924009560 con 0x7f3938100060 2024-03-22T10:56:30.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.609+0000 7f3936000640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 0x7f3938197960 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3924009230 tx=0x7f392402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:30.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.610+0000 7f392f400640 1 -- 172.21.15.115:0/4235567794 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39240035f0 con 0x7f3938100060 2024-03-22T10:56:30.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.610+0000 7f392f400640 1 -- 172.21.15.115:0/4235567794 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3924003c10 con 0x7f3938100060 2024-03-22T10:56:30.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.610+0000 7f392f400640 1 -- 172.21.15.115:0/4235567794 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3924005630 con 0x7f3938100060 2024-03-22T10:56:30.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.610+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393819c040 con 0x7f3938100060 2024-03-22T10:56:30.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.610+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393819c540 con 0x7f3938100060 2024-03-22T10:56:30.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.612+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393804e850 con 0x7f3938100060 2024-03-22T10:56:30.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.613+0000 7f392f400640 1 -- 172.21.15.115:0/4235567794 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f3924004ed0 con 0x7f3938100060 2024-03-22T10:56:30.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.614+0000 7f392f400640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f391c06df70 0x7f391c070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:30.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.615+0000 7f392f400640 1 -- 172.21.15.115:0/4235567794 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f39240afae0 con 0x7f3938100060 2024-03-22T10:56:30.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.615+0000 7f3936a00640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f391c06df70 0x7f391c070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:30.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.616+0000 7f3936a00640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f391c06df70 0x7f391c070420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f392800bbe0 tx=0x7f392800b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:30.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.621+0000 7f392f400640 1 -- 172.21.15.115:0/4235567794 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3924078ee0 con 0x7f3938100060 2024-03-22T10:56:30.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.928+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3938103710 con 0x7f3938100060 2024-03-22T10:56:30.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.930+0000 7f392f400640 1 -- 172.21.15.115:0/4235567794 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16966 (secure 0 0 0) 0x7f392407c5a0 con 0x7f3938100060 2024-03-22T10:56:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:30.925 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":83,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","created":"2024-03-22T10:29:37.233959+0000","modified":"2024-03-22T10:51:43.567993+0000","last_up_change":"2024-03-22T10:49:24.797169+0000","last_in_change":"2024-03-22T10:36:35.075110+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T10:35:17.870309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T10:38:27.996809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T10:38:28.878933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a34c0f70-4e9d-4044-b7b4-b6099b74ba6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6803","nonce":2453947745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6805","nonce":2453947745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6809","nonce":2453947745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6807","nonce":2453947745}]},"public_addr":"172.21.15.115:6803/2453947745","cluster_addr":"172.21.15.115:6805/2453947745","heartbeat_back_addr":"172.21.15.115:6809/2453947745","heartbeat_front_addr":"172.21.15.115:6807/2453947745","state":["exists","up"]},{"osd":1,"uuid":"1037db4c-35bc-48da-8950-460c772d50b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6811","nonce":3813430676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6813","nonce":3813430676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6817","nonce":3813430676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6815","nonce":3813430676}]},"public_addr":"172.21.15.115:6811/3813430676","cluster_addr":"172.21.15.115:6813/3813430676","heartbeat_back_addr":"172.21.15.115:6817/3813430676","heartbeat_front_addr":"172.21.15.115:6815/3813430676","state":["exists","up"]},{"osd":2,"uuid":"4e685a22-30ee-4404-bba5-ff944af9b858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6819","nonce":3285562920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6821","nonce":3285562920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6825","nonce":3285562920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6823","nonce":3285562920}]},"public_addr":"172.21.15.115:6819/3285562920","cluster_addr":"172.21.15.115:6821/3285562920","heartbeat_back_addr":"172.21.15.115:6825/3285562920","heartbeat_front_addr":"172.21.15.115:6823/3285562920","state":["exists","up"]},{"osd":3,"uuid":"1273664e-16c8-46c1-bfb4-c51e3e84ea6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6801","nonce":2792834726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6803","nonce":2792834726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6807","nonce":2792834726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6805","nonce":2792834726}]},"public_addr":"172.21.15.150:6801/2792834726","cluster_addr":"172.21.15.150:6803/2792834726","heartbeat_back_addr":"172.21.15.150:6807/2792834726","heartbeat_front_addr":"172.21.15.150:6805/2792834726","state":["exists","up"]},{"osd":4,"uuid":"574fd0d5-c58a-4e16-961d-b384e52d82b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6809","nonce":231697863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6811","nonce":231697863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6815","nonce":231697863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6813","nonce":231697863}]},"public_addr":"172.21.15.150:6809/231697863","cluster_addr":"172.21.15.150:6811/231697863","heartbeat_back_addr":"172.21.15.150:6815/231697863","heartbeat_front_addr":"172.21.15.150:6813/231697863","state":["exists","up"]},{"osd":5,"uuid":"cc37270e-923d-45cf-accd-d4119dcc1a7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6817","nonce":2534589159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6819","nonce":2534589159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6823","nonce":2534589159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6821","nonce":2534589159}]},"public_addr":"172.21.15.150:6817/2534589159","cluster_addr":"172.21.15.150:6819/2534589159","heartbeat_back_addr":"172.21.15.150:6823/2534589159","heartbeat_front_addr":"172.21.15.150:6821/2534589159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:04.542374+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:39.181161+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:16.028358+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:47.869896+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:22.969676+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:57.504061+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6827/2497876489":"2024-03-23T10:51:43.567943+0000","172.21.15.150:6826/2497876489":"2024-03-23T10:51:43.567943+0000","172.21.15.115:6829/4173107662":"2024-03-23T10:50:50.661077+0000","172.21.15.115:6828/4173107662":"2024-03-23T10:50:50.661077+0000","172.21.15.115:6827/3803356871":"2024-03-23T10:50:18.120225+0000","172.21.15.115:6801/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3152688785":"2024-03-23T10:41:27.672754+0000","172.21.15.115:0/450820820":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3897395096":"2024-03-23T10:44:04.061914+0000","172.21.15.115:0/243936731":"2024-03-23T10:44:04.061914+0000","172.21.15.115:6800/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3451764091":"2024-03-23T10:32:25.174415+0000","172.21.15.150:0/2901058091":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/3333444965":"2024-03-23T10:41:27.672754+0000","172.21.15.115:6800/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6801/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/880475782":"2024-03-23T10:44:04.061914+0000","172.21.15.115:0/3264237187":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/4100000632":"2024-03-23T10:44:04.061914+0000","172.21.15.150:0/3168349733":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/901503025":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/1202069415":"2024-03-23T10:41:27.672754+0000","172.21.15.115:0/3025276768":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/2275951342":"2024-03-23T10:30:58.613511+0000","172.21.15.150:6828/3274858902":"2024-03-23T10:42:28.351506+0000","172.21.15.150:0/3722858322":"2024-03-23T10:42:28.351506+0000","172.21.15.150:0/1874309885":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/3641576098":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/1251137983":"2024-03-23T10:41:27.672754+0000","172.21.15.150:0/2220372156":"2024-03-23T10:42:28.351506+0000","172.21.15.115:6800/1251137983":"2024-03-23T10:41:27.672754+0000","172.21.15.115:0/617745307":"2024-03-23T10:41:27.672754+0000","172.21.15.115:6800/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/246266260":"2024-03-23T10:44:04.061914+0000","172.21.15.115:6826/3803356871":"2024-03-23T10:50:18.120225+0000","172.21.15.115:6801/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.150:0/2001872342":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/2167508975":"2024-03-23T10:44:04.061914+0000","172.21.15.150:6829/3274858902":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/1384534494":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/400653309":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6800/246266260":"2024-03-23T10:44:04.061914+0000","172.21.15.115:0/401673952":"2024-03-23T10:44:04.061914+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T10:56:30.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.933+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f391c06df70 msgr2=0x7f391c070420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.933+0000 7f3937e00640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f391c06df70 0x7f391c070420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f392800bbe0 tx=0x7f392800b390 comp rx=0 tx=0).stop 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.933+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 msgr2=0x7f3938197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.933+0000 7f3937e00640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 0x7f3938197960 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3924009230 tx=0x7f392402c850 comp rx=0 tx=0).stop 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.933+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 shutdown_connections 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.933+0000 7f3937e00640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f391c06df70 0x7f391c070420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.933+0000 7f3937e00640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3938100060 0x7f3938197960 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.934+0000 7f3937e00640 1 --2- 172.21.15.115:0/4235567794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39380ff6c0 0x7f3938197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.934+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 >> 172.21.15.115:0/4235567794 conn(0x7f39380faf50 msgr2=0x7f39380fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.934+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 shutdown_connections 2024-03-22T10:56:30.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:30.934+0000 7f3937e00640 1 -- 172.21.15.115:0/4235567794 wait complete. 2024-03-22T10:56:30.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T10:35:17.870309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T10:38:27.996809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T10:38:28.878933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T10:56:30.952 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T10:56:31.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.168+0000 7f0c67200640 1 -- 172.21.15.115:0/96854779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c68067310 msgr2=0x7f0c68067760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:31.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.168+0000 7f0c67200640 1 --2- 172.21.15.115:0/96854779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c68067310 0x7f0c68067760 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009880 tx=0x7f0c5402bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.169+0000 7f0c67200640 1 -- 172.21.15.115:0/96854779 shutdown_connections 2024-03-22T10:56:31.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.169+0000 7f0c67200640 1 --2- 172.21.15.115:0/96854779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c68067310 0x7f0c68067760 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.169+0000 7f0c67200640 1 --2- 172.21.15.115:0/96854779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c681055c0 0x7f0c68105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.169+0000 7f0c67200640 1 -- 172.21.15.115:0/96854779 >> 172.21.15.115:0/96854779 conn(0x7f0c680743d0 msgr2=0x7f0c680747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:31.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.169+0000 7f0c67200640 1 -- 172.21.15.115:0/96854779 shutdown_connections 2024-03-22T10:56:31.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.169+0000 7f0c67200640 1 -- 172.21.15.115:0/96854779 wait complete. 2024-03-22T10:56:31.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.170+0000 7f0c67200640 1 Processor -- start 2024-03-22T10:56:31.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.170+0000 7f0c67200640 1 -- start start 2024-03-22T10:56:31.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.170+0000 7f0c67200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c68067310 0x7f0c68195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:31.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.170+0000 7f0c67200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c681055c0 0x7f0c68195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:31.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.170+0000 7f0c67200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c68195d10 con 0x7f0c681055c0 2024-03-22T10:56:31.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.170+0000 7f0c67200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c68199ae0 con 0x7f0c68067310 2024-03-22T10:56:31.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.171+0000 7f0c65400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c681055c0 0x7f0c68195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:31.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.171+0000 7f0c65400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c681055c0 0x7f0c68195680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45680/0 (socket says 172.21.15.115:45680) 2024-03-22T10:56:31.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.171+0000 7f0c65400640 1 -- 172.21.15.115:0/1486006806 learned_addr learned my addr 172.21.15.115:0/1486006806 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:31.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.171+0000 7f0c65e00640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c68067310 0x7f0c68195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:31.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.171+0000 7f0c65400640 1 -- 172.21.15.115:0/1486006806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c68067310 msgr2=0x7f0c68195140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:31.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.171+0000 7f0c65400640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c68067310 0x7f0c68195140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.171+0000 7f0c65400640 1 -- 172.21.15.115:0/1486006806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c54009560 con 0x7f0c681055c0 2024-03-22T10:56:31.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.171+0000 7f0c65400640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c681055c0 0x7f0c68195680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0c54000c00 tx=0x7f0c54004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:31.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.172+0000 7f0c5ea00640 1 -- 172.21.15.115:0/1486006806 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c540035f0 con 0x7f0c681055c0 2024-03-22T10:56:31.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.172+0000 7f0c5ea00640 1 -- 172.21.15.115:0/1486006806 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c54003c10 con 0x7f0c681055c0 2024-03-22T10:56:31.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.172+0000 7f0c5ea00640 1 -- 172.21.15.115:0/1486006806 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c54034440 con 0x7f0c681055c0 2024-03-22T10:56:31.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.172+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c68199d60 con 0x7f0c681055c0 2024-03-22T10:56:31.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.172+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6819a260 con 0x7f0c681055c0 2024-03-22T10:56:31.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.173+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6804e000 con 0x7f0c681055c0 2024-03-22T10:56:31.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.174+0000 7f0c5ea00640 1 -- 172.21.15.115:0/1486006806 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f0c54003750 con 0x7f0c681055c0 2024-03-22T10:56:31.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.175+0000 7f0c5ea00640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0c4c06df10 0x7f0c4c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:31.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.175+0000 7f0c5ea00640 1 -- 172.21.15.115:0/1486006806 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f0c540b01d0 con 0x7f0c681055c0 2024-03-22T10:56:31.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.175+0000 7f0c65e00640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0c4c06df10 0x7f0c4c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:31.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.176+0000 7f0c65e00640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0c4c06df10 0x7f0c4c0703c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c58005fd0 tx=0x7f0c58005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:31.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.181+0000 7f0c5ea00640 1 -- 172.21.15.115:0/1486006806 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0c54079620 con 0x7f0c681055c0 2024-03-22T10:56:31.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.403+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c6810c720 con 0x7f0c681055c0 2024-03-22T10:56:31.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.404+0000 7f0c5ea00640 1 -- 172.21.15.115:0/1486006806 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c5407cce0 con 0x7f0c681055c0 2024-03-22T10:56:31.476 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-03-22T10:56:31.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.406+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0c4c06df10 msgr2=0x7f0c4c0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:31.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.406+0000 7f0c67200640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0c4c06df10 0x7f0c4c0703c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c58005fd0 tx=0x7f0c58005850 comp rx=0 tx=0).stop 2024-03-22T10:56:31.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.406+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c681055c0 msgr2=0x7f0c68195680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:31.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.407+0000 7f0c67200640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c681055c0 0x7f0c68195680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0c54000c00 tx=0x7f0c54004040 comp rx=0 tx=0).stop 2024-03-22T10:56:31.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.407+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 shutdown_connections 2024-03-22T10:56:31.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.407+0000 7f0c67200640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f0c4c06df10 0x7f0c4c0703c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.407+0000 7f0c67200640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c681055c0 0x7f0c68195680 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.407+0000 7f0c67200640 1 --2- 172.21.15.115:0/1486006806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c68067310 0x7f0c68195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.407+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 >> 172.21.15.115:0/1486006806 conn(0x7f0c680743d0 msgr2=0x7f0c680fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:31.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.407+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 shutdown_connections 2024-03-22T10:56:31.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.407+0000 7f0c67200640 1 -- 172.21.15.115:0/1486006806 wait complete. 2024-03-22T10:56:31.478 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T10:56:31.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.701+0000 7fef02e00640 1 -- 172.21.15.115:0/1459979830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 msgr2=0x7feefc0fe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:31.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.701+0000 7fef02e00640 1 --2- 172.21.15.115:0/1459979830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 0x7feefc0fe960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feee8009880 tx=0x7feee802bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.702+0000 7fef02e00640 1 -- 172.21.15.115:0/1459979830 shutdown_connections 2024-03-22T10:56:31.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.702+0000 7fef02e00640 1 --2- 172.21.15.115:0/1459979830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 0x7feefc0fe960 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.702+0000 7fef02e00640 1 --2- 172.21.15.115:0/1459979830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feefc104500 0x7feefc1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.702+0000 7fef02e00640 1 -- 172.21.15.115:0/1459979830 >> 172.21.15.115:0/1459979830 conn(0x7feefc0f9ff0 msgr2=0x7feefc0fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:31.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.702+0000 7fef02e00640 1 -- 172.21.15.115:0/1459979830 shutdown_connections 2024-03-22T10:56:31.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.702+0000 7fef02e00640 1 -- 172.21.15.115:0/1459979830 wait complete. 2024-03-22T10:56:31.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.703+0000 7fef02e00640 1 Processor -- start 2024-03-22T10:56:31.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.703+0000 7fef02e00640 1 -- start start 2024-03-22T10:56:31.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.704+0000 7fef02e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 0x7feefc1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:31.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.704+0000 7fef02e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feefc104500 0x7feefc196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:31.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.704+0000 7fef02e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc196e00 con 0x7feefc0fe4f0 2024-03-22T10:56:31.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.704+0000 7fef02e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc196f40 con 0x7feefc104500 2024-03-22T10:56:31.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.704+0000 7feefbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 0x7feefc1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:31.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.704+0000 7feefbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 0x7feefc1961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45686/0 (socket says 172.21.15.115:45686) 2024-03-22T10:56:31.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.704+0000 7feefbe00640 1 -- 172.21.15.115:0/1242499411 learned_addr learned my addr 172.21.15.115:0/1242499411 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:31.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.704+0000 7feefb400640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feefc104500 0x7feefc196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:31.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.705+0000 7feefbe00640 1 -- 172.21.15.115:0/1242499411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feefc104500 msgr2=0x7feefc196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:31.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.705+0000 7feefbe00640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feefc104500 0x7feefc196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.705+0000 7feefbe00640 1 -- 172.21.15.115:0/1242499411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8009560 con 0x7feefc0fe4f0 2024-03-22T10:56:31.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.705+0000 7feefbe00640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 0x7feefc1961e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feeec00d8d0 tx=0x7feeec00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:31.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.705+0000 7feef8c00640 1 -- 172.21.15.115:0/1242499411 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeec00bea0 con 0x7feefc0fe4f0 2024-03-22T10:56:31.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.705+0000 7feef8c00640 1 -- 172.21.15.115:0/1242499411 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeec009c00 con 0x7feefc0fe4f0 2024-03-22T10:56:31.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.705+0000 7feef8c00640 1 -- 172.21.15.115:0/1242499411 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeec00f710 con 0x7feefc0fe4f0 2024-03-22T10:56:31.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.705+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc19f250 con 0x7feefc0fe4f0 2024-03-22T10:56:31.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.705+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc19f730 con 0x7feefc0fe4f0 2024-03-22T10:56:31.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.707+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc04e000 con 0x7feefc0fe4f0 2024-03-22T10:56:31.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.708+0000 7feef8c00640 1 -- 172.21.15.115:0/1242499411 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7feeec00f990 con 0x7feefc0fe4f0 2024-03-22T10:56:31.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.709+0000 7feef8c00640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feee006df10 0x7feee00703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.709+0000 7feef8c00640 1 -- 172.21.15.115:0/1242499411 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7feeec08dbe0 con 0x7feefc0fe4f0 2024-03-22T10:56:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.709+0000 7feefb400640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feee006df10 0x7feee00703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:31.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.710+0000 7feefb400640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feee006df10 0x7feee00703c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feee802c8b0 tx=0x7feee8005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:31.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.714+0000 7feef8c00640 1 -- 172.21.15.115:0/1242499411 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feeec056f30 con 0x7feefc0fe4f0 2024-03-22T10:56:31.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.928+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7feefc1069d0 con 0x7feefc0fe4f0 2024-03-22T10:56:31.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.929+0000 7feef8c00640 1 -- 172.21.15.115:0/1242499411 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7feeec05a5f0 con 0x7feefc0fe4f0 2024-03-22T10:56:31.923 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feee006df10 msgr2=0x7feee00703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feee006df10 0x7feee00703c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feee802c8b0 tx=0x7feee8005fd0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 msgr2=0x7feefc1961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 0x7feefc1961e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feeec00d8d0 tx=0x7feeec00dd90 comp rx=0 tx=0).stop 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 shutdown_connections 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7feee006df10 0x7feee00703c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feefc104500 0x7feefc196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 --2- 172.21.15.115:0/1242499411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feefc0fe4f0 0x7feefc1961e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 >> 172.21.15.115:0/1242499411 conn(0x7feefc0f9ff0 msgr2=0x7feefc0faa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 shutdown_connections 2024-03-22T10:56:31.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:31.933+0000 7fef02e00640 1 -- 172.21.15.115:0/1242499411 wait complete. 2024-03-22T10:56:31.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T10:56:32.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:31 smithi115 ceph-mon[78672]: pgmap v411: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:32.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:56:32.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:56:32.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:56:32.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:31 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/4235567794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:56:32.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:31 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1486006806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:56:32.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:31 smithi150 ceph-mon[65513]: pgmap v411: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:32.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:56:32.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:56:32.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:56:32.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:31 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/4235567794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:56:32.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:31 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1486006806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:56:32.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.170+0000 7fa356e00640 1 -- 172.21.15.115:0/17247443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa350105660 msgr2=0x7fa350105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:32.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.170+0000 7fa356e00640 1 --2- 172.21.15.115:0/17247443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa350105660 0x7fa350105a30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa340009880 tx=0x7fa34002bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.171+0000 7fa356e00640 1 -- 172.21.15.115:0/17247443 shutdown_connections 2024-03-22T10:56:32.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.171+0000 7fa356e00640 1 --2- 172.21.15.115:0/17247443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3500ff650 0x7fa3500ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.171+0000 7fa356e00640 1 --2- 172.21.15.115:0/17247443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa350105660 0x7fa350105a30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.171+0000 7fa356e00640 1 -- 172.21.15.115:0/17247443 >> 172.21.15.115:0/17247443 conn(0x7fa3500fb170 msgr2=0x7fa3500fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:32.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.171+0000 7fa356e00640 1 -- 172.21.15.115:0/17247443 shutdown_connections 2024-03-22T10:56:32.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.171+0000 7fa356e00640 1 -- 172.21.15.115:0/17247443 wait complete. 2024-03-22T10:56:32.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.172+0000 7fa356e00640 1 Processor -- start 2024-03-22T10:56:32.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.172+0000 7fa356e00640 1 -- start start 2024-03-22T10:56:32.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.173+0000 7fa356e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3500ff650 0x7fa350195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:32.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.173+0000 7fa356e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa350105660 0x7fa350195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:32.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.173+0000 7fa356e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa350195f20 con 0x7fa3500ff650 2024-03-22T10:56:32.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.173+0000 7fa356e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa350199cf0 con 0x7fa350105660 2024-03-22T10:56:32.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.173+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3500ff650 0x7fa350195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:32.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.173+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3500ff650 0x7fa350195350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60364/0 (socket says 172.21.15.115:60364) 2024-03-22T10:56:32.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.173+0000 7fa34fe00640 1 -- 172.21.15.115:0/2363933968 learned_addr learned my addr 172.21.15.115:0/2363933968 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.174+0000 7fa34f400640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa350105660 0x7fa350195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.174+0000 7fa34fe00640 1 -- 172.21.15.115:0/2363933968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa350105660 msgr2=0x7fa350195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.174+0000 7fa34fe00640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa350105660 0x7fa350195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.174+0000 7fa34fe00640 1 -- 172.21.15.115:0/2363933968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340009560 con 0x7fa3500ff650 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.174+0000 7fa34fe00640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3500ff650 0x7fa350195350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa340036040 tx=0x7fa340004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.174+0000 7fa34cc00640 1 -- 172.21.15.115:0/2363933968 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3400035f0 con 0x7fa3500ff650 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.174+0000 7fa34cc00640 1 -- 172.21.15.115:0/2363933968 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa340003c10 con 0x7fa3500ff650 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.174+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa350199f70 con 0x7fa3500ff650 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.174+0000 7fa34cc00640 1 -- 172.21.15.115:0/2363933968 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa340034440 con 0x7fa3500ff650 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.175+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35019a4d0 con 0x7fa3500ff650 2024-03-22T10:56:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.176+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35004e000 con 0x7fa3500ff650 2024-03-22T10:56:32.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.177+0000 7fa34cc00640 1 -- 172.21.15.115:0/2363933968 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa340003750 con 0x7fa3500ff650 2024-03-22T10:56:32.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.178+0000 7fa34cc00640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa33406df10 0x7fa3340703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:32.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.178+0000 7fa34cc00640 1 -- 172.21.15.115:0/2363933968 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa3400b01e0 con 0x7fa3500ff650 2024-03-22T10:56:32.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.178+0000 7fa34f400640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa33406df10 0x7fa3340703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:32.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.179+0000 7fa34f400640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa33406df10 0x7fa3340703c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa350196960 tx=0x7fa33c005c60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:32.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.182+0000 7fa34cc00640 1 -- 172.21.15.115:0/2363933968 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa340079630 con 0x7fa3500ff650 2024-03-22T10:56:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.421+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa35019a100 con 0x7fa3500ff650 2024-03-22T10:56:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.422+0000 7fa34cc00640 1 -- 172.21.15.115:0/2363933968 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa35019a100 con 0x7fa3500ff650 2024-03-22T10:56:32.416 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa33406df10 msgr2=0x7fa3340703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa33406df10 0x7fa3340703c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa350196960 tx=0x7fa33c005c60 comp rx=0 tx=0).stop 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3500ff650 msgr2=0x7fa350195350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3500ff650 0x7fa350195350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa340036040 tx=0x7fa340004040 comp rx=0 tx=0).stop 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 shutdown_connections 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fa33406df10 0x7fa3340703c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa350105660 0x7fa350195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 --2- 172.21.15.115:0/2363933968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3500ff650 0x7fa350195350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 >> 172.21.15.115:0/2363933968 conn(0x7fa3500fb170 msgr2=0x7fa3500fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 shutdown_connections 2024-03-22T10:56:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.425+0000 7fa356e00640 1 -- 172.21.15.115:0/2363933968 wait complete. 2024-03-22T10:56:32.439 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T10:56:32.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.641+0000 7fab6d400640 1 -- 172.21.15.115:0/3136728527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab681013c0 msgr2=0x7fab681098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:32.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.641+0000 7fab6d400640 1 --2- 172.21.15.115:0/3136728527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab681013c0 0x7fab681098b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fab540098b0 tx=0x7fab5402bd70 comp rx=0 tx=0).stop 2024-03-22T10:56:32.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.642+0000 7fab6d400640 1 -- 172.21.15.115:0/3136728527 shutdown_connections 2024-03-22T10:56:32.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.642+0000 7fab6d400640 1 --2- 172.21.15.115:0/3136728527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab681013c0 0x7fab681098b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.642+0000 7fab6d400640 1 --2- 172.21.15.115:0/3136728527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab68100ab0 0x7fab68100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.642+0000 7fab6d400640 1 -- 172.21.15.115:0/3136728527 >> 172.21.15.115:0/3136728527 conn(0x7fab680fa300 msgr2=0x7fab680fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:32.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.642+0000 7fab6d400640 1 -- 172.21.15.115:0/3136728527 shutdown_connections 2024-03-22T10:56:32.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.642+0000 7fab6d400640 1 -- 172.21.15.115:0/3136728527 wait complete. 2024-03-22T10:56:32.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.643+0000 7fab6d400640 1 Processor -- start 2024-03-22T10:56:32.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.643+0000 7fab6d400640 1 -- start start 2024-03-22T10:56:32.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.643+0000 7fab6d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab68100ab0 0x7fab68197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:32.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.643+0000 7fab6d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab681013c0 0x7fab68197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.643+0000 7fab6d400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab681981c0 con 0x7fab68100ab0 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.643+0000 7fab6d400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab68198330 con 0x7fab681013c0 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.643+0000 7fab66800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab68100ab0 0x7fab68197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.643+0000 7fab66800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab68100ab0 0x7fab68197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60382/0 (socket says 172.21.15.115:60382) 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.643+0000 7fab66800640 1 -- 172.21.15.115:0/2132343302 learned_addr learned my addr 172.21.15.115:0/2132343302 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.644+0000 7fab66800640 1 -- 172.21.15.115:0/2132343302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab681013c0 msgr2=0x7fab68197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.644+0000 7fab66800640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab681013c0 0x7fab68197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.644+0000 7fab66800640 1 -- 172.21.15.115:0/2132343302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab54009560 con 0x7fab68100ab0 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.644+0000 7fab66800640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab68100ab0 0x7fab68197680 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c8a0 tx=0x7fab5800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.644+0000 7fab5f400640 1 -- 172.21.15.115:0/2132343302 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab58007b80 con 0x7fab68100ab0 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.644+0000 7fab5f400640 1 -- 172.21.15.115:0/2132343302 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab58007ce0 con 0x7fab68100ab0 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.644+0000 7fab5f400640 1 -- 172.21.15.115:0/2132343302 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab58018810 con 0x7fab68100ab0 2024-03-22T10:56:32.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.644+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6819c1c0 con 0x7fab68100ab0 2024-03-22T10:56:32.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.645+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6819c6d0 con 0x7fab68100ab0 2024-03-22T10:56:32.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.646+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6804e000 con 0x7fab68100ab0 2024-03-22T10:56:32.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.647+0000 7fab5f400640 1 -- 172.21.15.115:0/2132343302 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fab58018a80 con 0x7fab68100ab0 2024-03-22T10:56:32.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.648+0000 7fab5f400640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab4c06dfd0 0x7fab4c070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:32.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.648+0000 7fab5f400640 1 -- 172.21.15.115:0/2132343302 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fab5808e040 con 0x7fab68100ab0 2024-03-22T10:56:32.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.648+0000 7fab65e00640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab4c06dfd0 0x7fab4c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:32.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.649+0000 7fab65e00640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab4c06dfd0 0x7fab4c070480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fab54036040 tx=0x7fab54000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:32.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.653+0000 7fab5f400640 1 -- 172.21.15.115:0/2132343302 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab58057390 con 0x7fab68100ab0 2024-03-22T10:56:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.946+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab68066860 con 0x7fab68100ab0 2024-03-22T10:56:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.947+0000 7fab5f400640 1 -- 172.21.15.115:0/2132343302 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5072 (secure 0 0 0) 0x7fab5805aa50 con 0x7fab68100ab0 2024-03-22T10:56:32.941 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:32.941 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44297,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/1804848703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1804848703},{"type":"v1","addr":"172.21.15.150:6825","nonce":1804848703}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44321,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6827/2053213512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2053213512},{"type":"v1","addr":"172.21.15.150:6827","nonce":2053213512}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:52:05.899739+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34476,"mds_1":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34476":{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":35,"state":"up:active","state_seq":23,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34508":{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":1,"incarnation":39,"state":"up:active","state_seq":18,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34476,"qdb_cluster":[34476,34508]},"id":1}]} 2024-03-22T10:56:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2024-03-22T10:56:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.949+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab4c06dfd0 msgr2=0x7fab4c070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.949+0000 7fab6d400640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab4c06dfd0 0x7fab4c070480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fab54036040 tx=0x7fab54000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.949+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab68100ab0 msgr2=0x7fab68197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.949+0000 7fab6d400640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab68100ab0 0x7fab68197680 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c8a0 tx=0x7fab5800cd60 comp rx=0 tx=0).stop 2024-03-22T10:56:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.950+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 shutdown_connections 2024-03-22T10:56:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.950+0000 7fab6d400640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fab4c06dfd0 0x7fab4c070480 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.950+0000 7fab6d400640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab681013c0 0x7fab68197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.950+0000 7fab6d400640 1 --2- 172.21.15.115:0/2132343302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab68100ab0 0x7fab68197680 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.950+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 >> 172.21.15.115:0/2132343302 conn(0x7fab680fa300 msgr2=0x7fab680feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.950+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 shutdown_connections 2024-03-22T10:56:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:32.950+0000 7fab6d400640 1 -- 172.21.15.115:0/2132343302 wait complete. 2024-03-22T10:56:32.959 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 12, 'max_mds': 2, 'flags': 18} 2024-03-22T10:56:32.960 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 13 2024-03-22T10:56:33.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:32 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1242499411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T10:56:33.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:32 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2363933968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T10:56:33.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:32 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1242499411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T10:56:33.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:32 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2363933968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T10:56:33.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.162+0000 7f535b000640 1 -- 172.21.15.115:0/3063087124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 msgr2=0x7f5354101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:33.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.162+0000 7f535b000640 1 --2- 172.21.15.115:0/3063087124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 0x7f5354101b20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5340009880 tx=0x7f534002bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.162+0000 7f535b000640 1 -- 172.21.15.115:0/3063087124 shutdown_connections 2024-03-22T10:56:33.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.162+0000 7f535b000640 1 --2- 172.21.15.115:0/3063087124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 0x7f5354101b20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.162+0000 7f535b000640 1 --2- 172.21.15.115:0/3063087124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53541076c0 0x7f5354107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.162+0000 7f535b000640 1 -- 172.21.15.115:0/3063087124 >> 172.21.15.115:0/3063087124 conn(0x7f53540fd1b0 msgr2=0x7f53540ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.162+0000 7f535b000640 1 -- 172.21.15.115:0/3063087124 shutdown_connections 2024-03-22T10:56:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.162+0000 7f535b000640 1 -- 172.21.15.115:0/3063087124 wait complete. 2024-03-22T10:56:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.164+0000 7f535b000640 1 Processor -- start 2024-03-22T10:56:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.164+0000 7f535b000640 1 -- start start 2024-03-22T10:56:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.164+0000 7f535b000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 0x7f53541972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.164+0000 7f535b000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53541076c0 0x7f5354197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.164+0000 7f535b000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354197e30 con 0x7f53541016b0 2024-03-22T10:56:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.164+0000 7f535b000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354197fa0 con 0x7f53541076c0 2024-03-22T10:56:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.165+0000 7f5359c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 0x7f53541972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.165+0000 7f5359c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 0x7f53541972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60410/0 (socket says 172.21.15.115:60410) 2024-03-22T10:56:33.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.165+0000 7f5359c00640 1 -- 172.21.15.115:0/4096105721 learned_addr learned my addr 172.21.15.115:0/4096105721 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.165+0000 7f5359c00640 1 -- 172.21.15.115:0/4096105721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53541076c0 msgr2=0x7f5354197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.165+0000 7f5359200640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53541076c0 0x7f5354197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.165+0000 7f5359c00640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53541076c0 0x7f5354197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.165+0000 7f5359c00640 1 -- 172.21.15.115:0/4096105721 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5340009560 con 0x7f53541016b0 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.165+0000 7f5359c00640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 0x7f53541972f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f534800c8f0 tx=0x7f534800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.166+0000 7f5352000640 1 -- 172.21.15.115:0/4096105721 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53480041f0 con 0x7f53541016b0 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.166+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535419be30 con 0x7f53541016b0 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.166+0000 7f5352000640 1 -- 172.21.15.115:0/4096105721 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5348007e10 con 0x7f53541016b0 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.166+0000 7f5352000640 1 -- 172.21.15.115:0/4096105721 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f534800f450 con 0x7f53541016b0 2024-03-22T10:56:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.166+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535419c410 con 0x7f53541016b0 2024-03-22T10:56:33.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.167+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535404e000 con 0x7f53541016b0 2024-03-22T10:56:33.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.169+0000 7f5352000640 1 -- 172.21.15.115:0/4096105721 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5348007950 con 0x7f53541016b0 2024-03-22T10:56:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.170+0000 7f5352000640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f533c06df10 0x7f533c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.170+0000 7f5352000640 1 -- 172.21.15.115:0/4096105721 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f5348013070 con 0x7f53541016b0 2024-03-22T10:56:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.170+0000 7f5359200640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f533c06df10 0x7f533c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.171+0000 7f5359200640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f533c06df10 0x7f533c0703c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5340036040 tx=0x7f5340005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:33.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.175+0000 7f5352000640 1 -- 172.21.15.115:0/4096105721 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5348057280 con 0x7f53541016b0 2024-03-22T10:56:33.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.403+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 13, "format": "json"} v 0) v1 -- 0x7f5354198640 con 0x7f53541016b0 2024-03-22T10:56:33.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.405+0000 7f5352000640 1 -- 172.21.15.115:0/4096105721 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 13, "format": "json"}]=0 dumped fsmap epoch 13 v40) v1 ==== 107+0+4883 (secure 0 0 0) 0x7f534805a940 con 0x7f53541016b0 2024-03-22T10:56:33.399 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:33.399 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:41:04.070466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14454,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14454":{"gid":14454,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6827/3803356871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3803356871},{"type":"v1","addr":"172.21.15.115:6827","nonce":3803356871}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi150.zvyrzc","rank":1,"incarnation":6,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6825/1103557222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1103557222},{"type":"v1","addr":"172.21.15.150:6825","nonce":1103557222}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:33.399 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 13 2024-03-22T10:56:33.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f533c06df10 msgr2=0x7f533c0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:33.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f533c06df10 0x7f533c0703c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5340036040 tx=0x7f5340005fd0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 msgr2=0x7f53541972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:33.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 0x7f53541972f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f534800c8f0 tx=0x7f534800cdb0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 shutdown_connections 2024-03-22T10:56:33.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f533c06df10 0x7f533c0703c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53541076c0 0x7f5354197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 --2- 172.21.15.115:0/4096105721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53541016b0 0x7f53541972f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 >> 172.21.15.115:0/4096105721 conn(0x7f53540fd1b0 msgr2=0x7f53540fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:33.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 shutdown_connections 2024-03-22T10:56:33.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.407+0000 7f535b000640 1 -- 172.21.15.115:0/4096105721 wait complete. 2024-03-22T10:56:33.418 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-03-22T10:56:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.615+0000 7f4b59400640 1 -- 172.21.15.115:0/2483830936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 msgr2=0x7f4b541064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.615+0000 7f4b59400640 1 --2- 172.21.15.115:0/2483830936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 0x7f4b541064e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009880 tx=0x7f4b3c02bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.615+0000 7f4b59400640 1 -- 172.21.15.115:0/2483830936 shutdown_connections 2024-03-22T10:56:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.615+0000 7f4b59400640 1 --2- 172.21.15.115:0/2483830936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 0x7f4b541064e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.615+0000 7f4b59400640 1 --2- 172.21.15.115:0/2483830936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5410bb60 0x7f4b5410bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.615+0000 7f4b59400640 1 -- 172.21.15.115:0/2483830936 >> 172.21.15.115:0/2483830936 conn(0x7f4b54075260 msgr2=0x7f4b54077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.615+0000 7f4b59400640 1 -- 172.21.15.115:0/2483830936 shutdown_connections 2024-03-22T10:56:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.615+0000 7f4b59400640 1 -- 172.21.15.115:0/2483830936 wait complete. 2024-03-22T10:56:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.616+0000 7f4b59400640 1 Processor -- start 2024-03-22T10:56:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.616+0000 7f4b59400640 1 -- start start 2024-03-22T10:56:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b59400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 0x7f4b541a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b59400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5410bb60 0x7f4b541a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b59400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419b690 con 0x7f4b54106090 2024-03-22T10:56:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b59400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419b800 con 0x7f4b5410bb60 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b53e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 0x7f4b541a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b53e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 0x7f4b541a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60420/0 (socket says 172.21.15.115:60420) 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b53e00640 1 -- 172.21.15.115:0/2104735694 learned_addr learned my addr 172.21.15.115:0/2104735694 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b4be00640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5410bb60 0x7f4b541a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b53e00640 1 -- 172.21.15.115:0/2104735694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5410bb60 msgr2=0x7f4b541a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b53e00640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5410bb60 0x7f4b541a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.617+0000 7f4b53e00640 1 -- 172.21.15.115:0/2104735694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c009560 con 0x7f4b54106090 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.618+0000 7f4b53e00640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 0x7f4b541a1620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d6f0 tx=0x7f4b4400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.618+0000 7f4b51600640 1 -- 172.21.15.115:0/2104735694 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b44009a30 con 0x7f4b54106090 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.618+0000 7f4b51600640 1 -- 172.21.15.115:0/2104735694 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b440054e0 con 0x7f4b54106090 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.618+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5419bae0 con 0x7f4b54106090 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.618+0000 7f4b51600640 1 -- 172.21.15.115:0/2104735694 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b44010460 con 0x7f4b54106090 2024-03-22T10:56:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.618+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5419c040 con 0x7f4b54106090 2024-03-22T10:56:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.620+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5404e000 con 0x7f4b54106090 2024-03-22T10:56:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.620+0000 7f4b51600640 1 -- 172.21.15.115:0/2104735694 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4b44009b90 con 0x7f4b54106090 2024-03-22T10:56:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.621+0000 7f4b51600640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4b3806df10 0x7f4b380703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.621+0000 7f4b51600640 1 -- 172.21.15.115:0/2104735694 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4b4408d920 con 0x7f4b54106090 2024-03-22T10:56:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.621+0000 7f4b4be00640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4b3806df10 0x7f4b380703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.622+0000 7f4b4be00640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4b3806df10 0x7f4b380703c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c036040 tx=0x7f4b3c005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:33.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.628+0000 7f4b51600640 1 -- 172.21.15.115:0/2104735694 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4b44056cf0 con 0x7f4b54106090 2024-03-22T10:56:33.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.891+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7f4b5410def0 con 0x7f4b54106090 2024-03-22T10:56:33.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.892+0000 7f4b51600640 1 -- 172.21.15.115:0/2104735694 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v40) v1 ==== 107+0+4884 (secure 0 0 0) 0x7f4b4405a3b0 con 0x7f4b54106090 2024-03-22T10:56:33.886 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:33.887 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:43:22.786262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14454,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14454":{"gid":14454,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6827/3803356871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3803356871},{"type":"v1","addr":"172.21.15.115:6827","nonce":3803356871}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi150.zvyrzc","rank":1,"incarnation":6,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6825/1103557222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1103557222},{"type":"v1","addr":"172.21.15.150:6825","nonce":1103557222}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:33.887 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 14 2024-03-22T10:56:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4b3806df10 msgr2=0x7f4b380703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4b3806df10 0x7f4b380703c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c036040 tx=0x7f4b3c005fd0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 msgr2=0x7f4b541a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 0x7f4b541a1620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d6f0 tx=0x7f4b4400dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 shutdown_connections 2024-03-22T10:56:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4b3806df10 0x7f4b380703c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b5410bb60 0x7f4b541a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 --2- 172.21.15.115:0/2104735694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b54106090 0x7f4b541a1620 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 >> 172.21.15.115:0/2104735694 conn(0x7f4b54075260 msgr2=0x7f4b54076d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 shutdown_connections 2024-03-22T10:56:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:33.895+0000 7f4b59400640 1 -- 172.21.15.115:0/2104735694 wait complete. 2024-03-22T10:56:33.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-03-22T10:56:34.068 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:33 smithi115 ceph-mon[78672]: pgmap v412: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:34.068 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:33 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2132343302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T10:56:34.068 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:33 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/4096105721' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-03-22T10:56:34.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.132+0000 7fe771200640 1 -- 172.21.15.115:0/504290433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c067310 msgr2=0x7fe76c067760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:34.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.132+0000 7fe771200640 1 --2- 172.21.15.115:0/504290433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c067310 0x7fe76c067760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe758009880 tx=0x7fe75802bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.132+0000 7fe771200640 1 -- 172.21.15.115:0/504290433 shutdown_connections 2024-03-22T10:56:34.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.132+0000 7fe771200640 1 --2- 172.21.15.115:0/504290433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c067310 0x7fe76c067760 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.132+0000 7fe771200640 1 --2- 172.21.15.115:0/504290433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe76c1055c0 0x7fe76c105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.132+0000 7fe771200640 1 -- 172.21.15.115:0/504290433 >> 172.21.15.115:0/504290433 conn(0x7fe76c0743d0 msgr2=0x7fe76c0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:34.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.132+0000 7fe771200640 1 -- 172.21.15.115:0/504290433 shutdown_connections 2024-03-22T10:56:34.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.132+0000 7fe771200640 1 -- 172.21.15.115:0/504290433 wait complete. 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.133+0000 7fe771200640 1 Processor -- start 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.133+0000 7fe771200640 1 -- start start 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.133+0000 7fe771200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe76c067310 0x7fe76c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.133+0000 7fe771200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c1055c0 0x7fe76c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.133+0000 7fe771200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c197f20 con 0x7fe76c1055c0 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.133+0000 7fe771200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c19bc80 con 0x7fe76c067310 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.134+0000 7fe76aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c1055c0 0x7fe76c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.134+0000 7fe76aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c1055c0 0x7fe76c197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60442/0 (socket says 172.21.15.115:60442) 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.134+0000 7fe76aa00640 1 -- 172.21.15.115:0/706116697 learned_addr learned my addr 172.21.15.115:0/706116697 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:34.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.134+0000 7fe76b400640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe76c067310 0x7fe76c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:34.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.134+0000 7fe76aa00640 1 -- 172.21.15.115:0/706116697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe76c067310 msgr2=0x7fe76c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:34.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.134+0000 7fe76aa00640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe76c067310 0x7fe76c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.134+0000 7fe76aa00640 1 -- 172.21.15.115:0/706116697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758009560 con 0x7fe76c1055c0 2024-03-22T10:56:34.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.135+0000 7fe76aa00640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c1055c0 0x7fe76c197890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe758009230 tx=0x7fe758004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:34.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.135+0000 7fe763e00640 1 -- 172.21.15.115:0/706116697 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7580035f0 con 0x7fe76c1055c0 2024-03-22T10:56:34.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.135+0000 7fe763e00640 1 -- 172.21.15.115:0/706116697 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe758003c10 con 0x7fe76c1055c0 2024-03-22T10:56:34.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.135+0000 7fe763e00640 1 -- 172.21.15.115:0/706116697 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe758005670 con 0x7fe76c1055c0 2024-03-22T10:56:34.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.135+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76c19bf00 con 0x7fe76c1055c0 2024-03-22T10:56:34.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.135+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c19c3b0 con 0x7fe76c1055c0 2024-03-22T10:56:34.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.137+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76c04e000 con 0x7fe76c1055c0 2024-03-22T10:56:34.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.138+0000 7fe763e00640 1 -- 172.21.15.115:0/706116697 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe758004dd0 con 0x7fe76c1055c0 2024-03-22T10:56:34.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.139+0000 7fe763e00640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe75006df10 0x7fe7500703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:34.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.139+0000 7fe763e00640 1 -- 172.21.15.115:0/706116697 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe7580b0150 con 0x7fe76c1055c0 2024-03-22T10:56:34.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.140+0000 7fe76b400640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe75006df10 0x7fe7500703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:34.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.140+0000 7fe76b400640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe75006df10 0x7fe7500703c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005fd0 tx=0x7fe75c005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.144+0000 7fe763e00640 1 -- 172.21.15.115:0/706116697 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe7580795a0 con 0x7fe76c1055c0 2024-03-22T10:56:34.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:33 smithi150 ceph-mon[65513]: pgmap v412: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:34.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:33 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2132343302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T10:56:34.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:33 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/4096105721' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-03-22T10:56:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.441+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7fe76c1986a0 con 0x7fe76c1055c0 2024-03-22T10:56:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.442+0000 7fe763e00640 1 -- 172.21.15.115:0/706116697 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v40) v1 ==== 107+0+4898 (secure 0 0 0) 0x7fe75807cc60 con 0x7fe76c1055c0 2024-03-22T10:56:34.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:34.436 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:46:58.862168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14454,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14454":{"gid":14454,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6827/3803356871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3803356871},{"type":"v1","addr":"172.21.15.115:6827","nonce":3803356871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi150.zvyrzc","rank":1,"incarnation":6,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6825/1103557222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1103557222},{"type":"v1","addr":"172.21.15.150:6825","nonce":1103557222}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14454,"qdb_cluster":[14454,24241]},"id":1}]} 2024-03-22T10:56:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 15 2024-03-22T10:56:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe75006df10 msgr2=0x7fe7500703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe75006df10 0x7fe7500703c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005fd0 tx=0x7fe75c005850 comp rx=0 tx=0).stop 2024-03-22T10:56:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c1055c0 msgr2=0x7fe76c197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c1055c0 0x7fe76c197890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe758009230 tx=0x7fe758004040 comp rx=0 tx=0).stop 2024-03-22T10:56:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 shutdown_connections 2024-03-22T10:56:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe75006df10 0x7fe7500703c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe76c1055c0 0x7fe76c197890 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 --2- 172.21.15.115:0/706116697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe76c067310 0x7fe76c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 >> 172.21.15.115:0/706116697 conn(0x7fe76c0743d0 msgr2=0x7fe76c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 shutdown_connections 2024-03-22T10:56:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.446+0000 7fe771200640 1 -- 172.21.15.115:0/706116697 wait complete. 2024-03-22T10:56:34.461 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-03-22T10:56:34.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.662+0000 7f71a4200640 1 -- 172.21.15.115:0/2469691939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0fe8b0 msgr2=0x7f719c0fec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:34.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.662+0000 7f71a4200640 1 --2- 172.21.15.115:0/2469691939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0fe8b0 0x7f719c0fec80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7190009880 tx=0x7f719002bd90 comp rx=0 tx=0).stop 2024-03-22T10:56:34.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.662+0000 7f71a4200640 1 -- 172.21.15.115:0/2469691939 shutdown_connections 2024-03-22T10:56:34.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.662+0000 7f71a4200640 1 --2- 172.21.15.115:0/2469691939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f719c0ff1c0 0x7f719c103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.662+0000 7f71a4200640 1 --2- 172.21.15.115:0/2469691939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0fe8b0 0x7f719c0fec80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.662+0000 7f71a4200640 1 -- 172.21.15.115:0/2469691939 >> 172.21.15.115:0/2469691939 conn(0x7f719c0fa2f0 msgr2=0x7f719c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:34.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.662+0000 7f71a4200640 1 -- 172.21.15.115:0/2469691939 shutdown_connections 2024-03-22T10:56:34.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.662+0000 7f71a4200640 1 -- 172.21.15.115:0/2469691939 wait complete. 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.663+0000 7f71a4200640 1 Processor -- start 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.663+0000 7f71a4200640 1 -- start start 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.664+0000 7f71a4200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f719c0fe8b0 0x7f719c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.664+0000 7f71a4200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0ff1c0 0x7f719c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.664+0000 7f71a4200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c19b9d0 con 0x7f719c0ff1c0 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.664+0000 7f71a4200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c19bb40 con 0x7f719c0fe8b0 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.664+0000 7f71a0e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0ff1c0 0x7f719c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.664+0000 7f71a0e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0ff1c0 0x7f719c1a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60470/0 (socket says 172.21.15.115:60470) 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.664+0000 7f71a0e00640 1 -- 172.21.15.115:0/4201533535 learned_addr learned my addr 172.21.15.115:0/4201533535 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:34.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f71a1800640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f719c0fe8b0 0x7f719c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:34.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f71a0e00640 1 -- 172.21.15.115:0/4201533535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f719c0fe8b0 msgr2=0x7f719c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:34.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f71a0e00640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f719c0fe8b0 0x7f719c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f71a0e00640 1 -- 172.21.15.115:0/4201533535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190009560 con 0x7f719c0ff1c0 2024-03-22T10:56:34.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f71a0e00640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0ff1c0 0x7f719c1a1ea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f718800d6f0 tx=0x7f718800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:34.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f719a000640 1 -- 172.21.15.115:0/4201533535 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7188009a30 con 0x7f719c0ff1c0 2024-03-22T10:56:34.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f719a000640 1 -- 172.21.15.115:0/4201533535 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71880054e0 con 0x7f719c0ff1c0 2024-03-22T10:56:34.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c19be20 con 0x7f719c0ff1c0 2024-03-22T10:56:34.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f719a000640 1 -- 172.21.15.115:0/4201533535 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7188010460 con 0x7f719c0ff1c0 2024-03-22T10:56:34.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.665+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c19c380 con 0x7f719c0ff1c0 2024-03-22T10:56:34.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.667+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c04e000 con 0x7f719c0ff1c0 2024-03-22T10:56:34.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.670+0000 7f719a000640 1 -- 172.21.15.115:0/4201533535 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f7188009b90 con 0x7f719c0ff1c0 2024-03-22T10:56:34.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.671+0000 7f719a000640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718406df10 0x7f71840703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:34.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.671+0000 7f719a000640 1 -- 172.21.15.115:0/4201533535 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f718808dd20 con 0x7f719c0ff1c0 2024-03-22T10:56:34.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.671+0000 7f71a1800640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718406df10 0x7f71840703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:34.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.672+0000 7f71a1800640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718406df10 0x7f71840703c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7190005ab0 tx=0x7f7190005a40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:34.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.676+0000 7f719a000640 1 -- 172.21.15.115:0/4201533535 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7188057230 con 0x7f719c0ff1c0 2024-03-22T10:56:34.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.983+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f719c066860 con 0x7f719c0ff1c0 2024-03-22T10:56:34.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.984+0000 7f719a000640 1 -- 172.21.15.115:0/4201533535 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v40) v1 ==== 107+0+4897 (secure 0 0 0) 0x7f7188020090 con 0x7f719c0ff1c0 2024-03-22T10:56:34.979 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:34.979 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:47:17.573634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14454,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14454":{"gid":14454,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6827/3803356871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3803356871},{"type":"v1","addr":"172.21.15.115:6827","nonce":3803356871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi150.zvyrzc","rank":1,"incarnation":6,"state":"up:active","state_seq":5,"addr":"172.21.15.150:6825/1103557222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1103557222},{"type":"v1","addr":"172.21.15.150:6825","nonce":1103557222}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14454,"qdb_cluster":[24241,14454]},"id":1}]} 2024-03-22T10:56:34.979 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2024-03-22T10:56:34.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718406df10 msgr2=0x7f71840703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:34.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718406df10 0x7f71840703c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7190005ab0 tx=0x7f7190005a40 comp rx=0 tx=0).stop 2024-03-22T10:56:34.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0ff1c0 msgr2=0x7f719c1a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:34.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0ff1c0 0x7f719c1a1ea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f718800d6f0 tx=0x7f718800dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 shutdown_connections 2024-03-22T10:56:34.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f718406df10 0x7f71840703c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719c0ff1c0 0x7f719c1a1ea0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 --2- 172.21.15.115:0/4201533535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f719c0fe8b0 0x7f719c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:34.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 >> 172.21.15.115:0/4201533535 conn(0x7f719c0fa2f0 msgr2=0x7f719c10fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:34.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 shutdown_connections 2024-03-22T10:56:34.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:34.987+0000 7f71a4200640 1 -- 172.21.15.115:0/4201533535 wait complete. 2024-03-22T10:56:35.003 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-03-22T10:56:35.072 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:34 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2104735694' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-03-22T10:56:35.072 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:56:35.072 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:34 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/706116697' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-03-22T10:56:35.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:34 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2104735694' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-03-22T10:56:35.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:56:35.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:34 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/706116697' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-03-22T10:56:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.228+0000 7fd080a00640 1 -- 172.21.15.115:0/3291025686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 msgr2=0x7fd07c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.228+0000 7fd080a00640 1 --2- 172.21.15.115:0/3291025686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 0x7fd07c105890 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009880 tx=0x7fd06c02bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.229+0000 7fd080a00640 1 -- 172.21.15.115:0/3291025686 shutdown_connections 2024-03-22T10:56:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.229+0000 7fd080a00640 1 --2- 172.21.15.115:0/3291025686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd07c0ff4b0 0x7fd07c0ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.229+0000 7fd080a00640 1 --2- 172.21.15.115:0/3291025686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 0x7fd07c105890 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.229+0000 7fd080a00640 1 -- 172.21.15.115:0/3291025686 >> 172.21.15.115:0/3291025686 conn(0x7fd07c0fafb0 msgr2=0x7fd07c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.229+0000 7fd080a00640 1 -- 172.21.15.115:0/3291025686 shutdown_connections 2024-03-22T10:56:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.229+0000 7fd080a00640 1 -- 172.21.15.115:0/3291025686 wait complete. 2024-03-22T10:56:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.230+0000 7fd080a00640 1 Processor -- start 2024-03-22T10:56:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.230+0000 7fd080a00640 1 -- start start 2024-03-22T10:56:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.230+0000 7fd080a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd07c0ff4b0 0x7fd07c195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.230+0000 7fd080a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 0x7fd07c1956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.230+0000 7fd080a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c195d50 con 0x7fd07c1054c0 2024-03-22T10:56:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.230+0000 7fd080a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c199b20 con 0x7fd07c0ff4b0 2024-03-22T10:56:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.231+0000 7fd073e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 0x7fd07c1956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:35.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.231+0000 7fd073e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 0x7fd07c1956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60482/0 (socket says 172.21.15.115:60482) 2024-03-22T10:56:35.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.231+0000 7fd073e00640 1 -- 172.21.15.115:0/1828883247 learned_addr learned my addr 172.21.15.115:0/1828883247 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:35.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.231+0000 7fd07b400640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd07c0ff4b0 0x7fd07c195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:35.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.231+0000 7fd073e00640 1 -- 172.21.15.115:0/1828883247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd07c0ff4b0 msgr2=0x7fd07c195180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:35.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.231+0000 7fd073e00640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd07c0ff4b0 0x7fd07c195180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.231+0000 7fd073e00640 1 -- 172.21.15.115:0/1828883247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c009560 con 0x7fd07c1054c0 2024-03-22T10:56:35.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.231+0000 7fd073e00640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 0x7fd07c1956c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd06400d6f0 tx=0x7fd06400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:35.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.232+0000 7fd078c00640 1 -- 172.21.15.115:0/1828883247 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd064009a30 con 0x7fd07c1054c0 2024-03-22T10:56:35.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.232+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c199e00 con 0x7fd07c1054c0 2024-03-22T10:56:35.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.232+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07c19a360 con 0x7fd07c1054c0 2024-03-22T10:56:35.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.232+0000 7fd078c00640 1 -- 172.21.15.115:0/1828883247 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0640054e0 con 0x7fd07c1054c0 2024-03-22T10:56:35.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.232+0000 7fd078c00640 1 -- 172.21.15.115:0/1828883247 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd064010460 con 0x7fd07c1054c0 2024-03-22T10:56:35.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.233+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07c04e000 con 0x7fd07c1054c0 2024-03-22T10:56:35.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.236+0000 7fd078c00640 1 -- 172.21.15.115:0/1828883247 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fd064009b90 con 0x7fd07c1054c0 2024-03-22T10:56:35.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.237+0000 7fd078c00640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd06006df10 0x7fd0600703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:35.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.237+0000 7fd078c00640 1 -- 172.21.15.115:0/1828883247 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd06408da10 con 0x7fd07c1054c0 2024-03-22T10:56:35.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.237+0000 7fd07b400640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd06006df10 0x7fd0600703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:35.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.238+0000 7fd07b400640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd06006df10 0x7fd0600703c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd06c006010 tx=0x7fd06c000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:35.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.247+0000 7fd078c00640 1 -- 172.21.15.115:0/1828883247 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd064056d60 con 0x7fd07c1054c0 2024-03-22T10:56:35.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.588+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fd07c107990 con 0x7fd07c1054c0 2024-03-22T10:56:35.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.589+0000 7fd078c00640 1 -- 172.21.15.115:0/1828883247 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v40) v1 ==== 107+0+4893 (secure 0 0 0) 0x7fd06405a420 con 0x7fd07c1054c0 2024-03-22T10:56:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:35.584 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:49:29.807826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14454,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14454":{"gid":14454,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6827/3803356871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3803356871},{"type":"v1","addr":"172.21.15.115:6827","nonce":3803356871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi150.zvyrzc","rank":1,"incarnation":6,"state":"up:stopping","state_seq":5,"addr":"172.21.15.150:6825/1103557222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1103557222},{"type":"v1","addr":"172.21.15.150:6825","nonce":1103557222}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14454,"qdb_cluster":[14454]},"id":1}]} 2024-03-22T10:56:35.584 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 17 2024-03-22T10:56:35.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.591+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd06006df10 msgr2=0x7fd0600703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.591+0000 7fd080a00640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd06006df10 0x7fd0600703c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd06c006010 tx=0x7fd06c000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.591+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 msgr2=0x7fd07c1956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.591+0000 7fd080a00640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 0x7fd07c1956c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd06400d6f0 tx=0x7fd06400dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.591+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 shutdown_connections 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.591+0000 7fd080a00640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fd06006df10 0x7fd0600703c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.591+0000 7fd080a00640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c1054c0 0x7fd07c1956c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.591+0000 7fd080a00640 1 --2- 172.21.15.115:0/1828883247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd07c0ff4b0 0x7fd07c195180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.591+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 >> 172.21.15.115:0/1828883247 conn(0x7fd07c0fafb0 msgr2=0x7fd07c0fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.592+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 shutdown_connections 2024-03-22T10:56:35.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.592+0000 7fd080a00640 1 -- 172.21.15.115:0/1828883247 wait complete. 2024-03-22T10:56:35.611 DEBUG:tasks.fs:max_mds reduced in epoch 17 2024-03-22T10:56:35.612 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-22T10:56:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.839+0000 7fde4d400640 1 -- 172.21.15.115:0/4169784190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 msgr2=0x7fde48105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.839+0000 7fde4d400640 1 --2- 172.21.15.115:0/4169784190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 0x7fde48105890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fde38009880 tx=0x7fde3802bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.840+0000 7fde4d400640 1 -- 172.21.15.115:0/4169784190 shutdown_connections 2024-03-22T10:56:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.840+0000 7fde4d400640 1 --2- 172.21.15.115:0/4169784190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde480ff4b0 0x7fde480ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.840+0000 7fde4d400640 1 --2- 172.21.15.115:0/4169784190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 0x7fde48105890 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.840+0000 7fde4d400640 1 -- 172.21.15.115:0/4169784190 >> 172.21.15.115:0/4169784190 conn(0x7fde480fafb0 msgr2=0x7fde480fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.840+0000 7fde4d400640 1 -- 172.21.15.115:0/4169784190 shutdown_connections 2024-03-22T10:56:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.840+0000 7fde4d400640 1 -- 172.21.15.115:0/4169784190 wait complete. 2024-03-22T10:56:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.841+0000 7fde4d400640 1 Processor -- start 2024-03-22T10:56:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.841+0000 7fde4d400640 1 -- start start 2024-03-22T10:56:35.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.842+0000 7fde4d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde480ff4b0 0x7fde48072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:35.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.842+0000 7fde4d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 0x7fde48072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:35.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.842+0000 7fde4d400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde48072fe0 con 0x7fde481054c0 2024-03-22T10:56:35.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.842+0000 7fde4d400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde48073150 con 0x7fde480ff4b0 2024-03-22T10:56:35.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.842+0000 7fde47400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 0x7fde48072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:35.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.842+0000 7fde47400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 0x7fde48072aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60504/0 (socket says 172.21.15.115:60504) 2024-03-22T10:56:35.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.842+0000 7fde47400640 1 -- 172.21.15.115:0/2400957331 learned_addr learned my addr 172.21.15.115:0/2400957331 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:35.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.842+0000 7fde47e00640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde480ff4b0 0x7fde48072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:35.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.843+0000 7fde47400640 1 -- 172.21.15.115:0/2400957331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde480ff4b0 msgr2=0x7fde48072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:35.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.843+0000 7fde47400640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde480ff4b0 0x7fde48072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:35.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.843+0000 7fde47400640 1 -- 172.21.15.115:0/2400957331 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde38009560 con 0x7fde481054c0 2024-03-22T10:56:35.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.843+0000 7fde47400640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 0x7fde48072aa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d6f0 tx=0x7fde3400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:35.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.843+0000 7fde44c00640 1 -- 172.21.15.115:0/2400957331 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde34009a30 con 0x7fde481054c0 2024-03-22T10:56:35.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.843+0000 7fde44c00640 1 -- 172.21.15.115:0/2400957331 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde340054e0 con 0x7fde481054c0 2024-03-22T10:56:35.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.843+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde480733d0 con 0x7fde481054c0 2024-03-22T10:56:35.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.843+0000 7fde44c00640 1 -- 172.21.15.115:0/2400957331 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde34010460 con 0x7fde481054c0 2024-03-22T10:56:35.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.843+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde481a1710 con 0x7fde481054c0 2024-03-22T10:56:35.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.845+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4804e000 con 0x7fde481054c0 2024-03-22T10:56:35.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.846+0000 7fde44c00640 1 -- 172.21.15.115:0/2400957331 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fde34009b90 con 0x7fde481054c0 2024-03-22T10:56:35.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.847+0000 7fde44c00640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde2c06df10 0x7fde2c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:35.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.847+0000 7fde44c00640 1 -- 172.21.15.115:0/2400957331 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fde3408d870 con 0x7fde481054c0 2024-03-22T10:56:35.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.847+0000 7fde47e00640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde2c06df10 0x7fde2c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:35.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.848+0000 7fde47e00640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde2c06df10 0x7fde2c0703c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fde38000c00 tx=0x7fde3802cf90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:35.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:35.853+0000 7fde44c00640 1 -- 172.21.15.115:0/2400957331 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde34056bc0 con 0x7fde481054c0 2024-03-22T10:56:36.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:35 smithi115 ceph-mon[78672]: pgmap v413: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:36.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:35 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/4201533535' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-03-22T10:56:36.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:35 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1828883247' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-22T10:56:36.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.130+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7fde48107990 con 0x7fde481054c0 2024-03-22T10:56:36.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.130+0000 7fde44c00640 1 -- 172.21.15.115:0/2400957331 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v40) v1 ==== 107+0+4894 (secure 0 0 0) 0x7fde3405a280 con 0x7fde481054c0 2024-03-22T10:56:36.125 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:36.125 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:49:37.577141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14454,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14454":{"gid":14454,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6827/3803356871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3803356871},{"type":"v1","addr":"172.21.15.115:6827","nonce":3803356871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi150.zvyrzc","rank":1,"incarnation":6,"state":"up:stopping","state_seq":5,"addr":"172.21.15.150:6825/1103557222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1103557222},{"type":"v1","addr":"172.21.15.150:6825","nonce":1103557222}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14454,"qdb_cluster":[14454]},"id":1}]} 2024-03-22T10:56:36.125 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 18 2024-03-22T10:56:36.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde2c06df10 msgr2=0x7fde2c0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:36.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde2c06df10 0x7fde2c0703c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fde38000c00 tx=0x7fde3802cf90 comp rx=0 tx=0).stop 2024-03-22T10:56:36.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 msgr2=0x7fde48072aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:36.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 0x7fde48072aa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d6f0 tx=0x7fde3400dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 shutdown_connections 2024-03-22T10:56:36.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fde2c06df10 0x7fde2c0703c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde481054c0 0x7fde48072aa0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 --2- 172.21.15.115:0/2400957331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde480ff4b0 0x7fde48072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 >> 172.21.15.115:0/2400957331 conn(0x7fde480fafb0 msgr2=0x7fde480fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:36.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 shutdown_connections 2024-03-22T10:56:36.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.134+0000 7fde4d400640 1 -- 172.21.15.115:0/2400957331 wait complete. 2024-03-22T10:56:36.153 DEBUG:tasks.fs:max_mds reduced in epoch 18 2024-03-22T10:56:36.153 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-22T10:56:36.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:35 smithi150 ceph-mon[65513]: pgmap v413: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:36.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:35 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/4201533535' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-03-22T10:56:36.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:35 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1828883247' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-22T10:56:36.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.358+0000 7f2a60e00640 1 -- 172.21.15.115:0/53702834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 msgr2=0x7f2a5c10bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:36.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.359+0000 7f2a60e00640 1 --2- 172.21.15.115:0/53702834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 0x7f2a5c10bbd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009880 tx=0x7f2a4802bd70 comp rx=0 tx=0).stop 2024-03-22T10:56:36.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.359+0000 7f2a60e00640 1 -- 172.21.15.115:0/53702834 shutdown_connections 2024-03-22T10:56:36.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.359+0000 7f2a60e00640 1 --2- 172.21.15.115:0/53702834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 0x7f2a5c10bbd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.359+0000 7f2a60e00640 1 --2- 172.21.15.115:0/53702834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a5c0725a0 0x7f2a5c072970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.359+0000 7f2a60e00640 1 -- 172.21.15.115:0/53702834 >> 172.21.15.115:0/53702834 conn(0x7f2a5c0fb370 msgr2=0x7f2a5c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:36.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.359+0000 7f2a60e00640 1 -- 172.21.15.115:0/53702834 shutdown_connections 2024-03-22T10:56:36.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.359+0000 7f2a60e00640 1 -- 172.21.15.115:0/53702834 wait complete. 2024-03-22T10:56:36.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.360+0000 7f2a60e00640 1 Processor -- start 2024-03-22T10:56:36.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.360+0000 7f2a60e00640 1 -- start start 2024-03-22T10:56:36.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.360+0000 7f2a60e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a5c0725a0 0x7f2a5c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.360+0000 7f2a60e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 0x7f2a5c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.360+0000 7f2a60e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c1975b0 con 0x7f2a5c072eb0 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.360+0000 7f2a60e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c197720 con 0x7f2a5c0725a0 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.361+0000 7f2a59400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 0x7f2a5c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.361+0000 7f2a59400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 0x7f2a5c19da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60528/0 (socket says 172.21.15.115:60528) 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.361+0000 7f2a59400640 1 -- 172.21.15.115:0/2978363294 learned_addr learned my addr 172.21.15.115:0/2978363294 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.361+0000 7f2a59e00640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a5c0725a0 0x7f2a5c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.361+0000 7f2a59400640 1 -- 172.21.15.115:0/2978363294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a5c0725a0 msgr2=0x7f2a5c19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.361+0000 7f2a59400640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a5c0725a0 0x7f2a5c19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.361+0000 7f2a59400640 1 -- 172.21.15.115:0/2978363294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48009560 con 0x7f2a5c072eb0 2024-03-22T10:56:36.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.362+0000 7f2a59400640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 0x7f2a5c19da80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a48036040 tx=0x7f2a4802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:36.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.362+0000 7f2a52a00640 1 -- 172.21.15.115:0/2978363294 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a480035f0 con 0x7f2a5c072eb0 2024-03-22T10:56:36.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.362+0000 7f2a52a00640 1 -- 172.21.15.115:0/2978363294 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a48003c10 con 0x7f2a5c072eb0 2024-03-22T10:56:36.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.362+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c1979a0 con 0x7f2a5c072eb0 2024-03-22T10:56:36.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.362+0000 7f2a52a00640 1 -- 172.21.15.115:0/2978363294 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a48005690 con 0x7f2a5c072eb0 2024-03-22T10:56:36.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.362+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c197f00 con 0x7f2a5c072eb0 2024-03-22T10:56:36.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.363+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c04e000 con 0x7f2a5c072eb0 2024-03-22T10:56:36.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.364+0000 7f2a52a00640 1 -- 172.21.15.115:0/2978363294 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f2a48004ed0 con 0x7f2a5c072eb0 2024-03-22T10:56:36.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.365+0000 7f2a52a00640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2a4006deb0 0x7f2a40070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:36.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.365+0000 7f2a52a00640 1 -- 172.21.15.115:0/2978363294 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2a480b0a20 con 0x7f2a5c072eb0 2024-03-22T10:56:36.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.365+0000 7f2a59e00640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2a4006deb0 0x7f2a40070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:36.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.366+0000 7f2a59e00640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2a4006deb0 0x7f2a40070360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c005fd0 tx=0x7f2a4c005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:36.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.374+0000 7f2a52a00640 1 -- 172.21.15.115:0/2978363294 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a48079df0 con 0x7f2a5c072eb0 2024-03-22T10:56:36.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.744+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f2a5c066860 con 0x7f2a5c072eb0 2024-03-22T10:56:36.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.745+0000 7f2a52a00640 1 -- 172.21.15.115:0/2978363294 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v40) v1 ==== 107+0+4095 (secure 0 0 0) 0x7f2a4807d4b0 con 0x7f2a5c072eb0 2024-03-22T10:56:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:36.744 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:02.579068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14454},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14454":{"gid":14454,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6827/3803356871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3803356871},{"type":"v1","addr":"172.21.15.115:6827","nonce":3803356871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14454,"qdb_cluster":[14454]},"id":1}]} 2024-03-22T10:56:36.744 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 19 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2a4006deb0 msgr2=0x7f2a40070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2a4006deb0 0x7f2a40070360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c005fd0 tx=0x7f2a4c005850 comp rx=0 tx=0).stop 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 msgr2=0x7f2a5c19da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 0x7f2a5c19da80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a48036040 tx=0x7f2a4802c8e0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 shutdown_connections 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f2a4006deb0 0x7f2a40070360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a5c072eb0 0x7f2a5c19da80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 --2- 172.21.15.115:0/2978363294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a5c0725a0 0x7f2a5c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 >> 172.21.15.115:0/2978363294 conn(0x7f2a5c0fb370 msgr2=0x7f2a5c106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:36.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 shutdown_connections 2024-03-22T10:56:36.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.749+0000 7f2a60e00640 1 -- 172.21.15.115:0/2978363294 wait complete. 2024-03-22T10:56:36.761 DEBUG:tasks.fs:max_mds reduced in epoch 19 2024-03-22T10:56:36.761 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-22T10:56:36.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.976+0000 7f6e2aa00640 1 -- 172.21.15.115:0/3680624114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e241054a0 msgr2=0x7f6e24105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:36.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.976+0000 7f6e2aa00640 1 --2- 172.21.15.115:0/3680624114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e241054a0 0x7f6e24105870 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009880 tx=0x7f6e1802bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.976+0000 7f6e2aa00640 1 -- 172.21.15.115:0/3680624114 shutdown_connections 2024-03-22T10:56:36.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.976+0000 7f6e2aa00640 1 --2- 172.21.15.115:0/3680624114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e240ff490 0x7f6e240ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.976+0000 7f6e2aa00640 1 --2- 172.21.15.115:0/3680624114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e241054a0 0x7f6e24105870 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.976+0000 7f6e2aa00640 1 -- 172.21.15.115:0/3680624114 >> 172.21.15.115:0/3680624114 conn(0x7f6e240fafb0 msgr2=0x7f6e240fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:36.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.976+0000 7f6e2aa00640 1 -- 172.21.15.115:0/3680624114 shutdown_connections 2024-03-22T10:56:36.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.976+0000 7f6e2aa00640 1 -- 172.21.15.115:0/3680624114 wait complete. 2024-03-22T10:56:36.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.977+0000 7f6e2aa00640 1 Processor -- start 2024-03-22T10:56:36.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.977+0000 7f6e2aa00640 1 -- start start 2024-03-22T10:56:36.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e2aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e240ff490 0x7f6e241951b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:36.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e2aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e241054a0 0x7f6e241956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:36.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e2aa00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24195d80 con 0x7f6e240ff490 2024-03-22T10:56:36.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e2aa00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24199b50 con 0x7f6e241054a0 2024-03-22T10:56:36.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e29600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e240ff490 0x7f6e241951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e29600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e240ff490 0x7f6e241951b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60542/0 (socket says 172.21.15.115:60542) 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e29600640 1 -- 172.21.15.115:0/2729371704 learned_addr learned my addr 172.21.15.115:0/2729371704 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e28c00640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e241054a0 0x7f6e241956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e29600640 1 -- 172.21.15.115:0/2729371704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e241054a0 msgr2=0x7f6e241956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e29600640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e241054a0 0x7f6e241956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.978+0000 7f6e29600640 1 -- 172.21.15.115:0/2729371704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18009560 con 0x7f6e240ff490 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.979+0000 7f6e29600640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e240ff490 0x7f6e241951b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e1802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.979+0000 7f6e22000640 1 -- 172.21.15.115:0/2729371704 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e18004520 con 0x7f6e240ff490 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.979+0000 7f6e22000640 1 -- 172.21.15.115:0/2729371704 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e18003ab0 con 0x7f6e240ff490 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.979+0000 7f6e22000640 1 -- 172.21.15.115:0/2729371704 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e18034440 con 0x7f6e240ff490 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.979+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e24199dd0 con 0x7f6e240ff490 2024-03-22T10:56:36.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.979+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2419a2d0 con 0x7f6e240ff490 2024-03-22T10:56:36.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.981+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2404e000 con 0x7f6e240ff490 2024-03-22T10:56:36.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.981+0000 7f6e22000640 1 -- 172.21.15.115:0/2729371704 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6e18003c20 con 0x7f6e240ff490 2024-03-22T10:56:36.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.982+0000 7f6e22000640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6e0c06df10 0x7f6e0c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:36.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.982+0000 7f6e22000640 1 -- 172.21.15.115:0/2729371704 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f6e180b00f0 con 0x7f6e240ff490 2024-03-22T10:56:36.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.983+0000 7f6e28c00640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6e0c06df10 0x7f6e0c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:36.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.983+0000 7f6e28c00640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6e0c06df10 0x7f6e0c0703c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e10005fd0 tx=0x7f6e10005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:36.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:36.987+0000 7f6e22000640 1 -- 172.21.15.115:0/2729371704 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6e18079540 con 0x7f6e240ff490 2024-03-22T10:56:37.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:36 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2400957331' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-22T10:56:37.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:36 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2978363294' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-22T10:56:37.093 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:36 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2400957331' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-22T10:56:37.093 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:36 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2978363294' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-22T10:56:37.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.354+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f6e240008d0 con 0x7f6e240ff490 2024-03-22T10:56:37.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.355+0000 7f6e22000640 1 -- 172.21.15.115:0/2729371704 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v40) v1 ==== 107+0+4877 (secure 0 0 0) 0x7f6e1807cc00 con 0x7f6e240ff490 2024-03-22T10:56:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:37.350 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:02.579068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14454},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14454":{"gid":14454,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6827/3803356871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3803356871},{"type":"v1","addr":"172.21.15.115:6827","nonce":3803356871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14454,"qdb_cluster":[14454]},"id":1}]} 2024-03-22T10:56:37.350 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 20 2024-03-22T10:56:37.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6e0c06df10 msgr2=0x7f6e0c0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:37.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6e0c06df10 0x7f6e0c0703c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e10005fd0 tx=0x7f6e10005850 comp rx=0 tx=0).stop 2024-03-22T10:56:37.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e240ff490 msgr2=0x7f6e241951b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:37.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e240ff490 0x7f6e241951b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e1802c850 comp rx=0 tx=0).stop 2024-03-22T10:56:37.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 shutdown_connections 2024-03-22T10:56:37.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6e0c06df10 0x7f6e0c0703c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e241054a0 0x7f6e241956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 --2- 172.21.15.115:0/2729371704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e240ff490 0x7f6e241951b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 >> 172.21.15.115:0/2729371704 conn(0x7f6e240fafb0 msgr2=0x7f6e240fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:37.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 shutdown_connections 2024-03-22T10:56:37.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.358+0000 7f6e2aa00640 1 -- 172.21.15.115:0/2729371704 wait complete. 2024-03-22T10:56:37.371 DEBUG:tasks.fs:max_mds reduced in epoch 20 2024-03-22T10:56:37.371 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-22T10:56:37.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.576+0000 7fdaa9600640 1 -- 172.21.15.115:0/767400861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 msgr2=0x7fdaa4107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:37.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.576+0000 7fdaa9600640 1 --2- 172.21.15.115:0/767400861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 0x7fdaa4107a90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fda94009880 tx=0x7fda9402bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.576+0000 7fdaa9600640 1 -- 172.21.15.115:0/767400861 shutdown_connections 2024-03-22T10:56:37.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.576+0000 7fdaa9600640 1 --2- 172.21.15.115:0/767400861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa41016b0 0x7fdaa4101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.576+0000 7fdaa9600640 1 --2- 172.21.15.115:0/767400861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 0x7fdaa4107a90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.576+0000 7fdaa9600640 1 -- 172.21.15.115:0/767400861 >> 172.21.15.115:0/767400861 conn(0x7fdaa40fd1b0 msgr2=0x7fdaa40ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:37.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.576+0000 7fdaa9600640 1 -- 172.21.15.115:0/767400861 shutdown_connections 2024-03-22T10:56:37.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.576+0000 7fdaa9600640 1 -- 172.21.15.115:0/767400861 wait complete. 2024-03-22T10:56:37.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.577+0000 7fdaa9600640 1 Processor -- start 2024-03-22T10:56:37.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.577+0000 7fdaa9600640 1 -- start start 2024-03-22T10:56:37.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.578+0000 7fdaa9600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa41016b0 0x7fdaa4197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:37.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.578+0000 7fdaa9600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 0x7fdaa4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:37.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.578+0000 7fdaa9600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4197ee0 con 0x7fdaa41076c0 2024-03-22T10:56:37.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.578+0000 7fdaa9600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa419bcb0 con 0x7fdaa41016b0 2024-03-22T10:56:37.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.578+0000 7fdaa3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 0x7fdaa4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:37.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.578+0000 7fdaa3400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 0x7fdaa4197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60570/0 (socket says 172.21.15.115:60570) 2024-03-22T10:56:37.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.578+0000 7fdaa3400640 1 -- 172.21.15.115:0/3515764875 learned_addr learned my addr 172.21.15.115:0/3515764875 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:37.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.578+0000 7fdaa3e00640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa41016b0 0x7fdaa4197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:37.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.579+0000 7fdaa3400640 1 -- 172.21.15.115:0/3515764875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa41016b0 msgr2=0x7fdaa4197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:37.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.579+0000 7fdaa3400640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa41016b0 0x7fdaa4197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.579+0000 7fdaa3400640 1 -- 172.21.15.115:0/3515764875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda94009560 con 0x7fdaa41076c0 2024-03-22T10:56:37.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.579+0000 7fdaa3400640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 0x7fdaa4197850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fda9000d6f0 tx=0x7fda9000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:37.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.579+0000 7fdaa0c00640 1 -- 172.21.15.115:0/3515764875 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda90009a30 con 0x7fdaa41076c0 2024-03-22T10:56:37.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.579+0000 7fdaa0c00640 1 -- 172.21.15.115:0/3515764875 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda900054e0 con 0x7fdaa41076c0 2024-03-22T10:56:37.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.579+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa419bf90 con 0x7fdaa41076c0 2024-03-22T10:56:37.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.579+0000 7fdaa0c00640 1 -- 172.21.15.115:0/3515764875 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda90010460 con 0x7fdaa41076c0 2024-03-22T10:56:37.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.580+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa419c570 con 0x7fdaa41076c0 2024-03-22T10:56:37.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.581+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa404e000 con 0x7fdaa41076c0 2024-03-22T10:56:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.583+0000 7fdaa0c00640 1 -- 172.21.15.115:0/3515764875 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fda90009b90 con 0x7fdaa41076c0 2024-03-22T10:56:37.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.584+0000 7fdaa0c00640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fda8806df10 0x7fda880703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:37.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.585+0000 7fdaa0c00640 1 -- 172.21.15.115:0/3515764875 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fda9008d920 con 0x7fdaa41076c0 2024-03-22T10:56:37.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.585+0000 7fdaa3e00640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fda8806df10 0x7fda880703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:37.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.586+0000 7fdaa3e00640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fda8806df10 0x7fda880703c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fda94006010 tx=0x7fda94000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:37.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.590+0000 7fdaa0c00640 1 -- 172.21.15.115:0/3515764875 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fda90056cf0 con 0x7fdaa41076c0 2024-03-22T10:56:37.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.844+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fdaa4198660 con 0x7fdaa41076c0 2024-03-22T10:56:37.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.845+0000 7fdaa0c00640 1 -- 172.21.15.115:0/3515764875 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v40) v1 ==== 107+0+4077 (secure 0 0 0) 0x7fda9005a3b0 con 0x7fdaa41076c0 2024-03-22T10:56:37.839 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:37.840 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:18.120737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:37.840 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 21 2024-03-22T10:56:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fda8806df10 msgr2=0x7fda880703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fda8806df10 0x7fda880703c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fda94006010 tx=0x7fda94000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 msgr2=0x7fdaa4197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 0x7fdaa4197850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fda9000d6f0 tx=0x7fda9000dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 shutdown_connections 2024-03-22T10:56:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fda8806df10 0x7fda880703c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa41076c0 0x7fdaa4197850 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 --2- 172.21.15.115:0/3515764875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa41016b0 0x7fdaa4197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 >> 172.21.15.115:0/3515764875 conn(0x7fdaa40fd1b0 msgr2=0x7fdaa40feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 shutdown_connections 2024-03-22T10:56:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:37.849+0000 7fdaa9600640 1 -- 172.21.15.115:0/3515764875 wait complete. 2024-03-22T10:56:37.863 DEBUG:tasks.fs:max_mds reduced in epoch 21 2024-03-22T10:56:37.863 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-22T10:56:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.089+0000 7f6b05800640 1 -- 172.21.15.115:0/3426678614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b001054a0 msgr2=0x7f6b00105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.089+0000 7f6b05800640 1 --2- 172.21.15.115:0/3426678614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b001054a0 0x7f6b00105870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009880 tx=0x7f6af002bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.090+0000 7f6b05800640 1 -- 172.21.15.115:0/3426678614 shutdown_connections 2024-03-22T10:56:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.090+0000 7f6b05800640 1 --2- 172.21.15.115:0/3426678614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b000ff490 0x7f6b000ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.090+0000 7f6b05800640 1 --2- 172.21.15.115:0/3426678614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b001054a0 0x7f6b00105870 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.090+0000 7f6b05800640 1 -- 172.21.15.115:0/3426678614 >> 172.21.15.115:0/3426678614 conn(0x7f6b000fafb0 msgr2=0x7f6b000fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.090+0000 7f6b05800640 1 -- 172.21.15.115:0/3426678614 shutdown_connections 2024-03-22T10:56:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.090+0000 7f6b05800640 1 -- 172.21.15.115:0/3426678614 wait complete. 2024-03-22T10:56:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.091+0000 7f6b05800640 1 Processor -- start 2024-03-22T10:56:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.091+0000 7f6b05800640 1 -- start start 2024-03-22T10:56:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.092+0000 7f6b05800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b000ff490 0x7f6b001951b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.092+0000 7f6b05800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b001054a0 0x7f6b001956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.092+0000 7f6b05800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00195d80 con 0x7f6b000ff490 2024-03-22T10:56:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.092+0000 7f6b05800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00199b50 con 0x7f6b001054a0 2024-03-22T10:56:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.092+0000 7f6affe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b000ff490 0x7f6b001951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.092+0000 7f6affe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b000ff490 0x7f6b001951b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60586/0 (socket says 172.21.15.115:60586) 2024-03-22T10:56:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.092+0000 7f6affe00640 1 -- 172.21.15.115:0/216068522 learned_addr learned my addr 172.21.15.115:0/216068522 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.093+0000 7f6affe00640 1 -- 172.21.15.115:0/216068522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b001054a0 msgr2=0x7f6b001956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.093+0000 7f6affe00640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b001054a0 0x7f6b001956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.093+0000 7f6affe00640 1 -- 172.21.15.115:0/216068522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af0009560 con 0x7f6b000ff490 2024-03-22T10:56:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.093+0000 7f6affe00640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b000ff490 0x7f6b001951b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009230 tx=0x7f6af0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.093+0000 7f6afcc00640 1 -- 172.21.15.115:0/216068522 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af00035f0 con 0x7f6b000ff490 2024-03-22T10:56:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.093+0000 7f6afcc00640 1 -- 172.21.15.115:0/216068522 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6af0003c10 con 0x7f6b000ff490 2024-03-22T10:56:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.093+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b00199dd0 con 0x7f6b000ff490 2024-03-22T10:56:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.093+0000 7f6afcc00640 1 -- 172.21.15.115:0/216068522 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af0034440 con 0x7f6b000ff490 2024-03-22T10:56:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.093+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0019a2d0 con 0x7f6b000ff490 2024-03-22T10:56:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.095+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0004e000 con 0x7f6b000ff490 2024-03-22T10:56:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.097+0000 7f6afcc00640 1 -- 172.21.15.115:0/216068522 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6af0004fd0 con 0x7f6b000ff490 2024-03-22T10:56:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.098+0000 7f6afcc00640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6ae406df10 0x7f6ae40703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.098+0000 7f6afcc00640 1 -- 172.21.15.115:0/216068522 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f6af00b02b0 con 0x7f6b000ff490 2024-03-22T10:56:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.098+0000 7f6aff400640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6ae406df10 0x7f6ae40703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.099+0000 7f6aff400640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6ae406df10 0x7f6ae40703c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6b001967c0 tx=0x7f6aec005bf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.103+0000 7f6afcc00640 1 -- 172.21.15.115:0/216068522 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6af0079640 con 0x7f6b000ff490 2024-03-22T10:56:38.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:37 smithi150 ceph-mon[65513]: pgmap v414: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:38.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:37 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2729371704' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-22T10:56:38.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:37 smithi115 ceph-mon[78672]: pgmap v414: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:38.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:37 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2729371704' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-22T10:56:38.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.364+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f6b00196500 con 0x7f6b000ff490 2024-03-22T10:56:38.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.365+0000 7f6afcc00640 1 -- 172.21.15.115:0/216068522 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v40) v1 ==== 107+0+4088 (secure 0 0 0) 0x7f6af007cd00 con 0x7f6b000ff490 2024-03-22T10:56:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:38.360 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:18.128814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":0,"incarnation":22,"state":"up:replay","state_seq":3,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:38.360 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 22 2024-03-22T10:56:38.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6ae406df10 msgr2=0x7f6ae40703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6ae406df10 0x7f6ae40703c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6b001967c0 tx=0x7f6aec005bf0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b000ff490 msgr2=0x7f6b001951b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b000ff490 0x7f6b001951b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009230 tx=0x7f6af0004040 comp rx=0 tx=0).stop 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 shutdown_connections 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f6ae406df10 0x7f6ae40703c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b001054a0 0x7f6b001956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 --2- 172.21.15.115:0/216068522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b000ff490 0x7f6b001951b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 >> 172.21.15.115:0/216068522 conn(0x7f6b000fafb0 msgr2=0x7f6b000fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.368+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 shutdown_connections 2024-03-22T10:56:38.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.369+0000 7f6b05800640 1 -- 172.21.15.115:0/216068522 wait complete. 2024-03-22T10:56:38.380 DEBUG:tasks.fs:max_mds reduced in epoch 22 2024-03-22T10:56:38.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-22T10:56:38.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.577+0000 7ff4e0e00640 1 -- 172.21.15.115:0/176732829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 msgr2=0x7ff4dc0feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:38.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.577+0000 7ff4e0e00640 1 --2- 172.21.15.115:0/176732829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 0x7ff4dc0feac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0098b0 tx=0x7ff4cc02bcd0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.577+0000 7ff4e0e00640 1 -- 172.21.15.115:0/176732829 shutdown_connections 2024-03-22T10:56:38.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.577+0000 7ff4e0e00640 1 --2- 172.21.15.115:0/176732829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4dc0ff000 0x7ff4dc109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.577+0000 7ff4e0e00640 1 --2- 172.21.15.115:0/176732829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 0x7ff4dc0feac0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.577+0000 7ff4e0e00640 1 -- 172.21.15.115:0/176732829 >> 172.21.15.115:0/176732829 conn(0x7ff4dc073d20 msgr2=0x7ff4dc074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:38.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.577+0000 7ff4e0e00640 1 -- 172.21.15.115:0/176732829 shutdown_connections 2024-03-22T10:56:38.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.577+0000 7ff4e0e00640 1 -- 172.21.15.115:0/176732829 wait complete. 2024-03-22T10:56:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.578+0000 7ff4e0e00640 1 Processor -- start 2024-03-22T10:56:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.579+0000 7ff4e0e00640 1 -- start start 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.579+0000 7ff4e0e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 0x7ff4dc1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.579+0000 7ff4e0e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4dc0ff000 0x7ff4dc197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.579+0000 7ff4e0e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc1980c0 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.579+0000 7ff4e0e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc19be90 con 0x7ff4dc0ff000 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.579+0000 7ff4d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 0x7ff4dc1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.579+0000 7ff4d9e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 0x7ff4dc1974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60598/0 (socket says 172.21.15.115:60598) 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.579+0000 7ff4d9e00640 1 -- 172.21.15.115:0/3733818936 learned_addr learned my addr 172.21.15.115:0/3733818936 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.580+0000 7ff4d9e00640 1 -- 172.21.15.115:0/3733818936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4dc0ff000 msgr2=0x7ff4dc197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.580+0000 7ff4d9e00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4dc0ff000 0x7ff4dc197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.580+0000 7ff4d9e00640 1 -- 172.21.15.115:0/3733818936 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc009560 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.580+0000 7ff4d9e00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 0x7ff4dc1974f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009880 tx=0x7ff4cc02ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:38.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.580+0000 7ff4d2a00640 1 -- 172.21.15.115:0/3733818936 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4cc0035f0 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.580+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc19c110 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.580+0000 7ff4d2a00640 1 -- 172.21.15.115:0/3733818936 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4cc003c10 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.580+0000 7ff4d2a00640 1 -- 172.21.15.115:0/3733818936 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4cc034440 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.580+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc19c610 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.582+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc04e000 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.583+0000 7ff4d2a00640 1 -- 172.21.15.115:0/3733818936 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff4cc004f60 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.585+0000 7ff4d2a00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff4c006df10 0x7ff4c00703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:38.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.585+0000 7ff4d2a00640 1 -- 172.21.15.115:0/3733818936 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff4cc0b02b0 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.585+0000 7ff4d3e00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff4c006df10 0x7ff4c00703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:38.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.586+0000 7ff4d3e00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff4c006df10 0x7ff4c00703c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005fd0 tx=0x7ff4c4005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:38.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.593+0000 7ff4d2a00640 1 -- 172.21.15.115:0/3733818936 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff4cc079600 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.885+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7ff4dc109540 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.886+0000 7ff4d2a00640 1 -- 172.21.15.115:0/3733818936 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v40) v1 ==== 107+0+4093 (secure 0 0 0) 0x7ff4cc07ccc0 con 0x7ff4dc0fe6f0 2024-03-22T10:56:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:38.880 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:24.089241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":0,"incarnation":22,"state":"up:reconnect","state_seq":177,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 23 2024-03-22T10:56:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.888+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff4c006df10 msgr2=0x7ff4c00703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.888+0000 7ff4e0e00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff4c006df10 0x7ff4c00703c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005fd0 tx=0x7ff4c4005850 comp rx=0 tx=0).stop 2024-03-22T10:56:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.888+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 msgr2=0x7ff4dc1974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.888+0000 7ff4e0e00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 0x7ff4dc1974f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009880 tx=0x7ff4cc02ce90 comp rx=0 tx=0).stop 2024-03-22T10:56:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.888+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 shutdown_connections 2024-03-22T10:56:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.888+0000 7ff4e0e00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff4c006df10 0x7ff4c00703c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.888+0000 7ff4e0e00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4dc0ff000 0x7ff4dc197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.888+0000 7ff4e0e00640 1 --2- 172.21.15.115:0/3733818936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4dc0fe6f0 0x7ff4dc1974f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.888+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 >> 172.21.15.115:0/3733818936 conn(0x7ff4dc073d20 msgr2=0x7ff4dc100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.889+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 shutdown_connections 2024-03-22T10:56:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:38.889+0000 7ff4e0e00640 1 -- 172.21.15.115:0/3733818936 wait complete. 2024-03-22T10:56:38.904 DEBUG:tasks.fs:max_mds reduced in epoch 23 2024-03-22T10:56:38.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-22T10:56:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.122+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1225257599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 msgr2=0x7f5a54100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.122+0000 7f5a5ae00640 1 --2- 172.21.15.115:0/1225257599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 0x7f5a54100e90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5a440098b0 tx=0x7f5a4402bd70 comp rx=0 tx=0).stop 2024-03-22T10:56:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.123+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1225257599 shutdown_connections 2024-03-22T10:56:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.123+0000 7f5a5ae00640 1 --2- 172.21.15.115:0/1225257599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a541013d0 0x7f5a541098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.123+0000 7f5a5ae00640 1 --2- 172.21.15.115:0/1225257599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 0x7f5a54100e90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.123+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1225257599 >> 172.21.15.115:0/1225257599 conn(0x7f5a540fa2f0 msgr2=0x7f5a540fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.123+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1225257599 shutdown_connections 2024-03-22T10:56:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.123+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1225257599 wait complete. 2024-03-22T10:56:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.124+0000 7f5a5ae00640 1 Processor -- start 2024-03-22T10:56:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.124+0000 7f5a5ae00640 1 -- start start 2024-03-22T10:56:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a5ae00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 0x7f5a540fe710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a5ae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a541013d0 0x7f5a540fec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a5ae00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a540ff190 con 0x7f5a54100ac0 2024-03-22T10:56:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a5ae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a540ff300 con 0x7f5a541013d0 2024-03-22T10:56:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 0x7f5a540fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 0x7f5a540fe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60610/0 (socket says 172.21.15.115:60610) 2024-03-22T10:56:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a53e00640 1 -- 172.21.15.115:0/1138655937 learned_addr learned my addr 172.21.15.115:0/1138655937 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a53400640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a541013d0 0x7f5a540fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a53e00640 1 -- 172.21.15.115:0/1138655937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a541013d0 msgr2=0x7f5a540fec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a53e00640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a541013d0 0x7f5a540fec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.125+0000 7f5a53e00640 1 -- 172.21.15.115:0/1138655937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a44009560 con 0x7f5a54100ac0 2024-03-22T10:56:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.126+0000 7f5a53e00640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 0x7f5a540fe710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5a4402c2b0 tx=0x7f5a44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.126+0000 7f5a50c00640 1 -- 172.21.15.115:0/1138655937 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a440037c0 con 0x7f5a54100ac0 2024-03-22T10:56:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.126+0000 7f5a50c00640 1 -- 172.21.15.115:0/1138655937 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a44003de0 con 0x7f5a54100ac0 2024-03-22T10:56:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.126+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a540ff580 con 0x7f5a54100ac0 2024-03-22T10:56:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.126+0000 7f5a50c00640 1 -- 172.21.15.115:0/1138655937 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4403e450 con 0x7f5a54100ac0 2024-03-22T10:56:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.126+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a541a1a40 con 0x7f5a54100ac0 2024-03-22T10:56:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.128+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5404e000 con 0x7f5a54100ac0 2024-03-22T10:56:39.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.129+0000 7f5a50c00640 1 -- 172.21.15.115:0/1138655937 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5a44004fd0 con 0x7f5a54100ac0 2024-03-22T10:56:39.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.130+0000 7f5a50c00640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5a3806df10 0x7f5a380703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:39.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.130+0000 7f5a50c00640 1 -- 172.21.15.115:0/1138655937 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f5a440b0980 con 0x7f5a54100ac0 2024-03-22T10:56:39.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.130+0000 7f5a53400640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5a3806df10 0x7f5a380703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:39.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.131+0000 7f5a53400640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5a3806df10 0x7f5a380703c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a40005e10 tx=0x7f5a40005da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.135+0000 7f5a50c00640 1 -- 172.21.15.115:0/1138655937 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a44079eb0 con 0x7f5a54100ac0 2024-03-22T10:56:39.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:38 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3515764875' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-22T10:56:39.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:38 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/216068522' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-22T10:56:39.167 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:38 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3515764875' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-22T10:56:39.167 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:38 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/216068522' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-22T10:56:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.409+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f5a540ff520 con 0x7f5a54100ac0 2024-03-22T10:56:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.410+0000 7f5a50c00640 1 -- 172.21.15.115:0/1138655937 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v40) v1 ==== 107+0+4090 (secure 0 0 0) 0x7f5a4407d570 con 0x7f5a54100ac0 2024-03-22T10:56:39.404 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:39.404 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:24.358113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":0,"incarnation":22,"state":"up:rejoin","state_seq":178,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 24 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5a3806df10 msgr2=0x7f5a380703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5a3806df10 0x7f5a380703c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a40005e10 tx=0x7f5a40005da0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 msgr2=0x7f5a540fe710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 0x7f5a540fe710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5a4402c2b0 tx=0x7f5a44004270 comp rx=0 tx=0).stop 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 shutdown_connections 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5a3806df10 0x7f5a380703c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a541013d0 0x7f5a540fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 --2- 172.21.15.115:0/1138655937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54100ac0 0x7f5a540fe710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 >> 172.21.15.115:0/1138655937 conn(0x7f5a540fa2f0 msgr2=0x7f5a54104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 shutdown_connections 2024-03-22T10:56:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.413+0000 7f5a5ae00640 1 -- 172.21.15.115:0/1138655937 wait complete. 2024-03-22T10:56:39.426 DEBUG:tasks.fs:max_mds reduced in epoch 24 2024-03-22T10:56:39.426 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-22T10:56:39.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.638+0000 7fe6dbe00640 1 -- 172.21.15.115:0/1815290806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc067310 msgr2=0x7fe6dc067780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:39.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.638+0000 7fe6dbe00640 1 --2- 172.21.15.115:0/1815290806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc067310 0x7fe6dc067780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009880 tx=0x7fe6c802bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.639+0000 7fe6dbe00640 1 -- 172.21.15.115:0/1815290806 shutdown_connections 2024-03-22T10:56:39.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.639+0000 7fe6dbe00640 1 --2- 172.21.15.115:0/1815290806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc067310 0x7fe6dc067780 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.639+0000 7fe6dbe00640 1 --2- 172.21.15.115:0/1815290806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6dc105530 0x7fe6dc105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.639+0000 7fe6dbe00640 1 -- 172.21.15.115:0/1815290806 >> 172.21.15.115:0/1815290806 conn(0x7fe6dc0743d0 msgr2=0x7fe6dc0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.639+0000 7fe6dbe00640 1 -- 172.21.15.115:0/1815290806 shutdown_connections 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.639+0000 7fe6dbe00640 1 -- 172.21.15.115:0/1815290806 wait complete. 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.640+0000 7fe6dbe00640 1 Processor -- start 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.640+0000 7fe6dbe00640 1 -- start start 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.640+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6dc067310 0x7fe6dc0fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.640+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc105530 0x7fe6dc0ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.640+0000 7fe6dbe00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc0ff910 con 0x7fe6dc105530 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.640+0000 7fe6dbe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc0ffa80 con 0x7fe6dc067310 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6da000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc105530 0x7fe6dc0ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:39.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6da000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc105530 0x7fe6dc0ff3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60618/0 (socket says 172.21.15.115:60618) 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6da000640 1 -- 172.21.15.115:0/168568001 learned_addr learned my addr 172.21.15.115:0/168568001 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6daa00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6dc067310 0x7fe6dc0fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6da000640 1 -- 172.21.15.115:0/168568001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6dc067310 msgr2=0x7fe6dc0fee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6da000640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6dc067310 0x7fe6dc0fee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6da000640 1 -- 172.21.15.115:0/168568001 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8009560 con 0x7fe6dc105530 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6daa00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6dc067310 0x7fe6dc0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6da000640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc105530 0x7fe6dc0ff3d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009230 tx=0x7fe6c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.641+0000 7fe6e0a00640 1 -- 172.21.15.115:0/168568001 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6c80035f0 con 0x7fe6dc105530 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.642+0000 7fe6e0a00640 1 -- 172.21.15.115:0/168568001 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6c8003c10 con 0x7fe6dc105530 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.642+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc0ffd00 con 0x7fe6dc105530 2024-03-22T10:56:39.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.642+0000 7fe6e0a00640 1 -- 172.21.15.115:0/168568001 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6c803e460 con 0x7fe6dc105530 2024-03-22T10:56:39.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.642+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc1a1670 con 0x7fe6dc105530 2024-03-22T10:56:39.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.643+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6dc04e000 con 0x7fe6dc105530 2024-03-22T10:56:39.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.645+0000 7fe6e0a00640 1 -- 172.21.15.115:0/168568001 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe6c8004fd0 con 0x7fe6dc105530 2024-03-22T10:56:39.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.646+0000 7fe6e0a00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe6c006df10 0x7fe6c00703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:39.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.646+0000 7fe6e0a00640 1 -- 172.21.15.115:0/168568001 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe6c80b0500 con 0x7fe6dc105530 2024-03-22T10:56:39.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.646+0000 7fe6daa00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe6c006df10 0x7fe6c00703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:39.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.647+0000 7fe6daa00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe6c006df10 0x7fe6c00703c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00baa0 tx=0x7fe6cc005d90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:39.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.655+0000 7fe6e0a00640 1 -- 172.21.15.115:0/168568001 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe6c80798d0 con 0x7fe6dc105530 2024-03-22T10:56:39.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.900+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fe6dc0ffd00 con 0x7fe6dc105530 2024-03-22T10:56:39.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.901+0000 7fe6e0a00640 1 -- 172.21.15.115:0/168568001 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v40) v1 ==== 107+0+4099 (secure 0 0 0) 0x7fe6dc0ffd00 con 0x7fe6dc105530 2024-03-22T10:56:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:39.895 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:26.435082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":0,"incarnation":22,"state":"up:active","state_seq":179,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[14468]},"id":1}]} 2024-03-22T10:56:39.895 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 25 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.904+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe6c006df10 msgr2=0x7fe6c00703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.904+0000 7fe6dbe00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe6c006df10 0x7fe6c00703c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00baa0 tx=0x7fe6cc005d90 comp rx=0 tx=0).stop 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.904+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc105530 msgr2=0x7fe6dc0ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.904+0000 7fe6dbe00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc105530 0x7fe6dc0ff3d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009230 tx=0x7fe6c8004040 comp rx=0 tx=0).stop 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.905+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 shutdown_connections 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.905+0000 7fe6dbe00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe6c006df10 0x7fe6c00703c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.905+0000 7fe6dbe00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6dc105530 0x7fe6dc0ff3d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.905+0000 7fe6dbe00640 1 --2- 172.21.15.115:0/168568001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6dc067310 0x7fe6dc0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.905+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 >> 172.21.15.115:0/168568001 conn(0x7fe6dc0743d0 msgr2=0x7fe6dc0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.905+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 shutdown_connections 2024-03-22T10:56:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:39.905+0000 7fe6dbe00640 1 -- 172.21.15.115:0/168568001 wait complete. 2024-03-22T10:56:39.923 DEBUG:tasks.fs:max_mds reduced in epoch 25 2024-03-22T10:56:39.923 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-22T10:56:40.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.128+0000 7f4ee7e00640 1 -- 172.21.15.115:0/672944262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 msgr2=0x7f4ee8067780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:40.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.128+0000 7f4ee7e00640 1 --2- 172.21.15.115:0/672944262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 0x7f4ee8067780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009880 tx=0x7f4ed402bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.128+0000 7f4ee7e00640 1 -- 172.21.15.115:0/672944262 shutdown_connections 2024-03-22T10:56:40.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.128+0000 7f4ee7e00640 1 --2- 172.21.15.115:0/672944262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 0x7f4ee8067780 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.128+0000 7f4ee7e00640 1 --2- 172.21.15.115:0/672944262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ee8100290 0x7f4ee8100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.128+0000 7f4ee7e00640 1 -- 172.21.15.115:0/672944262 >> 172.21.15.115:0/672944262 conn(0x7f4ee80743d0 msgr2=0x7f4ee80747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:40.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.128+0000 7f4ee7e00640 1 -- 172.21.15.115:0/672944262 shutdown_connections 2024-03-22T10:56:40.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.128+0000 7f4ee7e00640 1 -- 172.21.15.115:0/672944262 wait complete. 2024-03-22T10:56:40.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.129+0000 7f4ee7e00640 1 Processor -- start 2024-03-22T10:56:40.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.129+0000 7f4ee7e00640 1 -- start start 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee7e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 0x7f4ee81972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee7e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ee8100290 0x7f4ee8197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee7e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8197ec0 con 0x7f4ee8067310 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee7e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee819bc90 con 0x7f4ee8100290 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 0x7f4ee81972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 0x7f4ee81972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60638/0 (socket says 172.21.15.115:60638) 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee6a00640 1 -- 172.21.15.115:0/2987802214 learned_addr learned my addr 172.21.15.115:0/2987802214 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee6a00640 1 -- 172.21.15.115:0/2987802214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ee8100290 msgr2=0x7f4ee8197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee6a00640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ee8100290 0x7f4ee8197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.130+0000 7f4ee6a00640 1 -- 172.21.15.115:0/2987802214 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4009560 con 0x7f4ee8067310 2024-03-22T10:56:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.131+0000 7f4ee6a00640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 0x7f4ee81972f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c8f0 tx=0x7f4ed800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:40.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.131+0000 7f4edf400640 1 -- 172.21.15.115:0/2987802214 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed8007b80 con 0x7f4ee8067310 2024-03-22T10:56:40.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.131+0000 7f4edf400640 1 -- 172.21.15.115:0/2987802214 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ed80054e0 con 0x7f4ee8067310 2024-03-22T10:56:40.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.131+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee819bf70 con 0x7f4ee8067310 2024-03-22T10:56:40.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.131+0000 7f4edf400640 1 -- 172.21.15.115:0/2987802214 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed800f450 con 0x7f4ee8067310 2024-03-22T10:56:40.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.131+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee819c550 con 0x7f4ee8067310 2024-03-22T10:56:40.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.133+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee804e000 con 0x7f4ee8067310 2024-03-22T10:56:40.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.134+0000 7f4edf400640 1 -- 172.21.15.115:0/2987802214 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4ed8004f60 con 0x7f4ee8067310 2024-03-22T10:56:40.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.135+0000 7f4edf400640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ecc06df10 0x7f4ecc0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:40.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.135+0000 7f4edf400640 1 -- 172.21.15.115:0/2987802214 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4ed808dbe0 con 0x7f4ee8067310 2024-03-22T10:56:40.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.136+0000 7f4ee6000640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ecc06df10 0x7f4ecc0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:40.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.136+0000 7f4ee6000640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ecc06df10 0x7f4ecc0703c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ed402efd0 tx=0x7f4ed4000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:40.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.142+0000 7f4edf400640 1 -- 172.21.15.115:0/2987802214 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ed8056fb0 con 0x7f4ee8067310 2024-03-22T10:56:40.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:39 smithi150 ceph-mon[65513]: pgmap v415: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:40.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:39 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3733818936' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-22T10:56:40.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:39 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1138655937' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-22T10:56:40.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:39 smithi115 ceph-mon[78672]: pgmap v415: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:40.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:39 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3733818936' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-22T10:56:40.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:39 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1138655937' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-22T10:56:40.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.408+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f4ee810eb00 con 0x7f4ee8067310 2024-03-22T10:56:40.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.409+0000 7f4edf400640 1 -- 172.21.15.115:0/2987802214 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v40) v1 ==== 107+0+4915 (secure 0 0 0) 0x7f4ed801d090 con 0x7f4ee8067310 2024-03-22T10:56:40.403 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:40.403 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:26.435082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi115.wjotmn","rank":0,"incarnation":22,"state":"up:active","state_seq":179,"addr":"172.21.15.115:6829/4173107662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4173107662},{"type":"v1","addr":"172.21.15.115:6829","nonce":4173107662}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[14468]},"id":1}]} 2024-03-22T10:56:40.404 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 26 2024-03-22T10:56:40.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ecc06df10 msgr2=0x7f4ecc0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:40.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ecc06df10 0x7f4ecc0703c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ed402efd0 tx=0x7f4ed4000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 msgr2=0x7f4ee81972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:40.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 0x7f4ee81972f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c8f0 tx=0x7f4ed800cdb0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 shutdown_connections 2024-03-22T10:56:40.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ecc06df10 0x7f4ecc0703c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ee8100290 0x7f4ee8197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 --2- 172.21.15.115:0/2987802214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee8067310 0x7f4ee81972f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 >> 172.21.15.115:0/2987802214 conn(0x7f4ee80743d0 msgr2=0x7f4ee80fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:40.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 shutdown_connections 2024-03-22T10:56:40.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.412+0000 7f4ee7e00640 1 -- 172.21.15.115:0/2987802214 wait complete. 2024-03-22T10:56:40.429 DEBUG:tasks.fs:max_mds reduced in epoch 26 2024-03-22T10:56:40.429 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-22T10:56:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.632+0000 7fe38a800640 1 -- 172.21.15.115:0/59518967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe384100ac0 msgr2=0x7fe384100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:40.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.632+0000 7fe38a800640 1 --2- 172.21.15.115:0/59518967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe384100ac0 0x7fe384100e90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe3740098b0 tx=0x7fe37402bd60 comp rx=0 tx=0).stop 2024-03-22T10:56:40.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.632+0000 7fe38a800640 1 -- 172.21.15.115:0/59518967 shutdown_connections 2024-03-22T10:56:40.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.632+0000 7fe38a800640 1 --2- 172.21.15.115:0/59518967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3841013d0 0x7fe3841098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.632+0000 7fe38a800640 1 --2- 172.21.15.115:0/59518967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe384100ac0 0x7fe384100e90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.632+0000 7fe38a800640 1 -- 172.21.15.115:0/59518967 >> 172.21.15.115:0/59518967 conn(0x7fe3840fa2f0 msgr2=0x7fe3840fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:40.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.632+0000 7fe38a800640 1 -- 172.21.15.115:0/59518967 shutdown_connections 2024-03-22T10:56:40.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.632+0000 7fe38a800640 1 -- 172.21.15.115:0/59518967 wait complete. 2024-03-22T10:56:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.633+0000 7fe38a800640 1 Processor -- start 2024-03-22T10:56:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.633+0000 7fe38a800640 1 -- start start 2024-03-22T10:56:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe38a800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe384100ac0 0x7fe38419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe38a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3841013d0 0x7fe38419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe38a800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3841975f0 con 0x7fe3841013d0 2024-03-22T10:56:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe38a800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe384197760 con 0x7fe384100ac0 2024-03-22T10:56:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe382a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3841013d0 0x7fe38419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe382a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3841013d0 0x7fe38419dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60660/0 (socket says 172.21.15.115:60660) 2024-03-22T10:56:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe382a00640 1 -- 172.21.15.115:0/1033251519 learned_addr learned my addr 172.21.15.115:0/1033251519 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe383400640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe384100ac0 0x7fe38419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe382a00640 1 -- 172.21.15.115:0/1033251519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe384100ac0 msgr2=0x7fe38419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe382a00640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe384100ac0 0x7fe38419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe382a00640 1 -- 172.21.15.115:0/1033251519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe374009560 con 0x7fe3841013d0 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.634+0000 7fe383400640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe384100ac0 0x7fe38419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.635+0000 7fe382a00640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3841013d0 0x7fe38419dac0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe37000d6f0 tx=0x7fe37000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.635+0000 7fe37be00640 1 -- 172.21.15.115:0/1033251519 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe370009a30 con 0x7fe3841013d0 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.635+0000 7fe37be00640 1 -- 172.21.15.115:0/1033251519 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3700054e0 con 0x7fe3841013d0 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.635+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe384197a40 con 0x7fe3841013d0 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.635+0000 7fe37be00640 1 -- 172.21.15.115:0/1033251519 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe370010460 con 0x7fe3841013d0 2024-03-22T10:56:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.635+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe384197fa0 con 0x7fe3841013d0 2024-03-22T10:56:40.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.636+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38404e000 con 0x7fe3841013d0 2024-03-22T10:56:40.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.637+0000 7fe37be00640 1 -- 172.21.15.115:0/1033251519 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe370009b90 con 0x7fe3841013d0 2024-03-22T10:56:40.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.638+0000 7fe37be00640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe36806df10 0x7fe3680703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:40.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.638+0000 7fe37be00640 1 -- 172.21.15.115:0/1033251519 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe37008d8a0 con 0x7fe3841013d0 2024-03-22T10:56:40.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.639+0000 7fe383400640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe36806df10 0x7fe3680703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:40.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.640+0000 7fe383400640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe36806df10 0x7fe3680703c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe37402c920 tx=0x7fe374005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.643+0000 7fe37be00640 1 -- 172.21.15.115:0/1033251519 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe370056cf0 con 0x7fe3841013d0 2024-03-22T10:56:40.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.912+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fe384198b00 con 0x7fe3841013d0 2024-03-22T10:56:40.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.914+0000 7fe37be00640 1 -- 172.21.15.115:0/1033251519 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v40) v1 ==== 107+0+4111 (secure 0 0 0) 0x7fe37005a3b0 con 0x7fe3841013d0 2024-03-22T10:56:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:40.908 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:50.661368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:40.908 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 27 2024-03-22T10:56:40.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.916+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe36806df10 msgr2=0x7fe3680703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:40.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.916+0000 7fe38a800640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe36806df10 0x7fe3680703c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe37402c920 tx=0x7fe374005fd0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.916+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3841013d0 msgr2=0x7fe38419dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:40.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.916+0000 7fe38a800640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3841013d0 0x7fe38419dac0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe37000d6f0 tx=0x7fe37000dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.916+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 shutdown_connections 2024-03-22T10:56:40.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.916+0000 7fe38a800640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fe36806df10 0x7fe3680703c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.916+0000 7fe38a800640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3841013d0 0x7fe38419dac0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.916+0000 7fe38a800640 1 --2- 172.21.15.115:0/1033251519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe384100ac0 0x7fe38419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:40.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.916+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 >> 172.21.15.115:0/1033251519 conn(0x7fe3840fa2f0 msgr2=0x7fe3840feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:40.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.917+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 shutdown_connections 2024-03-22T10:56:40.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:40.917+0000 7fe38a800640 1 -- 172.21.15.115:0/1033251519 wait complete. 2024-03-22T10:56:40.929 DEBUG:tasks.fs:max_mds reduced in epoch 27 2024-03-22T10:56:40.929 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-22T10:56:41.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.150+0000 7f8b85e00640 1 -- 172.21.15.115:0/505957996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 msgr2=0x7f8b80107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:41.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.150+0000 7f8b85e00640 1 --2- 172.21.15.115:0/505957996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 0x7f8b80107c70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009880 tx=0x7f8b7002bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.150+0000 7f8b85e00640 1 -- 172.21.15.115:0/505957996 shutdown_connections 2024-03-22T10:56:41.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.150+0000 7f8b85e00640 1 --2- 172.21.15.115:0/505957996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80101890 0x7f8b80101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.150+0000 7f8b85e00640 1 --2- 172.21.15.115:0/505957996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 0x7f8b80107c70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.150+0000 7f8b85e00640 1 -- 172.21.15.115:0/505957996 >> 172.21.15.115:0/505957996 conn(0x7f8b800fd390 msgr2=0x7f8b800ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:41.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.150+0000 7f8b85e00640 1 -- 172.21.15.115:0/505957996 shutdown_connections 2024-03-22T10:56:41.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.150+0000 7f8b85e00640 1 -- 172.21.15.115:0/505957996 wait complete. 2024-03-22T10:56:41.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.151+0000 7f8b85e00640 1 Processor -- start 2024-03-22T10:56:41.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.151+0000 7f8b85e00640 1 -- start start 2024-03-22T10:56:41.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.152+0000 7f8b85e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80101890 0x7f8b801975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:41.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.152+0000 7f8b85e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 0x7f8b80197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.152+0000 7f8b85e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b801981b0 con 0x7f8b801078a0 2024-03-22T10:56:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.152+0000 7f8b85e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8019bf80 con 0x7f8b80101890 2024-03-22T10:56:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.152+0000 7f8b7e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 0x7f8b80197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.152+0000 7f8b7e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 0x7f8b80197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60678/0 (socket says 172.21.15.115:60678) 2024-03-22T10:56:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.152+0000 7f8b7e000640 1 -- 172.21.15.115:0/1636090805 learned_addr learned my addr 172.21.15.115:0/1636090805 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.152+0000 7f8b7ea00640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80101890 0x7f8b801975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.153+0000 7f8b7e000640 1 -- 172.21.15.115:0/1636090805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80101890 msgr2=0x7f8b801975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.153+0000 7f8b7e000640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80101890 0x7f8b801975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.153+0000 7f8b7e000640 1 -- 172.21.15.115:0/1636090805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70009560 con 0x7f8b801078a0 2024-03-22T10:56:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.153+0000 7f8b7e000640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 0x7f8b80197b20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00d6f0 tx=0x7f8b6c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.153+0000 7f8b84a00640 1 -- 172.21.15.115:0/1636090805 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b6c009a30 con 0x7f8b801078a0 2024-03-22T10:56:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.153+0000 7f8b84a00640 1 -- 172.21.15.115:0/1636090805 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b6c0054e0 con 0x7f8b801078a0 2024-03-22T10:56:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.153+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8019c260 con 0x7f8b801078a0 2024-03-22T10:56:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.153+0000 7f8b84a00640 1 -- 172.21.15.115:0/1636090805 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b6c010460 con 0x7f8b801078a0 2024-03-22T10:56:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.153+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8019c7c0 con 0x7f8b801078a0 2024-03-22T10:56:41.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.155+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8004e000 con 0x7f8b801078a0 2024-03-22T10:56:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.157+0000 7f8b84a00640 1 -- 172.21.15.115:0/1636090805 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f8b6c009b90 con 0x7f8b801078a0 2024-03-22T10:56:41.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:40 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/168568001' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-22T10:56:41.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:40 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2987802214' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-22T10:56:41.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.158+0000 7f8b84a00640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b6406df10 0x7f8b640703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:41.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.158+0000 7f8b84a00640 1 -- 172.21.15.115:0/1636090805 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8b6c08d9e0 con 0x7f8b801078a0 2024-03-22T10:56:41.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.158+0000 7f8b7ea00640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b6406df10 0x7f8b640703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.159+0000 7f8b7ea00640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b6406df10 0x7f8b640703c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8b70006010 tx=0x7f8b70000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.163+0000 7f8b84a00640 1 -- 172.21.15.115:0/1636090805 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b6c056d70 con 0x7f8b801078a0 2024-03-22T10:56:41.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:40 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/168568001' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-22T10:56:41.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:40 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2987802214' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-22T10:56:41.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.441+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f8b8010eb80 con 0x7f8b801078a0 2024-03-22T10:56:41.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.442+0000 7f8b84a00640 1 -- 172.21.15.115:0/1636090805 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v40) v1 ==== 107+0+4122 (secure 0 0 0) 0x7f8b6c05a430 con 0x7f8b801078a0 2024-03-22T10:56:41.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:41.436 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:50.668530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":0,"incarnation":28,"state":"up:replay","state_seq":2,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:41.436 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 28 2024-03-22T10:56:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b6406df10 msgr2=0x7f8b640703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b6406df10 0x7f8b640703c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8b70006010 tx=0x7f8b70000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 msgr2=0x7f8b80197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 0x7f8b80197b20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00d6f0 tx=0x7f8b6c00dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 shutdown_connections 2024-03-22T10:56:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f8b6406df10 0x7f8b640703c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b801078a0 0x7f8b80197b20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 --2- 172.21.15.115:0/1636090805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80101890 0x7f8b801975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 >> 172.21.15.115:0/1636090805 conn(0x7f8b800fd390 msgr2=0x7f8b800fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.445+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 shutdown_connections 2024-03-22T10:56:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.446+0000 7f8b85e00640 1 -- 172.21.15.115:0/1636090805 wait complete. 2024-03-22T10:56:41.459 DEBUG:tasks.fs:max_mds reduced in epoch 28 2024-03-22T10:56:41.459 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-22T10:56:41.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.675+0000 7ff1a2000640 1 -- 172.21.15.115:0/1820979392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 msgr2=0x7ff19c107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:41.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.675+0000 7ff1a2000640 1 --2- 172.21.15.115:0/1820979392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 0x7ff19c107c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009880 tx=0x7ff18c02bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.675+0000 7ff1a2000640 1 -- 172.21.15.115:0/1820979392 shutdown_connections 2024-03-22T10:56:41.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.675+0000 7ff1a2000640 1 --2- 172.21.15.115:0/1820979392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff19c101890 0x7ff19c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.675+0000 7ff1a2000640 1 --2- 172.21.15.115:0/1820979392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 0x7ff19c107c70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.675+0000 7ff1a2000640 1 -- 172.21.15.115:0/1820979392 >> 172.21.15.115:0/1820979392 conn(0x7ff19c0fd390 msgr2=0x7ff19c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:41.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.675+0000 7ff1a2000640 1 -- 172.21.15.115:0/1820979392 shutdown_connections 2024-03-22T10:56:41.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.675+0000 7ff1a2000640 1 -- 172.21.15.115:0/1820979392 wait complete. 2024-03-22T10:56:41.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.676+0000 7ff1a2000640 1 Processor -- start 2024-03-22T10:56:41.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.676+0000 7ff1a2000640 1 -- start start 2024-03-22T10:56:41.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.677+0000 7ff1a2000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff19c101890 0x7ff19c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:41.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.677+0000 7ff1a2000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 0x7ff19c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:41.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.677+0000 7ff1a2000640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c1981b0 con 0x7ff19c1078a0 2024-03-22T10:56:41.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.677+0000 7ff1a2000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c19bf80 con 0x7ff19c101890 2024-03-22T10:56:41.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.677+0000 7ff19aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 0x7ff19c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:41.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.677+0000 7ff19aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 0x7ff19c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60698/0 (socket says 172.21.15.115:60698) 2024-03-22T10:56:41.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.677+0000 7ff19aa00640 1 -- 172.21.15.115:0/2136310480 learned_addr learned my addr 172.21.15.115:0/2136310480 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.678+0000 7ff19b400640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff19c101890 0x7ff19c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.678+0000 7ff19aa00640 1 -- 172.21.15.115:0/2136310480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff19c101890 msgr2=0x7ff19c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.678+0000 7ff19aa00640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff19c101890 0x7ff19c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.678+0000 7ff19aa00640 1 -- 172.21.15.115:0/2136310480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c009560 con 0x7ff19c1078a0 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.678+0000 7ff19aa00640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 0x7ff19c197b20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff18800d6f0 tx=0x7ff18800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.678+0000 7ff193e00640 1 -- 172.21.15.115:0/2136310480 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff188009a30 con 0x7ff19c1078a0 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.678+0000 7ff193e00640 1 -- 172.21.15.115:0/2136310480 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1880054e0 con 0x7ff19c1078a0 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.678+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c19c260 con 0x7ff19c1078a0 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.678+0000 7ff193e00640 1 -- 172.21.15.115:0/2136310480 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff188010460 con 0x7ff19c1078a0 2024-03-22T10:56:41.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.679+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c19c7c0 con 0x7ff19c1078a0 2024-03-22T10:56:41.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.680+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c04e000 con 0x7ff19c1078a0 2024-03-22T10:56:41.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.681+0000 7ff193e00640 1 -- 172.21.15.115:0/2136310480 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff188009b90 con 0x7ff19c1078a0 2024-03-22T10:56:41.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.682+0000 7ff193e00640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff18006df10 0x7ff1800703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:41.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.682+0000 7ff193e00640 1 -- 172.21.15.115:0/2136310480 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff18808d8a0 con 0x7ff19c1078a0 2024-03-22T10:56:41.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.683+0000 7ff19b400640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff18006df10 0x7ff1800703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:41.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.683+0000 7ff19b400640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff18006df10 0x7ff1800703c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff18c036040 tx=0x7ff18c02cf90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:41.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.686+0000 7ff193e00640 1 -- 172.21.15.115:0/2136310480 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff188056cf0 con 0x7ff19c1078a0 2024-03-22T10:56:41.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.927+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7ff19c198930 con 0x7ff19c1078a0 2024-03-22T10:56:41.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.928+0000 7ff193e00640 1 -- 172.21.15.115:0/2136310480 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v40) v1 ==== 107+0+4127 (secure 0 0 0) 0x7ff18805a3b0 con 0x7ff19c1078a0 2024-03-22T10:56:41.922 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:41.922 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:56.374327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":184,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:41.922 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 29 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.931+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff18006df10 msgr2=0x7ff1800703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.931+0000 7ff1a2000640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff18006df10 0x7ff1800703c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff18c036040 tx=0x7ff18c02cf90 comp rx=0 tx=0).stop 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.931+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 msgr2=0x7ff19c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.931+0000 7ff1a2000640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 0x7ff19c197b20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff18800d6f0 tx=0x7ff18800dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.931+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 shutdown_connections 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.931+0000 7ff1a2000640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7ff18006df10 0x7ff1800703c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.931+0000 7ff1a2000640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c1078a0 0x7ff19c197b20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.931+0000 7ff1a2000640 1 --2- 172.21.15.115:0/2136310480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff19c101890 0x7ff19c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.931+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 >> 172.21.15.115:0/2136310480 conn(0x7ff19c0fd390 msgr2=0x7ff19c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.932+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 shutdown_connections 2024-03-22T10:56:41.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:41.932+0000 7ff1a2000640 1 -- 172.21.15.115:0/2136310480 wait complete. 2024-03-22T10:56:41.943 DEBUG:tasks.fs:max_mds reduced in epoch 29 2024-03-22T10:56:41.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-22T10:56:42.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:41 smithi150 ceph-mon[65513]: pgmap v416: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:42.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:41 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1033251519' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-22T10:56:42.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:41 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1636090805' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-22T10:56:42.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.163+0000 7f492a600640 1 -- 172.21.15.115:0/504198193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 msgr2=0x7f49240ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:42.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.163+0000 7f492a600640 1 --2- 172.21.15.115:0/504198193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 0x7f49240ffce0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4910009880 tx=0x7f491002bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.163+0000 7f492a600640 1 -- 172.21.15.115:0/504198193 shutdown_connections 2024-03-22T10:56:42.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.163+0000 7f492a600640 1 --2- 172.21.15.115:0/504198193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 0x7f49240ffce0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.163+0000 7f492a600640 1 --2- 172.21.15.115:0/504198193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4924105880 0x7f4924105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.163+0000 7f492a600640 1 -- 172.21.15.115:0/504198193 >> 172.21.15.115:0/504198193 conn(0x7f49240fb370 msgr2=0x7f49240fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:42.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.163+0000 7f492a600640 1 -- 172.21.15.115:0/504198193 shutdown_connections 2024-03-22T10:56:42.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.163+0000 7f492a600640 1 -- 172.21.15.115:0/504198193 wait complete. 2024-03-22T10:56:42.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.164+0000 7f492a600640 1 Processor -- start 2024-03-22T10:56:42.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.165+0000 7f492a600640 1 -- start start 2024-03-22T10:56:42.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.165+0000 7f492a600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 0x7f4924072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:42.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.165+0000 7f492a600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4924105880 0x7f4924072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:42.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.165+0000 7f492a600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924072fe0 con 0x7f49240ff870 2024-03-22T10:56:42.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.165+0000 7f492a600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924073150 con 0x7f4924105880 2024-03-22T10:56:42.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.165+0000 7f4923400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 0x7f4924072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:42.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.166+0000 7f4923400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 0x7f4924072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37578/0 (socket says 172.21.15.115:37578) 2024-03-22T10:56:42.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.166+0000 7f4923400640 1 -- 172.21.15.115:0/2682313081 learned_addr learned my addr 172.21.15.115:0/2682313081 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:42.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.166+0000 7f4922a00640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4924105880 0x7f4924072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:42.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.166+0000 7f4923400640 1 -- 172.21.15.115:0/2682313081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4924105880 msgr2=0x7f4924072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:42.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.166+0000 7f4923400640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4924105880 0x7f4924072aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.166+0000 7f4923400640 1 -- 172.21.15.115:0/2682313081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910009560 con 0x7f49240ff870 2024-03-22T10:56:42.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.166+0000 7f4923400640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 0x7f4924072560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f491400c8f0 tx=0x7f491400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:42.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.167+0000 7f491be00640 1 -- 172.21.15.115:0/2682313081 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49140041f0 con 0x7f49240ff870 2024-03-22T10:56:42.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.167+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49240733d0 con 0x7f49240ff870 2024-03-22T10:56:42.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.167+0000 7f491be00640 1 -- 172.21.15.115:0/2682313081 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4914007e70 con 0x7f49240ff870 2024-03-22T10:56:42.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.167+0000 7f491be00640 1 -- 172.21.15.115:0/2682313081 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49140049d0 con 0x7f49240ff870 2024-03-22T10:56:42.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.167+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49241a1b40 con 0x7f49240ff870 2024-03-22T10:56:42.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.168+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492404e000 con 0x7f49240ff870 2024-03-22T10:56:42.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.169+0000 7f491be00640 1 -- 172.21.15.115:0/2682313081 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4914004350 con 0x7f49240ff870 2024-03-22T10:56:42.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.170+0000 7f491be00640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f490806df10 0x7f49080703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:42.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.171+0000 7f491be00640 1 -- 172.21.15.115:0/2682313081 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f491408daf0 con 0x7f49240ff870 2024-03-22T10:56:42.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.171+0000 7f4922a00640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f490806df10 0x7f49080703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:42.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.172+0000 7f4922a00640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f490806df10 0x7f49080703c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4924073d50 tx=0x7f4910000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:42.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.176+0000 7f491be00640 1 -- 172.21.15.115:0/2682313081 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4914056e40 con 0x7f49240ff870 2024-03-22T10:56:42.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:41 smithi115 ceph-mon[78672]: pgmap v416: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:42.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:41 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1033251519' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-22T10:56:42.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:41 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1636090805' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-22T10:56:42.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.508+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f4924066860 con 0x7f49240ff870 2024-03-22T10:56:42.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.509+0000 7f491be00640 1 -- 172.21.15.115:0/2682313081 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v40) v1 ==== 107+0+4124 (secure 0 0 0) 0x7f491405a500 con 0x7f49240ff870 2024-03-22T10:56:42.504 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:42.504 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:56.968203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":185,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:42.504 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-03-22T10:56:42.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.512+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f490806df10 msgr2=0x7f49080703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:42.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.512+0000 7f492a600640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f490806df10 0x7f49080703c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4924073d50 tx=0x7f4910000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.512+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 msgr2=0x7f4924072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:42.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.512+0000 7f492a600640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 0x7f4924072560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f491400c8f0 tx=0x7f491400cdb0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.513+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 shutdown_connections 2024-03-22T10:56:42.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.513+0000 7f492a600640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f490806df10 0x7f49080703c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.513+0000 7f492a600640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4924105880 0x7f4924072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.513+0000 7f492a600640 1 --2- 172.21.15.115:0/2682313081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49240ff870 0x7f4924072560 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.513+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 >> 172.21.15.115:0/2682313081 conn(0x7f49240fb370 msgr2=0x7f49240fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:42.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.513+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 shutdown_connections 2024-03-22T10:56:42.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.513+0000 7f492a600640 1 -- 172.21.15.115:0/2682313081 wait complete. 2024-03-22T10:56:42.524 DEBUG:tasks.fs:max_mds reduced in epoch 30 2024-03-22T10:56:42.524 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-22T10:56:42.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.728+0000 7f1694c00640 1 -- 172.21.15.115:0/1261898378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 msgr2=0x7f1690067310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:42.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.728+0000 7f1694c00640 1 --2- 172.21.15.115:0/1261898378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 0x7f1690067310 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f167c009880 tx=0x7f167c02bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.729+0000 7f1694c00640 1 -- 172.21.15.115:0/1261898378 shutdown_connections 2024-03-22T10:56:42.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.729+0000 7f1694c00640 1 --2- 172.21.15.115:0/1261898378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 0x7f1690067310 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.729+0000 7f1694c00640 1 --2- 172.21.15.115:0/1261898378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690105550 0x7f1690105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.729+0000 7f1694c00640 1 -- 172.21.15.115:0/1261898378 >> 172.21.15.115:0/1261898378 conn(0x7f16900743d0 msgr2=0x7f16900747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:42.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.729+0000 7f1694c00640 1 -- 172.21.15.115:0/1261898378 shutdown_connections 2024-03-22T10:56:42.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.729+0000 7f1694c00640 1 -- 172.21.15.115:0/1261898378 wait complete. 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.730+0000 7f1694c00640 1 Processor -- start 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.730+0000 7f1694c00640 1 -- start start 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.730+0000 7f1694c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690105550 0x7f1690197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.730+0000 7f1694c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 0x7f1690197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.730+0000 7f1694c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1690197f10 con 0x7f1690105e60 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.730+0000 7f1694c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169019bce0 con 0x7f1690105550 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.731+0000 7f168ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 0x7f1690197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.731+0000 7f168ea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 0x7f1690197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37592/0 (socket says 172.21.15.115:37592) 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.731+0000 7f168ea00640 1 -- 172.21.15.115:0/4181262591 learned_addr learned my addr 172.21.15.115:0/4181262591 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:42.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.731+0000 7f168f400640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690105550 0x7f1690197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:42.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.731+0000 7f168ea00640 1 -- 172.21.15.115:0/4181262591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690105550 msgr2=0x7f1690197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:42.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.731+0000 7f168ea00640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690105550 0x7f1690197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:42.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.731+0000 7f168ea00640 1 -- 172.21.15.115:0/4181262591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c009560 con 0x7f1690105e60 2024-03-22T10:56:42.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.732+0000 7f168ea00640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 0x7f1690197880 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f167c036040 tx=0x7f167c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:42.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.732+0000 7f1687e00640 1 -- 172.21.15.115:0/4181262591 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f167c0035f0 con 0x7f1690105e60 2024-03-22T10:56:42.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.732+0000 7f1687e00640 1 -- 172.21.15.115:0/4181262591 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f167c003c10 con 0x7f1690105e60 2024-03-22T10:56:42.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.732+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169019bf60 con 0x7f1690105e60 2024-03-22T10:56:42.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.732+0000 7f1687e00640 1 -- 172.21.15.115:0/4181262591 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f167c034440 con 0x7f1690105e60 2024-03-22T10:56:42.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.732+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169019c4c0 con 0x7f1690105e60 2024-03-22T10:56:42.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.733+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169004e000 con 0x7f1690105e60 2024-03-22T10:56:42.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.735+0000 7f1687e00640 1 -- 172.21.15.115:0/4181262591 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f167c004f60 con 0x7f1690105e60 2024-03-22T10:56:42.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.737+0000 7f1687e00640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f167406df10 0x7f16740703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:42.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.737+0000 7f1687e00640 1 -- 172.21.15.115:0/4181262591 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f167c0b0270 con 0x7f1690105e60 2024-03-22T10:56:42.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.737+0000 7f168f400640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f167406df10 0x7f16740703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:42.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.738+0000 7f168f400640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f167406df10 0x7f16740703c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f16800058c0 tx=0x7f168000a4c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:42.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:42.742+0000 7f1687e00640 1 -- 172.21.15.115:0/4181262591 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f167c079640 con 0x7f1690105e60 2024-03-22T10:56:42.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.002+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f1690198690 con 0x7f1690105e60 2024-03-22T10:56:42.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.003+0000 7f1687e00640 1 -- 172.21.15.115:0/4181262591 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v40) v1 ==== 107+0+4949 (secure 0 0 0) 0x7f167c07cd00 con 0x7f1690105e60 2024-03-22T10:56:42.997 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:42.997 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44271,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/2616517800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2616517800},{"type":"v1","addr":"172.21.15.150:6825","nonce":2616517800}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:58.971074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":0,"incarnation":28,"state":"up:active","state_seq":186,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24255,"qdb_cluster":[24255]},"id":1}]} 2024-03-22T10:56:42.998 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 31 2024-03-22T10:56:43.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.006+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f167406df10 msgr2=0x7f16740703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:43.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.006+0000 7f1694c00640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f167406df10 0x7f16740703c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f16800058c0 tx=0x7f168000a4c0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.006+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 msgr2=0x7f1690197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:43.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.006+0000 7f1694c00640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 0x7f1690197880 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f167c036040 tx=0x7f167c004040 comp rx=0 tx=0).stop 2024-03-22T10:56:43.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.007+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 shutdown_connections 2024-03-22T10:56:43.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.007+0000 7f1694c00640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f167406df10 0x7f16740703c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.007+0000 7f1694c00640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1690105e60 0x7f1690197880 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.007+0000 7f1694c00640 1 --2- 172.21.15.115:0/4181262591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690105550 0x7f1690197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.007+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 >> 172.21.15.115:0/4181262591 conn(0x7f16900743d0 msgr2=0x7f16900ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:43.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.007+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 shutdown_connections 2024-03-22T10:56:43.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.007+0000 7f1694c00640 1 -- 172.21.15.115:0/4181262591 wait complete. 2024-03-22T10:56:43.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:42 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2136310480' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-22T10:56:43.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:42 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2682313081' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-22T10:56:43.019 DEBUG:tasks.fs:max_mds reduced in epoch 31 2024-03-22T10:56:43.019 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-22T10:56:43.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:42 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2136310480' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-22T10:56:43.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:42 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2682313081' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-22T10:56:43.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.224+0000 7f4f86e00640 1 -- 172.21.15.115:0/3420998406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105e60 msgr2=0x7f4f80067310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:43.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.225+0000 7f4f86e00640 1 --2- 172.21.15.115:0/3420998406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105e60 0x7f4f80067310 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009880 tx=0x7f4f6c02bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.225+0000 7f4f86e00640 1 -- 172.21.15.115:0/3420998406 shutdown_connections 2024-03-22T10:56:43.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.225+0000 7f4f86e00640 1 --2- 172.21.15.115:0/3420998406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105e60 0x7f4f80067310 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.225+0000 7f4f86e00640 1 --2- 172.21.15.115:0/3420998406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f80105550 0x7f4f80105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.225+0000 7f4f86e00640 1 -- 172.21.15.115:0/3420998406 >> 172.21.15.115:0/3420998406 conn(0x7f4f800743d0 msgr2=0x7f4f800747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:43.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.225+0000 7f4f86e00640 1 -- 172.21.15.115:0/3420998406 shutdown_connections 2024-03-22T10:56:43.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.225+0000 7f4f86e00640 1 -- 172.21.15.115:0/3420998406 wait complete. 2024-03-22T10:56:43.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.226+0000 7f4f86e00640 1 Processor -- start 2024-03-22T10:56:43.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.226+0000 7f4f86e00640 1 -- start start 2024-03-22T10:56:43.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.227+0000 7f4f86e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105550 0x7f4f80195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:43.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.227+0000 7f4f86e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f80105e60 0x7f4f80195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:43.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.227+0000 7f4f86e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80195cd0 con 0x7f4f80105550 2024-03-22T10:56:43.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.227+0000 7f4f86e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80199aa0 con 0x7f4f80105e60 2024-03-22T10:56:43.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.227+0000 7f4f85a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105550 0x7f4f80195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:43.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.227+0000 7f4f85a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105550 0x7f4f80195100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37600/0 (socket says 172.21.15.115:37600) 2024-03-22T10:56:43.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.227+0000 7f4f85a00640 1 -- 172.21.15.115:0/3135852824 learned_addr learned my addr 172.21.15.115:0/3135852824 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:43.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.228+0000 7f4f85a00640 1 -- 172.21.15.115:0/3135852824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f80105e60 msgr2=0x7f4f80195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.228+0000 7f4f85a00640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f80105e60 0x7f4f80195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.228+0000 7f4f85a00640 1 -- 172.21.15.115:0/3135852824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c009560 con 0x7f4f80105550 2024-03-22T10:56:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.228+0000 7f4f85a00640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105550 0x7f4f80195100 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400d6f0 tx=0x7f4f7400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.228+0000 7f4f7e000640 1 -- 172.21.15.115:0/3135852824 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f74009a30 con 0x7f4f80105550 2024-03-22T10:56:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.228+0000 7f4f7e000640 1 -- 172.21.15.115:0/3135852824 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f740054e0 con 0x7f4f80105550 2024-03-22T10:56:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.228+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f80199d80 con 0x7f4f80105550 2024-03-22T10:56:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.228+0000 7f4f7e000640 1 -- 172.21.15.115:0/3135852824 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f74010460 con 0x7f4f80105550 2024-03-22T10:56:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.228+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8019a360 con 0x7f4f80105550 2024-03-22T10:56:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.230+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8004e000 con 0x7f4f80105550 2024-03-22T10:56:43.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.232+0000 7f4f7e000640 1 -- 172.21.15.115:0/3135852824 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4f74009b90 con 0x7f4f80105550 2024-03-22T10:56:43.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.232+0000 7f4f7e000640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4f6806df10 0x7f4f680703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:43.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.233+0000 7f4f7e000640 1 -- 172.21.15.115:0/3135852824 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4f7408d8a0 con 0x7f4f80105550 2024-03-22T10:56:43.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.233+0000 7f4f85000640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4f6806df10 0x7f4f680703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:43.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.233+0000 7f4f85000640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4f6806df10 0x7f4f680703c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c036040 tx=0x7f4f6c000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:43.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.238+0000 7f4f7e000640 1 -- 172.21.15.115:0/3135852824 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f74056cf0 con 0x7f4f80105550 2024-03-22T10:56:43.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.478+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f4f8010c920 con 0x7f4f80105550 2024-03-22T10:56:43.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.479+0000 7f4f7e000640 1 -- 172.21.15.115:0/3135852824 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v40) v1 ==== 107+0+4167 (secure 0 0 0) 0x7f4f7405a3b0 con 0x7f4f80105550 2024-03-22T10:56:43.473 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:43.474 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:58.971074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":0,"incarnation":28,"state":"up:active","state_seq":186,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24255,"qdb_cluster":[24255]},"id":1}]} 2024-03-22T10:56:43.474 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 32 2024-03-22T10:56:43.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4f6806df10 msgr2=0x7f4f680703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:43.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4f6806df10 0x7f4f680703c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c036040 tx=0x7f4f6c000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105550 msgr2=0x7f4f80195100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:43.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105550 0x7f4f80195100 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400d6f0 tx=0x7f4f7400dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 shutdown_connections 2024-03-22T10:56:43.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4f6806df10 0x7f4f680703c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f80105e60 0x7f4f80195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 --2- 172.21.15.115:0/3135852824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f80105550 0x7f4f80195100 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 >> 172.21.15.115:0/3135852824 conn(0x7f4f800743d0 msgr2=0x7f4f800fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:43.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.483+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 shutdown_connections 2024-03-22T10:56:43.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.484+0000 7f4f86e00640 1 -- 172.21.15.115:0/3135852824 wait complete. 2024-03-22T10:56:43.495 DEBUG:tasks.fs:max_mds reduced in epoch 32 2024-03-22T10:56:43.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-22T10:56:43.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.709+0000 7f5e5e800640 1 -- 172.21.15.115:0/7495206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e580fedc0 msgr2=0x7f5e580ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:43.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.709+0000 7f5e5e800640 1 --2- 172.21.15.115:0/7495206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e580fedc0 0x7f5e580ff230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009880 tx=0x7f5e4402bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.709+0000 7f5e5e800640 1 -- 172.21.15.115:0/7495206 shutdown_connections 2024-03-22T10:56:43.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.709+0000 7f5e5e800640 1 --2- 172.21.15.115:0/7495206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e580fedc0 0x7f5e580ff230 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.709+0000 7f5e5e800640 1 --2- 172.21.15.115:0/7495206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e581034f0 0x7f5e581038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.709+0000 7f5e5e800640 1 -- 172.21.15.115:0/7495206 >> 172.21.15.115:0/7495206 conn(0x7f5e580743d0 msgr2=0x7f5e580747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:43.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.709+0000 7f5e5e800640 1 -- 172.21.15.115:0/7495206 shutdown_connections 2024-03-22T10:56:43.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.709+0000 7f5e5e800640 1 -- 172.21.15.115:0/7495206 wait complete. 2024-03-22T10:56:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.710+0000 7f5e5e800640 1 Processor -- start 2024-03-22T10:56:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.710+0000 7f5e5e800640 1 -- start start 2024-03-22T10:56:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.710+0000 7f5e5e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e580fedc0 0x7f5e581972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.710+0000 7f5e5e800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e581034f0 0x7f5e581977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.710+0000 7f5e5e800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e58197e80 con 0x7f5e581034f0 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.710+0000 7f5e5e800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5819bc50 con 0x7f5e580fedc0 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.711+0000 7f5e5ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e581034f0 0x7f5e581977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.711+0000 7f5e5ca00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e581034f0 0x7f5e581977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37606/0 (socket says 172.21.15.115:37606) 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.711+0000 7f5e5ca00640 1 -- 172.21.15.115:0/3697277419 learned_addr learned my addr 172.21.15.115:0/3697277419 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.711+0000 7f5e5d400640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e580fedc0 0x7f5e581972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.711+0000 7f5e5ca00640 1 -- 172.21.15.115:0/3697277419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e580fedc0 msgr2=0x7f5e581972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.711+0000 7f5e5ca00640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e580fedc0 0x7f5e581972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.711+0000 7f5e5ca00640 1 -- 172.21.15.115:0/3697277419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44009560 con 0x7f5e581034f0 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.711+0000 7f5e5d400640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e580fedc0 0x7f5e581972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:56:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.712+0000 7f5e5ca00640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e581034f0 0x7f5e581977f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e44000c00 tx=0x7f5e4402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.712+0000 7f5e56000640 1 -- 172.21.15.115:0/3697277419 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e44004520 con 0x7f5e581034f0 2024-03-22T10:56:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.712+0000 7f5e56000640 1 -- 172.21.15.115:0/3697277419 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e44003ab0 con 0x7f5e581034f0 2024-03-22T10:56:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.712+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5819bed0 con 0x7f5e581034f0 2024-03-22T10:56:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.712+0000 7f5e56000640 1 -- 172.21.15.115:0/3697277419 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e44034440 con 0x7f5e581034f0 2024-03-22T10:56:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.712+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5819c450 con 0x7f5e581034f0 2024-03-22T10:56:43.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.713+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5804e000 con 0x7f5e581034f0 2024-03-22T10:56:43.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.714+0000 7f5e56000640 1 -- 172.21.15.115:0/3697277419 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5e44003c20 con 0x7f5e581034f0 2024-03-22T10:56:43.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.716+0000 7f5e56000640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5e4006df10 0x7f5e400703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:43.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.716+0000 7f5e56000640 1 -- 172.21.15.115:0/3697277419 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f5e440b00f0 con 0x7f5e581034f0 2024-03-22T10:56:43.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.716+0000 7f5e5d400640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5e4006df10 0x7f5e400703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:43.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.717+0000 7f5e5d400640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5e4006df10 0x7f5e400703c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009b40 tx=0x7f5e4c005e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:43.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:43.723+0000 7f5e56000640 1 -- 172.21.15.115:0/3697277419 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5e44079540 con 0x7f5e581034f0 2024-03-22T10:56:44.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.031+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f5e581986a0 con 0x7f5e581034f0 2024-03-22T10:56:44.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.031+0000 7f5e56000640 1 -- 172.21.15.115:0/3697277419 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v40) v1 ==== 107+0+4983 (secure 0 0 0) 0x7f5e4407cc00 con 0x7f5e581034f0 2024-03-22T10:56:44.025 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:44.026 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44297,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/1804848703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1804848703},{"type":"v1","addr":"172.21.15.150:6825","nonce":1804848703}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:50:58.971074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi150.slzwwb","rank":0,"incarnation":28,"state":"up:active","state_seq":186,"addr":"172.21.15.150:6827/2497876489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2497876489},{"type":"v1","addr":"172.21.15.150:6827","nonce":2497876489}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24255,"qdb_cluster":[24255]},"id":1}]} 2024-03-22T10:56:44.026 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 33 2024-03-22T10:56:44.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5e4006df10 msgr2=0x7f5e400703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:44.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5e4006df10 0x7f5e400703c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009b40 tx=0x7f5e4c005e50 comp rx=0 tx=0).stop 2024-03-22T10:56:44.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e581034f0 msgr2=0x7f5e581977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:44.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e581034f0 0x7f5e581977f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e44000c00 tx=0x7f5e4402c850 comp rx=0 tx=0).stop 2024-03-22T10:56:44.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 shutdown_connections 2024-03-22T10:56:44.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f5e4006df10 0x7f5e400703c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e581034f0 0x7f5e581977f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 --2- 172.21.15.115:0/3697277419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e580fedc0 0x7f5e581972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 >> 172.21.15.115:0/3697277419 conn(0x7f5e580743d0 msgr2=0x7f5e58108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:44.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 shutdown_connections 2024-03-22T10:56:44.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.034+0000 7f5e5e800640 1 -- 172.21.15.115:0/3697277419 wait complete. 2024-03-22T10:56:44.051 DEBUG:tasks.fs:max_mds reduced in epoch 33 2024-03-22T10:56:44.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-22T10:56:44.126 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:43 smithi115 ceph-mon[78672]: pgmap v417: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:44.126 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:43 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/4181262591' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-22T10:56:44.126 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:43 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3135852824' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-22T10:56:44.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:43 smithi150 ceph-mon[65513]: pgmap v417: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:44.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:43 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/4181262591' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-22T10:56:44.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:43 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3135852824' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-22T10:56:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.266+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3043861573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 msgr2=0x7fc6f8105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.266+0000 7fc6fcc00640 1 --2- 172.21.15.115:0/3043861573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 0x7fc6f8105920 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009880 tx=0x7fc6e802bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.266+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3043861573 shutdown_connections 2024-03-22T10:56:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.266+0000 7fc6fcc00640 1 --2- 172.21.15.115:0/3043861573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f8067310 0x7fc6f8067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.266+0000 7fc6fcc00640 1 --2- 172.21.15.115:0/3043861573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 0x7fc6f8105920 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.266+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3043861573 >> 172.21.15.115:0/3043861573 conn(0x7fc6f80743d0 msgr2=0x7fc6f80747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.266+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3043861573 shutdown_connections 2024-03-22T10:56:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.266+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3043861573 wait complete. 2024-03-22T10:56:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.267+0000 7fc6fcc00640 1 Processor -- start 2024-03-22T10:56:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.268+0000 7fc6fcc00640 1 -- start start 2024-03-22T10:56:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.268+0000 7fc6fcc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f8067310 0x7fc6f8195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.268+0000 7fc6fcc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 0x7fc6f8195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.268+0000 7fc6fcc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f8195d20 con 0x7fc6f8105550 2024-03-22T10:56:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.268+0000 7fc6fcc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f8199af0 con 0x7fc6f8067310 2024-03-22T10:56:44.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.268+0000 7fc6f6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 0x7fc6f8195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:44.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.268+0000 7fc6f6a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 0x7fc6f8195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37634/0 (socket says 172.21.15.115:37634) 2024-03-22T10:56:44.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.268+0000 7fc6f6a00640 1 -- 172.21.15.115:0/3208470496 learned_addr learned my addr 172.21.15.115:0/3208470496 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.269+0000 7fc6f7400640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f8067310 0x7fc6f8195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.269+0000 7fc6f6a00640 1 -- 172.21.15.115:0/3208470496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f8067310 msgr2=0x7fc6f8195150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.269+0000 7fc6f6a00640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f8067310 0x7fc6f8195150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.269+0000 7fc6f6a00640 1 -- 172.21.15.115:0/3208470496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8009560 con 0x7fc6f8105550 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.269+0000 7fc6f6a00640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 0x7fc6f8195690 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400d6f0 tx=0x7fc6e400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.269+0000 7fc6efe00640 1 -- 172.21.15.115:0/3208470496 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e4009a30 con 0x7fc6f8105550 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.269+0000 7fc6efe00640 1 -- 172.21.15.115:0/3208470496 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6e40054e0 con 0x7fc6f8105550 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.269+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f8199dd0 con 0x7fc6f8105550 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.269+0000 7fc6efe00640 1 -- 172.21.15.115:0/3208470496 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e4010460 con 0x7fc6f8105550 2024-03-22T10:56:44.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.270+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f819a330 con 0x7fc6f8105550 2024-03-22T10:56:44.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.271+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f804e000 con 0x7fc6f8105550 2024-03-22T10:56:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.272+0000 7fc6efe00640 1 -- 172.21.15.115:0/3208470496 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc6e4009b90 con 0x7fc6f8105550 2024-03-22T10:56:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.273+0000 7fc6efe00640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc6dc06df10 0x7fc6dc0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.273+0000 7fc6efe00640 1 -- 172.21.15.115:0/3208470496 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc6e408da20 con 0x7fc6f8105550 2024-03-22T10:56:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.274+0000 7fc6f7400640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc6dc06df10 0x7fc6dc0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.275+0000 7fc6f7400640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc6dc06df10 0x7fc6dc0703c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8006010 tx=0x7fc6e8000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:44.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.278+0000 7fc6efe00640 1 -- 172.21.15.115:0/3208470496 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6e4056d70 con 0x7fc6f8105550 2024-03-22T10:56:44.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.674+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fc6f81964a0 con 0x7fc6f8105550 2024-03-22T10:56:44.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.675+0000 7fc6efe00640 1 -- 172.21.15.115:0/3208470496 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v40) v1 ==== 107+0+4179 (secure 0 0 0) 0x7fc6e405a430 con 0x7fc6f8105550 2024-03-22T10:56:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:44.670 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44297,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/1804848703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1804848703},{"type":"v1","addr":"172.21.15.150:6825","nonce":1804848703}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:51:43.568502+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:44.670 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 34 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc6dc06df10 msgr2=0x7fc6dc0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc6dc06df10 0x7fc6dc0703c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8006010 tx=0x7fc6e8000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 msgr2=0x7fc6f8195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 0x7fc6f8195690 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400d6f0 tx=0x7fc6e400dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 shutdown_connections 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc6dc06df10 0x7fc6dc0703c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6f8105550 0x7fc6f8195690 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 --2- 172.21.15.115:0/3208470496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f8067310 0x7fc6f8195150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 >> 172.21.15.115:0/3208470496 conn(0x7fc6f80743d0 msgr2=0x7fc6f80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 shutdown_connections 2024-03-22T10:56:44.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.678+0000 7fc6fcc00640 1 -- 172.21.15.115:0/3208470496 wait complete. 2024-03-22T10:56:44.698 DEBUG:tasks.fs:max_mds reduced in epoch 34 2024-03-22T10:56:44.699 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-22T10:56:44.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.907+0000 7f5535600640 1 -- 172.21.15.115:0/2330569503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 msgr2=0x7f553010ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:44.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.907+0000 7f5535600640 1 --2- 172.21.15.115:0/2330569503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 0x7f553010ffb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f551c009880 tx=0x7f551c02bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.907+0000 7f5535600640 1 -- 172.21.15.115:0/2330569503 shutdown_connections 2024-03-22T10:56:44.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.907+0000 7f5535600640 1 --2- 172.21.15.115:0/2330569503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 0x7f553010ffb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.907+0000 7f5535600640 1 --2- 172.21.15.115:0/2330569503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5530074790 0x7f5530074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.907+0000 7f5535600640 1 -- 172.21.15.115:0/2330569503 >> 172.21.15.115:0/2330569503 conn(0x7f55300fd370 msgr2=0x7f55300ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:44.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.908+0000 7f5535600640 1 -- 172.21.15.115:0/2330569503 shutdown_connections 2024-03-22T10:56:44.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.908+0000 7f5535600640 1 -- 172.21.15.115:0/2330569503 wait complete. 2024-03-22T10:56:44.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.908+0000 7f5535600640 1 Processor -- start 2024-03-22T10:56:44.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.909+0000 7f5535600640 1 -- start start 2024-03-22T10:56:44.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.909+0000 7f5535600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5530074790 0x7f55301a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:44.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.909+0000 7f5535600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 0x7f55301a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:44.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.909+0000 7f5535600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553019b6d0 con 0x7f55300750a0 2024-03-22T10:56:44.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.909+0000 7f5535600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553019b840 con 0x7f5530074790 2024-03-22T10:56:44.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.909+0000 7f552e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 0x7f55301a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:44.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.909+0000 7f552e000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 0x7f55301a1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37654/0 (socket says 172.21.15.115:37654) 2024-03-22T10:56:44.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.909+0000 7f552e000640 1 -- 172.21.15.115:0/3508470628 learned_addr learned my addr 172.21.15.115:0/3508470628 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:44.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.910+0000 7f552ea00640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5530074790 0x7f55301a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:44.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.910+0000 7f552e000640 1 -- 172.21.15.115:0/3508470628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5530074790 msgr2=0x7f55301a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:44.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.910+0000 7f552e000640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5530074790 0x7f55301a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:44.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.910+0000 7f552e000640 1 -- 172.21.15.115:0/3508470628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c009560 con 0x7f55300750a0 2024-03-22T10:56:44.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.910+0000 7f552ea00640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5530074790 0x7f55301a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:56:44.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.910+0000 7f552e000640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 0x7f55301a1ba0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f551c036040 tx=0x7f551c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:44.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.910+0000 7f5527400640 1 -- 172.21.15.115:0/3508470628 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f551c004aa0 con 0x7f55300750a0 2024-03-22T10:56:44.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.911+0000 7f5527400640 1 -- 172.21.15.115:0/3508470628 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f551c003cc0 con 0x7f55300750a0 2024-03-22T10:56:44.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.911+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553019bac0 con 0x7f55300750a0 2024-03-22T10:56:44.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.911+0000 7f5527400640 1 -- 172.21.15.115:0/3508470628 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f551c034440 con 0x7f55300750a0 2024-03-22T10:56:44.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.911+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553019c020 con 0x7f55300750a0 2024-03-22T10:56:44.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.912+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553004e000 con 0x7f55300750a0 2024-03-22T10:56:44.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.913+0000 7f5527400640 1 -- 172.21.15.115:0/3508470628 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f551c005060 con 0x7f55300750a0 2024-03-22T10:56:44.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.914+0000 7f5527400640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f551406df10 0x7f55140703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:44.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.914+0000 7f5527400640 1 -- 172.21.15.115:0/3508470628 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f551c0b03f0 con 0x7f55300750a0 2024-03-22T10:56:44.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.915+0000 7f552ea00640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f551406df10 0x7f55140703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:44.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.915+0000 7f552ea00640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f551406df10 0x7f55140703c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55200058c0 tx=0x7f5520005d30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:44.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:44.919+0000 7f5527400640 1 -- 172.21.15.115:0/3508470628 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f551c0797c0 con 0x7f55300750a0 2024-03-22T10:56:45.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:44 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3697277419' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-22T10:56:45.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:44 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3208470496' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-22T10:56:45.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.158+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f553010d690 con 0x7f55300750a0 2024-03-22T10:56:45.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.159+0000 7f5527400640 1 -- 172.21.15.115:0/3508470628 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v40) v1 ==== 107+0+4224 (secure 0 0 0) 0x7f551c07ce80 con 0x7f55300750a0 2024-03-22T10:56:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:45.153 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44297,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/1804848703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1804848703},{"type":"v1","addr":"172.21.15.150:6825","nonce":1804848703}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:51:43.581747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34476},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34476":{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:45.153 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 35 2024-03-22T10:56:45.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.161+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f551406df10 msgr2=0x7f55140703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:45.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.161+0000 7f5535600640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f551406df10 0x7f55140703c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55200058c0 tx=0x7f5520005d30 comp rx=0 tx=0).stop 2024-03-22T10:56:45.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.161+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 msgr2=0x7f55301a1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:45.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.161+0000 7f5535600640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 0x7f55301a1ba0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f551c036040 tx=0x7f551c004270 comp rx=0 tx=0).stop 2024-03-22T10:56:45.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.162+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 shutdown_connections 2024-03-22T10:56:45.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.162+0000 7f5535600640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f551406df10 0x7f55140703c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.162+0000 7f5535600640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300750a0 0x7f55301a1ba0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.162+0000 7f5535600640 1 --2- 172.21.15.115:0/3508470628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5530074790 0x7f55301a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.162+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 >> 172.21.15.115:0/3508470628 conn(0x7f55300fd370 msgr2=0x7f55300fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:45.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.162+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 shutdown_connections 2024-03-22T10:56:45.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.162+0000 7f5535600640 1 -- 172.21.15.115:0/3508470628 wait complete. 2024-03-22T10:56:45.179 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:44 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3697277419' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-22T10:56:45.179 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:44 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3208470496' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-22T10:56:45.180 DEBUG:tasks.fs:max_mds reduced in epoch 35 2024-03-22T10:56:45.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-22T10:56:45.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.392+0000 7f2978a00640 1 -- 172.21.15.115:0/3416084080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974107680 msgr2=0x7f2974107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:45.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.392+0000 7f2978a00640 1 --2- 172.21.15.115:0/3416084080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974107680 0x7f2974107a50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2964009880 tx=0x7f296402bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.392+0000 7f2978a00640 1 -- 172.21.15.115:0/3416084080 shutdown_connections 2024-03-22T10:56:45.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.392+0000 7f2978a00640 1 --2- 172.21.15.115:0/3416084080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974101670 0x7f2974101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.392+0000 7f2978a00640 1 --2- 172.21.15.115:0/3416084080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974107680 0x7f2974107a50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.392+0000 7f2978a00640 1 -- 172.21.15.115:0/3416084080 >> 172.21.15.115:0/3416084080 conn(0x7f29740fd170 msgr2=0x7f29740ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:45.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.392+0000 7f2978a00640 1 -- 172.21.15.115:0/3416084080 shutdown_connections 2024-03-22T10:56:45.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.392+0000 7f2978a00640 1 -- 172.21.15.115:0/3416084080 wait complete. 2024-03-22T10:56:45.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.394+0000 7f2978a00640 1 Processor -- start 2024-03-22T10:56:45.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.394+0000 7f2978a00640 1 -- start start 2024-03-22T10:56:45.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.394+0000 7f2978a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974101670 0x7f2974197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:45.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.394+0000 7f2978a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974107680 0x7f2974197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:45.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.394+0000 7f2978a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2974197ed0 con 0x7f2974101670 2024-03-22T10:56:45.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.394+0000 7f2978a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297419bca0 con 0x7f2974107680 2024-03-22T10:56:45.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.395+0000 7f2973400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974101670 0x7f2974197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:45.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.395+0000 7f2973400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974101670 0x7f2974197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37670/0 (socket says 172.21.15.115:37670) 2024-03-22T10:56:45.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.395+0000 7f2973400640 1 -- 172.21.15.115:0/3317896335 learned_addr learned my addr 172.21.15.115:0/3317896335 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:45.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.395+0000 7f2972a00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974107680 0x7f2974197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:45.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.395+0000 7f2973400640 1 -- 172.21.15.115:0/3317896335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974107680 msgr2=0x7f2974197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:45.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.395+0000 7f2973400640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974107680 0x7f2974197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.395+0000 7f2973400640 1 -- 172.21.15.115:0/3317896335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964009560 con 0x7f2974101670 2024-03-22T10:56:45.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.395+0000 7f2972a00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974107680 0x7f2974197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:56:45.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.396+0000 7f2973400640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974101670 0x7f2974197300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f296402c220 tx=0x7f296402c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.396+0000 7f296be00640 1 -- 172.21.15.115:0/3317896335 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2964004520 con 0x7f2974101670 2024-03-22T10:56:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.396+0000 7f296be00640 1 -- 172.21.15.115:0/3317896335 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2964003ab0 con 0x7f2974101670 2024-03-22T10:56:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.396+0000 7f296be00640 1 -- 172.21.15.115:0/3317896335 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2964034440 con 0x7f2974101670 2024-03-22T10:56:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.396+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297419bf20 con 0x7f2974101670 2024-03-22T10:56:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.396+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297419c450 con 0x7f2974101670 2024-03-22T10:56:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.398+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297404e000 con 0x7f2974101670 2024-03-22T10:56:45.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.400+0000 7f296be00640 1 -- 172.21.15.115:0/3317896335 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f2964003c20 con 0x7f2974101670 2024-03-22T10:56:45.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.402+0000 7f296be00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806df10 0x7f29580703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:45.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.402+0000 7f296be00640 1 -- 172.21.15.115:0/3317896335 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f29640b07b0 con 0x7f2974101670 2024-03-22T10:56:45.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.402+0000 7f2972a00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806df10 0x7f29580703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:45.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.403+0000 7f2972a00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806df10 0x7f29580703c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f29600058c0 tx=0x7f2960005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:45.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.405+0000 7f296be00640 1 -- 172.21.15.115:0/3317896335 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2964079b80 con 0x7f2974101670 2024-03-22T10:56:45.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.667+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f2974198650 con 0x7f2974101670 2024-03-22T10:56:45.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.668+0000 7f296be00640 1 -- 172.21.15.115:0/3317896335 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v40) v1 ==== 107+0+4228 (secure 0 0 0) 0x7f296407d240 con 0x7f2974101670 2024-03-22T10:56:45.662 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:45.662 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44297,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/1804848703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1804848703},{"type":"v1","addr":"172.21.15.150:6825","nonce":1804848703}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:51:49.596193+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34476},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34476":{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":35,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:45.663 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 36 2024-03-22T10:56:45.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.670+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806df10 msgr2=0x7f29580703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:45.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.670+0000 7f2978a00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806df10 0x7f29580703c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f29600058c0 tx=0x7f2960005850 comp rx=0 tx=0).stop 2024-03-22T10:56:45.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.670+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974101670 msgr2=0x7f2974197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:45.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.670+0000 7f2978a00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974101670 0x7f2974197300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f296402c220 tx=0x7f296402c7a0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.671+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 shutdown_connections 2024-03-22T10:56:45.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.671+0000 7f2978a00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f295806df10 0x7f29580703c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.671+0000 7f2978a00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2974107680 0x7f2974197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.671+0000 7f2978a00640 1 --2- 172.21.15.115:0/3317896335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2974101670 0x7f2974197300 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.671+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 >> 172.21.15.115:0/3317896335 conn(0x7f29740fd170 msgr2=0x7f29740feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:45.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.671+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 shutdown_connections 2024-03-22T10:56:45.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.671+0000 7f2978a00640 1 -- 172.21.15.115:0/3317896335 wait complete. 2024-03-22T10:56:45.681 DEBUG:tasks.fs:max_mds reduced in epoch 36 2024-03-22T10:56:45.681 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-03-22T10:56:45.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.873+0000 7fc78dc00640 1 -- 172.21.15.115:0/3357764922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 msgr2=0x7fc788107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:45.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.873+0000 7fc78dc00640 1 --2- 172.21.15.115:0/3357764922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 0x7fc788107a90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc778009880 tx=0x7fc77802bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.874+0000 7fc78dc00640 1 -- 172.21.15.115:0/3357764922 shutdown_connections 2024-03-22T10:56:45.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.874+0000 7fc78dc00640 1 --2- 172.21.15.115:0/3357764922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7881016b0 0x7fc788101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.874+0000 7fc78dc00640 1 --2- 172.21.15.115:0/3357764922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 0x7fc788107a90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.874+0000 7fc78dc00640 1 -- 172.21.15.115:0/3357764922 >> 172.21.15.115:0/3357764922 conn(0x7fc7880fd1b0 msgr2=0x7fc7880ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:45.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.874+0000 7fc78dc00640 1 -- 172.21.15.115:0/3357764922 shutdown_connections 2024-03-22T10:56:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.874+0000 7fc78dc00640 1 -- 172.21.15.115:0/3357764922 wait complete. 2024-03-22T10:56:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.875+0000 7fc78dc00640 1 Processor -- start 2024-03-22T10:56:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.875+0000 7fc78dc00640 1 -- start start 2024-03-22T10:56:45.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.876+0000 7fc78dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7881016b0 0x7fc788197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:45.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.876+0000 7fc78dc00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 0x7fc7881979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:45.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.876+0000 7fc78dc00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc788198040 con 0x7fc7881076c0 2024-03-22T10:56:45.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.876+0000 7fc78dc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78819be10 con 0x7fc7881016b0 2024-03-22T10:56:45.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.876+0000 7fc77fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 0x7fc7881979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:45.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.876+0000 7fc77fe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 0x7fc7881979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37698/0 (socket says 172.21.15.115:37698) 2024-03-22T10:56:45.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.876+0000 7fc77fe00640 1 -- 172.21.15.115:0/3598595788 learned_addr learned my addr 172.21.15.115:0/3598595788 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:45.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.876+0000 7fc787e00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7881016b0 0x7fc788197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:45.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.877+0000 7fc77fe00640 1 -- 172.21.15.115:0/3598595788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7881016b0 msgr2=0x7fc788197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:45.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.877+0000 7fc77fe00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7881016b0 0x7fc788197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:45.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.877+0000 7fc77fe00640 1 -- 172.21.15.115:0/3598595788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778009560 con 0x7fc7881076c0 2024-03-22T10:56:45.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.877+0000 7fc77fe00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 0x7fc7881979b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc77000b420 tx=0x7fc77000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:45.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.878+0000 7fc785600640 1 -- 172.21.15.115:0/3598595788 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc770011800 con 0x7fc7881076c0 2024-03-22T10:56:45.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.878+0000 7fc785600640 1 -- 172.21.15.115:0/3598595788 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc770011e20 con 0x7fc7881076c0 2024-03-22T10:56:45.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.878+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78819c0f0 con 0x7fc7881076c0 2024-03-22T10:56:45.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.878+0000 7fc785600640 1 -- 172.21.15.115:0/3598595788 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc77000f2e0 con 0x7fc7881076c0 2024-03-22T10:56:45.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.878+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78819c650 con 0x7fc7881076c0 2024-03-22T10:56:45.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.879+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78804e000 con 0x7fc7881076c0 2024-03-22T10:56:45.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.880+0000 7fc785600640 1 -- 172.21.15.115:0/3598595788 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc770010380 con 0x7fc7881076c0 2024-03-22T10:56:45.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.882+0000 7fc785600640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76c06df10 0x7fc76c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:45.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.882+0000 7fc785600640 1 -- 172.21.15.115:0/3598595788 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc77008dd80 con 0x7fc7881076c0 2024-03-22T10:56:45.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.882+0000 7fc787e00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76c06df10 0x7fc76c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:45.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.883+0000 7fc787e00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76c06df10 0x7fc76c0703c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc778006010 tx=0x7fc778000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:45.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:45.888+0000 7fc785600640 1 -- 172.21.15.115:0/3598595788 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc770057210 con 0x7fc7881076c0 2024-03-22T10:56:45.954 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:45 smithi115 ceph-mon[78672]: pgmap v418: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:45.954 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:45 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3508470628' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-22T10:56:45.954 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:45 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3317896335' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-22T10:56:46.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:45 smithi150 ceph-mon[65513]: pgmap v418: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:46.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:45 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3508470628' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-22T10:56:46.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:45 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3317896335' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-22T10:56:46.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.222+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7fc7881987c0 con 0x7fc7881076c0 2024-03-22T10:56:46.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.223+0000 7fc785600640 1 -- 172.21.15.115:0/3598595788 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v40) v1 ==== 107+0+4225 (secure 0 0 0) 0x7fc77005a8d0 con 0x7fc7881076c0 2024-03-22T10:56:46.217 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:46.217 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44297,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/1804848703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1804848703},{"type":"v1","addr":"172.21.15.150:6825","nonce":1804848703}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:51:50.270609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34476},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34476":{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":35,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-22T10:56:46.218 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 37 2024-03-22T10:56:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76c06df10 msgr2=0x7fc76c0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76c06df10 0x7fc76c0703c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc778006010 tx=0x7fc778000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 msgr2=0x7fc7881979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 0x7fc7881979b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc77000b420 tx=0x7fc77000b8e0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 shutdown_connections 2024-03-22T10:56:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fc76c06df10 0x7fc76c0703c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7881076c0 0x7fc7881979b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 --2- 172.21.15.115:0/3598595788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7881016b0 0x7fc788197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 >> 172.21.15.115:0/3598595788 conn(0x7fc7880fd1b0 msgr2=0x7fc7880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:46.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 shutdown_connections 2024-03-22T10:56:46.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.226+0000 7fc78dc00640 1 -- 172.21.15.115:0/3598595788 wait complete. 2024-03-22T10:56:46.240 DEBUG:tasks.fs:max_mds reduced in epoch 37 2024-03-22T10:56:46.240 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-03-22T10:56:46.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.445+0000 7fca5a800640 1 -- 172.21.15.115:0/1159325539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54101850 msgr2=0x7fca54101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:46.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.445+0000 7fca5a800640 1 --2- 172.21.15.115:0/1159325539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54101850 0x7fca54101cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009880 tx=0x7fca3c02bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.445+0000 7fca5a800640 1 -- 172.21.15.115:0/1159325539 shutdown_connections 2024-03-22T10:56:46.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.445+0000 7fca5a800640 1 --2- 172.21.15.115:0/1159325539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54101850 0x7fca54101cc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.445+0000 7fca5a800640 1 --2- 172.21.15.115:0/1159325539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca54107860 0x7fca54107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.445+0000 7fca5a800640 1 -- 172.21.15.115:0/1159325539 >> 172.21.15.115:0/1159325539 conn(0x7fca540fd370 msgr2=0x7fca540ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:46.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.445+0000 7fca5a800640 1 -- 172.21.15.115:0/1159325539 shutdown_connections 2024-03-22T10:56:46.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.445+0000 7fca5a800640 1 -- 172.21.15.115:0/1159325539 wait complete. 2024-03-22T10:56:46.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.446+0000 7fca5a800640 1 Processor -- start 2024-03-22T10:56:46.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.446+0000 7fca5a800640 1 -- start start 2024-03-22T10:56:46.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.446+0000 7fca5a800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca54101850 0x7fca541974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:46.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.446+0000 7fca5a800640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54107860 0x7fca541979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:46.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.446+0000 7fca5a800640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca54198080 con 0x7fca54107860 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.446+0000 7fca5a800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5419be50 con 0x7fca54101850 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.447+0000 7fca4aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54107860 0x7fca541979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.447+0000 7fca4aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54107860 0x7fca541979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37724/0 (socket says 172.21.15.115:37724) 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.447+0000 7fca4aa00640 1 -- 172.21.15.115:0/1817954535 learned_addr learned my addr 172.21.15.115:0/1817954535 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.447+0000 7fca53400640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca54101850 0x7fca541974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.447+0000 7fca4aa00640 1 -- 172.21.15.115:0/1817954535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca54101850 msgr2=0x7fca541974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.447+0000 7fca4aa00640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca54101850 0x7fca541974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.447+0000 7fca4aa00640 1 -- 172.21.15.115:0/1817954535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c009560 con 0x7fca54107860 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.447+0000 7fca53400640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca54101850 0x7fca541974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.448+0000 7fca4aa00640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54107860 0x7fca541979f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca3c000c00 tx=0x7fca3c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.448+0000 7fca50c00640 1 -- 172.21.15.115:0/1817954535 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3c004520 con 0x7fca54107860 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.448+0000 7fca50c00640 1 -- 172.21.15.115:0/1817954535 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca3c003ab0 con 0x7fca54107860 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.448+0000 7fca50c00640 1 -- 172.21.15.115:0/1817954535 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3c034440 con 0x7fca54107860 2024-03-22T10:56:46.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.448+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5419c0d0 con 0x7fca54107860 2024-03-22T10:56:46.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.448+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5419c650 con 0x7fca54107860 2024-03-22T10:56:46.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.449+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5404e000 con 0x7fca54107860 2024-03-22T10:56:46.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.450+0000 7fca50c00640 1 -- 172.21.15.115:0/1817954535 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fca3c003c20 con 0x7fca54107860 2024-03-22T10:56:46.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.451+0000 7fca50c00640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fca3806df10 0x7fca380703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:46.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.452+0000 7fca50c00640 1 -- 172.21.15.115:0/1817954535 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fca3c0b0130 con 0x7fca54107860 2024-03-22T10:56:46.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.452+0000 7fca53400640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fca3806df10 0x7fca380703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:46.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.452+0000 7fca53400640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fca3806df10 0x7fca380703c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca54102990 tx=0x7fca44009400 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:46.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.456+0000 7fca50c00640 1 -- 172.21.15.115:0/1817954535 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca3c079540 con 0x7fca54107860 2024-03-22T10:56:46.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.765+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7fca54198800 con 0x7fca54107860 2024-03-22T10:56:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.765+0000 7fca50c00640 1 -- 172.21.15.115:0/1817954535 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v40) v1 ==== 107+0+5050 (secure 0 0 0) 0x7fca3c07cc00 con 0x7fca54107860 2024-03-22T10:56:46.760 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:46.760 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44297,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/1804848703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1804848703},{"type":"v1","addr":"172.21.15.150:6825","nonce":1804848703}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44321,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6827/2053213512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2053213512},{"type":"v1","addr":"172.21.15.150:6827","nonce":2053213512}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:51:52.280217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34476},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34476":{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":35,"state":"up:active","state_seq":23,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34476,"qdb_cluster":[34476]},"id":1}]} 2024-03-22T10:56:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 38 2024-03-22T10:56:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fca3806df10 msgr2=0x7fca380703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fca3806df10 0x7fca380703c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca54102990 tx=0x7fca44009400 comp rx=0 tx=0).stop 2024-03-22T10:56:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54107860 msgr2=0x7fca541979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54107860 0x7fca541979f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca3c000c00 tx=0x7fca3c02c850 comp rx=0 tx=0).stop 2024-03-22T10:56:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 shutdown_connections 2024-03-22T10:56:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fca3806df10 0x7fca380703c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca54107860 0x7fca541979f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 --2- 172.21.15.115:0/1817954535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca54101850 0x7fca541974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 >> 172.21.15.115:0/1817954535 conn(0x7fca540fd370 msgr2=0x7fca540fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 shutdown_connections 2024-03-22T10:56:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.768+0000 7fca5a800640 1 -- 172.21.15.115:0/1817954535 wait complete. 2024-03-22T10:56:46.780 DEBUG:tasks.fs:max_mds reduced in epoch 38 2024-03-22T10:56:46.780 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-03-22T10:56:46.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.985+0000 7fb9c5c00640 1 -- 172.21.15.115:0/3068916906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c0105880 msgr2=0x7fb9c0105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:46.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.985+0000 7fb9c5c00640 1 --2- 172.21.15.115:0/3068916906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c0105880 0x7fb9c0105c50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009880 tx=0x7fb9b002bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.985+0000 7fb9c5c00640 1 -- 172.21.15.115:0/3068916906 shutdown_connections 2024-03-22T10:56:46.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.985+0000 7fb9c5c00640 1 --2- 172.21.15.115:0/3068916906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9c00ff870 0x7fb9c00ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.985+0000 7fb9c5c00640 1 --2- 172.21.15.115:0/3068916906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c0105880 0x7fb9c0105c50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.985+0000 7fb9c5c00640 1 -- 172.21.15.115:0/3068916906 >> 172.21.15.115:0/3068916906 conn(0x7fb9c00fb370 msgr2=0x7fb9c00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:46.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.985+0000 7fb9c5c00640 1 -- 172.21.15.115:0/3068916906 shutdown_connections 2024-03-22T10:56:46.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.985+0000 7fb9c5c00640 1 -- 172.21.15.115:0/3068916906 wait complete. 2024-03-22T10:56:46.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.986+0000 7fb9c5c00640 1 Processor -- start 2024-03-22T10:56:46.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.986+0000 7fb9c5c00640 1 -- start start 2024-03-22T10:56:46.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.986+0000 7fb9c5c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c00ff870 0x7fb9c01954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:46.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.986+0000 7fb9c5c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9c0105880 0x7fb9c0195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:46.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.986+0000 7fb9c5c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01960b0 con 0x7fb9c00ff870 2024-03-22T10:56:46.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.986+0000 7fb9c5c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0199e80 con 0x7fb9c0105880 2024-03-22T10:56:46.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.987+0000 7fb9bea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c00ff870 0x7fb9c01954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:46.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.987+0000 7fb9bea00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c00ff870 0x7fb9c01954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37738/0 (socket says 172.21.15.115:37738) 2024-03-22T10:56:46.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.987+0000 7fb9bea00640 1 -- 172.21.15.115:0/2178155233 learned_addr learned my addr 172.21.15.115:0/2178155233 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:46.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.987+0000 7fb9be000640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9c0105880 0x7fb9c0195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:46.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.987+0000 7fb9bea00640 1 -- 172.21.15.115:0/2178155233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9c0105880 msgr2=0x7fb9c0195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:46.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.987+0000 7fb9bea00640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9c0105880 0x7fb9c0195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:46.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.987+0000 7fb9bea00640 1 -- 172.21.15.115:0/2178155233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0009560 con 0x7fb9c00ff870 2024-03-22T10:56:46.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.987+0000 7fb9be000640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9c0105880 0x7fb9c0195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:56:46.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.987+0000 7fb9bea00640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c00ff870 0x7fb9c01954e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb9b002c220 tx=0x7fb9b002c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:46.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.988+0000 7fb9b7400640 1 -- 172.21.15.115:0/2178155233 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b0004520 con 0x7fb9c00ff870 2024-03-22T10:56:46.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.988+0000 7fb9b7400640 1 -- 172.21.15.115:0/2178155233 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9b0003ab0 con 0x7fb9c00ff870 2024-03-22T10:56:46.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.988+0000 7fb9b7400640 1 -- 172.21.15.115:0/2178155233 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b0034440 con 0x7fb9c00ff870 2024-03-22T10:56:46.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.988+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c019a100 con 0x7fb9c00ff870 2024-03-22T10:56:46.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.988+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c019a680 con 0x7fb9c00ff870 2024-03-22T10:56:46.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.989+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c004e000 con 0x7fb9c00ff870 2024-03-22T10:56:46.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.990+0000 7fb9b7400640 1 -- 172.21.15.115:0/2178155233 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb9b0003c20 con 0x7fb9c00ff870 2024-03-22T10:56:46.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.992+0000 7fb9b7400640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9a406df10 0x7fb9a40703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:46.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.992+0000 7fb9b7400640 1 -- 172.21.15.115:0/2178155233 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb9b00b0200 con 0x7fb9c00ff870 2024-03-22T10:56:46.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.992+0000 7fb9be000640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9a406df10 0x7fb9a40703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:46.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:46.993+0000 7fb9be000640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9a406df10 0x7fb9a40703c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0196af0 tx=0x7fb9ac005e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:46.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.000+0000 7fb9b7400640 1 -- 172.21.15.115:0/2178155233 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb9b0079550 con 0x7fb9c00ff870 2024-03-22T10:56:47.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:46 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3598595788' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-22T10:56:47.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:46 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1817954535' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-22T10:56:47.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:46 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3598595788' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-22T10:56:47.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:46 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1817954535' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-22T10:56:47.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.403+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7fb9c010ca40 con 0x7fb9c00ff870 2024-03-22T10:56:47.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.404+0000 7fb9b7400640 1 -- 172.21.15.115:0/2178155233 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v40) v1 ==== 107+0+5067 (secure 0 0 0) 0x7fb9b007cc10 con 0x7fb9c00ff870 2024-03-22T10:56:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:47.398 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44297,"name":"cephfs.smithi150.zvyrzc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6825/1804848703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":1804848703},{"type":"v1","addr":"172.21.15.150:6825","nonce":1804848703}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44321,"name":"cephfs.smithi150.slzwwb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6827/2053213512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2053213512},{"type":"v1","addr":"172.21.15.150:6827","nonce":2053213512}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T10:38:29.359957+0000","modified":"2024-03-22T10:52:04.884633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34476,"mds_1":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34476":{"gid":34476,"name":"cephfs.smithi115.ggxfzi","rank":0,"incarnation":35,"state":"up:active","state_seq":23,"addr":"172.21.15.115:6827/2107356580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2107356580},{"type":"v1","addr":"172.21.15.115:6827","nonce":2107356580}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34508":{"gid":34508,"name":"cephfs.smithi115.wjotmn","rank":1,"incarnation":39,"state":"up:starting","state_seq":1,"addr":"172.21.15.115:6829/3270789943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3270789943},{"type":"v1","addr":"172.21.15.115:6829","nonce":3270789943}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34476,"qdb_cluster":[34476]},"id":1}]} 2024-03-22T10:56:47.399 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 39 2024-03-22T10:56:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9a406df10 msgr2=0x7fb9a40703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9a406df10 0x7fb9a40703c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0196af0 tx=0x7fb9ac005e50 comp rx=0 tx=0).stop 2024-03-22T10:56:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c00ff870 msgr2=0x7fb9c01954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c00ff870 0x7fb9c01954e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb9b002c220 tx=0x7fb9b002c7a0 comp rx=0 tx=0).stop 2024-03-22T10:56:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 shutdown_connections 2024-03-22T10:56:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fb9a406df10 0x7fb9a40703c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9c0105880 0x7fb9c0195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 --2- 172.21.15.115:0/2178155233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c00ff870 0x7fb9c01954e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 >> 172.21.15.115:0/2178155233 conn(0x7fb9c00fb370 msgr2=0x7fb9c00fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 shutdown_connections 2024-03-22T10:56:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.407+0000 7fb9c5c00640 1 -- 172.21.15.115:0/2178155233 wait complete. 2024-03-22T10:56:47.417 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-22T10:56:47.428 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-22T10:56:47.429 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:56:47.429 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T10:56:47.474 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:56:47.475 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T10:56:47.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T10:56:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.826+0000 7f57a0a00640 1 -- 172.21.15.115:0/4162796533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 msgr2=0x7f579c105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.826+0000 7f57a0a00640 1 --2- 172.21.15.115:0/4162796533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 0x7f579c105850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f578c009880 tx=0x7f578c02bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.826+0000 7f57a0a00640 1 -- 172.21.15.115:0/4162796533 shutdown_connections 2024-03-22T10:56:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.826+0000 7f57a0a00640 1 --2- 172.21.15.115:0/4162796533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579c0ff470 0x7f579c0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.826+0000 7f57a0a00640 1 --2- 172.21.15.115:0/4162796533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 0x7f579c105850 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.826+0000 7f57a0a00640 1 -- 172.21.15.115:0/4162796533 >> 172.21.15.115:0/4162796533 conn(0x7f579c0faf90 msgr2=0x7f579c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.827+0000 7f57a0a00640 1 -- 172.21.15.115:0/4162796533 shutdown_connections 2024-03-22T10:56:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.827+0000 7f57a0a00640 1 -- 172.21.15.115:0/4162796533 wait complete. 2024-03-22T10:56:47.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.828+0000 7f57a0a00640 1 Processor -- start 2024-03-22T10:56:47.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.828+0000 7f57a0a00640 1 -- start start 2024-03-22T10:56:47.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f57a0a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579c0ff470 0x7f579c195080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:47.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f57a0a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 0x7f579c1955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:47.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f57a0a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c195c50 con 0x7f579c105480 2024-03-22T10:56:47.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f57a0a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c199a20 con 0x7f579c0ff470 2024-03-22T10:56:47.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f5799e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 0x7f579c1955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:47.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f5799e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 0x7f579c1955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37750/0 (socket says 172.21.15.115:37750) 2024-03-22T10:56:47.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f5799e00640 1 -- 172.21.15.115:0/1084457104 learned_addr learned my addr 172.21.15.115:0/1084457104 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:47.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f579a800640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579c0ff470 0x7f579c195080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:47.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f5799e00640 1 -- 172.21.15.115:0/1084457104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579c0ff470 msgr2=0x7f579c195080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:47.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f5799e00640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579c0ff470 0x7f579c195080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:47.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.829+0000 7f5799e00640 1 -- 172.21.15.115:0/1084457104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009560 con 0x7f579c105480 2024-03-22T10:56:47.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.830+0000 7f5799e00640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 0x7f579c1955c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f578800d6f0 tx=0x7f578800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:47.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.830+0000 7f5793400640 1 -- 172.21.15.115:0/1084457104 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5788009a30 con 0x7f579c105480 2024-03-22T10:56:47.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.830+0000 7f5793400640 1 -- 172.21.15.115:0/1084457104 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57880054e0 con 0x7f579c105480 2024-03-22T10:56:47.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.830+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c199d00 con 0x7f579c105480 2024-03-22T10:56:47.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.830+0000 7f5793400640 1 -- 172.21.15.115:0/1084457104 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5788010460 con 0x7f579c105480 2024-03-22T10:56:47.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.830+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c19a260 con 0x7f579c105480 2024-03-22T10:56:47.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.831+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c04e850 con 0x7f579c105480 2024-03-22T10:56:47.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.833+0000 7f5793400640 1 -- 172.21.15.115:0/1084457104 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5788009b90 con 0x7f579c105480 2024-03-22T10:56:47.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.835+0000 7f5793400640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f578006dfd0 0x7f5780070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:47.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.835+0000 7f5793400640 1 -- 172.21.15.115:0/1084457104 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f578808d8a0 con 0x7f579c105480 2024-03-22T10:56:47.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.835+0000 7f579a800640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f578006dfd0 0x7f5780070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:47.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.836+0000 7f579a800640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f578006dfd0 0x7f5780070480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f578c006010 tx=0x7f578c000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:47.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:47.840+0000 7f5793400640 1 -- 172.21.15.115:0/1084457104 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5788056cf0 con 0x7f579c105480 2024-03-22T10:56:48.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:47 smithi115 ceph-mon[78672]: pgmap v419: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:48.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:47 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2178155233' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-22T10:56:48.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.035+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f579c1963d0 con 0x7f579c105480 2024-03-22T10:56:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.037+0000 7f5793400640 1 -- 172.21.15.115:0/1084457104 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16966 (secure 0 0 0) 0x7f578805a3b0 con 0x7f579c105480 2024-03-22T10:56:48.032 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:48.033 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":83,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","created":"2024-03-22T10:29:37.233959+0000","modified":"2024-03-22T10:51:43.567993+0000","last_up_change":"2024-03-22T10:49:24.797169+0000","last_in_change":"2024-03-22T10:36:35.075110+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T10:35:17.870309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T10:38:27.996809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T10:38:28.878933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a34c0f70-4e9d-4044-b7b4-b6099b74ba6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6803","nonce":2453947745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6805","nonce":2453947745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6809","nonce":2453947745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6807","nonce":2453947745}]},"public_addr":"172.21.15.115:6803/2453947745","cluster_addr":"172.21.15.115:6805/2453947745","heartbeat_back_addr":"172.21.15.115:6809/2453947745","heartbeat_front_addr":"172.21.15.115:6807/2453947745","state":["exists","up"]},{"osd":1,"uuid":"1037db4c-35bc-48da-8950-460c772d50b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6811","nonce":3813430676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6813","nonce":3813430676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6817","nonce":3813430676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6815","nonce":3813430676}]},"public_addr":"172.21.15.115:6811/3813430676","cluster_addr":"172.21.15.115:6813/3813430676","heartbeat_back_addr":"172.21.15.115:6817/3813430676","heartbeat_front_addr":"172.21.15.115:6815/3813430676","state":["exists","up"]},{"osd":2,"uuid":"4e685a22-30ee-4404-bba5-ff944af9b858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6819","nonce":3285562920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6821","nonce":3285562920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6825","nonce":3285562920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6823","nonce":3285562920}]},"public_addr":"172.21.15.115:6819/3285562920","cluster_addr":"172.21.15.115:6821/3285562920","heartbeat_back_addr":"172.21.15.115:6825/3285562920","heartbeat_front_addr":"172.21.15.115:6823/3285562920","state":["exists","up"]},{"osd":3,"uuid":"1273664e-16c8-46c1-bfb4-c51e3e84ea6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6801","nonce":2792834726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6803","nonce":2792834726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6807","nonce":2792834726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6805","nonce":2792834726}]},"public_addr":"172.21.15.150:6801/2792834726","cluster_addr":"172.21.15.150:6803/2792834726","heartbeat_back_addr":"172.21.15.150:6807/2792834726","heartbeat_front_addr":"172.21.15.150:6805/2792834726","state":["exists","up"]},{"osd":4,"uuid":"574fd0d5-c58a-4e16-961d-b384e52d82b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6809","nonce":231697863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6811","nonce":231697863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6815","nonce":231697863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6813","nonce":231697863}]},"public_addr":"172.21.15.150:6809/231697863","cluster_addr":"172.21.15.150:6811/231697863","heartbeat_back_addr":"172.21.15.150:6815/231697863","heartbeat_front_addr":"172.21.15.150:6813/231697863","state":["exists","up"]},{"osd":5,"uuid":"cc37270e-923d-45cf-accd-d4119dcc1a7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6817","nonce":2534589159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6819","nonce":2534589159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6823","nonce":2534589159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6821","nonce":2534589159}]},"public_addr":"172.21.15.150:6817/2534589159","cluster_addr":"172.21.15.150:6819/2534589159","heartbeat_back_addr":"172.21.15.150:6823/2534589159","heartbeat_front_addr":"172.21.15.150:6821/2534589159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:04.542374+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:39.181161+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:16.028358+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:47.869896+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:22.969676+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:57.504061+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6827/2497876489":"2024-03-23T10:51:43.567943+0000","172.21.15.150:6826/2497876489":"2024-03-23T10:51:43.567943+0000","172.21.15.115:6829/4173107662":"2024-03-23T10:50:50.661077+0000","172.21.15.115:6828/4173107662":"2024-03-23T10:50:50.661077+0000","172.21.15.115:6827/3803356871":"2024-03-23T10:50:18.120225+0000","172.21.15.115:6801/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3152688785":"2024-03-23T10:41:27.672754+0000","172.21.15.115:0/450820820":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3897395096":"2024-03-23T10:44:04.061914+0000","172.21.15.115:0/243936731":"2024-03-23T10:44:04.061914+0000","172.21.15.115:6800/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3451764091":"2024-03-23T10:32:25.174415+0000","172.21.15.150:0/2901058091":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/3333444965":"2024-03-23T10:41:27.672754+0000","172.21.15.115:6800/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6801/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/880475782":"2024-03-23T10:44:04.061914+0000","172.21.15.115:0/3264237187":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/4100000632":"2024-03-23T10:44:04.061914+0000","172.21.15.150:0/3168349733":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/901503025":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/1202069415":"2024-03-23T10:41:27.672754+0000","172.21.15.115:0/3025276768":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/2275951342":"2024-03-23T10:30:58.613511+0000","172.21.15.150:6828/3274858902":"2024-03-23T10:42:28.351506+0000","172.21.15.150:0/3722858322":"2024-03-23T10:42:28.351506+0000","172.21.15.150:0/1874309885":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/3641576098":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/1251137983":"2024-03-23T10:41:27.672754+0000","172.21.15.150:0/2220372156":"2024-03-23T10:42:28.351506+0000","172.21.15.115:6800/1251137983":"2024-03-23T10:41:27.672754+0000","172.21.15.115:0/617745307":"2024-03-23T10:41:27.672754+0000","172.21.15.115:6800/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/246266260":"2024-03-23T10:44:04.061914+0000","172.21.15.115:6826/3803356871":"2024-03-23T10:50:18.120225+0000","172.21.15.115:6801/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.150:0/2001872342":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/2167508975":"2024-03-23T10:44:04.061914+0000","172.21.15.150:6829/3274858902":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/1384534494":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/400653309":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6800/246266260":"2024-03-23T10:44:04.061914+0000","172.21.15.115:0/401673952":"2024-03-23T10:44:04.061914+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T10:56:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f578006dfd0 msgr2=0x7f5780070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f578006dfd0 0x7f5780070480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f578c006010 tx=0x7f578c000bc0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 msgr2=0x7f579c1955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 0x7f579c1955c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f578800d6f0 tx=0x7f578800dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 shutdown_connections 2024-03-22T10:56:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f578006dfd0 0x7f5780070480 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c105480 0x7f579c1955c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 --2- 172.21.15.115:0/1084457104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579c0ff470 0x7f579c195080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 >> 172.21.15.115:0/1084457104 conn(0x7f579c0faf90 msgr2=0x7f579c0fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:48.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 shutdown_connections 2024-03-22T10:56:48.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.041+0000 7f57a0a00640 1 -- 172.21.15.115:0/1084457104 wait complete. 2024-03-22T10:56:48.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T10:35:17.870309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T10:38:27.996809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T10:38:28.878933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T10:56:48.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T10:56:48.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:47 smithi150 ceph-mon[65513]: pgmap v419: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:48.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:47 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2178155233' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-22T10:56:48.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.275+0000 7f128d400640 1 -- 172.21.15.115:0/2333408993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12881055c0 msgr2=0x7f1288105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:48.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.275+0000 7f128d400640 1 --2- 172.21.15.115:0/2333408993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12881055c0 0x7f1288105990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1278009880 tx=0x7f127802bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.276+0000 7f128d400640 1 -- 172.21.15.115:0/2333408993 shutdown_connections 2024-03-22T10:56:48.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.276+0000 7f128d400640 1 --2- 172.21.15.115:0/2333408993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1288067310 0x7f1288067760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.276+0000 7f128d400640 1 --2- 172.21.15.115:0/2333408993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12881055c0 0x7f1288105990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.276+0000 7f128d400640 1 -- 172.21.15.115:0/2333408993 >> 172.21.15.115:0/2333408993 conn(0x7f12880743d0 msgr2=0x7f12880747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:48.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.276+0000 7f128d400640 1 -- 172.21.15.115:0/2333408993 shutdown_connections 2024-03-22T10:56:48.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.276+0000 7f128d400640 1 -- 172.21.15.115:0/2333408993 wait complete. 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.277+0000 7f128d400640 1 Processor -- start 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.277+0000 7f128d400640 1 -- start start 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.277+0000 7f128d400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1288067310 0x7f1288197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.277+0000 7f128d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12881055c0 0x7f1288197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.277+0000 7f128d400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1288197f20 con 0x7f1288067310 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.277+0000 7f128d400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128819bcf0 con 0x7f12881055c0 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.277+0000 7f1287e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1288067310 0x7f1288197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.278+0000 7f1287e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1288067310 0x7f1288197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37768/0 (socket says 172.21.15.115:37768) 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.278+0000 7f1287e00640 1 -- 172.21.15.115:0/3081469597 learned_addr learned my addr 172.21.15.115:0/3081469597 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:48.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.278+0000 7f1287400640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12881055c0 0x7f1288197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:48.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.278+0000 7f1287e00640 1 -- 172.21.15.115:0/3081469597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12881055c0 msgr2=0x7f1288197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:48.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.278+0000 7f1287e00640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12881055c0 0x7f1288197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.278+0000 7f1287e00640 1 -- 172.21.15.115:0/3081469597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278009560 con 0x7f1288067310 2024-03-22T10:56:48.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.278+0000 7f1287e00640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1288067310 0x7f1288197350 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1278009230 tx=0x7f1278004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:48.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.279+0000 7f1284c00640 1 -- 172.21.15.115:0/3081469597 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12780035f0 con 0x7f1288067310 2024-03-22T10:56:48.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.279+0000 7f1284c00640 1 -- 172.21.15.115:0/3081469597 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1278003c10 con 0x7f1288067310 2024-03-22T10:56:48.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.279+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128819bf70 con 0x7f1288067310 2024-03-22T10:56:48.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.279+0000 7f1284c00640 1 -- 172.21.15.115:0/3081469597 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1278005670 con 0x7f1288067310 2024-03-22T10:56:48.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.279+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128819c470 con 0x7f1288067310 2024-03-22T10:56:48.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.280+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128804e000 con 0x7f1288067310 2024-03-22T10:56:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.281+0000 7f1284c00640 1 -- 172.21.15.115:0/3081469597 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f1278004dd0 con 0x7f1288067310 2024-03-22T10:56:48.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.282+0000 7f1284c00640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f126c06dec0 0x7f126c070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:48.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.282+0000 7f1284c00640 1 -- 172.21.15.115:0/3081469597 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f12780b0280 con 0x7f1288067310 2024-03-22T10:56:48.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.282+0000 7f1287400640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f126c06dec0 0x7f126c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:48.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.283+0000 7f1287400640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f126c06dec0 0x7f126c070370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1274005fd0 tx=0x7f1274005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:48.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.287+0000 7f1284c00640 1 -- 172.21.15.115:0/3081469597 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f12780795d0 con 0x7f1288067310 2024-03-22T10:56:48.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.556+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f128810c720 con 0x7f1288067310 2024-03-22T10:56:48.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.556+0000 7f1284c00640 1 -- 172.21.15.115:0/3081469597 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f127807cc90 con 0x7f1288067310 2024-03-22T10:56:48.551 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f126c06dec0 msgr2=0x7f126c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f126c06dec0 0x7f126c070370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1274005fd0 tx=0x7f1274005850 comp rx=0 tx=0).stop 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1288067310 msgr2=0x7f1288197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1288067310 0x7f1288197350 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1278009230 tx=0x7f1278004040 comp rx=0 tx=0).stop 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 shutdown_connections 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f126c06dec0 0x7f126c070370 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12881055c0 0x7f1288197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 --2- 172.21.15.115:0/3081469597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1288067310 0x7f1288197350 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 >> 172.21.15.115:0/3081469597 conn(0x7f12880743d0 msgr2=0x7f1288106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:48.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.559+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 shutdown_connections 2024-03-22T10:56:48.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.560+0000 7f128d400640 1 -- 172.21.15.115:0/3081469597 wait complete. 2024-03-22T10:56:48.573 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T10:56:48.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.778+0000 7f4ad5400640 1 -- 172.21.15.115:0/2976060212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 msgr2=0x7f4ad00ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:48.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.778+0000 7f4ad5400640 1 --2- 172.21.15.115:0/2976060212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 0x7f4ad00ff900 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009880 tx=0x7f4ab802bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.778+0000 7f4ad5400640 1 -- 172.21.15.115:0/2976060212 shutdown_connections 2024-03-22T10:56:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.778+0000 7f4ad5400640 1 --2- 172.21.15.115:0/2976060212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 0x7f4ad00ff900 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.778+0000 7f4ad5400640 1 --2- 172.21.15.115:0/2976060212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ad01054a0 0x7f4ad0105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.778+0000 7f4ad5400640 1 -- 172.21.15.115:0/2976060212 >> 172.21.15.115:0/2976060212 conn(0x7f4ad00fafb0 msgr2=0x7f4ad00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.778+0000 7f4ad5400640 1 -- 172.21.15.115:0/2976060212 shutdown_connections 2024-03-22T10:56:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.778+0000 7f4ad5400640 1 -- 172.21.15.115:0/2976060212 wait complete. 2024-03-22T10:56:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.779+0000 7f4ad5400640 1 Processor -- start 2024-03-22T10:56:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.780+0000 7f4ad5400640 1 -- start start 2024-03-22T10:56:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.780+0000 7f4ad5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 0x7f4ad0072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.780+0000 7f4ad5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ad01054a0 0x7f4ad0072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.780+0000 7f4ad5400640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0073070 con 0x7f4ad00ff490 2024-03-22T10:56:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.780+0000 7f4ad5400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00731b0 con 0x7f4ad01054a0 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.780+0000 7f4acfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 0x7f4ad0072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.781+0000 7f4acfe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 0x7f4ad0072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37784/0 (socket says 172.21.15.115:37784) 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.781+0000 7f4acfe00640 1 -- 172.21.15.115:0/2347764268 learned_addr learned my addr 172.21.15.115:0/2347764268 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.781+0000 7f4ac7e00640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ad01054a0 0x7f4ad0072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.781+0000 7f4acfe00640 1 -- 172.21.15.115:0/2347764268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ad01054a0 msgr2=0x7f4ad0072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.781+0000 7f4acfe00640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ad01054a0 0x7f4ad0072aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.781+0000 7f4acfe00640 1 -- 172.21.15.115:0/2347764268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab8009560 con 0x7f4ad00ff490 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.781+0000 7f4ac7e00640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ad01054a0 0x7f4ad0072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.781+0000 7f4acfe00640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 0x7f4ad0072560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000d6f0 tx=0x7f4ac000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.782+0000 7f4acd600640 1 -- 172.21.15.115:0/2347764268 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac0009a30 con 0x7f4ad00ff490 2024-03-22T10:56:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.782+0000 7f4acd600640 1 -- 172.21.15.115:0/2347764268 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ac00054e0 con 0x7f4ad00ff490 2024-03-22T10:56:48.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.782+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad01a1320 con 0x7f4ad00ff490 2024-03-22T10:56:48.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.782+0000 7f4acd600640 1 -- 172.21.15.115:0/2347764268 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac0010460 con 0x7f4ad00ff490 2024-03-22T10:56:48.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.782+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad01a1750 con 0x7f4ad00ff490 2024-03-22T10:56:48.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.783+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad004e000 con 0x7f4ad00ff490 2024-03-22T10:56:48.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.784+0000 7f4acd600640 1 -- 172.21.15.115:0/2347764268 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4ac0009b90 con 0x7f4ad00ff490 2024-03-22T10:56:48.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.785+0000 7f4acd600640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ab406df10 0x7f4ab40703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:48.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.786+0000 7f4acd600640 1 -- 172.21.15.115:0/2347764268 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4ac008da20 con 0x7f4ad00ff490 2024-03-22T10:56:48.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.786+0000 7f4ac7e00640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ab406df10 0x7f4ab40703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:48.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.787+0000 7f4ac7e00640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ab406df10 0x7f4ab40703c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8005ec0 tx=0x7f4ab8005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:48.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:48.790+0000 7f4acd600640 1 -- 172.21.15.115:0/2347764268 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ac0056d70 con 0x7f4ad00ff490 2024-03-22T10:56:49.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.093+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f4ad0073a90 con 0x7f4ad00ff490 2024-03-22T10:56:49.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.093+0000 7f4acd600640 1 -- 172.21.15.115:0/2347764268 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4ac005a430 con 0x7f4ad00ff490 2024-03-22T10:56:49.088 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-03-22T10:56:49.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ab406df10 msgr2=0x7f4ab40703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:49.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ab406df10 0x7f4ab40703c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8005ec0 tx=0x7f4ab8005fd0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 msgr2=0x7f4ad0072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:49.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 0x7f4ad0072560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000d6f0 tx=0x7f4ac000dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 shutdown_connections 2024-03-22T10:56:49.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f4ab406df10 0x7f4ab40703c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ad01054a0 0x7f4ad0072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 --2- 172.21.15.115:0/2347764268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ad00ff490 0x7f4ad0072560 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 >> 172.21.15.115:0/2347764268 conn(0x7f4ad00fafb0 msgr2=0x7f4ad00fb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:49.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 shutdown_connections 2024-03-22T10:56:49.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.096+0000 7f4ad5400640 1 -- 172.21.15.115:0/2347764268 wait complete. 2024-03-22T10:56:49.109 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T10:56:49.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:48 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1084457104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:56:49.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:48 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3081469597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:56:49.176 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:48 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1084457104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:56:49.177 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:48 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3081469597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:56:49.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.323+0000 7fcc2d200640 1 -- 172.21.15.115:0/3618049148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 msgr2=0x7fcc28067760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:49.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.323+0000 7fcc2d200640 1 --2- 172.21.15.115:0/3618049148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 0x7fcc28067760 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009880 tx=0x7fcc1002bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.324+0000 7fcc2d200640 1 -- 172.21.15.115:0/3618049148 shutdown_connections 2024-03-22T10:56:49.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.324+0000 7fcc2d200640 1 --2- 172.21.15.115:0/3618049148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 0x7fcc28067760 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.324+0000 7fcc2d200640 1 --2- 172.21.15.115:0/3618049148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc281055c0 0x7fcc28105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.324+0000 7fcc2d200640 1 -- 172.21.15.115:0/3618049148 >> 172.21.15.115:0/3618049148 conn(0x7fcc280743d0 msgr2=0x7fcc280747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:49.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.324+0000 7fcc2d200640 1 -- 172.21.15.115:0/3618049148 shutdown_connections 2024-03-22T10:56:49.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.324+0000 7fcc2d200640 1 -- 172.21.15.115:0/3618049148 wait complete. 2024-03-22T10:56:49.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.325+0000 7fcc2d200640 1 Processor -- start 2024-03-22T10:56:49.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.325+0000 7fcc2d200640 1 -- start start 2024-03-22T10:56:49.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.325+0000 7fcc2d200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 0x7fcc28195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:49.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.325+0000 7fcc2d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc281055c0 0x7fcc28195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:49.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.325+0000 7fcc2d200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28195d10 con 0x7fcc28067310 2024-03-22T10:56:49.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.325+0000 7fcc2d200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28199ae0 con 0x7fcc281055c0 2024-03-22T10:56:49.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.325+0000 7fcc27400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 0x7fcc28195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:49.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.325+0000 7fcc27400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 0x7fcc28195140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37800/0 (socket says 172.21.15.115:37800) 2024-03-22T10:56:49.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.325+0000 7fcc27400640 1 -- 172.21.15.115:0/3989864912 learned_addr learned my addr 172.21.15.115:0/3989864912 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:49.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.326+0000 7fcc27400640 1 -- 172.21.15.115:0/3989864912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc281055c0 msgr2=0x7fcc28195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:49.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.326+0000 7fcc1fe00640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc281055c0 0x7fcc28195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:49.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.326+0000 7fcc27400640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc281055c0 0x7fcc28195680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.326+0000 7fcc27400640 1 -- 172.21.15.115:0/3989864912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10009560 con 0x7fcc28067310 2024-03-22T10:56:49.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.326+0000 7fcc27400640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 0x7fcc28195140 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800d6f0 tx=0x7fcc1800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:49.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.326+0000 7fcc24c00640 1 -- 172.21.15.115:0/3989864912 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc18009a30 con 0x7fcc28067310 2024-03-22T10:56:49.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.326+0000 7fcc24c00640 1 -- 172.21.15.115:0/3989864912 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc180054e0 con 0x7fcc28067310 2024-03-22T10:56:49.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.326+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc28199dc0 con 0x7fcc28067310 2024-03-22T10:56:49.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.326+0000 7fcc24c00640 1 -- 172.21.15.115:0/3989864912 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc18010460 con 0x7fcc28067310 2024-03-22T10:56:49.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.327+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2819a320 con 0x7fcc28067310 2024-03-22T10:56:49.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.328+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2804e000 con 0x7fcc28067310 2024-03-22T10:56:49.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.329+0000 7fcc24c00640 1 -- 172.21.15.115:0/3989864912 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fcc18009b90 con 0x7fcc28067310 2024-03-22T10:56:49.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.330+0000 7fcc24c00640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc0c06df10 0x7fcc0c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:49.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.330+0000 7fcc24c00640 1 -- 172.21.15.115:0/3989864912 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcc1808d9a0 con 0x7fcc28067310 2024-03-22T10:56:49.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.330+0000 7fcc1fe00640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc0c06df10 0x7fcc0c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:49.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.331+0000 7fcc1fe00640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc0c06df10 0x7fcc0c0703c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc10036040 tx=0x7fcc1002cf90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:49.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.335+0000 7fcc24c00640 1 -- 172.21.15.115:0/3989864912 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc18056d70 con 0x7fcc28067310 2024-03-22T10:56:49.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.629+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fcc28107990 con 0x7fcc28067310 2024-03-22T10:56:49.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.630+0000 7fcc24c00640 1 -- 172.21.15.115:0/3989864912 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcc1805a430 con 0x7fcc28067310 2024-03-22T10:56:49.624 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-03-22T10:56:49.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc0c06df10 msgr2=0x7fcc0c0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc0c06df10 0x7fcc0c0703c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc10036040 tx=0x7fcc1002cf90 comp rx=0 tx=0).stop 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 msgr2=0x7fcc28195140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 0x7fcc28195140 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800d6f0 tx=0x7fcc1800dbb0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 shutdown_connections 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7fcc0c06df10 0x7fcc0c0703c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc281055c0 0x7fcc28195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 --2- 172.21.15.115:0/3989864912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc28067310 0x7fcc28195140 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 >> 172.21.15.115:0/3989864912 conn(0x7fcc280743d0 msgr2=0x7fcc280fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.632+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 shutdown_connections 2024-03-22T10:56:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.633+0000 7fcc2d200640 1 -- 172.21.15.115:0/3989864912 wait complete. 2024-03-22T10:56:49.644 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T10:56:49.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.849+0000 7f4947e00640 1 -- 172.21.15.115:0/3914146686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49481055c0 msgr2=0x7f4948105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:49.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.849+0000 7f4947e00640 1 --2- 172.21.15.115:0/3914146686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49481055c0 0x7f4948105990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4938009880 tx=0x7f493802bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.849+0000 7f4947e00640 1 -- 172.21.15.115:0/3914146686 shutdown_connections 2024-03-22T10:56:49.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.849+0000 7f4947e00640 1 --2- 172.21.15.115:0/3914146686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4948067310 0x7f4948067760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.849+0000 7f4947e00640 1 --2- 172.21.15.115:0/3914146686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49481055c0 0x7f4948105990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.849+0000 7f4947e00640 1 -- 172.21.15.115:0/3914146686 >> 172.21.15.115:0/3914146686 conn(0x7f49480743d0 msgr2=0x7f49480747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:49.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.849+0000 7f4947e00640 1 -- 172.21.15.115:0/3914146686 shutdown_connections 2024-03-22T10:56:49.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.849+0000 7f4947e00640 1 -- 172.21.15.115:0/3914146686 wait complete. 2024-03-22T10:56:49.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.851+0000 7f4947e00640 1 Processor -- start 2024-03-22T10:56:49.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.851+0000 7f4947e00640 1 -- start start 2024-03-22T10:56:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.851+0000 7f4947e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4948067310 0x7f49481974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.851+0000 7f4947e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49481055c0 0x7f49481979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.852+0000 7f4947e00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4948198080 con 0x7f4948067310 2024-03-22T10:56:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.852+0000 7f4947e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494819be50 con 0x7f49481055c0 2024-03-22T10:56:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.852+0000 7f4946a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4948067310 0x7f49481974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.852+0000 7f4946a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4948067310 0x7f49481974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37824/0 (socket says 172.21.15.115:37824) 2024-03-22T10:56:49.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.852+0000 7f4946a00640 1 -- 172.21.15.115:0/1131370806 learned_addr learned my addr 172.21.15.115:0/1131370806 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:49.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.852+0000 7f4946a00640 1 -- 172.21.15.115:0/1131370806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49481055c0 msgr2=0x7f49481979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:49.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.852+0000 7f4946a00640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49481055c0 0x7f49481979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:49.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.852+0000 7f4946a00640 1 -- 172.21.15.115:0/1131370806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938009560 con 0x7f4948067310 2024-03-22T10:56:49.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.852+0000 7f4946a00640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4948067310 0x7f49481974b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4938036040 tx=0x7f4938004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:49.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.853+0000 7f493f400640 1 -- 172.21.15.115:0/1131370806 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49380035f0 con 0x7f4948067310 2024-03-22T10:56:49.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.853+0000 7f493f400640 1 -- 172.21.15.115:0/1131370806 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4938003c10 con 0x7f4948067310 2024-03-22T10:56:49.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.853+0000 7f493f400640 1 -- 172.21.15.115:0/1131370806 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4938034440 con 0x7f4948067310 2024-03-22T10:56:49.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.853+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494819c0d0 con 0x7f4948067310 2024-03-22T10:56:49.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.853+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494819c630 con 0x7f4948067310 2024-03-22T10:56:49.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.854+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494804e000 con 0x7f4948067310 2024-03-22T10:56:49.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.855+0000 7f493f400640 1 -- 172.21.15.115:0/1131370806 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4938005060 con 0x7f4948067310 2024-03-22T10:56:49.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.856+0000 7f493f400640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f492c06df10 0x7f492c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:49.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.856+0000 7f493f400640 1 -- 172.21.15.115:0/1131370806 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f49380b0230 con 0x7f4948067310 2024-03-22T10:56:49.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.856+0000 7f4946000640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f492c06df10 0x7f492c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:49.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.857+0000 7f4946000640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f492c06df10 0x7f492c0703c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4948198ac0 tx=0x7f4934006c90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:49.861+0000 7f493f400640 1 -- 172.21.15.115:0/1131370806 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4938079600 con 0x7f4948067310 2024-03-22T10:56:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:49 smithi115 ceph-mon[78672]: pgmap v420: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:49 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2347764268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T10:56:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:56:50.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:49 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3989864912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T10:56:50.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.097+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4948198800 con 0x7f4948067310 2024-03-22T10:56:50.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.098+0000 7f493f400640 1 -- 172.21.15.115:0/1131370806 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v83) v1 ==== 81+0+2580 (secure 0 0 0) 0x7f4948198800 con 0x7f4948067310 2024-03-22T10:56:50.093 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:6827/2497876489 2024-03-23T10:51:43.567943+0000 2024-03-22T10:56:50.093 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:6826/2497876489 2024-03-23T10:51:43.567943+0000 2024-03-22T10:56:50.093 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6829/4173107662 2024-03-23T10:50:50.661077+0000 2024-03-22T10:56:50.093 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6828/4173107662 2024-03-23T10:50:50.661077+0000 2024-03-22T10:56:50.093 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6827/3803356871 2024-03-23T10:50:18.120225+0000 2024-03-22T10:56:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/2729066911 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3152688785 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/450820820 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3897395096 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/243936731 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/778669970 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3451764091 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/2901058091 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3333444965 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/2729066911 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/778669970 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/880475782 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3264237187 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/4100000632 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/3168349733 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/901503025 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1202069415 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3025276768 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2275951342 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:6828/3274858902 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:50.095 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/3722858322 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:50.097 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/1874309885 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3641576098 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/1251137983 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/2220372156 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/1251137983 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/617745307 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/2262961610 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/246266260 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6826/3803356871 2024-03-23T10:50:18.120225+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/2262961610 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/2001872342 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:50.098 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2167508975 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:50.099 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:6829/3274858902 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:50.099 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1384534494 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:50.099 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/400653309 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:50.099 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/246266260 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:50.099 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/401673952 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:listed 43 entries 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f492c06df10 msgr2=0x7f492c0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f492c06df10 0x7f492c0703c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4948198ac0 tx=0x7f4934006c90 comp rx=0 tx=0).stop 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4948067310 msgr2=0x7f49481974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4948067310 0x7f49481974b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4938036040 tx=0x7f4938004270 comp rx=0 tx=0).stop 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 shutdown_connections 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f492c06df10 0x7f492c0703c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49481055c0 0x7f49481979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 --2- 172.21.15.115:0/1131370806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4948067310 0x7f49481974b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 >> 172.21.15.115:0/1131370806 conn(0x7f49480743d0 msgr2=0x7f49480fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:50.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 shutdown_connections 2024-03-22T10:56:50.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.101+0000 7f4947e00640 1 -- 172.21.15.115:0/1131370806 wait complete. 2024-03-22T10:56:50.113 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-22T10:56:50.113 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T10:56:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:49 smithi150 ceph-mon[65513]: pgmap v420: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:50.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:49 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2347764268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T10:56:50.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:56:50.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:49 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3989864912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T10:56:50.156 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T10:56:50.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.391+0000 7f55af200640 1 -- 172.21.15.115:0/898082809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 msgr2=0x7f55b01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:50.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.391+0000 7f55af200640 1 --2- 172.21.15.115:0/898082809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 0x7f55b01058a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009c20 tx=0x7f55a002c1b0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.392+0000 7f55af200640 1 -- 172.21.15.115:0/898082809 shutdown_connections 2024-03-22T10:56:50.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.392+0000 7f55af200640 1 --2- 172.21.15.115:0/898082809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00ff4c0 0x7f55b00ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.392+0000 7f55af200640 1 --2- 172.21.15.115:0/898082809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 0x7f55b01058a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.392+0000 7f55af200640 1 -- 172.21.15.115:0/898082809 >> 172.21.15.115:0/898082809 conn(0x7f55b00fafc0 msgr2=0x7f55b00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:50.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.392+0000 7f55af200640 1 -- 172.21.15.115:0/898082809 shutdown_connections 2024-03-22T10:56:50.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.392+0000 7f55af200640 1 -- 172.21.15.115:0/898082809 wait complete. 2024-03-22T10:56:50.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.393+0000 7f55af200640 1 Processor -- start 2024-03-22T10:56:50.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.393+0000 7f55af200640 1 -- start start 2024-03-22T10:56:50.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.393+0000 7f55af200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00ff4c0 0x7f55b0195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:50.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.393+0000 7f55af200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 0x7f55b0195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:50.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.393+0000 7f55af200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0195b50 con 0x7f55b01054d0 2024-03-22T10:56:50.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.393+0000 7f55af200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0195cc0 con 0x7f55b00ff4c0 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.394+0000 7f55a5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 0x7f55b0195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.394+0000 7f55a5400640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 0x7f55b0195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37832/0 (socket says 172.21.15.115:37832) 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.394+0000 7f55a5400640 1 -- 172.21.15.115:0/3738886809 learned_addr learned my addr 172.21.15.115:0/3738886809 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.394+0000 7f55ade00640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00ff4c0 0x7f55b0195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.394+0000 7f55a5400640 1 -- 172.21.15.115:0/3738886809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00ff4c0 msgr2=0x7f55b0195010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.394+0000 7f55a5400640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00ff4c0 0x7f55b0195010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.394+0000 7f55a5400640 1 -- 172.21.15.115:0/3738886809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0009900 con 0x7f55b01054d0 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.394+0000 7f55ade00640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00ff4c0 0x7f55b0195010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.395+0000 7f55a5400640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 0x7f55b0195550 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f559800c8a0 tx=0x7f559800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.395+0000 7f55a7400640 1 -- 172.21.15.115:0/3738886809 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5598007b80 con 0x7f55b01054d0 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.395+0000 7f55a7400640 1 -- 172.21.15.115:0/3738886809 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5598007ce0 con 0x7f55b01054d0 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.395+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b0199b50 con 0x7f55b01054d0 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.395+0000 7f55a7400640 1 -- 172.21.15.115:0/3738886809 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5598018810 con 0x7f55b01054d0 2024-03-22T10:56:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.395+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b019a0b0 con 0x7f55b01054d0 2024-03-22T10:56:50.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.396+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b004e850 con 0x7f55b01054d0 2024-03-22T10:56:50.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.397+0000 7f55a7400640 1 -- 172.21.15.115:0/3738886809 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5598018a80 con 0x7f55b01054d0 2024-03-22T10:56:50.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.399+0000 7f55a7400640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f559406dfd0 0x7f5594070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:50.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.399+0000 7f55a7400640 1 -- 172.21.15.115:0/3738886809 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f559808dfc0 con 0x7f55b01054d0 2024-03-22T10:56:50.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.399+0000 7f55ade00640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f559406dfd0 0x7f5594070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.400+0000 7f55ade00640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f559406dfd0 0x7f5594070480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005e30 tx=0x7f55a0005da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:50.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.406+0000 7f55a7400640 1 -- 172.21.15.115:0/3738886809 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5598057390 con 0x7f55b01054d0 2024-03-22T10:56:50.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.606+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55b010c900 con 0x7f55b01054d0 2024-03-22T10:56:50.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.608+0000 7f55a7400640 1 -- 172.21.15.115:0/3738886809 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16966 (secure 0 0 0) 0x7f559805aa50 con 0x7f55b01054d0 2024-03-22T10:56:50.603 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-22T10:56:50.603 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":83,"fsid":"fd1caff4-e836-11ee-95cd-87774f69a715","created":"2024-03-22T10:29:37.233959+0000","modified":"2024-03-22T10:51:43.567993+0000","last_up_change":"2024-03-22T10:49:24.797169+0000","last_in_change":"2024-03-22T10:36:35.075110+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T10:35:17.870309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T10:38:27.996809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T10:38:28.878933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a34c0f70-4e9d-4044-b7b4-b6099b74ba6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6803","nonce":2453947745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6805","nonce":2453947745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6809","nonce":2453947745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2453947745},{"type":"v1","addr":"172.21.15.115:6807","nonce":2453947745}]},"public_addr":"172.21.15.115:6803/2453947745","cluster_addr":"172.21.15.115:6805/2453947745","heartbeat_back_addr":"172.21.15.115:6809/2453947745","heartbeat_front_addr":"172.21.15.115:6807/2453947745","state":["exists","up"]},{"osd":1,"uuid":"1037db4c-35bc-48da-8950-460c772d50b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6811","nonce":3813430676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6813","nonce":3813430676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6817","nonce":3813430676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3813430676},{"type":"v1","addr":"172.21.15.115:6815","nonce":3813430676}]},"public_addr":"172.21.15.115:6811/3813430676","cluster_addr":"172.21.15.115:6813/3813430676","heartbeat_back_addr":"172.21.15.115:6817/3813430676","heartbeat_front_addr":"172.21.15.115:6815/3813430676","state":["exists","up"]},{"osd":2,"uuid":"4e685a22-30ee-4404-bba5-ff944af9b858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6819","nonce":3285562920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6821","nonce":3285562920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6825","nonce":3285562920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3285562920},{"type":"v1","addr":"172.21.15.115:6823","nonce":3285562920}]},"public_addr":"172.21.15.115:6819/3285562920","cluster_addr":"172.21.15.115:6821/3285562920","heartbeat_back_addr":"172.21.15.115:6825/3285562920","heartbeat_front_addr":"172.21.15.115:6823/3285562920","state":["exists","up"]},{"osd":3,"uuid":"1273664e-16c8-46c1-bfb4-c51e3e84ea6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6801","nonce":2792834726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6803","nonce":2792834726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6807","nonce":2792834726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2792834726},{"type":"v1","addr":"172.21.15.150:6805","nonce":2792834726}]},"public_addr":"172.21.15.150:6801/2792834726","cluster_addr":"172.21.15.150:6803/2792834726","heartbeat_back_addr":"172.21.15.150:6807/2792834726","heartbeat_front_addr":"172.21.15.150:6805/2792834726","state":["exists","up"]},{"osd":4,"uuid":"574fd0d5-c58a-4e16-961d-b384e52d82b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6809","nonce":231697863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6811","nonce":231697863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6815","nonce":231697863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":231697863},{"type":"v1","addr":"172.21.15.150:6813","nonce":231697863}]},"public_addr":"172.21.15.150:6809/231697863","cluster_addr":"172.21.15.150:6811/231697863","heartbeat_back_addr":"172.21.15.150:6815/231697863","heartbeat_front_addr":"172.21.15.150:6813/231697863","state":["exists","up"]},{"osd":5,"uuid":"cc37270e-923d-45cf-accd-d4119dcc1a7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6817","nonce":2534589159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6819","nonce":2534589159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6823","nonce":2534589159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2534589159},{"type":"v1","addr":"172.21.15.150:6821","nonce":2534589159}]},"public_addr":"172.21.15.150:6817/2534589159","cluster_addr":"172.21.15.150:6819/2534589159","heartbeat_back_addr":"172.21.15.150:6823/2534589159","heartbeat_front_addr":"172.21.15.150:6821/2534589159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:04.542374+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:34:39.181161+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:16.028358+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:35:47.869896+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:22.969676+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T10:36:57.504061+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6827/2497876489":"2024-03-23T10:51:43.567943+0000","172.21.15.150:6826/2497876489":"2024-03-23T10:51:43.567943+0000","172.21.15.115:6829/4173107662":"2024-03-23T10:50:50.661077+0000","172.21.15.115:6828/4173107662":"2024-03-23T10:50:50.661077+0000","172.21.15.115:6827/3803356871":"2024-03-23T10:50:18.120225+0000","172.21.15.115:6801/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/3152688785":"2024-03-23T10:41:27.672754+0000","172.21.15.115:0/450820820":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3897395096":"2024-03-23T10:44:04.061914+0000","172.21.15.115:0/243936731":"2024-03-23T10:44:04.061914+0000","172.21.15.115:6800/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/3451764091":"2024-03-23T10:32:25.174415+0000","172.21.15.150:0/2901058091":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/3333444965":"2024-03-23T10:41:27.672754+0000","172.21.15.115:6800/2729066911":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6801/778669970":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/880475782":"2024-03-23T10:44:04.061914+0000","172.21.15.115:0/3264237187":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/4100000632":"2024-03-23T10:44:04.061914+0000","172.21.15.150:0/3168349733":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/901503025":"2024-03-23T10:32:25.174415+0000","172.21.15.115:0/1202069415":"2024-03-23T10:41:27.672754+0000","172.21.15.115:0/3025276768":"2024-03-23T10:30:18.281079+0000","172.21.15.115:0/2275951342":"2024-03-23T10:30:58.613511+0000","172.21.15.150:6828/3274858902":"2024-03-23T10:42:28.351506+0000","172.21.15.150:0/3722858322":"2024-03-23T10:42:28.351506+0000","172.21.15.150:0/1874309885":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/3641576098":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/1251137983":"2024-03-23T10:41:27.672754+0000","172.21.15.150:0/2220372156":"2024-03-23T10:42:28.351506+0000","172.21.15.115:6800/1251137983":"2024-03-23T10:41:27.672754+0000","172.21.15.115:0/617745307":"2024-03-23T10:41:27.672754+0000","172.21.15.115:6800/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.115:6801/246266260":"2024-03-23T10:44:04.061914+0000","172.21.15.115:6826/3803356871":"2024-03-23T10:50:18.120225+0000","172.21.15.115:6801/2262961610":"2024-03-23T10:30:18.281079+0000","172.21.15.150:0/2001872342":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/2167508975":"2024-03-23T10:44:04.061914+0000","172.21.15.150:6829/3274858902":"2024-03-23T10:42:28.351506+0000","172.21.15.115:0/1384534494":"2024-03-23T10:30:58.613511+0000","172.21.15.115:0/400653309":"2024-03-23T10:32:25.174415+0000","172.21.15.115:6800/246266260":"2024-03-23T10:44:04.061914+0000","172.21.15.115:0/401673952":"2024-03-23T10:44:04.061914+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T10:56:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.611+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f559406dfd0 msgr2=0x7f5594070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.611+0000 7f55af200640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f559406dfd0 0x7f5594070480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005e30 tx=0x7f55a0005da0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.611+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 msgr2=0x7f55b0195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.611+0000 7f55af200640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 0x7f55b0195550 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f559800c8a0 tx=0x7f559800cd60 comp rx=0 tx=0).stop 2024-03-22T10:56:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.612+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 shutdown_connections 2024-03-22T10:56:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.612+0000 7f55af200640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f559406dfd0 0x7f5594070480 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.612+0000 7f55af200640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b01054d0 0x7f55b0195550 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.612+0000 7f55af200640 1 --2- 172.21.15.115:0/3738886809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00ff4c0 0x7f55b0195010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.612+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 >> 172.21.15.115:0/3738886809 conn(0x7f55b00fafc0 msgr2=0x7f55b00fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:50.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.612+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 shutdown_connections 2024-03-22T10:56:50.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.612+0000 7f55af200640 1 -- 172.21.15.115:0/3738886809 wait complete. 2024-03-22T10:56:50.632 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T10:35:17.870309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T10:38:27.996809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T10:38:28.878933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T10:56:50.632 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T10:56:50.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.834+0000 7f30a1c00640 1 -- 172.21.15.115:0/3431598269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c101880 msgr2=0x7f309c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:50.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.834+0000 7f30a1c00640 1 --2- 172.21.15.115:0/3431598269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c101880 0x7f309c101cf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3084009880 tx=0x7f308402bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.834+0000 7f30a1c00640 1 -- 172.21.15.115:0/3431598269 shutdown_connections 2024-03-22T10:56:50.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.834+0000 7f30a1c00640 1 --2- 172.21.15.115:0/3431598269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c101880 0x7f309c101cf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.834+0000 7f30a1c00640 1 --2- 172.21.15.115:0/3431598269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f309c107890 0x7f309c107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.834+0000 7f30a1c00640 1 -- 172.21.15.115:0/3431598269 >> 172.21.15.115:0/3431598269 conn(0x7f309c0fd360 msgr2=0x7f309c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:50.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.834+0000 7f30a1c00640 1 -- 172.21.15.115:0/3431598269 shutdown_connections 2024-03-22T10:56:50.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.834+0000 7f30a1c00640 1 -- 172.21.15.115:0/3431598269 wait complete. 2024-03-22T10:56:50.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.836+0000 7f30a1c00640 1 Processor -- start 2024-03-22T10:56:50.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.836+0000 7f30a1c00640 1 -- start start 2024-03-22T10:56:50.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.836+0000 7f30a1c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f309c101880 0x7f309c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:50.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.836+0000 7f30a1c00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c107890 0x7f309c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:50.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.836+0000 7f30a1c00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c198090 con 0x7f309c107890 2024-03-22T10:56:50.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.836+0000 7f30a1c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c19be60 con 0x7f309c101880 2024-03-22T10:56:50.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.837+0000 7f309a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c107890 0x7f309c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:50.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.837+0000 7f309a000640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c107890 0x7f309c197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37836/0 (socket says 172.21.15.115:37836) 2024-03-22T10:56:50.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.837+0000 7f309a000640 1 -- 172.21.15.115:0/3474580143 learned_addr learned my addr 172.21.15.115:0/3474580143 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:50.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.837+0000 7f309aa00640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f309c101880 0x7f309c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:50.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.837+0000 7f309a000640 1 -- 172.21.15.115:0/3474580143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f309c101880 msgr2=0x7f309c1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:50.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.837+0000 7f309a000640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f309c101880 0x7f309c1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:50.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.837+0000 7f309a000640 1 -- 172.21.15.115:0/3474580143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3084009560 con 0x7f309c107890 2024-03-22T10:56:50.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.838+0000 7f309a000640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c107890 0x7f309c197a00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3084009230 tx=0x7f3084004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:50.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.838+0000 7f3093400640 1 -- 172.21.15.115:0/3474580143 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30840035f0 con 0x7f309c107890 2024-03-22T10:56:50.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.838+0000 7f3093400640 1 -- 172.21.15.115:0/3474580143 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3084003c10 con 0x7f309c107890 2024-03-22T10:56:50.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.838+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c19c0e0 con 0x7f309c107890 2024-03-22T10:56:50.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.838+0000 7f3093400640 1 -- 172.21.15.115:0/3474580143 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3084005670 con 0x7f309c107890 2024-03-22T10:56:50.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.838+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c19c660 con 0x7f309c107890 2024-03-22T10:56:50.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.840+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c04e000 con 0x7f309c107890 2024-03-22T10:56:50.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.841+0000 7f3093400640 1 -- 172.21.15.115:0/3474580143 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f3084004dd0 con 0x7f309c107890 2024-03-22T10:56:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.843+0000 7f3093400640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f308006df10 0x7f30800703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.843+0000 7f3093400640 1 -- 172.21.15.115:0/3474580143 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f30840b01d0 con 0x7f309c107890 2024-03-22T10:56:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.843+0000 7f309aa00640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f308006df10 0x7f30800703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:50.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.844+0000 7f309aa00640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f308006df10 0x7f30800703c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f309c1029c0 tx=0x7f308c009870 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:50.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:50.848+0000 7f3093400640 1 -- 172.21.15.115:0/3474580143 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f30840795a0 con 0x7f309c107890 2024-03-22T10:56:51.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:50 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1131370806' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T10:56:51.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:50 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3738886809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:56:51.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.051+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f309c10eb10 con 0x7f309c107890 2024-03-22T10:56:51.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.052+0000 7f3093400640 1 -- 172.21.15.115:0/3474580143 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f308407cc60 con 0x7f309c107890 2024-03-22T10:56:51.046 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.054+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f308006df10 msgr2=0x7f30800703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.054+0000 7f30a1c00640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f308006df10 0x7f30800703c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f309c1029c0 tx=0x7f308c009870 comp rx=0 tx=0).stop 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.054+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c107890 msgr2=0x7f309c197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.054+0000 7f30a1c00640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c107890 0x7f309c197a00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3084009230 tx=0x7f3084004040 comp rx=0 tx=0).stop 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.054+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 shutdown_connections 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.055+0000 7f30a1c00640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f308006df10 0x7f30800703c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.055+0000 7f30a1c00640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f309c107890 0x7f309c197a00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.055+0000 7f30a1c00640 1 --2- 172.21.15.115:0/3474580143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f309c101880 0x7f309c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.055+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 >> 172.21.15.115:0/3474580143 conn(0x7f309c0fd360 msgr2=0x7f309c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.055+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 shutdown_connections 2024-03-22T10:56:51.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.055+0000 7f30a1c00640 1 -- 172.21.15.115:0/3474580143 wait complete. 2024-03-22T10:56:51.066 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T10:56:51.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:50 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1131370806' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T10:56:51.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:50 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3738886809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T10:56:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.277+0000 7f6189a00640 1 -- 172.21.15.115:0/3892417028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61841054c0 msgr2=0x7f6184105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.277+0000 7f6189a00640 1 --2- 172.21.15.115:0/3892417028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61841054c0 0x7f6184105890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6174009880 tx=0x7f617402bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.277+0000 7f6189a00640 1 -- 172.21.15.115:0/3892417028 shutdown_connections 2024-03-22T10:56:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.277+0000 7f6189a00640 1 --2- 172.21.15.115:0/3892417028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61840ff4b0 0x7f61840ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.277+0000 7f6189a00640 1 --2- 172.21.15.115:0/3892417028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61841054c0 0x7f6184105890 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.277+0000 7f6189a00640 1 -- 172.21.15.115:0/3892417028 >> 172.21.15.115:0/3892417028 conn(0x7f61840fafb0 msgr2=0x7f61840fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.278+0000 7f6189a00640 1 -- 172.21.15.115:0/3892417028 shutdown_connections 2024-03-22T10:56:51.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.278+0000 7f6189a00640 1 -- 172.21.15.115:0/3892417028 wait complete. 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.278+0000 7f6189a00640 1 Processor -- start 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.279+0000 7f6189a00640 1 -- start start 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.279+0000 7f6189a00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61840ff4b0 0x7f6184072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.279+0000 7f6189a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61841054c0 0x7f6184072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.279+0000 7f6189a00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6184072fe0 con 0x7f61840ff4b0 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.279+0000 7f6189a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6184073150 con 0x7f61841054c0 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.279+0000 7f6183e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61840ff4b0 0x7f6184072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.279+0000 7f6183e00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61840ff4b0 0x7f6184072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37858/0 (socket says 172.21.15.115:37858) 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.279+0000 7f6183e00640 1 -- 172.21.15.115:0/1153096465 learned_addr learned my addr 172.21.15.115:0/1153096465 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.280+0000 7f6183e00640 1 -- 172.21.15.115:0/1153096465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61841054c0 msgr2=0x7f6184072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.280+0000 7f6183400640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61841054c0 0x7f6184072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.280+0000 7f6183e00640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61841054c0 0x7f6184072aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.280+0000 7f6183e00640 1 -- 172.21.15.115:0/1153096465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6174009560 con 0x7f61840ff4b0 2024-03-22T10:56:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.280+0000 7f6183e00640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61840ff4b0 0x7f6184072560 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6174009230 tx=0x7f6174004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.281+0000 7f6180c00640 1 -- 172.21.15.115:0/1153096465 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61740035f0 con 0x7f61840ff4b0 2024-03-22T10:56:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.281+0000 7f6180c00640 1 -- 172.21.15.115:0/1153096465 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6174003c10 con 0x7f61840ff4b0 2024-03-22T10:56:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.281+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6184073370 con 0x7f61840ff4b0 2024-03-22T10:56:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.281+0000 7f6180c00640 1 -- 172.21.15.115:0/1153096465 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6174005670 con 0x7f61840ff4b0 2024-03-22T10:56:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.281+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61841a1660 con 0x7f61840ff4b0 2024-03-22T10:56:51.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.282+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618404e000 con 0x7f61840ff4b0 2024-03-22T10:56:51.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.283+0000 7f6180c00640 1 -- 172.21.15.115:0/1153096465 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6174004dd0 con 0x7f61840ff4b0 2024-03-22T10:56:51.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.284+0000 7f6180c00640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f616806dec0 0x7f6168070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:51.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.285+0000 7f6180c00640 1 -- 172.21.15.115:0/1153096465 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f61740b0280 con 0x7f61840ff4b0 2024-03-22T10:56:51.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.285+0000 7f6183400640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f616806dec0 0x7f6168070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:51.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.285+0000 7f6183400640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f616806dec0 0x7f6168070370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6170005fd0 tx=0x7f6170005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:51.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.289+0000 7f6180c00640 1 -- 172.21.15.115:0/1153096465 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f61740795d0 con 0x7f61840ff4b0 2024-03-22T10:56:51.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.491+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f6184073bc0 con 0x7f61840ff4b0 2024-03-22T10:56:51.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.492+0000 7f6180c00640 1 -- 172.21.15.115:0/1153096465 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f617407cc90 con 0x7f61840ff4b0 2024-03-22T10:56:51.486 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-03-22T10:56:51.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.494+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f616806dec0 msgr2=0x7f6168070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.494+0000 7f6189a00640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f616806dec0 0x7f6168070370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6170005fd0 tx=0x7f6170005850 comp rx=0 tx=0).stop 2024-03-22T10:56:51.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.494+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61840ff4b0 msgr2=0x7f6184072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.494+0000 7f6189a00640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61840ff4b0 0x7f6184072560 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6174009230 tx=0x7f6174004040 comp rx=0 tx=0).stop 2024-03-22T10:56:51.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.494+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 shutdown_connections 2024-03-22T10:56:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.494+0000 7f6189a00640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f616806dec0 0x7f6168070370 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.494+0000 7f6189a00640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61841054c0 0x7f6184072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.494+0000 7f6189a00640 1 --2- 172.21.15.115:0/1153096465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61840ff4b0 0x7f6184072560 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.494+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 >> 172.21.15.115:0/1153096465 conn(0x7f61840fafb0 msgr2=0x7f61840fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.495+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 shutdown_connections 2024-03-22T10:56:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.495+0000 7f6189a00640 1 -- 172.21.15.115:0/1153096465 wait complete. 2024-03-22T10:56:51.507 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T10:56:51.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.743+0000 7f1b71600640 1 -- 172.21.15.115:0/1093026917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c105880 msgr2=0x7f1b6c105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.743+0000 7f1b71600640 1 --2- 172.21.15.115:0/1093026917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c105880 0x7f1b6c105c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009880 tx=0x7f1b5c02bcf0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.743+0000 7f1b71600640 1 -- 172.21.15.115:0/1093026917 shutdown_connections 2024-03-22T10:56:51.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.743+0000 7f1b71600640 1 --2- 172.21.15.115:0/1093026917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c0ff870 0x7f1b6c0ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.743+0000 7f1b71600640 1 --2- 172.21.15.115:0/1093026917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c105880 0x7f1b6c105c50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.743+0000 7f1b71600640 1 -- 172.21.15.115:0/1093026917 >> 172.21.15.115:0/1093026917 conn(0x7f1b6c0fb370 msgr2=0x7f1b6c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:51.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.743+0000 7f1b71600640 1 -- 172.21.15.115:0/1093026917 shutdown_connections 2024-03-22T10:56:51.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.743+0000 7f1b71600640 1 -- 172.21.15.115:0/1093026917 wait complete. 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.744+0000 7f1b71600640 1 Processor -- start 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.744+0000 7f1b71600640 1 -- start start 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.744+0000 7f1b71600640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c0ff870 0x7f1b6c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.744+0000 7f1b71600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c105880 0x7f1b6c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.744+0000 7f1b71600640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c19e190 con 0x7f1b6c0ff870 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.744+0000 7f1b71600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c197670 con 0x7f1b6c105880 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.745+0000 7f1b6aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c0ff870 0x7f1b6c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.745+0000 7f1b6aa00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c0ff870 0x7f1b6c19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37884/0 (socket says 172.21.15.115:37884) 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.745+0000 7f1b6aa00640 1 -- 172.21.15.115:0/2596639166 learned_addr learned my addr 172.21.15.115:0/2596639166 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.745+0000 7f1b6a000640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c105880 0x7f1b6c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:51.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.745+0000 7f1b6aa00640 1 -- 172.21.15.115:0/2596639166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c105880 msgr2=0x7f1b6c19db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.745+0000 7f1b6aa00640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c105880 0x7f1b6c19db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.745+0000 7f1b6aa00640 1 -- 172.21.15.115:0/2596639166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c009560 con 0x7f1b6c0ff870 2024-03-22T10:56:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.745+0000 7f1b6aa00640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c0ff870 0x7f1b6c19d600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009230 tx=0x7f1b5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.746+0000 7f1b63400640 1 -- 172.21.15.115:0/2596639166 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b5c0037c0 con 0x7f1b6c0ff870 2024-03-22T10:56:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.746+0000 7f1b63400640 1 -- 172.21.15.115:0/2596639166 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b5c003de0 con 0x7f1b6c0ff870 2024-03-22T10:56:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.746+0000 7f1b63400640 1 -- 172.21.15.115:0/2596639166 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b5c03e460 con 0x7f1b6c0ff870 2024-03-22T10:56:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.746+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c1978c0 con 0x7f1b6c0ff870 2024-03-22T10:56:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.746+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c197e20 con 0x7f1b6c0ff870 2024-03-22T10:56:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.747+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c04e000 con 0x7f1b6c0ff870 2024-03-22T10:56:51.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.748+0000 7f1b63400640 1 -- 172.21.15.115:0/2596639166 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f1b5c004fd0 con 0x7f1b6c0ff870 2024-03-22T10:56:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.750+0000 7f1b63400640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1b5006df10 0x7f1b500703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.750+0000 7f1b63400640 1 -- 172.21.15.115:0/2596639166 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1b5c0b06e0 con 0x7f1b6c0ff870 2024-03-22T10:56:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.750+0000 7f1b6a000640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1b5006df10 0x7f1b500703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:51.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.751+0000 7f1b6a000640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1b5006df10 0x7f1b500703c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1b58005e10 tx=0x7f1b58005da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:51.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.754+0000 7f1b63400640 1 -- 172.21.15.115:0/2596639166 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1b5c079af0 con 0x7f1b6c0ff870 2024-03-22T10:56:51.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.955+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f1b6c066860 con 0x7f1b6c0ff870 2024-03-22T10:56:51.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.956+0000 7f1b63400640 1 -- 172.21.15.115:0/2596639166 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1b5c07d1b0 con 0x7f1b6c0ff870 2024-03-22T10:56:51.950 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-03-22T10:56:51.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.959+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1b5006df10 msgr2=0x7f1b500703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.959+0000 7f1b71600640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1b5006df10 0x7f1b500703c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1b58005e10 tx=0x7f1b58005da0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.959+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c0ff870 msgr2=0x7f1b6c19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:51.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.959+0000 7f1b71600640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c0ff870 0x7f1b6c19d600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009230 tx=0x7f1b5c004270 comp rx=0 tx=0).stop 2024-03-22T10:56:51.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.959+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 shutdown_connections 2024-03-22T10:56:51.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.959+0000 7f1b71600640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f1b5006df10 0x7f1b500703c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.959+0000 7f1b71600640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c105880 0x7f1b6c19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.959+0000 7f1b71600640 1 --2- 172.21.15.115:0/2596639166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6c0ff870 0x7f1b6c19d600 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:51.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.959+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 >> 172.21.15.115:0/2596639166 conn(0x7f1b6c0fb370 msgr2=0x7f1b6c0fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:51.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.960+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 shutdown_connections 2024-03-22T10:56:51.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:51.960+0000 7f1b71600640 1 -- 172.21.15.115:0/2596639166 wait complete. 2024-03-22T10:56:51.971 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T10:56:52.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:51 smithi150 ceph-mon[65513]: pgmap v421: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:52.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:51 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3474580143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:56:52.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:51 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/1153096465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T10:56:52.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.198+0000 7f79db200640 1 -- 172.21.15.115:0/3710149903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 msgr2=0x7f79d410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:52.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.198+0000 7f79db200640 1 --2- 172.21.15.115:0/3710149903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 0x7f79d410baa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009880 tx=0x7f79bc02bce0 comp rx=0 tx=0).stop 2024-03-22T10:56:52.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.198+0000 7f79db200640 1 -- 172.21.15.115:0/3710149903 shutdown_connections 2024-03-22T10:56:52.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.198+0000 7f79db200640 1 --2- 172.21.15.115:0/3710149903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 0x7f79d410baa0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:52.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.198+0000 7f79db200640 1 --2- 172.21.15.115:0/3710149903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d40fe430 0x7f79d40fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:52.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.198+0000 7f79db200640 1 -- 172.21.15.115:0/3710149903 >> 172.21.15.115:0/3710149903 conn(0x7f79d4075270 msgr2=0x7f79d4075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:52.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.198+0000 7f79db200640 1 -- 172.21.15.115:0/3710149903 shutdown_connections 2024-03-22T10:56:52.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.198+0000 7f79db200640 1 -- 172.21.15.115:0/3710149903 wait complete. 2024-03-22T10:56:52.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.199+0000 7f79db200640 1 Processor -- start 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.199+0000 7f79db200640 1 -- start start 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.200+0000 7f79db200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d40fe430 0x7f79d41973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.200+0000 7f79db200640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 0x7f79d41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.200+0000 7f79db200640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4197f80 con 0x7f79d40fed40 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.200+0000 7f79db200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d419bd50 con 0x7f79d40fe430 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.200+0000 7f79cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 0x7f79d41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.200+0000 7f79cbe00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 0x7f79d41978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51842/0 (socket says 172.21.15.115:51842) 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.200+0000 7f79cbe00640 1 -- 172.21.15.115:0/3776352412 learned_addr learned my addr 172.21.15.115:0/3776352412 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.200+0000 7f79d3e00640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d40fe430 0x7f79d41973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.201+0000 7f79cbe00640 1 -- 172.21.15.115:0/3776352412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d40fe430 msgr2=0x7f79d41973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.201+0000 7f79cbe00640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d40fe430 0x7f79d41973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:52.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.201+0000 7f79cbe00640 1 -- 172.21.15.115:0/3776352412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc009560 con 0x7f79d40fed40 2024-03-22T10:56:52.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.201+0000 7f79cbe00640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 0x7f79d41978f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009230 tx=0x7f79bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:52.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.201+0000 7f79d1600640 1 -- 172.21.15.115:0/3776352412 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79bc0037c0 con 0x7f79d40fed40 2024-03-22T10:56:52.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.201+0000 7f79d1600640 1 -- 172.21.15.115:0/3776352412 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79bc005d50 con 0x7f79d40fed40 2024-03-22T10:56:52.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.201+0000 7f79d1600640 1 -- 172.21.15.115:0/3776352412 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79bc034440 con 0x7f79d40fed40 2024-03-22T10:56:52.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.201+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d419bfd0 con 0x7f79d40fed40 2024-03-22T10:56:52.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.202+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d419c4d0 con 0x7f79d40fed40 2024-03-22T10:56:52.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.203+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d404e000 con 0x7f79d40fed40 2024-03-22T10:56:52.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.204+0000 7f79d1600640 1 -- 172.21.15.115:0/3776352412 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f79bc004dd0 con 0x7f79d40fed40 2024-03-22T10:56:52.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.205+0000 7f79d1600640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f79b806dfd0 0x7f79b8070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T10:56:52.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.205+0000 7f79d1600640 1 -- 172.21.15.115:0/3776352412 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f79bc0b0210 con 0x7f79d40fed40 2024-03-22T10:56:52.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.205+0000 7f79d3e00640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f79b806dfd0 0x7f79b8070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T10:56:52.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.206+0000 7f79d3e00640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f79b806dfd0 0x7f79b8070480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005fd0 tx=0x7f79c4005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T10:56:52.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.213+0000 7f79d1600640 1 -- 172.21.15.115:0/3776352412 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f79bc0795e0 con 0x7f79d40fed40 2024-03-22T10:56:52.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:51 smithi115 ceph-mon[78672]: pgmap v421: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:52.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:51 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3474580143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T10:56:52.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:51 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/1153096465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T10:56:52.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.485+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79d4109180 con 0x7f79d40fed40 2024-03-22T10:56:52.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.486+0000 7f79d1600640 1 -- 172.21.15.115:0/3776352412 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v83) v1 ==== 81+0+2580 (secure 0 0 0) 0x7f79bc0056b0 con 0x7f79d40fed40 2024-03-22T10:56:52.480 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:6827/2497876489 2024-03-23T10:51:43.567943+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:6826/2497876489 2024-03-23T10:51:43.567943+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6829/4173107662 2024-03-23T10:50:50.661077+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6828/4173107662 2024-03-23T10:50:50.661077+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6827/3803356871 2024-03-23T10:50:18.120225+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/2729066911 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3152688785 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/450820820 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3897395096 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/243936731 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/778669970 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3451764091 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/2901058091 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3333444965 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/2729066911 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/778669970 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/880475782 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3264237187 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/4100000632 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/3168349733 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/901503025 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1202069415 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3025276768 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2275951342 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:6828/3274858902 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/3722858322 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/1874309885 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3641576098 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/1251137983 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/2220372156 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/1251137983 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/617745307 2024-03-23T10:41:27.672754+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/2262961610 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/246266260 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6826/3803356871 2024-03-23T10:50:18.120225+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/2262961610 2024-03-23T10:30:18.281079+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:0/2001872342 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2167508975 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.150:6829/3274858902 2024-03-23T10:42:28.351506+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1384534494 2024-03-23T10:30:58.613511+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/400653309 2024-03-23T10:32:25.174415+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/246266260 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:52.483 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/401673952 2024-03-23T10:44:04.061914+0000 2024-03-22T10:56:52.484 INFO:teuthology.orchestra.run.smithi115.stderr:listed 43 entries 2024-03-22T10:56:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f79b806dfd0 msgr2=0x7f79b8070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f79b806dfd0 0x7f79b8070480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005fd0 tx=0x7f79c4005850 comp rx=0 tx=0).stop 2024-03-22T10:56:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 msgr2=0x7f79d41978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T10:56:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 0x7f79d41978f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009230 tx=0x7f79bc004270 comp rx=0 tx=0).stop 2024-03-22T10:56:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 shutdown_connections 2024-03-22T10:56:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:6800/3030573715,v1:172.21.15.115:6801/3030573715] conn(0x7f79b806dfd0 0x7f79b8070480 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79d40fed40 0x7f79d41978f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 --2- 172.21.15.115:0/3776352412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d40fe430 0x7f79d41973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T10:56:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 >> 172.21.15.115:0/3776352412 conn(0x7f79d4075270 msgr2=0x7f79d40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T10:56:52.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.490+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 shutdown_connections 2024-03-22T10:56:52.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-22T10:56:52.491+0000 7f79db200640 1 -- 172.21.15.115:0/3776352412 wait complete. 2024-03-22T10:56:52.503 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi115.front.sepia.ceph.com... 2024-03-22T10:56:52.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T10:56:52.503 DEBUG:teuthology.orchestra.run.smithi115:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-22T10:56:52.588 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T10:56:53.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:52 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/2596639166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T10:56:53.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:52 smithi150 ceph-mon[65513]: from='client.? 172.21.15.115:0/3776352412' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T10:56:53.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:52 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/2596639166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T10:56:53.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:52 smithi115 ceph-mon[78672]: from='client.? 172.21.15.115:0/3776352412' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T10:56:54.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:53 smithi150 ceph-mon[65513]: pgmap v422: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:54.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:53 smithi115 ceph-mon[78672]: pgmap v422: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:56.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:55 smithi150 ceph-mon[65513]: pgmap v423: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:56.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:55 smithi115 ceph-mon[78672]: pgmap v423: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:58.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:57 smithi150 ceph-mon[65513]: pgmap v424: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:56:58.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:57 smithi115 ceph-mon[78672]: pgmap v424: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:00.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:56:59 smithi115 ceph-mon[78672]: pgmap v425: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:00.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:56:59 smithi150 ceph-mon[65513]: pgmap v425: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:02.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:01 smithi115 ceph-mon[78672]: pgmap v426: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:02.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:01 smithi150 ceph-mon[65513]: pgmap v426: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:04.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:03 smithi115 ceph-mon[78672]: pgmap v427: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:04.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:03 smithi150 ceph-mon[65513]: pgmap v427: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:57:05.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:57:06.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:05 smithi115 ceph-mon[78672]: pgmap v428: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:06.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:05 smithi150 ceph-mon[65513]: pgmap v428: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:08.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:07 smithi115 ceph-mon[78672]: pgmap v429: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:08.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:07 smithi150 ceph-mon[65513]: pgmap v429: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:10.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:09 smithi115 ceph-mon[78672]: pgmap v430: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:10.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:09 smithi150 ceph-mon[65513]: pgmap v430: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:11.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:11 smithi150 ceph-mon[65513]: pgmap v431: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:11.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:11 smithi115 ceph-mon[78672]: pgmap v431: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:14.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:13 smithi115 ceph-mon[78672]: pgmap v432: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:14.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:13 smithi150 ceph-mon[65513]: pgmap v432: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:16.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:15 smithi115 ceph-mon[78672]: pgmap v433: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:16.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:15 smithi150 ceph-mon[65513]: pgmap v433: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:18.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:17 smithi115 ceph-mon[78672]: pgmap v434: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:18.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:17 smithi150 ceph-mon[65513]: pgmap v434: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:20.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:19 smithi150 ceph-mon[65513]: pgmap v435: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:20.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:57:20.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:19 smithi115 ceph-mon[78672]: pgmap v435: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:20.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:57:22.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:21 smithi150 ceph-mon[65513]: pgmap v436: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:22.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:21 smithi115 ceph-mon[78672]: pgmap v436: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:24.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:23 smithi150 ceph-mon[65513]: pgmap v437: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:24.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:23 smithi115 ceph-mon[78672]: pgmap v437: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:26.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:25 smithi115 ceph-mon[78672]: pgmap v438: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:26.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:25 smithi150 ceph-mon[65513]: pgmap v438: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:28.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:27 smithi150 ceph-mon[65513]: pgmap v439: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:28.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:27 smithi115 ceph-mon[78672]: pgmap v439: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:30.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:29 smithi150 ceph-mon[65513]: pgmap v440: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:30.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:29 smithi115 ceph-mon[78672]: pgmap v440: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:32.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:31 smithi150 ceph-mon[65513]: pgmap v441: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:32.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:31 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:57:32.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:31 smithi115 ceph-mon[78672]: pgmap v441: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:32.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:31 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:57:33.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:32 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:57:33.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:32 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:57:33.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:32 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:57:33.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:32 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:57:33.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:32 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:57:33.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:32 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:57:34.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:33 smithi150 ceph-mon[65513]: pgmap v442: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:34.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:33 smithi115 ceph-mon[78672]: pgmap v442: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:35.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:57:35.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:57:36.151 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:35 smithi150 ceph-mon[65513]: pgmap v443: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:36.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:35 smithi115 ceph-mon[78672]: pgmap v443: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:38.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:37 smithi150 ceph-mon[65513]: pgmap v444: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:38.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:37 smithi115 ceph-mon[78672]: pgmap v444: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:40.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:39 smithi150 ceph-mon[65513]: pgmap v445: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:40.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:39 smithi115 ceph-mon[78672]: pgmap v445: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:41.401 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:41 smithi150 ceph-mon[65513]: pgmap v446: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:41.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:41 smithi115 ceph-mon[78672]: pgmap v446: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:44.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:43 smithi115 ceph-mon[78672]: pgmap v447: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:44.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:43 smithi150 ceph-mon[65513]: pgmap v447: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:46.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:45 smithi150 ceph-mon[65513]: pgmap v448: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:46.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:45 smithi115 ceph-mon[78672]: pgmap v448: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:48.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:47 smithi150 ceph-mon[65513]: pgmap v449: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:48.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:47 smithi115 ceph-mon[78672]: pgmap v449: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:50.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:49 smithi150 ceph-mon[65513]: pgmap v450: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:50.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:57:50.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:49 smithi115 ceph-mon[78672]: pgmap v450: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:50.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:57:52.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:51 smithi150 ceph-mon[65513]: pgmap v451: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:52.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:51 smithi115 ceph-mon[78672]: pgmap v451: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:54.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:53 smithi150 ceph-mon[65513]: pgmap v452: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:54.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:53 smithi115 ceph-mon[78672]: pgmap v452: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:56.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:55 smithi150 ceph-mon[65513]: pgmap v453: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:56.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:55 smithi115 ceph-mon[78672]: pgmap v453: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:58.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:57 smithi150 ceph-mon[65513]: pgmap v454: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:57:58.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:57 smithi115 ceph-mon[78672]: pgmap v454: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:00.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:57:59 smithi150 ceph-mon[65513]: pgmap v455: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:00.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:57:59 smithi115 ceph-mon[78672]: pgmap v455: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:02.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:01 smithi150 ceph-mon[65513]: pgmap v456: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:02.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:01 smithi115 ceph-mon[78672]: pgmap v456: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:04.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:03 smithi150 ceph-mon[65513]: pgmap v457: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:04.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:03 smithi115 ceph-mon[78672]: pgmap v457: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:05.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:58:05.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:58:06.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:05 smithi150 ceph-mon[65513]: pgmap v458: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:06.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:05 smithi115 ceph-mon[78672]: pgmap v458: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:08.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:07 smithi150 ceph-mon[65513]: pgmap v459: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:08.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:07 smithi115 ceph-mon[78672]: pgmap v459: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:10.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:09 smithi150 ceph-mon[65513]: pgmap v460: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:10.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:09 smithi115 ceph-mon[78672]: pgmap v460: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:12.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:11 smithi150 ceph-mon[65513]: pgmap v461: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:12.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:11 smithi115 ceph-mon[78672]: pgmap v461: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:14.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:13 smithi150 ceph-mon[65513]: pgmap v462: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:14.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:13 smithi115 ceph-mon[78672]: pgmap v462: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:16.104 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:15 smithi115 ceph-mon[78672]: pgmap v463: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:16.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:15 smithi150 ceph-mon[65513]: pgmap v463: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:18.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:17 smithi150 ceph-mon[65513]: pgmap v464: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:18.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:17 smithi115 ceph-mon[78672]: pgmap v464: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:20.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:19 smithi150 ceph-mon[65513]: pgmap v465: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:20.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:58:20.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:19 smithi115 ceph-mon[78672]: pgmap v465: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:20.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:58:22.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:21 smithi150 ceph-mon[65513]: pgmap v466: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:22.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:21 smithi115 ceph-mon[78672]: pgmap v466: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:24.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:23 smithi150 ceph-mon[65513]: pgmap v467: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:24.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:23 smithi115 ceph-mon[78672]: pgmap v467: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:26.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:25 smithi150 ceph-mon[65513]: pgmap v468: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:26.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:25 smithi115 ceph-mon[78672]: pgmap v468: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:28.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:27 smithi115 ceph-mon[78672]: pgmap v469: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:28.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:27 smithi150 ceph-mon[65513]: pgmap v469: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:30.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:29 smithi115 ceph-mon[78672]: pgmap v470: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:30.318 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:29 smithi150 ceph-mon[65513]: pgmap v470: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:32.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:31 smithi115 ceph-mon[78672]: pgmap v471: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:32.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:31 smithi150 ceph-mon[65513]: pgmap v471: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:33.264 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:32 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:58:33.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:32 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:58:34.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:33 smithi115 ceph-mon[78672]: pgmap v472: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:34.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:33 smithi150 ceph-mon[65513]: pgmap v472: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:35.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:58:35.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:58:35.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:58:35.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:58:35.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:58:35.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:58:35.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:58:35.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:58:36.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:35 smithi115 ceph-mon[78672]: pgmap v473: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:36.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:35 smithi150 ceph-mon[65513]: pgmap v473: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:38.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:37 smithi115 ceph-mon[78672]: pgmap v474: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:38.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:37 smithi150 ceph-mon[65513]: pgmap v474: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:40.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:39 smithi115 ceph-mon[78672]: pgmap v475: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:40.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:39 smithi150 ceph-mon[65513]: pgmap v475: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:42.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:41 smithi115 ceph-mon[78672]: pgmap v476: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:42.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:41 smithi150 ceph-mon[65513]: pgmap v476: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:44.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:43 smithi115 ceph-mon[78672]: pgmap v477: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:44.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:43 smithi150 ceph-mon[65513]: pgmap v477: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:46.152 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:45 smithi150 ceph-mon[65513]: pgmap v478: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:46.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:45 smithi115 ceph-mon[78672]: pgmap v478: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:48.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:47 smithi115 ceph-mon[78672]: pgmap v479: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:48.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:47 smithi150 ceph-mon[65513]: pgmap v479: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:50.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:49 smithi115 ceph-mon[78672]: pgmap v480: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:50.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:58:50.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:49 smithi150 ceph-mon[65513]: pgmap v480: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:50.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:58:52.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:51 smithi115 ceph-mon[78672]: pgmap v481: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:52.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:51 smithi150 ceph-mon[65513]: pgmap v481: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:54.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:53 smithi115 ceph-mon[78672]: pgmap v482: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:54.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:53 smithi150 ceph-mon[65513]: pgmap v482: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:56.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:55 smithi115 ceph-mon[78672]: pgmap v483: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:56.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:56 smithi150 ceph-mon[65513]: pgmap v483: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:58.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:58:58 smithi115 ceph-mon[78672]: pgmap v484: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:58:58.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:58:58 smithi150 ceph-mon[65513]: pgmap v484: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:00.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:00 smithi115 ceph-mon[78672]: pgmap v485: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:00.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:00 smithi150 ceph-mon[65513]: pgmap v485: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:02.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:02 smithi115 ceph-mon[78672]: pgmap v486: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:02.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:02 smithi150 ceph-mon[65513]: pgmap v486: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:04.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:04 smithi150 ceph-mon[65513]: pgmap v487: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:04.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:04 smithi115 ceph-mon[78672]: pgmap v487: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:05.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:05 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:59:05.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:05 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:59:06.359 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:06 smithi150 ceph-mon[65513]: pgmap v488: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:06.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:06 smithi115 ceph-mon[78672]: pgmap v488: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:08.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:08 smithi150 ceph-mon[65513]: pgmap v489: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:08.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:08 smithi115 ceph-mon[78672]: pgmap v489: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:10.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:10 smithi150 ceph-mon[65513]: pgmap v490: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:10.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:10 smithi115 ceph-mon[78672]: pgmap v490: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:12.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:12 smithi115 ceph-mon[78672]: pgmap v491: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:12.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:12 smithi150 ceph-mon[65513]: pgmap v491: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:14.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:14 smithi150 ceph-mon[65513]: pgmap v492: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:14.418 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:14 smithi115 ceph-mon[78672]: pgmap v492: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:15.402 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:15 smithi150 ceph-mon[65513]: pgmap v493: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:15.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:15 smithi115 ceph-mon[78672]: pgmap v493: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:18.055 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:17 smithi150 ceph-mon[65513]: pgmap v494: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:18.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:17 smithi115 ceph-mon[78672]: pgmap v494: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:20.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:19 smithi115 ceph-mon[78672]: pgmap v495: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:20.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:59:20.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:19 smithi150 ceph-mon[65513]: pgmap v495: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:20.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:59:22.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:21 smithi150 ceph-mon[65513]: pgmap v496: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:22.272 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:21 smithi115 ceph-mon[78672]: pgmap v496: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:24.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:23 smithi150 ceph-mon[65513]: pgmap v497: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:24.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:23 smithi115 ceph-mon[78672]: pgmap v497: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:26.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:25 smithi150 ceph-mon[65513]: pgmap v498: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:26.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:25 smithi115 ceph-mon[78672]: pgmap v498: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:28.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:27 smithi150 ceph-mon[65513]: pgmap v499: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:28.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:27 smithi115 ceph-mon[78672]: pgmap v499: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:30.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:29 smithi150 ceph-mon[65513]: pgmap v500: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:30.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:29 smithi115 ceph-mon[78672]: pgmap v500: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:32.096 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:31 smithi150 ceph-mon[65513]: pgmap v501: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:32.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:31 smithi115 ceph-mon[78672]: pgmap v501: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:34.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:33 smithi150 ceph-mon[65513]: pgmap v502: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:34.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:33 smithi115 ceph-mon[78672]: pgmap v502: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:35.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:59:35.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:34 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:59:35.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:59:35.263 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:34 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T10:59:36.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:35 smithi150 ceph-mon[65513]: pgmap v503: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:36.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:59:36.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:59:36.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:59:36.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:35 smithi115 ceph-mon[78672]: pgmap v503: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:36.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T10:59:36.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T10:59:36.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T10:59:38.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:37 smithi150 ceph-mon[65513]: pgmap v504: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:38.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:37 smithi115 ceph-mon[78672]: pgmap v504: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:40.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:39 smithi150 ceph-mon[65513]: pgmap v505: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:40.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:39 smithi115 ceph-mon[78672]: pgmap v505: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:42.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:41 smithi150 ceph-mon[65513]: pgmap v506: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:42.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:41 smithi115 ceph-mon[78672]: pgmap v506: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:44.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:43 smithi150 ceph-mon[65513]: pgmap v507: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:44.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:43 smithi115 ceph-mon[78672]: pgmap v507: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:46.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:45 smithi150 ceph-mon[65513]: pgmap v508: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:46.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:45 smithi115 ceph-mon[78672]: pgmap v508: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:48.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:47 smithi115 ceph-mon[78672]: pgmap v509: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:48.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:47 smithi150 ceph-mon[65513]: pgmap v509: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:50.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:49 smithi115 ceph-mon[78672]: pgmap v510: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:50.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:59:50.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:49 smithi150 ceph-mon[65513]: pgmap v510: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:50.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T10:59:52.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:51 smithi115 ceph-mon[78672]: pgmap v511: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:52.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:51 smithi150 ceph-mon[65513]: pgmap v511: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:54.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:53 smithi115 ceph-mon[78672]: pgmap v512: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:54.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:53 smithi150 ceph-mon[65513]: pgmap v512: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:56.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:55 smithi115 ceph-mon[78672]: pgmap v513: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:56.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:55 smithi150 ceph-mon[65513]: pgmap v513: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:58.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:57 smithi150 ceph-mon[65513]: pgmap v514: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T10:59:58.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:57 smithi115 ceph-mon[78672]: pgmap v514: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:00.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 10:59:59 smithi115 ceph-mon[78672]: pgmap v515: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:00.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 10:59:59 smithi150 ceph-mon[65513]: pgmap v515: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:01.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:00 smithi115 ceph-mon[78672]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T11:00:01.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:00 smithi150 ceph-mon[65513]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-22T11:00:02.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:01 smithi115 ceph-mon[78672]: pgmap v516: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:02.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:01 smithi150 ceph-mon[65513]: pgmap v516: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:04.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:03 smithi115 ceph-mon[78672]: pgmap v517: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:04.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:03 smithi150 ceph-mon[65513]: pgmap v517: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:05.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:00:05.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:00:06.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:05 smithi115 ceph-mon[78672]: pgmap v518: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:06.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:05 smithi150 ceph-mon[65513]: pgmap v518: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:08.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:07 smithi115 ceph-mon[78672]: pgmap v519: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:08.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:07 smithi150 ceph-mon[65513]: pgmap v519: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:10.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:09 smithi115 ceph-mon[78672]: pgmap v520: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:10.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:09 smithi150 ceph-mon[65513]: pgmap v520: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:12.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:12 smithi115 ceph-mon[78672]: pgmap v521: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:12.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:12 smithi150 ceph-mon[65513]: pgmap v521: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:14.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:14 smithi115 ceph-mon[78672]: pgmap v522: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:14.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:14 smithi150 ceph-mon[65513]: pgmap v522: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:16.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:16 smithi115 ceph-mon[78672]: pgmap v523: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:16.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:16 smithi150 ceph-mon[65513]: pgmap v523: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:18.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:18 smithi115 ceph-mon[78672]: pgmap v524: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:18.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:18 smithi150 ceph-mon[65513]: pgmap v524: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:20.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:20 smithi150 ceph-mon[65513]: pgmap v525: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:20.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:20 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:00:20.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:20 smithi115 ceph-mon[78672]: pgmap v525: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:20.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:20 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:00:22.265 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:22 smithi115 ceph-mon[78672]: pgmap v526: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:22.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:22 smithi150 ceph-mon[65513]: pgmap v526: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:24.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:24 smithi150 ceph-mon[65513]: pgmap v527: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:24.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:24 smithi115 ceph-mon[78672]: pgmap v527: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:26.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:26 smithi150 ceph-mon[65513]: pgmap v528: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:26.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:26 smithi115 ceph-mon[78672]: pgmap v528: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:28.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:28 smithi150 ceph-mon[65513]: pgmap v529: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:28.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:28 smithi115 ceph-mon[78672]: pgmap v529: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:30.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:30 smithi150 ceph-mon[65513]: pgmap v530: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:30.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:30 smithi115 ceph-mon[78672]: pgmap v530: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:32.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:32 smithi115 ceph-mon[78672]: pgmap v531: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:32.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:32 smithi150 ceph-mon[65513]: pgmap v531: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:34.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:34 smithi150 ceph-mon[65513]: pgmap v532: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:34.518 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:34 smithi115 ceph-mon[78672]: pgmap v532: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:35.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:00:35.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:00:36.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:36 smithi150 ceph-mon[65513]: pgmap v533: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:36.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:36 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T11:00:36.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:36 smithi115 ceph-mon[78672]: pgmap v533: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:36.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:36 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T11:00:37.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:37 smithi150 ceph-mon[65513]: pgmap v534: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:37.423 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:37 smithi115 ceph-mon[78672]: pgmap v534: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:38.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T11:00:38.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T11:00:38.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T11:00:38.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T11:00:38.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T11:00:38.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T11:00:39.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:39 smithi150 ceph-mon[65513]: pgmap v535: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:39.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:39 smithi115 ceph-mon[78672]: pgmap v535: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:42.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:41 smithi150 ceph-mon[65513]: pgmap v536: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:42.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:41 smithi115 ceph-mon[78672]: pgmap v536: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:44.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:43 smithi150 ceph-mon[65513]: pgmap v537: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:44.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:43 smithi115 ceph-mon[78672]: pgmap v537: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:46.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:45 smithi150 ceph-mon[65513]: pgmap v538: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:46.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:45 smithi115 ceph-mon[78672]: pgmap v538: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:48.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:47 smithi115 ceph-mon[78672]: pgmap v539: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:48.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:47 smithi150 ceph-mon[65513]: pgmap v539: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:50.026 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:49 smithi150 ceph-mon[65513]: pgmap v540: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:50.027 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:00:50.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:49 smithi115 ceph-mon[78672]: pgmap v540: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:50.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:00:52.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:51 smithi150 ceph-mon[65513]: pgmap v541: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:52.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:51 smithi115 ceph-mon[78672]: pgmap v541: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:54.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:53 smithi150 ceph-mon[65513]: pgmap v542: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:54.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:53 smithi115 ceph-mon[78672]: pgmap v542: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:56.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:55 smithi150 ceph-mon[65513]: pgmap v543: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:56.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:55 smithi115 ceph-mon[78672]: pgmap v543: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:58.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:57 smithi150 ceph-mon[65513]: pgmap v544: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:00:58.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:57 smithi115 ceph-mon[78672]: pgmap v544: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:00.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:00:59 smithi150 ceph-mon[65513]: pgmap v545: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:00.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:00:59 smithi115 ceph-mon[78672]: pgmap v545: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:02.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:01 smithi150 ceph-mon[65513]: pgmap v546: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:02.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:01 smithi115 ceph-mon[78672]: pgmap v546: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:04.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:03 smithi150 ceph-mon[65513]: pgmap v547: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:04.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:03 smithi115 ceph-mon[78672]: pgmap v547: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:05.153 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:04 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:01:05.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:04 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:01:06.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:05 smithi115 ceph-mon[78672]: pgmap v548: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:06.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:05 smithi150 ceph-mon[65513]: pgmap v548: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:08.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:07 smithi115 ceph-mon[78672]: pgmap v549: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:08.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:07 smithi150 ceph-mon[65513]: pgmap v549: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:10.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:09 smithi150 ceph-mon[65513]: pgmap v550: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:10.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:09 smithi115 ceph-mon[78672]: pgmap v550: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:12.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:11 smithi115 ceph-mon[78672]: pgmap v551: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:12.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:11 smithi150 ceph-mon[65513]: pgmap v551: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:14.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:13 smithi115 ceph-mon[78672]: pgmap v552: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:14.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:13 smithi150 ceph-mon[65513]: pgmap v552: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:16.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:15 smithi115 ceph-mon[78672]: pgmap v553: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:16.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:15 smithi150 ceph-mon[65513]: pgmap v553: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:18.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:17 smithi115 ceph-mon[78672]: pgmap v554: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:18.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:17 smithi150 ceph-mon[65513]: pgmap v554: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:20.206 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:19 smithi150 ceph-mon[65513]: pgmap v555: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:20.207 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:19 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:01:20.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:19 smithi115 ceph-mon[78672]: pgmap v555: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:20.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:19 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:01:22.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:21 smithi115 ceph-mon[78672]: pgmap v556: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:22.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:21 smithi150 ceph-mon[65513]: pgmap v556: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:24.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:23 smithi115 ceph-mon[78672]: pgmap v557: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:24.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:23 smithi150 ceph-mon[65513]: pgmap v557: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:26.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:25 smithi115 ceph-mon[78672]: pgmap v558: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:26.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:25 smithi150 ceph-mon[65513]: pgmap v558: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:28.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:27 smithi115 ceph-mon[78672]: pgmap v559: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:28.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:27 smithi150 ceph-mon[65513]: pgmap v559: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:30.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:29 smithi115 ceph-mon[78672]: pgmap v560: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:30.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:29 smithi150 ceph-mon[65513]: pgmap v560: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:32.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:31 smithi115 ceph-mon[78672]: pgmap v561: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:32.403 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:31 smithi150 ceph-mon[65513]: pgmap v561: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:34.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:34 smithi115 ceph-mon[78672]: pgmap v562: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:34.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:34 smithi150 ceph-mon[65513]: pgmap v562: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:35.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:35 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:01:35.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:35 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:01:36.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:36 smithi115 ceph-mon[78672]: pgmap v563: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:36.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:36 smithi150 ceph-mon[65513]: pgmap v563: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:38.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:38 smithi150 ceph-mon[65513]: pgmap v564: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:38.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:38 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T11:01:38.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:38 smithi115 ceph-mon[78672]: pgmap v564: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:38.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:38 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T11:01:39.650 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T11:01:39.650 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:39 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T11:01:39.650 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:39 smithi115 ceph-mon[78672]: pgmap v565: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:39.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T11:01:39.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:39 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T11:01:39.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:39 smithi150 ceph-mon[65513]: pgmap v565: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:40.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T11:01:40.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T11:01:40.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:40 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T11:01:40.766 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T11:01:40.766 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T11:01:40.767 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:40 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' 2024-03-22T11:01:41.766 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:41 smithi115 ceph-mon[78672]: pgmap v566: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:41.904 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:41 smithi150 ceph-mon[65513]: pgmap v566: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:44.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:43 smithi150 ceph-mon[65513]: pgmap v567: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:44.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:43 smithi115 ceph-mon[78672]: pgmap v567: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:46.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:45 smithi150 ceph-mon[65513]: pgmap v568: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:46.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:45 smithi115 ceph-mon[78672]: pgmap v568: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:48.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:47 smithi150 ceph-mon[65513]: pgmap v569: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:48.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:47 smithi115 ceph-mon[78672]: pgmap v569: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:50.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:49 smithi150 ceph-mon[65513]: pgmap v570: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:50.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:49 smithi150 ceph-mon[65513]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:01:50.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:49 smithi115 ceph-mon[78672]: pgmap v570: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:50.266 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:49 smithi115 ceph-mon[78672]: from='mgr.34102 172.21.15.115:0/545326693' entity='mgr.smithi115.iqzwsz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T11:01:51.660 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-03-22T11:01:51.661 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-22T11:01:51.663 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-22T11:01:51.675 INFO:tasks.cephadm:Teardown begin 2024-03-22T11:01:51.675 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/cephadm.py", line 1867, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-22T11:01:51.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T11:01:51.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T11:01:51.883 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-22T11:01:51.883 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T11:01:51.932 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T11:01:51.988 INFO:tasks.cephadm:Stopping all daemons... 2024-03-22T11:01:51.988 INFO:tasks.cephadm.mon.smithi115:Stopping mon.smithi115... 2024-03-22T11:01:51.988 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115 2024-03-22T11:01:52.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:51 smithi115 ceph-mon[78672]: pgmap v571: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:52.154 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:51 smithi150 ceph-mon[65513]: pgmap v571: 65 pgs: 65 active+clean; 175 MiB data, 739 MiB used, 536 GiB / 536 GiB avail 2024-03-22T11:01:52.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:52 smithi115 systemd[1]: Stopping Ceph mon.smithi115 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T11:01:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:52 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[78668]: 2024-03-22T11:01:52.542+0000 7f7b8ea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi115 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T11:01:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:52 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115[78668]: 2024-03-22T11:01:52.542+0000 7f7b8ea00700 -1 mon.smithi115@0(leader) e3 *** Got Signal Terminated *** 2024-03-22T11:01:53.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:52 smithi115 podman[130696]: 2024-03-22 11:01:52.696668266 +0000 UTC m=+0.309846183 container died 52dfd9dbb5ad704e5db064ea29c5964479b4acfab8b08bc77ab35825254bc95b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-22T11:01:53.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:53 smithi115 podman[130696]: 2024-03-22 11:01:53.391644475 +0000 UTC m=+1.004822417 container cleanup 52dfd9dbb5ad704e5db064ea29c5964479b4acfab8b08bc77ab35825254bc95b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T11:01:53.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:53 smithi115 bash[130696]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115 2024-03-22T11:01:53.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Mar 22 11:01:53 smithi115 podman[130721]: 2024-03-22 11:01:53.651392423 +0000 UTC m=+0.944668382 container remove 52dfd9dbb5ad704e5db064ea29c5964479b4acfab8b08bc77ab35825254bc95b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi115, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-22T11:01:54.004 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi115.service' 2024-03-22T11:01:54.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T11:01:54.174 INFO:tasks.cephadm.mon.smithi115:Stopped mon.smithi115 2024-03-22T11:01:54.174 INFO:tasks.cephadm.mon.smithi150:Stopping mon.smithi150... 2024-03-22T11:01:54.174 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi150 2024-03-22T11:01:54.654 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:54 smithi150 systemd[1]: Stopping Ceph mon.smithi150 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T11:01:55.123 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:54 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150[65509]: 2024-03-22T11:01:54.769+0000 7f184d400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi150 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T11:01:55.124 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:54 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150[65509]: 2024-03-22T11:01:54.769+0000 7f184d400700 -1 mon.smithi150@1(peon) e3 *** Got Signal Terminated *** 2024-03-22T11:01:55.124 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:54 smithi150 podman[95794]: 2024-03-22 11:01:54.843285723 +0000 UTC m=+0.263609964 container died f35dc3320e8598c8a38f7bf12a64810fc7412d12a4483d231c6270c98da2cba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T11:01:55.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:55 smithi150 podman[95794]: 2024-03-22 11:01:55.346318795 +0000 UTC m=+0.766643049 container cleanup f35dc3320e8598c8a38f7bf12a64810fc7412d12a4483d231c6270c98da2cba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True) 2024-03-22T11:01:55.404 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:55 smithi150 bash[95794]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150 2024-03-22T11:01:55.904 INFO:journalctl@ceph.mon.smithi150.smithi150.stdout:Mar 22 11:01:55 smithi150 podman[95808]: 2024-03-22 11:01:55.5562042 +0000 UTC m=+0.693745747 container remove f35dc3320e8598c8a38f7bf12a64810fc7412d12a4483d231c6270c98da2cba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-mon-smithi150, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T11:01:56.073 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@mon.smithi150.service' 2024-03-22T11:01:56.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T11:01:56.203 INFO:tasks.cephadm.mon.smithi150:Stopped mon.smithi150 2024-03-22T11:01:56.203 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-22T11:01:56.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.0 2024-03-22T11:01:56.759 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:01:56 smithi115 systemd[1]: Stopping Ceph osd.0 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T11:01:57.017 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:01:56 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[85337]: 2024-03-22T11:01:56.762+0000 7f865ba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T11:01:57.017 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:01:56 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[85337]: 2024-03-22T11:01:56.762+0000 7f865ba00700 -1 osd.0 83 *** Got signal Terminated *** 2024-03-22T11:01:57.017 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:01:56 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0[85337]: 2024-03-22T11:01:56.762+0000 7f865ba00700 -1 osd.0 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T11:02:02.450 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:02 smithi115 podman[130838]: 2024-03-22 11:02:02.166875166 +0000 UTC m=+5.553738536 container died f8691567d116ed892bef35b3e110448b5d80597ec13e98fbde7cc2a541306e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-22T11:02:02.766 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:02 smithi115 podman[130838]: 2024-03-22 11:02:02.564610791 +0000 UTC m=+5.951474187 container cleanup f8691567d116ed892bef35b3e110448b5d80597ec13e98fbde7cc2a541306e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-22T11:02:02.766 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:02 smithi115 bash[130838]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0 2024-03-22T11:02:03.266 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:02 smithi115 podman[130856]: 2024-03-22 11:02:02.899307137 +0000 UTC m=+0.718247249 container remove f8691567d116ed892bef35b3e110448b5d80597ec13e98fbde7cc2a541306e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T11:02:03.735 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:03 smithi115 podman[130942]: 2024-03-22 11:02:03.391265385 +0000 UTC m=+0.045376378 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T11:02:03.735 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:03 smithi115 podman[130942]: 2024-03-22 11:02:03.491729044 +0000 UTC m=+0.145840029 container create d99b21e4375821d7069f391635051cb0a3eaa2fd56e6cf70d9e4bba948aaeeee (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-22T11:02:03.735 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:03 smithi115 podman[130942]: 2024-03-22 11:02:03.737795071 +0000 UTC m=+0.391906101 container init d99b21e4375821d7069f391635051cb0a3eaa2fd56e6cf70d9e4bba948aaeeee (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T11:02:04.017 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:03 smithi115 podman[130942]: 2024-03-22 11:02:03.750731185 +0000 UTC m=+0.404842215 container start d99b21e4375821d7069f391635051cb0a3eaa2fd56e6cf70d9e4bba948aaeeee (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-22T11:02:04.017 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:03 smithi115 podman[130942]: 2024-03-22 11:02:03.8795198 +0000 UTC m=+0.533630807 container attach d99b21e4375821d7069f391635051cb0a3eaa2fd56e6cf70d9e4bba948aaeeee (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-22T11:02:04.516 INFO:journalctl@ceph.osd.0.smithi115.stdout:Mar 22 11:02:04 smithi115 podman[130942]: 2024-03-22 11:02:04.255338855 +0000 UTC m=+0.909449902 container died d99b21e4375821d7069f391635051cb0a3eaa2fd56e6cf70d9e4bba948aaeeee (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-22T11:02:05.049 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.0.service' 2024-03-22T11:02:05.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T11:02:05.149 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-22T11:02:05.149 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-22T11:02:05.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.1 2024-03-22T11:02:05.516 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:05 smithi115 systemd[1]: Stopping Ceph osd.1 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T11:02:06.017 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:05 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[88648]: 2024-03-22T11:02:05.749+0000 7f9514a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T11:02:06.017 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:05 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[88648]: 2024-03-22T11:02:05.749+0000 7f9514a00700 -1 osd.1 83 *** Got signal Terminated *** 2024-03-22T11:02:06.017 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:05 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1[88648]: 2024-03-22T11:02:05.749+0000 7f9514a00700 -1 osd.1 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T11:02:11.616 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:11 smithi115 podman[131048]: 2024-03-22 11:02:11.321649479 +0000 UTC m=+5.729168550 container died 4e693719f2d78ed18e0f004cd0fab32a479c9fdc3e3eed7c5da10b8b423007d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T11:02:11.891 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:11 smithi115 podman[131048]: 2024-03-22 11:02:11.751523031 +0000 UTC m=+6.159042099 container cleanup 4e693719f2d78ed18e0f004cd0fab32a479c9fdc3e3eed7c5da10b8b423007d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-22T11:02:11.892 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:11 smithi115 bash[131048]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1 2024-03-22T11:02:12.266 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:11 smithi115 podman[131067]: 2024-03-22 11:02:11.953003713 +0000 UTC m=+0.617981361 container remove 4e693719f2d78ed18e0f004cd0fab32a479c9fdc3e3eed7c5da10b8b423007d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-22T11:02:12.970 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:12 smithi115 podman[131152]: 2024-03-22 11:02:12.576745243 +0000 UTC m=+0.044447032 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T11:02:12.970 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:12 smithi115 podman[131152]: 2024-03-22 11:02:12.729126369 +0000 UTC m=+0.196828194 container create 18776b6b193b8236a29fa85bf2fa69dc5212b333a0d1130a6201f2e104a9a6a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-22T11:02:13.267 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:12 smithi115 podman[131152]: 2024-03-22 11:02:12.973502476 +0000 UTC m=+0.441204276 container init 18776b6b193b8236a29fa85bf2fa69dc5212b333a0d1130a6201f2e104a9a6a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS) 2024-03-22T11:02:13.267 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:12 smithi115 podman[131152]: 2024-03-22 11:02:12.981172648 +0000 UTC m=+0.448874443 container start 18776b6b193b8236a29fa85bf2fa69dc5212b333a0d1130a6201f2e104a9a6a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-22T11:02:13.267 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:13 smithi115 podman[131152]: 2024-03-22 11:02:13.089707096 +0000 UTC m=+0.557408921 container attach 18776b6b193b8236a29fa85bf2fa69dc5212b333a0d1130a6201f2e104a9a6a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-22T11:02:13.767 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 22 11:02:13 smithi115 podman[131152]: 2024-03-22 11:02:13.503994341 +0000 UTC m=+0.971696172 container died 18776b6b193b8236a29fa85bf2fa69dc5212b333a0d1130a6201f2e104a9a6a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-22T11:02:14.285 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.1.service' 2024-03-22T11:02:14.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T11:02:14.409 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-22T11:02:14.409 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-22T11:02:14.410 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.2 2024-03-22T11:02:14.960 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:14 smithi115 systemd[1]: Stopping Ceph osd.2 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T11:02:15.267 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:14 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[91964]: 2024-03-22T11:02:14.962+0000 7f1573400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T11:02:15.267 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:14 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[91964]: 2024-03-22T11:02:14.962+0000 7f1573400700 -1 osd.2 83 *** Got signal Terminated *** 2024-03-22T11:02:15.267 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:14 smithi115 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2[91964]: 2024-03-22T11:02:14.962+0000 7f1573400700 -1 osd.2 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T11:02:20.874 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:20 smithi115 podman[131257]: 2024-03-22 11:02:20.532581132 +0000 UTC m=+5.709402114 container died 2e695f9baca8a550d04baaabbce66e3fd5e5ba7f7a1e5e89d00880ff8ff83f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T11:02:21.227 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:21 smithi115 podman[131257]: 2024-03-22 11:02:21.012291516 +0000 UTC m=+6.189112522 container cleanup 2e695f9baca8a550d04baaabbce66e3fd5e5ba7f7a1e5e89d00880ff8ff83f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T11:02:21.227 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:21 smithi115 bash[131257]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2 2024-03-22T11:02:21.516 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:21 smithi115 podman[131287]: 2024-03-22 11:02:21.230035092 +0000 UTC m=+0.683300987 container remove 2e695f9baca8a550d04baaabbce66e3fd5e5ba7f7a1e5e89d00880ff8ff83f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-22T11:02:22.231 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:21 smithi115 podman[131372]: 2024-03-22 11:02:21.853977801 +0000 UTC m=+0.045832388 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T11:02:22.231 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:21 smithi115 podman[131372]: 2024-03-22 11:02:21.989140082 +0000 UTC m=+0.180994664 container create d1a5d41468ebb650154d742799fbe9f58d5200352787c68405ac8adb20feb156 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-03-22T11:02:22.231 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:22 smithi115 podman[131372]: 2024-03-22 11:02:22.23435363 +0000 UTC m=+0.426208246 container init d1a5d41468ebb650154d742799fbe9f58d5200352787c68405ac8adb20feb156 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-22T11:02:22.517 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:22 smithi115 podman[131372]: 2024-03-22 11:02:22.246940589 +0000 UTC m=+0.438795168 container start d1a5d41468ebb650154d742799fbe9f58d5200352787c68405ac8adb20feb156 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-22T11:02:22.517 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:22 smithi115 podman[131372]: 2024-03-22 11:02:22.316884552 +0000 UTC m=+0.508739131 container attach d1a5d41468ebb650154d742799fbe9f58d5200352787c68405ac8adb20feb156 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-22T11:02:23.017 INFO:journalctl@ceph.osd.2.smithi115.stdout:Mar 22 11:02:22 smithi115 podman[131398]: 2024-03-22 11:02:22.721061574 +0000 UTC m=+0.030990027 container died d1a5d41468ebb650154d742799fbe9f58d5200352787c68405ac8adb20feb156 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T11:02:23.372 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.2.service' 2024-03-22T11:02:23.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T11:02:23.491 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-22T11:02:23.491 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-22T11:02:23.491 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.3 2024-03-22T11:02:23.904 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:23 smithi150 systemd[1]: Stopping Ceph osd.3 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T11:02:24.404 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:24 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T11:02:24.097+0000 7f7a7c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T11:02:24.405 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:24 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T11:02:24.097+0000 7f7a7c000700 -1 osd.3 83 *** Got signal Terminated *** 2024-03-22T11:02:24.405 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:24 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T11:02:24.097+0000 7f7a7c000700 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T11:02:24.904 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:24 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T11:02:24.607+0000 7f7a77800700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:01:59.801611+0000 front 2024-03-22T11:01:59.801672+0000 (oldest deadline 2024-03-22T11:02:23.901472+0000) 2024-03-22T11:02:24.904 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:24 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:24.443+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:25.904 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:25 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T11:02:25.588+0000 7f7a77800700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:01:59.801611+0000 front 2024-03-22T11:01:59.801672+0000 (oldest deadline 2024-03-22T11:02:23.901472+0000) 2024-03-22T11:02:25.904 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:25 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:25.430+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:26.778 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:26 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T11:02:26.547+0000 7f7a77800700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:01:59.801611+0000 front 2024-03-22T11:01:59.801672+0000 (oldest deadline 2024-03-22T11:02:23.901472+0000) 2024-03-22T11:02:26.778 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:26 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:26.412+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:27.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:26 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:26.780+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:27.822 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:27 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T11:02:27.580+0000 7f7a77800700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:01:59.801611+0000 front 2024-03-22T11:01:59.801672+0000 (oldest deadline 2024-03-22T11:02:23.901472+0000) 2024-03-22T11:02:27.823 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:27 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:27.459+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:28.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:27 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:27.825+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:28.837 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:28 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3[70677]: 2024-03-22T11:02:28.622+0000 7f7a77800700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:01:59.801611+0000 front 2024-03-22T11:01:59.801672+0000 (oldest deadline 2024-03-22T11:02:23.901472+0000) 2024-03-22T11:02:28.837 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:28 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:28.488+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:29.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:28 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:28.839+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:29.847 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:29 smithi150 podman[95930]: 2024-03-22 11:02:29.533795927 +0000 UTC m=+5.609240391 container died e01353665cf38622dacf031ec456182dd3b58e50bb79150b1fb472ac6f6c51bd (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-22T11:02:29.848 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:29 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:29.481+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:30.154 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:29 smithi150 podman[95930]: 2024-03-22 11:02:29.97557007 +0000 UTC m=+6.051014573 container cleanup e01353665cf38622dacf031ec456182dd3b58e50bb79150b1fb472ac6f6c51bd (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-22T11:02:30.155 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:29 smithi150 bash[95930]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3 2024-03-22T11:02:30.155 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:29 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:29.859+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:30.489 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:30 smithi150 podman[95949]: 2024-03-22 11:02:30.218629418 +0000 UTC m=+0.669194980 container remove e01353665cf38622dacf031ec456182dd3b58e50bb79150b1fb472ac6f6c51bd (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-22T11:02:30.876 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:30 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:30.491+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:31.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:30 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:30.879+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:31.155 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:31 smithi150 podman[96038]: 2024-03-22 11:02:30.983190254 +0000 UTC m=+0.035612813 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T11:02:31.155 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:31 smithi150 podman[96038]: 2024-03-22 11:02:31.092051065 +0000 UTC m=+0.144473640 container create 67c2ed50bca2832297fa98ae28bdebf69c3214eb4831d107c6f2e80b268f323c (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-22T11:02:31.490 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:31 smithi150 podman[96038]: 2024-03-22 11:02:31.346066812 +0000 UTC m=+0.398489397 container init 67c2ed50bca2832297fa98ae28bdebf69c3214eb4831d107c6f2e80b268f323c (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-22T11:02:31.490 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:31 smithi150 podman[96038]: 2024-03-22 11:02:31.356951747 +0000 UTC m=+0.409374328 container start 67c2ed50bca2832297fa98ae28bdebf69c3214eb4831d107c6f2e80b268f323c (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-22T11:02:31.490 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:31 smithi150 podman[96038]: 2024-03-22 11:02:31.444520911 +0000 UTC m=+0.496943493 container attach 67c2ed50bca2832297fa98ae28bdebf69c3214eb4831d107c6f2e80b268f323c (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-22T11:02:31.801 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:31 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:31.492+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:32.141 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:31 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:31.834+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:32.141 INFO:journalctl@ceph.osd.3.smithi150.stdout:Mar 22 11:02:31 smithi150 podman[96038]: 2024-03-22 11:02:31.804434328 +0000 UTC m=+0.856856930 container died 67c2ed50bca2832297fa98ae28bdebf69c3214eb4831d107c6f2e80b268f323c (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-22T11:02:32.595 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.3.service' 2024-03-22T11:02:32.712 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:32 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:32.464+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:32.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T11:02:32.716 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-22T11:02:32.716 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-22T11:02:32.716 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.4 2024-03-22T11:02:33.154 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:32 smithi150 systemd[1]: Stopping Ceph osd.4 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T11:02:33.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:32 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:32.829+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:33.654 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:33.350+0000 7f965da00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T11:02:33.655 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:33.350+0000 7f965da00700 -1 osd.4 83 *** Got signal Terminated *** 2024-03-22T11:02:33.655 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:33.350+0000 7f965da00700 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T11:02:33.655 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:33.416+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:34.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:33 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:33.861+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:34.654 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:34 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:34.388+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:35.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:34 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:34.825+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:35.654 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:35 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:35.377+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:36.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:35 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:35.812+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:36.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:35 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:35.812+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:36.654 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:36 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:36.372+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:36.654 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:36 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:36.372+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.587326+0000 front 2024-03-22T11:02:09.587290+0000 (oldest deadline 2024-03-22T11:02:35.486617+0000) 2024-03-22T11:02:37.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:36 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:36.787+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:37.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:36 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:36.787+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:37.654 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:37 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:37.415+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:00.785403+0000 front 2024-03-22T11:02:00.785318+0000 (oldest deadline 2024-03-22T11:02:23.685345+0000) 2024-03-22T11:02:37.655 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:37 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4[73558]: 2024-03-22T11:02:37.415+0000 7f9659200700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.587326+0000 front 2024-03-22T11:02:09.587290+0000 (oldest deadline 2024-03-22T11:02:35.486617+0000) 2024-03-22T11:02:38.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:37 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:37.772+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:38.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:37 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:37.772+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:39.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:38 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:38.818+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:39.155 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:38 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:38.818+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:39.155 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:38 smithi150 podman[96145]: 2024-03-22 11:02:38.853039114 +0000 UTC m=+5.643895518 container died c7c7943cac44275f987843c8df9552d1b35b152eaa7d83954d489ed5bc2a83f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-22T11:02:39.491 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:39 smithi150 podman[96145]: 2024-03-22 11:02:39.326005932 +0000 UTC m=+6.116862357 container cleanup c7c7943cac44275f987843c8df9552d1b35b152eaa7d83954d489ed5bc2a83f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T11:02:39.491 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:39 smithi150 bash[96145]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4 2024-03-22T11:02:39.783 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:39 smithi150 podman[96164]: 2024-03-22 11:02:39.493687192 +0000 UTC m=+0.624703788 container remove c7c7943cac44275f987843c8df9552d1b35b152eaa7d83954d489ed5bc2a83f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-22T11:02:40.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:39 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:39.786+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:40.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:39 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:39.786+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:40.612 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:40 smithi150 podman[96255]: 2024-03-22 11:02:40.199092262 +0000 UTC m=+0.033103572 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T11:02:40.613 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:40 smithi150 podman[96255]: 2024-03-22 11:02:40.328077009 +0000 UTC m=+0.162088341 container create 2198961de166012581c4b1ea90d05253e5ed695d876e2941128e5c09f1dd71ad (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T11:02:40.613 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:40 smithi150 podman[96255]: 2024-03-22 11:02:40.615265944 +0000 UTC m=+0.449277286 container init 2198961de166012581c4b1ea90d05253e5ed695d876e2941128e5c09f1dd71ad (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-22T11:02:40.905 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:40 smithi150 podman[96255]: 2024-03-22 11:02:40.626344861 +0000 UTC m=+0.460356199 container start 2198961de166012581c4b1ea90d05253e5ed695d876e2941128e5c09f1dd71ad (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T11:02:40.905 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:40 smithi150 podman[96255]: 2024-03-22 11:02:40.705631634 +0000 UTC m=+0.539643010 container attach 2198961de166012581c4b1ea90d05253e5ed695d876e2941128e5c09f1dd71ad (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-22T11:02:40.905 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:40 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:40.798+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:40.905 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:40 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:40.798+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:41.406 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 22 11:02:41 smithi150 podman[96281]: 2024-03-22 11:02:41.109571872 +0000 UTC m=+0.032892891 container died 2198961de166012581c4b1ea90d05253e5ed695d876e2941128e5c09f1dd71ad (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-22T11:02:41.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.4.service' 2024-03-22T11:02:41.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T11:02:41.890 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-22T11:02:41.891 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-22T11:02:41.891 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.5 2024-03-22T11:02:42.018 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:41 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:41.774+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:42.018 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:41 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:41.774+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:42.324 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:42 smithi150 systemd[1]: Stopping Ceph osd.5 for fd1caff4-e836-11ee-95cd-87774f69a715... 2024-03-22T11:02:42.654 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:42 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:42.543+0000 7f2ef6200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T11:02:42.654 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:42 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:42.543+0000 7f2ef6200700 -1 osd.5 83 *** Got signal Terminated *** 2024-03-22T11:02:42.655 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:42 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:42.543+0000 7f2ef6200700 -1 osd.5 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T11:02:43.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:42 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:42.769+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:43.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:42 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:42.769+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:44.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:43 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:43.739+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:44.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:43 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:43.739+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:45.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:44 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:44.780+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:45.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:44 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:44.780+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:46.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:45 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:45.757+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:46.155 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:45 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:45.757+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:46.155 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:45 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:45.757+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6822 osd.2 since back 2024-03-22T11:02:19.776935+0000 front 2024-03-22T11:02:19.776844+0000 (oldest deadline 2024-03-22T11:02:45.076513+0000) 2024-03-22T11:02:47.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:46 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:46.747+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-03-22T11:02:01.673154+0000 front 2024-03-22T11:02:01.673225+0000 (oldest deadline 2024-03-22T11:02:25.772679+0000) 2024-03-22T11:02:47.155 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:46 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:46.747+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6814 osd.1 since back 2024-03-22T11:02:09.775343+0000 front 2024-03-22T11:02:09.775478+0000 (oldest deadline 2024-03-22T11:02:35.075318+0000) 2024-03-22T11:02:47.155 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:46 smithi150 ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5[76444]: 2024-03-22T11:02:46.747+0000 7f2eeca00700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.115:6822 osd.2 since back 2024-03-22T11:02:19.776935+0000 front 2024-03-22T11:02:19.776844+0000 (oldest deadline 2024-03-22T11:02:45.076513+0000) 2024-03-22T11:02:48.389 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:48 smithi150 podman[96361]: 2024-03-22 11:02:48.080383633 +0000 UTC m=+5.713797726 container died 956493e4218861ccdfd039b61507a8a89dee7383beae26b6b1c72ff1ef07823b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-22T11:02:48.654 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:48 smithi150 podman[96361]: 2024-03-22 11:02:48.511278255 +0000 UTC m=+6.144692386 container cleanup 956493e4218861ccdfd039b61507a8a89dee7383beae26b6b1c72ff1ef07823b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T11:02:48.654 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:48 smithi150 bash[96361]: ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5 2024-03-22T11:02:49.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:48 smithi150 podman[96382]: 2024-03-22 11:02:48.762565159 +0000 UTC m=+0.671360063 container remove 956493e4218861ccdfd039b61507a8a89dee7383beae26b6b1c72ff1ef07823b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-22T11:02:49.889 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:49 smithi150 podman[96470]: 2024-03-22 11:02:49.509287494 +0000 UTC m=+0.033291570 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-22T11:02:49.890 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:49 smithi150 podman[96470]: 2024-03-22 11:02:49.621633486 +0000 UTC m=+0.145637584 container create b0e1d7efba47cef1368b177be17ba207fad386eca68af8fd546d611f3f131730 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T11:02:49.890 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:49 smithi150 podman[96470]: 2024-03-22 11:02:49.892175879 +0000 UTC m=+0.416179984 container init b0e1d7efba47cef1368b177be17ba207fad386eca68af8fd546d611f3f131730 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-22T11:02:50.154 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:49 smithi150 podman[96470]: 2024-03-22 11:02:49.902924122 +0000 UTC m=+0.426928230 container start b0e1d7efba47cef1368b177be17ba207fad386eca68af8fd546d611f3f131730 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-22T11:02:50.155 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:50 smithi150 podman[96470]: 2024-03-22 11:02:50.024093885 +0000 UTC m=+0.548098006 container attach b0e1d7efba47cef1368b177be17ba207fad386eca68af8fd546d611f3f131730 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD) 2024-03-22T11:02:50.654 INFO:journalctl@ceph.osd.5.smithi150.stdout:Mar 22 11:02:50 smithi150 podman[96470]: 2024-03-22 11:02:50.338871796 +0000 UTC m=+0.862875877 container died b0e1d7efba47cef1368b177be17ba207fad386eca68af8fd546d611f3f131730 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd1caff4-e836-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-22T11:02:51.060 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd1caff4-e836-11ee-95cd-87774f69a715@osd.5.service' 2024-03-22T11:02:51.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T11:02:51.183 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-22T11:02:51.183 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd1caff4-e836-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T11:02:51.478 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T11:02:58.165 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi115.stderr:ceph-fuse[66966]: fuse finished with error 0 and tester_r 0 2024-03-22T11:03:24.192 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd1caff4-e836-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T11:03:24.471 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T11:03:45.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T11:03:45.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T11:03:45.877 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-22T11:03:45.879 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273/remote/smithi115/crash 2024-03-22T11:03:45.880 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/crash -- . 2024-03-22T11:03:45.977 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T11:03:45.977 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2024-03-22T11:03:45.982 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273/remote/smithi150/crash 2024-03-22T11:03:45.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/crash -- . 2024-03-22T11:03:46.075 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T11:03:46.075 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-03-22T11:03:46.080 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-22T11:03:46.080 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-22T11:03:46.187 INFO:teuthology.orchestra.run.smithi115.stdout:2024-03-22T10:46:08.901624+0000 mon.smithi115 (mon.0) 254 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T11:03:46.189 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-22T11:03:46.189 INFO:tasks.cephadm:Compressing logs... 2024-03-22T11:03:46.189 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T11:03:46.194 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T11:03:46.254 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-22T11:03:46.255 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-22T11:03:46.256 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-22T11:03:46.256 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mgr.smithi115.iqzwsz.log 2024-03-22T11:03:46.257 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-22T11:03:46.258 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mgr.smithi115.iqzwsz.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mon.smithi115.log 2024-03-22T11:03:46.260 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi115.wjotmn.log 2024-03-22T11:03:46.261 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mon.smithi115.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-22T11:03:46.264 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi115.wjotmn.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi115.log 2024-03-22T11:03:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.1.log: /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi115.log: 91.5% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-22T11:03:46.265 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi115.ggxfzi.log 2024-03-22T11:03:46.268 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi115.ggxfzi.log: 94.2%gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-22T11:03:46.270 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-22T11:03:46.270 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi115.log.gz 2024-03-22T11:03:46.271 INFO:teuthology.orchestra.run.smithi115.stderr: 90.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-22T11:03:46.273 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.log 2024-03-22T11:03:46.275 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.cephadm.log: /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.log: 85.7% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-22T11:03:46.280 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-22T11:03:46.281 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-22T11:03:46.285 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-22T11:03:46.285 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-22T11:03:46.286 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi150.slzwwb.log 2024-03-22T11:03:46.287 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.2.log: /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-volume.log: 88.7% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-22T11:03:46.287 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mon.smithi150.log 2024-03-22T11:03:46.288 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi150.slzwwb.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi150.zvyrzc.log 2024-03-22T11:03:46.289 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mon.smithi150.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-22T11:03:46.289 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi150.log 2024-03-22T11:03:46.290 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi150.zvyrzc.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-22T11:03:46.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi150.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-22T11:03:46.293 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mgr.smithi150.fmoumv.log 2024-03-22T11:03:46.294 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.4.log: 94.2% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi150.log.gz 2024-03-22T11:03:46.295 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-22T11:03:46.297 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mgr.smithi150.fmoumv.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-22T11:03:46.302 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.5.log: /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-22T11:03:46.302 INFO:teuthology.orchestra.run.smithi150.stderr: 91.6% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-22T11:03:46.302 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.log 2024-03-22T11:03:46.304 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-22T11:03:46.305 INFO:teuthology.orchestra.run.smithi150.stderr: 85.7% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.cephadm.log.gz/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.log: 2024-03-22T11:03:46.308 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-03-22T11:03:46.313 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-volume.log: 88.7%/var/log/ceph/ceph-client.1.log: -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-22T11:03:46.322 INFO:teuthology.orchestra.run.smithi115.stderr: 91.5% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-22T11:03:46.338 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-22T11:03:46.354 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mgr.smithi150.fmoumv.log.gz 2024-03-22T11:03:46.749 INFO:teuthology.orchestra.run.smithi115.stderr: 88.3% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mgr.smithi115.iqzwsz.log.gz 2024-03-22T11:03:46.842 INFO:teuthology.orchestra.run.smithi150.stderr: 92.5% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mon.smithi150.log.gz 2024-03-22T11:03:47.432 INFO:teuthology.orchestra.run.smithi115.stderr: 90.4% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mon.smithi115.log.gz 2024-03-22T11:03:50.303 INFO:teuthology.orchestra.run.smithi150.stderr: 93.6% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-22T11:03:51.009 INFO:teuthology.orchestra.run.smithi115.stderr: 93.7% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-22T11:03:51.433 INFO:teuthology.orchestra.run.smithi115.stderr: 93.6% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-22T11:03:51.713 INFO:teuthology.orchestra.run.smithi150.stderr: 95.2% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi150.slzwwb.log.gz 2024-03-22T11:03:51.923 INFO:teuthology.orchestra.run.smithi115.stderr: 95.2% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi115.wjotmn.log.gz 2024-03-22T11:03:52.104 INFO:teuthology.orchestra.run.smithi150.stderr: 93.7% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-22T11:03:52.825 INFO:teuthology.orchestra.run.smithi115.stderr: 93.7% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-22T11:03:52.869 INFO:teuthology.orchestra.run.smithi150.stderr: 93.7% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-22T11:04:02.311 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-03-22T11:04:02.312 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-03-22T11:04:03.145 INFO:teuthology.orchestra.run.smithi115.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-22T11:04:28.317 INFO:teuthology.orchestra.run.smithi150.stderr: 93.2% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi150.zvyrzc.log.gz 2024-03-22T11:04:28.321 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-03-22T11:04:28.321 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m42.076s 2024-03-22T11:04:28.321 INFO:teuthology.orchestra.run.smithi150.stderr:user 1m15.334s 2024-03-22T11:04:28.321 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m5.221s 2024-03-22T11:04:38.568 INFO:teuthology.orchestra.run.smithi115.stderr: 93.4% -- replaced with /var/log/ceph/fd1caff4-e836-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi115.ggxfzi.log.gz 2024-03-22T11:04:38.572 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-03-22T11:04:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m52.348s 2024-03-22T11:04:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:user 1m26.423s 2024-03-22T11:04:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m6.637s 2024-03-22T11:04:38.574 INFO:tasks.cephadm:Archiving logs... 2024-03-22T11:04:38.574 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273/remote/smithi115/log 2024-03-22T11:04:38.575 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T11:04:50.610 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273/remote/smithi150/log 2024-03-22T11:04:50.611 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T11:05:01.142 INFO:tasks.cephadm:Removing cluster... 2024-03-22T11:05:01.142 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd1caff4-e836-11ee-95cd-87774f69a715 --force 2024-03-22T11:05:01.422 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T11:05:02.278 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd1caff4-e836-11ee-95cd-87774f69a715 --force 2024-03-22T11:05:02.561 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: fd1caff4-e836-11ee-95cd-87774f69a715 2024-03-22T11:05:03.395 INFO:tasks.cephadm:Removing cephadm ... 2024-03-22T11:05:03.395 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-22T11:05:03.452 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-22T11:05:03.507 INFO:tasks.cephadm:Teardown complete 2024-03-22T11:05:03.507 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-22T11:05:03.519 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-22T11:05:03.520 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-22T11:05:03.520 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T11:05:03.527 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T11:05:03.675 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-22T11:05:03.687 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-22T11:05:03.687 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T11:05:03.695 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T11:05:03.742 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-03-22T11:05:03.750 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-03-22T11:05:03.751 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T11:05:03.752 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-03-22T11:05:03.752 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 +135us[ +135us] +/- 115ms 2024-03-22T11:05:03.752 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv02.front.sepia.ceph.com 3 8 377 174 +92us[ +127us] +/- 73ms 2024-03-22T11:05:03.752 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -112us[ -112us] +/- 69ms 2024-03-22T11:05:03.752 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T11:05:03.759 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T11:05:03.759 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-03-22T11:05:03.759 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 +103us[ +119us] +/- 115ms 2024-03-22T11:05:03.759 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 +72us[ +87us] +/- 75ms 2024-03-22T11:05:03.759 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 -116us[ -101us] +/- 69ms 2024-03-22T11:05:03.759 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T11:05:03.761 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-22T11:05:03.772 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-22T11:05:03.773 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-22T11:05:03.785 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-22T11:05:04.021 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-22T11:05:04.245 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-22T11:05:04.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T11:05:04.347 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-22T11:05:04.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T11:05:04.449 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-22T11:05:04.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-22T11:05:04.472 INFO:teuthology.task.internal:Duration was 2676.714076 seconds 2024-03-22T11:05:04.473 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-22T11:05:04.484 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-22T11:05:04.485 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T11:05:04.492 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T11:05:04.649 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T11:05:04.654 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T11:05:05.103 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-22T11:05:05.103 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-03-22T11:05:05.103 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T11:05:05.164 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-03-22T11:05:05.164 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T11:05:05.237 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-22T11:05:05.237 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T11:05:05.245 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T11:05:05.336 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-22T11:05:05.336 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T11:05:05.341 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T11:05:06.163 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-22T11:05:06.176 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-22T11:05:06.176 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T11:05:06.266 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T11:05:06.367 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-22T11:05:06.379 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T11:05:06.387 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T11:05:06.489 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-03-22T11:05:06.491 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-03-22T11:05:06.552 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T11:05:06.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T11:05:06.602 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T11:05:06.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T11:05:06.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-22T11:05:06.666 INFO:teuthology.task.internal:Transferring archived files... 2024-03-22T11:05:06.667 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273/remote/smithi115 2024-03-22T11:05:06.668 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T11:05:06.793 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273/remote/smithi150 2024-03-22T11:05:06.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T11:05:06.905 INFO:teuthology.task.internal:Removing archive directory... 2024-03-22T11:05:06.905 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T11:05:06.914 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T11:05:07.000 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-22T11:05:07.014 INFO:teuthology.task.internal:Not uploading archives. 2024-03-22T11:05:07.014 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-22T11:05:07.027 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-22T11:05:07.027 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T11:05:07.035 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T11:05:07.085 INFO:teuthology.orchestra.run.smithi115.stdout: 395173 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 22 11:05 /home/ubuntu/cephtest 2024-03-22T11:05:07.085 INFO:teuthology.orchestra.run.smithi115.stdout: 812443 4 d--------- 2 ubuntu ubuntu 4096 Mar 22 10:39 /home/ubuntu/cephtest/mnt.0 2024-03-22T11:05:07.085 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-03-22T11:05:07.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T11:05:07.088 INFO:teuthology.orchestra.run.smithi115.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-03-22T11:05:07.089 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-22T11:05:07.090 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-22T11:05:07.101 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-22T11:05:07.115 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-22T11:05:07.115 INFO:teuthology.nuke:Checking targets against current locks 2024-03-22T11:05:07.140 DEBUG:teuthology.nuke:shortname: smithi115 2024-03-22T11:05:07.141 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T11:05:07.163 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 10:10:00.380312', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOZzXRohABkKc0eyosDcZ7Pj4wmSn9EHedxgyfbWG3nNkhXPgeagRbsOfQFehm0lfJc5UBWiZEDSOG1Krl1X61s='} 2024-03-22T11:05:07.167 DEBUG:teuthology.nuke:shortname: smithi150 2024-03-22T11:05:07.167 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T11:05:07.188 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:46:06-fs:upgrade-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616273', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 10:10:00.378428', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLZMjNswqm/B+vSDPOQOn4rTgjwKsbCB0d+IcJtfWnVO0J9bHO/l18dA42MwhBfuTiLS24cB6BBzzA8kpzoNHGs='} 2024-03-22T11:05:07.221 INFO:teuthology.orchestra.console.smithi115:Power off 2024-03-22T11:05:07.222 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T11:05:07.238 INFO:teuthology.orchestra.console.smithi150:Power off 2024-03-22T11:05:07.238 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T11:05:07.249 DEBUG:teuthology.orchestra.console.smithi115:power off output: Chassis Power Control: Down/Off 2024-03-22T11:05:07.249 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T11:05:07.260 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2024-03-22T11:05:07.261 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T11:05:07.271 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-03-22T11:05:07.285 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-03-22T11:05:11.273 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T11:05:11.286 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T11:05:11.384 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-03-22T11:05:11.395 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-03-22T11:05:15.388 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T11:05:15.396 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T11:05:15.498 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-03-22T11:05:15.505 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-03-22T11:05:15.606 INFO:teuthology.orchestra.console.smithi150:Power off completed 2024-03-22T11:05:19.502 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T11:05:19.611 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off 2024-03-22T11:05:19.712 INFO:teuthology.orchestra.console.smithi115:Power off completed 2024-03-22T11:05:19.817 INFO:teuthology.run:Summary data: description: fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2676.714075565338 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default host.a-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 host.b-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 owner: mchangir status: fail success: false 2024-03-22T11:05:19.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T11:05:19.898 INFO:teuthology.run:FAIL